Vagrant for Automating Your Virtual Box

Vagrant creates and configures virtual development environments. It can be seen as a higher-level wrapper around virtualization software such as VirtualBox, VMware, KVM and Linux Containers (LXC), and around configuration management software such as Ansible, Chef, Salt and Puppet.

Vagrant is not just about creating VMs, it is about automating the work of setting up development environments for our projects. Also, we can check Vagrantfiles into source control with each project, so the environment is essentially stored with the code, without having to find a way to store a VM itself.

Install VirtualBox
sudo apt-get install virtualbox

Install Vagrant
sudo apt-get install vagrant

OR Install Vagrant from here
https://www.vagrantup.com/downloads.html

mkdir vagrant_test
cd vagrant_test
vagrant init

This will create a Vagrantfile. As said Vagrantfile is config file for the virtual machine.

Next download an image.
vagrant box add ubuntu/trusty64
you can browse for available boxes here : https://atlas.hashicorp.com/boxes/search

This stores the box under a specific name so that multiple Vagrant environments can re-use it. Just like VM templates

[Note : The syntax for the vagrant box add subcommand is changed with the version 1.5, due to the Vagrant Cloud introduction.
vagrant box add {title} {url}
http://www.vagrantbox.es/]

Next change your Vagrantfile contens as below.

Vagrant.configure("2") do |config|
   config.vm.box = "ubuntu/trusty64"
   config.vm.network "private_network", ip: "xxx.xxx.xx.xx"
   config.vm.provider "virtualbox" do |vb|
     vb.cpus = 2
     vb.memory = "4096"
   end
end

Instead of using default box , we point config.vm.box to "ubuntu/trusty64" that we downloaded earlier.
config.vm.network : This allows to access any servers running on the box to be available to the network. You can configure public ip if you need.
Next , we set number of cpus 2 , also memory to 4GB.

Now bring up the box using following command
vagrant up

We can connect to the machine using
vagrant ssh

To destroy VM, we can use following command
vagrant destroy

Also note that vagrant halt will shut down the machine gracefully. And if you make any changes to Vagrantfile, to update the box we can use vagrant reload command.

AWS RDS - Take Snapshot, Delete Instance and Restore Instance using Snapshot - Scheduled Automation using Lambda

Create a test RDS instance which is db.t2.micro ( free tier ) , name it testdb and provide all parameters and create.

1. Create an IAM Role for Lambda with following policy. IAM → Roles → CreateNewRole

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Action": [
                "rds:AddTagsToResource",
                "rds:CopyDBSnapshot",
                "rds:CopyDBClusterSnapshot",
                "rds:DeleteDBInstance",
                "rds:DeleteDBSnapshot",
                "rds:RestoreDBInstanceFromDBSnapshot",
                "rds:Describe*",
                "rds:ListTagsForResource"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

2. Create a Lambda function for deleting RDS instance by taking latest snapshot.

3. Select Blank Function.

4. Configure Trigger Using CloudWatch Events – Schedule.

5. Enter Rule Name , Rule Description and Scheduled Expression ( in UTC ) time like : cron(0 0 21 ? * MON-FRI *) - This means it triggers every day from mon to friday at night 9 pm UTC time

6. Select python 2.7 and write Lambda Function ( change db_instance and region accordingly )

import boto3  
import datetime  
import time  
import sys

db_instance='testdb'  
region='us-west-2'

def lambda_handler(event, context):  
    try: 
        date=time.strftime("-%d-%m-%Y")
        snapshot_name = db_instance+date
        source = boto3.client('rds', region_name=region)
        global db_instance
        source.delete_db_instance(DBInstanceIdentifier=db_instance,SkipFinalSnapshot=False,FinalDBSnapshotIdentifier=snapshot_name)
    except Exception as e:
        raise e
    print '[main] End'

7. Select existing IAM role that we created in Step 1.
8. Create Lambda.
9. Test this Function and wait till snapshot created and instance deleted.




Restore :

1 . Create Lambda trigger at morning 9 am UTC

2. Add lambda code.

3.Write Lambda Function

import boto3  
import botocore  
import datetime  
import re  
import logging

region='us-west-2'  
db_instance_class='db.t2.micro'  
db_subnet='default'  
instances = ['testdb']

print('Loading function')

def byTimestamp(snap):  
  if 'SnapshotCreateTime' in snap:
    return datetime.datetime.isoformat(snap['SnapshotCreateTime'])
  else:
    return datetime.datetime.isoformat(datetime.datetime.now())

def lambda_handler(event, context):  
    source = boto3.client('rds', region_name=region)
    for instance in instances:
        try:
            source_snaps = source.describe_db_snapshots(DBInstanceIdentifier = instance)['DBSnapshots']
            print "DB_Snapshots:", source_snaps
            source_snap = sorted(source_snaps, key=byTimestamp, reverse=True)[0]['DBSnapshotIdentifier']
            snap_id = (re.sub( '-\d\d-\d\d-\d\d\d\d ?', '', source_snap))
            print('Will restore %s to %s' % (source_snap, snap_id))
            response = source.restore_db_instance_from_db_snapshot(DBInstanceIdentifier=snap_id,DBSnapshotIdentifier=source_snap,DBInstanceClass=db_instance_class, DBSubnetGroupName=db_subnet,MultiAZ=False,PubliclyAccessible=True)
            print(response)

        except botocore.exceptions.ClientError as e:
            raise Exception("Could not restore: %s" % e)

3. Select IAM Role.
4. Create Function
5. Test Function.