Thursday, December 15, 2016

Create Dynamo DB Table using Cloud Formation Template


AWS CloudFormation simplifies provisioning and management on AWS. It gives developers and systems administrators an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion. To provision and configure stack resources, we must understand AWS CloudFormation templates, which are formatted text files in JSON or YAML. These templates describe the resources that we want to provision in your AWS CloudFormation stacks. We can use the AWS CloudFormation Designer or any text editor to create and save templates.

Let us how designer works in an another blog entry. For now you can play with designer if you wish.
https://console.aws.amazon.com/cloudformation/designer

Creating Dynamo DB Using AWS Cloud Formation Template.

1. We need to create a custom IAM policy "createstack". This policy hels us to execute aws createstack command.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1449904348000",
            "Effect": "Allow",
            "Action": [
                "cloudformation:CreateStack"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

>

2. Attach the above created policy to user through whom we need to create dynamo db table.

3.install awscli and configure
  a. sudo apt-get install awscli
  b. aws configure
        now give proper details like access key and region. This will create a config file in ~/.aws directory.
  c. test if configured properly using command 'aws s3 ls'. This should list all the s3 buckets if you have any.

4.Create a Cloud formation temlate like below , for creating dynamo db.


{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Resources" : {
    "myDynamoDBTable" : {
      "Type" : "AWS::DynamoDB::Table",
      "Properties" : {
        "AttributeDefinitions" : [
          {
            "AttributeName" : "Name",
            "AttributeType" : "S"   
          },
          {
            "AttributeName" : "Age",
            "AttributeType" : "S"
          }
        ],
        "KeySchema" : [
          {
            "AttributeName" : "Name",
            "KeyType" : "HASH"
          },
          {
            "AttributeName" : "Age",
            "KeyType" : "RANGE"
          }
        ],
        "ProvisionedThroughput" : {
          "ReadCapacityUnits" : "5",
          "WriteCapacityUnits" : "5"
        },
        "TableName" : "Person"
      }
    }
  }
}


5. Save the above file in a s3 bucket and copy the URL of this file.

6. Now on your command line , you can enter following command to create dynamodb table.

aws cloudformation create-stack --stack-name <stack_name> --template-url <s3_bucket_template_url>

----------------------------------------------------------------------------------------------------------------------------

Now, this is too much of manual process. This can be done using a python code as well. ( or java , node.js etc, lets see python for now ).

Here is how.

1. Create a config file like below. Save it as 'awsconfig'

[default]
aws_access_key_id = xxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxxxxx
region = us-west-2

2. Create a shell script like below.

sudo apt-get install -fy --force-yes awscli python
sudo curl -s 'https://bootstrap.pypa.io/get-pip.py' | python2.7 && pip install boto awscli
sudo mkdir ~/.aws
cp awsconfig ~/.aws/config

The above script should install all the required tools for python to create a dynamo db. Allow execution permission and execute the script.

3. Create a python file using following code.


from __future__ import print_function # Python 2/3 compatibility
import boto3

dynamodb = boto3.resource('dynamodb', region_name='us-west-2')


table = dynamodb.create_table(
    TableName='Person',
    KeySchema=[
        {
            'AttributeName': 'name',
            'KeyType': 'HASH'  #Partition key
        },
        {
            'AttributeName': 'age',
            'KeyType': 'RANGE'  #Sort key
        }
    ],
    AttributeDefinitions=[
        {
            'AttributeName': 'name',
            'AttributeType': 'S'
        },
        {
            'AttributeName': 'age',
            'AttributeType': 'N'
        },

    ],
    ProvisionedThroughput={
        'ReadCapacityUnits': 5,
        'WriteCapacityUnits': 5
    }
)

print("Table status:", table.table_status)


4. Execute python code .

  python create_dynamo_table.py

5. Check your Dynamo DB service , table called person should have been created.


9 comments:

  1. Very nice article of Dynamo DB creation Using AWS Cloud Formation Template, thanks for providing this useful info!

    Best Regards,
    CourseIng - AWS Online Training in Hyderabad

    ReplyDelete
  2. do we need to add role for this template

    ReplyDelete
    Replies
    1. Yes if you are not a admin and you do not have permission to create a dynamo db, then you have to add a role and attach policy to execute the cf template.

      Delete
  3. {
    "AWSTemplateFormatVersion" : "2010-09-09",
    "Parameters": {

    "TableName": {
    "Description": "Name of DynamoDB Table",
    "Type": "String"
    }
    },
    "Resources" : {
    "myDynamoDBTable" : {
    "Type" : "AWS::DynamoDB::Table",
    "Properties" : {
    "AttributeDefinitions" : [
    {
    "AttributeName" : "Name",
    "AttributeType" : "S"
    },
    {
    "AttributeName" : "Age",
    "AttributeType" : "S"
    }
    ],
    "KeySchema" : [
    {
    "AttributeName" : "Name",
    "KeyType" : "HASH"
    },
    {
    "AttributeName" : "Age",
    "KeyType" : "RANGE"
    }
    ],
    "ProvisionedThroughput" : {
    "ReadCapacityUnits" : "5",
    "WriteCapacityUnits" : "5"
    },
    "TableName" : {
    "Ref": "TableName"
    }
    }
    }
    }
    }

    And you can execute like ...

    aws cloudformation create-stack --stack-name --template-url --parameters ParameterKey=TableName,ParameterValue=Person


    ReplyDelete
  4. Do you have any idea on why this template is failing to create the table?

    {
    "AWSTemplateFormatVersion" : "2010-09-09",
    "Resources" : {
    "myDynamoDBTable" : {
    "Type" : "AWS::DynamoDB::Table",
    "Properties" : {
    "AttributeDefinitions" : [
    {
    "AttributeName" : "Process",
    "AttributeType" : "S"
    },
    {
    "AttributeName" : "Flujo",
    "AttributeType" : "S"
    },
    {
    "AttributeName" : "Album",
    "AttributeType" : "S"
    }
    ],
    "KeySchema" : [
    {
    "AttributeName" : "Process",
    "KeyType" : "HASH"
    },
    {
    "AttributeName" : "Flujo",
    "KeyType" : "RANGE"
    }
    ],
    "ProvisionedThroughput" : {
    "ReadCapacityUnits" : "1",
    "WriteCapacityUnits" : "1"
    },
    "TableName" : "ConfiguracionApps"
    }
    }
    }
    }

    I tried to remove the "Album" attribute and worked, but I can't manage to add more attributes. In the console I got an error that said "Property AttributeDefinitions is inconsistent with the KeySchema of the table and the secondary indexes"

    ReplyDelete
    Replies
    1. Add global secondary index on Album.
      Refer : https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dynamodb-table.html

      Delete
  5. Nice blog! Thanks for providing abd update more things with us, Get more Knowledge on AWS Online Course Hyderabad

    ReplyDelete
  6. Hi,

    Thank you so much for taking effort to share such a useful information. I will definitely share your articles to my online portal DynamoDB Development blog.

    Ecommerce Website Development
    Aapthi Technologies
    Web Development Company Dubai

    ReplyDelete
  7. It is really a great work and the way in which you are sharing the knowledge is excellent.
    aws training in omr | aws training in velachery | best aws training center in chennai

    ReplyDelete