CyberKeeda In Social Media
Showing posts with label AWS Cloudformation. Show all posts
Showing posts with label AWS Cloudformation. Show all posts

AWS CloudFormation Interface Metadata



AWS CloudFormation Metadata
Metadata section define the details about the cloudformation template. 

Syntax Template

Metadata:
  Instances:
    Description"Instances details within dev environment"
  Applications:
    Description"Application details for dev environment"

There are three types of AWS Cloudformation specific Metadata keys.
  •   AWS::CloudFormation::Designer
It's auto generated during drag and drop of canvas within Cloudformation designer section.
  •   AWS::CloudFormation::Interface
It's used for parameter grouping and labeling as per requirement
  •   AWS::CloudFormation::Init
One of the most important section from Automation prospective, it's used for Application installation and configuration on our AWS EC2 instances.

So within this blog post, we will cover 
  • What is Interface Metadata.
  • Why and How it's used
  • How to customize orders of defined parameters.
  • How to group up parameters and mark them a identifying label.
  • How to define labels for the parameters.

AWS::CloudFormation::Interface

So during our stack creation, you might have noticed our defined parameters appears into an alphabetical order by their Logical ids, it has nothing to do with the way or order we define in our CFN template.

Interface Metadata Syntax
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - ParameterGroup
    ParameterLabels:
      - ParameterLabel

For this post, in order to understand more about it we will use the below CFN template as an example.

CFN Template

AWSTemplateFormatVersion: 2010-09-09
Description|
  CFN Script to demonstrate Cloudformation Interface Metadata Usage
  Interface Metadata : AWS::CloudFormation::Interface
  Interface Metadata can be used to
           Group Parameters and it's order.
           Labels for Parameter for user friendly description input.

Parameters:
  EnvironmentName:
    DescriptionSelect the environment.
    TypeString
    Defaultdev
    AllowedValues:
      - dev
      - prd
  EC2KeyName:
    DescriptionSelect your Key name from List.
    TypeAWS::EC2::KeyPair::KeyName
  EC2SecurityGroup:
    DescriptionSelect your security group from List.
    TypeAWS::EC2::SecurityGroup::Id
  EC2Subnet:
    DescriptionSelect your Subnet from List.
    TypeAWS::EC2::Subnet::Id
  EC2InstanceType:
    TypeString
    Defaultt2.micro
    AllowedValues:
      - t2.micro
      - t2.small
  ApplicationType:
    TypeString
    Defaultweb
    AllowedValues:
      - web
      - app
  MonitoringStatus:
    TypeString
    Defaultenabled
    AllowedValues:
      - enabled
      - disabled

When we will import the above template to create or update stack the console will appear something like below.



From above we can witness that irrespective of the order of allignment parameters within CFN script, it re-orders it within console alphabetically by Parameter's logical ids.

What if we want to group parameters and label them as a separate identifier, Interface metadata can help us to accomplish the goal

What are we going to do with above template ?
    Well we have two sections one relates to the  EC2 Configuration data and the other one is more specific to Application configuration, so we will define our parameter groups and label according to that only.

CFN Interface Metadata with Custom Parameter Group

AWSTemplateFormatVersion: 2010-09-09
Description|
  CFN Script to demonstrate Cloudformation Interface Metadata Usage
  Interface Metadata : AWS::CloudFormation::Interface
  Interface Metadata can be used to
           Group Parameters and it's order.
           Labels for Parameter for user friendly description input.
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default"EC2 Configuration"
        Parameters:
          - EC2InstanceType
          - EC2KeyName
          - EC2SecurityGroup
          - EC2Subnet
      - Label:
          default"Application Configuration"
        Parameters:
          - EnvironmentName
          - ApplicationType
          - MonitoringStatus

Once above file is imported, here is how the console screen looks likes.
We have two Parameter Groups and our parameters are ordered by the way we defined under these groups.
  • EC2 Configuration
  • Application Configuration.

Now, we will add some custom Labels to our Parameters to make it more user friendly and informative for user to avoid confusion.

Within our base template, we will intuit below Parameter Logical Ids with our own Label.
  • EnvironmentName
    • Be carefull while choosing your deployment environmnet
  • EC2KeyName
    • Ensure you have the keys before selecting
  • MonitoringStatus
    • It enables cloudwatch logs for application data

CFN Template to demonstrate Custom Labels for Parameters

Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default"EC2 Configuration"
        Parameters:
          - EC2InstanceType
          - EC2KeyName
          - EC2SecurityGroup
          - EC2Subnet
      - Label:
          default"Application Configuration"
        Parameters:
          - EnvironmentName
          - ApplicationType
          - MonitoringStatus
    ParameterLabels:
      EnvironmentName
        default"Be carefull while choosing your deployment environmnet"
      EC2KeyName:
        default"Ensure you have the keys before selecting"
      MonitoringStatus:
        default"It enables cloudwatch logs for application data"

Once imported, this is the output in cloudformation console.



From above, we can state few things to note while using a parameter label configuration.

  • ParameterLabels will substitute your logical Id for your parameter with the one we have declared as default within our ParameterLabels section.
  • Only Logical Id gets substituted, description mentioned under individual parameter section persists.

AWS CFN best practice, we must use Interface metadata for large stack creation this makes things quite helpful for end user.

Feel free to comment.  

Read more ...

AWS CloudFormation - EC2 UserData



AWS CloudFormation EC2 UserData.


EC2 UserData is a way to define multiple actions within ec2 instnace once the system gets first time started, if you are from a Linux background you might be familiar with rc.local file where we used to write all the commands or script action that needs to be executed whenever a system gets booted up.
It's somehow similar to that but not the same at all.

Something more about EC2 UserData
  • We can use UserData in CFN template for EC2. 
  • We need to use an intrinsic function Fn::Base64 with UserData in CFN template, this function return the Base64 representation of string, it passes encoded data to EC2 instance.
  • Multiple Line values can be used under Base64 followed by YAML Pipe ( | )
  • UserData scripts/commands runs during the boot cycle when we first launch the instance.
  • To Update and reflect the changes under the value of UserData, reboot is required.
  • Can be used the same way normal parameters are used via !Ref as an argument.
Usage Example.

Below CFN template along with UserData section is used to install to perform three steps.
  • Update YUM
  • Download Logstash using Wget
  • Install downloaded RPM
AWSTemplateFormatVersion: 2010-09-09
Description: A Cloudformation Script to demonstrate EC2 UserData

Resources:
CreateEC2Instance:
Type: "AWS::EC2::Instance"
Properties:
ImageId: ami-01460aa81365561fe
InstanceType: t2.micro
KeyName: kunal_eks_test
SubnetId: subnet-0cdcc621
IamInstanceProfile: "IamInstanceProfile"
SecurityGroups:
- launch-wizard-1
UserData:
Fn::Base64: |
#!/bin/bash
sudo yum update -y
sudo wget https://artifacts.elastic.co/downloads/logstash/logstash-7.1.0.rpm
sudo rpm -Uvh logstash/logstash-7.1.0.rpm

Feel free to comment !

Read more ...

AWS CloudFormation Psuedo Parameters




AWS Cloud Formation Pseudo Parameters.


Parameters section within CloudFormation template is way to gather inputs from user, which can be used within other sections of entire cloudformation script.

Pseudo Parameter
  • Type of parameters that are predefined by AWS Cloudformation.
  • We don't need to declare it within our template section.
  • Can be used the same way normal parameters are used via !Ref as an argument.
Usage Examples.

"AWS::Region" is one of the widely used Pseudo parameters, so below is the way we can use it under our CFN template.


CFN Output Section
Outputs:
  MyStacksRegion:
    Value: !Ref "AWS::Region"


CFN Mappings Section
Resources:
  CreateEC2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      ImageId: !FindInMap
      - ImageIdMap
      - !Ref !Ref "AWS::Region"
      - defaultAMI

So in the same there are few more pseudo parameters enlisted below.

AWS::AccountId
  • Returns the account id from where the stack is being created.
AWS::NotificationARNs
  • Returns the list of notification Amazon Resource Names (ARNs) for the current stack
AWS::NoValue
  • Removes the corresponding resource property when specified as a return value in the Fn::If intrinsic function.
AWS::Partition
  • Returns the partition that the resource is in. For standard AWS regions, the partition is aws. For resources in other partitions, the partition is aws-partitionname
AWS::Region
  • Return where the stack is being created, example ap-south-1
AWS::StackId
  • Returns the ID of the stack as specified with the aws cloudformation create-stack command, such as arn:aws:cloudformation:ap-south-1:123456789987:stack/cyberkeedastack/90af3dc0-d9a7-01e4-972e-1234567se123.
AWS::StackName
  • Returns the name of the stack as specified with the aws cloudformation create-stack command, such as cyberkeedastack.
AWS::URLSuffix
  • Returns the suffix for a domain. The suffix is typically amazonaws.com, but might differ by region.

Read more ...

AWS CloudFormation Mappings : How and where to use.



AWS Cloud Formation Mappings.


Mappings is one of the section under AWS Cloudformation templates, which matches a key to it's corresponding set of named values

Basic mapping template.
Mappings:
MyFirstMapping:
Key1:
Name: Value1
Key2:
Name: Value2

MySecondMapping:
dev:
Name: Server1
prod:
Name: Server2

So from above example few things to note:

  • There can be multiple mappings under mapping section.
  • First/Top level key name must be different, from above template ( Key1, Key2, dev, prod)
  • Second level key must be same, from above template ( Name )
Where can we use mappings.
  • For example, if we want to set a instance type based upon environment or instance AMI id based on environment, we can use it to mappings.
So our Key will be environment name and Instance type and AMI id as values for specific second level key.
  • We can use Fn::FindInMap intrinsic function to retrieve values in map.
So we will use the same example within our Cloud Formation script and understand it's section.

AWSTemplateFormatVersion: 2010-09-09
Description: |
CFN Script to demonstrate Cloudformation Mappings section.

Parameters:
EnvironmentName:
Description: Select the environment.
Type: String
Default: dev
AllowedValues:
- dev
- prd
EC2KeyName:
Description: Select your Key name from List.
Type: AWS::EC2::KeyPair::KeyName
EC2SecurityGroup:
Description: Select your security group from List.
Type: AWS::EC2::SecurityGroup::Id
EC2Subnet:
Description: Select your Subnet from List.
Type: AWS::EC2::Subnet::Id

Mappings:
InstanceTypeMap:
dev:
defaultInstanceType: t2.micro
prd:
defaultInstanceType: t2.small
ImageIdMap:
dev:
defaultAMI: ami-0732b62d310b80e97
prd:
defaultAMI: ami-0732b62d310b80e97

Resources:
CreateEC2Instance:
Type: "AWS::EC2::Instance"
Properties:
ImageId: !FindInMap
- ImageIdMap
- !Ref EnvironmentName
- defaultAMI
InstanceType: !FindInMap
- InstanceTypeMap
- !Ref EnvironmentName
- defaultInstanceType
KeyName: !Ref EC2KeyName
SubnetId: !Ref EC2Subnet
SecurityGroupIds:
- !Ref EC2SecurityGroup

So from above template, 
  • We can launch a basic EC2 instance and here mappings is used to decide which AMI ID and Instance Type based upon environment dev or prod.
  • Parameters section has the EnvironmentName parameter, from which an user can select his/her environment name which can be prod or dev.
Mappings section : We have created two maps as
  • InstanceTypeMap : 
    • First/Top level keys : dev and prod
    • Second Level key : defaultInstanceType.
    • Second level values : t2.micro and t2.small
  • ImageIdMap:
    • First/Top level keys : dev and prod
    • Second Level key : defaultAMI
    • Second level values : ami-0732b62d3 andami-0732b6223

How to Use/Call mapping values within our resource section using intrinsic function FindInMap.


Our above template has two mappings and we will know how it has been referenced under resource section.

EC2 Instance creation resource has a requirement of ImageId through which we can declare our AMI Id.

So we can use the below template as base template.
ImageId: !FindInMap
- MappingLogicalId
- !Ref ParameterLogicalId
- SecondLevelKey
With above base template can be replaced with our values as.
  • Mapping Logical Id : ImageIdMap
  • Top/First Level Key : We have used Parameter to grab it as user input.
  • Second Level Key : defaultAMI
ImageId: !FindInMap
- ImageIdMap
- !Ref EnvironmentName
- defaultAMI

Thus Same way, we have called mapping values under InstanceType's value.
InstanceType: !FindInMap
- InstanceTypeMap
- !Ref EnvironmentName
- defaultInstanceType

Within next post i will cover mappings with Mappings with Pseudo parameters.





Read more ...

AWS CloudFormation : AWS Specific Parameter Types



AWS Cloud Formation Parameters.


Parameters section within CloudFormation template is way to gather inputs from user, which can be used within other sections of entire cloudformation script.

Parameter Type
Parameter Type plays a very important, it enable CloudFormation to validate inputs earlier in the stack creation process, thus it is a way cloudformation can validate your input based upon type before even instantiating stack creation.

Here are the valid Parameter types supported by AWS CloudFormation

TypeDetails
String Can be used to validate normal string.
Number FAn integer or float
List<Number>RAn array of integers or floats
CommaDelimitedListHAn array of literal strings that are separated by commas
List<AWS::EC2::SecurityGroup::Id>Array of security group IDs
AWS::EC2::KeyPair::KeyNameAn Amazon EC2 key pair name
AWS::EC2::SecurityGroup::IdA security group ID
AWS::EC2::Subnet::Idsubnet ID
AWS::EC2::VPC::IdVPC ID
List<AWS::EC2::VPC::IdAn array of VPC IDs
List<AWS::EC2::Subnet::Id>An array of subnet IDs

Parameter section Example.



Parameters:
  EnvironmentName:
    DescriptionSelect the environment.
    TypeString
    Defaultdev
    AllowedValues:
      - dev
      - prd
  EC2InstanceType:
    TypeString
    Defaultt2.micro
    AllowedValues:
        - t2.micro
        - t2.small
  EC2KeyName:
    DescriptionSelect your Key name from List.
    TypeAWS::EC2::KeyPair::KeyName




Read more ...

AWS Cloudwatch : How to create CloudWatch Rule and Schedule Lambda Function Using it.



AWS CloudWatch


Amazon CloudWatch is a monitoring and management service that provides data and actionable insights for AWS, hybrid, and on-premises applications and infrastructure resources. With CloudWatch, you can collect and access all your performance and operational data in form of logs and metrics from a single platform.


CloudWatch Rule : Create rules to invoke Targets based on Events happening in your AWS environment.

So this tutorial is just a walk through to
  • How to create a Cloudwatch rule using AWS Console
  • How to create a Cloudwatch rule and invoke lambda function using Rule scheduler and Cron expression.
  • What is Constant Keys and what is it's use while we use Lambda function as Target.

What we are trying to achieve here : 
  • We have a Lambda function named as "Check_File" and it's function is to check existence of file.
  • File is dynamic type : Format --> YYMMDDYdemo-file-A.txt
  • Thus this part of file name demo-file-A.txt  is treated as file_suffix.
  • So we will be using Constant keys to send the required value, which will be used as EVENTS within our Lambda Function.
  • Our dynamic file gets uploaded everyday between 11:30 to 12:00, thus we will trigger Lambda Function via Cloudwatch rule with Custom cron to check file existence.
Lab Setup details

  • AWS Lambda Function name : Check_File
  • AWS Lambda Function Task : Checks existence of file uploaded daily between 11:30 to 12:00
  • Cron Timing : Every Five minute between 11:30 to 12:00 GMT
  • Constant Keys : { "bucket_sub_directory": "reports/", "file_suffix": "demo-file-A.txt", "bucket_name": "cyberkeeda-bucket-a", "fileType": "daily"}

Steps to create Cloudwatch Rule via AWS Console.

  • AWS Console --> Cloudwatch --> Rules --> Create Rule
  • Enter the required details.
    • Event Source : 
      • Schedule --> Custom Cron --> 30,35,40,45,50,55,59 11 ? * * *
      •  Target --> Lambda Function --> Check_File
      • Constant Keys --> { "bucket_sub_directory": "reports/", "file_suffix": "demo-file-A.txt", "bucket_name": "cyberkeeda-bucket-a", "fileType": "daily"}
  • Configure Rule details :
    • Name : Cloudwatch Rule name
    • Description : Describe about it and Click on Create Rule.
Note : Replace the above Inputs with your own.


    We are done with the creation of Cloudwatch Rule via Admin Console.

Verify Cloudwatch logs to confirm if scheduled lambda executed.


Cloudwatch Rule via CloudFormation Script.

Use the below AWS CFN template to create above cloudwatch rule via AWS Cloudformation.


AWSTemplateFormatVersion: 2010-09-09
DescriptionCFN Script to create Lambda Function and a CloudWatch Rule with cron to trigger it.
Parameters:
  LambdaFunctionNameARNParam:
    TypeString
    DescriptionProvide Lambda Function ARN Parameter. [a-z][a-z0-9]* 
  CloudwatchRuleNameParm:
    TypeString
    DescriptionProvide Cloudwatch Rule name. [a-z][a-z0-9]*

Resources:
  ScheduledRule
    TypeAWS::Events::Rule
    Properties
      Description!Sub "Scheduled Rule created to invoke Lambda Function: ${LambdaFunctionNameARNParam}"
      Name!Sub ${CloudwatchRuleNameParm}
      ScheduleExpressioncron(0/5 11-12 ? * * *)

      State"DISABLED"
      Targets
      - IdLambda1
        Arn!Sub ${LambdaFunctionNameARNParam}
        Input'{ "bucket_sub_directory": "reports/",
                   "file_suffix": "demo-file-A.txt",                    "bucket_name": "cyberkeeda-bucket-a",
                   "fileType": "random"
                  }'

  PermissionForEventsToInvokeLambda
    TypeAWS::Lambda::Permission
    Properties
      FunctionName!Sub ${LambdaFunctionNameARNParam}
      Action"lambda:InvokeFunction"
      Principal"events.amazonaws.com"
      SourceArn!GetAtt ScheduledRule.Arn



Read more ...

AWS SNS : How to create SNS Topic and add email Subscribers to it.



AWS SNS


Amazon Simple Notification Service (SNS) is a highly available, durable, secure, fully managed pub/sub messaging service that enables you to decouple microservices, distributed systems, and serverless applications. 
Additionally, SNS can be used to trigger notification end users using mobile push, SMS, and email.
Source : Official


Within this post, we will cover.

  • How to create SNS Topic and attach Email Subscribers to it via AWS Console.
  • How to create SNS Topic and attach Email Subscribers to it via  AWS CloudFormation script. 

We will start with the use case:

  • If you want to trigger email notifications coupled to any AWS services like AWS lambda function.
  • If you want to fan out AWS lambda function via SNS.

Steps:

Create SNS Topic using AWS Console.
  • AWS Console -->  SNS --> Create Topic --> Fill detail and Create Topic.
    • Topic Name : Identifier of topic name, could be anything.
    • Display Name : This one is important, when Notification email will land to your email this will be the name for the SNS email, so keep so that it will give you a hint about the notification content.
Once created we will get a Success Notification.




Attach email subscribers to SNS Topic and Verify the Subscription.
  • Under above created SNS topic, scroll down and Click on Create Subscription
  • Fill in the below required details and create Subscriptions.
    • Topic ARN --> Selected by default.
    • Protocol --> Email.
    • Endpoint --> Email address of recipient, if you want notification to your inbox put your email id.
  • Verify Subscription --> You will get a AWS Notification - Subscription Confirmation email.
  • Click on the link to confirm subscription.
  • With respect to filled input within screenshot, something similar you can expect.
  • SNS Topic ARN can be used later to send email notifications or fan out Lambda.
Note : Every time you want to send to notification to new email address, repeat the above process to add Subscriber under same SNS Topic.





Yeah above are the steps to create SNS topic and add subscribers via Console, below is the AWS CloudFormation Script to SNS topic and add email Subscribers to it.

AWS CloudFormation Script.


AWSTemplateFormatVersion: 2010-09-09
DescriptionCFN to create SNS topic and Add two different email Subscribers to it. 
Parameters:
 
  SNSTopicName:
    TypeString
    DescriptionProvide SNS Topic Name
  SNSTopicDisplayName:
    TypeString
    DescriptionProvide SNS Topic's Display Name, will be used as a name for SNS emails.
  SubscriptionEmail:
    TypeString
    DescriptionProvide DL or email for your SNS Topic.

Resources:
  SNSTopic:
    TypeAWS::SNS::Topic
    Properties:
      TopicName!Sub ${SNSTopicName}
      DisplayName!Sub ${SNSTopicDisplayName}
      Subscription:
      - Endpoint!Sub ${SubscriptionEmail}
        Protocolemail

In case if we want to add more email subscribers, we can append more -Endpoint and Protocol as email to it.

Below one is the hard coded Resource for SNS topic with two email subscribers,  


Resources:
  SNSTopic:
    TypeAWS::SNS::Topic
    Properties:
      TopicNamemySNSTopic
      DisplayNameSNS Admin
      Subscription:
      - Endpointadmin@cyberkeeda.com
        Protocolemail
      Subscription:
      - Endpointblogger@cyberkeeda.com
        Protocolemail

    Read more ...
    Designed By Jackuna