CyberKeeda In Social Media

AWS Cloudformation template to create Cloudwatch Event rule to trigger ECS Task

                             


Cloudformation Template that will created below resources.

  • IAM role for ECS Task and Cloudwatch rule.
  • CloudWatch schedule rule ( cron ) to trigger task defination.


Template

AWSTemplateFormatVersion: 2010-09-09
Description: | 
              1. IAM Role to be used by ECS task and cloudwatch event rule.
              2. CloudWatch Rule to trigger ecs tasks.
             
Parameters:
  ProductName:
    Description: Parent Product name.
    Type: String
    Default: cyberkeeda
  ProjectName:
    Description: Project Name
    Type: String
    Default: cyberkeeda-report
  Environment:
    Description: The equivalent CN name of the environment being worked on
    Type: String
    AllowedValues:
      - dev
      - uat
      - qa
  Region:
    Description: Ck Region specific parameter
    Type: String
    AllowedValues:
      - mum
      - hyd
  ECSClusterARN:
    Description: ECS Cluster ARN to schedule Task 
    Type: String
    Default: None
  CWEventRuleCron:
    Description: Cron Expression to schedule ECS task. 
    Type: String
    Default: "cron(0 9 * * ? *)"
  ECSTaskDefARN:
    Description: ARN for ECS Task defination
    Type: String

Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - 
        Label:
          default: Project based details
        Parameters:
          - ProductName
          - ProjectName
          - Environment
          - Region
      - 
        Label:
          default: ECS details.
        Parameters:
          - ECSClusterARN
          - ECSTaskDefARN
          - CWEventRuleCron
      
Resources:
  ExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-role"
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: [ 'ecs-tasks.amazonaws.com', 'events.amazonaws.com' ]
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
      Policies:
      - PolicyName: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-role-inlinePolicy"
        PolicyDocument: 
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                - ecs:RunTask
                Resource:
                - !Sub "${ECSTaskDefARN}:*"
              - Effect: Allow
                Action: iam:PassRole
                Resource:
                - "*"
                Condition:
                  StringLike:
                    iam:PassedToService: ecs-tasks.amazonaws.com
  TaskSchedule:
    Type: AWS::Events::Rule
    Properties:
      Description: Trigger Cyberkeeda Daily ECS task
      Name: !Sub  "${ProductName}-${Region}-${Environment}-${ProjectName}-daily-event-rule"
      ScheduleExpression: !Ref CWEventRuleCron
      State: ENABLED
      Targets:
        - Id: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-daily-event-rule-targetId"
          EcsParameters:
            LaunchType: EC2
            TaskDefinitionArn: !Ref TaskDefinition
            TaskCount: 1
          RoleArn:
            Fn::GetAtt:
            - ExecutionRole
            - Arn
          Arn: !Ref ECSClusterARN

Let me know, for any questions in comment box.

Read more ...

AWS Cloudformation template to create ECS Task definition.

 



Cloudformation Template that will created below resources.

  • IAM role for ECS Task execution
  • ECS Task definition


Template

AWSTemplateFormatVersion: 2010-09-09
Description: | 
              ECS Task is responsible to fetch files from sftp location.
              1. IAM Role to be used by ECS task and cloudwatch event rule.
              2. ECS Task defination with container env variables, please note credential needs to be created first within parameter store.
             
Parameters:
  ProductName:
    Description: Parent Product name.
    Type: String
    Default: cyberkeeda
  ProjectName:
    Description: Project Name
    Type: String
    Default: cyberkeeda-report
  Environment:
    Description: The equivalent CN name of the environment being worked on
    Type: String
    AllowedValues:
      - dev
      - uat
      - qa
  Region:
    Description: Ck Region specific parameter
    Type: String
    AllowedValues:
      - mum
      - hyd
  ECSTaskDefARN:
    Description: ARN for ECS Task defination
    Type: String
  SFTPHostFQDN:
    Description: Remote SFTP Host FQDN.
    Type: String
    Default: 123.111.11.1
  SFTPHostPort:
    Description: Remote SFTP Host Port.
    Type: String
    Default: 22
  SFTPUserName:
    Description: Remote SFTP Host username.
    Type: String
    Default: sftpadmin
  SFTPPasswordParameterStoreName:
    Description: Remote SFTP Host Parameter store name.
    Type: String
    Default: sftppass
  ContainerImageUrlwithTag:
    Description: Container Image URL with tag.
    Type: String
    Default: docker.io/jackuna/sftpnew
  ECSClusterARN:
    Description: ECS Cluster ARN to schedule Task 
    Type: String
    Default: arn:aws:ecs:ap-south-1:895678824142:cluster/sftp

Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - 
        Label:
          default: CK Project Details
        Parameters:
          - ProductName
          - ProjectName
          - Environment
          - Region
      - 
        Label:
          default: Remote SFTP Server details used as Container Environment Variables.
        Parameters:
          - SFTPHostFQDN
          - SFTPHostPort
          - SFTPUserName
          - SFTPPasswordParameterStoreName
      
Resources:
  ExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-role"
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: [ 'ecs-tasks.amazonaws.com', 'events.amazonaws.com' ]
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
      Policies:
      - PolicyName: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-role-inlinePolicy"
        PolicyDocument: 
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                - ssm:GetParameters
                Resource:
                - !Sub "arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/${Environment}.sftp-password" 
              - Effect: Allow
                Action:
                - ecs:RunTask
                Resource:
                - !Sub "${ECSTaskDefARN}:*"
              - Effect: Allow
                Action: iam:PassRole
                Resource:
                - "*"
                Condition:
                  StringLike:
                    iam:PassedToService: ecs-tasks.amazonaws.com
  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-ecs-task"
      Memory: 128
      NetworkMode: bridge 
      ExecutionRoleArn: !Ref ExecutionRole
      TaskRoleArn : !Ref ExecutionRole
      ContainerDefinitions:
        - Name: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-container"
          Image: !Ref ContainerImageUrlwithTag
          Memory: 128
          Cpu: 0
          MountPoints: 
            - 
              SourceVolume: "ecs-logs"
              ContainerPath: "/var/log/ecs"
          Command: 
            - python
            - sftp_python.py
          WorkingDirectory: "/usr/local/aws-swa"
          Secrets:
            - 
              Name: SFTP_PASSWORD
              ValueFrom: !Sub ${CNEnvironment}.sftp-password
          Environment: 
            - 
              Name: APPLICATION_LOGS
              Value: !Sub  "/var/log/ecs/${ProductName}-${Region}-${Environment}-${ProjectName}-ecs-task.logs"
            - 
              Name: SFTP_HOST
              Value: !Ref SFTPHostFQDN
            - 
              Name: SFTP_PORT
              Value: !Ref SFTPHostPort
            - 
              Name: SFTP_USERNAME
              Value: !Ref SFTPUserName

      RequiresCompatibilities:
        - EC2
      Volumes: 
        - 
          Host: 
            SourcePath: "/var/log/ecs"
          Name: "ecs-logs"

Let me know, for any questions in comment box.

Read more ...

Python Encode and Decode string using BASE64 module

 


base64 is a python library that can be used to encrypt and decrypt strings and characters, that can have a multiple use case.

One common use case is instead of directly pasting a plain text credential parameters into a file or as a parameter and that can be later decrypted using the decode statements within the program.

Within this blog post, we will cover.
  • How can we encrypt strings using base64
  • How can be decrypt the above base64 encrypted string using bas64 decoder.

Please note the two important points before we use this module.

  • base64 encode and decode functions both require a bytes-like object. In order to get our string into bytes, we must encode it first using Python's built in encode function. Most commonly, the UTF-8 encoding is used.
  • Encryption of same string using Linux command line interface and python shell differs, please use the same environment for both the encryption and decryption.

Here in this example, we will encrypt our string "cyberkeeda@123" and later we will decrypt it.

Encryption

# Encryption Block 

import base64
base64.b64encode(bytes("cyberkeeda@123", "utf-8"))
Output for the above.
b'Y3liZXJrZWVkYUAxMjM='


Decryption

Below code can be use to decrypt the above, please use only the string content enclosed within string to decrypt, so for the above example output consider string leaving the b ( byte ) identifier.
# Decryption Block 

import base64
base64.b64decode("Y3liZXJrZWVkYUAxMjM=").decode("utf-8")

Output for the above.
'cyberkeeda@123'

Hope this small piece of snippet will help you in some context.




Read more ...

AWS S3 Bucket Policy to grant access to other AWS account

 



AWS Bucket Policy to be used for the below requirements.

  • Grant access of S3 Bucket to other AWS account.
  • Restrict access to List and Download objects from it, nothing more nothing extra.

Script to extract yesterday date

{
"Sid": "Allow Bucket Read access from below AWS accounts", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::123456789012:root", "arn:aws:iam::121314151617:root", "arn:aws:iam::181912021222:root" ] }, "Action": [ "s3:Get*", "s3:List*" ], "Resource": "arn:aws:s3:::cyberkeeda-limited-access-bucket/*" } ] })


Hope this snippet, helps you !
Read more ...

How to get yesterday's date using Python timedelta

 

How to use datetime module to get yesterday's date.

All of us use Python's date time library for multiple date/time task, there is an additional function named as timedelta that can be used to get previous date as per requested varaible.

Let's go through it :)

Script to extract yesterday date

# Python script to get yesterday date.
from datetime import date, timedelta # Block to get present day data today = date.today() print("Today's date : ", today.strftime('%Y-%m-%d')) # Block to get yesterday data yesterday = today - timedelta(days = 1) print("Yesterday's date : ", yesterday.strftime('%Y-%m-%d'))

Output


Today's date : 2021-06-29
  Yesterday's date : 2021-06-28


Same way, we can change the timedelta to n number of day ( day = n )
Below, we will extract the 4 days old date


Script to extract yesterday date

# Python script to get yesterday date.
from datetime import date, timedelta # Block to get present day data today = date.today() print("Today's date : ", today.strftime('%Y-%m-%d')) # Block to get yesterday data yesterday = today - timedelta(days = 4) print("Yesterday's date : ", yesterday.strftime('%Y-%m-%d'))

Output


Today's date : 2021-06-29
  Yesterday's date : 2021-06-25


Read more ...

How to enable password based ssh authentication in ec2 instance

 



EC2 Linux SSH Authentication.

By default, preferred and default way of accessing any ec2 linux instance is key based authentication.
Here in this blog post, we will know 
  • How to enable basic authentication that is password based authentication in ec2 instance.
  • How to enable root login to ec2 instance.
I will keep on updating the post as per my learnings and used in practical scenarios.

Let's go through it :)

How to enable root login on linux ec2 instance.
  • Login to ec2 linux instance using it's private key.
  • Sudo to root
  • change password for root
  • Permit root login in sshd_config file

Syntax

[ec2-user@ip-10-0-1-116 ~]$ sudo su

Change root password from below command.


[root@ip-10-0-1-116 ec2-user]# passwd root

Permit root login by un-commenting the below line in sshd_config


[root@ip-10-0-1-116 ec2-user]# vi /etc/ssh/sshd_config

From


# PermitRootLogin yes

To

PermitRootLogin yes


How to enable password based authentication for ssh user.
  • Login to ec2 linux instance using it's private key.
  • Sudo to root
  • Permit root login in sshd_config file
  • Restart sshd service

Syntax

[ec2-user@ip-10-0-1-116 ~]$ sudo su

Permit root login by un-commenting the below line in sshd_config


[root@ip-10-0-1-116 ec2-user]# vi /etc/ssh/sshd_config

From


# PasswordAuthentication yes

To

PasswordAuthentication yes

Restart SSHD service

service sshd restart


Login and check !
Read more ...

idrac default password

racadm


The exceptionally One need to install srvadmin package to run racadm cli commands to use it further.

Once installed, we can use below command to find IP address details for IDRAC from installed base operating system.

Use racadmin utility to confirm IDRAC ip address, once you have this use default password.



root#  racadm getniccfg
NIC Enabled = 1
DHCP Enabled = 0
IP Address = 10.10.12.34
Subnet Mask = 255.255.255.0
Gateway = 10.10.12.1

Here IP address resembles to the configured IDRAC IP, use it to access installed server remotely. 

Deafault User : root
Default Password : calvin 
Read more ...

How to use, Python's DateTime Module

 


Python's Date Time Module:
    Python's datetime module ships with default python package which you can't skip for sure sooner or later you will come to know about it.
This module will help you to verify multiple checks and algorithms based upon date time filter.

I will keep on updating the post as per my learnings and used in practical scenarios.
Let's go through it :)

Basic Usage of date time
  • Import module and display current date.

Syntax
# Import datetime function from datetime library
from datetime import datetime
datetime.datetime(2021, 6, 7, 2, 43, 32, 276412)

# Create object and store value##
  dt = datetime.now()
# Display date, time without Formatting
print(dt)

Output:

datetime.datetime(2021, 6, 7, 2, 43, 32, 276412)


Filter datetime output as per our requirement.
  • Filter date

Syntax
# dt = datetime.now()
# show_date = dt.date()

# print (show_date)

Output:

datetime.date(2021, 6, 7)

Filter only Day

# dt = datetime.now()
# show_day = dt.day

# print (show_day)

Output:

7

  • Filter Month

Syntax
# dt = datetime.now()
# show_month = dt.month

# print (show_month)

Output:

6

Use datetime strftime to format date/time as per our need

    Below are some identifier to use while formatting.
    • %a  : Day of week (Short )
      • Mon, Tue, Wed, ... Sat, Sun
    • %A : Day of week (Full )
      • Monday, Tuesday, Wednesday .. Saturday, Sunday
    • %m : Month  ( Two digit number )
      • 01,02,03,04 .... 10,11,12
    • %y : Year ( Short, Two digit number )
      • 19,20,21
    • %Y : Year ( Full)
      • 2019,2020,2021
    • %d : Date ( Two digit number )
      • 01,02,03,04, 22,23,30,31

    Use Strftime as per our need.
    • With or Without Variables


    Syntax
    # dt = datetime.now()
    # show_val1 = dt.strftime('%m%d%y')
    # show_val2 = dt.strftime('%m%d%Y')
    # show_val3 = dt.strftime('%Y%m%d')
    # show_val4 = dt.strftime('%a %m %y')
    # show_val5 = dt.strftime('%A-%m-%Y')
    # show_val6 = dt.strftime('%I %p %S')
    # show_val7 = dt.strftime('%-j')
    # show_val8 = dt.strftime('myprefix-%m%d%y-mysuffix')

    # print (show_val1)
    # print (show_val2)
    # print (show_val3)
    # print (show_val4)
    # print (show_val5)
    # print (show_val6)
    # print (show_val7)

    Output:

    060721
    06072021
    20210607
    Mon 06 21
    Monday-06-2021
    02 AM 32
    158
    myprefix-060721-mysuffix


    There is lot of content with respect to datetime module, will keep this post updated ;)
    Read more ...

    Linux : Create dummy file of any size for test purpose using fallocate

    Requirement :
    Create dummy/fake file of desired size using Linux terminal
        There might be some test requirement to mimic the prod setup, in my case I have to write a script to validate files before downloading, instead of making request to prod site I just created a local setup and placed similar dummy files to replicate the prod environment.
    Let's know the one linux one liner using tool fallocate to create dummy file of desired size.

    • Fallocate is the Linux terminal utility.

    One Liner
    # fallocate -l <size_of_file> <desired_name_of_file>

    • Fallocate is the Linux terminal utility.

    Syntax Template

    # fallocate -l 15M myfile.img

    • <size_of_file >
      • in GB and MB : M for MB and G for GB
      • example : 15M for 15 MB and 5G for 5GB
    • <desired_name_of_file> 
      • It can be anything with our without extension : myfile.tar or myfile.img or myfile or anything

    Using Linux for loop and Seq to generate a series of file.
    Example : 
    • Below one liner can be used to generate 100 files of 15mb each with suffix changes as file count number.

    Syntax Template

    # for val in `seq 100`; do fallocate -l 15M demo060621000$val.tar; done;

    Output : File starts with name 
    demo0606210001.tar and ends with demo0606210099.tar

    Syntax Output

    # ls -ltr

    -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo0606210001.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo0606210002.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo0606210003.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo0606210004.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo0606210005.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo0606210006.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo0606210007.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo0606210008.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo0606210009.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100010.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100011.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100012.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100013.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100014.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100015.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100016.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100017.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100018.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100019.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100020.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100021.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100022.tar -rw-r--r-- 1 root root 15728640 Jun 6 23:25 demo06062100023.tar

    Read more ...
    Designed By Jackuna