CyberKeeda In Social Media
Showing posts with label Simple Storage Service. Show all posts
Showing posts with label Simple Storage Service. Show all posts

AWS S3 - Cross accounts copy data from one bucket to another.

Within this post, we will cover.

  • How to allow data copy from AWS Cross account S3 Bucekts.
  • Data from Bucket existing with one account can copy data to s3 bucket lying in another AWS account.

Setup is exactly similar to our last blog post : Link 

We have two different bucket and two files under those bucket within different AWS Accounts.
  • Bucket 1 name : cyberkeeda-bucket-account-a --> demo-file-A.txt
  • Bucket 2 name : cyberkeeda-bucket-account-b -> demo-file-B.txt


We will start by creating a bucket on Account B and modifying few things to allow our source bucket account owner to give access to our destination bucket.

We will assume we already have a bucket on account B, with all the public access to bucket denied, so we need to modify/add below changes within destination bucket Permission tab.

Below all modifications, we are doing at our destination account - B 
  •  Modify Public Access Rights : S3 --> choose your destination bucket --> Permission tab --> Click on Block Public Access --> Edit.
    • Uncheck : Block Public Access
    • Check : Block public access to buckets and objects granted through new access control lists (ACLs)
    • Check : Block public access to buckets and objects granted through any access control lists (ACLs)
    • Check : Block public access to buckets and objects granted through new public bucket or access point policies
    • Uncheck : Block public and cross-account access to buckets and objects through any public bucket or access point policies
  • In the above manner we are blocking every public access except for AWS Cross accounts.
  • Add Bucket Policy to allow read, write access to Account A:
    • S3 --> choose your destination bucket --> Permission tab --> Click on Block Policy --> Add the below lines.
    • Replace the AWS Account number with your source bucket owner account number, here our source account is for Account-A number.
    • And bucket with the destination bucket name, here our destination bucket name (cyberkeeda-bucket-account-b)
    • Update the variables Source Account number and Destination bucket name and save it.
{
    "Version": "2012-10-17",
    "Id": "Policy1586529665189",
    "Statement": [
        {
            "Sid": "SidtoAllowCrossAccountAccess",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::387789623977:root"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::cyberkeeda-bucket-account-b",
                "arn:aws:s3:::cyberkeeda-bucket-account-b/*"
            ]
        }
    ]
}

We are done with all required changes with Destination Bucket Account B, now lets move and do the needful at account A.

All below changes are made at Account -A ( Source Account )

Link for Cloudformation script : Link
Use the above cloudformation script to create instance based IAM role and replace the destination bucket with bucket name of Account B.

  • Stack Name : Name of the stack ( Could be anything )
  • Source Bucket name : Name of the bucket where we want to copy data from our source bucket, Account A bucket name (cyberkeeda-bucket-account-A)
  • Destination Bucket name : Name of the bucket where we want to copy data from our source bucket, Account B bucket name (cyberkeeda-bucket-account-b)
  • Role Name : Name of your IAM role ( Could be anything )
  • Inline Policy : Name of your policy, which will allow list,get,put object permission to buckets ( Could be anything )
  • Once Stack is created, follow the same process to attach IAM role to instance and after that we can use aws CLI commands as (LS,CP,SYNC)

Note
  1. This is really important stuff to share, whenever we copy any data/object from source s3 bucket to destination bucket while in Cross account, use sync --acl bucket-owner-full-control.
  2. This is mandatory else you can copy but the destination bucket owner will be unable to view/download any uploaded file/object from source account.

Now use the below AWS CLI command to Sync all file/content from one bucket to another with ACL as bucket owner.

 aws s3 sync --acl bucket-owner-full-control s3://cyberkeeda-bucket-account-A/  s3://cyberkeeda-bucket-account-B/

You can see a stream of data copying as an STDOUT after command is executed.



Read more ...
Designed By Jackuna