How to restrict AWS S3 Content to be accessed by CloudFront distribution only.

 

CloudFront is one of the popular services of AWS that gives Caching mechanism for our static contents like html, css, images and media files serving a very fast performance using it's globally available CDN networks of POP sites.

In this blog post, we will know 

• How to create a basic CloudFront distribution using S3 as Origin.
• How can we create a CloudFront distribution using S3 as Origin without making the Content of Origin(s3 Objects) public.
• What, Why and How about CloudFront OIA.

Here in this scenario, we will be using S3 bucket as an Origin for our CloudFront distribution

We will understand the problem first and then know, how Origin Access Identity can be used to address the request.

So we have quickly created a S3 bucket and CloudFront distribution using default settings with below details.

• S3 bucket name - s3-web-bucket
• Bucket Permissions - Block all Public Access
• CloudFront distribution default object - index.html
• CloudFront Origin - s3-web-bucket

Now, quickly upload a index.html file under the root of s3 bucket as s3-web-bucket/index.html.

We are done with the configuration, let's try to quickly access the CloudFront distribution and verify if everything is working perfectly or not.

$ curl -I https://d2wakmcndjowxj.cloudfront.net

HTTP/2 403
content-type: application/xml
date: Thu, 14 Jul 2022 07:28:37 GMT
server: AmazonS3
x-cache: Error from cloudfront
via: 1.1 ba846255b240e8319a67d7e11dc11506.cloudfront.net (CloudFront)
x-amz-cf-pop: MRS52-P4
x-amz-cf-id: BbAsVxxWfW9v3m1PD2uBHqRIj_7-J5U3fUzhhFiQQhbJj8a7lQlCvw==

We encountered 403 error, why ?
Ans : This is expected as we have kept the bucket permission level as Block All Public Access.

Okay, then let's modify the bucket permission and Allow Public Access, for this follow the below two steps.

• Enable Public Access from Console by unchecking the Check box "Block all public access" and Save it.

• Append the below Bucket Policy JSON statement to make all objects inside the Bucket as Public, the one highlighted in red can be replaced by your own Bucket name.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AddPerm",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::s3-web-bucket/*"
        }
    ]
}

• Save it, and your bucket permission section will appear with Red Amber signifying that your bucket is publicly accessible.

Done, Now let's try again to access the Website (index.html) from our CloudFront distribution.

$ curl -I https://d2wakmcndjowxj.cloudfront.net

HTTP/2 200
content-type: text/html
content-length: 557
date: Thu, 14 Jul 2022 07:47:58 GMT
last-modified: Wed, 13 Jul 2022 18:50:58 GMT
etag: "c255abee97060a02ae7b79db49ed7ec1"
accept-ranges: bytes
server: AmazonS3
x-cache: Miss from cloudfront
via: 1.1 ba055a10d278614dad75399031edff3c.cloudfront.net (CloudFront)
x-amz-cf-pop: MRS52-C2
x-amz-cf-id: Bhf_5IjA0sifp7jON4dpzZdjpCZCQTF5L7c5oenUbjc1vZzvL6ZUWA==

Good, we are able to access our webpage and now our static contents will be served from CDN network, but wait let's try to access the object(index.html) from bucket's S3 URLs too.

$ curl -I https://s3-web-bucket.s3.amazonaws.com/index.html

HTTP/1.1 200 OK
x-amz-id-2: OgLcIIYScHdVok2puZb09ccCjU5K9xNxOL6D1sVj/nBf6hm93vCjQQSpm3fxo4tXpdjUa3u2TS0=
x-amz-request-id: 588WXNR2BH9F37R9
Date: Thu, 14 Jul 2022 07:50:42 GMT
Last-Modified: Wed, 13 Jul 2022 18:50:58 GMT
ETag: "c255abee97060a02ae7b79db49ed7ec1"
Accept-Ranges: bytes
Content-Type: text/html
Server: AmazonS3
Content-Length: 557

Here is the loophole, naming standards for any s3 bucket urls and it's respective objects are quite easy to guess if one knows the name of the bucket only.

User, developer and hackers can bypass the CloudFront url and can get access to Objects directly from S3 Urls only, but you may think or what's the issue as they are anyhow public read in nature by permissions.

So to answer these questions, here are some points I would like to point, how accessing content via CloudFront URls is useful

• CloudFront URLs give you better performance.
• CloudFront URL can provide Authentication mechanism.
• CloudFront URL gives additional possibilities to trigger CloudFront Function, which can be used for custom solutions.
• Sometimes content of a website/API is designed to be served via CloudFront only, accessing it from S3 gives you a portion of it's content.

These are few counter points, but there are many more to support why should you disable public access to your s3 buckets.

Origin Access Identity ( OIA ) 

The above concern can be sorted out using OIA feature of CloudFront, where we can restrict the access of a S3 bucket from a CloudFront OIA user only.

To achieve it, here are the steps.

• First thing First, incase you have Public Access Enable on your bucket and it's object. ( In case, if it's a fresh bucket with default permission, you can skip this step)
• Disable Public Access to your bucket and it's object from S3 console.
• Remove the Bucket Policy which we have enabled to support public access.

• Create a  CloudFront user called an origin access identity (OAI) and associate it with the CloudFront distribution.
• Configure your S3 bucket permissions so that CloudFront can use the OAI to access the files in your bucket and serve them to your users.

We can configure both the above steps from AWS console.

Go to Distribution -- Select your distribution -- Click on Origin Tab --> Select your Origin -- Edit.

Follow the below order.

1. Check Use OIA
2. Create new OIA
3. Yes, Update the Bucket Policy
4. Save

And we are done, we can confirm the same by verifying the access and Looking into the Bucket Policy Section, it must be updated by something like below.

{
    "Version": "2008-10-17",
    "Id": "PolicyForCloudFrontPrivateContent",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E1WON59VMYUAH3"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::s3-web-bucket/*"
        }
    ]
}

Now Check again using the Curl Command, if we have access to the objects using CloudFront URLs using updates settings.

But before that, Invalidate the Cache from all POP location by creating the below Invalidation request,

Wait for the Invalidation to complete, once completed verify the website status.

$ curl -I https://d2wakmcndjowxj.cloudfront.net

HTTP/2 200
content-type: text/html
content-length: 557
date: Thu, 14 Jul 2022 07:47:58 GMT
last-modified: Wed, 13 Jul 2022 18:50:58 GMT

 curl -I https://s3-web-bucket.s3.amazonaws.com/index.html

HTTP/1.1 403 Forbidden
x-amz-request-id: 0SEWGEWRKXK7JS81
x-amz-id-2: 4iuN/FLtT6vEwq5WoE81dZTUNCvKRb0iDCmyOwmGWpTuL7yYu+jcgGrAGvuJO0wBT4i61rG3Lco=
Content-Type: application/xml
Date: Thu, 14 Jul 2022 08:23:51 GMT

We can see from the above Curl requests, we are able to retrieve our website content using CloudFront URLs only, forcing a user to go through it only.

Hope this helps you in some sort.