tag:blogger.com,1999:blog-16920341557973318832024-03-18T08:33:27.460+05:30CyberKeedaA complete Blog for Cyber addicts.shailhttp://www.blogger.com/profile/05532156483435343917noreply@blogger.comBlogger341125tag:blogger.com,1999:blog-1692034155797331883.post-5683720838783855162023-03-18T19:56:00.005+05:302023-03-18T21:05:07.487+05:30Most used AWS S3 Bucket Policies.<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIleRhJIkuWFc3vzlHZ9SRzbzlQFJnhW1B1NciEW2T4WHfAwVYSTqX2-6e-H7HJ84lKt2K4Vb5ROgPb0NVBWHMnhAnhLx0mgTyfEv_LZSInuMugLovaWenSEFh8EJUs7M2YVb1UGTIypl9meHGVJQX-RFwbBETO8LaFRCppOwZ97HXY-oCuvhi7M0/s1336/AWSBanner.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="715" data-original-width="1336" height="214" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIleRhJIkuWFc3vzlHZ9SRzbzlQFJnhW1B1NciEW2T4WHfAwVYSTqX2-6e-H7HJ84lKt2K4Vb5ROgPb0NVBWHMnhAnhLx0mgTyfEv_LZSInuMugLovaWenSEFh8EJUs7M2YVb1UGTIypl9meHGVJQX-RFwbBETO8LaFRCppOwZ97HXY-oCuvhi7M0/w400-h214/AWSBanner.png" width="400" /></a></div><br /><p></p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Bucket Policies are one of the key element when we have talk about security and compliance while using AWS S3 buckets to host our static contents.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">In this post, below are the some code snippets of most used bucket policy documents.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></p><h3 style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; text-align: left;">Policy 1 : Enable Public Read access to Bucket objects.</h3><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Turing OFF the Public Access Check from S3 bucket permission tab is not sufficient to enable public read access, additionally you need to add below bucket policy statement to enable.</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><b style="color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">
</b><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">{
"Version":"2012-10-17",
"Statement":[
{
"Sid":"EnablePublicRead",
"Effect":"Allow",
"Principal": "*",
"Action":["s3:GetObject"],
"Resource":["arn:aws:s3:::</span><span style="color: red;">ck-public-demo-bucket</span><span style="color: #2b00fe;">/*"]
}
]
}</span></b></span></span></pre><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></p><h3 style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Policy 2 : Allow only HTTPs Connections.</h3><div><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">AWS S3 allows both HTTP and HTTPS by default, in order to force clients to initiate only HTTPS connection, use below bucket policy document to force it.</span></div><div><br /></div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">{
"Id": "ExamplePolicy",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowOnlySSLRequests",
"Action": "s3:GetObject",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::</span><span style="color: red;">ck-public-demo-bucket</span><span style="color: #2b00fe;">/*"
],
</span><span style="color: red;">"Condition": {
"Bool": {
"aws:SecureTransport": "true"
}
},</span><span style="color: #2b00fe;">
"Principal": "*"
}
]
}</span></b></span></span></pre><div><h3 style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></h3><h3 style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Policy 3 : Allow access from a specific or range of IP address.</h3><div><br /></div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">{
"Version": "2012-10-17",
"Id": "AllowOnlyIpS3Policy",
"Statement": [
{
"Sid": "</span></b></span></span><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><span style="color: #2b00fe;">AllowOnlyIp</span></b><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><span style="color: #2b00fe;">",</span></b><br /><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;"> "Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::ck-public-demo-bucket",
"arn:aws:s3:::ck-public-demo-bucket/*"
],
</span><span style="color: red;">"Condition": {
"NotIpAddress": {"aws:SourceIp": "12.345.67.89/32"}
}</span><span style="color: #2b00fe;">
}
]
}</span></b></span></span></pre></div><div><br /></div><div><h3 style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Policy 4 : Cross Account Bucket access Policy.</h3><div><br /></div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::</span><span style="color: red;">REPLACE-WITH-YOUR-AWS-CROSS-ACCOUNT-NUMBER</span><span style="color: #2b00fe;">:root"
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::ck-public-demo-bucket/*"
]
}
]
}</span></b></span></span>
</pre></div><div><br /></div><div>Note : Do copy content from cross account with additional acl as bucket owner full control</div><div><br /></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>aws s3 cp demo-file.txt s3://ck-public-demo-bucket/ --acl bucket-owner-full-control</b></span></span></pre></div><div><br /></div>
Will keep on adding more..Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com4tag:blogger.com,1999:blog-1692034155797331883.post-91384800395938228192022-11-21T00:13:00.002+05:302023-03-18T21:08:22.026+05:30How to install Ansible on Ubuntu using Docker and Docker Compose <p> I was quite familiar with Ansible and has good worked on automating task using it, although I was not familiar with Ansible tower, have witnessed it dashboard some time but never worked directly with it.</p><p>Now, during my current I got a task of creating a automation using service now ticket.</p><p>This some how introduced me to Ansible API, Ansible Tower Webhooks and much more, during my exploration of Ansible tower and it's component I was somehow limited up to certain access, this insisted me to install Ansible AWX an Opensource version of Ansible Tower and I would again like to thank Redhat, for keeping an opensource version of it.</p><p><br /></p><p>The official release now doesn't support docker installation, instead they provide kubernetes installation guide, which for me personally is more hectic when it comes to testing and development.</p><p><br /></p><p>Here is the guide, I followed to install Ansible AWX on Ubuntu.</p><p><br /></p><p><a href="https://www.rogerperkin.co.uk/network-automation/ansible/how-to-install-ansible-awx/" target="_blank">External Link </a></p><p><br /></p><p>Don't forget to Thanks the author !</p>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-15720816504590606752022-09-11T17:37:00.004+05:302022-09-11T17:48:50.796+05:30IPV4 - Classes Range - Pictorial Representation<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzKjGaHgHoEGWKvBH6u6KkF_TiIECg0h2EtyKxrY0k3rrVPBmZHq2q0XmTi_JzwMPG4e4tWtzxAotfDt3vLJxSOnQyskdt3viUZBadBp9sl_qhGuXu-FEVGBsvcMOdc_Yfxi8I_IKq-6D90HdczyHHPh4J8JQb0Eq9ArW-ZM4fGzx8aviaDf-8Hyc/s721/network-calculator.drawio.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="481" data-original-width="721" height="426" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzKjGaHgHoEGWKvBH6u6KkF_TiIECg0h2EtyKxrY0k3rrVPBmZHq2q0XmTi_JzwMPG4e4tWtzxAotfDt3vLJxSOnQyskdt3viUZBadBp9sl_qhGuXu-FEVGBsvcMOdc_Yfxi8I_IKq-6D90HdczyHHPh4J8JQb0Eq9ArW-ZM4fGzx8aviaDf-8Hyc/w640-h426/network-calculator.drawio.png" width="640" /></a></div><br /><p></p><div><br /></div><div>BOGON IPs - Please note Bogon IPs are reserved for private network, thus can't be used by ISP providers to use the ranges to declare it as Public IP.</div><div><br /></div><div>Class A Bogon IP range.</div><div><ul style="text-align: left;"><li>CIDR - 10.0.0.0/8</li><li>Total IP - 32-8 = 24 </li><ul><li>2^24 = 16777216</li></ul></ul></div><div><div>Class B Bogon IP range.</div><div><ul><li>CIDR - 172.16.0.0/12</li><li>Total IP - 32-12 = 20 </li><ul><li>2^20 = 1048576</li></ul></ul></div><div><div>Class C Bogon IP range.</div><div><ul><li>CIDR - 192.168.0.0/16</li><li>Total IP - 32-16 = 16</li><ul><li>2^16 = 65536</li></ul></ul></div><div><br /></div></div></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-26900751497816596482022-09-11T17:12:00.003+05:302022-09-11T17:12:17.888+05:30Find IP Address Information Cheat Sheet by Example.<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgc3iryfN7EaloQhLRMimfwhZdl0Y9IAiVZEqRvu5y5D03KgJh8LwvE1ejpVYdyIFhlW0fJS3Lvxpl3UgW63TbaHg8cA6_KQfErLGCi6SZvw1zuA5C_cEzlN7QjwuSswk6OzNBdjFNnSj2Fjai66HdXQrYz7RuheeV-K2XoFaJbJRj2IZJ-PojfLHY/s721/ip-address-details.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="491" data-original-width="721" height="435" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgc3iryfN7EaloQhLRMimfwhZdl0Y9IAiVZEqRvu5y5D03KgJh8LwvE1ejpVYdyIFhlW0fJS3Lvxpl3UgW63TbaHg8cA6_KQfErLGCi6SZvw1zuA5C_cEzlN7QjwuSswk6OzNBdjFNnSj2Fjai66HdXQrYz7RuheeV-K2XoFaJbJRj2IZJ-PojfLHY/w640-h435/ip-address-details.png" width="640" /></a></div><br /><p></p>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-68514951382236455052022-07-28T12:46:00.004+05:302022-07-28T12:46:53.501+05:30How to scan IP addresses details on your network using NMAP<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRAvS9Ok-07T0fa6845MJdMKQpaVEbpr8OgBhJLDrb43_18s-AGS7E9Y8PqNakJjp86dceDavAjZa3eTHDEC3Xz8JwOdYEc_TAg9bkncq1rUNezp8YdeVaYpi2LnEgP7ZnRLkURn6OgEWZwkr1WBW8-MZbXYr6dsFBfrnNJSlpknPRcGVbCs1k588/s2048/terminal.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="2048" data-original-width="1792" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRAvS9Ok-07T0fa6845MJdMKQpaVEbpr8OgBhJLDrb43_18s-AGS7E9Y8PqNakJjp86dceDavAjZa3eTHDEC3Xz8JwOdYEc_TAg9bkncq1rUNezp8YdeVaYpi2LnEgP7ZnRLkURn6OgEWZwkr1WBW8-MZbXYr6dsFBfrnNJSlpknPRcGVbCs1k588/s320/terminal.png" width="280" /></a></div><br /><p></p><p>You know using Linux is a kind fun, think about a requirement and you can see a wide number opensource tools gives you wings to your idea, no hurdles just go with your goal, they all will support you..</p><p>I would like to share you, what made me search internet and write this blog post.</p><p>So within my Lab environment, it's a very frequent task to configure, update IP configuration of other virtual machines, so to tackle this task, I have already written an Ansible Role, which basically configures the IP address for the host which has existing dhcp address assigned to it.</p><p>Now still there are some information I need to provide ansible before I proceed to run the playbook and the information it needs is, I need to manually look for free IPs in my current network.</p><p>So I was curious how to scan my network for used and free IP addresses, thus I surfed the internet and found, my friendly network troubleshooting tool NMAP gives the insight about it.</p><p>Let's see what command can be used to find those details.</p><p>Using below one lines to search for used IPs within your network.</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; font-weight: bold; letter-spacing: 1px; white-space: pre-wrap;">$ </span><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>nmap <span style="color: red;">-sP</span> 192.168.29.0/24</b></span></span></pre><p>Output</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>
Starting Nmap 6.40 ( http://nmap.org ) at 2022-06-16 17:10 IST
Nmap scan report for 192.168.29.1
Host is up (0.0078s latency).
Nmap scan report for 192.168.29.9
Host is up (0.0050s latency).
Nmap scan report for 192.168.29.21
Host is up (0.0043s latency).
Nmap scan report for 192.168.29.30
Host is up (0.0015s latency).
Nmap done: 256 IP addresses (4 hosts up) scanned in 2.59 seconds</b></span></span><span style="color: #333333; font-size: 14px; letter-spacing: 1px;">
</span></pre><div><br /></div><p></p><p>Now let's scan again the same network and look for the listening ports along with the host ip</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; font-weight: bold; letter-spacing: 1px; white-space: pre-wrap;">$ </span><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>sudo nmap <span style="color: red;">-sT</span> 192.168.29.0/24</b></span></span></pre><p>Output</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>Starting Nmap 6.40 ( http://nmap.org ) at 2022-06-16 17:17 IST
Nmap scan report for 192.168.29.1
Host is up (0.0061s latency).
Not shown: 992 filtered ports
PORT STATE SERVICE
80/tcp open http
443/tcp open https
1900/tcp open upnp
2869/tcp closed icslap
7443/tcp open oracleas-https
8080/tcp open http-proxy
8200/tcp closed trivnet1
8443/tcp open https-alt
MAC Address: AA:HA:IC:PF:P3:C1 (Unknown)
Nmap scan report for 192.168.29.9
Host is up (0.0083s latency).
Not shown: 998 closed ports
PORT STATE SERVICE
80/tcp open http
554/tcp open rtsp
MAC Address: 14:07:o8:g5:7E:99 (Private)
Nmap scan report for 192.168.29.21
Host is up (0.0051s latency).
Not shown: 998 closed ports
PORT STATE SERVICE
22/tcp open ssh
80/tcp open http
MAC Address: 08:76:20:00:75:D5 (Cadmus Computer Systems)
Nmap scan report for 192.168.29.25
Host is up (0.0057s latency).
Not shown: 999 filtered ports
PORT STATE SERVICE
135/tcp open msrpc
MAC Address: F0:76:30:60:8E:21 (Unknown)
Nmap scan report for 192.168.29.30
Host is up (0.0018s latency).
Not shown: 997 closed ports
PORT STATE SERVICE
22/tcp open ssh
8000/tcp open http-alt
8080/tcp open http-proxy
Nmap done: 256 IP addresses (5 hosts up) scanned in 7.84 seconds</b></span></span>
</pre><div><br /></div><p>If you need additional details like Host OS details and some more, then run the scan again with below command</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; font-weight: bold; letter-spacing: 1px; white-space: pre-wrap;">$ </span><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>sudo nmap <span style="color: red;">-sT -O</span> 192.168.29.0/24</b></span></span></pre><p>Output</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>Nmap scan report for 192.168.29.30
Host is up (0.00026s latency).
Not shown: 997 closed ports
PORT STATE SERVICE
22/tcp open ssh
8000/tcp open http-alt
8080/tcp open http-proxy
Device type: general purpose
Running: Linux 3.X
OS CPE: cpe:/o:linux:linux_kernel:3
OS details: Linux 3.7 - 3.9
Network Distance: 0 hops</b></span></span></pre><div><br /></div><div>Hope this post will help you in some sort !</div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com4tag:blogger.com,1999:blog-1692034155797331883.post-27388844349508051582022-07-21T10:12:00.001+05:302022-07-21T10:12:05.061+05:30How to remove last character from the last line of a file using SED<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimIyauCDPbwEYs4-7f3Ipw7RsEpXmhOTrIf1Vl19dCg_m_91Jdt8GbGFbLvFwjOVSgWfj4pdmBIe_sLXdkJMsqHr9q7VFcEoKdB146TfxCeKQ7PmZXCJjaQCoU0E_OUt2y7_eFoB8hCFbxwBhOC2d6JeA5BSLyJwWP5mLoB9V4Pm1UMDfsBC9ulj4/s2048/terminal.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="2048" data-original-width="1792" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimIyauCDPbwEYs4-7f3Ipw7RsEpXmhOTrIf1Vl19dCg_m_91Jdt8GbGFbLvFwjOVSgWfj4pdmBIe_sLXdkJMsqHr9q7VFcEoKdB146TfxCeKQ7PmZXCJjaQCoU0E_OUt2y7_eFoB8hCFbxwBhOC2d6JeA5BSLyJwWP5mLoB9V4Pm1UMDfsBC9ulj4/s320/terminal.png" width="280" /></a></div><p></p><p>This could be very relatable hack for you as we all are dealing with JSON object now a days, and during automation using bash aka shell scripts, we may need to parse our json data.</p><p>Okay so here is the data, and what I have</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">$ </span>cat account_address.txt<span style="color: #2b00fe;">
"59598532c58EBeB13A70a37159F0C3AB2e0aB623": { "balance": "10000" },
"A281753296De2A35c2Ae6D613b317b71F76F6aE2": { "balance": "10000" },
"2eAc363b2ffAfbc9b5dE9E2004057a778313d4Ac": { "balance": "10000" },
"3FD7893E53D35A93A240Be3B4112A24746F8d858": { "balance": "10000" },
"dfd46B5F7B194133C48562d84A970358E13d64f7": { "balance": "10000" },
"8F3D701F3963d41935C4D2FeeFb3E072FBc613Ee": { "balance": "10000" },</span></b></span></span></pre><div>And here is the data, and what I need.</div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">$ </span>cat account_address.txt<span style="color: #2b00fe;">
"59598532c58EBeB13A70a37159F0C3AB2e0aB623": { "balance": "10000" },
"A281753296De2A35c2Ae6D613b317b71F76F6aE2": { "balance": "10000" },
"2eAc363b2ffAfbc9b5dE9E2004057a778313d4Ac": { "balance": "10000" },
"3FD7893E53D35A93A240Be3B4112A24746F8d858": { "balance": "10000" },
"dfd46B5F7B194133C48562d84A970358E13d64f7": { "balance": "10000" },
</span><span style="color: red;">"8F3D701F3963d41935C4D2FeeFb3E072FBc613Ee": { "balance": "10000" }</span></b></span></span></pre><p>Using SED one liner, we can do this stuff.</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">$ </span>cat account_address.txt </b></span></span>| <b><span style="color: #2b00fe; font-size: medium;">sed '$ s/.$//'</span></b></pre><p><br /></p><p>That's it !</p></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-79105782561389318152022-07-14T14:00:00.002+05:302022-07-14T14:00:31.268+05:30How to restrict AWS S3 Content to be accessed by CloudFront distribution only.<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEguUx8U-QoxcsHAatoJ-0oZTuqJZuWYrmvpuiGFYlQE8vtxW2m4I1z56l83GLu9v1Te7bboMU0xiqjhyCPpjFKywhLm1apzZsL3tF0rtJGFHSpIRge_jfyLt3euVWvuCDfaFxYMkO4kZ5HwW9RCKElWqi6wB366ZHcxAe-c57NIwfneiztN-iwe2C0" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="670" data-original-width="1191" height="360" src="https://blogger.googleusercontent.com/img/a/AVvXsEguUx8U-QoxcsHAatoJ-0oZTuqJZuWYrmvpuiGFYlQE8vtxW2m4I1z56l83GLu9v1Te7bboMU0xiqjhyCPpjFKywhLm1apzZsL3tF0rtJGFHSpIRge_jfyLt3euVWvuCDfaFxYMkO4kZ5HwW9RCKElWqi6wB366ZHcxAe-c57NIwfneiztN-iwe2C0=w640-h360" width="640" /></a></div><br /><p></p><p>CloudFront is one of the popular services of AWS that gives Caching mechanism for our static contents like html, css, images and media files serving a very fast performance using it's globally available CDN networks of POP sites.</p><p>In this blog post, we will know </p><p></p><ul style="text-align: left;"><li>How to create a basic CloudFront distribution using S3 as Origin.</li><li>How can we create a CloudFront distribution using S3 as Origin without making the Content of Origin(s3 Objects) public.</li><li>What, Why and How about CloudFront OIA.</li></ul><p></p><p><br /></p><p>Here in this scenario, we will be using S3 bucket as an Origin for our CloudFront distribution</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg8dsosWzKHVE0twV5LdT7QWYSAWHo_QBbkbzl6lEBICawTk-0stNcS06GVEOLiqezHd8yuR5eZrbFtJyKMjmcYHdvSTCqSj9Vfw2-WwjdkgyzZYA8uekYpmh8wclRS1oB1KcLGnOHwltPY0MjgnalJ82cleJlEmNH7x6SVnYVTOVZ1pcLLmf3Lxx0" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="223" data-original-width="505" height="141" src="https://blogger.googleusercontent.com/img/a/AVvXsEg8dsosWzKHVE0twV5LdT7QWYSAWHo_QBbkbzl6lEBICawTk-0stNcS06GVEOLiqezHd8yuR5eZrbFtJyKMjmcYHdvSTCqSj9Vfw2-WwjdkgyzZYA8uekYpmh8wclRS1oB1KcLGnOHwltPY0MjgnalJ82cleJlEmNH7x6SVnYVTOVZ1pcLLmf3Lxx0" width="320" /></a></div><br /><br /><p></p><p>We will understand the problem first and then know, how Origin Access Identity can be used to address the request.</p><p>So we have quickly created a S3 bucket and CloudFront distribution using default settings with below details.</p><div style="text-align: left;"><ul style="text-align: left;"><li>S3 bucket name - s3-web-bucket</li><li>Bucket Permissions - Block all Public Access</li><li>CloudFront distribution default object - index.html</li><li>CloudFront Origin - s3-web-bucket</li></ul></div><p>Now, quickly upload a index.html file under the root of s3 bucket as s3-web-bucket/index.html.</p><p>We are done with the configuration, let's try to quickly access the CloudFront distribution and verify if everything is working perfectly or not.</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">$ curl -I </span><span style="color: #ffa400;">https://d2wakmcndjowxj.cloudfront.net</span></b></span></span></pre><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: red;">
HTTP/2 403</span><span style="color: #2b00fe;">
content-type: application/xml
date: Thu, 14 Jul 2022 07:28:37 GMT
server: AmazonS3
x-cache: Error from cloudfront
via: 1.1 ba846255b240e8319a67d7e11dc11506.cloudfront.net (CloudFront)
x-amz-cf-pop: MRS52-P4
x-amz-cf-id: BbAsVxxWfW9v3m1PD2uBHqRIj_7-J5U3fUzhhFiQQhbJj8a7lQlCvw==</span></b></span></span></pre><div style="text-align: left;">We encountered 403 error, why ?<br />Ans : This is expected as we have kept the bucket permission level as Block All Public Access.</div><p>Okay, then let's modify the bucket permission and Allow Public Access, for this follow the below two steps.</p><p></p><ul style="text-align: left;"><li>Enable Public Access from Console by unchecking the Check box "Block all public access" and Save it.</li></ul><p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEh5zI_RZ9wugjyItgthdlt3gxFvFlcE8SohUz3HdlAqrOTbswYWgkHC3x-Zq80M9nogM_8ZB2yarN-fTPGyGtZze-piBM3bnPsm-X4r_ZtXK4AZdRFtka3v8Y66WJZswK36xHKtBKfdy2j3-hcOm6h2txK2m2uLgfgg2FHhkpJRz1Ektxf2TAGfXo8" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="712" data-original-width="976" height="291" src="https://blogger.googleusercontent.com/img/a/AVvXsEh5zI_RZ9wugjyItgthdlt3gxFvFlcE8SohUz3HdlAqrOTbswYWgkHC3x-Zq80M9nogM_8ZB2yarN-fTPGyGtZze-piBM3bnPsm-X4r_ZtXK4AZdRFtka3v8Y66WJZswK36xHKtBKfdy2j3-hcOm6h2txK2m2uLgfgg2FHhkpJRz1Ektxf2TAGfXo8=w400-h291" width="400" /></a></div><div class="separator" style="clear: both; text-align: left;"><ul><li>Append the below Bucket Policy JSON statement to make all objects inside the Bucket as Public, the one highlighted in red can be replaced by your own Bucket name.</li></ul><br /></div><p></p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::</span><span style="color: red;">s3-web-bucket</span><span style="color: #2b00fe;">/*"
}
]
}</span></b></span></span></pre><div class="separator" style="clear: both; text-align: left;"></div><p></p><div class="separator" style="clear: both; text-align: left;"><ul style="text-align: left;"><li>Save it, and your bucket permission section will appear with Red Amber signifying that your bucket is publicly accessible.</li></ul></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgTMBl0UHAMgmTx74g_9GI-ZeqskCq7HBKAVZdNJ85oC1XjLOoBc1y5ODKnXf6nftHu5YZlNcj5tSPTofknvuWHqMq-yT1ptsHZOz6vGd4xCmzUhT1Jo2aEWWlUqDgX_F6blR3XRhYStVJqAAT7zY02m5YGoLhWaLM8x2A4vVpU1y5-ghdvd8tTSoY" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="405" data-original-width="801" height="162" src="https://blogger.googleusercontent.com/img/a/AVvXsEgTMBl0UHAMgmTx74g_9GI-ZeqskCq7HBKAVZdNJ85oC1XjLOoBc1y5ODKnXf6nftHu5YZlNcj5tSPTofknvuWHqMq-yT1ptsHZOz6vGd4xCmzUhT1Jo2aEWWlUqDgX_F6blR3XRhYStVJqAAT7zY02m5YGoLhWaLM8x2A4vVpU1y5-ghdvd8tTSoY" width="320" /></a></div><br />Done, Now let's try again to access the Website (index.html) from our CloudFront distribution.<div><br /><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">$ curl -I </span><span style="color: #ffa400;">https://d2wakmcndjowxj.cloudfront.net</span></b></span></span></pre><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">
</span><span style="color: #04ff00;">HTTP/2 200</span><span style="color: #2b00fe;">
content-type: text/html
content-length: 557
date: Thu, 14 Jul 2022 07:47:58 GMT
last-modified: Wed, 13 Jul 2022 18:50:58 GMT
etag: "c255abee97060a02ae7b79db49ed7ec1"
accept-ranges: bytes
server: AmazonS3
x-cache: Miss from cloudfront
via: 1.1 ba055a10d278614dad75399031edff3c.cloudfront.net (CloudFront)
x-amz-cf-pop: MRS52-C2
x-amz-cf-id: Bhf_5IjA0sifp7jON4dpzZdjpCZCQTF5L7c5oenUbjc1vZzvL6ZUWA==</span></b></span></span></pre><p>Good, we are able to access our webpage and now our static contents will be served from CDN network, but wait let's try to access the object(index.html) from bucket's S3 URLs too.</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">$ curl -I </span><span style="color: #ffa400;">https://s3-web-bucket.s3.amazonaws.com/index.html</span></b></span></span></pre><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">
</span><span style="color: #04ff00;">HTTP/1.1 200 OK</span><span style="color: #2b00fe;">
x-amz-id-2: OgLcIIYScHdVok2puZb09ccCjU5K9xNxOL6D1sVj/nBf6hm93vCjQQSpm3fxo4tXpdjUa3u2TS0=
x-amz-request-id: 588WXNR2BH9F37R9
Date: Thu, 14 Jul 2022 07:50:42 GMT
Last-Modified: Wed, 13 Jul 2022 18:50:58 GMT
ETag: "c255abee97060a02ae7b79db49ed7ec1"
Accept-Ranges: bytes
Content-Type: text/html
Server: AmazonS3
Content-Length: 557</span></b></span></span></pre><p>Here is the loophole, naming standards for any s3 bucket urls and it's respective objects are quite easy to guess if one knows the name of the bucket only.</p><p>User, developer and hackers can bypass the CloudFront url and can get access to Objects directly from S3 Urls only, but you may think or what's the issue as they are anyhow public read in nature by permissions.</p><p>So to answer these questions, here are some points I would like to point, how accessing content via CloudFront URls is useful</p><p></p><ul style="text-align: left;"><li>CloudFront URLs give you better performance.</li><li>CloudFront URL can provide Authentication mechanism.</li><li>CloudFront URL gives additional possibilities to trigger CloudFront Function, which can be used for custom solutions.</li><li>Sometimes content of a website/API is designed to be served via CloudFront only, accessing it from S3 gives you a portion of it's content.</li></ul>These are few counter points, but there are many more to support why should you disable public access to your s3 buckets.<p></p><span><a name='more'></a></span><h3 style="text-align: left;">Origin Access Identity ( OIA ) </h3><div>The above concern can be sorted out using OIA feature of CloudFront, where we can restrict the access of a S3 bucket from a CloudFront OIA user only.</div><div>To achieve it, here are the steps.</div><div><br /></div><div style="text-align: left;"><ul style="text-align: left;"><li>First thing First, incase you have Public Access Enable on your bucket and it's object. ( In case, if it's a fresh bucket with default permission, you can skip this step)</li><ul><li>Disable Public Access to your bucket and it's object from S3 console.</li><li>Remove the Bucket Policy which we have enabled to support public access.</li></ul></ul></div><ul style="text-align: left;"><li>Create a CloudFront user called an origin access identity (OAI) and associate it with the CloudFront distribution.</li><li>Configure your S3 bucket permissions so that CloudFront can use the OAI to access the files in your bucket and serve them to your users.</li></ul><p></p><p>We can configure both the above steps from AWS console.</p><p>Go to Distribution -- Select your distribution -- Click on Origin Tab --> Select your Origin -- Edit.</p><div style="text-align: left;">Follow the below order.</div><div style="text-align: left;"><ol style="text-align: left;"><li>Check <b>Use OIA</b></li><li><b>Create new OIA</b></li><li>Yes, Update the Bucket Policy</li><li>Save</li></ol></div><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgNeTqfhHK2QePZQBciZYinCACiqB9P_B2huzik3tuNGEIjl5xKt5M1YLkpMA8GCy26OjxZ77bJBfHa2uSrO-WFTyJ-OABsuqSK3iFXM6vR0ZDdWOAT7HtAN78MEG6zo27J1TNWQcR0FDoZ4mPsvYI-Ndm0547SHs-C2rlXWkksAotys90KHxaAJNE" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="751" data-original-width="799" height="602" src="https://blogger.googleusercontent.com/img/a/AVvXsEgNeTqfhHK2QePZQBciZYinCACiqB9P_B2huzik3tuNGEIjl5xKt5M1YLkpMA8GCy26OjxZ77bJBfHa2uSrO-WFTyJ-OABsuqSK3iFXM6vR0ZDdWOAT7HtAN78MEG6zo27J1TNWQcR0FDoZ4mPsvYI-Ndm0547SHs-C2rlXWkksAotys90KHxaAJNE=w640-h602" width="640" /></a></div><br />And we are done, we can confirm the same by verifying the access and Looking into the Bucket Policy Section, it must be updated by something like below.<p></p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>{
"Version": "2008-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"Statement": [
{
"Sid": "1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E1WON59VMYUAH3"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::s3-web-bucket/*"
}
]
}</b></span></span></pre><p>Now Check again using the Curl Command, if we have access to the objects using CloudFront URLs using updates settings.</p><p>But before that, Invalidate the Cache from all POP location by creating the below Invalidation request,</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjgJLIVz9NorFFzKKKnavUtV3jJ4ColaEAzt-IKnV2mtJ4Wy2Z0FDutfCWYZObMkvGCQCB1KXreV8Z2vfRf63oY9G8yNY42NoMCeH1tyPQFcf_gthkmbJogmyfa7HdMwwccZQsSjsL4_rJJ2_9JTMlKbe0cHHxTvEglXqV-pJfrBmjdoYaTGIQ7MwA" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="517" data-original-width="892" height="370" src="https://blogger.googleusercontent.com/img/a/AVvXsEjgJLIVz9NorFFzKKKnavUtV3jJ4ColaEAzt-IKnV2mtJ4Wy2Z0FDutfCWYZObMkvGCQCB1KXreV8Z2vfRf63oY9G8yNY42NoMCeH1tyPQFcf_gthkmbJogmyfa7HdMwwccZQsSjsL4_rJJ2_9JTMlKbe0cHHxTvEglXqV-pJfrBmjdoYaTGIQ7MwA=w640-h370" width="640" /></a></div><br />Wait for the Invalidation to complete, once completed verify the website status.</div><div><br /></div><div><p></p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">$ curl -I </span><span style="color: #ffa400;">https://d2wakmcndjowxj.cloudfront.net</span></b></span></span></pre><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">
</span><span style="color: #04ff00;">HTTP/2 200</span><span style="color: #2b00fe;">
content-type: text/html
content-length: 557
date: Thu, 14 Jul 2022 07:47:58 GMT
last-modified: Wed, 13 Jul 2022 18:50:58 GMT</span></b></span></span></pre><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;"> curl -I </span></b></span></span><span style="color: #ffa400; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>https://s3-web-bucket.s3.amazonaws.com/index.html</b></span></span></pre><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: red;">HTTP/1.1 403 Forbidden
</span><span style="color: #2b00fe;">x-amz-request-id: 0SEWGEWRKXK7JS81
x-amz-id-2: 4iuN/FLtT6vEwq5WoE81dZTUNCvKRb0iDCmyOwmGWpTuL7yYu+jcgGrAGvuJO0wBT4i61rG3Lco=
Content-Type: application/xml
Date: Thu, 14 Jul 2022 08:23:51 GMT</span></b></span></span></pre><p>We can see from the above Curl requests, we are able to retrieve our website content using CloudFront URLs only, forcing a user to go through it only.</p><p><br /></p><p>Hope this helps you in some sort. </p></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-10377337738914304592022-07-14T01:36:00.003+05:302022-07-14T01:36:18.913+05:30CloudFront : How to host multiple buckets from single CloudFront domain<div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><br /></div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjHOyADLGwUB8VYI88iBPftAnx9oo0OZB1Cm0W67xOQhPY_sLsQMTkv_0YtARkwhlbTobqbPakT9eBy8xneTBZHdCe6nG_Yf0b7HulJGqhZQX9sFxDO4Lmpd2Szu4d4W5dyEIN4fHIfJ_lLkej2AehP5gBjOFqki929C2cI5YglDAzQDDfRCEESV-E" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="652" data-original-width="1159" height="360" src="https://blogger.googleusercontent.com/img/a/AVvXsEjHOyADLGwUB8VYI88iBPftAnx9oo0OZB1Cm0W67xOQhPY_sLsQMTkv_0YtARkwhlbTobqbPakT9eBy8xneTBZHdCe6nG_Yf0b7HulJGqhZQX9sFxDO4Lmpd2Szu4d4W5dyEIN4fHIfJ_lLkej2AehP5gBjOFqki929C2cI5YglDAzQDDfRCEESV-E=w640-h360" width="640" /></a></div><br /><br /><p></p><p>As far if you follow this blog's posts, here mostly posts are related to cloud tasks assigned to me as an requirement, you can think as one of the industry standard requirements too.</p><p>In this blog post, we will see how we can achieve the above scenario that is One CloudFront domain to host multiple S3 buckets as origin.</p><p>Let's follow the steps as follow.</p><p>Create 3 different S3 buckets as per above architecture diagram.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgnkB4uSe4eq3eVjDRCoPp_AyMd1sHByAmUh6OQQ_91H6wFHGTR9yzYE-_F-2NUDoeonOiWNGRai-vSfkoXUFmwolQNRwX7OpFW5PTYQIEs-Rp3Q6hv_T1uXpdLFDFi6MKWM-vvsnr2X7Jg7-Pk7IjX-G4aBpCQMwuLa1iWlsAa3Ooml2VvyvWcLfc" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="331" data-original-width="1453" height="146" src="https://blogger.googleusercontent.com/img/a/AVvXsEgnkB4uSe4eq3eVjDRCoPp_AyMd1sHByAmUh6OQQ_91H6wFHGTR9yzYE-_F-2NUDoeonOiWNGRai-vSfkoXUFmwolQNRwX7OpFW5PTYQIEs-Rp3Q6hv_T1uXpdLFDFi6MKWM-vvsnr2X7Jg7-Pk7IjX-G4aBpCQMwuLa1iWlsAa3Ooml2VvyvWcLfc=w640-h146" width="640" /></a></div><br />As per the architecture diagram, create respective directories to match the URI path that is<p></p><p></p><ul style="text-align: left;"><li>http://d233xxyxzzz.cloudfront.net/web1 --> <b>s3-web1-bucket</b> --> Create <b>web1</b> directory inside <b>s3-web1-bucket/</b></li><li>htttp://d233xxyxzzz.cloudfront.net/web2 <b>--> s3-web2-bucket --> Create <b>web2</b><span style="font-weight: 400;"> directory inside </span><b>s3-web2-bucket/</b></b></li></ul><p></p><p>Dump 3 individual index.html files, that resembles to be an identifier as content served from that specific bucket.</p><p></p><ul style="text-align: left;"><li>index.html path for s3-web-bucket -- s3-web-bucket/index.html</li><li>index.html path for s3-web1-bucket -- s3-web-bucket/<b>web1/</b>index.html</li><li>index.html path for s3-web2-bucket -- s3-web-bucket/<b>web2/</b>index.html</li></ul><p></p><p>This is how my three different index.html looks like.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg0QlU4rm4j9v7CUKl656rj7Oc6gGizi_Owe25S-1XAmFJAUyshMVNccFvN4X8f6cy6k99T2zXXfZi8T_JTDtxbzTihlOixobt_xe31AS9Mv1muS28FI7qF68CXO1FxGxjvgFEPfZaxNEb6PGKFLqJKZ7EuyLYOHq3HZ85M_1582Hn32LoiZktvhRk" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="895" data-original-width="946" height="605" src="https://blogger.googleusercontent.com/img/a/AVvXsEg0QlU4rm4j9v7CUKl656rj7Oc6gGizi_Owe25S-1XAmFJAUyshMVNccFvN4X8f6cy6k99T2zXXfZi8T_JTDtxbzTihlOixobt_xe31AS9Mv1muS28FI7qF68CXO1FxGxjvgFEPfZaxNEb6PGKFLqJKZ7EuyLYOHq3HZ85M_1582Hn32LoiZktvhRk=w640-h605" width="640" /></a></div><p><br /></p>We are set from the Bucket part, let's jump to CloudFront and create a basic CloudFront distribution with one of the s3 bucket as origin, here we have chosen s3-web-bucket as our origin for CloudFront distribution with other default settings.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhv7aVaTWhDulIeupZRg5YruaK9je0ttql16y7uHtI_8RjXhVcKqeMz9J5nV0itUrmkABeugrxG7XSPcKj0NbDOandAea0DzGpJNCZaW5ISpKauhKnDWWW_s14_jijC8uYEP9nKI9Lqs9TUbUzbFNEm6vCSjUhZP2jVYF2I8Uf69YIftVnBXGf9_HQ" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="595" data-original-width="1078" height="354" src="https://blogger.googleusercontent.com/img/a/AVvXsEhv7aVaTWhDulIeupZRg5YruaK9je0ttql16y7uHtI_8RjXhVcKqeMz9J5nV0itUrmkABeugrxG7XSPcKj0NbDOandAea0DzGpJNCZaW5ISpKauhKnDWWW_s14_jijC8uYEP9nKI9Lqs9TUbUzbFNEm6vCSjUhZP2jVYF2I8Uf69YIftVnBXGf9_HQ=w640-h354" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgjmsMcSb4yBKNf1ofPfur6iFkmEk-NE4awGFvliUJp39y1dB9AFsDikhqQaNs_7fqozg_1SXS9pAEFoheqikXNNsUeFRuT3PtlwZHxWfA9DbePXRBT_hnNw2sCoa4xU7-4xl287s8whz0uEIFmHxki12SsoTxw4fZXzknJCul4N3_JM0GNjdZ5w1g" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="376" data-original-width="1525" height="158" src="https://blogger.googleusercontent.com/img/a/AVvXsEgjmsMcSb4yBKNf1ofPfur6iFkmEk-NE4awGFvliUJp39y1dB9AFsDikhqQaNs_7fqozg_1SXS9pAEFoheqikXNNsUeFRuT3PtlwZHxWfA9DbePXRBT_hnNw2sCoa4xU7-4xl287s8whz0uEIFmHxki12SsoTxw4fZXzknJCul4N3_JM0GNjdZ5w1g=w640-h158" width="640" /></a></div><br /><p>Note : Default root object to index.html, else we have to append index.html manually every time after /</p><p></p><p>Now here comes the fun, we have our CloudFront URL in active state and thus according to our architecture this is what we are expecting overall.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEis0ywX3b8N3qaU5DLhYsMcYoZFB9yFgQ4vVmQjnnpMd2fg5VzZ0G2GQtlzM5XG-Qxh4C79THtSiOnwDArG3MePRxetze8JvoEPHMQlvHANxmu5RnDmnTLWjsZPXhgXG4vLFds9ZUZZiN9Hew2EPTPemxrzoFNZGf1SY3cChIbazFUz31hUCs1X13Q" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="636" data-original-width="958" height="424" src="https://blogger.googleusercontent.com/img/a/AVvXsEis0ywX3b8N3qaU5DLhYsMcYoZFB9yFgQ4vVmQjnnpMd2fg5VzZ0G2GQtlzM5XG-Qxh4C79THtSiOnwDArG3MePRxetze8JvoEPHMQlvHANxmu5RnDmnTLWjsZPXhgXG4vLFds9ZUZZiN9Hew2EPTPemxrzoFNZGf1SY3cChIbazFUz31hUCs1X13Q=w640-h424" width="640" /></a></div><br /><br /><p></p><h3 style="text-align: left;">Create Origins for S3 buckets.</h3><p>Let's add other two more origin, which are the two other remaining s3 buckets.</p><p>Origin Configuration for S3 bucket "<b>s3-web1-bucket</b>"</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgqJRBw-NWeIg-K5YqP0qNKindB1pjPjfC6mcxcWn4BPSWNEZ2AIEX34Kq9vJ15H0y_SyYi9-Kn6mQFtWT7J6FJRUSOKcvGA81aTiz2jrQKk4hhCuK7QMLDTcgUUABVKTmLczQM0Rf78nG9w0VSfGWu3Vxy5LSld0pfd1NyH_Jr3qQ2OYEQsrmTT8o" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="699" data-original-width="823" height="543" src="https://blogger.googleusercontent.com/img/a/AVvXsEgqJRBw-NWeIg-K5YqP0qNKindB1pjPjfC6mcxcWn4BPSWNEZ2AIEX34Kq9vJ15H0y_SyYi9-Kn6mQFtWT7J6FJRUSOKcvGA81aTiz2jrQKk4hhCuK7QMLDTcgUUABVKTmLczQM0Rf78nG9w0VSfGWu3Vxy5LSld0pfd1NyH_Jr3qQ2OYEQsrmTT8o=w640-h543" width="640" /></a></div><br /><p style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"></p><p></p><p style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"><span style="font-weight: 400;">Origin Configuration for S3 bucket "</span><b>s3-web2-bucket</b>"</p><p style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiw7kkFYGZv6WGUNYgmXfz7EIEbQ35iFj368ntSxXLvJR1PAuTzWkXu_v6D2wRzZrexxJ6vLfY8eEhh4gYJBOLqrBD0sK96GhyCD6x8w5SSYfJ7auO9h5lEdXioH6BWWzAcwIZG4dFU6roo1qB6QBAyR6annR0cfU4NASOgI3Sw024YP1TP3VcpbIQ" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="697" data-original-width="790" height="565" src="https://blogger.googleusercontent.com/img/a/AVvXsEiw7kkFYGZv6WGUNYgmXfz7EIEbQ35iFj368ntSxXLvJR1PAuTzWkXu_v6D2wRzZrexxJ6vLfY8eEhh4gYJBOLqrBD0sK96GhyCD6x8w5SSYfJ7auO9h5lEdXioH6BWWzAcwIZG4dFU6roo1qB6QBAyR6annR0cfU4NASOgI3Sw024YP1TP3VcpbIQ=w640-h565" width="640" /></a></div><br /><h3 style="text-align: left;">Create Behaviors for the above origins.</h3><p></p><p style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">So far, we have added all the s3 buckets as origin, now let's create the behavior which is path aka URI based routing.</p><p style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"><span style="font-weight: 400;">Behavior 1 - </span><b>/web1</b><span style="font-weight: 400;"> routes to</span><b> s3-web1-bucket</b></p><p>Behavior 2 - <b>/web2</b> routes to<b> s3-web2-bucket</b></p><div>Overall, within the Behavior tab, it should look as below <b> </b></div><div><b><br /></b></div><div><b><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhGwPuPk6c7qKoutvTN0is7DNsdiSrMHmFa-bbk-axbzYbinvlK6cSxdInvzqCBSbUEQbxDPcvLoi40wEaQd_KTNm2TgwWSzQNtLB7kaNw_8xWEB0pTE-Z0M7c_m9Hg8Ro1FiltqcFsC4ooOoQZ_U7SRsaxxajE7gBcXYI8LQ4Ad3nt0mLagIPqBS8" style="margin-left: 1em; margin-right: 1em;"></a><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhGwPuPk6c7qKoutvTN0is7DNsdiSrMHmFa-bbk-axbzYbinvlK6cSxdInvzqCBSbUEQbxDPcvLoi40wEaQd_KTNm2TgwWSzQNtLB7kaNw_8xWEB0pTE-Z0M7c_m9Hg8Ro1FiltqcFsC4ooOoQZ_U7SRsaxxajE7gBcXYI8LQ4Ad3nt0mLagIPqBS8" style="margin-left: 1em; margin-right: 1em;"></a><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgDfpLS_I0n_KYBgR5Pyjrm25M8-OG9E3LsFqJ23CBqJw5Rx-Bm3kjxQJ3rWfsAnJ97XdOC5kdpamz5HG0lycEknKyr6Itbb47u3R_KCNXqiPEnjYSPCeysuOTzDgJHWSyl4OxVRgd11eOEjuSHx4VyQEOhL_yccjCpxafD7W2kvTkmPN_kU-xXm-0" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="450" data-original-width="1254" height="230" src="https://blogger.googleusercontent.com/img/a/AVvXsEgDfpLS_I0n_KYBgR5Pyjrm25M8-OG9E3LsFqJ23CBqJw5Rx-Bm3kjxQJ3rWfsAnJ97XdOC5kdpamz5HG0lycEknKyr6Itbb47u3R_KCNXqiPEnjYSPCeysuOTzDgJHWSyl4OxVRgd11eOEjuSHx4VyQEOhL_yccjCpxafD7W2kvTkmPN_kU-xXm-0" width="640" /></a></div></div><br /></b></div><div><div>That's it !</div><div>Let's Open Browser and test the urls one by one.</div><div><br /></div><div><ul style="text-align: left;"><li>https://d2wakmcndjowxj.cloudfront.net</li><li>https://d2wakmcndjowxj.cloudfront.net/web1/index.html</li><li>https://d2wakmcndjowxj.cloudfront.net/web2/index.html</li></ul></div><div> <b> </b></div><div><b>Hope this will help you in some sort !</b></div><div><b><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhGwPuPk6c7qKoutvTN0is7DNsdiSrMHmFa-bbk-axbzYbinvlK6cSxdInvzqCBSbUEQbxDPcvLoi40wEaQd_KTNm2TgwWSzQNtLB7kaNw_8xWEB0pTE-Z0M7c_m9Hg8Ro1FiltqcFsC4ooOoQZ_U7SRsaxxajE7gBcXYI8LQ4Ad3nt0mLagIPqBS8" style="margin-left: 1em; margin-right: 1em;"></a></div></b></div></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-23132170779838779292022-07-13T18:45:00.001+05:302022-07-13T18:46:32.356+05:30Kubernetes Inter-pod communication within a cluster. <p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyABxtXvnPpYOqZmlX8UiyvkYGYVHfxq3IT7GkEM-eXhxAWisiItFagNCSQB8Bd5uR7-izzzb65SX7GCE2MiXwgwp6grrGtwCdCP6Qzf8tYL4Eo7iE-w19gNdPz56llj3VqUyHqtTLEsOjEonDUOMGyFzqYFy67VDM9ivMf2b3NM-5qoRp7N9cZHo/s800/Kubernetes-logo.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="535" data-original-width="800" height="268" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyABxtXvnPpYOqZmlX8UiyvkYGYVHfxq3IT7GkEM-eXhxAWisiItFagNCSQB8Bd5uR7-izzzb65SX7GCE2MiXwgwp6grrGtwCdCP6Qzf8tYL4Eo7iE-w19gNdPz56llj3VqUyHqtTLEsOjEonDUOMGyFzqYFy67VDM9ivMf2b3NM-5qoRp7N9cZHo/w400-h268/Kubernetes-logo.png" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; text-align: start;">In this post, what are the ways through which we can configure our pods to communicate with each other within the same Kubernetes cluster.</p><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">In order to understand the same, we have create a lab scenario where we have two pods running inside the same cluster.</span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div style="text-align: left;"><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;">We will focus on two namespace </span></span></div><div style="text-align: left;"><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;">default</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;">web-apps</span></span></li></ul><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Let's see what are the pods running in both namespaces.</span></span></div><div><ul style="text-align: left;"><li>Pods running under default namespace.</li></ul></div></div><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; text-align: start;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEijuoOgJ59uTsjqJeUcfP3u5OoIVCNMB3zcfwopLjQ9ZGprliFNgbKCj8bDHM4zoso3PVfRfGftsNy3d-tfdiuOR7LtXA4psd_Zxb3-tNItDOfynahI3rWfMYhmV3NxZzNpIga4sukd4fAorPRl_ISmpmULOLSD3VM9sH7hGcUwhAz0qP6LtNb7jWc" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="99" data-original-width="1207" height="52" src="https://blogger.googleusercontent.com/img/a/AVvXsEijuoOgJ59uTsjqJeUcfP3u5OoIVCNMB3zcfwopLjQ9ZGprliFNgbKCj8bDHM4zoso3PVfRfGftsNy3d-tfdiuOR7LtXA4psd_Zxb3-tNItDOfynahI3rWfMYhmV3NxZzNpIga4sukd4fAorPRl_ISmpmULOLSD3VM9sH7hGcUwhAz0qP6LtNb7jWc=w640-h52" width="640" /></a></div><br /><p style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: 1px; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"></p><p></p><div style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"><div><ul style="text-align: left;"><li>Pods running under web-apps namespace.</li></ul><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg_cC5gkoSSdhbjP1RwNiQs65CWkNX6BAiQR-BSeItnnfRQl6fbaMyhfKnx5-n7PlwAOZlpWXYvrYV8_wBc0kW8WDlLyscVJI-PTu0Lobdp8IlK5WbCA3vuIDQxbg4oAAaQrt8b8L1PXaYqSLXjmyou4mirBvlMBItvOmqOXnqAKK4DmbylXy7teEQ" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="88" data-original-width="1294" height="44" src="https://blogger.googleusercontent.com/img/a/AVvXsEg_cC5gkoSSdhbjP1RwNiQs65CWkNX6BAiQR-BSeItnnfRQl6fbaMyhfKnx5-n7PlwAOZlpWXYvrYV8_wBc0kW8WDlLyscVJI-PTu0Lobdp8IlK5WbCA3vuIDQxbg4oAAaQrt8b8L1PXaYqSLXjmyou4mirBvlMBItvOmqOXnqAKK4DmbylXy7teEQ=w640-h44" width="640" /></a></div><br /><br /></div></div><div>What's the application - So we have our application pod named as "genache-cli-deploymnet" running under default namespace, within this lab environment we will know how we can establish communication between microservices like my-shell and weapp-shell to genache-cli-core.</div><div><br /></div><div>Here are the different ways..</div></div><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; text-align: start;"></p><h3 style="text-align: left;">Using Pod's IP.</h3><div style="text-align: left;">Every pod gets an IP from the defined CIDR range, which can be used to communicate directly from each other, irrespective of namespaces.</div><div style="text-align: left;">Thus a simple pattern of <b>http://<pod-ip-address>:<container-port-number></b></div><div style="text-align: left;"><br /></div><div style="text-align: left;">So as per our lab environment, we will try to establish a connection to <b>genache-cli</b> running with IP Address as curl <b>10.1.1.160</b> and on Port <b>8545</b></div><div style="text-align: left;"><b><br /></b></div><div style="text-align: left;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>> kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
genache-cli-deployment-8f48b88fb-dqnkx 1/1 Running 20 (2d10h ago) 30d 10.1.1.160 docker-desktop <none> <none>
my-shell 1/1 Running 0 37m 10.1.1.162 docker-desktop <none> <none></b></span></span></pre></div><div><div style="text-align: left;">Output from my-shell running on <span style="font-weight: 700;">web-apps</span> namespace</div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; text-align: left; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">root@webapp-shell:/# curl http://10.1.1.160:8545/
</span><span style="color: #ffa400;">400 Bad Request </span></b></span></span></pre></div><h3 style="text-align: left;"><div><span style="font-size: small; font-weight: normal;">Output from webapp-shell running on </span><span style="font-size: small;">default</span><span style="font-size: small; font-weight: normal;"> namespace.</span></div></h3><h3 style="text-align: left;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">root@my-shell:/# curl http://10.1.1.160:8545/
400 Bad Request</span><b style="font-size: 14px; font-weight: 400; letter-spacing: 1px; white-space: pre-wrap;"><br /></b></span></pre></h3><span><a name='more'></a></span><h3 style="text-align: left;"><br /></h3><h3 style="text-align: left;">Create Service for Pod to Pod Communication for same namespace.</h3><div style="text-align: left;">For the above genache-cli deployment, we have created a service by below details.</div><div style="text-align: left;"><ul style="text-align: left;"><li>Name - <b>genache-cli-service</b></li><li>Service<b> </b>type - ClusterIP</li><li>Service Port - 8545</li></ul></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg6zg_GZm7oOsfa7LpECqCGGtA21FzEHT5Cv96v83xfkIPVFRzZ-L1ulTfjoyKYO6U-rwZXbpe0HuZAvc-MLzilARAKsuHe5ndGP1-_wu_35QKNOQoMOOB8hbCAw-TOsWTdXTDRFHlCwaXu6d4JStdhc80l1Fw0kLsuKFrlhTTX-uRlemSIv3rnYGA" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="94" data-original-width="1173" height="52" src="https://blogger.googleusercontent.com/img/a/AVvXsEg6zg_GZm7oOsfa7LpECqCGGtA21FzEHT5Cv96v83xfkIPVFRzZ-L1ulTfjoyKYO6U-rwZXbpe0HuZAvc-MLzilARAKsuHe5ndGP1-_wu_35QKNOQoMOOB8hbCAw-TOsWTdXTDRFHlCwaXu6d4JStdhc80l1Fw0kLsuKFrlhTTX-uRlemSIv3rnYGA=w640-h52" width="640" /></a></div><br /><div style="text-align: left;">Now, we would like to establish the connectivity from pod name <b>my-shell </b>to genachi-cli pod via genache-cli-service. </div></div><div style="text-align: left;"><ul style="text-align: left;"><li>Using Environment variables to fetch Service Name and Service Port.</li><ul><li>Login to my-shell pod and execute command env</li></ul></ul><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgBK42pQDNQohHLTOFV9Qk3q84OCsUJXtDo748CWtsWyB3zKuKtOstWd9GEbkzOFlGyoTl335w4XBy4WsdhspmDAI9h8SKcBtSr9oIk76FBKBrcKvQnHvoM90weMPHxFJFwrPqkg2J8C2BEcMKdQ_ap6HJjLLq5wUPPxR8ZVrwNpT21MbsqvpcG9zg" style="margin-left: 1em; margin-right: 1em;"><span> </span><span> </span><img alt="" data-original-height="832" data-original-width="745" height="640" src="https://blogger.googleusercontent.com/img/a/AVvXsEgBK42pQDNQohHLTOFV9Qk3q84OCsUJXtDo748CWtsWyB3zKuKtOstWd9GEbkzOFlGyoTl335w4XBy4WsdhspmDAI9h8SKcBtSr9oIk76FBKBrcKvQnHvoM90weMPHxFJFwrPqkg2J8C2BEcMKdQ_ap6HJjLLq5wUPPxR8ZVrwNpT21MbsqvpcG9zg=w573-h640" width="573" /></a><br /><ul style="text-align: left;"><ul><li>Here we are interested on genache cli service details that is host and port, we will filter it out using grep for the same above command.</li></ul></ul><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgtpN0zMwg_NdO7XtzAVmhORFr0uStPE-74JALbgXYOIEWpPadSCZ9gcZ3kpbE_iDJRWz53gUbfNQpqu3DyVVNqn__YGnr-xpBzDgr0XfwqbhRzLh7p2uQBzFqfrVoVxBv6trB_eUQ-V5sSsEVF0RX4zvh9_nDtXKNA8hvcOELxa7vEcwcrtyi7i_Q" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="235" data-original-width="690" height="218" src="https://blogger.googleusercontent.com/img/a/AVvXsEgtpN0zMwg_NdO7XtzAVmhORFr0uStPE-74JALbgXYOIEWpPadSCZ9gcZ3kpbE_iDJRWz53gUbfNQpqu3DyVVNqn__YGnr-xpBzDgr0XfwqbhRzLh7p2uQBzFqfrVoVxBv6trB_eUQ-V5sSsEVF0RX4zvh9_nDtXKNA8hvcOELxa7vEcwcrtyi7i_Q=w640-h218" width="640" /></a></div><br /><ul style="text-align: left;"><ul><li>So we can fetch our details and use the pattern as below to establish a connection between pods.</li></ul></ul><b style="background-color: whitesmoke; color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><span> </span>http://<SERVICE_NAME>_SERVICE_HOST}:${<SERVICE_NAME>_SERVICE_PORT}</b><br /><ul style="text-align: left;"><li>Here in this case, we can replace it with the following as per our lab environment.</li><ul><li>SERVICE_NAME -- <b>GENACHE_CLI_SERVICE</b></li></ul></ul><div>Now, in case if we want to connect it using environment variables from our pods.</div><div><br /></div><div>From pod <b>"my-shell</b>"</div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">root@my-shell:/# curl http://${GENACHE_CLI_SERVICE_SERVICE_HOST}:${GENACHE_CLI_SERVICE_SERVICE_PORT}
</span><span style="color: #ffa400;">400 Bad Request</span></b></span></span></pre></div><div><br /></div><div> From pod <b>"webapps-shell</b>"</div><div></div><div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">root@webapps-shell:/# curl http://${GENACHE_CLI_SERVICE_SERVICE_HOST}:${GENACHE_CLI_SERVICE_SERVICE_PORT}
</span></b></span></span><span style="color: #ffa400; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>curl: (3) URL using bad/illegal format or missing URL</b></span></span></pre></div><div><br /></div></div><ul style="text-align: left;"><li>Using Service Names.</li></ul><div>We can directly establish a connection using the format as <b>http://<service-name>:<service-port></b></div><div><b>I</b>n our lab environment, we can replace the same by <b>http://<span style="color: red;">genache-cli-service</span>:<span style="color: #2b00fe;">8545</span></b></div><div><b><span style="color: #2b00fe;"><br /></span></b></div><div>Now, let's try to establish a connection using the same from our pods.</div><div><br /></div><div><div>From pod <b>"my-shell</b>"</div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="color: #2b00fe;"><b style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">root@my-shell:/# </b><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>curl http://genache-cli-service:8545</b></span><b style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">
</b></span><span style="color: #ffa400; font-size: 14px; font-weight: bold; letter-spacing: 1px; white-space: pre-wrap;">400 Bad Request</span></span></pre></div><div><br /></div><div> From pod <b>"webapps-shell</b>"</div><div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="color: #2b00fe;"><b style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">root@webapps-shell:/# </b><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>curl http://genache-cli-service:8545</b></span><b style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">
</b></span></span><span style="color: #ffa400; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>curl: (6) Could not resolve host: genache-cli-service</b></span></span></pre></div></div><div><h4 style="text-align: left;">Conclusion : </h4><div><ul style="text-align: left;"><li>Service name can be used to establish connection within same namespace.</li><li>Service name should be used instead of POD IPs directly.</li><span></span></ul><br /></div><span><!--more--></span><h3>Service Communication for different namespace.</h3><div style="text-align: left;">In order to establish the connection between pods separated by namespaces, we can use the name based service on cluster.</div><div style="text-align: left;">Use the below naming standards to establish communication between services across different name space.</div><div><div style="background-color: white;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><service-name>.<namespace-name>.svc.cluster.local</b></span></span><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">
</span></span></pre><div style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">For our lab environment, we will replace it with our environment as</div><div style="font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; letter-spacing: normal; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #ffa400;">genache-cli-service</span><span style="color: #ffa400; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="letter-spacing: 1px; white-space: pre-wrap;"><b>.</b></span></span><span style="color: #2b00fe;">default</span><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="letter-spacing: 1px; white-space: pre-wrap;"><b>.svc.cluster.local</b></span></span><span style="color: #333333; font-family: Lora, serif;"><span style="letter-spacing: 1px;">
</span></span></pre><div style="color: #333333;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="letter-spacing: 1px; white-space: pre-wrap;"><b><br /></b></span></span></div></div><div><div style="color: black; font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;">Now, let's try to establish a connection using the same from our pods.</div><div style="color: black; font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;"><br /></div><div><div style="color: black; font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;">From pod <b>"my-shell</b>"</div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="color: #2b00fe;"><b style="color: black; font-family: "Times New Roman"; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">root@my-shell:/# </b><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>curl http://genache-cli-service.default.svc.cluster.local:8545</b></span><b style="color: black; font-family: "Times New Roman"; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">
</b></span><span style="color: #ffa400; font-family: "Times New Roman"; font-size: 14px; font-weight: bold; letter-spacing: 1px; white-space: pre-wrap;">400 Bad Request</span></span></pre></div><div style="color: black; font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;"><br /></div><div style="color: black; font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;"> From pod <b>"webapps-shell</b>"</div><div style="color: black; font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;"></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="color: #2b00fe;"><b style="color: black; font-family: "Times New Roman"; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">root@webapps-shell:/# </b><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>curl http://genache-cli-service.default.svc.cluster.local:8545
400 Bad Request</b></span></span></span></pre></div></div></div></div></div></div></div></div></div><p></p>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-30955317945566141972022-04-25T02:02:00.004+05:302022-04-25T02:02:46.500+05:30Selenium XPATH Cheat Sheet<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbLeyq0S0nigZrr3wB-t3doULtLoocZi3evTuwofcyKiIOD_B-tCqKdN2uMqRYgZGfqn1orK-qbdCz3iINjl_7KZWEgv-4Ka79s2jaHIf81Yz_rE4xpwM8A1HPiARujb1mrCzIiCln5b9dOtL1X2x2OeBzhSno3oM984-jeUqcOl34XngLoBlntY8/s800/sublime-icon.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="600" data-original-width="800" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbLeyq0S0nigZrr3wB-t3doULtLoocZi3evTuwofcyKiIOD_B-tCqKdN2uMqRYgZGfqn1orK-qbdCz3iINjl_7KZWEgv-4Ka79s2jaHIf81Yz_rE4xpwM8A1HPiARujb1mrCzIiCln5b9dOtL1X2x2OeBzhSno3oM984-jeUqcOl34XngLoBlntY8/s320/sublime-icon.png" width="320" /></a></div><br /><p><br /></p><p>XPATH that contains Partial Text.</p><p><br /></p><p>Example to Consider.</p><p><span class="style-scope ytd-grid-video-renderer">1 day ago</span></p><p><span class="style-scope ytd-grid-video-renderer">84K views</span></p><p><span class="style-scope ytd-grid-video-renderer">1 hour ago</span></p><p><span class="style-scope ytd-grid-video-renderer">8 hour ago</span></p><div><br /></div><p>Need to Grep elements with Text that contains <b>partial</b> word as hour</p><p>Code Snippet.</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #333333; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px;"><span style="white-space: pre-wrap;"><b><span style="color: #2b00fe;"> </span></b></span></span><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>//*[contains(text(), "hour")]</b></span></span></pre><div><br /></div><div><p>In case if to Grep elements that contains <b>partial</b> tag.</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #333333; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px;"><span style="white-space: pre-wrap;"><b><span style="color: #2b00fe;"> </span></b></span></span><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>//*[contains(@id, "title")]</b></span></span></pre><div><br /></div></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-78677382051432895102022-03-15T17:59:00.001+05:302022-03-15T17:59:04.458+05:30Fix : AWS SAM IAM Error : arn:aws:cloudformation:us-east-1:aws:transform/Serverless-2016-10-31<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjJQM4xEcLTxIpKfn2SZIR9Selvv-zQPcefQyCmV4hwHhlEVKLhjqMW15rJWxCv_XGNbBjbF8EFsE8gTklz5K9VNFq4YWbfdMewrFbx7sK5qG-rpKO3ONzLNELU5f2-d5xp3HRmuyddOqyBXp6GQ0dQOTDYAXwQkjMrcpzzfcEODEaDc1xbvcquEsk=s300" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="300" data-original-width="300" height="300" src="https://blogger.googleusercontent.com/img/a/AVvXsEjJQM4xEcLTxIpKfn2SZIR9Selvv-zQPcefQyCmV4hwHhlEVKLhjqMW15rJWxCv_XGNbBjbF8EFsE8gTklz5K9VNFq4YWbfdMewrFbx7sK5qG-rpKO3ONzLNELU5f2-d5xp3HRmuyddOqyBXp6GQ0dQOTDYAXwQkjMrcpzzfcEODEaDc1xbvcquEsk" width="300" /></a></div><br /><p></p><p><br /></p><p>Stack trace :</p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: red; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>User: arn:aws:sts::455734o955:assumed-role/xx-xx-xx-app-role/i-xxxxxxxxxx is not authorized to perform: cloudformation:CreateChangeSet on resource: arn:aws:cloudformation:us-east-1:aws:transform/Serverless-2016-10-31 because no identity-based policy allows the cloudformation:CreateChangeSet action.</b></span></span></pre><p><br /></p><p>Fix.</p><p>Add the below resource within your JSON policy statement.</p><p>Note : <span style="background-color: whitesmoke;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; font-weight: bold; letter-spacing: 1px; white-space: pre-wrap;">c</span><span style="color: #ffa400; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; font-weight: bold; letter-spacing: 1px; white-space: pre-wrap;">loudformation:*</span><span style="color: #2b00fe;"><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"> is strictly </b><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>discouraged</b></span></span><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">, fine tune your access permissions.</b></span></span></p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;"> </span><span style="color: #2b00fe;"> </span><span style="color: #2b00fe;">{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "cloudformation:*",
"Resource": [
"</span><span style="color: red;">arn:aws:cloudformation:us-east-1:aws:transform/Serverless-2016-10-31</span><span style="color: #2b00fe;">"
]
}</span></b></span></span></pre><p><br /></p><p><br /></p>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-58672643691734688232022-03-01T13:19:00.005+05:302022-03-02T16:26:50.946+05:30Continuous Integration of Lambda Function using AWS SAM<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjExPBQg0dlHSUCxh6f0TkgyqtdkPRt9SWT18rsZYm8zpy4SuOtVobVcZWNwtLSUn6g_jW4dGqJqCMhUUC14QkEkcIrR54-R9wYixiIXLAf_PnrxiLcc3dZaCqOOjrQeMz7zJsmVn-FjR1n5y3CPftosPj1v_pYWbRvqMNyYkl-xiNud_-nywIvDU8=s571" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="313" data-original-width="571" height="175" src="https://blogger.googleusercontent.com/img/a/AVvXsEjExPBQg0dlHSUCxh6f0TkgyqtdkPRt9SWT18rsZYm8zpy4SuOtVobVcZWNwtLSUn6g_jW4dGqJqCMhUUC14QkEkcIrR54-R9wYixiIXLAf_PnrxiLcc3dZaCqOOjrQeMz7zJsmVn-FjR1n5y3CPftosPj1v_pYWbRvqMNyYkl-xiNud_-nywIvDU8=s320" width="320" /></a></div><br /><p></p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>AWS Lambda Function </b>is awesome and trust me if you are working on AWS, sooner or later you have to deal with AWS Lambda.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">In this blog post, we will cover the below use cases.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"></p><ul style="text-align: left;"><li>What is AWS SAM</li><li>How to create Lambda Function using AWS SAM</li><li>How to delete Lambda Function created using AWS SAM</li><li>How to integrate AWS SAM with Docker.</li><li>How to create a continuous integration pipeline with Jenkins, GitHub, Docker and SAM.</li></ul><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></p>What is AWS SAM.<p></p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">One can find a huge detailed information in official documentation, here is the AWS link for the same.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">I would like to share my level of short understanding with you, that can give you some idea of AWS SAM and it's respective components.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"></p><ul style="text-align: left;"><li>AWS SAM that is focused on the creating Application using AWS PAAS services such as API Gateway, AWS Lambda, SNS, SES etc.</li><li>SAM templates are somehow similar to CloudFormation templates, so one who has idea of CloudFormation can easily adapt SAM templates too.</li><li>SAM shorten the code when it's being compared to CloudFormation for Serverless services deployment.</li><li>SAM in backend create CloudFormation stacks to deploy the AWS services, that means it's avoid some part of code that has to be written by user and does the job for you by adding those lines.</li><li>In order to use SAM, one needs an download additional binary/package which is not being clubbed with AWS CLI.</li></ul><div><b>How to create Lambda Function using SAM ?</b></div><div><br /></div><div>Before you directly jump into it, first know the must know stuffs from file and directory prospective.</div><div><br /></div><div><ul style="text-align: left;"><li>samconfig.toml : Configuration file that will be used during the SAM commands ( init, test, build, validate etc)</li><li>template.yml : SAM template, similar to CloudFormation template to define Parameter, Resource, Metadata, Output, Mapping etc.</li><li>events - Directory to store events for testing our Lambda code using events.json file.</li><li>tests - Directory that contains the unit test files</li></ul><div>Lab setup details -</div><div><span> - </span>We will be deploying a Lambda Function with Python 3.7 runtime.</div><div> - Name of our sam application is <b>sam-deployer_with_sam</b></div><div><b> - </b>This is how our Lambda Function looks like in console and it's basic task is to check the status of port, ie Open or Closed </div><div><span> - Our files and templates follow the CICD approach, so we have kept our code for two environment</span><br /></div><div><span><span> </span><span> that is ( default, dev, uat )</span><br /></span></div><div><span> </span><br /></div><div><span><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhk8RXM1h6o6MlWPBM-jrsw5UySepKwJCj2SHSDSv-0XoYTkA3PU9AwQy-3OgCjxCOeE5Idgit339iaOO2OfKFYXtLj2HXvRRKcpGxMaaayJ8aP9p9fYP2HMu24YUn_srnSPtyvB8uXMgJIT0okqJuYgbK1wRSTvTS8czh5pTweP0Us-FfPEs1i8p8" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="598" data-original-width="931" height="257" src="https://blogger.googleusercontent.com/img/a/AVvXsEhk8RXM1h6o6MlWPBM-jrsw5UySepKwJCj2SHSDSv-0XoYTkA3PU9AwQy-3OgCjxCOeE5Idgit339iaOO2OfKFYXtLj2HXvRRKcpGxMaaayJ8aP9p9fYP2HMu24YUn_srnSPtyvB8uXMgJIT0okqJuYgbK1wRSTvTS8czh5pTweP0Us-FfPEs1i8p8" width="400" /></a></div><br /><br /></span></div><div><br /></div><div>Steps.</div><div><ul style="text-align: left;"><li>Install AWS SAM CLI first.</li><li>All tutorial you might have went through will ask you to go through the sam init, sam build and all, this blog is post is little baked one by using existing templates.</li><li>Create a new directory to use in this project</li></ul><div><div style="background-color: white;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px;"><span style="white-space: pre-wrap;"><b>$ mkdir lambda-deployer_with_sam</b></span></span></pre></div></div></div></div><div><ul><li>Create a new files following the below directory structure layout</li></ul><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>lambda-deployer_with_sam/
├── events
│ └── event.json
├── samconfig.toml
├── src
└── template.yaml
2 directories, 3 files</b></span></span></pre></div><div><br /></div><div>Here is the basic content to post in respective files.</div><div><br /></div><div><span style="background-color: #fcff01;">Contents of <b>samconfig.toml</b></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>version = 0.1
[default]
[default.deploy]
[default.deploy.parameters]
stack_name = "default-lambda-deployer-with-sam-Stack"
s3_bucket = "lambda-deployer-sam-bucket"
s3_prefix = "sam-lambda-stack"
region = "ap-south-1"
capabilities = "CAPABILITY_IAM"
disable_rollback = true
image_repositories = []
[dev]
[dev.deploy]
[dev.deploy.parameters]
stack_name = "dev-lambda-deployer-with-sam-Stack"
s3_bucket = "lambda-deployer-sam-bucket"
s3_prefix = "dev-sam-lambda-stack"
region = "ap-south-1"
capabilities = "CAPABILITY_IAM"
disable_rollback = true
image_repositories = []
parameter_overrides = "Environment=\"dev\""
[uat]
[uat.deploy]
[uat.deploy.parameters]
stack_name = "uat-lambda-deployer-with-sam-Stack"
s3_bucket = "lambda-deployer-sam-bucket"
s3_prefix = "uat-sam-lambda-stack"
region = "ap-south-1"
capabilities = "CAPABILITY_IAM"
disable_rollback = true
image_repositories = []
parameter_overrides = "Environment=\"uat\""
</b></span></span>
</pre></div><div><br /></div><div>Lets understand the content of sam configuration file, that is samconfig.toml.</div><div><ul style="text-align: left;"><li>This file will be used later during the deployment operation Lambda and it's respective resource.</li><li>This file can be used to categorize environment specific paramters.</li><li>The first line <b>[ default ], [ dev ] , </b><b style="background-color: whitesmoke; color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">[uat]</b>defines the name of the environment</li><li>All the next lines coming after Second and Third Line <b style="background-color: whitesmoke; color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">[uat.deploy.parameters] </b>is provide environment specifc paramters.</li><li><span style="color: #2b00fe;">parameter_overrides</span> is the one, that is used to override the default parameter provided in the template.yml file, which is equivalent to cloudformation template.</li></ul></div><div><div><span style="background-color: #fcff01;">Contents of <b>template.yml</b></span></div><div></div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px;"><div><span style="color: #569cd6;">AWSTemplateFormatVersion</span>: <span style="color: #ce9178;">'2010-09-09'</span></div><div><span style="color: #569cd6;">Transform</span>: <span style="color: #ce9178;">'AWS::Serverless-2016-10-31'</span></div><div><span style="color: #569cd6;">Description</span>: <span style="color: #ce9178;">An AWS Serverless Specification template describing your function.</span></div><br /><div><span style="color: #569cd6;">Parameters</span>:</div><div> <span style="color: #569cd6;">Environment</span>:</div><div> <span style="color: #569cd6;">Description</span>: <span style="color: #ce9178;">Please specify the target environment.</span></div><div> <span style="color: #569cd6;">Type</span>: <span style="color: #ce9178;">String</span></div><div> <span style="color: #569cd6;">Default</span>: <span style="color: #ce9178;">"dev"</span></div><div> <span style="color: #569cd6;">AllowedValues</span>:</div><div> - <span style="color: #ce9178;">dev</span></div><div> - <span style="color: #ce9178;">uat</span></div><div> <span style="color: #569cd6;">AppName</span>:</div><div> <span style="color: #569cd6;">Description</span>: <span style="color: #ce9178;">Application name.</span></div><div> <span style="color: #569cd6;">Type</span>: <span style="color: #ce9178;">String</span></div><div> <span style="color: #569cd6;">Default</span>: <span style="color: #ce9178;">"find-port-status"</span></div><br /><div><span style="color: #569cd6;">Mappings</span>:</div><div> <span style="color: #569cd6;">EnvironmentMap</span>:</div><div> <span style="color: #569cd6;">dev</span>:</div><div> <span style="color: #569cd6;">IAMRole</span>: <span style="color: #ce9178;">'arn:aws:iam::897248824142:role/service-role/vpclambda-role-27w9b8uq'</span></div><div> <span style="color: #569cd6;">uat</span>:</div><div> <span style="color: #569cd6;">IAMRole</span>: <span style="color: #ce9178;">'arn:aws:iam::897248824142:role/service-role/vpclambda-role-27w9b8uq'</span></div><div> <span style="color: #569cd6;">stg</span>:</div><div> <span style="color: #569cd6;">IAMRole</span>: <span style="color: #ce9178;">'arn:aws:iam::897248824142:role/service-role/vpclambda-role-27w9b8uq'</span></div><br /><div><span style="color: #569cd6;">Resources</span>:</div><div> <span style="color: #569cd6;">LambdabySam</span>:</div><div> <span style="color: #569cd6;">Type</span>: <span style="color: #ce9178;">'AWS::Serverless::Function'</span></div><div> <span style="color: #569cd6;">Properties</span>:</div><div> <span style="color: #569cd6;">FunctionName</span>: <span style="color: #569cd6;">!Sub</span> <span style="color: #ce9178;">'ck-${Environment}-${AppName}'</span></div><div> <span style="color: #569cd6;">Handler</span>: <span style="color: #ce9178;">lambda_function.lambda_handler</span></div><div> <span style="color: #569cd6;">Runtime</span>: <span style="color: #ce9178;">python3.7</span></div><div> <span style="color: #569cd6;">CodeUri</span>: <span style="color: #ce9178;">src/</span></div><div> <span style="color: #569cd6;">Description</span>: <span style="color: #ce9178;">'Lambda Created by SAM template'</span></div><div> <span style="color: #569cd6;">MemorySize</span>: <span style="color: #b5cea8;">128</span></div><div> <span style="color: #569cd6;">Timeout</span>: <span style="color: #b5cea8;">3</span></div><div> <span style="color: #569cd6;">Role</span>: <span style="color: #569cd6;">!FindInMap</span> [<span style="color: #ce9178;">EnvironmentMap</span>, <span style="color: #569cd6;">!Ref</span> <span style="color: #ce9178;">Environment</span>, <span style="color: #ce9178;">IAMRole</span>]</div><div> <span style="color: #569cd6;">VpcConfig</span>:</div><div> <span style="color: #569cd6;">SecurityGroupIds</span>:</div><div> - <span style="color: #ce9178;">sg-a0f856da</span></div><div> <span style="color: #569cd6;">SubnetIds</span>:</div><div> - <span style="color: #ce9178;">subnet-e9c898a5</span></div><div> - <span style="color: #ce9178;">subnet-bdbb59d6</span></div><div> <span style="color: #569cd6;">Environment</span>:</div><div> <span style="color: #569cd6;">Variables</span>:</div><div> <span style="color: #569cd6;">Name</span>: <span style="color: #569cd6;">!Sub</span> <span style="color: #ce9178;">'ck-${Environment}-${AppName}'</span></div><div> <span style="color: #569cd6;">Owner</span>: <span style="color: #ce9178;">CyberkeedaAdmin</span></div><div> <span style="color: #569cd6;">Tags</span>:</div><div> <span style="color: #569cd6;">Name</span>: <span style="color: #569cd6;">!Sub</span> <span style="color: #ce9178;">'ck-${Environment}-${AppName}'</span></div><div> <span style="color: #569cd6;">Owner</span>: <span style="color: #ce9178;">CyberkeedaAdmin</span></div><br /></div></pre></div><div><br /></div><div>Now, our last step is put our Lambda code into src diectory.</div><div><br /></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ touch src/lambda_function.py </b></span></span></pre></div><div><br /></div><div>Contents of src/lambda_function.py</div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>import json
import socket
def isOpen(ip,port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.connect((ip, int(port)))
s.settimeout(1)
return True
except:
time.sleep(1)
return False
def lambda_handler(event, context):
if isOpen('142.250.195.196',443):
code = 200
else:
code = 500
return {
'statusCode': code,
'body': json.dumps("Port status")
}</b></span></span></pre></div><div><br /></div><div>Now, we have everything in place, let's deploy or Lambda code using SAM.</div><div><br /></div><div>Initiate SAM build with respective environment, defined in samconfig.toml</div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ sam build --config-env dev</b></span></span></pre></div><div><br /></div><div>Output will look, something like below.</div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>Building codeuri: /home/kunal/aws_sam_work/lambda-deployer_with_sam/src runtime: python3.7 metadata: {} architecture: x86_64 functions: ['LambdabySam']
requirements.txt file not found. Continuing the build without dependencies.
Running PythonPipBuilder:CopySource
Build Succeeded
Built Artifacts : .aws-sam/build
Built Template : .aws-sam/build/template.yaml
Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Test Function in the Cloud: sam sync --stack-name {stack-name} --watch
[*] Deploy: sam deploy --guided</b></span></span></pre></div><div><br /></div><div>Now, we have build ready to be deployed.. let's initiate sam deploy.</div><div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ sam deploy --config-env dev</b></span></span></pre></div><div><div>Output will look, something like below.</div><div></div></div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>Uploading to dev-sam-lambda-stack/dccfd91235d686ff0c5dcab3c4d44652 400 / 400 (100.00%)
Deploying with following values
===============================
Stack name : dev-lambda-deployer-with-sam-Stack
Region : ap-south-1
Confirm changeset : False
Disable rollback : True
Deployment s3 bucket : 9-bucket
Capabilities : ["CAPABILITY_IAM"]
Parameter overrides : {"Environment": "dev"}
Signing Profiles : {}
Initiating deployment
=====================
Uploading to dev-sam-lambda-stack/b6c26b6d535bf3b43f5b0bb71a88daa1.template 1627 / 1627 (100.00%)
Waiting for changeset to be created..
CloudFormation stack changeset
---------------------------------------------------------------------------------------------------------------------
Operation LogicalResourceId ResourceType Replacement
---------------------------------------------------------------------------------------------------------------------
+ Add LambdabySam AWS::Lambda::Function N/A
---------------------------------------------------------------------------------------------------------------------
Changeset created successfully. arn:aws:cloudformation:ap-south-1:897248824142:changeSet/samcli-deploy1646210098/97de1b9e-ed08-45fe-8e65-fb0c0928e8f7
2022-03-02 14:05:09 - Waiting for stack create/update to complete
CloudFormation events from stack operations
---------------------------------------------------------------------------------------------------------------------
ResourceStatus ResourceType LogicalResourceId ResourceStatusReason
---------------------------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS AWS::Lambda::Function LambdabySam -
CREATE_IN_PROGRESS AWS::Lambda::Function LambdabySam Resource creation Initiated
Initiate SAM build with respective environment, defined in samconfig.toml
CREATE_COMPLETE AWS::Lambda::Function LambdabySam -
CREATE_COMPLETE AWS::CloudFormation::Stack dev-lambda-deployer-with- -
sam-Stack
---------------------------------------------------------------------------------------------------------------------
Successfully created/updated stack - dev-lambda-deployer-with-sam-Stack in ap-south-1</b></span></span>
</pre></div><div><br /></div><div>This step, will create the required services and it's respective configuration, confirm the same from lambda console, this is how it looks like.</div><div><br /></div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjQnO_Zey8l-_t1IXkahpejSjbyAPFjkAPM3spNfDSSsl49lH2RmyoAiIbI9EEWKgvG_v4q-oK7ywOY3Q8sEH2_uYRa0TYUOdKdcdq3F3AQcCeV3wghViY07MWhPTjkooeaPo0PM-zNoxx2Tb6zC_3AaloR3VKVPUEQ2GmOt-678a55YgKD3YazELU" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="658" data-original-width="904" height="291" src="https://blogger.googleusercontent.com/img/a/AVvXsEjQnO_Zey8l-_t1IXkahpejSjbyAPFjkAPM3spNfDSSsl49lH2RmyoAiIbI9EEWKgvG_v4q-oK7ywOY3Q8sEH2_uYRa0TYUOdKdcdq3F3AQcCeV3wghViY07MWhPTjkooeaPo0PM-zNoxx2Tb6zC_3AaloR3VKVPUEQ2GmOt-678a55YgKD3YazELU=w400-h291" width="400" /></a></div><br /><br /></div><div>Please note, every time we make any changes in lambda_function.py file, we need to re-build and deploy.</div><div><br /></div><div>That's in this post, we will know later in next post about below stuffs.</div><div><ul><li>How to delete Lambda Function created using AWS SAM</li><li>How to integrate AWS SAM with Docker.</li><li>How to create a continuous integration pipeline with Jenkins, GitHub, Docker and SAM.</li></ul></div><div><br /></div><div><div style="background-color: white;"></div></div></div><p></p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></p>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-50080452105877970172022-02-24T21:56:00.001+05:302022-02-24T21:56:03.882+05:30AWS IAM Policy to Allow All Operations except IAM<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEikxUr722BTVBIkMnNvEUK-uVjs9apKM40o4Mnc-4w_pzEhA-VtGCFN1mI0o8ckuaHCNusclevfChdphkazfOo9lvbAI_QkReAZfNSDnZbktSGy9zo27ez-8-90GKES2dcqOB-yovidmbmTYfLtm2rhjCePWiNScYY99CJNT2BHv8-ftsozwVlr_tg=s300" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="300" data-original-width="300" height="300" src="https://blogger.googleusercontent.com/img/a/AVvXsEikxUr722BTVBIkMnNvEUK-uVjs9apKM40o4Mnc-4w_pzEhA-VtGCFN1mI0o8ckuaHCNusclevfChdphkazfOo9lvbAI_QkReAZfNSDnZbktSGy9zo27ez-8-90GKES2dcqOB-yovidmbmTYfLtm2rhjCePWiNScYY99CJNT2BHv8-ftsozwVlr_tg" width="300" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><br /></div><br /><p></p><p><br /></p><p>Below policy template can be used to provide access to a user or add policy to a role with below set of permissions.</p><p></p><ul style="text-align: left;"><li><span style="color: #38761d;">Allow all Services.</span></li><li><span style="color: #38761d;">Allow all Resources</span></li><li><span style="color: #38761d;">Allow all actions linked to every resource</span></li><li><span style="color: red;">Except IAM all operations and actions.</span></li></ul><div><br /></div><div></div><p></p><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #333333; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "*",
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Deny",
"Action": "iam:*",
"Resource": "*"
}
]
}</b></span></span></pre><p><br /></p><p>I have spent time to explore little template, hope this finds you via google to save some of yours time.</p><p><br /></p>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-12702766762369356192022-02-22T19:34:00.006+05:302022-02-25T13:36:27.674+05:30Jenkins Pipeline for Continuous Integration of AWS Lambda Function with GitHub repository<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjwaHSe1UbvM175hqJYCU5B8bx3_gASBbL4328YK5dqR7KCUR9R702wHCyqxvVaBPKtavxmj8rIpMs0UD_JyjpUdoT7ymyUO7c0tmpMFY5xa4UW1Zo0AvZmzhggFqaVFOVjHNVcQuxC1zXyl1KfLXhTGNt_cZULMrnEm1KtjaTKsaWzqjtLyJ-8vpg" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="436" data-original-width="853" height="205" src="https://blogger.googleusercontent.com/img/a/AVvXsEjwaHSe1UbvM175hqJYCU5B8bx3_gASBbL4328YK5dqR7KCUR9R702wHCyqxvVaBPKtavxmj8rIpMs0UD_JyjpUdoT7ymyUO7c0tmpMFY5xa4UW1Zo0AvZmzhggFqaVFOVjHNVcQuxC1zXyl1KfLXhTGNt_cZULMrnEm1KtjaTKsaWzqjtLyJ-8vpg=w400-h205" width="400" /></a></div><br /><p></p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>AWS Lambda Function </b>is awesome and trust me if you are working on AWS, sooner or later you have to deal with AWS Lambda.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; text-align: left;">Due to the nature of being a PAAS service, we can't ignore the ways of Lambda deployment and it's test methods which is somehow more or less through the Lambda Function Console.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; text-align: left;">Off course there are ways to write code, test code and deploy code directly through IDEs, but keep in mind you still need an ACCESS Key and ACCESS Secret. </p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; text-align: left;">So what about the code base, how will we track the code changes done in Lambda Function itself.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; text-align: left;">In this post, we will cover the challenges with Lambda approaches for CICD and one of the my proposed solutions to address some part of it.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Let's know some of the challenges and their probable solutions.</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"></p><ul style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; line-height: 1.4; margin: 0.5em 0px; padding: 0px 2.5em; text-align: left;"><li style="margin: 0px 0px 0.25em; padding: 0px;">Lambda Deployment : We can use Terraform and CloudFormation for the same, then what's is the challenge ?</li><ul><li style="margin: 0px 0px 0.25em; padding: 0px;">CloudFormation :</li><ul><li style="margin: 0px 0px 0.25em; padding: 0px;">We can use Inline methods to put our Lambda Code under the <b>Code </b>block of ZipFile, but what about the 3rd party modules like panda, we can't use the code block under CloudFormation.</li><li style="margin: 0px 0px 0.25em; padding: 0px;">One can still package those third party modules and other code together, but still one needs to upload those in S3 bucket and think of a way of handling changes before using it.</li></ul></ul><li>Lambda Function Code base : </li><ul><li>We still need to have snapshots of our lambda function code to track the daily changes and to be later use in deployment pipeline.</li></ul></ul><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">There are some more, challenges with Lambda Function, but in this blog post we will try to cover the basic of CICD, that is replicating our Lambda Function code from Console to GitHub repository.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><ul><li>The moment when we talk about CICD, mostly pipeline uses git to get source code and then use it for further process like, checkout, build, release, deploy, test..</li><li>Here the case is somehow different, due to nature of PAAS, we have to test our code's functionality in Lambda Console first, then it can be pushed to repository to save our source code.</li><li>Yes, AWS SAM is yet another option of testing Lambda Function code within our local environment, but not in the case when Lambda is hosted in VPC and it uses other services to communicate.</li></ul><div><br /></div></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Below is one of my proposed solution to achieve the same.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEihk0XOOIgIL7m17Kfd3sL3DguiBp2Na6aZzCzby3Y3TxwFjUcYMc90nydz52BRN07w20-_G7WHWI4K8E9ElxJ8YbiaOcT7YbgzI5MWppRUfZbTyonFCR_Apetpmkt7GGMy3cUMPGWU8ZpuY1-FEXmVASyYg8ZDlq6JvN8OUWz0j33Na8XQSKTjW7A" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="412" data-original-width="840" height="314" src="https://blogger.googleusercontent.com/img/a/AVvXsEihk0XOOIgIL7m17Kfd3sL3DguiBp2Na6aZzCzby3Y3TxwFjUcYMc90nydz52BRN07w20-_G7WHWI4K8E9ElxJ8YbiaOcT7YbgzI5MWppRUfZbTyonFCR_Apetpmkt7GGMy3cUMPGWU8ZpuY1-FEXmVASyYg8ZDlq6JvN8OUWz0j33Na8XQSKTjW7A=w640-h314" width="640" /></a></div><br /><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Prerequisites.</span></span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">IAM Access Key Secrets or IAM Role attached to EC2 instance, from where the Jenkins job is triggered.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">GitHub Personal Access Token</span></span></li></ul></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Here is the flow...</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><ol style="text-align: left;"><li>I assume, developer is initially testing his/her code's functionality on Lambda Console, Once Developer is okay, with his/her Lambda Function code, we will move to next step.</li><li>SysAdmin/Developer can check-in his/her code directly from Lambda Function to GitHub repository using Jenkins Job.</li><li>Jenkins Job has scripted pipeline attached to it, thus it will go through below stages.</li><ul><li>Stage : Check Out code to appropriate branch.</li><li>Stage : Build Docker image from Docker File for Ansible.</li><li>Stage : Run Ansible container from above created Docker image and run Ansible Playbook command to execute Ansible role and it's relative ansible tasks.</li></ul><ol><ol><li>Task 1 - </li><ul><li>Download Lambda Code from Lambda Console using Python Script, which is using boto3 module.</li><li>Unzip the downloaded code into specific directory to track the changes as a file, else changes in zip file can't be tracked.</li></ul><li>Task 2 - </li><ul><li>Clone existing repository from git, replace the existing lambda source code with the newer one downloaded in above step.</li><li> Git add, commit and push it into git repository.</li></ul></ol></ol></ol></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Here is the lab setup.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Our Lambda Function in console has something by name "<b>ck-uat-Lambda-Authorizer</b>"</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg5adWTmCUkwGtUPiOFwSUbxrtEyDOoXEqAlMXPEIKbbpW7Utk8GzSee9-qQzD9x_8x7OuBAa5BTTl_E8ZHWOd6odclAVfDU0Lrz9bvJhxXBfPtxhWRepA2KNvhrBlg1-yBWpRffeA0B8pQp1J8yIyDT2NpJbfX2nei30rHJkZWKMF1yaJnPTrHg1I" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="616" data-original-width="991" height="249" src="https://blogger.googleusercontent.com/img/a/AVvXsEg5adWTmCUkwGtUPiOFwSUbxrtEyDOoXEqAlMXPEIKbbpW7Utk8GzSee9-qQzD9x_8x7OuBAa5BTTl_E8ZHWOd6odclAVfDU0Lrz9bvJhxXBfPtxhWRepA2KNvhrBlg1-yBWpRffeA0B8pQp1J8yIyDT2NpJbfX2nei30rHJkZWKMF1yaJnPTrHg1I=w400-h249" width="400" /></a></div><br />And it's code looks like something below in console.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgxefo0kOmEsEsYsyAooGKcf2IT03LSzsFer4Xx50cZqQd-9Gzm0mNXLsg0S2otwtE2hYW6NQanygSpHYMzqe2quL-S0G9xU5Yr3ofgZHIvFQV7AQUdaAaernGG5kebEHMH8s3F-QbJK3x5nXI1eqCIceWwmZNgF9FlB8cNDoEA3bdIsQbJpReSmYU" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="603" data-original-width="975" height="248" src="https://blogger.googleusercontent.com/img/a/AVvXsEgxefo0kOmEsEsYsyAooGKcf2IT03LSzsFer4Xx50cZqQd-9Gzm0mNXLsg0S2otwtE2hYW6NQanygSpHYMzqe2quL-S0G9xU5Yr3ofgZHIvFQV7AQUdaAaernGG5kebEHMH8s3F-QbJK3x5nXI1eqCIceWwmZNgF9FlB8cNDoEA3bdIsQbJpReSmYU=w400-h248" width="400" /></a></div><br /><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">GitHub repository where I want to publish my code.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Link : <a href="https://github.com/Jackuna/aws-swa/tree/lambda_deployer">https://github.com/Jackuna/aws-swa/tree/lambda_deployer</a></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Repo Snapshot.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhb72pgYnAUPGkq4EAx8aHk39kMNTMYWaGbtflA6lANKp9y9JA1qzwwNiDsoqFM8Uyo04KtXtcd709mIFm7FUs2LotQBiGoPshluypNZ0N_IZx8nzRGssN9tzbhEgjpSwVP6K_ivxUvw4Rf_SAjdTZsjfRwfJedClIhVa7VZeQu6O1vqgI3ihlbIH4" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="856" data-original-width="1375" height="249" src="https://blogger.googleusercontent.com/img/a/AVvXsEhb72pgYnAUPGkq4EAx8aHk39kMNTMYWaGbtflA6lANKp9y9JA1qzwwNiDsoqFM8Uyo04KtXtcd709mIFm7FUs2LotQBiGoPshluypNZ0N_IZx8nzRGssN9tzbhEgjpSwVP6K_ivxUvw4Rf_SAjdTZsjfRwfJedClIhVa7VZeQu6O1vqgI3ihlbIH4=w400-h249" width="400" /></a></div><br /><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Directory Layout for the same...</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhhkDXShXgZj2NLOfd97Wv1cbM4zaXRmMfKALk_JG9uxOxrPu_F5liuPisIQuf0_Q_fFYem4e583FaE24-oCc53pwqYvGc3jvIijNbSK8VfL9LG4l5Zbf8oEEowTdxJs3YzIeFVyG1Y0cVt1jSU8q3TEkyWo01Cd2aKlsSs5H44varckJNJV7xP094" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="526" data-original-width="475" height="400" src="https://blogger.googleusercontent.com/img/a/AVvXsEhhkDXShXgZj2NLOfd97Wv1cbM4zaXRmMfKALk_JG9uxOxrPu_F5liuPisIQuf0_Q_fFYem4e583FaE24-oCc53pwqYvGc3jvIijNbSK8VfL9LG4l5Zbf8oEEowTdxJs3YzIeFVyG1Y0cVt1jSU8q3TEkyWo01Cd2aKlsSs5H44varckJNJV7xP094=w362-h400" width="362" /></a></div></div><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br />Our Final Intention is to dump or lambda function code under src directory, that is <b>lambda_folder</b>/<b>src</b></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">So according to the flow stated earlier in the post, I will paste the code..</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Jenkins Scripted Pipeline code.</span></span></div><div><br /></div><div>Note: Do mask the additional secrets to avoid it to be appear in plain text.</div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>def gituser = env.GIT_USERNAME
def gituserpass = env.GIT_PASSWORD
def ACCESS_KEY = env.AWS_ACCESS_KEY
def KEY_ID = env.AWS_SECRET_ACCESS_KEY
def DEBUG_MODE = env.LOG_TYPE
node('master'){
try {
stage('Git Checkout'){
checkout scm
sh "git checkout lambda_deployer"
}
stage('build'){
sh "ls -ltr"
echo "Building docker image via dockerfile..."
sh "docker build -t ansible:2.10-$BUILD_ID ."
}
stage('deploy'){
echo "Infrastructure deployment started...."
wrap([$class: "MaskPasswordsBuildWrapper",
varPasswordPairs: [[password: gituserpass, var: gituserpass] ]]) {
sh "docker run --rm \
-e gituser=$gituser \
-e gituserpass=$gituserpass \
-e AWS_ACCESS_KEY_ID=$ACCESS_KEY \
-e AWS_SECRET_ACCESS_KEY=$KEY_ID \
-e AWS_DEFAULT_REGION='ap-south-1' \
ansible:2.10-$BUILD_ID ansible-playbook -$DEBUG_MODE --extra-vars 'env=dev1 git_username=${gituser} token=${gituserpass}' lambda_folder/root_lambda_project.yml"
}
}
}
catch (e){
echo "Error occurred - " + e.toString()
throw e
}
finally {
deleteDir()
sh 'docker rmi -f ansible:2.10-$BUILD_ID && echo "ansible:2.10-$BUILD_ID local image deleted."'
}
}</b></span></span></pre></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Build Pipe Line should have something like below in Jenkins Console.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiozwjZip1iQf_MY8uVi-BFfq-lc_dEjUN4fswUtdJIr0NMSDUYz2Z49Du-1K5uL01YtjLcWAaQJFBsL_G8J1OHqrOxtBzpYyFXQC9MjgAHYK4r2fnsKbFqOsb7OrMchb9w2F89YHI3dNFPOmIFvQqBlGDeiFv6cOsi9VLLOJf5fhXiiJw24AzgLRo" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="855" data-original-width="988" height="240" src="https://blogger.googleusercontent.com/img/a/AVvXsEiozwjZip1iQf_MY8uVi-BFfq-lc_dEjUN4fswUtdJIr0NMSDUYz2Z49Du-1K5uL01YtjLcWAaQJFBsL_G8J1OHqrOxtBzpYyFXQC9MjgAHYK4r2fnsKbFqOsb7OrMchb9w2F89YHI3dNFPOmIFvQqBlGDeiFv6cOsi9VLLOJf5fhXiiJw24AzgLRo" width="277" /></a></div><br /><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Jenkins One of the Stage : Build will build docker image from Docker File, here is the docker file source code.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>FROM python:3.7
RUN python3 -m pip install ansible==2.10 boto3 awscli
RUN rm -rf /usr/local/ansible/
copy lambda_folder /usr/local/ansible/lambda_folder
WORKDIR usr/local/ansible/
CMD ["ansible-playbook", "--version"]</b></span></span></pre></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Once Docker Images is created, next step is to run Docker container from the above created Ansible image.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Here is the Ansible Role and it's respective tasks.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Ansible Root Playbook YAML -- <b>root_lambda_project.yml</b></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>---
- hosts: localhost
connection: local
gather_facts: False
roles:
- role-</b></span></span></pre></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Ansible Variable file under roles -- <b>lambda_folder/role/vars/dev1/main.yml</b></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>---
region: us-east-1
function_name: ck-uat-LambdaAuthorizer
git_repo_name: aws-swa
git_repo_branch: lambda_deployer</b></span></span></pre></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Python Script, that will be called on one of the Ansible Task to download Lambda Function code </span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Note : It's an edited version of existing version of code from stackoverflow.</span></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><pre style="overflow-wrap: break-word; white-space: pre-wrap;">"""
Script to download individual Lambda Function and dump code in specified directory
"""
import os
import sys
from urllib.request import urlopen
import zipfile
from io import BytesIO
import boto3
def get_lambda_functions_code_url(fn_name):
client = boto3.client("lambda")
functions_code_url = []
fn_code = client.get_function(FunctionName=fn_name)["Code"]
fn_code["FunctionName"] = fn_name
functions_code_url.append(fn_code)
return functions_code_url
def download_lambda_function_code(fn_name, fn_code_link, dir_path):
function_path = os.path.join(dir_path, fn_name)
if not os.path.exists(function_path):
os.mkdir(function_path)
with urlopen(fn_code_link) as lambda_extract:
with zipfile.ZipFile(BytesIO(lambda_extract.read())) as zfile:
zfile.extractall(function_path)
if __name__ == "__main__":
inp = sys.argv[1:]
print("Destination folder {}".format(inp))
if inp and os.path.exists(inp[0]):
dest = os.path.abspath(inp[0])
fc = get_lambda_functions_code_url(sys.argv[2])
for i, f in enumerate(fc):
print("Downloading Lambda function {}".format(f["FunctionName"]))
download_lambda_function_code(f["FunctionName"], f["Location"], dest)
else:
print("Destination folder doesn't exist")</pre></pre></div><div><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><br /></b></span></span></div><div><br /></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Ansible Task 1 : <b>lambda_folder/role/tasks/download_lambda_code.yml</b></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b><br /></b></span></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>---
- name: Read Variables
include_vars:
file: "role/vars/{{ env }}/main.yml"
- name: Download Lambda Function using Python script..
command:
argv:
- python3
- role/files/download_lambda.py
- src
- "{{ function_name }}"</b></span></span></pre></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Ansible Task 2 : </span><b style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">lambda_folder/role/tasks/</b><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b>update_repository.yml</b></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b><br /></b></span></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>---
- name: Git clone source repository..
command:
argv:
- git
- clone
- https://{{ git_username }}:{{ token }}@github.com/Jackuna/{{ git_repo_name }}.git
- -b
- "{{ git_repo_branch }}"
- name: Git add Lambda function source code to repo..
command: >
cp -r src {{ git_repo_name }}/lambda_folder
- name: Git add recent changes..
command: >
git add --all lambda_folder/src
args:
chdir: "{{ git_repo_name }}"
- name: Git Config username..
command: >
git config user.name {{ git_username }}
args:
chdir: "{{ git_repo_name }}"
- name: Git Config email..
command: >
git config user.email {{ git_username }}@cyberkeeda.com
args:
chdir: "{{ git_repo_name }}"
- name: Git commit recent changes..
command: >
git commit -m "Updated Latest code"
args:
chdir: "{{ git_repo_name }}"
- name: Git push recent changes..
command:
argv:
- git
- push
- https://{{ git_username }}:{{ token }}@github.com/Jackuna/{{ git_repo_name }}.git
- -u
- "{{ git_repo_branch }}"
args:
chdir: "{{ git_repo_name }}"
register: git_push_output </b></span></span></pre></div><div><br /></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">That's all you need.. in case of hurdles or issues, do comment !</span></span></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-58847470067069449932022-01-26T17:41:00.007+05:302022-01-26T19:57:13.655+05:30What are terraform providers and how to use it.<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiapdvfUSUJcGT-z3Xm7p11urFrP8rXZ9TQx72PBwFtmDuON6ODlIfgDjRL3HsGKbXFvyDFU0JNtz6dxT1F5vSv5tXnNq5xb66XCdUEHM5bJ7NKEKF0oUtlDGPh3UCODvox2SP7jfHRuuo/" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="988" data-original-width="1845" height="214" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiapdvfUSUJcGT-z3Xm7p11urFrP8rXZ9TQx72PBwFtmDuON6ODlIfgDjRL3HsGKbXFvyDFU0JNtz6dxT1F5vSv5tXnNq5xb66XCdUEHM5bJ7NKEKF0oUtlDGPh3UCODvox2SP7jfHRuuo/w400-h214/image.png" width="400" /></a></div><p></p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"> </p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Within this post, we will cover </p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"></p><p></p><ul style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; line-height: 1.4; margin: 0.5em 0px; padding: 0px 2.5em; text-align: left;"><li style="margin: 0px 0px 0.25em; padding: 0px;">What are terraform providers</li><li style="margin: 0px 0px 0.25em; padding: 0px;">Documentation link for providers.</li><li style="margin: 0px 0px 0.25em; padding: 0px;">How to choose providers</li><li style="margin: 0px 0px 0.25em; padding: 0px;">How to define providers within your terraform file.</li><li style="margin: 0px 0px 0.25em; padding: 0px;">Providers Versions.</li><ul><li style="margin: 0px 0px 0.25em; padding: 0px;">How to find terraform provider versions.</li><li style="margin: 0px 0px 0.25em; padding: 0px;">How to explicitly mention provider version in terraform file. </li></ul></ul><div><br /></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>What are terraform providers ?</b></span></div><div style="text-align: left;"><br /></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Terraform support <b>N</b> number of providers.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">When we say providers it's basically terraform supported binaries and plugins for individual provider's subcategories like for example ( aws, azure, gcp etc ).</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">To be specific, terraform documentation categorized providers into multiple parts, which are mainly.</span></span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Major Clouds</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">AWS</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">GCP</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Azure</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">OCI</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Digital Ocean</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">VMware</span></span></li></ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Clouds</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Other Cloud providers.</span></span></li></ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Infrastructure Software.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Network</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">VCS</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Monitor and System Management</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Database</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Community.</span></span></li></ul></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>Documentation link for providers.</b></span></div><div style="text-align: left;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;"><span style="color: #333333; font-family: Lora, serif;">Link : <a href="https://registry.terraform.io/browse/providers">https://registry.terraform.io/browse/providers</a></span></span></div><div style="text-align: left;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;"><span style="color: #333333; font-family: Lora, serif;"><b><br /></b></span></span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>How to choose providers ?</b></span></div><div style="text-align: left;"><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Before, you start writing your first terraform file, you must choose appropriate provider to provision</span></span></div><div style="text-align: left;"><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">desired infrastructure.</span></span></div><div style="text-align: left;"><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">For example, incase if you want to create a VPC subnet in AWS, you must choose <b>AWS</b> provider and define the same within your terraform script.</span></span></div><div style="text-align: left;"><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div style="text-align: left;"><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Navigate to official link to know more about supported provider. : <b><a href=" https://registry.terraform.io/browse/providers" target="_blank">Link</a></b></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Please note, there are labels which also differentiate providers authors and owners.</span></span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Official</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Officially maintained and supported and tested by Hashicorp</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Note : They can be installed directly by executing terraform init command.</span></span></li></ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Verified.</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Verified modules are reviewed by Hashicorp and are actively maintained by contributors, these badges appear next after the verification by Hashicorp.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b>Note : They can't be installed directly by executing terraform init command.</b></span></span></li></ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Community</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">3rd Party plugin and modules, not actively maintained.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b>Note : They can't be installed directly by executing terraform init command.</b></span></span></li></ul></ul><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">How to define providers within your terraform file ?</span></div></div><div style="text-align: left;"><ul style="text-align: left;"><li><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Create an empty file within your IDE and give it a extension of <b>.tf</b></span></li></ul><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; color: #333333; font-size: 14px; letter-spacing: 1px; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace;"><span style="white-space: pre-wrap;"><b>$ touch create_new_ec2_instance.tf</b></span></span></pre></div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;">Next step is to choose format to define provider from our official terraform documentation.</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;">Navigate to official provider <a href="https://registry.terraform.io/browse/providers" target="_blank">Link</a></span></span></li><li><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Select your provider as per your requirement.</span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;">For example, I need to create an EC2 instance, hence I must select AWS as provider.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;">Incase If I want to create a Azure Blob Container, I must select azure as my provider.</span></span></li></ul></ul><li><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Once provider is selected, toggle to the Documentation from Navigation bar.</span></li></ul><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivrQ_J17Ig1S8RkPzviq1898poluzftfgj97S42690BxGQvUw0CTDg6AQKiaRHStHolZonitDZFCC560ETwqqoisp9XSzgW74equeqPSWQiVnAnZmaUGowPwWQjIDQCACiz-DwfmzOdSg/" style="margin-left: 1em; margin-right: 1em;"><img data-original-height="456" data-original-width="1828" height="100" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivrQ_J17Ig1S8RkPzviq1898poluzftfgj97S42690BxGQvUw0CTDg6AQKiaRHStHolZonitDZFCC560ETwqqoisp9XSzgW74equeqPSWQiVnAnZmaUGowPwWQjIDQCACiz-DwfmzOdSg/w400-h100/image.png" width="400" /></a></div><br /></span></span></div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Within documentation. scroll to the Example Usage section and look for provider section, how it has been defined.</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Please note before you define, providers and start executing your terraform you must have the authentication mechanism ready with you, It's very obvious if you want to provision any infrastructure on any public cloud, you must be authenticated first.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Every Providers has different way of authentication.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">It's not mandatory or even discouraged to keep credentials hardcoded in a file, one work around is to define environment variables and import it during runtime.</span></span></li></ul></ul><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>provider "aws" {
region = "us-east-1"
access_key = "AKIXXXXXXXXHB5PO7T6G"
secret_key = "UdB1/aXJ9QgbQUSBS8BS9NWdrjr3wRbjE7hKddTD"
}</b></span></span></pre><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">In case, if we want to use the export method of key, we can export keys and secret during terraform init command.</span></span></div><div style="text-align: left;"><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div style="text-align: left;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>provider "aws" {}</b></span></span></pre><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ export AWS_ACCESS_KEY_ID="myaccesskey"
$ export AWS_SECRET_ACCESS_KEY="myaccesssecret"
$ export AWS_DEFAULT_REGION="us-east-1"
$ terraform plan</b></span></span></pre><ul style="text-align: left;"><li><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Below snipped is to define Azure provider</span></li><ul><li><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Azure authentication can be done using multiple methods like Azure cli authentication, service principle and other too.</span></li></ul></ul></div><div style="text-align: left;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b># Configure the Microsoft Azure Provider
provider "azurerm" {
features {}
}</b></span></span></pre></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b><u>Provider Versions.</u></b></span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div style="text-align: left;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjcLbZSF7MzuyKS-uMAkGH2gcDmrdTWPIyFbAFEBjpZpJCrc390VpTkBq5-8uoDjDKjUQQsPdIQCGgwWUwU2N9o1b3u77rqhCXCPKxvNK-XccP_-1jul7KMhyqrvmOukna-p_wHoNCGF7u6guh4xC70XZZv7WMzMEZVBviYW9Osz1VETFhpsx5LZy0=s656" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="271" data-original-width="656" height="165" src="https://blogger.googleusercontent.com/img/a/AVvXsEjcLbZSF7MzuyKS-uMAkGH2gcDmrdTWPIyFbAFEBjpZpJCrc390VpTkBq5-8uoDjDKjUQQsPdIQCGgwWUwU2N9o1b3u77rqhCXCPKxvNK-XccP_-1jul7KMhyqrvmOukna-p_wHoNCGF7u6guh4xC70XZZv7WMzMEZVBviYW9Osz1VETFhpsx5LZy0=w400-h165" width="400" /></a></div><br /><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Provider sits in between terraform binary and Infrastructure provisioning, Providers are set of plugins that invokes APIs to create requested infrastructure in terraform file.</span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Here in above diagram, we will be creating a EC2 resource from terraform file named as create_ec2.tf</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: red;">provider "aws" {
region = "us-east-1"
access_key = "AKIA5BMYACCESSKEY"
secret_key = "UdB1/MYACCESSSECRETIWIW7EH303"
}</span><span style="color: #2b00fe;">
resource "aws_instance" "my-ec2-instance" {
ami = "ami-08e4e35cccc6189f4" # us-west-1
instance_type = "t2.micro"
tags = {
Name = "my-ec2"
}
}</span></b></span></span></pre></div><div style="text-align: left;"><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;">Provider used here is AWS.</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;">Please note under the provider section, we nowhere mentioned the version of aws provider.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white;"><span style="font-size: 14px; letter-spacing: 1px;">Incase, if provider version is not explicitly mentioned, it will download the latest version available during the terraform init command.</span></span></span></li></ul></ul></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">How to find, version of providers ?</span></div><div style="text-align: left;"><ul style="text-align: left;"><li><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">For this, navigate to <a href="https://registry.terraform.io/browse/providers" target="_blank">official providers page </a>and select your provider.</span></li><li><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Here I am demonstrating the version of providers for aws.</span></li><ul><li><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Navigate to provider link. <a href="https://registry.terraform.io/providers/hashicorp/aws/latest">https://registry.terraform.io/providers/hashicorp/aws/latest</a></span></li><li><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Under version, you can look for all available versions.</span></li></ul></ul><div><span style="color: #333333; font-family: Lora, serif;"><div class="separator" style="clear: both; font-size: 14px; letter-spacing: 1px; text-align: center;"><br /></div><div style="font-size: 14px; letter-spacing: 1px;"><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiJdCVDAL7ocFQK0SPHumBcucNsAHGdHlCSrfOxygL3BMwpe7JMes6Re_B77Ati1rfsZVB8U3Tofvz8bWm8it0kQQvq09jYeRdlV5jpfJ1_lgXpNYCZQwMcsp0CtBbP4QfxSxpFJ2H8j0umK0vr6TXqHjNwFjkyrFD7fekKryZJShoyx0bn-6x7_hs=s1168" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="771" data-original-width="1168" height="264" src="https://blogger.googleusercontent.com/img/a/AVvXsEiJdCVDAL7ocFQK0SPHumBcucNsAHGdHlCSrfOxygL3BMwpe7JMes6Re_B77Ati1rfsZVB8U3Tofvz8bWm8it0kQQvq09jYeRdlV5jpfJ1_lgXpNYCZQwMcsp0CtBbP4QfxSxpFJ2H8j0umK0vr6TXqHjNwFjkyrFD7fekKryZJShoyx0bn-6x7_hs=w400-h264" width="400" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><br /></div></span></span></div><div style="font-size: 14px; letter-spacing: 1px;"><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><span style="font-size: 14px; letter-spacing: 1px;">How to define provider version explicitly in terraform file ?</span></span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">This is very useful, as this is the ideal way of using providers in production environment to avoid the adverse effect of new release to our existing infrastructure.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Below is the way, how we can define provider version in terraform file.</span></span></li></ul><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>provider "aws" {
region = "us-east-1"
version = "3.70.0"
}</b></span></span></pre></div></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">We can also use operators to define as like any other language, use version equal to, greater than, less than like below.</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b style="background-color: whitesmoke; color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; white-space: pre-wrap;">version = "3.70.0"</b></span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b style="background-color: whitesmoke; color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; white-space: pre-wrap;">version = "<=3.70.0"</b></span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b style="background-color: whitesmoke; color: #2b00fe; font-family: monaco, menlo, consolas, "courier new", monospace; white-space: pre-wrap;">version = ">=3.70.0"</b> </span></span></li></ul></ul><div></div><span style="color: #333333; font-family: Lora, serif;"><br /></span></div></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">As on Terraform version greater then 0.13+, Version and Providers can be stated like below.</span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div style="text-align: left;"><span style="background-color: white;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace; font-size: 14px; letter-spacing: normal;"><span style="letter-spacing: 1px; white-space: pre-wrap;"><b> </b></span></span><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">terraform {
required_providers {
aws = {
source = "hashicorp/aws"
</span><span style="color: red;">version = "3.73.0"</span><span style="color: #2b00fe;">
}
}
}
provider "aws" {
region = "us-east-1"
access_key = "XXXXXXXXXXXXXXXXXXXX"
secret_key = "UdB1/YYYYYYYYYYYYYYYYYYYYYYYYYYY"
}
<br /></span></b></span></span></pre></span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">This is all about providers in this post, there are still more to explore and apply, will keep this thread updated.</span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div style="text-align: left;"><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Hope, this document helps you in some way !<br /><br /></span></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-81176784918465433152022-01-07T22:22:00.019+05:302022-01-18T11:50:40.599+05:30Jenkins Pipeline to create CloudFront Distribution using S3 as Origin <p> </p><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhr10jojmEHBgFU4cV1H-qnxcFhVy3zyBQqkjt37LkzjnbvIxLi4gYa_7L5xXMj35cVmzNsQ2Lxv08PbQYV5qIz_ADupyeG7bqTi0c91R92cS2ukz6vxtPe4Ad1zHe81HAPqR2SExpfrCc/" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="862" data-original-width="1740" height="318" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhr10jojmEHBgFU4cV1H-qnxcFhVy3zyBQqkjt37LkzjnbvIxLi4gYa_7L5xXMj35cVmzNsQ2Lxv08PbQYV5qIz_ADupyeG7bqTi0c91R92cS2ukz6vxtPe4Ad1zHe81HAPqR2SExpfrCc/w640-h318/image.png" width="640" /></a></div></div><p></p><br /><p></p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"> Within this post, we will cover </p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"></p><ul style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; line-height: 1.4; margin: 0.5em 0px; padding: 0px 2.5em; text-align: left;"><li>Complete Jenkins Scripted pipeline.</li><li>Complete IAAC (Infrastructure As A Code ) to deploy AWS services and It's respective configuration.</li><li style="margin: 0px 0px 0.25em; padding: 0px;">Jenkins integration with GitHub as code repository.</li><li style="margin: 0px 0px 0.25em; padding: 0px;">Jenkins integration with Docker to make the deployment platform independent.</li><li style="margin: 0px 0px 0.25em; padding: 0px;">Jenkins integration with Ansible to call AWS CloudFormation scripts.</li><li style="margin: 0px 0px 0.25em; padding: 0px;">Using Ansible roles to fill the gaps of AWS CloudFormation, basically in this blog post and lab environment I'm using it to bypass usage of AWS CloudFormation Stack Sets and Custom Resources.</li></ul><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Flow diagram explaining the automation.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiSg-2f-bRTTcb4tnvKQO0e16qYR0FrBetdexUWGgrg3ftAPj7TOtik3o2p5JsCbQUoT0Sghzy7BxYaHiBA0aAP2Xj3L2JXxT7GSivp2VdkvWyhMNrKEan5CWjBXAYpSQPovRtNDHfMtKfdCjg_ftfBxTVqmbG9seqE3n43Bo1RtAO990Hf_JMTpVg=s661" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="261" data-original-width="661" src="https://blogger.googleusercontent.com/img/a/AVvXsEiSg-2f-bRTTcb4tnvKQO0e16qYR0FrBetdexUWGgrg3ftAPj7TOtik3o2p5JsCbQUoT0Sghzy7BxYaHiBA0aAP2Xj3L2JXxT7GSivp2VdkvWyhMNrKEan5CWjBXAYpSQPovRtNDHfMtKfdCjg_ftfBxTVqmbG9seqE3n43Bo1RtAO990Hf_JMTpVg=s16000" /></a></div><br /></span></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Explanation of above flow diagram.</span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Once Jenkins Job is triggered with appropriate input variables.</span></span></div><div><ol style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">It starts with fetching source code from git repository, which contains.</span></span></li><ol><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Source Code for applications ( HTML, CSS, JS )</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">IAAC code to support infrastructures deployment.</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Ansible Role, playbooks.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">CloudFormation templates.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Jenkins File, which has scripted pipeline defined.</span></span></li></ul></ol><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Once source code is downloaded, it will look for Jenkins pipeline file named a Jenkinsfile.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Once Jenkins file is executed, it will initiate the pipeline in below stages.</span></span></li><ol><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Stage <b>Checkout</b> : It looks for deployment type, as normal build or rollback and based upon it, it will checkout to respective git branch or tag.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Stage <b>Build : </b>To make the pipeline, platform independent and reusable in nature, instead of directly triggering jobs on Jenkins node via bash or powershell commands, we will be using docker containers to run our CLI commands.</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Here we will use Ansible Playbooks to create Infrastructure, thus in this step we will build a Ansible docker image from Docker file.</span></span></li></ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Stage </span><b style="font-size: 14px; letter-spacing: 1px;">Deploy</b><span style="font-size: 14px; letter-spacing: 1px;">: Once our pre-requisites are ready ( Ansible Docker Image ), we will run ansible container and trigger ansible-playbook command on the fly with appropriate Environment variables and Variables.</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Ansible playbook ( <b>root.yml</b> ) is executed, which has the roles defined under it by name <b>ansible_role</b></span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">I have removed non used default directories like (meta, default, handlers, tests etc. ) as these are not being used within our requirement.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Ansible role has three task playbook files with below operations.</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Create S3 bucket : It will use ansible's role amazon.aws.s3_bucket to creates s3 bucket with tags and restricted public access. </span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Create empty directories within above created S3 buckets.: It will use ansible's role amazon.aws.aws_s3 to create bucket objects.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Create CloudFormation distributions : It will use ansible's role amazon.aws.cloudformation option to create CloudFront distribution via CloudFormation template.</span></span></li></ul></ul></ol></ol><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b>Jenkins file used in this lab.</b></span></span></div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>def ENVT = env.ENVIRONMENT
def VERSION = env.VERSION
def JOBTYPE = env.JOBTYPE
def ACCESS_KEY = env.AWS_ACCESS_KEY
def KEY_ID = env.AWS_SECRET_ACCESS_KEY
node('master'){
try {
stage('checkout'){
if ( "${VERSION}" == 'default') {
checkout scm
}
else {
checkout scm
sh "git checkout $VERSION"
}
}
stage('build'){
sh "ls -ltr"
echo "Building docker image via dockerfile..."
sh "docker build -t ck-pwdgen-app/ansible:2.10-$BUILD_ID ."
}
stage('deploy'){
echo "Infrastructure deployment started...."
wrap([$class: "MaskPasswordsBuildWrapper",
varPasswordPairs: [[password: ACCESS_KEY, var: ACCESS_KEY], [password: KEY_ID, var: KEY_ID] ]]) {
sh "docker run \
-e AWS_ACCESS_KEY_ID=$ACCESS_KEY \
-e AWS_SECRET_ACCESS_KEY=$KEY_ID \
-e AWS_DEFAULT_REGION='us-west-1' \
ck-pwdgen-app/ansible:2.10-$BUILD_ID ansible-playbook -vvv --extra-vars 'Environment=${ENVT}' root.yml"
}
}
}
catch (e){
echo "Error occurred - " + e.toString()
throw e
}
finally {
deleteDir()
if ( "${JOBTYPE}" == 'build-deploy') {
sh 'docker rmi -f ck-pwdgen-app/ansible:2.10-$BUILD_ID && echo "ck-pwdgen-app/ansible:2.10-$BUILD_ID local image deleted."'
}
}
}</b></span></span></pre></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b><br /></b></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b>Jenkins Pipeline job will look something like below.</b></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhjh6qToWYyUXw8bNkUrm0aXJt4mUjk26yd0x1SziPs3I9kOVUNmuC3vO2sbdsX4HOwY5bF68GuvtCZ5Rrrvzhw0p0lnz2RW8vPEz1kfFa-tfHVktjEAoN48HD_pcicAExkmF9RtnXkBkmiPzOq458EKKN-7uXfR2w55K4INDfXM9cNOWF2HT7tihk=s1006" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="736" data-original-width="1006" height="293" src="https://blogger.googleusercontent.com/img/a/AVvXsEhjh6qToWYyUXw8bNkUrm0aXJt4mUjk26yd0x1SziPs3I9kOVUNmuC3vO2sbdsX4HOwY5bF68GuvtCZ5Rrrvzhw0p0lnz2RW8vPEz1kfFa-tfHVktjEAoN48HD_pcicAExkmF9RtnXkBkmiPzOq458EKKN-7uXfR2w55K4INDfXM9cNOWF2HT7tihk=w400-h293" width="400" /></a></div><br /> </b></span></span></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>Dockerfile used to create Ansible Image</b></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>FROM python:3.7
RUN python3 -m pip install ansible==2.10 boto3 awscli && ansible-galaxy collection install amazon.aws
ADD root.yml /usr/local/ansible/
COPY ansible_role /usr/local/ansible/ansible_role
WORKDIR usr/local/ansible/
CMD ["ansible-playbook", "--version"]</b></span></span></pre></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b><br /></b></span></div><div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>Ansible Role directory structure and it's respective file contents.</b></span></div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>root.yml
|
ansible_role/
├── README.md
├── tasks
│ ├── create_bucket_directories.yml
│ ├── create_cloudfront_dist.yml
│ ├── create_s3_bucket.yml
│ └── main.yml
└── vars
└── int
└── main.yml
3 directories, 6 files</b></span></span></pre></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>Ansible Entry Playbook file ( root.yml ), we will initiate the ansible tasks using roles defined in below file.</b></span></div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">
</span>$ cat root.yml
<span style="color: #2b00fe;">
---
- hosts: localhost
connection: local
gather_facts: False
roles:
- ansible_role</span></b></span></span></pre></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>Ansible Roles Variable file content.</b></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">
</span>$ cat ansible_role/vars/int/main.yml
<span style="color: #2b00fe;">
---
# default variables
region: us-east-1
ProductName: ck
ProjectName: pwdgen
Environment: int
PrimaryRegion: us-east-1
SecondaryRegion: us-east-2
bucketCfg:
int:
Environment: "{{ Environment }}"
PrimarBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-primary-bucket"
SecondaryBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-secondary-bucket"
CDNLogBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-cdn-logs-bucket"
DevopsBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-devops-bucket"
PrimaryBucketRegion: "{{ PrimaryRegion }}"
SecondaryBucketRegion: "{{SecondaryRegion}}"
DevopsBucketRegion: "{{ PrimaryRegion }}"
bucketTags:
int:
PrimaryBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-primary"
SecondaryBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-secondary"
Environment: "{{ Environment }}"
Owner: "admin@cyberkeeda.com"</span></b></span></span></pre></div></div><div><br /></div><div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>Ansible Role Tasks file contents.</b></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ cat ansible_role/tasks/main.yml
<span style="color: #2b00fe;">
---
- import_tasks: create_s3_bucket.yml
- import_tasks: create_bucket_directories.yml
- import_tasks: create_cloudfront_dist.yml</span></b></span></span></pre></div></div><div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>Ansible Role Tasks file contents.</b></span></div><div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">$ </b><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>cat ansible_role/tasks/create_s3_bucket.yml</b></span></span></pre></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span><span style="color: #2b00fe;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>
- name: Read environment specific variables.
include_vars:
file: "ansible_role/vars/{{ Environment }}/main.yml"
- name: Create static-ck application buckets in us-east-1 region.
s3_bucket:
name: "{{ item }}"
state: absent
tags:
Name: "{{ item }}"
Environment: "{{ Environment }}"
Owner: "{{ bucketTags[Environment]['Owner'] }}"
region: us-east-1
public_access:
block_public_acls: true
ignore_public_acls: true
block_public_policy: true
restrict_public_buckets: true
with_items:
- "{{ bucketCfg[Environment]['PrimarBucketName'] }}"
- "{{ bucketCfg[Environment]['DevopsBucketName'] }}"
- "{{ bucketCfg[Environment]['CDNLogBucketName'] }}"
- name: Create static-ck application buckets in us-east-2 region.
s3_bucket:
name: "{{ item }}"
state: absent
tags:
Name: "{{ item }}"
Environment: "{{ Environment }}"
Owner: "{{ bucketTags[Environment]['Owner'] }}"
region: us-east-2
public_access:
block_public_acls: true
ignore_public_acls: true
block_public_policy: true
restrict_public_buckets: true
with_items:
- "{{ bucketCfg[Environment]['SecondaryBucketName'] }}"
</b></span><b style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">
</b></span></span></span></pre></div><div><br /></div><div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>Ansible Role Tasks file contents.</b></span></div><div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">$ </b><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>cat ansible_role/tasks/create_bucket_directories.yml</b></span></span></pre></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>---
- name: Read environment specific variables.
include_vars:
file: "ansible_role/vars/{{ Environment }}/main.yml"
- name: Create empty directories to store build artifacts.
aws_s3:
bucket: "{{ item.bucket_name }}"
object: "{{ item.artifact_dir }}"
mode: create
with_items:
- { bucket_name: "{{ bucketCfg[Environment]['PrimarBucketName'] }}", artifact_dir: "/app1/artifacts" }
- { bucket_name: "{{ bucketCfg[Environment]['SecondaryBucketName'] }}", artifact_dir: "/app1/artifacts" }
- name: Create empty directories to deploy latest build.
aws_s3:
bucket: "{{ item.bucket_name }}"
object: "{{ item.latest_dir }}"
mode: create
with_items:
- { bucket_name: "{{ bucketCfg[Environment]['PrimarBucketName'] }}", latest_dir: "/app1/latest" }
- { bucket_name: "{{ bucketCfg[Environment]['SecondaryBucketName'] }}", latest_dir: "/app1/latest" }</b></span><b style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">
</b></span></span></pre></div></div><div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>Ansible Role Tasks file contents.</b></span></div><div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">$ </b><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>cat ansible_role/tasks/create_cloudfront_dist.yml</b></span></span></pre></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>AWSTemplateFormatVersion: '2010-09-09'
Description: 'CF Template to setup infra for static password generator application'
Parameters:
Environment:
Description: Please specify the target environment.
Type: String
Default: "int"
AllowedValues:
- int
- pre-prod
- prod
AppName:
Description: Application name.
Type: String
Default: "pwdgen"
AlternateDomainNames:
Description: CNAMEs (alternate domain names)
Type: String
Default: "jackuna.github.io"
IPV6Enabled:
Description: Should CloudFront to respond to IPv6 DNS requests with an IPv6 address for your distribution.
Type: String
Default: true
AllowedValues:
- true
- false
OriginProtocolPolicy:
Description: CloudFront Origin Protocol Policy to apply to your origin.
Type: String
Default: "https-only"
AllowedValues:
- http-only
- match-viewer
- https-only
Compress:
Description: CloudFront Origin Protocol Policy to apply to your origin.
Type: String
Default: "true"
AllowedValues:
- true
- false
DefaultTTL:
Description: The default time in seconds that objects stay in CloudFront caches before CloudFront forwards another request to your custom origin. By default, AWS CloudFormation specifies 86400 seconds (one day).
Type: String
Default: "540.0"
MaxTTL:
Description: The maximum time in seconds that objects stay in CloudFront caches before CloudFront forwards another request to your custom origin. By default, AWS CloudFormation specifies 31536000 seconds (one year).
Type: String
Default: "600.0"
MinTTL:
Description: The minimum amount of time that you want objects to stay in the cache before CloudFront queries your origin to see whether the object has been updated.
Type: String
Default: "1.0"
SmoothStreaming:
Description: Indicates whether to use the origin that is associated with this cache behavior to distribute media files in the Microsoft Smooth Streaming format.
Type: String
Default: "false"
AllowedValues:
- true
- false
QueryString:
Description: Indicates whether you want CloudFront to forward query strings to the origin that is associated with this cache behavior.
Type: String
Default: "false"
AllowedValues:
- true
- false
ForwardCookies:
Description: Forwards specified cookies to the origin of the cache behavior.
Type: String
Default: "none"
AllowedValues:
- all
- whitelist
- none
ViewerProtocolPolicy:
Description: The protocol that users can use to access the files in the origin that you specified in the TargetOriginId property when the default cache behavior is applied to a request.
Type: String
Default: "https-only"
AllowedValues:
- redirect-to-https
- allow-all
- https-only
PriceClass:
Description: The price class that corresponds with the maximum price that you want to pay for CloudFront service. If you specify PriceClass_All, CloudFront responds to requests for your objects from all CloudFront edge locations.
Type: String
Default: "PriceClass_100"
AllowedValues:
- PriceClass_All
- PriceClass_100
- PriceClass_200
SslSupportMethod:
Description: Specifies how CloudFront serves HTTPS requests.
Type: String
Default: "sni-only"
AllowedValues:
- sni-only
- vip
MinimumProtocolVersion:
Description: The minimum version of the SSL protocol that you want CloudFront to use for HTTPS connections.
Type: String
Default: "TLSv1.2_2021"
AllowedValues:
- TLSv1.2_2021
- TLSv1.2_2019
- TLSv1.1_2018
OriginKeepaliveTimeout:
Description: You can create a custom keep-alive timeout. All timeout units are in seconds. The default keep-alive timeout is 5 seconds, but you can configure custom timeout lengths. The minimum timeout length is 1 second; the maximum is 60 seconds.
Type: String
Default: "60"
OriginReadTimeout:
Description: You can create a custom origin read timeout. All timeout units are in seconds. The default origin read timeout is 30 seconds, but you can configure custom timeout lengths. The minimum timeout length is 4 seconds; the maximum is 60 seconds.
Type: String
Default: "30"
BucketVersioning:
Description: The versioning state of an Amazon S3 bucket. If you enable versioning, you must suspend versioning to disable it.
Type: String
Default: "Suspended"
AllowedValues:
- Enabled
- Suspended
Resources:
# Bucket Policy for primary and secondary buckets.
PrimaryBucketReadPolicy:
Type: 'AWS::S3::BucketPolicy'
Properties:
Bucket: !Sub 'ck-${Environment}-${AppName}-primary-bucket'
PolicyDocument:
Statement:
- Action:
- 's3:GetObject'
Effect: Allow
Resource: !Sub 'arn:aws:s3:::ck-${Environment}-${AppName}-primary-bucket/*'
Principal:
CanonicalUser: !GetAtt PrimaryBucketCloudFrontOriginAccessIdentity.S3CanonicalUserId
SecondaryBucketReadPolicy:
Type: 'AWS::S3::BucketPolicy'
Properties:
Bucket: !Sub 'ck-${Environment}-${AppName}-secondary-bucket'
PolicyDocument:
Statement:
- Action:
- 's3:GetObject'
Effect: Allow
Resource: !Sub 'arn:aws:s3:::ck-${Environment}-${AppName}-secondary-bucket/*'
Principal:
CanonicalUser: !GetAtt SecondaryBucketCloudFrontOriginAccessIdentity.S3CanonicalUserId
# Cloud Front OAI
PrimaryBucketCloudFrontOriginAccessIdentity:
Type: 'AWS::CloudFront::CloudFrontOriginAccessIdentity'
Properties:
CloudFrontOriginAccessIdentityConfig:
Comment: !Sub 'ck-${Environment}-${AppName}-primary'
SecondaryBucketCloudFrontOriginAccessIdentity:
Type: 'AWS::CloudFront::CloudFrontOriginAccessIdentity'
Properties:
CloudFrontOriginAccessIdentityConfig:
Comment: !Sub 'ck-${Environment}-${AppName}-secondary'
# Cloudfront Cache Policy
CDNCachePolicy:
Type: AWS::CloudFront::CachePolicy
Properties:
CachePolicyConfig:
Comment: 'Max TTL 600 to validate frequent changes'
DefaultTTL: !Ref DefaultTTL
MaxTTL: !Ref MaxTTL
MinTTL: !Ref MinTTL
Name: !Sub 'ck-${Environment}-${AppName}-cache-policy'
ParametersInCacheKeyAndForwardedToOrigin:
CookiesConfig:
CookieBehavior: none
EnableAcceptEncodingBrotli: True
EnableAcceptEncodingGzip: True
HeadersConfig:
HeaderBehavior: none
QueryStringsConfig:
QueryStringBehavior: none
# CLOUDFRONT DISTRIBUTION
CloudFrontDistribution:
Type: 'AWS::CloudFront::Distribution'
DependsOn:
- CDNCachePolicy
Properties:
DistributionConfig:
Comment: 'Cyberkeeda Password Generator application'
Enabled: true
HttpVersion: http2
IPV6Enabled: true
DefaultRootObject: version.json
Origins:
- DomainName: !Sub 'ck-${Environment}-${AppName}-primary.s3.amazonaws.com'
Id: !Sub 'ck-${Environment}-${AppName}-primary-origin'
OriginPath: "/v1/latest"
ConnectionAttempts: 1
ConnectionTimeout: 2
S3OriginConfig:
OriginAccessIdentity: !Sub 'origin-access-identity/cloudfront/${PrimaryBucketCloudFrontOriginAccessIdentity}'
- DomainName: !Sub 'ck-${Environment}-${AppName}-secondary.s3.amazonaws.com'
Id: !Sub 'ck-${Environment}-${AppName}-secondary-origin'
OriginPath: "/v1/latest"
ConnectionAttempts: 1
ConnectionTimeout: 2
S3OriginConfig:
OriginAccessIdentity: !Sub 'origin-access-identity/cloudfront/${SecondaryBucketCloudFrontOriginAccessIdentity}'
OriginGroups:
Quantity: 1
Items:
- Id: !Sub 'ck-${Environment}-${AppName}-cdn-origin-group'
FailoverCriteria:
StatusCodes:
Items:
- 500
- 502
- 503
- 504
- 403
- 404
Quantity: 6
Members:
Quantity: 2
Items:
- OriginId: !Sub 'ck-${Environment}-${AppName}-primary-origin'
- OriginId: !Sub 'ck-${Environment}-${AppName}-secondary-origin'
CacheBehaviors:
- CachePolicyId: !GetAtt 'CDNCachePolicy.Id'
PathPattern: '*'
ViewerProtocolPolicy: !Ref 'ViewerProtocolPolicy'
TargetOriginId: !Sub 'ck-${Environment}-${AppName}-cdn-origin-group'
DefaultCacheBehavior:
AllowedMethods:
- GET
- HEAD
TargetOriginId: !Sub 'ck-${Environment}-${AppName}-cdn-origin-group'
ViewerProtocolPolicy: !Ref 'ViewerProtocolPolicy'
CachePolicyId: !GetAtt 'CDNCachePolicy.Id'
Outputs:
CDNCloudfrontURL:
Description: CloudFront CDN Url.
Value: !GetAtt 'CloudFrontDistribution.DomainName'</b></span></span></pre></div></div><div><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><br /></b></span></span></div><div><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><br /></b></span></span></div><div>Once the above file and it's respective contents are dumped within source code repository, we can use to create AWS services using Jenkins pipeline job.</div></div><div><br /></div><div>If we breakdown the blog post, this post can be used for other techinacl refrences too, such as.</div><div><ul style="text-align: left;"><li>Jenkins Scripted pipeline using parameters.</li><li>How to hash/mask passwords and sensitive environments. </li><li>Leverage the power of docker to make codes uniform across environments and platform.</li><ul><li>If you notice, we can easily install ansible packages within build machine and run the ansible playbook directly, but we are not touching any third party application within our build machine.</li><li>Even once our task is done, we are removing the container.</li></ul><li>How to build docker image from docker file using jenkins.</li><li>Docker file to build ansible image.</li><li>Real world example of Ansible Roles.</li><li>Ansible to create S3 buckets with tags.</li><li>How to disable s3 bucket public access using ansible.</li><li>How to create s3 bucket directories and objects using Ansible.</li><li>How to use Ansible to create CloudFormation stack using parameters.</li><li>CloudFormation template to create below resources.</li><ul><li>S3 Bucket Policy</li><li>CloudFront Origin Access Identity.</li><li>CloudFront Cache Policy.</li><li>CloudFront Distribution with Origin Group and S3 as a Origin.</li></ul></ul></div><div><br /></div><div>Hope this blog post, help you in some use case.</div><div><br /></div><div>There might be definitely errors and areas of improvement within this blog post or better wat to handle such deployment, please share your valuable comments.</div><div><br /></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-91382736907392812822022-01-07T16:16:00.000+05:302022-01-07T16:16:02.900+05:30Cloudformation template to create S3 bucket with Tags and disable public access<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi5wOw0rf6dntNw8OcPSsP7GaI31QcAlZY4N2gOvWv5JRtH5pXuo7w0Hf7XG5oF8ibMGdVU_3k1LjgI4vTlmkyRoJRvWRVsa26NHeKNc4aJf3UkhijCkzc7FP3Eaov6H3wdnVcjN83DmfwSC1bHscCo4K0oysLhteERxFcMfdHVSf_YOnm0hQvewC4=s1336" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="715" data-original-width="1336" height="214" src="https://blogger.googleusercontent.com/img/a/AVvXsEi5wOw0rf6dntNw8OcPSsP7GaI31QcAlZY4N2gOvWv5JRtH5pXuo7w0Hf7XG5oF8ibMGdVU_3k1LjgI4vTlmkyRoJRvWRVsa26NHeKNc4aJf3UkhijCkzc7FP3Eaov6H3wdnVcjN83DmfwSC1bHscCo4K0oysLhteERxFcMfdHVSf_YOnm0hQvewC4=w400-h214" width="400" /></a></div><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></p><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Below CloudFormation template can be used for the following tasks.</span></span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Create S3 bucket.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Add tags to S3 bucket.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Disable public access.</span></span></li></ul></div><div style="background-color: #1e1e1e; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div style="color: #d4d4d4;"> <span style="color: #ce9178;">S3BUCKET</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">PrimaryBucket</span>:</div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">Type</span>: <span style="color: #ce9178;">AWS::S3::Bucket</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">Properties</span>: </div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">BucketName</span>: <span style="color: #569cd6;">!Sub</span> <span style="color: #ce9178;">'</span><span style="color: #ce9178;">cyberkeeda</span><span style="color: #ce9178;">-${Environment}-${AppName}-bucket'</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">AccessControl</span>: <span style="color: #ce9178;">Private</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">PublicAccessBlockConfiguration</span>:</div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">BlockPublicAcls</span>: <span style="color: #569cd6;">True</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">BlockPublicPolicy</span>: <span style="color: #569cd6;">True</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">IgnorePublicAcls</span>: <span style="color: #569cd6;">True</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">RestrictPublicBuckets</span>: <span style="color: #569cd6;">True</span> </div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">Tags</span>:</div><div style="color: #d4d4d4;"> - <span style="color: #569cd6;">Key</span>: <span style="color: #ce9178;">Name</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">Value</span>: <span style="color: #569cd6;">!Sub</span> <span style="color: #ce9178;">'cyberkeeda-${Environment}-${AppName}'</span></div><div style="color: #d4d4d4;"> - <span style="color: #569cd6;">Key</span>: <span style="color: #ce9178;">Environment</span></div><div><span style="color: #d4d4d4;"> <span style="color: #569cd6;">Value</span>: "</span><span style="color: #d4d4d4;">Development"</span></div><div style="color: #d4d4d4;"> - <span style="color: #569cd6;">Key</span>: <span style="color: #ce9178;">Creator</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">Value</span>: <span style="color: #569cd6;">!Sub</span> <span style="color: #ce9178;">"${Creator}"</span></div><div style="color: #d4d4d4;"> - <span style="color: #569cd6;">Key</span>: <span style="color: #ce9178;">Appname</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">Value</span>: <span style="color: #569cd6;">!Sub</span> <span style="color: #ce9178;">"${</span><span style="color: #ce9178;">Appname</span><span style="color: #ce9178;">}"</span></div><div><span style="color: #d4d4d4;"> - </span><span style="color: #569cd6;">Key</span><span style="color: #d4d4d4;">: </span><span style="color: #ce9178;">Unit</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">Value</span>: <span style="color: #569cd6;">!Sub</span> <span style="color: #ce9178;">"${</span><span style="color: #ce9178;">Unit</span><span style="color: #ce9178;">}"</span></div><div style="color: #d4d4d4;"> - <span style="color: #569cd6;">Key</span>: <span style="color: #ce9178;">Owner</span></div><div style="color: #d4d4d4;"> <span style="color: #569cd6;">Value</span>: <span style="color: #ce9178;">admin@ck.com</span></div><div style="color: #d4d4d4;"><br /></div></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-47457649851556267182022-01-06T20:17:00.007+05:302022-01-07T12:29:56.073+05:30Ansible role to create S3 bucket and directories<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi9XPUn2DxgnQ2DJKkn66KZEDAxHk98DSUohvk_ckJJUJUVhHlblRAFbaBoQa3VN2GUhIeBPT6xIWx_YDuJivPAimqQNSkH9BnR8uhIC-Kym1HApaqQ_tHW4O6-Z90upV48x7iGBsbVgiqvG_HABcq6YbhUhh216AnzI8ajykAG5wmwsxUpaIcPmzo=s427" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="427" data-original-width="420" height="320" src="https://blogger.googleusercontent.com/img/a/AVvXsEi9XPUn2DxgnQ2DJKkn66KZEDAxHk98DSUohvk_ckJJUJUVhHlblRAFbaBoQa3VN2GUhIeBPT6xIWx_YDuJivPAimqQNSkH9BnR8uhIC-Kym1HApaqQ_tHW4O6-Z90upV48x7iGBsbVgiqvG_HABcq6YbhUhh216AnzI8ajykAG5wmwsxUpaIcPmzo=s320" width="315" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><p><span style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"> Ansible roles can be defined as</span></p><p></p><ul style="text-align: left;"><li><span style="background-color: white;"><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Collection of multiple playbooks within directories for several tasks and operations.</span></span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white;"><span style="font-size: 14px; letter-spacing: 1px;">It's a way of maintaining playbooks in a structured and identical manner.</span></span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white;"><span style="font-size: 14px; letter-spacing: 1px;">It's a way of breaking lengthy playbooks into small plays.</span></span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="background-color: white; font-size: 14px; letter-spacing: 1px;">Roles can be uploaded to Ansible galaxy, which can be reused as an ansible library or module.</span></span></li></ul><div><br /></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">How can we create Ansible roles.</span></span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">We can use ansible-galaxy command to download existing roles uploaded on website https://galaxy.ansible.com/</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">We can use ansible-galaxy command to create new role.</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">While creating a new role ansible-galaxy creates roles in the default directory as /etc/ansible/roles followed by name of the role.</span></span></li></ul><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Below commands can be used as per need.</span></span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Ansible galaxy command to check installed roles.</span></span></li></ul></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ ansible-galaxy collection list</b></span></span><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">
</b></pre></div><div><div><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Ansible galaxy command to create role in default directory</span></span></li></ul></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ ansible-galaxy init /etc/ansible/roles/my-role --offline</b></span></span><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">
</b></pre></div></div></div><div><div><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Ansible galaxy command to create role in present working directory</span></span></li></ul></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ ansible-galaxy init my-role</b></span></span><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">
</b></pre></div></div><div><div><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Ansible galaxy command to install roles from ansible galaxy website collections</span></span></li></ul></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ ansible-galaxy collection install amazon.aws</b></span></span></pre></div></div><div><div style="text-align: left;"><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div style="text-align: left;"><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Ansible role directory structure looks like below, we can take example of above created ansible role by name <b>my-role</b></span></div><div style="text-align: left;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ ansible-galaxy init my-role
- Role my-role was created successfully</b></span></span></pre><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ tree my-role/
my-role/
├── defaults
│ └── main.yml
├── files
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── README.md
├── tasks
│ └── main.yml
├── templates
├── tests
│ ├── inventory
│ └── test.yml
└── vars
└── main.yml
8 directories, 8 files</b></span></span></pre></div></div><div><br /></div><div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">How can we use roles</span></span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Till now, we have seen how to create role and once created what's it's default directory structure looks like, Ansible roles can be used in there ways</span></span></li><ul><li>with the roles option: This is the classic way of using roles in a play.</li><li>tasks level with include_role: one can reuse roles dynamically anywhere in the tasks section of a play using include_role.</li><li>tasks level with import_role: You can reuse roles statically anywhere in the tasks section of a play using import_role.</li></ul></ul><div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Here we will know more about the classic way of using roles, that is by using the roles option in playbook.</span></div></div></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">So instead of going through all conventional method of installing apache or ngnix, I will share a real-time custom role, that has the following task/operations to do.</span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Create multiple AWS S3 buckets by regions.</span></li><li><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Create directory structure within two of above created bucket.</span></li></ul><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">First let's go through the playbook, that can be independently used to do the entire operation without creating ansible roles.</span></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Note: </span><span style="color: red; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">amazon.aws galaxy collection must be update to recent version, in order to use option s3_bucket </span></div><div><span style="color: #2b00fe;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">$ </b><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>ansible-galaxy collection install amazon.aws</b></span></pre></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">---
- hosts: localhost
connection: local
gather_facts: False
tasks:</span>
<span style="color: #2b00fe;"> - name: Read environment specific variables.
include_vars:
file: "ansible_role/vars/{{ Environment }}/main.yml"</span>
- name: Create static-ck application buckets in us-east-1 region.
s3_bucket:
name: "{{ item }}"
state: present
tags:
Name: "{{ item }}"
Environment: "{{ Environment }}"
Owner: "{{ bucketTags[Environment]['Owner'] }}"
region: us-east-1
public_access:
block_public_acls: true
ignore_public_acls: true
block_public_policy: true
restrict_public_buckets: true
with_items:
- "{{ bucketCfg[Environment]['PrimarBucketName'] }}"
- "{{ bucketCfg[Environment]['DevopsBucketName'] }}"
- "{{ bucketCfg[Environment]['CDNLogBucketName'] }}"<span style="color: #2b00fe;">
- name: Create static-ck application buckets in us-east-2 region.
s3_bucket:
name: "{{ item }}"
state: present
tags:
Name: "{{ item }}"
Environment: "{{ Environment }}"
Owner: "{{ bucketTags[Environment]['Owner'] }}"
region: us-east-2
public_access:
block_public_acls: true
ignore_public_acls: true
block_public_policy: true
restrict_public_buckets: true
with_items:
- "{{ bucketCfg[Environment]['SecondaryBucketName'] }}"</span>
<span style="color: #7f6000;"> - name: Create empty directories to store build artifacts.
aws_s3:
bucket: "{{ item.bucket_name }}"
object: "{{ item.artifact_dir }}"
mode: create
with_items:
- { bucket_name: "{{ bucketCfg[Environment]['PrimarBucketName'] }}", artifact_dir: "/app1/artifcats" }
- { bucket_name: "{{ bucketCfg[Environment]['SecondaryBucketName'] }}", artifact_dir: "/app1/artifcats" }</span>
<span style="color: #e69138;"> - name: Create empty directories to deploy latest build.
aws_s3:
bucket: "{{ item.bucket_name }}"
object: "{{ item.latest_dir }}"
mode: create
with_items:
- { bucket_name: "{{ bucketCfg[Environment]['PrimarBucketName'] }}", latest_dir: "/app1/latest" }
- { bucket_name: "{{ bucketCfg[Environment]['SecondaryBucketName'] }}", latest_dir: "/app1/latest" }</span></b></span></span></pre></div></div></div><div><br /></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Above playbook can be triggered independently using the below command as.</span></span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ ansible-playbook -vv --extra-vars "Environment=int" main.yml</b></span></span></pre></div><div><br /></div><div><br /></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">The same deployment can be done using ansible roles in below manner following the below steps.</span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Create a new ansible role by name <b>ansible_role</b></span></li></ul></div><div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ ansible-galaxy init ansible_role</b></span></span></pre></div></div><div><div><ul><li><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Create a new root/entry playbook to initiate deployment</span></li></ul></div><div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ touch root.yml</b></span></span></pre></div><div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Include the below lines and indicate the use role option to call our role, please note that we have used the option "<b>roles" </b>to call our newly created role directory by name <b>ansible_role, </b>while using roles option please make a note of the below points for main.yml file.</span></span></li><ul><li>When you use the roles option at the play level, for each role ‘x’:</li><li>If roles/x/tasks/main.yml exists, Ansible adds the tasks in that file to the play.</li><li>If roles/x/handlers/main.yml exists, Ansible adds the handlers in that file to the play.</li><li>If roles/x/vars/main.yml exists, Ansible adds the variables in that file to the play.</li><li>If roles/x/defaults/main.yml exists, Ansible adds the variables in that file to the play.</li><li>If roles/x/meta/main.yml exists, Ansible adds any role dependencies in that file to the list of roles.</li><li>Any copy, script, template or include tasks (in the role) can reference files in roles/x/{files,templates,tasks}/ (dir depends on task) without having to path them relatively or absolutely.</li></ul></ul></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">---
- hosts: localhost
connection: local
gather_facts: False
</span><span style="color: red;">roles</span><span style="color: #2b00fe;">:
- ansible_role </span></b></span></span></pre></div></div></div></div><div><ul><li><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Below the directory structure we follow within our newly created role.</span></li></ul><div><div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>root.yml
|
ansible_role/
├── defaults
│ └── main.yml
├── files
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── README.md
├── tasks
│ ├── create_bucket_directories.yml
│ ├── create_s3_bucket.yml
│ └── main.yml
├── templates
├── tests
│ ├── inventory
│ └── test.yml
└── vars
└── int
└── main.yml</b></span></span></pre></div></div></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">So from above directory layout, we have the below files and directories to create.</span><div><div></div><div></div></div></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">We have divided our tasks into parts</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Create S3 buckets</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Create directories within S3</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">All the above two tasks will be defined individually under two different file by name</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">create_s3_bucket.yml</span></span></li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">create_bucket_directories.yml</span></span></li></ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Where as ansible_roles/tasks/main.yml is entry point for these two task, which we will be importing using option <b>import tasks</b></span></span></li></ul></ul><div style="font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><div style="font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ cat </b></span></span><b style="background-color: transparent; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">ansible_role/tasks/main.yml</b></pre></div><div style="color: black; font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;"><div></div></div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>---
- import_tasks: create_s3_bucket.yml
- import_tasks: create_bucket_directories.yml</b></span></span></pre></div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div>This is how, my other two task files look like.</span></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></div><div><div style="font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><div style="font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ cat </b></span></span><b style="background-color: transparent; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">ansible_role/tasks/</b><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">create_s3_bucket.yml</b></pre></div><div style="color: black; font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;"><div></div></div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>---
- name: Read environment specific variables.
include_vars:
file: "ansible_role/vars/{{ Environment }}/main.yml"
- name: Create static-ck application buckets in us-east-1 region.
s3_bucket:
name: "{{ item }}"
state: present
tags:
Name: "{{ item }}"
Environment: "{{ Environment }}"
Owner: "{{ bucketTags[Environment]['Owner'] }}"
region: us-east-1
public_access:
block_public_acls: true
ignore_public_acls: true
block_public_policy: true
restrict_public_buckets: true
with_items:
- "{{ bucketCfg[Environment]['PrimarBucketName'] }}"
- "{{ bucketCfg[Environment]['DevopsBucketName'] }}"
- "{{ bucketCfg[Environment]['CDNLogBucketName'] }}"
- name: Create static-ck application buckets in us-east-2 region.
s3_bucket:
name: "{{ item }}"
state: present
tags:
Name: "{{ item }}"
Environment: "{{ Environment }}"
Owner: "{{ bucketTags[Environment]['Owner'] }}"
region: us-east-2
public_access:
block_public_acls: true
ignore_public_acls: true
block_public_policy: true
restrict_public_buckets: true
with_items:
- "{{ bucketCfg[Environment]['SecondaryBucketName'] }}"</b></span></span>
</pre></div></div><div><br /></div><div><div style="font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><div style="font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ cat </b></span></span><b style="background-color: transparent; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">ansible_role/tasks/</b><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">create_bucket_directories</b><b style="font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">.yml</b></pre></div><div style="color: black; font-family: "Times New Roman"; font-size: medium; letter-spacing: normal;"><div></div></div></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>---
- name: Read environment specific variables.
include_vars:
file: "ansible_role/vars/{{ Environment }}/main.yml"
- name: Create empty directories to store build artifacts.
aws_s3:
bucket: "{{ item.bucket_name }}"
object: "{{ item.artifact_dir }}"
mode: create
with_items:
- { bucket_name: "{{ bucketCfg[Environment]['PrimarBucketName'] }}", artifact_dir: "/v1/artifcats" }
- { bucket_name: "{{ bucketCfg[Environment]['SecondaryBucketName'] }}", artifact_dir: "/v1/artifcats" }</b></span><b style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;">
</b></span></pre></div><div><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><br /></b></span></span></div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">We have added an additional directory as "<b>int</b>", which is the short form of Internal environment, following the same we can create more directories that can relate to other environmental specific files for <b>prod </b>and <b>non-prod </b>environmet too.</span></span></li><ul><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Within file <b>ansible_role/vars/int/main.yml</b> we defined key value pairs that can be used later while running our playbook</span></span></li></ul></ul></div><div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ cat </b></span></span><b style="background-color: transparent; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">ansible_role/vars/int/main.yml</b></pre></div></div><div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>---
# default variables
region: us-east-1
ProductName: ck
ProjectName: static-app
Environment: int
PrimaryRegion: us-east-1
SecondaryRegion: us-east-2
regions:
us-east-1:
preferredMaintenanceWindow: "sat:06:00-sat:06:30"
us-east-2:
preferredMaintenanceWindow: "sat:05:00-sat:05:30"
bucketCfg:
int:
Environment: "{{ Environment }}"
PrimarBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-primary-cyberkeeda-bucket-01"
SecondaryBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-secondary-cyberkeeda-bucket-01"
CDNLogBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-cdn-logs-cyberkeeda-bucket-01"
DevopsBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-devops-cyberkeeda-bucket-01"
PrimaryBucketRegion: "{{ PrimaryRegion }}"
SecondaryBucketRegion: "{{SecondaryRegion}}"
DevopsBucketRegion: "{{ PrimaryRegion }}"
bucketTags:
int:
PrimaryBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-primary"
SecondaryBucketName: "{{ ProductName }}-{{Environment}}-{{ ProjectName }}-secondary"
Environment: "{{ Environment }}"
CreatorID: "admin@cyberkeeda.com"
Owner: "admin@cyberkeeda.com"</b></span></span></pre></div></div><div><br /></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Once the above templates are created and save, we can run our playbook with below ansible-playbook command.</span></span></div><div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: #2b00fe; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>$ ansible-playbook -vv --extra-vars "Environment=int" root.yml</b></span></span></pre></div><div><div></div></div></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Below is the details for the above paramter used along with the ansible-playbook command.</span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">-vv : Verbrose mode for debugging in STDOUT</span></li><li><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">--extra-vars : Key-Value pair to be used within playbook</span></li></ul></div><div><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></span></div><div><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;">Hope this blog post will help any one in some sort, please comment in case if you have any difficulties following steps.</span></span></div><div><br /></div><p></p>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-10882154745270006102021-12-30T14:36:00.005+05:302022-01-07T16:06:32.365+05:30Ansible script for AWS S3 Bucket automation.<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgp1tMWsQAum8baYOoh_T0NWW5XqSNJYUpPcgXNXEom1gSDSy75a4q-BTg3DMQRYuVr12ebNiWoZwvnytij3-ujB8XxocMzYXhseZ1PfIypjmY_UAuGEkpcPMQlBC7fKJj3vBni0gHmHq0Y9if_qZ_5rby2s_SVdPOXyNmeKaAZYyczbVsiXP3xDZE=s512" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="512" data-original-width="512" height="200" src="https://blogger.googleusercontent.com/img/a/AVvXsEgp1tMWsQAum8baYOoh_T0NWW5XqSNJYUpPcgXNXEom1gSDSy75a4q-BTg3DMQRYuVr12ebNiWoZwvnytij3-ujB8XxocMzYXhseZ1PfIypjmY_UAuGEkpcPMQlBC7fKJj3vBni0gHmHq0Y9if_qZ_5rby2s_SVdPOXyNmeKaAZYyczbVsiXP3xDZE=w200-h200" width="200" /></a></div><p></p><h3 style="background-color: white; margin: 0px; position: relative;"><div style="color: #333333; font-family: lora, serif; letter-spacing: 1px;"><span style="font-size: small;">Ansible Script/Tasks/Playbook for AWS S3 Operations.</span></div><div style="color: #333333; font-family: lora, serif; letter-spacing: 1px;"><span style="font-size: x-small;"><br /></span></div><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><span style="font-size: 14px; font-weight: 400;">Within this post, we have a ansible playbook/script that can be used for below tasks.</span></div><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><ol style="text-align: left;"><li><span style="font-size: 14px; font-weight: 400;">Create empty s3 buckets, we will use loop to create two empty buckets.</span></li><ol><li><span style="font-size: 14px; font-weight: 400;">Bucket in specific region.</span></li><li><span style="font-size: 14px; font-weight: 400;">Bucket with specific tags</span></li><li><span style="font-size: 14px; font-weight: 400;">Disable bucket public access</span></li></ol><li><span style="font-size: 14px; font-weight: 400;">Create empty directories within bucket.</span></li></ol></div><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><span style="font-size: 14px; font-weight: 400;"><div>Note: <span style="color: red;">amazon.aws galaxy collection must be updated to recent version, in order to use option s3_bucket's updated functions (like. public_access )</span></div><div><span style="color: #2b00fe;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><b style="font-family: monaco, menlo, consolas, "courier new", monospace; white-space: pre-wrap;">$ </b><span style="white-space: pre-wrap;"><b>ansible-galaxy collection install amazon.aws</b></span></pre></span></div></span></div><div><div style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 16.25px;"><span style="font-size: 14px; letter-spacing: 1px;"><span style="color: #c65d09;">---
- hosts: localhost
connection: local
gather_facts: False
tasks:
</span>- name: Create empty buckets
s3_bucket:
name: "{{ item }}"
state: present
tags:
Name: </span>"{{ item }}"<span style="font-size: 14px; letter-spacing: 1px;">
Owner: admin
region: us-west-1
public_access:
block_public_acls: true
ignore_public_acls: true
block_public_policy: true
restrict_public_buckets: true
with_items:
- cyberkeeda-ansible-bucket1
- cyberkeeda-ansible-bucket2<span style="color: #c65d09;">
- name: Create empty directories to store artifacts.
aws_s3:
bucket: "{{ item.bucket_name }}"
object: "{{ item.artifact_dir }}"
mode: delobj
with_items:
- { bucket_name: cyberkeeda-ansible-bucket1, artifact_dir: "/v1/artifcats" }
- { bucket_name: cyberkeeda-ansible-bucket2, artifact_dir: "/v1/artifcats" }
</span></span></pre></div></div><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><span style="font-size: 14px; font-weight: 400;"><br /></span></div><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><span style="font-size: 14px; font-weight: 400;"><br /></span></div><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><span style="font-size: 14px; font-weight: 400;"><br /></span></div></h3><div class="separator" style="clear: both; text-align: center;"><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; text-align: start;">Will add more to this thread for more s3 specific opeation.</p></div><p><br /></p>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-63462377615365245112021-08-03T21:19:00.001+05:302021-08-03T21:19:26.243+05:30Python Script to create new JIRA tickets using JIRA API<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimgnZ6YMuf-oX-YLfNQv6HPyK_bJW5dTXqbTcy7Xt1YoaAD0x-2O3fNOn0_Vqm0M2FY87lXk7mYMqNUeEu1H9lxpNU5aCVtDd24RmHyVvOWjm2PnDYEolYno-Ds9_NP7neBwrqjMng_UY/s601/python-logo-master-v3-TM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="203" data-original-width="601" height="135" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimgnZ6YMuf-oX-YLfNQv6HPyK_bJW5dTXqbTcy7Xt1YoaAD0x-2O3fNOn0_Vqm0M2FY87lXk7mYMqNUeEu1H9lxpNU5aCVtDd24RmHyVvOWjm2PnDYEolYno-Ds9_NP7neBwrqjMng_UY/w400-h135/python-logo-master-v3-TM.png" width="400" /></a></div><p></p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><span face="verdana, sans-serif"><br /></span></p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Python Script with below JIRA operations</p><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"></p><ul style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; line-height: 1.4; margin: 0.5em 0px; padding: 0px 2.5em;"><li style="margin: 0px 0px 0.25em; padding: 0px;"><span face="verdana, sans-serif">Fetch JIRA ticket details using JIRA API.</span></li><li style="margin: 0px 0px 0.25em; padding: 0px;"><span face="verdana, sans-serif">Create JIRA ticket using JIRA API</span></li></ul><p style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"></p><div style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></div><div style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Information to collect before using script.</div><div style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><ul style="text-align: left;"><li>Get your organization JIRA URL handy with you.</li><ul><li>Can be retrieved from any JIRA ticket URL, respective URL has FQDN that's the JIRA URL dedicated for your organization.</li></ul><li>Know your JIRA project/space name.</li><ul><li>Login to any of your JIRA tickets.</li><li>From top navigation panel, select projects and check the name associated with the project, project name will be single word without any space.</li></ul><li>Know JIRA field/mandatory fields within your ticket, before you create a ticket via API.</li><ul><li>Will know how to fetch details about it from our python script, <b>get_jita_details </b>method can be used to get details for field</li></ul></ul></div><div style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></div><div style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Script has one class and two methods, will know how and when to use one by one.</div><div style="background-color: white;"><ul style="text-align: left;"><li style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>JiraHandler </b>( Class )</li><li style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><b>get_jira_details</b> ( Method ) - Can be used to fetch JIRA ticket details.</li><li><span style="color: #333333; font-family: Lora, serif;"><span style="font-size: 14px; letter-spacing: 1px;"><b>create_jira_cr_ticket</b></span></span><span style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">( Method ) - Can be used to create new JIRA ticket.</span> </li></ul><div>Note : </div><div><ul style="text-align: left;"><li>For simplicity, i have used basic authentication method to authenticate to JIRA servers, although for some sort of security instead of using plain text password, have encoded it using base64 authentication.</li><li>You need to ready with the Payload data json file, before you go ahead and create a new JIRA ticket, this file can be of any name but please content is in JSON format.</li></ul><span style="background-color: whitesmoke; font-family: monaco, menlo, consolas, "courier new", monospace; font-size: 14px; font-weight: 700; letter-spacing: 1px; white-space: pre-wrap;"><span> </span><span> </span>payload_data.json</span></div><div> </div></div><div style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><a href="https://github.com/Jackuna/PythonXample/blob/master/jira_api_handler.py" target="_blank">Github script link</a>:</div><div style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></div><h4 style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; margin: 0px; position: relative;">Script.</h4><h4 style="background-color: white; margin: 0px; position: relative;"><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, "courier new", monospace;"><span><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><span style="color: #2b00fe;">import requests
import json
import base64
from requests.auth import HTTPBasicAuth
# Inorder to encrypt/decrypt your credentials using base64 module as below.
# To encode --> base64.b64encode(bytes("random", "utf-8"))
# To decode --> base64.b64decode("cmFuZG9t").decode("utf-8")</span>
class JiraHandler:
def __init__(self, username):
print('Loading Instnace variables...')
self.username = username
self.securestring = base64.b64decode("replaceItwithYourCredential").decode("utf-8")
self.url = "https://jira.yourownjiradomain.com/rest/api/2/issue/"
def get_jira_details(self,jira_ticket):
try :
auth = HTTPBasicAuth(self.username, self.securestring)
get_details_url = self.url + jira_ticket
headers = {
"Accept": "application/json"
}
print("retrieveing", jira_ticket, "details...")
response = requests.request(
"GET",
get_details_url,
headers=headers,
auth=auth
)
print(json.dumps(json.loads(response.text), sort_keys=True, indent=4, separators=(",", ": ")))
except Exception as e:
return e
def create_jira_cr_ticket(self, filename):
get_details_url = self.url
auth = HTTPBasicAuth(self.username, self.securestring)
headers = {
"Accept": "application/json",
"Content-Type": "application/json"
}
try:
with open(filename, "r") as re_read_file:
payload = re_read_file.read()
print("Creating new JIRA...")
response = requests.request(
"POST",
get_details_url,
data=payload,
headers=headers,
auth=auth
)
print(json.dumps(json.loads(response.text), sort_keys=True, indent=4, separators=(",", ": ")))
except Exception as filenotfound:
print("Can't load file..", filenotfound)
<span style="color: #2b00fe;">
</span></span><b style="color: black; font-family: "Times New Roman"; font-size: 14px; font-weight: 400; letter-spacing: 1px; white-space: pre-wrap;">
</b></span></span></pre></div></h4><div><br /></div><div>How to use it, </div><div><ul style="text-align: left;"><li>Load Instance variable by calling class and provide your JIRA username as an input.</li></ul></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; font-weight: 700; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; text-align: left; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, "courier new", monospace;"><span><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"># <span style="color: #2b00fe;">Initiate Class and load instance variable</span>.
d = JiraHandler("your_JIRA_Username")</span></span></span></pre></div><div><span style="font-family: monaco, menlo, consolas, "courier new", monospace;"><span><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><div style="font-family: "Times New Roman"; font-size: medium; letter-spacing: normal; white-space: normal;"><ul><li>Call method <b>get_jira_details </b>and add JIRA ticket as method input, for example if we want to get details about a specific JIRA ticket CKPROJ-6162, we must call the method as described below.</li></ul></div><div style="font-family: "Times New Roman"; font-size: medium; letter-spacing: normal; white-space: normal;"></div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; font-weight: 700; letter-spacing: normal; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, "courier new", monospace;"><span><span style="letter-spacing: 1px; white-space: pre-wrap;"># <span style="color: #2b00fe;">Call below method to get JIRA ticket details.</span>
d.get_jira_details("<span style="color: red;">CKPROJ-6162</span>")</span></span></span></pre></span></span></span></div><div><ul style="text-align: left;"><li>Before calling method <b>create_jira_cr_ticket, </b>dump json content within a json file, for instance i have created a file named payload_data.json and it looks somehow like below.</li><ul><li>Thinking how to find it, use method get_jira_details, it will give you an idea for the field used within your project's jira ticket.</li></ul></ul><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>{
"fields": {
"project":
{
"key": "CKPROJ"
},
"summary": "My Dummy Ticket created by REST API",
"description": "Dummy CR for REST API Test",
"confluence": "Dummy Confluence Page Link",
"verification_steps": "Verification Plan",
"issuetype": {
"name": "Change Request"
}
}
}</b></span></span></pre></div></div><div>Once you are ready with the payload data, save it within a JSON file and call method providing payload_data.json file as an input to it, as an output script will return the JIRA ticket details.</div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; font-size: 14px; font-weight: 700; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, "courier new", monospace;"><span><span style="letter-spacing: 1px; white-space: pre-wrap;"># <span style="color: #2b00fe;">Call below method to create new JIRA ticket</span>
d.create_jira_cr_ticket("C:\\Users\\cyberkeeda\\payload_data.json")</span></span></span></pre></div><div><br /></div><div>Hope this script can have help in any sort, for any help comment please.</div><div><br /></div><br />Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com7tag:blogger.com,1999:blog-1692034155797331883.post-86111937060151984042021-07-24T14:17:00.008+05:302021-07-24T14:17:50.818+05:30AWS Cloudformation template to create Cloudwatch Event rule to trigger ECS Task<p> <span> </span><span> </span><span> </span><span> </span><span> </span><span> </span><span> </span><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjovEfAFALfzQD8Kp7MKbJ8ndhvLeCQz84zkm5imXsRzYXZSfwShaHOjzUPzHlFbB00vhjjaQNcRXi-2CLEtnXDnIRzUORvvSbheuxlmhXQQzur3lNyAG9Wz5H686-Ky5A5LTIs60Isq10/s1333/cfn-cloudwatch-ecs.png" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="713" data-original-width="1333" height="214" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjovEfAFALfzQD8Kp7MKbJ8ndhvLeCQz84zkm5imXsRzYXZSfwShaHOjzUPzHlFbB00vhjjaQNcRXi-2CLEtnXDnIRzUORvvSbheuxlmhXQQzur3lNyAG9Wz5H686-Ky5A5LTIs60Isq10/w400-h214/cfn-cloudwatch-ecs.png" width="400" /></a></p><br /><p></p><p><span face="verdana, sans-serif" style="color: #333333; font-size: 14px; letter-spacing: 1px;">Cloudformation Template that will created below resources.</span></p><p></p><ul><li><span face="verdana, sans-serif" style="color: #333333;"><span style="font-size: 14px; letter-spacing: 1px;">IAM role for ECS Task and Cloudwatch rule.</span></span></li><li><span face="verdana, sans-serif" style="color: #333333;"><span style="font-size: 14px; letter-spacing: 1px;">CloudWatch schedule rule ( cron ) to trigger task defination.</span></span></li></ul><p></p><div style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></div><h4 style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; margin: 0px; position: relative;">Template</h4><h4 style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; margin: 0px; position: relative;"><div style="color: black; font-family: "Times New Roman"; font-size: medium; font-weight: 400; letter-spacing: normal;"><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b><span style="color: #2b00fe;">AWSTemplateFormatVersion</span>: 2010-09-09
<span style="color: #2b00fe;">Description</span>: |
1. IAM Role to be used by ECS task and cloudwatch event rule.
2. CloudWatch Rule to trigger ecs tasks.
<span style="color: #2b00fe;">Parameters</span>:
ProductName:
Description: Parent Product name.
Type: String
Default: cyberkeeda
ProjectName:
Description: Project Name
Type: String
Default: cyberkeeda-report
Environment:
Description: The equivalent CN name of the environment being worked on
Type: String
AllowedValues:
- dev
- uat
- qa
Region:
Description: Ck Region specific parameter
Type: String
AllowedValues:
- mum
- hyd
ECSClusterARN:
Description: ECS Cluster ARN to schedule Task
Type: String
Default: None
CWEventRuleCron:
Description: Cron Expression to schedule ECS task.
Type: String
Default: "cron(0 9 * * ? *)"
ECSTaskDefARN:
Description: ARN for ECS Task defination
Type: String
<span style="color: #2b00fe;">Metadata</span>:
AWS::CloudFormation::Interface:
ParameterGroups:
-
Label:
default: Project based details
Parameters:
- ProductName
- ProjectName
- Environment
- Region
-
Label:
default: ECS details.
Parameters:
- ECSClusterARN
- ECSTaskDefARN
- CWEventRuleCron
<span style="color: #2b00fe;">Resources</span>:
ExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-role"
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ 'ecs-tasks.amazonaws.com', 'events.amazonaws.com' ]
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
Policies:
- PolicyName: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-role-inlinePolicy"
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- ecs:RunTask
Resource:
- !Sub "${ECSTaskDefARN}:*"
- Effect: Allow
Action: iam:PassRole
Resource:
- "*"
Condition:
StringLike:
iam:PassedToService: ecs-tasks.amazonaws.com
<span style="color: #2b00fe;">TaskSchedule</span>:
Type: AWS::Events::Rule
Properties:
Description: Trigger Cyberkeeda Daily ECS task
Name: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-daily-event-rule"
ScheduleExpression: !Ref CWEventRuleCron
State: ENABLED
Targets:
- Id: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-daily-event-rule-targetId"
EcsParameters:
LaunchType: EC2
TaskDefinitionArn: !Ref TaskDefinition
TaskCount: 1
RoleArn:
Fn::GetAtt:
- ExecutionRole
- Arn
Arn: !Ref ECSClusterARN</b></span></span></pre></div><div style="font-weight: 400;"><br /></div><div style="font-weight: 400;">Let me know, for any questions in comment box.</div></h4>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-66044612365791007722021-07-24T13:56:00.006+05:302021-07-24T14:15:44.031+05:30AWS Cloudformation template to create ECS Task definition.<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhs3wSK2lG0oYrnjBOr_OWxwFuadA15O0cUYbHELjFJrwmlv6MLoGSk8hWCFeZGeE9fjDDHDN939IN28TPzVl7lXH4GZ2DC1JzpD33_qNvFkqOz-FgDhrRlA3n-g8LNVxqcfSFojPHommc/s1287/ecs-task-defination.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="725" data-original-width="1287" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhs3wSK2lG0oYrnjBOr_OWxwFuadA15O0cUYbHELjFJrwmlv6MLoGSk8hWCFeZGeE9fjDDHDN939IN28TPzVl7lXH4GZ2DC1JzpD33_qNvFkqOz-FgDhrRlA3n-g8LNVxqcfSFojPHommc/w400-h225/ecs-task-defination.png" width="400" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><br /></div><p></p><p><span face="verdana, sans-serif" style="color: #333333; font-size: 14px; letter-spacing: 1px;">Cloudformation Template that will created below resources.</span></p><p></p><ul style="text-align: left;"><li><span face="verdana, sans-serif" style="color: #333333;"><span style="font-size: 14px; letter-spacing: 1px;">IAM role for ECS Task execution</span></span></li><li><span style="color: #333333;"><span style="font-size: 14px; letter-spacing: 1px;">ECS Task definition</span></span></li></ul><p></p><div style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></div><h4 style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; margin: 0px; position: relative;">Template</h4><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>AWSTemplateFormatVersion: 2010-09-09
Description: |
ECS Task is responsible to fetch files from sftp location.
1. IAM Role to be used by ECS task and cloudwatch event rule.
2. ECS Task defination with container env variables, please note credential needs to be created first within parameter store.
Parameters:
ProductName:
Description: Parent Product name.
Type: String
Default: cyberkeeda
ProjectName:
Description: Project Name
Type: String
Default: cyberkeeda-report
Environment:
Description: The equivalent CN name of the environment being worked on
Type: String
AllowedValues:
- dev
- uat
- qa
Region:
Description: Ck Region specific parameter
Type: String
AllowedValues:
- mum
- hyd
ECSTaskDefARN:
Description: ARN for ECS Task defination
Type: String
SFTPHostFQDN:
Description: Remote SFTP Host FQDN.
Type: String
Default: 123.111.11.1
SFTPHostPort:
Description: Remote SFTP Host Port.
Type: String
Default: 22
SFTPUserName:
Description: Remote SFTP Host username.
Type: String
Default: sftpadmin
SFTPPasswordParameterStoreName:
Description: Remote SFTP Host Parameter store name.
Type: String
Default: sftppass
ContainerImageUrlwithTag:
Description: Container Image URL with tag.
Type: String
Default: docker.io/jackuna/sftpnew
ECSClusterARN:
Description: ECS Cluster ARN to schedule Task
Type: String
Default: arn:aws:ecs:ap-south-1:895678824142:cluster/sftp
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
-
Label:
default: CK Project Details
Parameters:
- ProductName
- ProjectName
- Environment
- Region
-
Label:
default: Remote SFTP Server details used as Container Environment Variables.
Parameters:
- SFTPHostFQDN
- SFTPHostPort
- SFTPUserName
- SFTPPasswordParameterStoreName
Resources:
ExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-role"
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ 'ecs-tasks.amazonaws.com', 'events.amazonaws.com' ]
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
Policies:
- PolicyName: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-role-inlinePolicy"
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- ssm:GetParameters
Resource:
- !Sub "arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/${Environment}.sftp-password"
- Effect: Allow
Action:
- ecs:RunTask
Resource:
- !Sub "${ECSTaskDefARN}:*"
- Effect: Allow
Action: iam:PassRole
Resource:
- "*"
Condition:
StringLike:
iam:PassedToService: ecs-tasks.amazonaws.com
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-ecs-task"
Memory: 128
NetworkMode: bridge
ExecutionRoleArn: !Ref ExecutionRole
TaskRoleArn : !Ref ExecutionRole
ContainerDefinitions:
- Name: !Sub "${ProductName}-${Region}-${Environment}-${ProjectName}-container"
Image: !Ref ContainerImageUrlwithTag
Memory: 128
Cpu: 0
MountPoints:
-
SourceVolume: "ecs-logs"
ContainerPath: "/var/log/ecs"
Command:
- python
- sftp_python.py
WorkingDirectory: "/usr/local/aws-swa"
Secrets:
-
Name: SFTP_PASSWORD
ValueFrom: !Sub ${CNEnvironment}.sftp-password
Environment:
-
Name: APPLICATION_LOGS
Value: !Sub "/var/log/ecs/${ProductName}-${Region}-${Environment}-${ProjectName}-ecs-task.logs"
-
Name: SFTP_HOST
Value: !Ref SFTPHostFQDN
-
Name: SFTP_PORT
Value: !Ref SFTPHostPort
-
Name: SFTP_USERNAME
Value: !Ref SFTPUserName
RequiresCompatibilities:
- EC2
Volumes:
-
Host:
SourcePath: "/var/log/ecs"
Name: "ecs-logs"</b></span></span></pre></div><div style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><br /></div><div style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;">Let me know, for any questions in comment box.</div><div style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><span face="verdana, sans-serif"><br /></span></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com3tag:blogger.com,1999:blog-1692034155797331883.post-43664394729935229972021-07-23T16:18:00.002+05:302021-07-23T16:18:12.071+05:30Python Encode and Decode string using BASE64 module<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimgnZ6YMuf-oX-YLfNQv6HPyK_bJW5dTXqbTcy7Xt1YoaAD0x-2O3fNOn0_Vqm0M2FY87lXk7mYMqNUeEu1H9lxpNU5aCVtDd24RmHyVvOWjm2PnDYEolYno-Ds9_NP7neBwrqjMng_UY/s601/python-logo-master-v3-TM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="203" data-original-width="601" height="135" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimgnZ6YMuf-oX-YLfNQv6HPyK_bJW5dTXqbTcy7Xt1YoaAD0x-2O3fNOn0_Vqm0M2FY87lXk7mYMqNUeEu1H9lxpNU5aCVtDd24RmHyVvOWjm2PnDYEolYno-Ds9_NP7neBwrqjMng_UY/w400-h135/python-logo-master-v3-TM.png" width="400" /></a></div><br /><p></p><h2 style="background-color: white; margin: 0px; position: relative;"><span style="color: #333333; font-family: verdana, sans-serif;"><span style="font-size: 14px; letter-spacing: 1px;">base64 </span><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;">is a python library that can be used to encrypt and decrypt strings and characters, that can have a multiple use case.</span></span></h2><div><span style="color: #333333; font-family: verdana, sans-serif;"><span style="font-size: 14px; letter-spacing: 1px;">One common use case is instead of directly pasting a plain text credential parameters into a file or as a parameter and that can be later decrypted using the decode statements within the program.</span></span></div><div style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;"><br /></span></div><div style="background-color: white; color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;">Within this blog post, we will cover.</span></div><div style="background-color: white;"><ul style="color: #333333; font-family: Lora, serif; font-size: 14px; letter-spacing: 1px; line-height: 1.4; margin: 0.5em 0px; padding: 0px 2.5em;"><li style="margin: 0px 0px 0.25em; padding: 0px;"><span style="font-family: verdana, sans-serif;">How can we encrypt strings using base64</span></li><li style="margin: 0px 0px 0.25em; padding: 0px;"><span style="font-family: verdana, sans-serif;">How can be decrypt the above base64 encrypted string using bas64 decoder.</span></li></ul><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;"><br /></span></div><div><span style="font-family: verdana, sans-serif;"><span style="color: #333333;"><span style="font-size: 14px; letter-spacing: 1px;">Please note the two important points before we use this module.</span></span></span></div><div><span style="font-family: verdana, sans-serif;"><span style="color: #333333;"><span style="font-size: 14px; letter-spacing: 1px;"><br /></span></span></span></div><div><ul style="text-align: left;"><li><span style="color: #333333; font-family: verdana, sans-serif;"><span style="font-size: 14px; letter-spacing: 1px;">base64 encode and decode functions both require a bytes-like object. In order to get our string into bytes, we must encode it first using Python's built in encode function. Most commonly, the UTF-8 encoding is used.</span></span></li><li><span style="color: #333333; font-family: verdana, sans-serif; font-size: 14px; letter-spacing: 1px;">Encryption</span><span style="color: #333333; font-family: verdana, sans-serif; font-size: 14px; letter-spacing: 1px;"> of same string using Linux command line interface and python shell differs, please use the same environment for both the encryption and decryption.</span></li></ul></div><div><br /></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;">Here in this example, we will encrypt our string "<b>cyberkeeda@123</b>" and later we will decrypt it.</span></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><br /></div><h4 style="color: #333333; font-size: 14px; letter-spacing: 1px; text-align: left;">Encryption</h4><div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: blue; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b># Encryption Block
import base64
base64.b64encode(bytes("cyberkeeda@123", "utf-8"))</b></span></span></pre></div></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;">Output for the above.</span></div><div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: blue; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>b'Y3liZXJrZWVkYUAxMjM='</b></span></span></pre><div style="color: black; font-size: medium; letter-spacing: normal;"><h4 style="color: #333333; font-size: 14px; letter-spacing: 1px;"><br /></h4><h4 style="color: #333333; font-size: 14px; letter-spacing: 1px;">Decryption</h4></div></div></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;">Below code can be use to decrypt the above, please use only the string content enclosed within string to decrypt, so for the above example output consider string leaving the b ( byte ) identifier.</span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: blue; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b># Decryption Block
import base64
base64.b64decode("Y3liZXJrZWVkYUAxMjM=").decode("utf-8")</b></span></span></pre></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;"><br /></span></div><div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;">Output for the above.</span></div><div><pre style="background-color: whitesmoke; border-radius: 0px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; line-height: 1.42857; margin-bottom: 10.5px; overflow-wrap: break-word; padding: 10px; word-break: break-all;"><span style="color: blue; font-family: monaco, menlo, consolas, courier new, monospace;"><span style="font-size: 14px; letter-spacing: 1px; white-space: pre-wrap;"><b>'cyberkeeda@123'</b></span></span></pre></div></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;"><br /></span></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;">Hope this small piece of snippet will help you in some context.</span></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;"><br /></span></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;"><br /></span></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;"><br /></span></div><div style="color: #333333; font-size: 14px; letter-spacing: 1px;"><span style="font-family: verdana, sans-serif;"><br /></span></div></div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0tag:blogger.com,1999:blog-1692034155797331883.post-29615433730691188502021-07-13T15:38:00.003+05:302021-07-13T15:38:32.271+05:30AWS S3 Bucket Policy to grant access to other AWS account<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHyPSoDDiM2D-09H3fL6e5Tu6_ue1PTRzrTsCh_XH_a7m9ldtg5Q2bczIPC_3FJxJuaCnnx_TRiowL3UJT_klp3aIWbI258Q6WsBf-v1X6nMWehKm6BiG9rLF1XqvhE5WLlJ9mht_iSc4/s1347/AWS+S3.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="751" data-original-width="1347" height="223" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHyPSoDDiM2D-09H3fL6e5Tu6_ue1PTRzrTsCh_XH_a7m9ldtg5Q2bczIPC_3FJxJuaCnnx_TRiowL3UJT_klp3aIWbI258Q6WsBf-v1X6nMWehKm6BiG9rLF1XqvhE5WLlJ9mht_iSc4/w400-h223/AWS+S3.png" width="400" /></a></div><br /><p></p><p><br /></p><p>AWS Bucket Policy to be used for the below requirements.</p><p></p><ul style="text-align: left;"><li>Grant access of S3 Bucket to other AWS account.</li><li>Restrict access to List and Download objects from it, nothing more nothing extra.</li></ul><div><br /></div><div></div><p></p><h3 style="background-color: white; margin: 0px; position: relative;"><div class="codehead" style="background: rgb(213, 232, 215); border-top-left-radius: 4px; border-top-right-radius: 4px; color: #599a60; font-family: consolas; font-size: 16px; font-weight: 500; letter-spacing: 1px; margin-bottom: 0px; padding: 4px 15px;">Script to extract yesterday date</div><pre class="code" style="background: rgb(237, 239, 244); border-bottom-left-radius: 4px; border-bottom-right-radius: 4px; line-height: 1.4; margin-top: 0px; overflow: auto; padding-bottom: 10px; padding-left: 10px; padding-top: 10px;"><div style="background-color: #1e1e1e; line-height: 19px;"><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><span style="color: #569cd6; font-family: consolas, "courier new", monospace;"><span style="font-size: 14px; font-weight: 400;"><br /></span></span></div><div><span style="color: #569cd6; font-family: consolas, "courier new", monospace; letter-spacing: 1px;"><span style="font-size: 14px; font-weight: 400;"> </span></span><span style="color: #569cd6; font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;"> {</span></span></div><span style="color: #569cd6; font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;"> "Sid": "Allow Bucket Read access from below AWS accounts",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::123456789012:root",
"arn:aws:iam::121314151617:root",
"arn:aws:iam::181912021222:root"
]
},
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": "arn:aws:s3:::cyberkeeda-limited-access-bucket/*"
}
]
}</span></span><span style="color: #569cd6; font-family: consolas, "courier new", monospace; letter-spacing: 1px;"><span style="font-size: 14px; font-weight: 400;">)</span></span></div><div style="background-color: #1e1e1e; line-height: 19px;"><span style="color: #569cd6; font-family: consolas, "courier new", monospace; letter-spacing: 1px;"><span style="font-size: 14px; font-weight: 400;"><br /></span></span></div></pre></h3><div><br /></div><div>Hope this snippet, helps you !</div>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com5tag:blogger.com,1999:blog-1692034155797331883.post-33281902126544175872021-06-29T17:22:00.005+05:302021-06-29T17:29:13.436+05:30How to get yesterday's date using Python timedelta<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimgnZ6YMuf-oX-YLfNQv6HPyK_bJW5dTXqbTcy7Xt1YoaAD0x-2O3fNOn0_Vqm0M2FY87lXk7mYMqNUeEu1H9lxpNU5aCVtDd24RmHyVvOWjm2PnDYEolYno-Ds9_NP7neBwrqjMng_UY/s601/python-logo-master-v3-TM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="203" data-original-width="601" height="135" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimgnZ6YMuf-oX-YLfNQv6HPyK_bJW5dTXqbTcy7Xt1YoaAD0x-2O3fNOn0_Vqm0M2FY87lXk7mYMqNUeEu1H9lxpNU5aCVtDd24RmHyVvOWjm2PnDYEolYno-Ds9_NP7neBwrqjMng_UY/w400-h135/python-logo-master-v3-TM.png" width="400" /></a></div><p></p><h3 style="background-color: white; color: #333333; font-family: Lora, serif; letter-spacing: 1px; margin: 0px; position: relative;"><div><span style="font-family: helvetica;">How to use datetime module to get yesterday's date.</span></div><div><span style="font-family: helvetica;"><br /></span></div><div><div><span style="font-size: small;"><span style="font-weight: 400;">All of us use Python's date time library for multiple date/time task, there is an additional function named as </span>timedelta <span style="font-weight: normal;">that can be used to get previous date as per requested varaible.</span></span></div><div><br /></div><div><span style="font-size: x-small;">Let's go through it :)</span></div><div><span style="font-size: small;"><br /></span></div></div></h3><h3 style="background-color: white; margin: 0px; position: relative;"><div class="codehead" style="background: rgb(213, 232, 215); border-top-left-radius: 4px; border-top-right-radius: 4px; color: #599a60; font-family: consolas; font-size: 16px; font-weight: 500; letter-spacing: 1px; margin-bottom: 0px; padding: 4px 15px;">Script to extract yesterday date</div><pre class="code" style="background: rgb(237, 239, 244); border-bottom-left-radius: 4px; border-bottom-right-radius: 4px; line-height: 1.4; margin-top: 0px; overflow: auto; padding-bottom: 10px; padding-left: 10px; padding-top: 10px;"><div style="background-color: #1e1e1e; line-height: 19px;"><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><span style="color: #569cd6; font-family: consolas, "courier new", monospace;"><span style="font-size: 14px; font-weight: 400;"><br /></span></span></div><div><span style="color: #569cd6; font-family: consolas, "courier new", monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;"> </span></span><span style="color: #569cd6; font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;"># Python script to get yesterday date.</span></span></div><span style="font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;"><span style="color: #569cd6;"> from datetime import date, timedelta
# Block to get present day data
today = date.today()
print("Today's date : ", today.strftime('%Y-%m-%d'))
# Block to get yesterday data
yesterday = today - timedelta(</span><span style="color: red;">days = 1</span><span style="color: #569cd6;">)
print("</span></span></span><span style="color: #569cd6; font-family: consolas, "courier new", monospace; font-size: 14px; font-weight: 400; letter-spacing: 1px;">Yesterday's date </span><span style="color: #569cd6; font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;">: ", yesterday.strftime('%Y-%m-%d'))</span></span></div><div style="background-color: #1e1e1e; line-height: 19px;"><span style="color: #569cd6; font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;"><br /></span></span></div></pre></h3><h3 style="background-color: white; color: #333333; letter-spacing: 1px; margin: 0px; position: relative;"><span style="font-family: helvetica; font-size: small; font-weight: 400;">Output</span></h3><div><span style="font-family: helvetica; font-size: x-small;"><span style="font-weight: 400;"><br /></span></span></div><h3 style="background-color: white; color: #333333; font-family: Lora, serif; letter-spacing: 1px; margin: 0px; position: relative;"><div></div></h3><h3 style="background-color: white; margin: 0px; position: relative;"><pre class="code" style="background: rgb(237, 239, 244); border-bottom-left-radius: 4px; border-bottom-right-radius: 4px; line-height: 1.4; margin-top: 0px; overflow: auto; padding-bottom: 10px; padding-left: 10px; padding-top: 10px;"><div style="background-color: #1e1e1e; line-height: 19px;"><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><span style="color: #569cd6; font-family: consolas, "courier new", monospace;"><span style="font-size: 14px; font-weight: 400;"> </span></span></div><div><span style="color: #569cd6; font-family: consolas, "courier new", monospace; letter-spacing: 1px;"><span style="font-size: 14px;"> </span></span><span style="color: #569cd6; font-family: consolas, "courier new", monospace; font-size: 14px; font-weight: 400; letter-spacing: 1px;">Today's date : </span><span style="color: #569cd6; font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;">2021-06-29</span></span></div><div><span style="color: #569cd6; font-family: consolas, "courier new", monospace; font-size: 14px; font-weight: 400; letter-spacing: 1px;"><span> </span></span><span style="color: #569cd6; font-family: consolas, "courier new", monospace; font-size: 14px; font-weight: 400; letter-spacing: 1px;">Yesterday's date : </span><span style="color: #569cd6; font-family: consolas, "courier new", monospace; font-size: 14px; font-weight: 400; letter-spacing: 1px;">2021-06-28</span></div><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><br style="font-size: 16.38px;" /></div></div></pre></h3><h3 style="background-color: white; color: #333333; letter-spacing: 1px; margin: 0px; position: relative;"><div style="font-family: Lora, serif;"></div><div style="font-family: Lora, serif;"><span style="font-family: helvetica; font-size: small;"><br /></span></div><div style="font-family: Lora, serif;"><span style="font-family: helvetica; font-size: small;">Same way, we can change the timedelta to n number of day ( day = n )</span></div><div><span style="font-family: helvetica; font-size: x-small;">Below, we will extract the 4 days old date</span></div><div style="font-family: Lora, serif;"><br /></div><div style="font-family: Lora, serif;"><span style="font-family: helvetica; font-size: small;"><br /></span></div></h3><h3 style="background-color: white; margin: 0px; position: relative;"><div class="codehead" style="background: rgb(213, 232, 215); border-top-left-radius: 4px; border-top-right-radius: 4px; color: #599a60; font-family: consolas; font-size: 16px; font-weight: 500; letter-spacing: 1px; margin-bottom: 0px; padding: 4px 15px;">Script to extract yesterday date</div><pre class="code" style="background: rgb(237, 239, 244); border-bottom-left-radius: 4px; border-bottom-right-radius: 4px; line-height: 1.4; margin-top: 0px; overflow: auto; padding-bottom: 10px; padding-left: 10px; padding-top: 10px;"><div style="background-color: #1e1e1e; line-height: 19px;"><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><span style="color: #569cd6; font-family: consolas, "courier new", monospace;"><span style="font-size: 14px; font-weight: 400;"><br /></span></span></div><div><span style="color: #569cd6; font-family: consolas, "courier new", monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;"> </span></span><span style="color: #569cd6; font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;"># Python script to get yesterday date.</span></span></div><span style="font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;"><span style="color: #569cd6;"> from datetime import date, timedelta
# Block to get present day data
today = date.today()
print("Today's date : ", today.strftime('%Y-%m-%d'))
# Block to get yesterday data
yesterday = today - timedelta(</span><span style="color: red;">days = 4</span><span style="color: #569cd6;">)
print("</span></span></span><span style="color: #569cd6; font-family: consolas, "courier new", monospace; font-size: 14px; font-weight: 400; letter-spacing: 1px;">Yesterday's date </span><span style="color: #569cd6; font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;">: ", yesterday.strftime('%Y-%m-%d'))</span></span></div><div style="background-color: #1e1e1e; line-height: 19px;"><span style="color: #569cd6; font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;"><br /></span></span></div></pre></h3><h3 style="background-color: white; color: #333333; letter-spacing: 1px; margin: 0px; position: relative;"><span style="font-family: helvetica; font-size: small; font-weight: 400;">Output</span></h3><h3 style="background-color: white; color: #333333; font-family: Lora, serif; letter-spacing: 1px; margin: 0px; position: relative;"><span style="font-family: helvetica; font-size: x-small;"><span style="font-weight: 400;"><br /></span></span></h3><h3 style="background-color: white; color: #333333; font-family: Lora, serif; letter-spacing: 1px; margin: 0px; position: relative;"><div></div></h3><h3 style="background-color: white; margin: 0px; position: relative;"><pre class="code" style="background: rgb(237, 239, 244); border-bottom-left-radius: 4px; border-bottom-right-radius: 4px; line-height: 1.4; margin-top: 0px; overflow: auto; padding-bottom: 10px; padding-left: 10px; padding-top: 10px;"><div style="background-color: #1e1e1e; line-height: 19px;"><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><span style="color: #569cd6; font-family: consolas, "courier new", monospace;"><span style="font-size: 14px; font-weight: 400;"> </span></span></div><div><span style="color: #569cd6; font-family: consolas, "courier new", monospace; letter-spacing: 1px;"><span style="font-size: 14px;"> </span></span><span style="color: #569cd6; font-family: consolas, "courier new", monospace; font-size: 14px; font-weight: 400; letter-spacing: 1px;">Today's date : </span><span style="color: #569cd6; font-family: consolas, courier new, monospace;"><span style="font-size: 14px; font-weight: 400; letter-spacing: 1px;">2021-06-29</span></span></div><div><span style="color: #569cd6; font-family: consolas, "courier new", monospace; font-size: 14px; font-weight: 400; letter-spacing: 1px;"> </span><span style="color: #569cd6; font-family: consolas, "courier new", monospace; font-size: 14px; font-weight: 400; letter-spacing: 1px;">Yesterday's date : </span><span style="color: #569cd6; font-family: consolas, "courier new", monospace; font-size: 14px; font-weight: 400; letter-spacing: 1px;">2021-06-25</span></div><div style="color: #333333; font-family: Lora, serif; letter-spacing: 1px;"><br style="font-size: 16.38px;" /></div></div></pre></h3><h3 style="background-color: white; color: #333333; font-family: Lora, serif; letter-spacing: 1px; margin: 0px; position: relative;"><div></div><div><br /></div></h3>Admin@CyberKeedahttp://www.blogger.com/profile/04693345984696492143noreply@blogger.com0