Automation of Application (Jenkins) Deployment & Configuration on AWS using CloudFormation Init

In this blog series, we have followed many infrastructure automation techniques as well as the setup and configuration of the Jenkins application. Using Terraform in Part 1 and CloudFormation as the IaC tool in Part 2, we demonstrated how this was accomplished.

In the third and final blog of this series, we will illustrate how to automate an infrastructure setup with Jenkins installation & configuration using AWS Cloudformation Init

Need for Automation

Here in our scenario, we are going to launch an EC2 instance using the Amazon Linux operating system and create the required infrastructure set up in the AWS cloud with Jenkins installed and configured for creating and building a simple pipeline job.

If we launched ‘n’ instances in accordance with our needs, it would take more time to complete this with a manual procedure. When the instance is launched with the necessary infrastructure and Jenkins Configurations, this could be a risk. In order to meet the criteria, in this case, automation using AWS Cloudformation Init is the perfect solution.

Automation Tools

• Terraform
• Cloudformation-userdata
• Cloudformation Init

Why AWS Cloudformation Init

Cfn-init can be used to retrieve and interpret resource metadata, install packages, create files and start services. Cfn-init helper script reads template metadata from the AWS::CloudFormation::Init key and acts accordingly to Fetch and parse metadata from CloudFormation, Install packages, write files to disk, Enable/Disable and Start/Stop services.

Cfn-signal can be used to signal with a creation policy or wait condition, so you can synchronize other resources in the stack when the prerequisite resource or application is ready.

Before AWS Cloudformation Init

Previously, Terraform was a popular choice, but CloudFormation Init has simplified automation because it automates and orchestrates the creation, update, and deletion of any AWS resources. Additionally, CloudFormation Init supports deployment failure recovery and offers fine-grained permissions. It can check the logs for any error /var/log/cfn-init.log. On the other hand, Terraform does not provide an automatic rollback function for incorrect changes of resources.

Challenges:

1. The main challenge while performing the cfn-init was that the Jenkins automation for disabling the setup wizard was difficult to execute.
2. There was an issue with the new build after disabling the setup wizard.
3. The git clone was not done correctly so we got an error (The code has multiple lines)

Tips & Tricks:

1. To overcome the challenge of Jenkins automation for disabling the setup wizard we add the groovy file.

2. We solved the problem with the new build by installing the workflow aggregator plugin

3. Check the git clone URL before updating the stack to avoid (The code has multiple lines) errors.

AWS CloudFormation creates the following resources:

• VPC with subnets and gateways associated.

• EC2 instance with amazon linux OS.

• Security groups with respective ports opened.

Process Summary:

We create an EC2 instance with Amazon Linux OS with existing VPC and the networking components by using cfn-init. Installation of cfn-init isn’t necessary because Amazon Linux OS has cfn-init by default.

Using Cloudformation Init:

By using metadata, the following steps for the installation and configuration of Jenkins can be done:

1. We mount the disk volume for equal sharing of the disk space.
2. We install the prerequisites such as jdk and git, then install Jenkins tools. 
3. Generally, configsets are used when there are multiple configs that you want to run. If not, we can use the default config.
4. The configsets will define how we handle the additional EBS volume that we create with the instance.
5. We mount the volume in var/lib/jenkins.

By using userdata the following steps for the automation and configuration of jenkins can be done:

1. Disable the Initial Setup Wizard by creating jenkins.install.UpgradeWizard.state with a “2.0” content.
2. Provide a secure login id and password for default admin account using the basic-security.groovy file with respective as given in the git repository within the init.groovy.d folder.
3. For the installation of the required plugins we need to download the Jenkins CLI in the instance.

Finally, we validate by using Jenkins CLI commands to create and build the job by retrieving the jenkins file from the CodeCommit Repository.

 Output:

Other possible ways:

In Terraform the same process can be accomplished by using the userdata. Since terraform doesn’t have a rollback option and we cannot check through the logs, we have opted for cfn-init. 

Reference:

 Link for disabling the initial setup wizard:

https://github.com/geerlingguy/ansible-role-jenkins/issues/50

Link for cfn-init user guide:  https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-init.html

Link for github repository:  https://github.com/Roshna-R/cfn-init.git

If you have any questions or suggestions, please reach out to us at contactus@1cloudhub.com

Written by:  Pavithra S , Roshna R, , Umashankar N, and Yuvanshankar G,

Published On: July 7, 2022