From ClickOps to DevOps: Provisioning Cloud Infrastructure using CloudFormation

The process of deploying and managing cloud infrastructure can take either of two modes. Infrastructure can either be deployed using clickable options, fondly called ClickOps, or some form of automation. ClickOps is a UI based way to manually provision resources. This means that everything has a graphic UI from where you could explore available services and manually click on the console to provision infrastructure.

Why would one want to deploy resources to the cloud using ClickOps?

For starters, its pretty much the easiest way to get started in the cloud. Additionally, its a no-code process, for the most part. You do not need to write code to deploy your services.

Despite these benefits, ClickOps has major pain points, some of which are:

1. It’s a manual process.

2. Its prone to errors because its difficult to complete a series of tasks without making mistakes.

3. You can’t track what others have done – no version control, no auditing process.
4. If you need to re-provision a resource, it takes a lot of time to perform disaster recovery. For example, if you have 50 instances and something goes wrong, you’ll need to fix them one at a time.

DevOps to the rescue!

Long time ago in a galaxy far away, traditional software development was between two different teams – the developer team and operations team. Both teams kept to themselves and had separate and sometimes conflicting goals. The developers were interested in building stuff, while the operations team were more concerned about stability and reliability. Agile methodology saw the emergence of DevOps. DevOps merges the once separated and sometimes conflicting software development and operations teams, thereby eliminating that wall of confusion between them. There is an entire argument out there concerning whether DevOps is a culture or a role. There’s probably merit on both sides of the aisle but for the purpose of this blogpost, I’ll restrict myself to how AWS sees DevOps.

> DevOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes.

What are some benefits of DevOps?

1. It’s a faster deployment strategy – DevOps increases the frequency and pace of product releases.
2. There is improved product quality – a greater collaboration between development and operations teams, in addition to feedback from end user leads to better products.
3. Scale – with DevOps, you can operate and manage your infrastructure and development process at scale.
4. Reduced cost – since maintenance and new updates are brought together, there is reduced management and production costs.
5. Reliability – DevOps ensures the quality of applications updates and infrastructure changes so you can reliably deliver at a more rapid pace while maintaining a positive experience for clients.

What are some of the trends and common questions?

1. As more companies embrace agile methodology, DevOps (and other positions) will continue to be a pivotal role within the cloud space and tech in general.
2. What exactly are DevOps engineers responsible for? DevOps engineers are responsible for infrastructure. Infrastructure here does not mean brick and mortar lol, but hardware and software that products and applications run on. DevOps engineers are responsible for proactively automating and maintaining infrastructure which helps reduce the manual and time cost of ClickOps we discussed earlier.
3. Another question that gets asked a lot is if DevOps engineers need to code? The classic engineering answer? It depends. It’s a heated industry debate and it depends on the environment and the team structure. It’s common to encounter workflows that includes ClickOps, scripting, or building an entire platform.

Automating the boring stuff

A huge part of DevOps is concerned with automation, as opposed to manually provisioning cloud infrastructure. So the question ‘why is automation important?’ comes up.

1. With automation you eliminate human errors. This means there is a low risk of misconfiguration.
2. You can literally stand-up new environments on demand.
3. Easier peer review with new team members. It’s a lot easier to review someone’s shared code as opposed to watching over their shoulder as they make changes on the UI.
4. It allows organizations to quickly scale without redundant manual work. For example, the difference between provisioning 1 instance and 1000 instances is changing the count value.

Software development life cycle (SDLC) and Infrastructure life cycle (ILC)

Software development life cycle is the end-to-end life cycle of software. It is broken up into the six phases of analysis, design, development, testing, deployment, and maintenance. Common tools used in SDLC are Git, CI/CD, Linters, and Jira.

ILC is pretty much same as SDLC, except its for your infrastructure. With infrastructure life cycle, we see this unending loop of continuous improvement. Common tools for infrastructure life cycle are infrastructure as code, configuration management, GitOps, and CI/CD.

Infrastructure as Code (IaC)

IaC is a method we could use to automate provisioning and deploying cloud infrastructure using scripts. IaC tools are commonly configured using YAML, JSON, HarshiCorp Configuration Language. For AWS, three important infrastructure as code tools are CloudFormation, Cloud Development Kit (CDK), and Terraform.

Here's a YAML code with which you could deploy an Amazon EC2 instance using CloudFormation.

AWSTemplateFormatVersion: 2010-09-09
Resources:
  myInstance:
    Type: AWS::EC2::Instance
    Properties:
      # You'll have to grab the image ID for the EC2 intsance for your preferred region
      #This image ID is for Canada central-1 for Amazon Linux 2
      ImageId: ami-01d29fca5bdf8f4b4
      #You'll need to grab the default security group ID in your AWS account
      SecurityGroupIds: ["sg-0c8ba2b706e2581e0"]
      #You'll need to grab a subnetID in the default security group's associated VPC
      SubnetId: subnet-0a6311ab0cdf8fa1b
      InstanceType: t2.micro

CloudFormation Tips and Tricks for YAML

1. Indentation has to be exactly correct.
2. Validate YAML syntax before deploying - this could be done in the CloudFormation template designer.
3. Reference the AWS documentation.
4. Determine which properties are required.
5. Reference IDs and values to AWS resources within your AWS console.
6. Verify availability zone from where you intend to deploy the resources (AMI differs across availability zones. To deploy an EC2 instance in us-east-1, for instance, you have to use the AMI from us-east-1. You'll definitely get a rollback if you use an AMI from a different availability zone).
7. If you get a rollback/error, look under 'Events' tab in CloudFormation to see what went wrong.
8. If the error on CloudFormation is not descriptive enough, you could open up CloudTrail to get additional hints.