In a previous blog post, we covered the fundamentals of Terraform and Infrastructure as Code (IaC) Check here. Building upon that foundation, this blog will delve into advanced concepts by showcasing a real-world project: Automated AWS Infrastructure & Deployment. We'll explore how to leverage Terraform for provisioning AWS resources and integrate GitHub Actions for seamless automation.

Prerequisites:

Before diving into the Automated AWS Infrastructure & Deployment with Terraform and GitHub Actions project, ensure you have the following prerequisites in place:

1. AWS Account:

   • Ensure you have an AWS account. If you don't have one, you can create it here.

2. AWS CLI:

   • Install the AWS Command Line Interface (CLI) on your local machine. You can download it from the official AWS CLI page.

3. AWS Access Key and Secret Access Key:

   • Set up an AWS Access Key ID and Secret Access Key. These credentials are required for programmatic access to your AWS account. Follow the AWS documentation on Managing Access Keys to create and manage these credentials securely.

4. Terraform Installed:

   • Install Terraform on your local machine. You can download Terraform from the official Terraform website. Ensure that the Terraform binary is added to your system's PATH.

Once you have these prerequisites in place, you'll be ready to explore the project, automate AWS infrastructure, and leverage GitHub Actions.

Project Overview:

The project aims to showcase the automation of AWS infrastructure deployment using Terraform and GitHub Actions. The primary goals include demonstrating a modular, scalable, and highly available architecture for hosting web applications on AWS.

• Infrastructure Definition: Utilizing Terraform to define and manage the complete AWS infrastructure.

• Continuous Deployment: Implementing GitHub Actions for a streamlined CI/CD pipeline, automating testing and deployment processes.

Project Structure:

The project's architecture employs a modular and automated approach, utilizing Terraform and GitHub Actions to orchestrate the deployment of a scalable and highly available AWS infrastructure. Each component plays a vital role in achieving the project's goals of consistency, efficiency, and scalability.

Terraform Workflow:

In the Automated AWS Infrastructure & Deployment project, the Terraform configurations follow a modular structure to enhance organization, reusability, and maintainability. Each infrastructure component, such as EC2 instances, Virtual Private Cloud (VPC), Application Load Balancer (ALB), and Security Groups (SG), is defined in separate Terraform modules. These modules are then composed in the main.tf file to create a comprehensive and modular infrastructure. Let's explore the main components:

1. Terraform Modules:

- EC2 Module (modules/ec2):

• Defines the configuration for EC2 instances hosting the web application.

• Utilizes parameters for customization, allowing flexibility in instance types, AMIs, and other settings.

- VPC Module (modules/vpc):

• Configures the Virtual Private Cloud (VPC) with private and public subnets for enhanced network isolation.

• Parameters enable customization of CIDR blocks, availability zones, and other VPC settings.

- ALB Module (modules/alb):

• Defines the Application Load Balancer (ALB) to distribute traffic across EC2 instances.

• Parameters support customization of listener rules, Target groups and other load balancer settings.

- SG Module (modules/sg):

• Configures Security Groups (SG) to control inbound and outbound traffic for resources.

• Customizable parameters allow for specific rules, port configurations, and other security settings.

2. Main Configuration (main.tf):

In the main.tf file, these modules are composed to define the overall infrastructure. This file acts as the orchestrator, pulling in the individual modules and specifying their interconnections.

module "VPC" {
  source       = "./modules/vpc"
  subnet_names = var.subnet_names
}
module "security_group" {
  source = "./modules/sg"
  vpc_id = module.VPC.vpc_id
}
module "EC2" {
  source    = "./modules/ec2"
  sg_id     = module.security_group.sg_id // variable = module.<module_name>.<output_name>
  subnet_id = module.VPC.subnet_id
}
module "alb" {
  source = "./modules/alb"
  sg_id = module.security_group.sg_id
  subnet_id = module.VPC.subnet_id  
  vpc_id = module.VPC.vpc_id
  instance_ids = module.EC2.instance_ids
}


# outputs
output "sg_id" {
  value = module.security_group.sg_id
}
output "ec2_public_ip" {
  value = module.EC2.aws_instance_public_ip
}
output "ec2_ids" {
  value = module.EC2.instance_ids
}
output "user_data" {
  value = module.EC2.user_data
}

Explore Full Code:

Note that this only the main file(main.tf). For the complete Terraform project code, including modules and additional configurations, please visit this GitHub repository.

Benefits of Modular Approach:

• Organization: Clear separation of concerns for each component.

• Reusability: Modules can be reused in various projects or environments.

• Customization: Parameters allow for easy customization without modifying module internals.

• Maintainability: Simplifies maintenance by isolating updates within specific modules.

GitHub Actions Workflow:

Before delving into the GitHub Actions workflow for your Automated AWS Infrastructure & Deployment project, ensure the following prerequisites are set up:

1. GitHub Repository Structure:

   • Create a dedicated folder .github/workflows, in your GitHub repository to store your GitHub Actions workflow files (YAML file). This helps maintain a clean and organized repository structure and github for checking workflow files.

2. GitHub Secrets for AWS Credentials:

Add the necessary AWS credentials as secrets in your GitHub repository. This includes AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION. Secrets provide a secure way to store sensitive information and are used.

**
Setting Up Secrets in GitHub**

To securely store sensitive information, such as AWS credentials, GitHub provides a feature called secrets. Here's a brief guide on how to set up secrets in your GitHub repository:

1. Navigate to Your Repository:

Go to your GitHub repository, and in the top navigation bar, click on "Settings."

2. Access Secrets:

In the left sidebar, find and click on "Secrets" under the "Settings" section.

3. Add a New Secret:

Click on the "New repository secret" button.

4. Enter Secret Details:

• Name: Enter a name for your secret (e.g., AWS_ACCESS_KEY_ID).

• Value: Input the corresponding secret value (e.g., your AWS Access Key ID).

5. Save the Secret:

Click on the "Add secret" button to save the secret. Repeat the process for additional secrets.

name: terraform-deploy-cicd
run-name: ${{github.actor}}-terraform-deploy-cicd 
# github.actor refers to the username of the user or the name of the 
# GitHub Actions bot that triggered the workflow. 
# It represents the entity that initiated the event
on:
  push:
    branches:
      - main
    paths-ignore:
      - 'README.md'
      - 'LICENSE'
      - '**.md'
env:
  AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY_ID}}
  AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET_ACCESS_KEY}}
  AWS_REGION: ${{secrets.AWS_REGION}}

jobs:
  deploy:
    name: Terraform Deployment
    runs-on: ubuntu-latest
    steps:
      - name: Step 1: Checkout Code
        uses: actions/checkout@v2

      - name: Step 2: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{secrets.AWS_ACCESS_KEY_ID}}
          aws-secret-access-key: ${{secrets.AWS_SECRET_ACCESS_KEY}}
          aws-region: ${{secrets.AWS_REGION}}

      - name: Step 3: Terraform Setup
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_wrapper: false 

      - name: Step 4: Terraform Init
        run: terraform init

      - name: Step 5: Terraform Validate
        run: terraform validate 

      - name: Step 6: Terraform Apply
        run: |
          TF_LOG=DEBUG
          terraform apply -auto-approve

      - name: Step 7: Terraform Destroy
        run: |
          TF_LOG=DEBUG
          terraform destroy -auto-approve

Steps:

Step 1: Checkout Code

• Uses GitHub Actions' built-in actions/checkout to fetch the Terraform code from the repository.

Step 2: Configure AWS Credentials

• Utilizes the aws-actions/configure-aws-credentials action to securely configure AWS credentials using GitHub Secrets.

Step 3: Terraform Setup

• Leverages hashicorp/setup-terraform action to set up the correct version of Terraform in the GitHub Actions environment, ensuring compatibility.

Step 4: Terraform Init

• Executes terraform init to initialize the working directory and download necessary Terraform providers.

Step 5: Terraform Validate

• Performs terraform validate to check Terraform configurations for syntax errors and potential issues.

Step 6: Terraform Apply

• Initiates terraform apply -auto-approve to automatically apply infrastructure changes. Detailed logs are captured using TF_LOG=DEBUG.

Step 7: Terraform Destroy

• Executes terraform destroy -auto-approve to clean up the infrastructure, providing a complete deployment and destruction cycle.

After you create the infrastructure remember to destroy the resources by running or directly addingterraform destroycommand to destroy the created resources to not incur any charges.

Overview of Deployment:

1. EC2 Instances

The EC2 module deploys instances hosting the web application, leveraging Amazon Linux 2 and nginx.

2. Application Load Balancer (ALB)

The ALB module ensures high availability by evenly distributing traffic across multiple EC2 instances.

4. Terraform Orchestrated Deployment

This is the deployed site on both the ec2 instances after installing a nginx server by passing a shell script to auser_dataattribute of ec2 instance resource in terraform alternatively you can also install or created files after installing nginx on instance.

As depicted in the images above, observe how the Application Load Balancer efficiently distributes incoming traffic across both instances, ensuring optimal load balancing and high availability for the hosted application.

This dynamic distribution mechanism plays a crucial role in enhancing the performance and reliability of the infrastructure by effectively managing the workload among the deployed instances.

Conclusion

In conclusion, the Automated AWS Infrastructure & Deployment project, driven by Terraform and GitHub Actions, exemplifies the efficiency and scalability achieved through Infrastructure as Code (IaC). The modular Terraform structure, coupled with a streamlined GitHub Actions workflow, ensures a consistent and automated deployment pipeline.

For a hands-on experience, explore the complete Terraform project on GitHub. Happy coding!

Next Steps:

Encourage readers to explore the project on GitHub, experiment with the provided code, and adapt it to their own use cases. Mention any related resources, documentation, or additional topics they might find beneficial and do checkout my other blogs.

In the rapidly evolving landscape of cloud computing, managing infrastructure efficiently has become a critical aspect of software development. Enter Terraform, a revolutionary Infrastructure as Code (IaC) tool that empowers developers to provision and manage infrastructure in a declarative and version-controlled manner. In this blog, we will embark on a journey to unravel the core concepts of Terraform, exploring its inner workings, state management, and the role of providers.

What is Terraform?

At its core, Terraform is an open-source IaC tool developed by HashiCorp. It enables developers to define and provision infrastructure using a declarative configuration language. This means you describe the desired state of your infrastructure, and Terraform takes care of bringing it to life.

Infrastructure as Code (IaC)

IaC is a paradigm shift in infrastructure management. Instead of manually configuring servers and resources, IaC allows developers to codify infrastructure using scripts. This approach brings numerous benefits, including repeatability, version control, and collaboration.

Terraform embraces IaC by allowing users to define infrastructure in HashiCorp Configuration Language (HCL), a simple and human-readable language. With IaC, infrastructure becomes more predictable, scalable, and manageable and quite resembles javascript objects or JSON.

Core Concepts of Terraform

1. Providers:

At the heart of Terraform are providers. Providers define and manage the lifecycle of resources, such as virtual machines, networks, or databases. Providers interact with APIs of various cloud or on-premises services.

In this blog, we will consider AWS as an example but you can choose from multiple provider plugins from the Terraform site.

terraform { // terraform block defines the version of terraform to be used and the backend configuration
 // local backend configuration
  backend "local" {
    path = "terraform.tfstate"
  }
  required_providers {
    aws = {
      source  = "hashicorp/aws" //  location of the provider plugin.
      version = "~> 5.0"        //  Terraform registry path and 5+ version
    }  

  }
}

In this example, we declare an AWS provider and specify the region for resource provisioning.

1. terraform block: This block defines the Terraform version to be used. In this case, it's not explicitly set, so it will use the default version installed on your system.

2. backend "local" {…}: This block defines a local Terraform backend, which stores the Terraform state file on the local filesystem. The path specified is terraform.tfstate and it maintains the state file and keeps track of the desired state.

3. required_providers {aws = {…}: This block defines the required provider plugin for AWS. The source is set to hashicorp/aws, which indicates that the plugin should be downloaded from the HashiCorp registry. The version is set to "~> 5.0", which means that Terraform will use the latest version of the plugin that is greater than or equal to 5.0.

2. Resources:

Resources are the building blocks of your infrastructure. They represent the cloud components you want to create, such as instances, storage, or databases. Terraform uses resource configurations to understand what infrastructure to create.

resource "aws_instance" "Terraform-instance" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
  tags = {
    Name = "terraformGettingStarted"
  }
  count = 2
}

Here, we define an AWS instance with a specific Amazon Machine Image (AMI) and instance type with a count of 2 instances and tags for reference. The resource type and resource name both act as a unique ID for the resource for identification.

3. Variables:

Variables provide a way to parameterize your configurations. They allow you to reuse and customize configurations for different environments or use cases.

variable "instance_type" {
  description = "The type of EC2 instance"
  default     = "t2.micro"
}

This example creates a variable for the instance type, allowing flexibility in configuration.

4. Modules:

Modules enable code reuse and maintainability. They encapsulate a set of resources and configurations, allowing you to organize and share infrastructure code.

module "web_server" {
  source = "./modules/web_server"

  instance_count = 3
}

Here, we use a module to create multiple web server instances with a shared configuration.

Terraform State

Terraform maintains a state file that records the current state of your infrastructure. This state is crucial for understanding what Terraform has provisioned and for making informed decisions about changes.

State files can be stored remotely or locally, depending on your configuration. Terraform uses the state to plan and apply changes, ensuring that the desired and actual states align.

Terraform Workflow

1. The user writes a Terraform configuration file, which defines the desired state of the infrastructure.

2. The user runs the terraform plan command to generate a plan of the changes that will be made if the configuration file is applied.

3. The user reviews the plan and, if they are satisfied, runs the terraform apply command to apply the changes.

   Terraform will then create, update, or delete resources as needed to match the desired state defined in the configuration file.

Example Screenshot

Here is an example from the above configuration.

Conclusion

Terraform's power lies in its simplicity, flexibility, and scalability. By embracing IaC principles, developers can efficiently manage infrastructure, automate repetitive tasks, and collaborate seamlessly. Understanding core concepts like providers, resources, variables, and modules is key to harnessing the full potential of Terraform.

In subsequent blog posts, we'll dive deeper into advanced Terraform topics, exploring best practices, integration with CI/CD pipelines, and tackling real-world infrastructure challenges. Until then, happy Terraforming!

In the ever-evolving landscape of cloud computing, serverless technology has emerged as a game-changer. Among the leading platforms that offer serverless capabilities, Amazon Web Services (AWS) has been a pioneer with its Lambda service. In this blog, we'll delve into my hands-on experience with AWS Lambda, explore how serverless works, and unravel the fascinating world of Lambda functions, layers, triggers, destinations, and related concepts.

Understanding Serverless Computing

To kick things off, let's clarify what "serverless" actually means. Contrary to the name, it doesn't mean there are no servers involved. Instead, it implies that developers no longer need to manage the infrastructure. They can focus solely on writing code while AWS takes care of provisioning, scaling, and maintaining the servers.

Serverless computing enables developers to build applications with less operational overhead and more agility. AWS Lambda, being a serverless computing service, plays a pivotal role in this serverless revolution.

AWS Lambda: The Heart of Serverless

What is AWS Lambda?

AWS Lambda is a computing service that automatically scales your code in response to incoming requests. It allows you to run code without provisioning or managing servers, making it a fundamental component of serverless architecture.

Hands-On with Lambda Functions

My journey began by creating Lambda functions. These are small, single-purpose units of code that can be triggered by various AWS services or HTTP requests. AWS provides support for multiple programming languages, which means you can write your Lambda functions in your language of choice like Nodejs, Python etc.

To create a Lambda function, you write your code and specify the event source that triggers the function's execution. Events can range from HTTP requests through Amazon API Gateway to database changes using Amazon DynamoDB streams.

Once the Lambda function is set up, AWS automatically takes care of scaling it according to the incoming request rate. This means you don't have to worry about server provisioning or maintenance.

Lambda Layers: Reusable Code Components

While working with Lambda functions, I discovered the utility of Lambda Layers. These are a way to manage and share code and data across multiple functions. Instead of duplicating code across functions, you can package common libraries, custom runtimes, or even custom tools as layers.

Lambda Layers promote code reusability, making it easier to manage and update shared components without having to redeploy every function that uses them.

Triggers and Destinations: Orchestrating Workflows

AWS Lambda functions can be triggered by a variety of event sources. These include AWS services like S3, SQS, and SNS, as well as custom events through the Amazon API Gateway. The choice of trigger depends on the specific use case and the need for seamless integration within your application architecture.

In my hands-on experience, I configured triggers for my Lambda functions that enabled them to respond to specific events. This allowed me to build event-driven, serverless workflows with ease.

Moreover, AWS Lambda offers the ability to configure destinations for asynchronous invocations. You can send the results of a Lambda function's execution to various AWS services, making it simpler to manage the flow of data and processing within your serverless applications.

Here's a simple example of an AWS Lambda function written in Node.js

export const handler = async (event, context) => {
    // Convert to JS string
    const body = JSON.stringify(event.body);

    // Perform some processing
    const result = processInput(body);

    // Prepare the response
    const response = {
        statusCode: 200,
        body: JSON.stringify(result),
        functionName: context.functionName,
        requestID: context.awsRequestId, 
    };
   // The context object is particularly useful for monitoring and 
   // logging, as you can use the information it provides to record execution details, 
   // log to CloudWatch Logs

    return response;
};

function processInput(data) {
    // In this example, we'll simply return the input data as is.
    // You can replace this with more complex processing as needed.
    return data;
}

Output:

Note: To get the output "Hello From lambda" you need to create a test event cause we haven't set any triggers as of now, you can do that in the test and create a new test event and simply add { "body": "Hello From Lambda 101" } or any other string you would like and make sure to deploy the changes.

Conclusion

My hands-on experience with AWS Lambda was a revelation. It not only provided a glimpse into the world of serverless computing but also offered a practical understanding of how to build scalable, event-driven applications with ease. Lambda functions, layers, triggers, and destinations are all integral parts of this serverless ecosystem, providing a powerful toolset for developers to create efficient, cost-effective, and scalable applications.

As serverless technology continues to evolve, AWS Lambda remains at the forefront, enabling developers to focus on code while AWS manages the infrastructure. If you're looking to embrace the serverless revolution, AWS Lambda is a fantastic place to start.