GCP Application Infrastructure

Notes

This is a WIP as I build out various pieces of the infrastructure. The goal of this infrastructure is to having a template I can apply to any of my future projects whenever it's time to deploy.

Overview

This project contain an application deployed on Google Cloud Platform (GCP). It integrates with Kubernetes (GKE) for container orchestration, Compute Engine for Prometheus and Grafana monitoring, and Terraform for infrastructure as code. The app is designed to provide insights into its operations using Prometheus for metrics collection and Grafana for visualization.

Features

• Kubernetes Cluster (GKE): Hosts the application in a scalable and managed environment.
• Compute Engine: Runs Prometheus and Grafana for monitoring and visualization.
• Infrastructure as Code (Terraform): Automates the provisioning of all cloud resources.
• Networking (VPC): Configures separate subnets for GKE and Compute Engine instances for isolation.

Architecture

1. VPC Configuration
   • Subnet-1: Dedicated for GKE.
   • Subnet-2: Dedicated for Compute Engine.
2. Compute Engine:
   • Runs Prometheus and Grafana for monitoring.
3. GKE Cluster:
   • Hosts the application container.

Prerequisites

1. Tools Required:
   • Terraform
   • Google Cloud SDK
   • Kubectl
   • Ansible (for Compute Engine setup)
2. GCP Requirements:
   • Enabled APIs: Compute Engine, Kubernetes Engine, Artifact Registry.
   • A billing-enabled project.
3. Access:
   • Admin permissions to create and manage resources.

Setup Instructions

Step 1: Clone the Repository

git clone <repository-url>
cd <repository-folder>

Step 2: Initialize Terraform

terraform init

Step 3: Create Terraform Variables

Edit terraform.tfvars to match your project setup:

project_id = "<your-project-id>"
region = "us-central1"
compute_zone = "us-central1-f"
compute_subnet_name = "subnet-2"
compute_subnet_cidr = "10.0.1.0/24"
gke_subnet_name = "subnet-1"
gke_subnet_cidr = "10.0.0.0/24"
vpc_name = "devops-vpc"
cluster_name = "devops-app-cluster"

Step 4: Apply Terraform Configuration

terraform apply

Step 5: Configure Ansible for Compute Engine

1. Update the inventory/hosts file with the Compute Engine public IP address.
2. Run the Ansible playbook:

   ansible-playbook -i inventory/hosts playbooks/setup-prometheus-grafana.yml

Step 6: Deploy Application to GKE

1. Build and push the application image:

docker build -t /:v1 . docker push /:v1

2. Apply Kubernetes deployment and service configurations:
```bash
kubectl apply -f kubernetes/devops-app-deployment.yaml
kubectl apply -f kubernetes/devops-app-service.yaml

3. Verify the application is running:

   kubectl get pods
   kubectl get services

Step 7: Access the Application

• Access the application using the external IP of the service (LoadBalancer type). Use:

  kubectl get services

Step 8: Access Prometheus and Grafana

• Prometheus: http://<compute-engine-ip>:9090
• Grafana: http://<compute-engine-ip>:3000

Monitoring and Troubleshooting

1. Verify Terraform:

   terraform plan

2. Check Kubernetes Resources:

   kubectl get all

3. Validate Ansible Playbook:

   ansible-playbook -i inventory/hosts playbooks/setup-prometheus-grafana.yml --check

4. Debug Compute Engine:
   • SSH into the instance:

     gcloud compute ssh <instance-name> --zone <zone>

   • Check Prometheus and Grafana logs.

Cleanup

To clean up all resources:

terraform destroy

References

• Terraform Documentation
• Google Cloud SDK
• Prometheus Documentation
• Grafana Documentation