DevOps Zero to Hero: Part 4 - Building Your First CI/CD Pipeline

Introduction

Continuous Integration and Continuous Deployment (CI/CD) are the backbone of modern DevOps practices. In this part, you'll build automated pipelines that test, build, and deploy your application automatically whenever you push code changes. We'll use GitHub Actions, but the concepts apply to any CI/CD platform.

Understanding CI/CD

Continuous Integration (CI)

• Developers frequently merge code into a shared repository

• Automated builds and tests run on every commit

• Issues are detected and fixed early

• Maintains a always-deployable main branch

Continuous Delivery (CD)

• Code changes are automatically prepared for release

• Automated testing through multiple environments

• Manual approval for production deployment

• Reduces time between writing code and using it

Continuous Deployment

• Every change that passes tests is deployed automatically

• No manual intervention required

• Requires robust testing and monitoring

• Enables rapid iteration and feedback

CI/CD Pipeline Stages

A typical pipeline includes:

1. Source: Code repository trigger

2. Build: Compile/package application

3. Test: Run automated tests

4. Analyze: Code quality and security scans

5. Package: Create deployable artifacts

6. Deploy: Release to environments

7. Monitor: Track application health

GitHub Actions Fundamentals

Core Concepts

• Workflows: Automated processes defined in YAML

• Events: Triggers that start workflows

• Jobs: Sets of steps that execute on the same runner

• Steps: Individual tasks within a job

• Actions: Reusable units of code

• Runners: Servers that execute workflows

• Artifacts: Files produced by workflows

• Secrets: Encrypted environment variables

Workflow Syntax

name: Workflow Name
on: [push, pull_request]  # Triggers
jobs:
  job-name:
    runs-on: ubuntu-latest  # Runner
    steps:
      - uses: actions/checkout@v3  # Action
      - name: Run a command  # Step
        run: echo "Hello World"

Setting Up Your First Pipeline

Let's create a comprehensive CI/CD pipeline for our Node.js application.

Basic CI Workflow

Create .github/workflows/ci.yml:

name: CI Pipeline

# Triggers
on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]
  workflow_dispatch:  # Manual trigger

# Environment variables
env:
  NODE_VERSION: '18'
  
jobs:
  # Job 1: Linting
  lint:
    name: Lint Code
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Run ESLint
        run: npm run lint
  
  # Job 2: Testing
  test:
    name: Run Tests
    runs-on: ubuntu-latest
    needs: lint  # Run after lint job
    
    strategy:
      matrix:
        node-version: [16, 18, 20]
        os: [ubuntu-latest, windows-latest, macos-latest]
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      
      - name: Setup Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Run tests
        run: npm test
      
      - name: Generate coverage report
        if: matrix.node-version == '18' && matrix.os == 'ubuntu-latest'
        run: npm run test:coverage
      
      - name: Upload coverage to Codecov
        if: matrix.node-version == '18' && matrix.os == 'ubuntu-latest'
        uses: codecov/codecov-action@v3
        with:
          file: ./coverage/lcov.info
          fail_ci_if_error: true
  
  # Job 3: Security Scan
  security:
    name: Security Scan
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      
      - name: Run Snyk Security Scan
        uses: snyk/actions/node@master
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
        with:
          args: --severity-threshold=high
      
      - name: Run npm audit
        run: npm audit --audit-level=moderate
  
  # Job 4: Build Docker Image
  build:
    name: Build Docker Image
    runs-on: ubuntu-latest
    needs: [lint, test]
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      
      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}
      
      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: ${{ secrets.DOCKER_USERNAME }}/devops-web-app
          tags: |
            type=ref,event=branch
            type=ref,event=pr
            type=semver,pattern={{version}}
            type=semver,pattern={{major}}.{{minor}}
            type=sha,prefix={{branch}}-
      
      - name: Build and push Docker image
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
          platforms: linux/amd64,linux/arm64

Advanced CI/CD Pipeline

Complete Production Pipeline

Create .github/workflows/cd.yml:

name: CD Pipeline

on:
  push:
    tags:
      - 'v*'
  workflow_run:
    workflows: ["CI Pipeline"]
    branches: [main]
    types:
      - completed

env:
  AWS_REGION: us-east-1
  ECR_REPOSITORY: devops-web-app
  ECS_SERVICE: web-app-service
  ECS_CLUSTER: production-cluster
  ECS_TASK_DEFINITION: task-definition.json

jobs:
  # Job 1: Build and Push to ECR
  build-and-push:
    name: Build and Push to ECR
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion == 'success' || github.event_name == 'push' }}
    
    outputs:
      image: ${{ steps.image.outputs.image }}
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}
      
      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1
      
      - name: Build, tag, and push image to Amazon ECR
        id: image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          echo "image=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT
  
  # Job 2: Deploy to Staging
  deploy-staging:
    name: Deploy to Staging
    runs-on: ubuntu-latest
    needs: build-and-push
    environment:
      name: staging
      url: https://staging.example.com
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}
      
      - name: Deploy to ECS Staging
        run: |
          aws ecs update-service \
            --cluster staging-cluster \
            --service web-app-staging \
            --force-new-deployment \
            --region ${{ env.AWS_REGION }}
      
      - name: Wait for deployment
        run: |
          aws ecs wait services-stable \
            --cluster staging-cluster \
            --services web-app-staging \
            --region ${{ env.AWS_REGION }}
      
      - name: Run smoke tests
        run: |
          npm run test:smoke -- --url=https://staging.example.com
  
  # Job 3: Deploy to Production
  deploy-production:
    name: Deploy to Production
    runs-on: ubuntu-latest
    needs: deploy-staging
    environment:
      name: production
      url: https://example.com
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}
      
      - name: Fill in the new image ID in the Amazon ECS task definition
        id: task-def
        uses: aws-actions/amazon-ecs-render-task-definition@v1
        with:
          task-definition: ${{ env.ECS_TASK_DEFINITION }}
          container-name: web-app
          image: ${{ needs.build-and-push.outputs.image }}
      
      - name: Deploy Amazon ECS task definition
        uses: aws-actions/amazon-ecs-deploy-task-definition@v1
        with:
          task-definition: ${{ steps.task-def.outputs.task-definition }}
          service: ${{ env.ECS_SERVICE }}
          cluster: ${{ env.ECS_CLUSTER }}
          wait-for-service-stability: true
      
      - name: Notify deployment
        uses: 8398a7/action-slack@v3
        with:
          status: ${{ job.status }}
          text: 'Production deployment completed!'
          webhook_url: ${{ secrets.SLACK_WEBHOOK }}
        if: always()

Setting Up Secrets and Variables

GitHub Secrets Configuration

1. Go to Settings → Secrets → Actions

2. Add the following secrets:

# Docker Hub
DOCKER_USERNAME: your-dockerhub-username
DOCKER_PASSWORD: your-dockerhub-password

# AWS
AWS_ACCESS_KEY_ID: your-aws-access-key
AWS_SECRET_ACCESS_KEY: your-aws-secret-key

# Snyk
SNYK_TOKEN: your-snyk-token

# Slack
SLACK_WEBHOOK: your-slack-webhook-url

Environment Protection Rules

1. Go to Settings → Environments

2. Create environments: staging, production

3. Add protection rules:

   • Required reviewers

   • Deployment branches

   • Environment secrets

Testing Strategies in CI/CD

Unit Tests

Create tests/unit/app.test.js:

const request = require('supertest');
const app = require('../../src/app');

describe('Unit Tests', () => {
  describe('GET /', () => {
    it('should return 200 OK', async () => {
      const res = await request(app).get('/');
      expect(res.statusCode).toBe(200);
    });

    it('should return correct message', async () => {
      const res = await request(app).get('/');
      expect(res.body.message).toBe('Welcome to DevOps Web App');
    });
  });

  describe('GET /health', () => {
    it('should return healthy status', async () => {
      const res = await request(app).get('/health');
      expect(res.statusCode).toBe(200);
      expect(res.body.status).toBe('healthy');
    });
  });
});

Integration Tests

Create tests/integration/api.test.js:

const request = require('supertest');
const app = require('../../src/app');

describe('Integration Tests', () => {
  let server;

  beforeAll(() => {
    server = app.listen(4000);
  });

  afterAll((done) => {
    server.close(done);
  });

  it('should handle concurrent requests', async () => {
    const requests = Array(10).fill().map(() => 
      request(app).get('/health')
    );
    
    const responses = await Promise.all(requests);
    responses.forEach(res => {
      expect(res.statusCode).toBe(200);
    });
  });
});

Smoke Tests

Create tests/smoke/smoke.test.js:

const axios = require('axios');

const URL = process.env.TEST_URL || 'http://localhost:3000';

describe('Smoke Tests', () => {
  it('should respond to health check', async () => {
    const response = await axios.get(`${URL}/health`);
    expect(response.status).toBe(200);
    expect(response.data.status).toBe('healthy');
  });

  it('should have required endpoints', async () => {
    const endpoints = ['/

', '/health', '/info', '/metrics'];
    
    for (const endpoint of endpoints) {
      const response = await axios.get(`${URL}${endpoint}`);
      expect(response.status).toBe(200);
    }
  });
});

Code Quality and Analysis

ESLint Configuration

Create .eslintrc.json:

{
  "env": {
    "node": true,
    "es2021": true,
    "jest": true
  },
  "extends": [
    "eslint:recommended",
    "plugin:security/recommended"
  ],
  "parserOptions": {
    "ecmaVersion": 12
  },
  "rules": {
    "indent": ["error", 2],
    "quotes": ["error", "single"],
    "semi": ["error", "always"],
    "no-unused-vars": ["error", { "argsIgnorePattern": "^_" }],
    "no-console": ["warn", { "allow": ["warn", "error"] }]
  },
  "plugins": ["security"]
}

SonarQube Integration

Add to workflow:

- name: SonarQube Scan
  uses: SonarSource/sonarcloud-github-action@master
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
  with:
    args: >
      -Dsonar.projectKey=your-project
      -Dsonar.organization=your-org
      -Dsonar.sources=src
      -Dsonar.tests=tests
      -Dsonar.javascript.lcov.reportPaths=coverage/lcov.info

Deployment Strategies

Blue-Green Deployment

name: Blue-Green Deployment

jobs:
  deploy:
    steps:
      - name: Deploy to Green Environment
        run: |
          # Deploy new version to green environment
          aws ecs update-service --cluster green-cluster --service app-service --force-new-deployment
          
      - name: Run Health Checks
        run: |
          # Wait for green environment to be healthy
          ./scripts/health-check.sh https://green.example.com
          
      - name: Switch Traffic
        run: |
          # Update load balancer to point to green
          aws elbv2 modify-listener --listener-arn $LISTENER_ARN --default-actions Type=forward,TargetGroupArn=$GREEN_TG_ARN
          
      - name: Monitor
        run: |
          # Monitor for 5 minutes
          sleep 300
          ./scripts/check-metrics.sh
          
      - name: Cleanup Old Blue
        if: success()
        run: |
          # Stop old blue environment
          aws ecs update-service --cluster blue-cluster --service app-service --desired-count 0

Canary Deployment

name: Canary Deployment

jobs:
  deploy:
    steps:
      - name: Deploy Canary
        run: |
          # Deploy to 10% of infrastructure
          aws ecs update-service \
            --cluster production \
            --service app-canary \
            --desired-count 1
            
      - name: Monitor Canary
        run: |
          # Monitor metrics for 10 minutes
          ./scripts/monitor-canary.sh
          
      - name: Promote or Rollback
        run: |
          if [ "$CANARY_SUCCESS" = "true" ]; then
            # Full deployment
            aws ecs update-service --cluster production --service app-main --force-new-deployment
          else
            # Rollback
            aws ecs update-service --cluster production --service app-canary --desired-count 0
          fi

Monitoring and Notifications

Slack Notifications

Create .github/workflows/notify.yml:

name: Deployment Notifications

on:
  workflow_run:
    workflows: ["CD Pipeline"]
    types: [completed]

jobs:
  notify:
    runs-on: ubuntu-latest
    steps:
      - name: Send Slack Notification
        uses: 8398a7/action-slack@v3
        with:
          status: ${{ github.event.workflow_run.conclusion }}
          text: |
            Deployment ${{ github.event.workflow_run.conclusion }}!
            Repository: ${{ github.repository }}
            Branch: ${{ github.event.workflow_run.head_branch }}
            Commit: ${{ github.event.workflow_run.head_sha }}
            Author: ${{ github.actor }}
          webhook_url: ${{ secrets.SLACK_WEBHOOK }}
          fields: repo,commit,author,eventName,ref,workflow

### Email Notifications

```yaml
- name: Send Email Notification
  uses: dawidd6/action-send-mail@v3
  with:
    server_address: smtp.gmail.com
    server_port: 465
    username: ${{ secrets.EMAIL_USERNAME }}
    password: ${{ secrets.EMAIL_PASSWORD }}
    subject: Deployment Status - ${{ job.status }}
    to: team@example.com
    from: CI/CD Pipeline
    body: |
      Build job of ${{ github.repository }} completed.
      Status: ${{ job.status }}
      Commit: ${{ github.sha }}
      See: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}

Pipeline Optimization

Caching Dependencies

- name: Cache node modules
  uses: actions/cache@v3
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

- name: Cache Docker layers
  uses: actions/cache@v3
  with:
    path: /tmp/.buildx-cache
    key: ${{ runner.os }}-buildx-${{ github.sha }}
    restore-keys: |
      ${{ runner.os }}-buildx-

Parallel Jobs

jobs:
  tests:
    strategy:
      matrix:
        test-suite: [unit, integration, e2e]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm run test:${{ matrix.test-suite }}

Conditional Execution

- name: Deploy to Production
  if: github.ref == 'refs/heads/main' && github.event_name == 'push'
  run: ./deploy.sh production

- name: Run expensive tests
  if: contains(github.event.head_commit.message, '[full-test]')
  run: npm run test:full

Self-Hosted Runners

Setting Up Self-Hosted Runner

# Download runner
mkdir actions-runner && cd actions-runner
curl -o actions-runner-linux-x64-2.311.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.311.0/actions-runner-linux-x64-2.311.0.tar.gz
tar xzf ./actions-runner-linux-x64-2.311.0.tar.gz

# Configure
./config.sh --url https://github.com/YOUR_ORG/YOUR_REPO --token YOUR_TOKEN

# Run as service
sudo ./svc.sh install
sudo ./svc.sh start

Using Self-Hosted Runner

jobs:
  build:
    runs-on: self-hosted
    steps:
      - uses: actions/checkout@v3
      - run: ./build.sh

Advanced GitHub Actions Features

Reusable Workflows

Create .github/workflows/reusable-deploy.yml:

name: Reusable Deploy Workflow

on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
      image-tag:
        required: true
        type: string
    secrets:
      AWS_ACCESS_KEY_ID:
        required: true
      AWS_SECRET_ACCESS_KEY:
        required: true

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment }}
    steps:
      - name: Configure AWS
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1
      
      - name: Deploy to ECS
        run: |
          echo "Deploying ${{ inputs.image-tag }} to ${{ inputs.environment }}"
          # Deployment logic here

Using the reusable workflow:

jobs:
  deploy-staging:
    uses: ./.github/workflows/reusable-deploy.yml
    with:
      environment: staging
      image-tag: ${{ github.sha }}
    secrets: inherit

Composite Actions

Create .github/actions/node-setup/action.yml:

name: 'Node.js Setup'
description: 'Set up Node.js with caching'
inputs:
  node-version:
    description: 'Node.js version'
    required: false
    default: '18'

runs:
  using: "composite"
  steps:
    - name: Setup Node.js
      uses: actions/setup-node@v3
      with:
        node-version: ${{ inputs.node-version }}
        cache: 'npm'
    
    - name: Install dependencies
      run: npm ci
      shell: bash
    
    - name: Cache build
      uses: actions/cache@v3
      with:
        path: .next/cache
        key: ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}

Security in CI/CD

Dependency Scanning

- name: Run Dependabot Security Updates
  uses: dependabot/fetch-metadata@v1
  with:
    github-token: "${{ secrets.GITHUB_TOKEN }}"

- name: OWASP Dependency Check
  uses: dependency-check/Dependency-Check_Action@main
  with:
    project: 'DevOps Web App'
    path: '.'
    format: 'HTML'

Container Scanning

- name: Run Trivy vulnerability scanner
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: '${{ secrets.DOCKER_USERNAME }}/devops-web-app:${{ github.sha }}'
    format: 'sarif'
    output: 'trivy-results.sarif'

- name: Upload Trivy results to GitHub Security
  uses: github/codeql-action/upload-sarif@v2
  with:
    sarif_file: 'trivy-results.sarif'

Secrets Scanning

- name: TruffleHog OSS
  uses: trufflesecurity/trufflehog@main
  with:
    path: ./
    base: ${{ github.event.repository.default_branch }}
    head: HEAD

Complete CI/CD Example

Full Production Pipeline

Create .github/workflows/production.yml:

name: Production Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

env:
  NODE_VERSION: '18'
  DOCKER_REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  # Quality Gates
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Lint
        run: npm run lint
      
      - name: Type check
        run: npm run type-check
      
      - name: Security audit
        run: npm audit --audit-level=moderate

  # Testing
  test:
    name: Test Suite
    runs-on: ubuntu-latest
    needs: quality
    
    services:
      redis:
        image: redis:alpine
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 6379:6379
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Run unit tests
        run: npm run test:unit
      
      - name: Run integration tests
        env:
          REDIS_URL: redis://localhost:6379
        run: npm run test:integration
      
      - name: Generate coverage
        run: npm run test:coverage
      
      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          file: ./coverage/lcov.info

  # Build and Push
  build:
    name: Build and Push
    runs-on: ubuntu-latest
    needs: [quality, test]
    if: github.event_name == 'push'
    
    permissions:
      contents: read
      packages: write
    
    outputs:
      image-tag: ${{ steps.meta.outputs.tags }}
      image-digest: ${{ steps.build.outputs.digest }}
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      
      - name: Log in to GitHub Container Registry
        uses: docker/login-action@v2
        with:
          registry: ${{ env.DOCKER_REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      
      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: ${{ env.DOCKER_REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=ref,event=branch
            type=sha,prefix={{branch}}-
            type=raw,value=latest,enable={{is_default_branch}}
      
      - name: Build and push Docker image
        id: build
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
          platforms: linux/amd64,linux/arm64

  # Security Scanning
  security:
    name: Security Scanning
    runs-on: ubuntu-latest
    needs: build
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: ${{ needs.build.outputs.image-tag }}
          format: 'sarif'
          output: 'trivy-results.sarif'
      
      - name: Upload Trivy results
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: 'trivy-results.sarif'

  # Deploy to Staging
  deploy-staging:
    name: Deploy to Staging
    runs-on: ubuntu-latest
    needs: [build, security]
    environment:
      name: staging
      url: https://staging.example.com
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Deploy to Staging
        run: |
          echo "Deploying ${{ needs.build.outputs.image-tag }} to staging"
          # Add actual deployment commands here
      
      - name: Smoke Tests
        run: |
          sleep 30
          curl -f https://staging.example.com/health || exit 1

  # Deploy to Production
  deploy-production:
    name: Deploy to Production
    runs-on: ubuntu-latest
    needs: deploy-staging
    environment:
      name: production
      url: https://example.com
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Deploy to Production
        run: |
          echo "Deploying to production"
          # Add actual deployment commands here
      
      - name: Verify Deployment
        run: |
          sleep 30
          curl -f https://example.com/health || exit 1
      
      - name: Notify Success
        if: success()
        uses: 8398a7/action-slack@v3
        with:
          status: success
          text: 'Production deployment successful!'
          webhook_url: ${{ secrets.SLACK_WEBHOOK }}

Monitoring Your Pipeline

Pipeline Metrics

Create a dashboard script scripts/pipeline-metrics.js:

const { Octokit } = require("@octokit/rest");

const octokit = new Octokit({
  auth: process.env.GITHUB_TOKEN,
});

async function getPipelineMetrics() {
  const { data: workflows } = await octokit.actions.listWorkflowRuns({
    owner: 'your-org',
    repo: 'your-repo',
    per_page: 100,
  });

  const metrics = {
    total_runs: workflows.total_count,
    success_rate: 0,
    average_duration: 0,
    failed_runs: [],
  };

  const successful = workflows.workflow_runs.filter(run => run.conclusion === 'success');
  metrics.success_rate = (successful.length / workflows.workflow_runs.length) * 100;

  const durations = workflows.workflow_runs.map(run => {
    const start = new Date(run.created_at);
    const end = new Date(run.updated_at);
    return (end - start) / 1000 / 60; // minutes
  });

  metrics.average_duration = durations.reduce((a, b) => a + b, 0) / durations.length;

  metrics.failed_runs = workflows.workflow_runs
    .filter(run => run.conclusion === 'failure')
    .map(run => ({
      id: run.id,
      branch: run.head_branch,
      commit: run.head_sha.substring(0, 7),
      message: run.head_commit.message,
    }));

  console.log(JSON.stringify(metrics, null, 2));
  return metrics;
}

getPipelineMetrics();

Troubleshooting CI/CD Issues

Common Problems and Solutions

1. Workflow not triggering

# Check your triggers
on:
  push:
    branches: [main]  # Ensure branch name is correct
  pull_request:
    types: [opened, synchronize, reopened]

2. Permissions errors

# Add necessary permissions
permissions:
  contents: read
  packages: write
  issues: write
  pull-requests: write

3. Secrets not available

# Pass secrets explicitly to reusable workflows
jobs:
  call-workflow:
    uses: ./.github/workflows/reusable.yml
    secrets: inherit  # or pass specific secrets

4. Job failing silently

# Add debugging
- name: Debug
  run: |
    echo "Event: ${{ github.event_name }}"
    echo "Ref: ${{ github.ref }}"
    echo "SHA: ${{ github.sha }}"
  env:
    ACTIONS_STEP_DEBUG: true

Best Practices

1. Keep Workflows DRY

• Use reusable workflows

• Create composite actions

• Use workflow templates

2. Optimize for Speed

• Run jobs in parallel when possible

• Use caching effectively

• Minimize Docker image sizes

3. Security First

• Never hardcode secrets

• Use least-privilege permissions

• Scan for vulnerabilities

• Sign commits and images

4. Fail Fast

• Run quick checks first

• Use matrix strategy for parallel testing

• Set timeouts for jobs

5. Monitor and Iterate

• Track pipeline metrics

• Set up alerts for failures

• Continuously improve based on data

Key Takeaways

• CI/CD automates the software delivery process from code to production

• GitHub Actions provides a powerful, integrated CI/CD platform

• Pipelines should include quality gates, testing, security scanning, and staged deployments

• Proper secret management and security scanning are crucial

• Monitor pipeline performance and continuously optimize

• Use deployment strategies like blue-green or canary for safe releases

What's Next?

In Part 5, we'll explore Infrastructure as Code with Terraform. You'll learn:

• Terraform fundamentals

• Writing Terraform configurations

• Managing state

• Creating AWS resources

• Terraform modules and best practices

Additional Resources

• GitHub Actions Documentation

• GitHub Actions Marketplace

• CI/CD Best Practices

• The DevOps Handbook

• GitHub Actions Examples

---

Ready to manage infrastructure as code? Continue with Part 5: Infrastructure as Code with Terraform!