DevOps Engineer's Guide

CI/CD integration, cloud setup, and operational deployment

Welcome DevOps Engineers!

Target Audience: DevOps Engineers, SRE, Infrastructure Engineers

Reading Time: 75 minutes | Goal: Deploy and integrate framework in CI/CD pipeline

This guide gets you integrating the framework into your CI/CD pipeline FAST:

✅ Maven lifecycle and phases
✅ CI/CD integration patterns
✅ Google Cloud setup
✅ Configuration management
✅ Monitoring and reporting

Quick CI/CD Setup

GitHub Actions Example

# .github/workflows/test.yml
name: Automated Tests

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]
  schedule:
    - cron: '0 9 * * *'  # Daily at 9 AM

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up JDK 23
        uses: actions/setup-java@v4
        with:
          java-version: '23'
          distribution: 'temurin'
      
      - name: Cache Maven packages
        uses: actions/cache@v4
        with:
          path: ~/.m2
          key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
      
      - name: Run tests
        run: mvn clean verify -Dheadless=true
      
      - name: Upload Allure Results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: allure-results
          path: target/allure-results

Jenkins Pipeline Example

pipeline {
    agent any
    
    tools {
        maven 'Maven 3.9'
        jdk 'JDK 23'
    }
    
    stages {
        stage('Checkout') {
            steps {
                git branch: 'main', url: 'https://github.com/your-org/automation-framework.git'
            }
        }
        
        stage('Build') {
            steps {
                sh 'mvn clean compile'
            }
        }
        
        stage('Test') {
            steps {
                sh 'mvn verify -Dheadless=true'
            }
        }
        
        stage('Report') {
            steps {
                allure includeProperties: false, 
                       results: [[path: 'target/allure-results']]
            }
        }
    }
    
    post {
        always {
            junit 'target/surefire-reports/*.xml'
            archiveArtifacts artifacts: 'target/screenshots/**/*.png', 
                             allowEmptyArchive: true
        }
    }
}

GitLab CI Example

image: maven:3.9-eclipse-temurin-23

stages:
  - test
  - report

variables:
  MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"

cache:
  paths:
    - .m2/repository

test:
  stage: test
  script:
    - mvn clean verify -Dheadless=true
  artifacts:
    when: always
    reports:
      junit: target/surefire-reports/TEST-*.xml
    paths:
      - target/allure-results
      - target/screenshots
    expire_in: 1 week

report:
  stage: report
  image: allure/allure-cli:latest
  script:
    - allure generate target/allure-results -o allure-report
  artifacts:
    paths:
      - allure-report
  only:
    - main

Maven Lifecycle

Key Maven phases ↗ for test execution:

Phase	Purpose	Command
`validate`	Validate project structure	`mvn validate`
`compile`	Compile source code	`mvn compile`
`test`	Unit tests (Surefire)	`mvn test`
`integration-test`	Integration tests	`mvn integration-test`
`post-integration-test`	BigQuery upload, Screenshots	(automatic)
`verify`	Full test + validation	`mvn verify`

Complete Lifecycle Flow

mvn verify
  ↓
1. validate      - Validate project structure
  ↓
2. compile       - Compile source code
  ↓
3. test-compile  - Compile test code
  ↓
4. test          - Run unit tests (Surefire plugin)
  ↓
5. integration-test - Run integration tests
  ↓
6. post-integration-test - CLOUD UPLOADS HERE ✅
  ↓  ├─ BigQuery data export
  ↓  ├─ Cloud Storage screenshot upload
  ↓  └─ Allure report generation
  ↓
7. verify        - Verify results

Common Commands

# Clean and run all tests
mvn clean verify

# Skip tests (build only)
mvn clean install -DskipTests

# Run specific suite
mvn test -Dtest=TS_Smoke_JUnit

# Run by tag
mvn test -Dgroups="smoke"

# Run headless
mvn verify -Dbrowser.headless=true

# Run with parallel execution
mvn verify -DparallelExecution=true -DparallelExecutionThreads=4

Configuration Management

Environment-Specific Configs

# testConfig.properties (base)
browser.type=chrome
browser.headless=false
base.url=https://www.saucedemo.com

# Override via command line
mvn test -Dbrowser.headless=true -Dbase.url=https://staging.example.com

# Or via environment variables
export BROWSER_HEADLESS=true
export BASE_URL=https://staging.example.com

Key Configuration Properties

Property	Description	Default
`browser.type`	Browser type	chrome
`browser.headless`	Headless mode	false
`base.url`	Application URL	-
`implicit.wait`	Wait timeout (sec)	10
`screenshot.on.fail`	Auto screenshot	true
`parallelExecution`	Enable parallel	false
`parallelExecutionThreads`	Thread count	1

Google Cloud Setup

Prerequisites

Google Cloud project with billing enabled
BigQuery API enabled
Cloud Storage API enabled (for screenshots)
Service account with appropriate permissions
Service account JSON key file

Service Account Permissions

# BigQuery Permissions
bigquery.datasets.create
bigquery.tables.create
bigquery.tables.updateData
bigquery.jobs.create

# Cloud Storage Permissions
storage.objects.create
storage.objects.get
storage.buckets.get

Configuration

# testConfig.properties
############### Google Cloud Configuration ###############
googleCloudProjectId=your-project-id
googleCloudServiceAccountJsonFile=src/test/resources/.private/credentials.json

# BigQuery
googleCloudBigQueryExport=true
googleCloudBigQueryTimeout=10
googleCloudBigQueryMaxAttempts=10
googleCloudProjectDataset=dataset_automation_01
googleCloudProjectDatasetTableTestSteps=table_teststeps_02
googleCloudProjectDatasetTableTestStepsSchema=src/test/java/framework/data/FW_GoogleCloudBigQueryTableTestStepsSchema.json

# Cloud Storage
googleCloudStorageExport=true
googleCloudStorageBaseURL=https://storage.googleapis.com/
googleCloudStorageBucket=your-bucket-name

Creating Service Account

# Create service account
gcloud iam service-accounts create automation-framework \
    --display-name="Automation Framework"

# Grant BigQuery permissions
gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="serviceAccount:automation-framework@PROJECT_ID.iam.gserviceaccount.com" \
    --role="roles/bigquery.admin"

# Grant Storage permissions
gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="serviceAccount:automation-framework@PROJECT_ID.iam.gserviceaccount.com" \
    --role="roles/storage.objectCreator"

# Create and download key
gcloud iam service-accounts keys create credentials.json \
    --iam-account=automation-framework@PROJECT_ID.iam.gserviceaccount.com

CI/CD Integration Examples

GitHub Actions with Matrix

name: Cross-Browser Tests

on:
  push:
    branches: [ main ]
  schedule:
    - cron: '0 2 * * *'  # Nightly at 2 AM

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        browser: [chrome, firefox]
        environment: [staging, production]
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up JDK 23
        uses: actions/setup-java@v4
        with:
          java-version: '23'
          distribution: 'temurin'
      
      - name: Setup Google Cloud
        env:
          GCP_CREDENTIALS: ${{ secrets.GCP_CREDENTIALS }}
        run: |
          mkdir -p src/test/resources/.private
          echo "$GCP_CREDENTIALS" > src/test/resources/.private/credentials.json
      
      - name: Run Tests
        run: |
          mvn verify \
            -Dheadless=true \
            -Dbrowser=${{ matrix.browser }} \
            -Denvironment=${{ matrix.environment }}
      
      - name: Upload Results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: results-${{ matrix.browser }}-${{ matrix.environment }}
          path: |
            target/allure-results
            target/surefire-reports

Jenkins with Parallel Stages

pipeline {
    agent any
    
    environment {
        GOOGLE_APPLICATION_CREDENTIALS = credentials('gcp-service-account')
    }
    
    stages {
        stage('Parallel Tests') {
            parallel {
                stage('Chrome') {
                    steps {
                        sh 'mvn verify -Dbrowser=chrome -Dheadless=true'
                    }
                }
                stage('Firefox') {
                    steps {
                        sh 'mvn verify -Dbrowser=firefox -Dheadless=true'
                    }
                }
            }
        }
    }
    
    post {
        always {
            junit 'target/surefire-reports/*.xml'
            archiveArtifacts artifacts: 'target/screenshots/**/*.png', allowEmptyArchive: true
            cleanWs()
        }
        failure {
            emailext (
                subject: "Failed: ${env.JOB_NAME} - Build #${env.BUILD_NUMBER}",
                body: "Check console output at ${env.BUILD_URL}",
                to: 'team@example.com'
            )
        }
    }
}

Report Generation

# Generate and serve Allure report
allure serve target/allure-results

# Generate static report
allure generate target/allure-results -o target/allure-report --clean

# HTTP audit reports location
target/http-reports/
├── diagnostic/
│   └── *.html, *.md
└── summary/
    └── *.md

Report Locations

Report Type	Location
Allure Results	`target/allure-results/`
Allure Report	`target/allure-report/`
JUnit Reports	`target/surefire-reports/`
Screenshots	`target/screenshots/`
Test Data CSV	`target/data/TestStepsData.csv`
HTTP Reports	`target/http-reports/`

Troubleshooting

ChromeDriver version mismatch

Fix: WebDriverManager auto-downloads correct version. Ensure network access to download.

Headless mode fails

Fix: Install Chrome/Chromium on CI server. Add --no-sandbox for Docker.

BigQuery upload fails

Fix: Verify service account permissions and credentials path.

Out of memory

Fix: Increase Maven heap: MAVEN_OPTS="-Xmx2g"

Tests timeout in CI

Fix: Increase timeouts in testConfig.properties, use headless mode, check CI resources.

CSV file does not exist

Fix: Verify dataOutputPath in testConfig.properties.

Configuration Reference

Complete testConfig.properties Template

############### Browser Configuration ###############
browser=chrome
headless=false
chromeDriverPath=
firefoxDriverPath=
edgeDriverPath=

############### Test Execution ###############
parallelExecution=false
parallelExecutionThreads=1
implicitWait=10
pageLoadTimeout=30
scriptTimeout=30

############### Screenshot Configuration ###############
takeScreenshots=true
screenCaptureFormat=png
screenCaptureQuality=85
screenCapturePath=target/screenshots/
screenCaptureArchivePath=target/screenshots/archive/

############### Data Collection ###############
dataOutputPath=target/data/
dataTestStepsFilename=TestStepsData.csv
dataArchivePath=target/data/archive/

############### Google Cloud Configuration ###############
googleCloudProjectId=your-project-id
googleCloudServiceAccountJsonFile=src/test/resources/.private/credentials.json

# BigQuery
googleCloudBigQueryExport=true
googleCloudBigQueryTimeout=10
googleCloudBigQueryMaxAttempts=10
googleCloudProjectDataset=dataset_automation_01
googleCloudProjectDatasetTableTestSteps=table_teststeps_02

# Cloud Storage
googleCloudStorageExport=true
googleCloudStorageBaseURL=https://storage.googleapis.com/
googleCloudStorageBucket=your-bucket-name

############### mAi Advisor ###############
maiAdvisorEnabled=true
maiAdvisorAudiences=Developer,QA,DevOps,Security,PM,Executive

############### Network Configuration ###############
httpTimeout=30000
httpFollowRedirects=true
httpUserAgent=mAi-Automation-Framework/2.0

Security Best Practices

Credentials Management

✅ Store credentials in CI/CD secrets, not in code
✅ Use service accounts with minimal permissions
✅ Rotate credentials regularly
✅ Never commit credentials to version control
✅ Use environment variables for sensitive config

.gitignore Configuration

# Add to .gitignore
src/test/resources/.private/
*.json
testConfig.prod.properties
credentials.json

Key Rotation

# Rotate service account key
gcloud iam service-accounts keys create new-key.json \
    --iam-account=automation-framework@PROJECT.iam.gserviceaccount.com

# Delete old key
gcloud iam service-accounts keys delete OLD_KEY_ID \
    --iam-account=automation-framework@PROJECT.iam.gserviceaccount.com

IAM Best Practices

Least Privilege: Grant minimum required permissions
Service Accounts: Use service accounts (not user accounts)
Separate Environments: Different service accounts per environment
Audit: Enable Cloud Audit Logs
Monitoring: Set up alerts for unusual activity

Docker Support

# Dockerfile
FROM maven:3.9-eclipse-temurin-23

# Install Chrome
RUN apt-get update && apt-get install -y \
    chromium chromium-driver \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY . .

CMD ["mvn", "verify", "-Dbrowser.headless=true"]

Docker Compose

version: '3.8'
services:
  tests:
    build: .
    environment:
      - BROWSER=chrome
      - HEADLESS=true
    volumes:
      - ./target:/app/target
      - ./src/test/resources/.private:/app/src/test/resources/.private:ro

Quick Reference Card

Print this section!

CI/CD Commands

mvn clean verify -Dheadless=true          # Run all tests
mvn verify -P<profile>                     # Use profile
mvn verify -Dtest=TS_Smoke_JUnit          # Run smoke tests
allure generate target/allure-results      # Generate report

Cloud Operations

# BigQuery
bq query "SELECT * FROM dataset.table LIMIT 10"
bq ls dataset_automation_01

# Cloud Storage
gsutil ls gs://bucket-name/
gsutil cp file.png gs://bucket-name/

Environment Variables

export BASE_URL="https://staging.example.com"
export BROWSER="chrome"
export GCP_PROJECT_ID="project-id"
export HEADLESS="true"

Document	Description
BigQuery Upload Process	Cloud upload pipeline
Maven Lifecycle Flow	Test execution flow
Architecture Overview	Framework layers
Complete Reading Guide	Full documentation path
Test Writer's Guide	Writing tests
Framework Maintainer Guide	Extending framework