CI/CD Integration

CI/CD Integration

Automating structural API testing ensures that every deployment to the Supervised AI platform maintains schema integrity and prevents breaking changes in the communication layer. This section outlines how to integrate the testing-api suite into your existing delivery pipelines.

Overview

The testing-api is designed to be executed as a standalone validation step within your CI/CD workflow. It evaluates the structural response of your endpoints against the Supervised AI platform requirements, returning a non-zero exit code if validation fails.

Authentication and Environment Setup

To run tests in a remote environment, you must provide the necessary credentials via environment variables. These variables allow the testing suite to authenticate with the Supervised AI gateway and target the correct environment.

| Variable | Description | Required | | :--- | :--- | :--- | | SUPERVISED_AI_API_KEY | Your platform API key for authentication. | Yes | | TARGET_ENV | The environment to test against (e.g., staging, production). | Yes | | REPORT_OUTPUT_DIR | The directory where test results (JUnit/JSON) will be saved. | No |

Integration Examples

GitHub Actions

To integrate testing-api into a GitHub Actions workflow, add a step after your service has been deployed to a staging or preview environment.

jobs:
  api-validation:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Code
        uses: actions/checkout@v3

      - name: Install Testing API Suite
        run: npm install # Or the relevant installation command for your environment

      - name: Run Structural API Tests
        env:
          SUPERVISED_AI_API_KEY: ${{ secrets.SUPERVISED_AI_API_KEY }}
          TARGET_ENV: staging
        run: |
          npm run test:structural -- --endpoint https://api.staging.supervised.ai --schema ./schemas/main.json

GitLab CI/CD

Add a validation stage to your .gitlab-ci.yml file:

stages:
  - deploy
  - validate

verify_api_structure:
  stage: validate
  image: node:18-alpine
  script:
    - npm install
    - npm run test:structural -- --endpoint $STAGING_URL
  only:
    - merge_requests
    - main
  variables:
    SUPERVISED_AI_API_KEY: $CI_SUPERVISED_AI_KEY

CLI Interface & Arguments

The testing suite provides a CLI for manual execution and CI scripting. The primary entry point is the structural validation command.

Usage: testing-api validate [options]

Options:

• --endpoint <url>: The base URL of the API to be tested.
• --schema <path>: Path to the local JSON schema file or platform schema ID.
• --timeout <ms>: Set the timeout for API responses (default: 5000).
• --strict: If enabled, validation will fail if the API returns additional properties not defined in the schema.

Handling Test Failures

When integrated into a pipeline:

1. Exit Codes: The process will exit with 0 on success and 1 on structural mismatches or connection errors.
2. Reports: Detailed failure logs, including the specific path of the schema violation (e.g., body.data.user_id: expected string, received number), will be printed to stdout.
3. Artifacts: If REPORT_OUTPUT_DIR is defined, a report.json file is generated, which can be uploaded as a CI artifact for further debugging.