Contract Testing

Contract Testing

Contract testing ensures that both the Producers (the testing-api implementations) and the Consumers (services relying on these APIs) strictly adhere to the defined interface specification. This prevents integration failures within the Supervised AI platform by validating that the data exchange format, status codes, and payload structures remain consistent.

The Source of Truth

The testing-api serves as the formal specification for all testing-related interactions. Contract testing focuses on two primary perspectives:

1. Consumer Expectations: Services that call the testing API define what they expect the API to return.
2. Provider Compliance: The API implementation guarantees it can fulfill those expectations without breaking the interface.

Implementing Contract Tests

To maintain the integrity of the Supervised AI ecosystem, every integration must pass contract verification. This typically involves validating JSON schemas against the models defined in this repository.

1. Defining the Contract

Contracts should be derived from the core types provided by the testing-api. When a consumer makes a request, the provider must return a response that matches the expected schema.

// Example: Contract definition for a Test Execution endpoint
const testExecutionContract = {
  request: {
    method: 'POST',
    path: '/v1/execute',
    body: {
      testId: 'string',
      parameters: 'object'
    }
  },
  response: {
    status: 200,
    body: {
      executionId: 'string',
      status: 'queued' | 'running' | 'completed',
      timestamp: 'ISO-8601 string'
    }
  }
};

2. Provider Verification (Producer)

The provider service (the one implementing the testing-api) must run verification scripts to ensure that its current codebase honors all active contracts.

# Example: Running contract verification against a local provider instance
npm run test:contract:verify -- --provider-base-url=http://localhost:8080

3. Consumer Verification (Mocking)

Consumers should use the contract to generate mock servers during their own unit and integration tests. This ensures that the consumer code is written against a valid representation of the testing-api.

// Example: Using a contract-generated mock in a consumer test
const apiResponse = await testingClient.executeTest({
  testId: "diag-001",
  parameters: { threshold: 0.5 }
});

expect(apiResponse.status).toBe("queued");

Schema Validation Rules

The following rules are enforced during contract testing to ensure backward compatibility:

| Rule | Description | | :--- | :--- | | Field Additions | New optional fields can be added to the response without breaking the contract. | | Field Removals | Removing fields from a response is a breaking change and requires a version bump. | | Type Integrity | Data types (e.g., changing a string to an array) must not change for existing fields. | | Required vs. Optional | Changing a field from optional to required in a request is a breaking change for consumers. |

Continuous Integration

Contract tests are executed automatically in the CI/CD pipeline:

• On Pull Request: Ensures that new changes do not violate existing contracts.
• On Deployment: Verifies that the deployed version of the service is compatible with the latest version of the testing-api specification.