Mocking Strategy

Mocking Strategy

Mocking is a critical component of testing AI-integrated systems. Since interacting with production AI models can be expensive, slow, and non-deterministic, the testing-api provides a structured way to simulate model behaviors.

By adhering to the following strategies, you can ensure your tests are fast, reliable, and cover a wide range of edge cases without making actual upstream calls.

Core Mocking Principles

When simulating the Supervised AI platform, your mocks should prioritize:

• Deterministic Outputs: For a given input, the mock should return a consistent output unless testing for variability.
• Latency Simulation: Use configurable delays to test how your application handles model response times.
• Edge Case Coverage: Explicitly mock error states like token limit breaches, safety filters, and timeouts.

Implementing Basic Completions

The testing-api expects a specific structure for completion responses. To mock a standard text generation or classification task, mirror the response schema defined in the platform's interface.

import { CompletionResponse } from '@supervised-ai/testing-api';

const mockCompletion = (prompt: string): CompletionResponse => {
  return {
    id: "mock-task-123",
    object: "text_completion",
    created: Date.now(),
    model: "test-model-v1",
    choices: [
      {
        text: "This is a simulated AI response.",
        index: 0,
        finish_reason: "stop",
      }
    ],
    usage: {
      prompt_tokens: 5,
      completion_tokens: 7,
      total_tokens: 12
    }
  };
};

Simulating Streaming Behaviors

AI models often stream tokens incrementally. To test your UI's streaming capabilities, your mock should return an iterable or an event stream that emits partial chunks using the Chunk interface.

// Example: Mocking a token-by-token stream
const mockStream = async function* (content) {
  const words = content.split(" ");
  for (const word of words) {
    yield {
      delta: { content: word + " " },
      finish_reason: null
    };
    await new Promise(resolve => setTimeout(resolve, 50)); // Simulate network latency
  }
  yield { delta: {}, finish_reason: "stop" };
};

Error Simulation and Status Codes

Testing how your application handles failures is as important as testing the "happy path." Use the testing-api error structures to simulate common AI platform issues.

| Scenario | Simulated Status Code | Mocked Error Payload | | :--- | :--- | :--- | | Token Limit | 400 | { error: "context_length_exceeded", message: "..." } | | Rate Limiting | 429 | { error: "rate_limit_reached", retry_after: 30 } | | Safety Filter | 400 | { error: "content_filter", category: "violence" } | | Server Error | 500 | { error: "internal_model_error" } |

Best Practices for AI Mocking

1. Use "Golden" Datasets

Maintain a set of static JSON files representing "Golden" responses (the ideal output for a specific prompt). Use these files to hydrate your mocks to ensure your integration logic remains valid as your codebase evolves.

2. Introduce Artificial Jitter

To test the robustness of your async handling, don't just use a fixed delay. Introduce random jitter to your mock response times:

const delay = Math.floor(Math.random() * (500 - 100 + 1) + 100);
await new Promise(r => setTimeout(r, delay));

3. Validate Request Schemas

Before returning a mock response, validate that the incoming request matches the Request interface provided by the testing-api. This ensures your application is sending the correct parameters (e.g., temperature, max_tokens) to the platform.