Introduction to testing-api

Overview

The testing-api serves as the standardized interface for integrating automated testing workflows within the Supervised AI ecosystem. It acts as the bridge between your development environment and the Supervised AI platform, ensuring that model evaluations, performance metrics, and validation suites are consistently reported and tracked.

By providing a unified structure for test data, the API enables teams to monitor model regressions, validate data quality, and maintain high standards for AI reliability throughout the development lifecycle.

Core Role in the Ecosystem

Within the Supervised AI platform, the testing-api is responsible for:

• Result Standardization: Converting local test outputs into a format compatible with the platform's dashboard.
• Performance Tracking: Logging accuracy, precision, recall, and custom AI metrics for specific model versions.
• Dataset Validation: Ensuring that the data used for testing meets the required schemas and quality constraints of the platform.

Getting Started

To use the testing API, you must first authenticate your environment with your platform credentials. The API typically operates as a client-side wrapper around the Supervised AI backend services.

Basic Initialization

import { TestingAPI } from 'supervised-ai-testing';

const client = new TestingAPI({
  apiKey: process.env.SUPERVISED_AI_API_KEY,
  projectId: 'your-project-id',
  environment: 'staging'
});

Key API Interfaces

The testing-api exposes several public methods to manage the testing lifecycle.

1. Test Session Management

Every group of tests must be wrapped in a "Session" to aggregate results in the platform UI.

client.startSession(config)

• Input: config: { sessionName: string, modelTag: string }
• Returns: Promise<SessionID>

2. Reporting Results

Submit individual test case results or bulk evaluation metrics.

client.reportResult(data) | Parameter | Type | Description | | :--- | :--- | :--- | | testName | string | The unique identifier for the test case. | | status | Enum | PASSED, FAILED, or ERROR. | | metrics | Object | Key-value pairs of performance data (e.g., { "confidence": 0.98 }). | | metadata | Object | Optional context such as input payload or timestamps. |

Example Usage:

await client.reportResult({
  testName: "Entity Extraction Accuracy",
  status: "PASSED",
  metrics: {
    f1_score: 0.92,
    latency_ms: 150
  },
  metadata: {
    modelVersion: "v2.4-beta"
  }
});

3. Finalizing the Suite

Once all tests are completed, the session must be closed to trigger platform-side analytics and notifications.

client.completeSession(sessionId)

• Input: sessionId: string
• Description: Marks the session as finished and calculates the final pass/fail aggregates for the Supervised AI dashboard.

Configuration Options

While most internal logic is handled automatically, you can configure the following public variables during client instantiation:

• timeout: (Number) Maximum time in milliseconds to wait for a response from the platform.
• retries: (Number) Number of times to attempt re-sending test results in case of network failure.
• verbose: (Boolean) When enabled, logs detailed transmission status to the local console.