Data Types & Validation

Core Data Types

The Testing API utilizes specific data structures to ensure consistency and reliability across the Supervised AI platform. All request and response bodies are formatted as JSON.

Primitive Types and Formats

| Type | Format | Description | | :--- | :--- | :--- | | ID | UUID v4 | Unique identifiers for models, datasets, and test runs. | | Timestamp | ISO 8601 | Dates and times in UTC (e.g., 2023-10-27T10:30:00Z). | | Status | Enum | Categorical strings (e.g., PENDING, RUNNING, COMPLETED, FAILED). | | MetricValue | Float | Quantitative results of evaluation metrics. |

Object Schemas

TestCase

Represents a single unit of evaluation within a test suite.

{
  "id": "tc_9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d",
  "input_data": {
    "prompt": "What is the capital of France?",
    "context": "General knowledge"
  },
  "expected_output": "Paris",
  "metadata": {
    "category": "geography",
    "difficulty": "easy"
  }
}

EvaluationResult

The output generated after running a model against a specific TestCase.

• score (float): A normalized value between 0.0 and 1.0.
• passed (boolean): Result based on the defined success threshold.
• latency (int): Time taken in milliseconds.

Validation Logic

The API performs strict schema validation on all incoming requests. To ensure successful integration, your data must adhere to the following constraints:

String Constraints

• Names/Labels: Minimum 3 characters, maximum 128 characters.
• Descriptions: Maximum 1024 characters.
• IDs: Must be valid UUID strings.

Numerical Constraints

• Confidence Thresholds: Must be a float between 0.0 and 1.0.
• Batch Sizes: Integer between 1 and 500.

Validation Errors

If a request fails validation, the API returns a 422 Unprocessable Entity response. The response body provides a detailed map of the validation failures.

Example Validation Failure:

{
  "error": "Validation Failed",
  "details": [
    {
      "field": "confidence_threshold",
      "message": "Value must be less than or equal to 1.0",
      "received": 1.2
    },
    {
      "field": "test_suite_id",
      "message": "Invalid UUID format",
      "received": "invalid-id-123"
    }
  ]
}

Input/Output Type Mapping

When interacting with the SDK or the REST endpoints, the following type mappings apply:

| API Field | Type | Required | Default | | :--- | :--- | :--- | :--- | | model_version | string | Yes | N/A | | metrics | Array<string> | No | ["accuracy"] | | tags | Map<string, string> | No | {} | | timeout | integer (ms) | No | 30000 |

Internal Types

While the API handles these internally, users may see these referenced in advanced configuration:

• ValidationSchema: An internal representation of the expected JSON structure for a specific model input.
• TransformPipeline: Internal logic that maps raw data to model-specific features. These are not directly modifiable via the public API but influence the validation errors returned.