API reference

Evaluation endpoint#

1POST /v1/systemone
2Authorization: Bearer <provider API key>
3Content-Type: application/json

Send a JSON request to your model provider’s evaluation endpoint. Draft 0.1 uses the /v1/systemone path. Configure the full URL in DMP_ENDPOINT and keep credentials in your server environment.

The provider supplies the host, credentials, and model identifiers. A successful evaluation returns HTTP 200 with a JSON response. The DMP documentation site does not serve model requests.

Request fields#

Each question shares the request’s state. An answer uses the same name as its question. Questions cannot depend on other answers in the same request.

Draft 0.1 request and question objects accept only the fields listed here. Objects and arrays inside state, instructions, or a criterion may contain nested JSON values. Providers set their own input and question-count limits.

Choice#

Set type to choice. Criteria is a nonempty object: each key names an option, and its value describes that option. A description may be a string, object, array, or null.

1{
2  "type": "choice",
3  "choice": "operations",
4  "probabilities": {
5    "operations": 0.92,
6    "accounts": 0.05,
7    "other": 0.03
8  },
9  "confidence": 0.88
10}

Score#

Set type to score. Criteria is an ordered array of at least two level descriptions. Each description may be a string, object, or array. Level indices start at zero.

1{
2  "type": "score",
3  "score": 1.75,
4  "legend": {
5    "0": "No impact",
6    "1": "Delayed work",
7    "2": "Work blocked"
8  },
9  "probabilities": {
10    "0": 0.05,
11    "1": 0.15,
12    "2": 0.8
13  },
14  "confidence": 0.625
15}

For N levels, score lies between 0 and N − 1, subject to the rounding tolerance. A value between indices represents the expected index; clients should retain the distribution when assessing uncertainty.

Noul#

Set type to noul. Optional criteria is an object with true and false keys; either key may be omitted. Each value may be a string, object, or array describing that outcome.

1{
2  "type": "noul",
3  "noul": 0.96
4}

Noul has no separate confidence field. Choose an application threshold using evaluation data.

Response fields#

Choice and score distributions must sum to one within 0.0001. Probabilities and confidence must be finite. Confidence calculations remain provider-defined in Draft 0.1, so test thresholds for each model you use.

Clients should accept unknown response fields, including metadata such as latency. Providers may echo a model alias; request a pinned identifier when reproducibility matters. Token accounting follows the provider’s definition.

Request and response examples#

Select a question type to inspect a complete request and response. Replace example-model with a model identifier your provider accepts. Response values below are illustrative.

1{
2  "model": "example-model",
3  "state": "The nightly export stopped at 02:14. New reports are missing.",
4  "questions": {
5    "result": {
6      "type": "choice",
7      "instructions": "Which team should investigate?",
8      "criteria": {
9        "operations": "Failed jobs, missing data, or service outages",
10        "accounts": "Access, invitations, or account settings",
11        "other": "Requests outside these categories"
12      }
13    }
14  }
15}

Handle errors#

The table defines the draft’s client error-handling policy. Providers may use 503 or 529 to report overload. Error body formats remain provider-specific; the response schema covers successful evaluations.

Set a request timeout and a retry budget. Retried evaluations can incur another charge; this draft does not define an idempotency mechanism.

Validation schemas#

Use the DMP JSON Schemas to validate field types and required properties. Check answer names against request names, probability sums, the selected choice, and score arithmetic in application code.