Specification
Draft 0.1 defines an evaluation exchange for decision models.
Status and scope
The draft covers JSON requests, typed answers, and a synchronous HTTP binding. Training methods, model architecture, billing, and deployment are outside its scope. MUST and SHOULD identify proposed requirements and recommendations.
Request requirements
- A request MUST contain model, state, and a nonempty questions object.
- Each question MUST declare choice, score, or noul and include instructions.
- Choice criteria MUST contain at least one named option. Score criteria MUST contain at least two ordered descriptions.
- A provider MUST reject requests that exceed its supported limits and SHOULD document those limits.
Response requirements
- A successful response MUST include model and exactly one answer for each request question. Question names and types MUST match.
- Choice probabilities MUST use the criteria keys. The chosen key MUST have a maximal probability.
- Score probabilities and legend MUST use the zero-based criteria indices as string keys. Score MUST equal the probability-weighted index within 0.0001.
- Each probability and confidence MUST be finite and within [0, 1]. A distribution MUST sum to one within 0.0001.
- A noul answer MUST contain a noul value in [0, 1].
- Providers SHOULD document confidence calculations. Clients SHOULD ignore unknown response fields.
Schemas
The JSON Schemas check document structure, required fields, and value ranges. Cross-document rules, probability sums, argmax, and score arithmetic require additional checks.
Open questions
Further work is needed on a shared confidence calculation, version negotiation, capability discovery, a common error envelope, and a conformance suite. Draft 0.1 defines no model registry or certification process.
Implementation reference
Use the DMP API reference for field definitions and examples. The probability guide explains how to interpret distributions and choose application thresholds.