Connect an AI agent endpoint or your own model credential, define the contract you want tested, optionally seal the test set, and run an evidence-preserving evaluation whenever you need one.
A passing result means the properties exercised by this evaluation passed. It does not prove properties the evaluation could not observe.
A private browser session owns your Judge data. Clearing browser storage or using another browser creates a separate workspace.
BYOA calls your HTTPS agent endpoint using a simple JSON protocol. BYOK currently supports direct OpenAI model evaluation.
Each case needs a prompt and an expected output. This first self-serve surface uses deterministic scoring; uncertainty is not silently converted into a pass.
The run is bound to the exact system contract, test-set identity, evaluator identity where applicable, and evidence bundle.
The JUDGE turns “I tried my agent and it seemed to work” into a repeatable evaluation tied to a specific system, a defined set of requirements, and preserved evidence.
Did the system produce the result your test required? Each case receives PASS, FAIL, or NOT VERIFIED instead of hiding uncertainty inside a single score.
The run is bound to the system contract, test-set identity, and evaluator identity where applicable. A materially different configuration is a different evaluation identity.
The JUDGE preserves run evidence and produces a cryptographic fingerprint for the evidence bundle so later changes can be detected by its integrity check.
It means the evidence bundle passes The JUDGE's tamper-evidence check. The current integrity model uses Judge-controlled symmetric HMAC. It is not hardware attestation, independent third-party attestation, or proof of properties The JUDGE could not observe.
Sealing creates an immutable commitment to the evaluation material used for that run. This helps prove the committed test set was not silently rewritten afterward. The person who authored a self-serve test still knows its contents; sealing is not the same as double-blind evaluation.
A system should not receive a fabricated failure or success when the evaluator cannot establish a verdict. The JUDGE keeps uncertainty visible instead of pretending every result is certain.
Manual spot checks are hard to reproduce and easy to remember selectively. The JUDGE gives you a defined contract, repeatable cases, case-level verdicts, a bound run identity, and preserved evidence.
Choose 5–20 things your agent absolutely must do correctly in production. Make each requirement observable and specific. For example: return a required schema, refuse a forbidden action, preserve an exact field, choose the correct tool result, or produce a known deterministic answer. Run the same contract again after changing the model, prompt, tools, retrieval, or orchestration.
The JUDGE is free to use. If it saved you time or helped you verify something important, you can help cover the cost of keeping it online.
Support The JUDGE →If your production system depends on third-party AI models, TAB independently measures what they are actually delivering and what changes.
See TAB →The JUDGE stores provider credentials encrypted at rest and does not expose them in evidence. BYOA endpoints must be public HTTPS endpoints and redirects are refused in the public Judge service.