Playground

The playground is where prompt iteration meets governance. Every invocation is recorded as evidence, and dataset replays run through the same durable eval pipeline as any other run.

Invoke

Run a registry version (pick prompt + version; variables are auto-suggested from the template) or a draft template you edit inline. The result panel shows the completion, token counts, latency, the serving endpoint, and manifest cost. Missing variables render as empty strings and are flagged. The invocation proceeds so you can see exactly what the model received.

Every invocation is recorded. A playground.invoked audit event carries the prompt identity (registry id@version or draft content hash), endpoint, tokens, latency, and cost, and when tracing is enabled for your organization, the invocation also appears as an LLM span in the trace views. Playground traffic is never invisible.

Replay against a dataset

Replay renders a prompt version over every example of a frozen dataset version, completes each through the model, and judges the outputs with the evaluators you select, asynchronously. Submitting a replay returns a pending eval run immediately and takes you to its live run page; no model calls happen inside the request.

Two properties worth knowing:

• One budget. Generation tokens count against the run's maxJudgeTokens alongside judging, under the same hard reservation accounting, a replay can't silently outspend its budget, and cost_usd covers generation plus judging.
• Same evidence. A replay run writes Quality Center evidence exactly like any other eval run. Replaying a candidate prompt against your regression dataset produces a gate-consultable verdict, which makes "new prompt version + replay + compare runs" a reviewable, governed release process for prompts.

Cancellation works between phases. Cancel during generation and no judging starts; partial scores are kept either way.