Evals: test AI models under a policy, in your browser
Playgrounds and evals for AI model providers. Every run happens under an Agent Behaviour Policy (ABP): what the model is told, which tools it may call, how tool results are treated, and the checks its answer must pass. Everything runs in your browser; your key never reaches us.
The family
Shared core v0.1
Versioned ES modules: provider adapters, the ABP engine, the agent loop, the eval runner, tools over open data, a cost meter, key helpers and UI pieces, with in-browser tests.
Policy library
Agent Behaviour Policies as plain JSON. The first entries are the policies of the team that builds these sites.
Try the core
Run a policy against a recorded model reply with no key, or against a free OpenRouter model with your own key.
OpenRouter evals
The first provider site, on core v0.1: sign-in, model browser with free models first, chat, tools, side by side and evals. Planned for round 2; not live yet.
How a run works
- Pick a policy and a model. The policy becomes the system prompt and filters the tools the model can see.
- The agent loop calls the model. Each tool call is checked against the policy before it runs.
- Tool results are wrapped as untrusted data. Text in them that reads like an instruction is flagged, and the model is told not to follow it.
- The answer is checked against the policy: patterns, length, no secrets, which tools were used. The run records tokens, cost and time.
Your key
Keys stay in your browser: in memory by default. If you choose to remember one, it is kept encrypted in this browser, never in plain text, under a key pair your browser will not let page code export, and a button forgets it. That is not protection at rest: a copy of your browser profile could be used to recover it, so remember keys only on a device you trust. They are sent only to the provider, never to us, and never put in a URL, a log or an export.
Independent: not affiliated with or endorsed by OpenRouter or any model provider. Open data from data.promptugal.news (CC BY 4.0; each source keeps its own terms).