Evals

Shared core

Plain ES modules, no build step, no third-party code. Pin a version: a published version never changes in place.

import { createOpenRouter, createKeyStore, runAgent, runEval, promptugalTools, loadPolicy }
  from 'https://evals.agenticcells.site/core/v0.1/index.js';

Sites in the family copy or import a pinned version. Within v0.1 only fixes land; anything that changes a contract goes to v0.2. A site that needs bytes that never change should copy the files. The contract is in core/v0.1/README.md; the tests run in your browser at core/v0.1/tests/.

Modules in v0.1

ModuleWhat it does
abp.jsValidates a policy, builds its system prompt, filters tools, checks each tool call, wraps untrusted text and flags instructions in it, checks the answer.
agent.jsThe agent loop: model, tool calls, tool results, model, with a step-by-step trace.
eval.jsRuns one policy and a suite of cases across many models: pass or fail, tokens, cost, time. Retries a rate limit once.
providers/openrouter.jsOpenRouter adapter: models (free first), chat completions with tools, OAuth PKCE sign-in. Checked against OpenRouter's documentation on 18 November 2026.
providers/mock.jsA provider that replays recorded replies, for tests and demos with no key.
tools/promptugal.jsTwo read-only tools over data.promptugal.news: list_sources and get_latest.
cost.jsCost meter: the provider's reported cost first, listed prices as a fallback.
keys.jsKey store (memory by default, remember by choice, forget), masking, redaction, PKCE.
vault.jsEncrypted secret store: a Web Crypto key pair whose private key cannot be exported, a fresh AES-GCM key per secret, IndexedDB. Nothing in plain text.
ui.js and ui.cssUI pieces: the step-by-step trace, the checks, a key panel. Model and tool text is always set as text, never as HTML.