Skip to content

The lab

Test a decision model against LLMs on your own tickets

Connect JEV and one or more LLMs, add tickets together with the answer you expect, and see which one gets it right, how fast, and at what cost.

Connect your contenders

Your keys stay in this browser and are sent only with your own requests. The server never stores or logs them.
Decision model
JEV TypeSafe

Answers with a probability for every option.
TypeSafe key
LLM
Your language models

Answer in words and say how sure they are. Add as many as you want to compare.

OpenAI LiteLLM

OpenAI key

LiteLLM address LiteLLM key
This server calls the address you type, so only use one you trust. It never follows redirects and refuses cloud metadata addresses. Each request asks your proxy to skip its cache, so the latency you see is real.

Models to compare
Remember on this device Also run a keyword baseline (free, no key)

This page is served over plain HTTP: anyone on the network could read your keys. Use HTTPS or a VPN.
Choose the tickets

Each ticket needs the answer you expect, so the results can say who was right. The correct team is required; whether it should page on-call is optional.

Start from

Tickets to test, with the expected answers
Ticket Correct team Should page on-call? Remove

No tickets yet. Load a use case, add your own ticket, or paste a batch.
Run it and read the results

Settings
Send to a person below this confidence