Evals for your agents.
Graded against your judgment.

Your product fetches a versioned rubric, logs every case it judges, and you label them. MacroEval shows, in numbers, whether its verdicts agree with yours.

Start freeSee how it works

Sign in with Google. No credit card required.

Report · set-plate v3

ready · 120 labeled cases

Product

Your agent's own verdicts

Fail recall
64%
Fail precision
82%
Agreement
78%

Regrade

MacroEval's judge, run now

Fail recall
91%
Fail precision
87%
Agreement
90%
CaseHumanProductRegradeReason
shot-13failpassfailPerson visible at the left edge
shot-14failfailfailWindow frame is warped
shot-12passpasspassEmpty room, straight walls
An agreement report, with example numbers.

You shipped a grader.
Can you trust it?

  • Your grader passes what you'd reject.

    You added a model check to catch bad outputs. Nobody knows how often it's wrong until a customer finds one.

    Agreement reports put a number on it: fail recall and precision against your own labels.

  • Every rejection is explained, then lost.

    Reviewers say why in a thread or a spreadsheet, and the grader never hears about it.

    Label a fail with a reason and it becomes a rule in the next rubric version.

  • Changing the grader is a deploy and a guess.

    The judging prompt lives in your code, so every tweak ships blind and you find out later.

    Your product fetches the rubric at run time, and a new version gets a report before it goes live.

One API call.
The current rubric.

curl https://macroeval.com/api/v1/rubrics/set-plate -H "Authorization: Bearer me_live_..."

Your product reads its judging rules at run time, then logs each case and its verdict back. Change a rule here and the next fetch has it — no redeploy.

Label once.
A rubric that learns.

Your call, one click per case.

Mark what the product produced pass or fail. Say why it fails, and the reason becomes a rule in a new rubric version.

Try it — label the samples.

Cases · set-plate v3Label one
Kitchen plate, empty roomproduct: pass
Hallway plate, person at leftproduct: pass
Bedroom plate, warped windowproduct: fail

Every version kept.

Instructions and rules are versioned. Your product follows the current one, or pins a number.

v1v2v3 current

Numbers, not hunches.

Reports measure fail recall and precision for your product's verdicts, and for a fresh regrade by MacroEval's judge.

Product vs. your labels Regrade vs. your labels Every case, side by side

Try a version before it's live.

Save a version without making it current, run a report on it against your labels, and promote it only if it agrees with you more often.

Whatever your agent makes.
Graded the same way.

  • textJSON

    Support and sales replies

    A refund the policy doesn't allow. A reply that ignores the order on file.

    A ruleNever promise a refund the order doesn't qualify for.
  • video framesimages

    Generated video

    A second person in the shot. A face that changes between frames.

    A ruleOnly the characters in the shot list appear.
  • glTF filerenders

    3D models and renders

    Furniture the plan never had. A door in the wrong wall.

    A ruleEvery opening matches the floor plan.
  • structure fileJSON scores

    Protein designs

    Low confidence across a loop. A binder with too few contacts.

    A ruleMean pLDDT is at least 80.

A case is a list of typed parts, each with a role. The judge reads text, JSON, images and video frames; reviewers see everything, with 3D and structure viewers built in.

Know if your agent
is getting better.

Free to start. Create a project, make a key, and your product can fetch its first rubric today.

Start free