The Council is a 10-seat adversarial debate (5 bull seats vs 5 bear) that issues a chip. Now it grades its own seats too. Every Council run ends with each model blind-ranking the others, the rankings get aggregated across runs, and the per-seat scoreboard tells us which seat is actually earning its keep — empirically, not by intuition.
Ten seats — five bull, five bear — different model backbones, different priors, arguing under operator-locked role mandates. The verdict surfaces a chip plus a dissent log; that part has been working since we shipped the multi-seat structure.
What we couldn't answer: which seat is actually adding signal? The Skeptic argues "what if the Bull is wrong?" — but is it spotting real blind spots, or is it just noise? The Bear cites web sources Gemini found — but does the citation actually move the verdict, or do the local models reach the same conclusion without the URLs? The Cartographer recurses for 8 minutes — does the recursion produce something worth the wait?
We had no signal. We were running councils, reading the dissent log, and forming intuitions about seat quality. Intuition is fine for a week. Over months it drifts toward whichever seat happens to be most readable to a human, which is exactly the kind of bias the Council was built to escape.
So we added a new phase between Rebuttal and Verdict: every seat blind-ranks the other five. The trick is anonymization — each seat sees the other five positions as Response A through Response F, identities stripped, in a deterministic shuffle that's stable per run but rotates between runs.
The judge is asked to evaluate three things explicitly:
Then it produces a ranked list. We aggregate across all judges, average the position each seat received, and now we have a number.
Lower average rank = peer-judged stronger. The Quant adjudicator that writes the final chip ALSO gets the aggregate-rank summary as input — so when it weighs evidence in the verdict, it sees which positions the rest of the Council found most rigorous.
The anonymized cross-evaluation idea isn't ours. Andrej Karpathy posted llm-council as a Saturday hack — four frontier models answering a question, then peer-ranking each other anonymously, with a Chairman model synthesizing. We saw it, recognized exactly the missing layer in our debate structure, and ported the pattern into our multi-phase adversarial setup. Different problem (adversarial financial debate vs general Q&A), different roles (forced bull/bear/skeptic vs free-form ensemble), but the anonymized peer-rank phase translates cleanly.
Open work like that compounds. Crediting it openly is the minimum the field deserves.
Every Council run appends a row per seat to a local append-only ledger: average peer-rank received, whether that seat agreed with the final verdict, model used, latency. After enough runs we surface this on an admin page — sorted by avg-rank, best first.
What we expect to learn:
None of this resolves on day one. The point is the data starts accruing now.
The per-ticker Council is locked to a specific position with a specific data corpus. That's correct for stocks but useless for everything else. We have a sibling surface in development — a general-purpose Council that runs three frontier models on any free-form question, cross-ranks them, and lets the recursive cartographer compose the final answer. It's offline at the moment while we harden the prompt-injection defenses around free-form user input; back when that work lands.
This is the closest thing we have to Karpathy's original use case (book reading, general inquiry), and it's a useful demo for anything outside finance: research questions, technical comparisons, decisions under uncertainty. Same anonymized peer-ranking machinery, no domain corpus, no chip output — just the synthesized answer with the rankings visible.
One free-tier slot per user is shared with the per-ticker Council quota. Both spend roughly the same money.
Smaller win, real impact: the slowest seat (the proprietary recursive synthesis engine) used to run sequentially after a sibling local-syndicate seat. Both use the same local model backend. Loading that model from external disk takes ~10 seconds; unloading it for the next seat costs another 5; reloading it later costs another 10. Across the 10-seat roster with disk-resident weights, that adds up to a meaningful share of total wall-clock.
Now the Cartographer launches in a background thread at the start of Phase 1. The Bull seat fires its single fast call against the same loaded model — both share the warm GPU slot. The other four seats (different model, online APIs) run sequentially on top. By the time the loop reaches the Cartographer slot, its result is usually already done; if not, we wait the residual. Net: phase 1 wall-clock now bounded by max(cartographer, sum-of-other-five) instead of their sum. Saves 1-3 minutes per debate, more on cold-start runs.
For the avoidance of expectation: