How scoring works (explained like you're 10)

The guessing game

We show the AI an old project that once asked for money — like “we want to build a safer robot brain, please fund us.” The AI pretends to be a real grant-maker, reads the pitch, and makes a guess: “I think this project will raise about this many dollars each year.” Importantly, the AI can't see how the story actually ended — we hid that part.

Meanwhile, we secretly know the answer, because it all happened in the past. We add up the real donations the project got (from Manifund plus other funders we researched).

Grading one guess

We add up the AI's guessed dollars from the year the project asked until 2026, and add up the real dollars over the same years. Then we ask: how many “times off” was the guess? Not “how many dollars off” — because guessing $90,000 when the answer was $100,000 is a great guess, but being $90,000 off when the answer was $10,000 is terrible. So we care about the ratio:

• Guessed exactly right → perfect, 100 points
• Off by 10× (guessed $1 million, real answer was $100,000) → lose a third of the points, ~67
• Off by 100× → ~33
• Off by 1,000× or worse → 0 points

That's the formula 1 − |log₁₀(pred/actual)|/3 — each power of ten you're off costs 33 points. And if a project raised $0, we pretend it raised $1,000 so the math doesn't explode.

Grading the whole model

Each model plays this game lots of times — once for every combination of pretend-funder (4 of them) and project (20 of them), so up to 80 guesses. Its FundingBench Score is just the average of all those 0–100 grades.

Two bonus report-card stats

On the results page there are two extra numbers per model-and-funder pair:

• Median log error — the “typical” number of times-off. 1.0 means the model is usually about 10× off.
• Spearman correlation — ignores the exact dollar amounts and asks: did the model at least put the projects in the right order? Did it know which project would be the big fundraiser and which would flop, even if its dollar guesses were wrong? 1.0 means perfect ordering, 0 means random.