kb-bench

Which platform preserves figures in financial and contract documents?

Short answer

fastgpt — FastGPT retained 97% of the exact figures planted in the corpus, the highest of anything measured — but it is also the platform most likely to fail on a large filing, so the answer depends on document size as much as document type.

What this use case actually needs

How the candidates measured up

Platform Documents ingested Parsing Retrieval Peak memory Ready
fastgpt 8/11 97% 41% 3,227 MB 239 s
dify 11/11 82% 66% 2,572.7 MB 39 s
ragflow 11/11 80% 45% 10,326 MB 511 s

Every figure comes from the same run against the same documents, so these are directly comparable. The full per-document and per-query rows are in the raw data.

Why FastGPT wins this one

Parsing fidelity was measured by planting facts in real documents — long grouped figures of the kind that live inside tables, high-precision decimals, rare compound identifiers — and then searching for those exact strings in the chunks each platform stored. FastGPT retained 97% of them, against 82% for Dify and 80% for RAGFlow.

For a knowledge base built on contracts or financial statements, that is the number that matters most. An answer that reports a figure with a digit missing is worse than no answer, because it looks correct.

The catch, stated plainly

FastGPT’s 97% is marked with an asterisk on our tables because it covers only the eight documents it managed to ingest. The three it dropped were all large SEC filings, and they were dropped on a twenty-minute per-document limit rather than on an error — the platform was still working, just not fast enough.

So the honest form of the recommendation is narrower than the headline: FastGPT preserves figures better than anything else measured, on documents it can finish. If your contracts are ordinary documents rather than hundred-page filings, that caveat costs you nothing. If they are filings, look at Dify.

Why not simply pick the highest recall

Dify retrieved 66% of the answers against FastGPT’s 41%, and on most use cases that would settle it. It does not settle this one, because the two numbers measure different failures.

Recall asks whether the platform found the passage. Fidelity asks whether the figure inside that passage is still correct. A platform that retrieves the right page with a corrupted number has failed in a way that is much harder to notice than one that returns nothing.

If your documents are small enough that FastGPT finishes them, you can have the higher fidelity. If they are not, Dify’s combination of complete ingestion and 82% fidelity is the safer trade.

Where this recommendation stops applying