kb-bench

AnythingLLM 1.16.0

A single-container knowledge base and chat application, deployed and measured against the same corpus as every other platform here.

In one paragraph

Deployed from its published image, AnythingLLM runs 1 container and held 542.2 MB at peak during ingestion. It retained 98% of the facts planted in the corpus and answered 83% of the retrieval questions correctly.

Measured

MeasurementResultHow
Time to first served request 146 s Includes image pull, from a clean host
Containers 1 Running after the stack settles
Idle memory 258.1 MB Sum across containers, 60s after ready
Peak memory during ingestion 542.2 MB Sampled every 5s across the whole corpus
Documents ingested 36/36 Failures counted, not excluded
Chunks stored 1,900 Across the whole corpus
Parsing fidelity 98% Approximate: no chunk-listing API, measured by exhaustive retrieval
Retrieval accuracy 83% Answer present in top-5 context, 93 questions
Retrieval latency 274.3 / 307.2 ms p50 / p95, CPU embedding
Licence MIT Permits offering it as a service

What it does well

Where it falls short

Choose it when

Choose something else when

Deployment notes

The lightest deployment measured by a wide margin: one container, a few hundred megabytes, ready in a couple of minutes. If the comparison were only about cost to run, this would win it outright.

The default security posture

Worth stating plainly because it is easy to miss: a fresh deployment reports RequiresAuth: false and MultiUserMode: false, and POST /api/system/generate-api-key returns a working key to an unauthenticated caller. That is convenient for automation and dangerous for anything reachable from a network. Put authentication in front of it before exposing the port.

Why its parsing figure carries a tilde

There is no route that lists the chunks stored for a document, so the parsing measurement is taken by sweeping retrieval with a wide top-N and collecting what comes back. That can only under-report: a chunk retrieval never surfaces is invisible to the measurement. A missing fact here means not observed, not dropped.