kb-bench

AnythingLLM 1.16.0

A single-container knowledge base and chat application, deployed and measured against the same corpus as every other platform here.

In one paragraph

Deployed from its published image, AnythingLLM runs 1 container and held 394.2 MB at peak during ingestion. It retained 100% of the facts planted in the corpus and answered 41% of the retrieval questions correctly.

Measured

MeasurementResultHow
Time to first served request 146 s Includes image pull, from a clean host
Containers 1 Running after the stack settles
Idle memory 258.1 MB Sum across containers, 60s after ready
Peak memory during ingestion 394.2 MB Sampled every 5s across the whole corpus
Documents ingested 6/11 Failures counted, not excluded
Chunks stored 74 Across the whole corpus
Parsing fidelity 100% Approximate: no chunk-listing API, measured by exhaustive retrieval
Retrieval accuracy 41% Answer present in top-5 context, 29 questions
Retrieval latency 45,827.2 / 53,454.3 ms p50 / p95, CPU embedding
Licence MIT Permits offering it as a service

What it does well

Where it falls short

Choose it when

Choose something else when

Deployment notes

The lightest deployment measured by a wide margin: one container, a few hundred megabytes, ready in a couple of minutes. If the comparison were only about cost to run, this would win it outright.

The default security posture

Worth stating plainly because it is easy to miss: a fresh deployment reports RequiresAuth: false and MultiUserMode: false, and POST /api/system/generate-api-key returns a working key to an unauthenticated caller. That is convenient for automation and dangerous for anything reachable from a network. Put authentication in front of it before exposing the port.

Why its parsing figure carries a tilde

There is no route that lists the chunks stored for a document, so the parsing measurement is taken by sweeping retrieval with a wide top-N and collecting what comes back. That can only under-report: a chunk retrieval never surfaces is invisible to the measurement. A missing fact here means not observed, not dropped.