MemVerdict

Pilot results: 30 of 500 questions per system. Gaps of a few points are within noise; 95% intervals are shown. The full run is in progress.

Graphiti vs Plain RAG: accuracy, cost and latency

Graphiti: our pilot result was withdrawn and is being rerun. This page collects what is known so far: what each system is, how it is set up, and what the vendors report.

Graphiti

Memory systemRerun in progress
Vendor
Zep
Version
0.30.2
License
Apache-2.0
Result
Withdrawn, rerun in progress

Plain RAG

Baseline, not a memory system
Vendor
Baseline
Version
-
License
-
Result
86.7% (95% CI 70–95%)

Verdict: no head-to-head numbers yet

Pilot result withdrawn: our first adapter added each session as one episode, while Graphiti's own server adds each message as its own episode. Being rerun the canonical way.

Until then, this page shows what each system is, how it is set up, and the scores the vendors report. Those vendor numbers come from different setups, so they are not directly comparable. See the Graphiti page for status.

How the tested systems scored

Plain RAG is highlighted; other systems are muted for context, baselines lightest. Each name links to the system's page.

Accuracy

Higher is better

Share of 30 questions answered correctly (official judge). Whiskers: 95% interval.

  1. Cognee
    90.0%74–97%
  2. Hindsight
    90.0%74–97%
  3. Plain RAG
    86.7%70–95%
  4. Mem0
    83.3%66–93%
  5. Full context
    83.3%66–93%
  6. LangMem
    33.3%19–51%

Not shown: Graphiti (pilot result withdrawn, rerun in progress).

Cost per 1,000 questions

Lower is better

USD billed (prices as of 2026-10-08). Solid: memory side (ingestion and retrieval). Light: answering.

  1. Plain RAG
    $5.45
  2. Full context
    $13.1
  3. LangMem
    $61.6
  4. Mem0
    $93.0
  5. Cognee
    $141
  6. Hindsight
    $172

Not shown: Graphiti (pilot result withdrawn, rerun in progress).

Latency per question

Lower is better

Retrieval plus answer, in seconds. Solid: median (p50). Light: up to p90.

  1. LangMem
    3.5sp90 5.3s
  2. Cognee
    4.0sp90 8.1s
  3. Mem0
    4.3sp90 7.4s
  4. Full context
    4.8sp90 8.0s
  5. Plain RAG
    4.8sp90 7.4s
  6. Hindsight
    5.4sp90 8.1s

Not shown: Graphiti (pilot result withdrawn, rerun in progress).

Ingestion time

Lower is better

Median time to load one question's chat history into the system.

  1. Full context
    none
  2. Plain RAG
    1s
  3. Cognee
    2.8 min
  4. LangMem
    8.5 min
  5. Hindsight
    8.8 min
  6. Mem0
    15.2 min

Not shown: Graphiti (pilot result withdrawn, rerun in progress).

What each one is

Graphiti

The open-source temporal knowledge-graph engine behind Zep. Facts carry validity intervals, so old facts can be invalidated when they change.

Status of our run

Pilot result withdrawn: our first adapter added each session as one episode, while Graphiti's own server adds each message as its own episode. Being rerun the canonical way.

All Graphiti resultsSource repository

Plain RAG

No memory system. Each past session is embedded once; the 10 most similar sessions are given to the reader in date order.

How we ran it

text-embedding-3-small, cosine similarity, top 10 sessions.

All Plain RAG results

What the vendors report

Graphiti

LongMemEval scores reported for Graphiti
ScoreVariantReaderJudgeNoteSource
71.2%SGPT-4oGPT-4o, official promptsZep (paper).arxiv.org/abs/2501.13956
90.2%SGPT-5.4GPT-5.4 with chain-of-thought gradingZep Cloud.www.getzep.com/research

Vendor-reported scores and ours are measured differently, so a gap does not by itself mean either number is wrong. Our pilot uses openai/gpt-6-luna as the reader that writes each answer, gpt-4o-2024-08-06 with the official LongMemEval judge prompt, LongMemEval-S (cleaned, 2025-09) and 30 questions, with each system set up as described above. Scores move with the reader model, the judge model and its prompt, the dataset version, the number of questions, and whether a hosted platform or the open-source package is tested.

Frequently asked questions

Is Graphiti better than Plain RAG?

We do not have a valid result for Graphiti yet. Pilot result withdrawn: our first adapter added each session as one episode, while Graphiti's own server adds each message as its own episode. Being rerun the canonical way. Plain RAG scored 86.7% (95% CI 70–95%) on the same pilot, at $5.45 per 1,000 questions and a median latency of 4.8s.

Which is cheaper, Graphiti or Plain RAG?

Not measured yet: Graphiti's cost will be published with its rerun. Plain RAG: $5.45 per 1,000 questions, of which $2.08 (38%) is memory-side (ingestion and retrieval) and $3.37 is answering.

Which is faster, Graphiti or Plain RAG?

Not measured yet for Graphiti. Plain RAG answers in a median 4.8s (p90 7.4s) and ingests one chat history in 1s (median).

How were Graphiti and Plain RAG tested?

Plain RAG was run on the same 30 questions of LongMemEval-S (cleaned, 2025-09) as every other system, by the same harness. For each question, the system ingests that question's chat history, then retrieves context that one reader model (openai/gpt-6-luna) uses to answer with the official LongMemEval prompt. Answers are graded by the official judge (gpt-4o-2024-08-06) and cross-checked by 3 other judges; all judges agreed on 95% of graded answers. Costs are what the providers billed (prices as of 2026-10-08). Graphiti: Pilot result withdrawn: our first adapter added each session as one episode, while Graphiti's own server adds each message as its own episode. Being rerun the canonical way. Plain RAG: text-embedding-3-small, cosine similarity, top 10 sessions.

Which should I use, Graphiti or Plain RAG?

Until the Graphiti rerun is published, our numbers cannot compare them. Vendor-reported scores are listed on this page, but they were measured with different reader models, judges and setups, so they are not directly comparable with our results or with each other. Graphiti is Apache-2.0 licensed.

More comparisons

Other comparisons with Graphiti

Other comparisons with Plain RAG

All comparisons