No real yardstick
Existing tests are short, static and judged by an LLM that scores the same answer differently across runs. None can ask what was true on a given date, after the facts changed.
Memception builds a fictional life, delivers it to your system the way real users would, and scores every answer against facts computed from that life. No judge model. No setup. Free to start.
More than 200 labs are working on long-term and organizational memory for AI. The benchmarks they have were built for short conversations and graded by another model, so the numbers that matter most are the ones nobody trusts.
Existing tests are short, static and judged by an LLM that scores the same answer differently across runs. None can ask what was true on a given date, after the facts changed.
Long-memory benchmarks take days to set up, cost real money to grade, and break in ways you only find late. Building a credible one in-house pulls your best people off the product.
When a lab writes the test, tunes against it and reports the score, buyers and investors discount the result. A neutral third party with private test sets makes the number mean something.
Every Memception life starts as data, not prose. Conversations and documents are rendered from it afterwards, so every question has an answer a script can compute.
A person, their people and a causal graph of events, dated to the day. It replays into a fact ledger where every fact has a start and an end.
Into what memory systems actually receive: voice conversations with realistic speech-recognition errors, shared documents, email and texts, and a stream of phone telemetry.
After every conversation, questions drawn from what had been disclosed so far. Each answer is a short formula over the ledger, or the question doesn’t ship.
Your system ingests the stream and answers. Grading is exact: numbers, dates and yes/no are compared directly, and fact questions are multiple choice drawn from the life itself.
Sign in with GitHub, get a key, and point your memory system at a life. Your system never writes replies. It only has to remember what it hears and answer what it’s asked.
# curl -s https://memception.io/sdk/memception.py -o memception.py
import memception
run = memception.start(corpus="maia-100k")
for item in run:
if item.kind == "turn": # a user or assistant line
memory.ingest(item)
elif item.kind == "question": # after each session
item.answer(memory.ask(item))
print(run.report_url) # every answer, graded, with proofEach call for the next item tells us you’re ready, so we time every turn without you adding anything.
A life arrives as dated conversations. Questions come after each session and reach back across everything so far. Time between sessions isn’t graded.
Every question shows your answer, the correct one, and the facts in the life it was computed from. Plus latency for every turn.
Three endpoints: start a run, get the next item, read the report. Use the SDK or call it from any language.
A memory system has to be right at a cost, speed and readiness production can accept. We score all of it.
The span one life is designed to cover, season by season.
Tokens delivered to a system over a full life, from about 16M tokens of authored material.
Every gold answer is a computed value. Correctness doesn’t depend on another model’s mood.
Every account gets the practice life free, forever. Bigger lives, more of the real world, and an official score cost more.
Priced per million tokens delivered
Early access on request
Same price for every lab
Custom lives for your domain: talk to us. Certification buys the right to publish, never the score.
Memception builds on BEAM (Tavakoli et al., 2025), which generates long-term memory benchmarks of up to ten million tokens. We adopt its foundation and add two things: a time-versioned world-state that can be asked about as of any date, and judge-free grading, where every answer is computed from that world.
Sign in with GitHub, create a key, and run the practice life in minutes. We only ask for your public profile and email.
Free forever on the practice life. Questions: hello@memception.io