# Cognoscenti: A Benchmark for Trustworthy AI Memory

Canonical HTML: https://achiral.ai/cognoscenti

Cognoscenti is Achiral's benchmark framework for trustworthy AI memory. It evaluates whether memory systems activate the right context, suppress stale or misleading context, adapt through use, and remain reproducible enough for public inspection.

AI memory does not fail only by forgetting. It also fails when it recalls obsolete instructions, activates plausible but wrong context, treats every stored fact as equal, or cannot explain why one memory was selected over another. Cognoscenti turns those behaviors into measurable benchmark dimensions.

## What Cognoscenti measures

- Activation: whether the current query, project, account, or workflow brings the right memory forward first.
- Selective forgetting: whether stale, superseded, low-signal, or misleading memories stay out of the active context.
- Interference resistance: whether related but wrong memories can be suppressed when they compete with the correct context.
- Consolidation: whether repeated and validated experience becomes easier to retrieve.
- Adaptation: whether new facts update older assumptions.
- Efficiency: whether useful memory remains selective as the memory set grows.

## Current public run

The first public organizational-memory run, dated 2026-08-02, compares RAG, agent-memory, and organic-memory reference baselines on the same inspectable JSONL workload.

Organic memory led the tested architectures on Top-1 accuracy and distractor suppression while matching agent memory on Recall@3:

| System | Top-1 Accuracy | Recall@3 | Precision@3 | Distractor Activation |
| --- | ---: | ---: | ---: | ---: |
| RAG reference baseline | 50.00% | 83.33% | 27.78% | 25.00% |
| Agent-memory reference baseline | 75.00% | 100.00% | 33.33% | 38.89% |
| Organic-memory reference baseline | 91.67% | 100.00% | 33.33% | 16.67% |

## Reproducible artifacts

- Benchmark report: https://achiral.ai/benchmarks/ai-memory-benchmark
- Markdown benchmark report: https://achiral.ai/benchmarks/ai-memory-benchmark.md
- Methodology: https://achiral.ai/benchmarks/organizational-memory-2026-08-02/methodology.md
- JSONL workload: https://achiral.ai/benchmarks/organizational-memory-2026-08-02/organizational_memory.jsonl
- Raw results: https://achiral.ai/benchmarks/organizational-memory-2026-08-02/memory_eval_results.json
- Metrics CSV: https://achiral.ai/benchmarks/organizational-memory-2026-08-02/memory_eval_metrics.csv
- Source repository: https://github.com/Achiral-AI/cognoscenti
- Zenodo software DOI: https://doi.org/10.5281/zenodo.21995304
- Figshare preprint DOI: https://doi.org/10.6084/m9.figshare.33288195
- Software Heritage source snapshot: https://archive.softwareheritage.org/swh:1:snp:c49295731ebb8a76713e877cfd8dc6399c0137d7

The public run does not claim to benchmark Mem0, Zep, Letta, Glean, LangGraph, or any other vendor product. Vendor results should be published only when provider endpoints, keys, commands, seeds, and raw outputs are recorded.

The Figshare preprint and benchmark artifact are shared under CC BY 4.0. Cognoscenti source code remains licensed under Apache-2.0.