Architecture

engrava is the memory database for AI agents — a Python library for storing, linking, searching, and evolving ideas. It is SQLite-first and designed to be embedded by larger cognitive systems.

Layer Model

Imports flow downward only:

+--------------------------------------------------+
|  CLI / MCP server / Consumer apps, scripts, ...  |
+--------------------------------------------------+
|  Extensions / Embeddings / MindQL                |
|  (dreaming, hooks, providers, query language)    |
+--------------------------------------------------+
|  Infrastructure                                  |
|  SqliteEngravaCore, schema, migrations           |
+--------------------------------------------------+
|  Domain                                          |
|  models, enums, protocols, exceptions            |
+--------------------------------------------------+

• Domain (src/engrava/domain/) — stdlib + Pydantic only. Frozen models, @runtime_checkable Protocols, zero infra imports.
• Infrastructure (src/engrava/infrastructure/) — SQLite implementation of domain protocols.
• Extensions (src/engrava/extensions/) — optional capabilities (dreaming, hooks) that depend on domain + infrastructure.
• MindQL (src/engrava/mindql/) — read-only query language.
• Embeddings (src/engrava/embeddings/) — pluggable embedding providers.
• CLI (src/engrava/cli/) — Click-based command-line interface.
• MCP server — a Model Context Protocol server shipped as the separate engrava-mcp package (not part of engrava core). Like the CLI it is a top-layer API consumer: it wraps engrava’s public async API over stdio so MCP clients (Claude Desktop, Cursor, …) can use a store. Install and run it independently with uvx engrava-mcp. See MCP server.

Core Components

SqliteEngravaCore

The primary store implementation. Provides:

• Thought CRUD (create, read, update, list, search)
• Edge CRUD (create, read, update, delete, traverse)
• Embedding storage and vector similarity search
• Full-text search (FTS5)
• Hybrid search (5-signal fusion — see below)
• Bi-temporal valid time — optional valid_from / valid_until bounds on thoughts and edges (a second time axis: when a fact is true, distinct from when it was recorded), queried via the four valid-time MindQL predicates, with invalidate_thought / invalidate_edge to close an interval without deleting. See Bi-temporal Model.
• Scoped retrieval — metadata filters (an AND of typed field predicates over a thought’s metadata) and a VisibilityQueryFilter for the “public-or-mine” pattern, both accepted by recall() and search_hybrid(). A visibility filter is a query refinement, not access control.
• Result collapse — collapse_key / collapse_max_per_unit keep one best-ranked row per caller-defined unit (a conversation, a document) and backfill the freed slots with deeper distinct units. A de-duplication convenience, not a filter.
• Action records — ActionRecord storage for what an agent did, alongside what it knows.
• Derived records — a producer hook can persist children from a stored thought; core writes them without re-entering on_store.
• Memory Hygiene — an off-by-default pass that archives inactive thoughts (ACTIVE → ARCHIVED, reversible via restore_thought — though the expiry cleared on archival is not restored) behind wall-clock guards, with permanent collection as a separate step.
• Hash-chain audit journal — off by default; when enabled, thought and edge create/update/delete operations are journaled as before/after deltas, but buffered access telemetry and the edges removed by a thought delete are not journaled individually. See Observability.
• Schema management and migrations

Hybrid Search (5-Signal Fusion)

search_hybrid() fuses five ranking signals:

final_score = w1*FTS + w2*Vector + w3*Recency + w4*Priority + w5*Graph

Disabled signals have their weight redistributed proportionally across the active ones. With neither FTS nor vector usable there is no arm to fuse, and search_hybrid() returns a query-less window instead, pre-ordered so that truncation keeps the freshest rows. See Hybrid Search for details.

Dreaming Extension

Periodic memory consolidation: a run evaluates candidates and may promote, link, cluster/reflect, and retire, each when its gates, thresholds, caps, embeddings, and feature settings permit.

1. Scores active thoughts via configurable signals.
2. May promote qualifying thoughts to P1 priority.
3. May create edges (ASSOCIATED, source=DREAMING) between promoted thoughts and their nearest neighbours.
4. May cluster + reflect — groups semantically related thoughts and creates ThoughtType.REFLECTION meta-thoughts with centroid embeddings and CONSOLIDATED_FROM edges.
5. May retire orphaned reflections — a REFLECTION whose source thoughts have all left ACTIVE summarises nothing live, so it is archived (ACTIVE → ARCHIVED, reversible) unless it is pinned or at a protected priority (P1 by default); see Dreaming → When a REFLECTION is retired. This sweep runs on every consolidation pass; it is gated neither by enable_reflections nor by Memory Hygiene.

Dreaming is a graph mutator and abstraction builder — each consolidation run can grow the thought graph with dream-discovered connections and create higher-order REFLECTION thoughts that aggregate clusters. Both feed into hybrid search, closing the dream → structure → retrieval loop.

See Dreaming for details.

Extension System

New behaviors plug in via EngravaHooksProtocol — on_store and on_retrieve transform thoughts through the lifecycle, and decay_function supplies the decay multiplier an enabled Memory Hygiene pass folds into its eviction score. Custom query commands are MindQLExtension entries carried on ExtensionManifest.mindql_extensions, or passed straight to MindQLExecutor; the hooks’ mindql_extension_registry() method is reserved and core never calls it. Application-level logic (planners, reasoners) belongs in consumers, not in engrava core. See Extensions.

Data Flow

   Ingest                    Dreaming                   Search
   ------                    --------                   ------
   create_thought() --+     run_consolidation()        search_hybrid()
   store_embedding()  |      |                          |
                      v      v                          v
              +-------------------+            +-----------------+
              |    SQLite DB      |            |  5-signal fusion |
              |  thoughts table   |<-----------|  FTS + Vec +    |
              |  edge table       |            |  Rec + Pri +    |
              |  embedding table  |            |  Graph          |
              +-------------------+            +-----------------+
                      ^
                      |
              Dream: ASSOCIATED edges on promotion
              Dream: REFLECTION thoughts from clusters

Upgrade Paths

Version-to-version upgrade notes live on their own pages rather than here: Upgrade to 0.7, which covers the user_version 20 to 21 core-schema migration, and the earlier Upgrade to 0.6 and Upgrade to 0.5. Older releases are recorded in the library’s own Upgrade Guide.