Concurrency

Engrava is built on SQLite, so it inherits SQLite’s concurrency model: many concurrent readers, one writer at a time. This page states what a single store guarantees concurrent callers, what it deliberately does not guarantee, and which writer topologies are supported.

WAL: many readers, one writer

File databases opened via from_config use WAL (write-ahead logging) mode by default — it is the database.wal_mode setting. Setting it to false skips Engrava’s WAL pragma on that call; a newly-created database then normally uses SQLite’s rollback journal, but an existing database that was already in WAL mode retains it, since the mode is stored in the file itself. Outside WAL, a writer and an open read can block each other, so the first guarantee below is WAL-specific; the second — only one writer at a time — is an ordinary SQLite property and holds either way. Under WAL:

• Readers don’t block the writer and the writer doesn’t block readers. A read sees a consistent snapshot while a write is in progress.
• There is still only one writer at a time. SQLite permits one writer at a time; a competing Engrava-owned connection waits up to five seconds — the configured busy timeout — and then raises OperationalError if the lock is still held, unless the call is one of the guarded write paths that convert that into WriteContentionError instead (see Busy timeout).

This is ideal for read-heavy agent-memory workloads: retrieval (the hot path) avoids per-result database writes, but calls sharing one store serialize their SQLite statements, and with access tracking enabled they can also buffer telemetry for a later write; ordinary writes are comparatively infrequent.

Many async tasks, one store

Share one store instance across the asyncio tasks in your event loop. You do not need a connection pool or multiple stores for in-process concurrency.

Every guarded write method on a store instance shares one in-process, task-reentrant write lock for its own write: create_thought, get_or_create, upsert_by_hash, bulk_store, update_thought, restore_thought, delete_thought, create_edge, update_edge, delete_edge, create_action, update_action, store_embedding, record_access, cleanup_expired, run_hygiene, and suspend_auto_commit(). Once a call holds the lock, its own guarded read and write run as one critical section: a different task’s call to any of these blocks until the one in progress has committed (or rolled back) rather than landing in the middle of it. run_hygiene() is the exception on the read side — it collects and scores its candidates before acquiring the lock, and only its archive/GC phase runs under it. The lock is task-reentrant, not call-reentrant — the same task may freely nest calls, notably a write issued from inside its own suspend_auto_commit() window, but a genuinely different task waits.

get_or_create and upsert_by_hash still have a real gap, on a miss — when the call is not itself nested inside the caller’s own suspend_auto_commit() window. Each runs an exploratory probe under the lock; when that probe finds nothing, the call releases the lock, runs the pre-insert preparation seam with nothing this call itself took still held, and only then reacquires the lock for a decisive probe before it inserts. A different task’s write can start and finish inside that gap. The decisive probe exists to catch exactly that: it is re-run before the insert, so a write that landed in the gap turns this call’s outcome into a confirmation bump instead of a duplicate row. Nested inside the caller’s own open window, none of that applies: the outer window’s hold of the lock never actually drops, so it stays held — blocking every other task — for the whole seam call too.

What this does not do: it does not make a read-modify-write your own code performs across two separate calls atomic — thought = await store.get_thought(id) followed later by await store.update_thought(id, field=thought.field + 1) is still two independent critical sections with a gap between them (see the safe idioms below). It also does not reach a second store on the same database file — see Multiple stores, one database file — and it does not help across event loops, since the lock itself is bound to one loop (below).

What is guaranteed

• Statements do not run concurrently. aiosqlite runs the actual SQLite calls on a dedicated background thread and marshals every call onto it, so two tasks’ statements are serialised rather than executed at once — and SQLite applies each statement atomically, so no query observes a half-written row.
• An edit writes only the columns it owns. update_thought writes the fields the call gave a new value to, plus the updated_at stamp and the revision bump — not the whole record. A field another task changed since this call read the row is preserved rather than rolled back. That includes the fields engrava maintains for you: access_count / last_accessed_at from record_access(), and confirmation_count.
• A deduplication sighting counts. create_thought(deduplicate=True) and get_or_create() bump confirmation_count relative to what is stored (confirmation_count + 1, evaluated by SQLite), so a bump made by another writer since the row was read is added to, never overwritten.
• Deduplication is serialised against itself, and now against every other guarded write on this instance too. create_thought(deduplicate=True), get_or_create() and upsert_by_hash() share an internal asyncio.Lock covering the whole “probe the content hash, then insert or bump” window, and that window is also held under the write lock above, so an unrelated guarded write from a different task cannot land inside it either. Except when caller code has already opened a transaction directly on the connection, these writes no longer duplicate across stores either — see Busy timeout.
• suspend_auto_commit() is a real, exclusive window. It holds the write lock for its entire duration, so a different task’s call to any guarded write path now waits for the window to close instead of joining its transaction. A nested call on the same task shares the outer transaction. Only the outermost call attempts the final commit or rollback; an already-quarantined connection is not rolled back, and cancellation during commit leaves the outcome indeterminate (see Error handling).

What is not guaranteed

• A caller-level read-modify-write across two separate calls is still not atomic. The lock above covers one call’s own internal read and write; it does not, and cannot, cover a gap between two calls your own code makes (get_thought then, later, update_thought). See the safe idioms for the caller-owned-lock pattern that closes that gap.
• get_or_create / upsert_by_hash have their own internal gap, on a miss. See above for what can land there and why the decisive probe exists.
• Same-field edits still resolve last-write-wins. Two tasks legitimately editing the same field is not a bug the lock removes — it removes the silent corruption of a call’s own read or write being torn by another call landing mid-operation, not the ordinary fact that two edits to the same field can only leave one final value. Whichever call’s critical section runs second determines the field’s value, deterministically and without error.
• StaleDataError does not detect “someone else wrote it,” in general. See Optimistic concurrency below for what the guard actually rejects.
• A same-task nested call is not blocked by its own lock. A caller-owned hook or callback that itself issues a write, invoked synchronously from inside another operation’s read-modify-write span (not a second asyncio task — a genuinely nested call on the same task), still runs freely: the lock is task-reentrant by design, so it does not — and structurally cannot — protect against this shape.
• A second store on the same database file is unaffected. See Multiple stores, one database file.
• One store belongs to one event loop. Not because of the connection — aiosqlite creates each operation’s future on the calling loop, so a plain read from a second loop works. It is the store’s own synchronisation: the write lock, the dedup lock, and the audit journal’s per-connection lock are all asyncio-bound locks that bind to the first loop that has to wait on one and then raise RuntimeError for a waiter from any other loop. An uncontended lock never binds, so a store shared across loops can look healthy right up to the first time two callers actually contend. Use one store per loop; within that loop, share it freely. (See Known Limitations.)

A deadlock this store cannot resolve raises, it does not hang

The write lock is task-reentrant, so a task that opened a suspend_auto_commit() window may issue writes of its own inside it freely — but a different task cannot reuse that grant. If the window’s own task spawns a task and awaits it before the window closes, neither can make progress: the spawned task can never get the lock the window’s task holds, and the window’s task can never release it while still awaiting the spawned task. This is already out of the documented contract (drive every write inside an open window from the one task that opened it), so past a bound, the acquiring call raises WriteLockTimeoutError instead of waiting forever — which also ends the deadlock, since the spawned task’s failure lets whatever awaited it unwind.

That bound has to clear a legitimate case, not just ordinary contention: bulk_store’s batch embedding call runs inside its own suspend_auto_commit() window, holding the lock for a genuine network round trip to an embedding provider that can take minutes for a large batch. The default — write_lock_acquire_timeout_seconds=600.0 — is sized with headroom over the shipped default provider’s own worst-case retry-exhaustion time; raise it for a slower provider or larger batches via SqliteEngravaCore(..., write_lock_acquire_timeout_seconds=...) or the equivalent from_config(...) argument.

The dedup lock’s own re-entry failure is narrower and has no legitimate case at all. Unlike the write lock, the in-process lock guarding the dedup probe-and-insert window (create_thought(deduplicate=True), get_or_create, upsert_by_hash) is not task-reentrant, because it has no legitimate reentrant use: a second acquisition by the same task — the concrete shape is upsert_by_hash()’s hit branch calling an overridden update_thought that calls back into a dedup entry point on the same task — raises DedupLockReentryError immediately instead of blocking on itself forever.

The safe idioms

• Hold one store for the process lifetime and share it across tasks — see Deployment.

• Let one task own a row, or partition edits so that no two tasks write the same field of the same row. This needs no locking at all.

• Or serialise the read-modify-write yourself when tasks genuinely compete for one field:

  # engrava does not make read-modify-write atomic; a caller-owned lock does.
  # edit_lock is one asyncio.Lock shared by every task that writes this way.
  async with edit_lock:
      thought = await store.get_thought(thought_id)
      if thought is not None:
          await store.update_thought(
              thought_id,
              confidence=min(1.0, (thought.confidence or 0.0) + 0.1),
          )

  This closes the window within one process only — see Multiple stores, one database file.

• suspend_auto_commit() no longer needs this discipline enforced by convention — a different task’s write now simply waits rather than racing in. It remains true that only the task that opened the window (or a write nested inside it) should be the one issuing writes inside it; interleaving unrelated writes through a long-running window just serialises them for no benefit.

Optimistic concurrency and StaleDataError

update_thought, restore_thought, update_edge and update_action each carry a revision guard, checked and incremented in the same atomic UPDATE; a changed upsert_by_hash hit goes through update_thought rather than a guard of its own. Every core row (thought, edge, action) has a revision column. No caller action arms it — the engine bumps revision on every guarded write to a row, automatically. update_edge and update_action carry this guard for the first time; before 0.7 neither could ever raise StaleDataError.

StaleDataError means the guarded UPDATE matched no row. Two different situations produce that, and the error does not tell them apart:

• the row’s revision is no longer the value this call read — another guarded write landed on the row in between, whatever field it touched; or
• the row no longer exists — a competing writer deleted it between this call’s read and its write. (ThoughtNotFoundError covers a row that was already missing when the call started, not one that vanished mid-call.)

The consequences are worth stating plainly:

• An ordinary concurrent edit now raises StaleDataError, whatever field it touched, the moment it lands on a row between this call’s own read and write. The rejected update then writes nothing at all — no field of it reaches storage. Recover by re-reading the record and recomputing the change; do not replay the original changes.
• In the ordinary case — two genuinely concurrent tasks on this store instance — this almost never fires, because the write lock above already serialises them end to end: the second call’s own read happens strictly after the first call’s write has committed, so its guard is captured against the post-first-call revision.
• What it does catch is exactly what the write lock cannot reach: a same-task nested write issued from inside a caller-owned hook (the lock is task-reentrant, so it does not block this), and a write from a second store on the same database file — a second connection in this process, or a second process. Both used to be silently lost; both now raise StaleDataError before anything is written. See Multiple stores, one database file for the cross-connection case in full.

StaleDataError is still not a general semantic-conflict detector: it is “this write found no row to apply to”. Do not read its absence as proof nobody else wrote. If your application needs caller-level staleness detection, keep your own version value in metadata and compare it yourself, or serialise the edits with the lock idiom above.

Busy timeout

When a connection can’t immediately get the lock it needs (another writer holds it), SQLite waits up to the busy timeout before giving up with database is locked.

How the timeout is set depends on how the store opened its connection:

• from_config and EngravaManager (engrava owns the connection) open it with PRAGMA busy_timeout=5000 explicitly. When SQLite invokes the busy handler, busy_timeout=5000 permits a wait of up to five seconds; a content-changing update_thought() can instead fail immediately during its FTS5-trigger lock upgrade. These paths also set PRAGMA synchronous=NORMAL, the documented-safe companion to WAL (durable across an application crash; only the most recent transactions are at risk on an OS crash or power loss).
• The manual SqliteEngravaCore(conn) constructor (you own the connection) changes none of these pragmas. The connection keeps whatever it was opened with; Python’s sqlite3/aiosqlite default busy_timeout already happens to be 5000 ms, and the default synchronous is FULL.

For workloads with more write contention you can raise the timeout on your own connection before handing it to the store, or after from_config via the store’s connection:

import aiosqlite
from engrava import SqliteEngravaCore

conn = await aiosqlite.connect("engrava.db")
conn.row_factory = aiosqlite.Row
await conn.execute("PRAGMA busy_timeout = 15000")  # wait up to 15s for a lock
store = SqliteEngravaCore(conn)
await store.ensure_schema()

A longer busy timeout trades latency-on-contention for fewer database is locked errors; tune it to your write pattern.

Two kinds of write convert lock contention into WriteContentionError instead of leaking a raw database is locked. create_thought(deduplicate=True), get_or_create(), upsert_by_hash() and bulk_store(deduplicate=True) open their “check existing, then insert or bump” window with BEGIN IMMEDIATE — unless your own code already opened a transaction directly on the connection — instead of an implicit deferred transaction, so lock contention surfaces at the start of the window rather than only once the eventual write needs the lock — closing the classic embedded-SQLite deadlock shape a deferred transaction invites. The busy timeout above still applies to each attempt; on top of it, the call retries up to three times total with a short backoff and, only once that is exhausted, raises WriteContentionError. The revision-guarded updates (update_thought, restore_thought, update_edge, update_action) convert the same busy/locked failure on their own single UPDATE, with no retry at this layer. Retrying either kind outright is safe — the failed call wrote nothing.

Multiple stores, one database file

Engrava does not currently support more than one store writing the same database file — whether those stores are in two processes or are two connections inside one process.

That is a statement about engrava, not about SQLite. WAL (by default) and PRAGMA busy_timeout=5000 are configured, and they do exactly what they say: the file tolerates several connections, readers do not block the writer, and SQLite serialises the writers so the database itself is never corrupted. In-process locks stop at the store instance, but two of the three cross-connection gaps this used to leave open are now closed at the database level instead:

1. An update landing in the window is now rejected, not silently discarded. No in-process lock can reach across a connection boundary, but the revision guard lives in the database itself: the guarded UPDATE’s WHERE revision = ? clause carries the value read before the write, so a second store’s committed bump in between makes it match no row, and the first store’s write raises StaleDataError instead of overwriting it. This is a correctness improvement, not a support statement — multiple writers on one file remain outside what this store is built and tested for.

2. Except when caller code has already opened a transaction directly on the connection, deduplicate=True, get_or_create(), upsert_by_hash() and bulk_store(deduplicate=True) no longer duplicate across stores. The probe-and-row window now opens with BEGIN IMMEDIATE (above). When Engrava can open BEGIN IMMEDIATE itself, exhaustion of its bounded retries raises WriteContentionError.

3. The audit journal’s sequence numbers can still collide. When journaling is enabled, appends are serialised by an asyncio.Lock keyed on the connection. A second store has a second connection and therefore a different lock — in another process, where no lock could be shared at all, but equally in this one. Two stores journaling the same database can race the journal’s monotonic sequence_number. The writer makes up to 5 total attempts (the initial attempt plus at most 4 retries) on the resulting UNIQUE collision; if contention persists it raises:

   RuntimeError: Failed to append journal entry after 5 retries due to sequence contention

   That error is the loud symptom of this remaining topology.

Two things do survive a second connection, and they are worth knowing so the rule is not read as broader than it is: reading (any number of processes may read one file under WAL) and the relative confirmation_count bump described above, which is evaluated by SQLite against the stored row rather than derived from a prior read.

Making every read-modify-write path correct across connections needs a transaction-level mechanism engrava does not have yet. It is deferred, not ruled out; until it ships, treat one writer per database file as the contract. If you need multiple independent writers, give each its own database (next section).

Per-service isolation

EngravaManager runs one database file per named service, each with its own connection and its own lock. This is the supported way to isolate writers (per tenant, per worker, per logical partition):

from engrava import EngravaManager, load_config

config = load_config("engrava.yaml")
async with await EngravaManager.from_config(config) as mgr:
    store_a = await mgr.get_store("tenant_a")  # tenant_a.db
    store_b = await mgr.get_store("tenant_b")  # tenant_b.db

Passing the whole config (its services section must be set) gives every service’s store the rest of engrava.yaml too: the journal, hygiene policy, TTL, hooks and dreaming. from_config(config.services) still works and wires only the PRAGMAs, the embedding provider and the vector backend; see Configuration → Multi-Service Usage.

Because each service is a separate file, writes to tenant_a never contend with writes to tenant_b, and each can be backed up or deleted independently. See the scoping section for when to choose per-service isolation over in-store filtering.

Summary

For the exhaustive package reference, see the OSS concurrency.md docs.