Persistence

Felix is cloud-agnostic: Postgres, a cache, and an object store behind Protocols — not a single vendor SDK. Repo: felix-run/felix.

Data plane

Store	Config	Holds
Postgres + pgvector	FELIX_DATABASE_URL	Sessions, audit, manifests, approvals, plans, jobs, fibers, eval, memory vectors, the skill library’s review records
Valkey / Redis	FELIX_REDIS_URL	Taskiq broker, rate-limit / short-lived cache, queue-tool lists
Object store	FELIX_OBJECT_STORE=fs|s3|gcs|memory	Artifacts, uploads, skill bytes (operator uploads and the tenant library), optional policy bundles / bulk blobs

Migrations: Alembic via felix migrate (or make migrate); make up runs them in a one-shot migrate service, and make db starts Postgres + Valkey alone for a checkout-run API. Local Compose uses pgvector + Valkey; fs object store needs no MinIO. Cloud notes: deploy, deploy/aws/, deploy/gcp/.

Tenant isolation

Tenant-owned tables lead with tenant_id in the primary key. Every store query filters on tenant, and the tenant tables carry the same row-level-security policy (felix_tenant_isolation: tenant_id = current_setting('app.tenant_id'), unless app.rls_bypass is on) — every tenant-owned skill table included. A cross-tenant worker sweep, such as the skill-job claim, runs under the bypass, because RLS with no tenant bound would silently return nothing.

Every store has an in-memory twin selected by FELIX_DATABASE_URL=memory://…, the supported no-infrastructure path; the skill library, feedback, evaluation and policy stores follow it, and the conformance suite (test_skill_library_store.py, test_skill_quality_store.py) runs one contract against both arms. Thread ids are {tenant_id}:{suffix} — caller suffixes containing : or # are rejected so the prefix cannot be smuggled.

Session events

PostgresSessionStore (felix.session.store) appends to session_events (monotonic seq, kind discriminators, message-shaped payload). Patterns never talk SQL directly — they use Session + SessionStrategy.

Strategies: full_replay, windowed:N, summarizing:N, semantic:N, compacting, plus any registered with register_session_strategy. memory.checkpointer chooses where that log lives — postgres (default), none (no session state), or a registered backend.

Relational highlights

Table / area	Notes
audit_events	Append-only; API buffers, worker flushes; optional warehouse spill
manifests + manifest_active	Append-only versions + active/canary pointer
approvals	Signature-indexed pending/approved/denied (+ TTL / one-shot). thread_id names the thread that first requested the row, which is the only attribution a durable run’s approval has
plans	Deep / plan tools; TTL expiry
jobs	Cron schedules; (tenant_id, name) PK
fibers	Durable sleep/step/resume state
eval_*	Datasets, items, runs. put_dataset upserts on (tenant_id, dataset_name, item_id) — it adds and updates items but never removes one, and re-writing an item leaves its created_at at first appearance
memory_vectors	pgvector HNSW, vector(768); kinds fact / event / instruction / task, plus procedure rows from procedural memory
document_chunks	Operator-ingested corpus; generated content_tsv (GIN) + nullable pgvector HNSW, so a deployment with no embedder still gets full-text retrieval
skill, skill_version, skill_file	The tenant skill library (migration 0022). skill is (tenant_id, name) with live_version, the only thing a catalog follows; null when archived. skill_version is (tenant_id, name, version), immutable: status draft / published / archived, source agent / operator / import, the review score, scan status and findings, and who decided it. The primary key is the save lock: two saves racing to one version collide on the row before either writes bytes. skill_file is (tenant_id, name, version, path) with the sha256 and size of what was written — never the content
skill_feedback, skill_eval, skill_policy	The library’s quality loop (migration 0023). Feedback and evaluations carry the worker’s claim (claim_token, heartbeat_at, attempts); a partial unique index holds a version to one queued or running evaluation. skill_policy is one row per tenant, read on top of FELIX_SKILL_PUBLISH_*
skill_version origin, skill_import_sighting	Skill imports (migration 0026). skill_version gains source import, nullable origin_source, origin_ref, origin_commit, origin_tree_hash, origin_license, origin_committed_at, and lineage_import (an import or a version built on one). skill_import_sighting is (tenant_id, origin_source, tree_hash) with first_seen_at: the import cooldown’s clock, pruned by the retention sweep after 366 days. skill_policy.import_min_age_days is the tenant’s side of the cooldown
skill_upstream	What an imported skill’s origin held when Felix last looked (migration 0029): (tenant_id, name) with origin_source, origin_ref, upstream_commit, upstream_tree_hash, first_seen_at, checked_at and the last failed check’s error. Backfilled from each skill’s newest import with checked_at null, which the periodic check takes first. The upstream listing and the library detail read it instead of asking GitHub; every row can be rebuilt by checking again. Migration 0030 adds the skill.update_available delivery to the same row, as 0019 put a run’s completion webhook on its fiber: notified_tree_hash (the digest last queued, which “once per digest” compares with), notify_status (pending / delivered / dead / superseded), notify_due_at, notify_attempts, notify_claim_until (a claim a crashed sweep lets lapse), notify_generation (bumped on every queue; part of the event’s webhook-id, and what a delivery’s save is guarded on), notify_checked_at (an older check never replaces it) and notify_state (the event body and per-endpoint progress). The partial index idx_skill_upstream_notify_due on (notify_due_at, tenant_id, name) holds pending rows only. Downgrading drops the columns and loses any queued event
skill.owner, skill_version.owner, skill_file.owner	Personal libraries (migration 0033). Each table gains owner TEXT NOT NULL DEFAULT '' and its primary key becomes (tenant_id, owner, name[, version[, path]]); '' is the tenant’s own library, so existing rows were not rewritten. A personal owner is `issuer
skill_version.promoted_from	Promotion (migration 0038). Source check gains promoted, and nullable promoted_from names the version of the promoter’s personal library a promoted draft was copied from. The owner is never stored on the tenant’s row; author is the promoter. The downgrade refuses while any promoted version exists
skill_version.adopted_from, skill_file.normalized_sha256	Adopting an import and the copy rule. Migration 0031 adds nullable skill_version.adopted_from: the import-lineage version an operator adopted into this one (POST …/versions/{version}/adopt), null on every row before it; the copy rule reads it, because an adopted version’s files still count as imported text. Migration 0032 adds nullable skill_file.normalized_sha256 — the sha256 of the file’s normalized text (NFKC, Cf characters removed, casefolded, whitespace collapsed, a SKILL.md’s body only), null for a binary asset — and the index idx_skill_file_tenant_normalized_sha256, built CONCURRENTLY. No backfill: the bytes live in the object store, and a version’s rows are immutable, so every file saved before 0032, imports included, stays null for good and is matched by its byte digest only. skills/copy_rule.py owns both digests as a stored format: changing either computation needs a migration that re-hashes the rows. Downgrading 0032 drops the index and column, and the copy rule falls back to bytes
skill_job_lease	Global rows (migration 0024, no tenant and no RLS, like memory_vector_config), one per sweep that must run on one worker at a time — the skill-job sweep, the upstream checks and the update notifications: holder and until_ms, taken with a conditional upsert and renewed before every claim. A row rather than a session advisory lock, because a lock taken through a transaction pooler such as PgBouncer stays on whichever server session ran it and leaks

Vector recall and the HNSW indexes

Each of memory_vectors and document_chunks has one HNSW index across every tenant, and recall filters to the caller’s tenant. For a tenant that holds most of a table’s rows, Postgres serves recall from that index, so the result is approximate: the nearest rows found, not provably the nearest. A smaller tenant is scanned exactly.

An HNSW scan looks at hnsw.ef_search candidates (40 by default) before the tenant filter applies, so a filtered scan could return fewer rows than it asked for. Each vector query therefore sets hnsw.iterative_scan = strict_order for its own transaction, so the scan continues until it has the rows requested or has visited hnsw.max_scan_tuples (20,000 by default). That setting needs pgvector 0.8 or later; on an older extension, recall runs as before and the harness logs one warning per process. The bundled Compose stack ships pgvector 0.8.

Object store

Used for artifact spill when spec.artifacts.enabled and output exceeds threshold_chars. Keys are tenant-scoped: artifacts/{tenant}/{manifest}/{id}.txt, with an {id}.owner object beside it recording the conversation that spilled it, which is what limits the model’s read_artifact to its own. Each spill is also recorded in the artifacts table (tenant RLS like every tenant table), because the store cannot be listed and an object nothing records is one nothing can collect. The row is written before the objects and deleted after them, so drift is always a row naming absent objects.

The worker’s nightly retention sweep drops spills older than FELIX_ARTIFACT_RETENTION_DAYS — 30 by default, 0 keeps forever — text, owner record and row together. Uploads (attachments/) are swept the same way but kept forever by default. Spills written before migration 0018 have no row and are not collected.

File checkpoints follow a similar pattern for /chat/rewind with restore_files. Each change the model’s file tools make in a thread’s own workspace keeps the file’s earlier content at checkpoints/{tenant}/{sha256}. That is once per tenant per distinct content, up to 512 KB a file and FELIX_FILE_CHECKPOINT_MAX_BYTES_PER_TENANT (256 MiB) in all. A file_checkpoints row records the change: its thread, the session event it is anchored to, the path, and the file’s digest before and after.

Content is deleted only once no row names it, and before the rows that named it, so drift is always a row naming an absent object. A rewind forgets the records of the branch it abandons. The nightly sweep forgets a file’s whole history in a thread once its newest record is older than FELIX_FILE_CHECKPOINT_RETENTION_DAYS (7 by default, 0 keeps forever). It never sweeps part of a file’s history, which would leave a rewind restoring an in-between state.

Small VMs: FELIX_OBJECT_STORE=fs under FELIX_DATA_DIR. Helm: enable persistence so /data survives restarts.

Skill bytes

Two prefixes, written by different people, never overlapping:

Prefix	Key	Written by	Read
skills/	skills/{tenant}/{name}/SKILL.md, skills/{name}/SKILL.md, and pinned …/{name}/{version}/SKILL.md	the operator, directly in the store	by the catalog loader for declared names
skill-library/	skill-library/{tenant}/{name}/{version}/{path} for the tenant’s library; skill-library/{tenant}/~{sha256(owner)[:32]}/{name}/{version}/{path} for a personal one	the library service only	live versions only, each checked against its skill_file digest

Library bytes never land under skills/, so an agent’s draft cannot overwrite an operator’s upload pinned at the same version, and a library version is read only through its skill_file rows, so a reader cannot be steered at an object the save did not write. A library SKILL.md whose bytes no longer match their digest is left out of the catalog; the management file route answers 500 version_corrupt, and a publish of it is blocked. Versions are never deleted, so their objects are not collected; a skill holds at most 200 versions.

A personal library’s segment is ~ and the first 32 hex characters of the sha256 of its owner, issuer|subject (skills/library_keys.py). ~ cannot start a skill name, so no personal key is an org one, and the digest keeps a subject, often an email, out of every key. The tenant’s keys are the ones every library key had before owners existed, so no stored bytes moved. A promotion copies a personal version’s bytes into the tenant’s keys; nothing is shared between the two.

Analytics warehouse (optional)

Postgres remains system of record. Optional append-only spill (FELIX_WAREHOUSE=duckdb|clickhouse|doris|none) — the worker exports after the Postgres write. Lean default is none.

The spill carries audit events only: export_audit_events is the sole write on the Warehouse Protocol, and felix.audit.store.flush is its only caller. Nothing in felix.eval reaches the warehouse; eval results are read back through GET /eval/runs/{run_id}.

Tip

felix doctor checks database URL, object store, auth, and consumer secret readiness without mutating state.