# NADILI: Architecture Source: https://manaienko.com/projects/nadili/architecture/ Author: Artem Manaienko # Architecture _and integration._ I design and integrate the web apps, API contracts, background processes and storage, including their recovery rules. **Runtime:** API, workers, scheduler and reconciliation **Contracts:** OpenAPI and generated TypeScript clients **State:** PostgreSQL, pgvector and durable jobs Explore the architecture 01 System topology **Separate processes, shared durable state** **Runtime**02 Data and contracts **One database, explicit boundaries** **Boundaries**03 Execution and failure **A late worker cannot commit an old attempt** **Recovery** Implemented in NADILI. Private, pre-launch. System topology ## Separate processes, shared durable state Two Next.js applications call FastAPI. Workers, scheduling and identity reconciliation run as separate Python processes. PostgreSQL holds durable product and job state. 01 / BROWSER SURFACES02 / APPLICATION RUNTIME03 / STATEPublic webNext.jsPrivate adminNext.jsFastAPIVersioned APIWorkersDurable jobsSchedulerLeader electedReconcileIdentityValkeyDerived statePostgreSQLAuthoritative state01 / BROWSER SURFACES02 / APPLICATION RUNTIME03 / STATEPublic webNext.jsPrivate adminNext.jsFastAPIVersioned APIWorkersDurable jobsSchedulerLeader electedReconcileIdentityValkeyDerived statePostgreSQLAuthoritative state Both web apps call FastAPI. API, workers, scheduler and reconciliation use PostgreSQL. Valkey holds transient state. Scheduled jobs reach workers through PostgreSQL. ### Where work runs **API:** Validate requests, enforce access and call application services. **Worker:** Claim jobs, acquire sources and execute processing pipelines. **Scheduler:** Evaluate due work and reconcile pending external AI operations. **Reconcile:** Run identity reconciliation independently. ### A web request does not own a background job Jobs and progress persist in PostgreSQL. A worker can resume eligible work after another process exits. Valkey handles transient coordination; PostgreSQL owns durable jobs. Trace a request through the system The API validates a request and calls an application service. Repositories run in a SQLAlchemy unit of work. Background work is recorded as a job for a worker to claim. The API, worker, scheduler and identity reconciliation run as separate services. Scheduled work reaches workers through persisted jobs. Data and contracts ## One database, explicit boundaries PostgreSQL stores product records, jobs, evidence and vector indexes. Both web applications use generated TypeScript clients from the API contract. ### What crosses each boundary **Web / API:** OpenAPI produces separate public and private client outputs. **Workspace:** Repositories scope owned records to the current workspace. **Knowledge:** Claims retain evidence; raw acquisition payloads are temporary. **Retrieval:** pgvector embeddings are keyed by content hash and model. ### The public contract has its own shape A maintained public OpenAPI fragment is merged into the exported contract. The TypeScript generator produces separate public and private clients with separate schema selections. Both applications use the shared client package. How contract drift and stale embeddings are handled The repository exports OpenAPI and checks it against the canonical contract. Generated-file drift has its own check. The public API shape stays explicit while private contracts evolve. Search embeddings live in PostgreSQL beside indexed content. Content identity and model identify each derived record. HNSW cosine indexes support vector retrieval with workspace scope. PostgreSQL is required in development, tests and CI. SQLite is unsupported because database-specific behavior matters here. Execution and failure ## A late worker cannot commit an old attempt A job claim carries an attempt number and an expiring lease. Before completion, the repository locks the job and checks current ownership. ### The job lifecycle **Claim:** FOR UPDATE SKIP LOCKED lets workers claim different eligible rows. **Recover:** Expired running jobs can be reclaimed within their attempt limit. **Fence:** Check workspace, public ID, running status, attempt and live lease. **Wait:** Persist waiting\_external while an external AI operation is pending. ### Cancellation must survive a late response A provider call can finish after cancellation or lease expiry. The completion check rejects a stale attempt before it writes state. Waiting on external AI work is recorded and releases the lease. Inspect the completion fence and scheduler failover Completion locks the job row, then checks workspace, job identity, running status, attempt number and an unexpired lease. A reclaimed job has a different attempt number; a cancelled job is no longer running. Either change rejects the old worker. Schedulers use a non-blocking PostgreSQL advisory lock on a dedicated AUTOCOMMIT connection. Standbys retry later; the leader probes its connection before evaluation. A connection loss during evaluation can still allow one overlapping cycle. The design relies on job deduplication and row locks for that boundary; leadership alone does not guarantee exactly-once execution. [Back to all six areas](https://manaienko.com/projects/#scope-overview-title)[Next: Applied AI ↗](https://manaienko.com/projects/nadili/ai/)I design and integrate the product, with coding agents supporting implementation.