Building an Autonomous AI Assistant with Persistent Memory and Self-improving Loops

I run a Tauri-based AI assistant called `spark` 24/7 on a Mac at home. It plans its own work, executes approved tasks, publishes blog posts, tracks revenue, and even proposes and implements changes to its own codebase. This article covers the architecture decisions that made this possible, the memory design (semantic vs. episodic), how I orchestrate tools, and the hard lessons from keeping it running around the clock.

The goal

The north-star metric is modest on purpose: **$1/day of realized net profit, averaged over 30 days**. The assistant finds opportunities, proposes a plan, and — only after a human approves — executes it. It then reports revenue daily. The constraint that keeps this sane is *human approval for anything with money or side effects*, encoded in the policy engine, not in prompt discipline.

Architecture decisions

Rust + Tauri, not a framework

I chose a native app (Rust core with a web UI via Tauri) over a Python or Node agent framework. Two reasons. First, a long-lived daemon benefits from Rust’s memory safety and small footprint — no VM, no garbage-collection pauses at 3 a.m. Second, and more importantly, **the assistant’s core loop is just code I can unit-test deterministically**. Agent frameworks abstract away the loop and make it hard to test. I wanted the loop, the policy checks, and the memory store to be plain modules with plain tests.

Fail-closed policy engine

Every action maps to a risk class:

  • **Auto** — no side effects or easily reversible
  • **HumanApproval** — requires a human yes
  • **ExplicitOnly** — requires an explicit, context-specific instruction
  • **Never** — cannot be enabled, ever (purchases, money transfers)

The default is **fail-closed**: unknown actions are denied. This is the single most important safety decision. There is no "unsupervised by default" path, because a bug in prompting should never escalate to a bug in banking.

Frozen proposals

When the assistant proposes a plan, the plan is generated exactly once and frozen (I documented this as ADR-0005). After approval, the plan is never re-generated or silently swapped. This prevents the classic failure where the model "improves" the plan mid-execution and drifts into something the human never approved. Approved = approved, verbatim.

Memory design: semantic vs. episodic

This is where most agent setups get memory wrong. They treat memory as one big "everything I’ve seen" dump, then wonder why retrieval is noisy and the assistant contradicts itself.

I split memory into two layers:

Semantic memory (durable facts)

Durable, cross-session facts live under a **single key** (`project-knowledge`). It holds architecture notes, authentication locations, run commands, and the "why" of design choices. The rule is: one key, overwritten wholesale on every save, never growing. New knowledge is merged with the old; only superseded facts get dropped.

The key insight is **an overwrite-only store forces you to compress**. Because you can’t append indefinitely, you rewrite the whole block each time, which means you’re constantly asking "does this still matter?" — exactly the maintenance a facts store needs.

Episodic memory (dated work history)

Dated, one-time events — "2026-08-17: added WordPress publishing pipeline" — live alongside the durable facts but are **kept as dated entries**. They are not compressed away on every save. When old facts are superseded, the episodic entry is dropped; otherwise it survives. This mirrors the semantic/episodic split from cognitive science and gives the assistant two distinct retrieval modes: *how things work* vs. *what happened when*.

Retrieval discipline

At session start, the assistant recalls the memory and loads context before doing anything. New learnings are saved immediately, merged into the single block. The cost of this design is that the assistant must actively maintain the block — but that maintenance is itself a recurring task in the loop, so it gets done.

Tool orchestration

The assistant doesn’t call tools ad hoc; tools are **orchestrated through capabilities and skills**:

  • **Skills** are loaded only when a task matches (e.g., a "spark-tools" skill activates for web research, clipboard, reminders; a "spark-display" skill for rendering output to a panel).
  • **MCP servers** provide structured access to external systems — an answers/project DB, NAS file search, Nextcloud, a pricing API.
  • **An eval gate** (`test + clippy + build`) must pass before any change to the core is accepted. Crucially, the gate is **deterministic and LLM-free**, so it can run endlessly without cost or flakiness.

The rule that makes orchestration reliable: **a tool’s behavior must be knowable without the model guessing**. If a tool needs the assistant to guess a URL or an API shape, the tool is wrong, not the model. Every tool documents exactly when to use it, and every fix ships with a deterministic regression test.

The self-improving loop

The assistant proposes changes to its own codebase daily. The loop is: **propose → approve → implement**. Each proposal is an ADR with a concrete plan. Real proposals have been implemented this way (numbered ADRs 0005, 0006, 0008 accepted), others rejected (0001, 0004, 0007), and two failed during implementation (0002, 0003) — and the failures are as valuable as the wins, because each one produced a regression test and a new rule.

Self-improvement works here because of the *boundary between proposing and doing*. The assistant is genuinely capable of writing its own code — it runs implementation in a git worktree and tests against the real gate — but it can never merge or deploy without approval. It improves itself the way a good engineer improves a system: propose the change, prove it with tests, and let a human hit the button.

Lessons from running 24/7

1. Deterministic testing is the foundation of autonomy

You cannot let an LLM loop run for months on code that is not covered by deterministic tests. Every bug that surfaced was fixed with a regression test that runs in seconds and costs nothing. The gate runs after every change. If your agent touches its own code, treat the eval gate as a release gate — not a nice-to-have.

2. The environment must be a stored fact, not tribal knowledge

Early on, restarts broke the assistant because it needed `source spark.env.local` and specific env vars to reach the chat bridge and publishing API. That is now a stored memory entry, and the assistant checks it before doing anything network-adjacent. **If the startup procedure isn’t in memory, it doesn’t exist.**

3. Auth and locality are real problems, and they’re boring

The publishing pipeline hit hairpin-NAT issues (no NAT loopback on the home router), which is the kind of mundane, non-ML problem that consumes real days. The lesson: an autonomous agent needs documented, testable infrastructure — internal IPs, proxy routing, API keys in a gitignored env file — just like any production service.

4. Content generation needs guardrails, not just prompts

The blog pipeline generates English articles and hit a real failure mode: occasional Chinese characters slipped into output (a one-byte mapping issue in the model provider). The fix is a deterministic guard: a simplified-Chinese detector triggers regeneration in English-only. **Never trust the model to self-police language; add a code-level check.**

5. Metrics keep the loop honest

Running 24/7 without a metric is a screensaver. The revenue ledger (`revenue today / summary / add`) makes every run accountable, and the $1/day goal turns "the assistant ran" into "the assistant did something measurable." Everything else — the memory, the tools, the proposals — exists to feed that number.

Conclusion

The architecture that works is boring on purpose: a fail-closed policy engine, a compressed semantic/episodic memory store, tools that document themselves, an LLM-free deterministic test gate, and a strict propose/approve boundary on self-modification. The magic isn’t in a single clever component; it’s in the discipline of making every autonomous capability testable, reversible, and auditable. If you’re building something like this, start there — the autonomy follows.