Three practitioner approaches to self-improving agent systems

Summary

Three builders working on production systems share their approaches to building self-improving agent loops. Román Olivera describes the Flat Fish System, a file-based architecture that preserves context across agent sessions using watched file changes and automated consolidation. Kevin Eulenberg documents how he transformed a prototype into a CMS-driven product by running repeated cycles with progressively refined skills, emphasizing nested loops within stages and the compact handoff of learnings to later runs. Rob Lineker walks through daily automated testing loops for a construction SaaS—MCP testing, issue discovery, and agent-driven fixes—without merging to production, keeping the final decision gate under human control. All three ground their approaches in the core principle: context is the asset, and the loop is the delivery mechanism. Read the full Peregian Digital Hub session.

Key Claims

• Static LLM memory (platform-managed context or chat) is insufficient for ongoing work; context needs to be application-level, persisted in the work system itself, not the platform.
• Platform hooks (Anthropic's stop_hook in Claude) tie agent workflows to a specific vendor; file-system watchers watching real changes decouple the loop from any one platform.
• Consolidation processes running at fixed intervals (e.g., daily at 1:00 a.m.) summarise conversation logs and agent traces into a single data file, filtering repetition and keeping only actionable learnings.
• Within a single project run, nested mini-loops (plan, execute, review, fix) inside each stage preserve finer-grained learnings before context compression; larger single-pass loops lose detail.
• Repeated failures are process failures, not model failures: if the same bug recurs, the loop did not capture the learning; every preventable re-occurrence is a signal to embed a rule or validation step.
• Skills improve fastest and reuse best when architecture is iterated in parallel with product iteration; the second build requires fewer mid-run corrections and produces cleaner code because earlier learnings are baked in.
• Guardrails on agent permissions (test-only accounts with partial scopes, environment isolation) are essential for letting agents run unattended; the cost of a mistake in a sandbox is cheap.
• Sub-agents spawned for specific review or validation tasks create natural handoff boundaries and reduce context pressure on the main agent.

Quotes

• "Context is the asset. You can hire a genius, but if you have to explain everything every time, it's like a genius with amnesia."
• "Failures are the raw material that brings you to the next stage. Begin with one real product problem, not a perfect workflow up front."
• "Make the workflow smarter, not the agent."
• "Every preventable repeated failure is a signal that the loop has missed something."
• "When the agent tells you it's fixed, that's a bad pattern. Capture the learning and convert it into a rule or a validation check, embed it in the skill file."