What I'm working on: Building a Workflow

This is a working summary of things I’ve been working on to build a workflow for agentic development over the last 6 months.

Agentic development will scale up all software development. I see two directions: further along a bad scale (big, clumsy teams that make poor software) or back toward a good scale of smaller, more capable teams. I feel the two paths map onto a split: bad open-loop workflows with no feedback, where the connection to stakeholders and mastery of the technology are lost, versus good closed-loop workflows that converge by building understanding of both.

So to “close the loop” I’m looking for “forcing functions”, constraints applied to guide development toward convergence. These can be informal or formal: a specification in human language, or in a formal modelling language. For software I prefer structured but informal, for systems a formal system description. The distinction comes as software at the core needs ‘slack’, but at a system level interfaces harden and should be locked down.

My two main areas of interest here are software development and systems modelling.

Software development

My main focus is structured workflows, such as pipelined processes and feedback loops, represented as agent skills.

Automated translation and compilation of Apple Lisa OS source code

The Lisa OS source code was released in 2023, but the codebase dates from the early 1980s: Clascal, a variant of Pascal, plus M68k assembler.

My question at the time was simply “can I compile and run this?” Some early investigation got a few Pascal files converted to C++ and compiling, and gave me a rough idea of how to handle the assembler and how much work the whole thing would be. Manually, a lot: which made it a perfect project to hand to an AI coding agent.

It’s also a good evaluation of coding-agent skill:

The languages are obsolete. The agent needs a general model of programming rather than pattern-matching a familiar stack.
The Apple source can’t be redistributed, so the agent can’t rewrite the OS, it has to write a tool that rewrites the OS.
It exercises the same skills needed by an agent for virtual platform emulation and cross-platform embedded development. (my day job)

Why the Lisa? Predating the Macintosh, it was the first personal computer with a WIMP GUI sold by a major personal computer maker. Running it on a modern machine makes a significant piece of GUI history far more accessible.

Current status

All non-Lisa code and tooling is agent-written, with no human code review.
All Lisa code is translated by agent-written tools, with no human code review.
The OS kernel and base libraries compiles on Linux.
Test utilities for the mouse and QuickDraw (much of it M68k assembler) run under Linux.
The boot ROM and OS boot sequence boot and display.
The hardware is virtualized only as far as needed to run the OS; display and input devices can be swapped for logging and test stimulus.
All of it runs in CI: https://www.shincbm.com/lisa-source-project/

QuickDraw Boot ROM OS boot

My Claude Skills Library

nakane1chome/claude-skills

I’ve used agent skills to codify workflows during development, I’ve focused on a few areas:

Technical document creation
Software development life-cycle
A harness for evaluating skills

Technical Document Creation Skills

This is a set of skills that help pull ideas from my brain into text. The key point about these skills is:

Each skill uses a pipeline to refine and edit a technical document, such as a spec or design.
The different skills represent different stages of document progress:
- flesh-out converts raw notes into a working document.
- review-steps should be a check, doing the clean-up.
- strong-edit should edit down to a final document.

I’ve described it in more detail here: Claude Beyond Code: Intent Expression and Agent Skills

The pipeline looks like this:

Software Development Life-cycle Skill

claude-skills/light-v-structure

I think iterative development via error-correcting feedback loops is the best way to converge on a desired system. For a team an Agile workflow with a tight code-release-evaluate-iterate cycle is a good harness for convergence, but for agentic development I think the story changes.

The traditional systems engineering V-model can provide the harness for an agent to work autonomously on the low-level implementation.
To enable that a human needs to work at a high level and provide the spec of what the software should do.
This is NOT waterfall! The point is discrete levels of abstraction that are documented or modeled, not discrete stages of development that are gated.

Some V-model development practices are not iterative, for example an anti-iteration practice is stage gating. For agentic development there is an inversion that means stage gating is not needed:

Costs have inverted. Code is now cheaper to produce and verify than specifications.
A specification may be cheaper to validate via an autonomous implementation loop, and use an Agile style release increment to collect stakeholder feedback.
But to iterate the full loop we need a memory of the project and rails for the iteration, this is now is the value of the V-model.
The cost of validation (does it match intent?) is hard to reduce, but the cost of verification (does it match the spec?) can be reduced by scaffolding and tests. That asymmetry, cost(validation) ≫ cost(verification), is why the human owns the top of the V and the agent owns the bottom.

NOTE: Removing stage gating ≠ removing release gating. On the contrary, we can have stronger release gating as we keep accumulated records of what the system does and why.

The idea is a human-dominated top of the V, and an agent-dominated bottom of the V, each with their own iterations driving correctness.

This general idea is similar to Spec-Driven Development (SDD): GitHub Spec Kit, AWS Kiro, OpenSpec, and more. However, the Light-V process is a generalization of a docs-as-code workflow I’ve been using for embedded systems for over 10 years. The main difference is that it is a portable document structure and an agent use discipline, not a tool to adopt. It also pulls in the V-model traceability symmetry: every right-side artifact (test, validation) must verify a left-side one (design, architecture). It targets building new ideas from scratch, growing structure from nothing rather than retrofitting a spec onto an existing code base.

Dimension	Spec-Driven Development	Light-V
Form	A tool/CLI plus templates and slash commands	A portable doc structure + agent use discipline (no tooling)
Primary artifact	The natural-language spec (including semi-formal like EARS or Gherkin); code via spec-as-source, or spec-anchored code and update	A hierarchy of natural-language docs: code via spec-as-source from design, and spec-anchored for higher layers
Flow	spec → plan → tasks → implement (largely linear)	paradigm → ADR → architecture → design → source → test → validate (iterate)
Verification	“Write testable requirements”	The right-side test verifies its left-side counterpart (design to unit tests, architecture to system tests)
Validation	Implicit - the spec is assumed to capture intent	An explicit human feedback loop at the top of the V, does the result meet the original intent?
Decision capture	The living spec	Architecture Decision Records (ADRs) governed by a paradigm document
Orientation	Often retrofitting onto existing code	Greenfield - building new ideas from scratch

Testing and Evaluating of Skills

Example test report generator-coding skill test run
End to end tests for skill : nakane1chome/claude-skills/tests/
Framework for testing skills : nakane1chome/claude-skills/tests_fw/

What I’m trying to do here is automated test & evaluation for Claude skills.

Test the skill - confirm it does what it claims.
Evaluate the skill - measure effectiveness. Add test points that depend on agent capability.
CI Test the skill - ensure each update maintains effective.

Test a Skill

A GHA runs the test and creates a report. The skill test report shows the following:

Summary of session (the skill, report, tokens, cost, duration, etc)
A table of the session details (user prompt, tool calls, requirements, expects, achievements)
List of files in the test project at the end of the test run.
Aggregate statistics (tokens, cache access, cost, duration etc)

A skill is evaluated according to the following:

require: Evaluates something that must happen, if it fails the test aborts.
expect: Evaluates something that is expected to happen, if it fails the test continues, but is regarded a fail.
achieve: Evaluates an optional achievement, the ratio of pass fail is

Test Skill Top

Test Skill End

End to End Report

The above test process is applied per model strength and per skill. A full report shows each test run.

This shows:

Results over time.
Result over model class.
Cost comparison.
Achievement comparison.

Test Report

Some early findings

For the full set of skills, Sonnet is slower and more expensive when doing the same task as Opus. This is concentrated on a few tests.
Agent generated tests are not good are defining achievements that show variation over model class. At the moment all model classes have the same achievement, this was not intended.

Generator (Meta-Coding) Skill

claude-skills/generator-coding.

Agents are capable enough to solve meta-problems, that is, code that generates code:

They often create scripts to make bulk changes instead of burning tokens on doing the work themselves.
If there is a common data model shared between many source files, correctness can be improved by using a code generator, rather than having an agent code each unit individually.
The skill encodes the pattern (Data Model → Parser → Helpers → Template → Output), and the rule: never edit generated output.
- One of the key points here is having the agent create a data model and a set of templates.
- The data model is a formal model that can act as a forcing function across the system.
- Code templates enhance code quality by reducing the variance in an agent’s output when it repeats the same activity.

System Modelling

These are projects focusing on system level development, using system models - rather than pure software.

System Modelling Language Translation and Query Model

This is the first project where I’ve applied the claude-skills/light-v-structure end-to-end and generated all code via an agent (Mostly Claude Opus 4.6 and 4.7 ).

This is a personal project to create a common interface to various System Modelling Languages - like Pan-doc translates between document formats, but for machine-readable system models.

Scoped to system topology - what components exist, how they connect, and their properties - not behavior (state machines, control flow, simulation).
The idea is to create formal system models as a single source of truth across implementation & verification.
These formal models should editable by agents, humans, or both. Concurrent editing should be possible.
The editing should be traceable via an immutable fact model (an append-only graph of facts, never updated in place; each correction is a new fact referencing the ones it supersedes, preserving full history and provenance)

Initial Targets focus on embedded systems:

Register Description: CMSIS-DVD (System View Description), Systematic (Register Description Language)
Hardware Modelling: VIDAL (VHS Hardware Description Language), structural subset
System Modelling: ADDLE (Architecture Analysis & Design Language)
General Modelling: SysML2 (Systems Modeling Language v2)

Implementation:

Architecture and design via: claude-skills/light-v-structure
All code & tests autonomously agent written, a combination of C++ and Python.
Testing grounded against human-owned use cases and third-party reference tools (existing parsers, round-trip conversion).
Model storage via an in-memory immutable fact store (looking for a better solution here)
Fact type system inspired by OMG Meta Object Facility
Model query interface based on Datalog

What worked and what did not:

First-order implementation worked: the agent implemented the parsers, generators, and their tests completely and correctly. Correctness was determined using third-party parsers to verify syntax, or round-trip conversion to verify equivalence.
Most target languages have no C++ parsers or libraries - the domain is dominated by Java. Some pushing was needed to get the agent to realize that implementing a new parser from open-source grammars was low-cost; otherwise it would create incomplete implementations, or claim features were impossible because no C++ version was available.
Much pushing was needed to get the agent to use consistent testing and code generation according to common guidelines.
Second-order implementation was less successful: the agent could not successfully generalize the implementations to architectural concepts. It preferred spaghetti to architecture. The architecture uses an explicit set of abstractions across languages (the MOF layers M1, M2, M3); when asked to design classes to represent those generalizations, the agent created concrete, inflexible designs that over-fit the use cases. I needed to give detailed instructions at this level - but the agent could still produce all the code.

Summary in Spec-Driven Development (SDD) terms - using the three levels of specification rigor:

The implementation in the first-order (that is from design docs) was successfully completed with the Spec-as-Source model of development - the spec generates the code, the human never edits it.
The implementation in the second-order (that is from architecture docs) was only completed with the Spec-Anchored model of development, although the agent was responsible for code editing.

Time Model for Distributed Simulation

This is a personal project being worked on as an exemplar for the systems-need-formal-models workflow: time a first-order dimension software can’t absorb, so a simulator that must be correct in the time domain needs a formal model, not just libraries.

The key idea is to create a logical model, formally specified, of time in distributed simulation

Logical model of how time as a first-order system property is communicated with a system constrained by real time and bandwidth.
Models real, virtual, and sampled time domains; nodes and channels on a timestamped network; using parameters and constraints to derive synchronization.
Core idea: all time is independent, but across the system it can be constrained by a model aware of the system’s topology.
Plan to model using SysMLv2 (or other language? TBD)

I expect an ordinary simulator (a program that computes an approximation of a real-world system) will become easier to write with agentic development. But writing a simulator with well-defined behavior in the time domain and full observability will stay hard without a formal model to constrain it. Unlike most software, a simulator must deal with system attributes such as time, and with the physical and logical layers of a system. For example, time is not well managed by software as software consumes time in its own operation, so it cannot separate the concern of time from compute.

Logical Model of Knowledge Understanding

Dimensions of Understanding

This is an informal model authored entirely by agents, this is the start of an attempt to build a logical architecture for understanding the knowledge structure common to humans and AI. The premise is that a model of understanding can exist independent of any implementation. If it can, several things follow: we could define interfaces to understanding, test and characterize implementations against them, interface to other systems below the text layer (via formal models), and partition models along those interfaces.