Loop Engineering: Why You’re Getting 1.5× From a 100× Machine

The term comes from "lights-out" manufacturing — a plant that runs with no humans on
the floor, the canonical example being a robotics maker whose robots build robots.
Applied to software, a dark factory is a pipeline that plans, writes, tests, and
ships code with no human reviewing a single line on the path to production. Students
learn the one precise distinction that separates it from the agent harness of EB-2:
the harness keeps a human review step before deployment; the dark factory removes
it. The defining characteristic is the absence of human code review in the
deployment path — everything downstream of merge is unchanged. The phrase is often
used loosely, so the module insists on this precision.

The reference architecture has four layers: an inputs layer owned by humans
(specs, holdout scenarios, instruction files, linter rules); an autonomous code
generation layer (the agent reads spec plus repo knowledge, generates, builds,
tests, self-reviews, opens a change); an isolated validation layer (standard CI
first — build, unit tests, static analysis — then an LLM-driven evaluator run
against an ephemeral deployment); and a merge & deploy layer that hands off to
the existing CI/CD pipeline. The same shape recurs in vendor write-ups as
planner → generator → evaluator → deployment. Students learn the single
non-negotiable invariant: the generation layer and the validation layer must be
completely isolated, because an agent that can see how it will be judged optimises
for the judge rather than the goal. Without that wall there is no quality gate, only
theatre.

The trust mechanism at the heart of the dark factory, borrowed directly from how ML
models are evaluated. Acceptance criteria are written as plain-language behavioural
scenarios kept in a location the coding agent cannot read; a separate
evaluator plans and executes the calls needed to exercise the described behaviour,
and an LLM judges whether the output satisfied it. The spec is the training data,
the scenarios are the held-out test set, and the agent never sees the test — when it
fails it gets only a terse failure message, not the scenario text, so it cannot game
the gate. Students learn why this isolation is what makes the gate meaningful, and a
practical bonus: because the evaluator interprets plain-English scenarios
dynamically, there is no brittle step-definition glue to rot as in traditional BDD.

A dynamic workflow is a harness an agent composes on the fly for the task in front of
it, rather than the single fixed harness it normally runs in. Students learn the core
mechanic: from a natural-language request the agent writes an orchestration script,
and a separate runtime executes that script in the background, spinning up tens to
hundreds of parallel subagents whose intermediate results live in script variables
rather than the main context window — so the conversation stays responsive and the
plan stays on track no matter how large the task. The motivating example is a
multi-hundred-thousand-line language port completed in days with the test suite kept
green. This is the harness fundamentals of EB-2 and the orchestrator pattern of EB-7,
now generated automatically and run at scale.

The practical discipline behind the slogan that you should stop prompting agents and
start designing the loops that prompt them. Students learn the canonical cycle —
plan, change, validate, observe, revise — and how to wire it: project instructions as
reusable rules, tools and connectors for the agent to act through, work isolation
(branch, worktree, container) so a run cannot damage production, and a built-in
verification step. They learn the bundled /loop-style skill pattern (e.g. a loop
that babysits open PRs, auto-fixes build breaks, and addresses review comments in a
worktree) and the principle that loops calling sharp, named skills get cheaper over
time while loops that re-derive everything do not — a direct tie to the compound
step of EB-10.

If a loop or workflow runs until a goal is met, the whole approach is only as good as
the check that defines "met". Verification engineering is the discipline of building
that signal. Students learn the spectrum — unit tests, type checks, linters,
benchmark suites, security scans, and LLM-as-judge — and the central rule taught
across the course: the approach only works where the validation signal is strong
(tests, type checks, clear acceptance criteria are friendly terrain; vague UX polish
is not). They learn to choose deterministic checks over model judgement wherever
possible, and how a code-plus-LLM split handles hard constraints with code and soft
constraints with a judge. This is the EB-6 evaluation discipline and the EB-10 holdout
gate, reframed as the thing that drives autonomous work rather than just measuring
it after the fact.

The harness is everything wrapping the model. Students learn the core agent loop
(the model proposes an action, the harness executes it and feeds back the result,
repeat) and the services a harness must provide around that loop: filesystem and
shell access, the tool layer, permission handling, and state. This is the
conceptual anchor for the entire module — every later topic is a component of, or a
variation on, the harness.

Skills package reusable behaviours as a folder containing a SKILL.md file plus
optional supporting scripts and templates. Frontmatter controls invocation: a skill
can be triggered manually, auto-invoked by the model when its description matches
the task, or both. The key technique is progressive disclosure — keep the
SKILL.md lean and move examples, edge cases, and templates into separate files
that are loaded only when relevant, keeping the context window clean. Skills follow
an open standard that works across multiple tools, and modern tooling unifies them
with slash commands so that every skill also exposes a command interface.

Theory . Why standard monitoring is insufficient for agents, which build their
own path at runtime rather than following fixed code. Students learn the pillars of
agent observability — execution traces, tool calls, decision steps, and failures — and
why you must read an agent's process, not just its final output.

Use cases . A support agent that returns a confident wrong answer and the
trace reveals it called the wrong tool; a coding agent whose final diff looks fine but
whose trace shows it never ran the tests; cost spikes traced to a single runaway
sub-agent. Each shows why the process, not just the output, is the unit of analysis.

Practical exercises .

• Concept-check: given a short annotated agent transcript, label each step as a
  trace event, tool call, decision, or failure, and identify where it first went wrong.
• Applied: turn on tracing for a provided sample agent, run one task, and read the
  resulting trace to narrate, in your own words, the path the agent actually took.

Guardrails work in layers, each catching what the others miss: model-level safety
for content, application-level validation for domain errors, tool-level permissions
for unauthorised actions, and human oversight for judgment calls. Students learn that
no single layer is sufficient and how the layers compose.