agent-system-improvementlisted
Install: claude install-skill SylphxAI/skills
# Agent System Improvement
Turn repeated agent behavior into a tested system change, not another reminder.
Read [references/learning-system-methods.md](references/learning-system-methods.md)
before choosing the intervention and evaluation design.
## Method
1. Define the recurring outcome, affected population, impact, baseline rate,
observation window, and evidence quality. Separate one incident from a
repeatable class.
2. Trace the behavior through the whole agent system: objective, instruction,
injected Skills, context and memory, model, tools, permissions, state,
evaluator, coordination, and feedback. Do not assume the prompt is the cause.
3. Form competing causal hypotheses and identify the observation that would
distinguish each one. Check whether the system is optimizing the wrong
objective before adding more instruction.
4. Choose the smallest intervention at the owning layer. Change policy,
procedure, context, tool contract, evaluator, or feedback only where the
causal mechanism lives.
5. Freeze the baseline, candidate, expected effect, countermetrics, rollback,
and promotion threshold. Use replay, a held-out task set, shadow execution,
a bounded experiment, or live comparison appropriate to the claim.
6. Compare outcome, critical failures, cost, latency, and transfer across task
and model families. Reject improvements that merely move the failure or
overfit the observed examples.
7. Promote, revise, or revert. Record the exact ch