tunelisted

# Tune — Live Skill RL Turn the user's reaction into a targeted SKILL.md overlay edit. Human-in-the-loop reinforcement learning for skills. ## Architecture ``` Base SKILL.md (read-only, shared) + Overlay.md (mutable, per-workspace) = Effective skill at invocation time Path A (cron): /api/improve ticks weekly, objective reward harvester Path B (live): THIS SKILL — user reaction = reward, edits overlay in-session Same overlay.md, same scorecard — both paths compound. ``` ## The reward schema (non-negotiable) | Signal | Reward | Source | |---|---|---| | Explicit positive | +1 | "good", "perfect", "keep it", "that nailed it" | | Implicit positive | +0.5 | user kept output unchanged, shipped it, moved on | | Neutral | 0 | no reaction, switched topic | | Implicit negative | −0.5 | user edited heavily, reran skill, tweaked params | | Explicit negative | −1 | "shit", "bad", "nope", "revert", "worse than before" | **Do not infer reward beyond these.** If you are unsure, classify as **neutral** and log; do NOT propose an edit on neutral. ## The loop (one tick) ``` 1. OBSERVE - User just invoked skill <X> at turn T - Skill produced output Y - User reacted with R 2. CLASSIFY - Map R to reward from the schema above - If reward in {0}: log and stop. Do not propose. 3. DIAGNOSE - If reward < 0: what specifically did the user reject? (a phrase? a structural choice? a missing section? tone?) - If reward > 0: what specifically did