docs-tune-ai-chatlisted
Install: claude install-skill Docsbook-io/docs-skills
# docs-tune-ai-chat — Tune AI chat system prompt from real feedback
## Workflow
1. **Verify MCP and plan** — confirm MCP transport is up and the workspace is on PRO or PRO+. On Free plan, stop and print an upgrade prompt. Confirm with the user that they want to modify the system prompt before proceeding.
2. **Pull negative feedback** — fetch 30 days of thumbs-down AI chat interactions, including the user question, the AI answer, and any free-text reason given.
3. **Pull unanswered questions** — fetch 30 days of interactions where the AI explicitly said it didn't know or retrieval returned nothing useful.
4. **Cluster by topic** — group the combined signal into 3–8 topic clusters. For each cluster, record a label, item count, up to three sample questions, and a one-sentence description of the inferred failure mode.
5. **Generate a prompt update** — read the current `system_prompt`. Produce a minimally invasive replacement that keeps all existing brand voice, persona, and refusal rules intact, and adds explicit guidance for the top 3–5 clusters. Cap the result at 1,500 tokens.
6. **Show the diff** — render a before/after diff with annotations mapping each changed chunk back to the cluster that motivates it.
7. **Apply on confirmation** — call `set_chat_system_prompt` only after the user explicitly confirms. Accept `yes`, `no`, or `edit`; on `edit`, loop back to the diff step with the user's revised version.
8. **Report** — confirm the update was applied, include the timestamp