← ClaudeAtlas

docs-tune-ai-chatlisted

Improve the AI chat system prompt of a Docsbook workspace using real negative feedback and unanswered questions from the last 30 days. Clusters failure patterns by topic, proposes a minimally invasive prompt update, shows a before/after diff, and applies the change only after explicit user confirmation. Requires PRO plan.
Docsbook-io/docs-skills · ★ 0 · AI & Automation · score 75
Install: claude install-skill Docsbook-io/docs-skills
# docs-tune-ai-chat — Tune AI chat system prompt from real feedback ## Workflow 1. **Verify MCP and plan** — confirm MCP transport is up and the workspace is on PRO or PRO+. On Free plan, stop and print an upgrade prompt. Confirm with the user that they want to modify the system prompt before proceeding. 2. **Pull negative feedback** — fetch 30 days of thumbs-down AI chat interactions, including the user question, the AI answer, and any free-text reason given. 3. **Pull unanswered questions** — fetch 30 days of interactions where the AI explicitly said it didn't know or retrieval returned nothing useful. 4. **Cluster by topic** — group the combined signal into 3–8 topic clusters. For each cluster, record a label, item count, up to three sample questions, and a one-sentence description of the inferred failure mode. 5. **Generate a prompt update** — read the current `system_prompt`. Produce a minimally invasive replacement that keeps all existing brand voice, persona, and refusal rules intact, and adds explicit guidance for the top 3–5 clusters. Cap the result at 1,500 tokens. 6. **Show the diff** — render a before/after diff with annotations mapping each changed chunk back to the cluster that motivates it. 7. **Apply on confirmation** — call `set_chat_system_prompt` only after the user explicitly confirms. Accept `yes`, `no`, or `edit`; on `edit`, loop back to the diff step with the user's revised version. 8. **Report** — confirm the update was applied, include the timestamp