grafana-llmops-forgelisted

Runs end-to-end AI/LLM observability on any Grafana (OSS, Cloud, Enterprise) with a single prerequisite, a reachable Grafana. Auto-discovers the instance and which LLM telemetry dialects exist (OpenTelemetry GenAI gen_ai.*, LiteLLM, vLLM, TGI, GPU/DCGM, eval scores), then generates and deploys dashboards (FinOps and cost by provider origin across US/EU/Asia, gateway SLOs, agent and RAG tracing, adoption, quality evaluations, self-hosted inference, and governance evidence read against the EU AI Act, ISO/IEC 42001 and NIST AI RMF). Also emits burn-rate SLO alerts, cost recording rules for Prometheus or the Prometheus Operator, English or French labels, and a visual verification pass. Use it whenever the user mentions Grafana, dashboards, AI or LLM monitoring, token costs, AI FinOps, LLMOps, agents or RAG, model adoption, AI Act, ISO 42001 or NIST AI RMF evidence, or Prometheus/Loki/Tempo applied to AI, even without the word dashboard, or to audit a stack emitting nothing yet.
alebgl77/grafana-llmops-forge · ★ 1 · AI & Automation · score 75

Install: claude install-skill alebgl77/grafana-llmops-forge

# Grafana LLMOps Forge Turns any Grafana instance into an AI/LLM command centre for a platform team: discovery, dashboard generation, alerting, FinOps, EU AI Act governance. Single prerequisite: `GRAFANA_URL` + a service-account token. Everything else is discovered or provisioned. ## Doctrine (what makes this different) 1. **Discovery-first, never assume.** Never generate a panel "just in case". Probe the instance and its datasources, capture the **real metric names**, and only build panels whose queries will return data. OTel exporters disagree on suffixes (`_seconds`, `_token`, `_total`): the capability map is the source of truth, not theory. 2. **Four telemetry dialects, one mental model.** LLM signals arrive in four practical shapes: OTel GenAI conventions (`gen_ai_*`, Development status, v1.4x, opt-in `OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental`), LiteLLM gateway (`litellm_*`, native USD spend), inference engines (`vllm:*`, `tgi_*`), and GPU (`DCGM_*`). Evaluation signals (`gen_ai_evaluation_*`, RAGAS, guardrails) form a fifth, optional one. Each blueprint is translated into whatever is actually emitted. 3. **Cost is computed, not hoped for.** Resolve one financial source across all datasources: the exact recorded total (`llm:cost_usd_per_second`), then native LiteLLM spend, then OTel composition against the price registry. Equal-priority candidates require `--datasource <UID|unique name>`; never silently mix scopes. Native spend is not an audited invoic