observability-and-opslisted
Install: claude install-skill Canhada-Labs/ceo-orchestration
# Observability and Ops
## Fail-Fast Rule
If any mandatory invariant, validation, or precondition fails, **stop and
return a structured failure**. Never guess, infer, smooth, approximate,
or "fix" business-critical data.
## Cardinal Rule
If you can't observe it, you can't operate it. Every critical path must
emit metrics, every failure must be logged with context, and every
assumption must have a health check that validates it.
## Three Pillars
### Metrics (What is happening?)
Prometheus-style metrics. Required for any production system:
```typescript
// Counters
metrics.counter('feed.events_received', { source, dataset });
metrics.counter('feed.events_rejected', { source, dataset, reason });
metrics.counter('feed.processing_errors', { dataset, error_type });
metrics.counter('ws.reconnects', { source, attempt });
metrics.counter('api.requests', { endpoint, status_code });
metrics.counter('entity.normalization_failures', { source, raw_id });
metrics.counter('state.transitions', { entity, from, to, reason });
// Gauges
metrics.gauge('feed.data_age_ms', age, { source, dataset });
metrics.gauge('feed.active_sessions', count, { source });
metrics.gauge('worker_queue_depth', count, { queue });
metrics.gauge('ws.active_connections', count, { source });
metrics.gauge('cache.entries', count);
// Histograms
metrics.histogram('feed.processing_latency_ms', duration, { source });
metrics.histogram('api.response_time_ms', duration, { endpoint });
```
Naming: `<domain>.<metric_na