bug-clustering

# Bug Clustering Process Step-by-step procedure for transforming raw XPost objects into structured, clustered bug candidates with PII redaction and reliability scoring. ## Instructions ### Step 1: Parse For each XPost, produce a BugCandidate with all 33 fields using `lib/parser.ts`: - Extract product_surface, feature_area, symptoms, error_strings, repro_hints - Extract urls, media_keys, language, conversation references - Determine source_type (mention, reply, quote_post, search_hit) ### Step 1.5: Deduplicate Before classification, run content-similarity deduplication using `lib/dedupe.ts`: - Call `deduplicateCandidates()` with parsed candidates and the `candidate_dedup.hybrid_similarity_threshold` from `config/cluster-matching-thresholds.json` (default 0.70) - Uses char-trigram + token-Jaccard hybrid similarity - Does NOT remove posts — tags them as duplicate groups with a canonical post (highest engagement) - Only canonical posts and non-duplicates (`forward_ids`) proceed to classification - Log dedup stats: `"{n} posts ({m} unique, {k} duplicate groups)"` ### Step 2: Classify Run `lib/classifier.ts` on each candidate: - Assign one of 12 classifications with confidence score (0.0-1.0) and rationale - Sarcastic bug reports get classified separately — still treated as signal ### Step 3: Redact PII Run `lib/redactor.ts` on each candidate: - Detect 6 PII types: email, API key, phone, account ID, media flag, URL token - Replace with `[REDACTED:type]` tags - Set pii_fla...