regex-llm-hybridlisted

# Regex-LLM Hybrid Extraction For text where >90% of items follow a repeating pattern, regex extraction plus a cheap-LLM fallback for the low-confidence remainder beats sending everything to an LLM. If the text is free-form and highly variable, skip regex and use an LLM directly. Pipeline: regex parse -> heuristic confidence score per item -> items below threshold (e.g. 0.95) go to the cheapest available LLM for correction; the rest pass through untouched. ## Production Metrics Quiz-parsing pipeline, 410 items: | Metric | Value | |--------|-------| | Regex success rate | 98.0% | | Low-confidence items | 8 (2.0%) | | LLM calls needed | ~5 | | Cost savings vs all-LLM | ~95% | ## Sketch ```python import re from dataclasses import dataclass, replace @dataclass(frozen=True) class Item: id: str text: str choices: tuple[str, ...] answer: str PATTERN = re.compile( r"(?P<id>\d+)\.\s*(?P<text>.+?)\n" r"(?P<choices>(?:[A-D]\..+?\n)+)" r"Answer:\s*(?P<answer>[A-D])", re.MULTILINE | re.DOTALL, ) def parse(content: str) -> list[Item]: return [ Item( id=m.group("id"), text=m.group("text").strip(), choices=tuple(re.findall(r"[A-D]\.\s*(.+)", m.group("choices"))), answer=m.group("answer"), ) for m in PATTERN.finditer(content) ] def confidence(item: Item) -> float: score = 1.0 if len(item.choices) < 3: score -= 0.3 # truncated choice block if not item.answer: