text-classificationlisted

# LLM-Based Text Classification for Social Science Research ## Instructions ### 1. Codebook Design - Before drafting the codebook, specify the population, sampling frame, and (for experimental data) the treatment condition each response is drawn from. These constrain which categories can plausibly exist and which demographic subgroups any bias assessment must cover. LLM classification extends, rather than replaces, the longer open-ended coding tradition in survey methodology (Geer 1988; Lupia 2018). - Treat codebook design as the most consequential decision in the classification pipeline. LLMs struggle with loose instructions and revert to general-purpose definitions rather than following researcher-specific operationalizations (Halterman & Keith 2025). - Structure each code with the following components (adapted from Halterman & Keith 2025): - **Label**: The exact output string the model should return - **Definition**: A single-sentence operationalization of the construct - **Clarification**: What IS included — boundary cases that belong in this category - **Negative clarification**: What is NOT included — common confusions and adjacent categories - **Examples**: 2-3 positive examples (correctly classified) and 2-3 negative examples (common misclassifications) - Keep the number of codes small (3-6) for initial classification. Larger coding schemes increase ambiguity and reduce inter-annotator agreement for both humans and LLMs (Chae & Davidson 2025). - Allow mul