The CICANT AISIC Lab will host its first webinar, marking the launch of a new internal seminar series focused on critical discussions at the intersection of artificial intelligence, communication, and social science research. The session is part of the Lab’s activities within the PhD programme and will be open to the wider CICANT community.
The webinar, titled What does the prompt know? Grain calibration for LLM coding of theoretical constructs, will take place on 28 May 2026, at 18h30 (from Lisbon), via online. It explores how large language models (LLMs) are increasingly used to code theoretical constructs in the social sciences and psychology, while questioning what prompts actually “know” about the theories they are applied to. It argues that most current approaches are theory-naïve, as they reproduce ground-truth labels without examining the inferential processes behind them, and introduces a grain-calibration architecture grounded in Messick’s substantive aspect of construct validity.
The approach proposes grain extraction, which decomposes a construct into component processes operationalised as layered codebooks, and grain calibration, which iteratively refines these layers while testing whether they align with the theorised mechanisms. Using two constructs from Moral Foundations Theory, the presentation shows how each codebook layer can reveal specific validity issues, such as overclaiming when the instrument responds to harm-adjacent vocabulary beyond the construct boundary, or underclaiming when it fails to capture implied group-based inequality.
Overall, the webinar proposes a feature-vector decomposition approach that provides evidence that LLM coding instruments can be aligned with theoretical constructs, shifting the focus from prompt optimisation to instrument construction and opening a broader methodological discussion on the use of LLMs in social science research.
