IC2S2 Tutorial

Organizers: Georg Ahnert, Maximilian Kreutner, Jens Rupprecht, Markus Strohmaier, Kristina Gligorić, und Indira Sen

Tutorial Homepage:https://dess-mannheim.github.io/tutorial_simulating_survey_responses/

Abstract.    This tutorial provides a hands-on introduction to simulating human survey responses with Large Language Models (LLMs), focusing on the methodological rigor required to use “silicon samples” to complement or extend human data. While this approach offers promise for rapid pretesting, counterfactual analysis, and enhancing statistical power through mixed-subjects designs, it introduces new methodological choices, assumptions, and risks that require careful scrutiny. To that end, this tutorial addresses analytic flexibility in silicon samples. Participants will learn to systematically explore how design decisions—such as persona construction and prompting strategies—meaningfully shift results, rather than treating LLM outputs as fixed. The tutorial introduces the QSTN framework, a tool designed to structure simulations and support transparent evaluation across design alternatives. Through guided Python exercises, participants will generate simulated responses for use cases like missing-data imputation and compare modelling choices using multiple evaluation metrics. The tutorial concludes with a critical discussion of methodological limitations, validation challenges, and ethical considerations surrounding autonomy and appropriate use cases for silicon samples. By the end of the tutorial, researchers will be equipped with a principled, transparent approach to integrating simulations into survey-centric social science workflows.