IS 612: Product Experimentation and Analytics

Content

This course examines how firms improve digital products and services – particularly their features, marketing, advertisement, retail, and pricing – through data-driven experimentation and analytics. In many organizations, critical product decisions must be made under uncertainty and at scale. Leading firms such as Amazon, Netflix, Google, Booking.com, and Coca-Cola address this challenge through continuous experimentation – for example, by A/B testing digital services and marketing campaigns – and systematic analysis of behavioral data. The course provides business students with a rigorous understanding of these approaches. Students learn how to define and interpret product metrics, design experiments, and trans­late empirical results into actionable business conclusions. The course covers key state-of-the-art methods in product experimentation and analytics, including A/B testing, multi-armed bandits, and difference-in-differences. The course is aimed at business students with an interest in analytics. The course combines conceptual foundations provided in lectures with hands-on application in exercise sessions. In the exercise sessions, students work on exercise sheets and analyze real-world data using R. No prior experience in programming or statistical software (e.g., R) is required. Students receive an introduction to R as part of the course.

Learning Outcomes

After successfully completing the course, students..:

• ..understand how firms use experimentation and analytics methods to inform product decisions about pricing, advertising, recommendations, and consumer experience
• ..can evaluate product experiments and apply modern analytics methods such as A/B testing, bandits, and difference-in-differences,
• ..analyze experimental and product data and interpret empirical results,
• ..draw business-relevant conclusions and communicate insights in a clear manner.

Necessary prerequisites:

Successful completion of at least one of the following courses: CC501, CC502, CC503, BE510, CS500, CS530, CS550, CS560, CS652, IE500, IE560. Concurrent enrollment is not sufficient.

Recommended prerequisites:

–