Unsuccessful and successful complex problem solvers – A log file analysis of complex problem solving strategies across multiple tasks

Nicolay, B., Krieger, F., Kuhn, J.-T., Graesser, A., Ifenthaler, D., Baker, R. S., & Greiff, S. (2023). Unsuccessful and successful complex problem solvers – a log file analysis of complex problem solving strategies across multiple tasks. Intelligence, 101793. https://doi.org/10.1016/j.intell.2023.101793

Complex problem solving (CPS) is a key competence in educational contexts with strong conceptual links to students' overall intelligence. However, the mechanisms underlying successful CPS are not fully understood. Therefore, this study investigated several factors presumed to be relevant to CPS success using log file data to code each individual student action during six CPS tasks with different characteristics (N = 1276). We coded individual strategy combinations per student for each item for different strategy combinations of vary-one-thing-at-a-time (VOTAT), hold-one-thing-at-a-time (HOTAT), vary-no-thing-at-a-time (NOTAT), and change-all (CA). Results from generalized linear mixed models showed that CPS success was likely to be achieved by using VOTAT. However, there was an increased chance of solving an item when additional strategies, such as NOTAT or NOTAT plus HOTAT were used. This result was moderated by the presence/absence of eigendynamics as an important determinant of item difficulty. Strategy combinations of VOTAT together with other strategies (all including NOTAT) showed higher chances of CPS success when eigendynamics were present. Additionally, higher chances of solving an item when using VOTAT with additional strategies was demonstrated for items without eigendynamics. Overall, our results suggest that flexibility in strategy application is the driving force for successful CPS performance. Implications are discussed in light of the presumed benefit of pedagogically relevant metacognitive skills, such as planning, monitoring and reflecting, for CPS success. Based on our findings, we provide specific recommendations for the development of computer-based learning simulations to train CPS and related competencies, ultimately enhancing students' skills in educational contexts.