When my colleagues from the Netherlands and I were preparing our NCME symposium submission on advanced models for using response times in educational measurement, we were not sure how well the topic would line up with the rest of the program, as there has not always been a strong focus on response time modeling at NCME.
However, it became clear that response time modeling was one of the hot topics of this year’s NCME, with dozens of presentations of research where response times and other process data were considered.
With the advance of computerized tests, research in educational measurement shows a strong move towards considering more response data than just recording whether an answer is correct, and many of the presentations showed interesting examples of the benefits that considering this process data can offer, as well as the challenges it poses.
From a psychometric perspective, it seems that we are entering a new and exciting era, where a wide variety of data is collected for the purpose of measurement, with the expectation that this will give us a more complete picture of the persons that we are assessing.
It is however also clear that this poses serious challenges for researchers in the field, both at the level of developing relevant research questions (what exactly do we want to use this data for, and what is its expected relevance?) and at the level of statistical modeling (how do we take these different sources of information into account, and how do we ensure that our statistical models result in valid inferences about the persons?).
Meeting these challenges will require us to critically consider exactly what we take to be the purpose of measurement, and how these novel sources of information relate to this.
With the surging research interests in considering process data, I am sure that we will see exciting new developments in this direction in upcoming conferences.
With ACTNext’s ambition to innovate assessment by incorporating process data, this presents us with a great opportunity to take a leading role in this important and growing scientific field.