Abstract: An observed response can be conceptualized as an outcome of a match (with a certain duration) between an item and a player. In modelling these observed responses, it is usually assumed that person abilities are not changing — at least not while making a test. If the skills of players or the difficulty of items possibly change over time (even after every response), this classical assumption of a constant ability fails. This is, for example, the case in learning environments with instructions or feedback. This presentation proposes an algorithm based on urns that is designed to track changing parameters. The skill of each player and the difficulty of each item is represented with an urn with a certain probability (configuration of balls) and a certain size. This results in binomial distributed random variables with known standard errors. Second, we illustrate the urnings algorithm by analyzing data from a large online adaptive learning environment (Klinkenberg, Straatemeier & Van der Maas, 2011) including both accuracy and response time data. We focus on the (developmental) relation between estimates based on speed and accuracy. Third, we show that this system allows for an adaptive selection of items to the skills of players, without any rating inflation.
An Urn-scheme to Track Accuracy and Response Times in Learning EnvironmentsPresentations
Presented: June 11, 2019
Presenter(s): Bolsinova, Maria | Maris, Gunter | van der Maas, Han | Deonovic, Benjamin| Brinkhuis, M. | Koops, Jesse