With the ambition to move education towards personalized learning systems, it is crucially important in digital learning systems to be able to accurately track learners’ skills and abilities as they develop during the learning process.

While current statistical methods are not well-equipped to address this challenge, this project develops novel, flexible, and efficient statistical methods that allow for dynamically tracking a large number of interrelated abilities and skills in learners as they develop over time, where the assessment of the learner’s ability levels are updated directly after every new response.

The new rating system has important statistical properties that make it different from existing systems.

First, it allows not only to track abilities but also provides a measure of uncertainty about them (standard error) which is important for rigorously testing hypotheses about the development of the abilities and the relations between them.

Second, the system corrects for adaptive selection of items in the learning system which without being corrected for destroys the measurement properties of the system.

Starting from a simple rating system for a single ability estimated only from the accuracy of the responses in the learning system, we develop our method to be able to handle multiple abilities and multiple sources of information on the learners (e.g., response times in addition to response accuracy).

Urnings is named after the group of statistics problems (dating back to the early 1700s) that involve drawing random colored balls out of an urn.