Connect with ACTNext at IMPS 2019!

Twitter: @pmetricsoc 



Tuesday, July 16

Symposium: Model what you can see, not what you imagine.

Time: 10:45 a.m. to 12:15 p.m.

Location: Salón Fresno

Organizer and chair: Gunter Maris

Topic: Virtually all statistical models used in educational measurement involve latent variables to represent properties of items and/or persons. This symposium brings together four studies where the focus is on modeling manifest rather than latent variables. The first will be about a general strategy to construct models that are fit for purpose acknowledging that the true model remains unknown. The second study deals with models for data from partially observed networks. The third is about the direct modeling of the covariance structure of high-dimensional response data and process data. Finally, the last study is about hypothesis testing on intra-class correlations to learn about the structure of hierarchical data. While the applications that are discussed are very different, the common theme of the talks is that models are used to provide an economical description of natural phenomena, that is, of the things we can actually observe, without imparting any existential meaning to latent variables or a model parameter. 

Talk 1: Towards a neo-classical test theory

Authors: Gunter Maris, Benjamin Deonovic, Maria Bolsinova, Lu Ou, Timo Bechger

Presenting Author: Timo Bechger

Abstract: As in the famous quote from Georg Box, any field that depends on statistical modeling, including psychometrics, has to deal with the fact that its models are not true. About a decade earlier, Georg Rasch noted that models need not be true to be applicable for a given purpose.  Following these insights, we propose three basic principles to guide model development in educational measurement. We use PISA data to illustrate how these principles are put in practice.

Talk 2: Modeling marginal networks

Authors: Gunter Maris, Benjamin Deonovic, Maria Bolsinova, Lu Ou, Timo Bechger

Presenting Author: Benjamin Deonovic

Abstract: Network models are increasingly important and popular in various scientific fields. For many of these models it is the case that the nodes of the network represent observed variables of interest. However, these observed variables are often only a small subset of all of the possible variables that could have been observed. Models for these networks which do not take this into account may lead to model misfit and/or model misinterpretation. If the relationships between the unobserved and observed variables is known we show how we can take into account these missing variables by approximating the distribution of the observed variables. Furthermore, with a completely observed set of variables, we show how we can use the same approximation to obtain closed form solutions to the first and second order moments. The derived model has connections to a variety of fields including structural equation modeling, models for deep learning, and cognitive diagnostic models.


Symposium: Process data in international large-scale assessments: Methods and applications

Time:1:30 p.m. to 3:00 p.m.

Location: Salón Fresno

Chair: Lu Ou

Panel discussion: How testing organizations have shaped the psychometric research and employment opportunities for psychometricians

Time: 4:10 p.m. to 5:40 p.m.

Location: Salón Fresno

Chair: Alina von Davier

Panelists include: Alina von Davier, Jorge Manzi, Matthias von Davier, Duanli Yan, Marilyn Stevenson, Anton Béguin


Wednesday, July 17

Panel discussion: Stories of successful careers in psychometrics and what we can learn from them

Time: 8:30 a.m. to 10:00 a.m.

Location: Salón Fresno

Chair: Alina von Davier

Panelists include: Duanli Yan, Marie Wiberg, Carolyn Anderson, Irini Moustaki, Susan Embretson, Jaqueline Meulman

Keynote Speaker Burr Settles – Improving language learning and assessment with data

Time: 10:20 a.m. to 11:20 a.m.

Location: Salón Fresno

Chair: Alina von Davier

Thursday, July 18

Invited Speaker Gunter Maris – The Wiring of Intelligence

Time: 9:55 a.m.-10:40 a.m.

Location: Salón Fresno

Chair: Jorge Gonzalez

Abstract: The positive manifold of intelligence has fascinated generations of scholars in human ability. In the past century, various formal explanations have been proposed, including the dominant g-factor, the revived sampling theory, and the recent multiplier effect model and mutualism model. In this article we propose a novel idiographic explanation. We formally conceptualize intelligence as evolving networks, in which new facts and procedures are wired together during development. The static model, an extension of the Fortuin-Kasteleyn model, provides a parsimonious explanation of the positive manifold and intelligence’s hierarchical factor structure. We show how it can explain the Matthew effect across developmental stages. Finally, we introduce a method for studying growth dynamics. Our truly idiographic approach offers a new view on a century-old construct, and ultimately allows the fields of human ability and human learning to coalesce.


Symposium: Advances in process data analysis

Time: 2:20 p.m. to 3:50 p.m.

Location: Salón Fresno

Chair: Jingchen Liu

Presentation: Learning & measurement of teamwork

Presenter: Alina von Davier

DEMRE Symposium: A major change of the national college admission system in Chile: opportunities to improve

Time: 2:20 p.m. to 3:50 p.m.

Location: Aula Magna

Chair: Daniela Jiménez

Discussion: Alina von Davier


Statistical methods

Time: 2:20 p.m. to 3:50 p.m.

Location: Auditorium 3

Chair: Cees Glas

Title: Machine learning for estimation in IRT models

Authors: Mariana Curi, Benjamin Deonovic, Pravin Chopade, Gunter Maris

Presenting Author: Mariana Curi

Abstract: High dimensional latent space is still a challenge for usual estimation methods in Item Response Theory (IRT) models, like MCMC or maximum likelihood. In this work, we propose a Variational Autoencoder (VAE) architecture, a kind of unsupervised deep neural network, for a multidimensional IRT model parameter estimation. Our approach allows us to model high latent trait dimensions, overcoming some of the limitations concerned to “big data” analysis. The simulation studies show that, given enough data, the proposed method is competitive with the state-of-the-art ones with respect to predictive power and is much faster in runtime performance. The new approach is applied to a real data set to illustrate the usefulness of the proposed method in the context of educational assessment.


Friday, July 19

Symposium: Modeling Heterogeneity with Time Series Data

Time: 8:30 a.m. to 10:00 a.m.

Location: Salón Fresno

Chair: Siwei Liu, University of California at Davis

Title: A Methodological Review on Qualitative Heterogeneity in Quantitative Changes

Presenting Author: Lu Ou

Abstract: Advancements in technology make it easier to collect multimodal time series data in real time. While the data afford quantitative within- and between-subject variability at desirable resolutions for each subject, researchers often need to summarize single time series into phases or group subjects into subgroups to make inferences of the whole population. The process of categorization is essential for drawing qualitative heterogeneity from the complex data, and can occur at either the manifest or latent level. Despite its popular utility, methods for categorizing multivariate intensive longitudinal time series data are still nascent to the field of social and behavioral sciences and require examination and demonstration. In this project, we review different parametric and non-parametric approaches for identifying clusters and/or phases, including functional data analysis, hidden Markov models, and regime-switching dynamic models, and examine through simulations their respective strengths and limitations. With an empirical example, we illustrate their usage in characterizing real-time tracking data in learning environments.

Measurement invariance and DIF III

Time: 1:10 p.m. to 2:40 p.m.

Location: Auditorium 2

Chair: Timo Bechger