Ten Days, Two Courses, One Transformative Journey into Item Response Theory

A former high school mathematics teacher from South Africa, a government curriculum officer from Ghana, and a PhD economist from Stellenbosch walked into the same training with little to no familiarity with item response theory. Ten days later, each of them stood up and presented diverse IRT models to their peers.

Teaching IRT is already a challenging task. To do so for a heterogeneous group of individuals from Burkina Faso, Uganda, Malawi, Mozambique, Kenya, Botswana, Tanzania, Zambia, Ghana, and South Africa is a genuine pedagogical challenge, one tackled with patience and curiosity by the AFLEARN team and Fulbright scholar and PhD candidate Tamlyn Lahoud, whose work at the University of Georgia focuses on enhancing the use of large-scale assessment data in South Africa through IRT.

A Widely Used Method, Poorly Understood

As Martin Gustafsson (2026)  has argued, IRT has become a marker of reliability in educational assessment, yet remains poorly understood among most education practitioners. This includes many whose work depends directly on it. In African contexts, where national assessments and large-scale learning studies increasingly rely on IRT to produce the trend data that informs policy, this gap is not merely academic. It is consequential. It is precisely this gap between the widespread use of IRT and the limited understanding of it among those whose work depends on it that made this training both timely and necessary.

A Deliberate Journey: From Concepts to Application

The training unfolded in two parts. The first course, held online from 13 to 17 April, focused on IRT literacy. It involved building the foundational concepts and the language participants need to engage with the IRT method meaningfully. By the end of the week, many left with a solid conceptual grounding of item response theory, foundational psychometric reasoning, classical test theory, and an understanding of why item-level thinking leads to more precise and informative analysis — alongside honest uncertainty about applying what they had learned. Several noted they needed more time, practice, and feedback before feeling confident working independently.

That honesty was, in many ways, exactly the point. Rather than rushing to application, the participants were given a week to sit with their new knowledge and let their questions sharpen. When the second course convened in person from 11 to 15 May, participants arrived with a clearer sense of what they did not yet know. 

The Pendulum of Frustration and Understanding

In person, participants used R to attempt their own IRT models on real educational data. R is a free, open-source platform accessible to practitioners across all African countries regardless of institutional resources. This is beneficial because it extends the reach of knowledge gained well beyond the training room.

What emerged was something anyone who has undergone serious skills training will recognise: a pendulum swinging between frustration and understanding. Participants were grappling with unfamiliar software, wrestling with datasets that resisted clean analysis, confronting the gap between knowing a concept and executing it — and then, those moments when something clicked. When a model ran cleanly. When a participant could look at an output and begin to interpret what it meant for the children behind the data. Participants built competence visibly and collectively.

For several participants, the datasets they arrived with did not meet the structural requirements IRT demands. This was not a failure, rather an honest reckoning with a challenge that is widespread across African educational contexts. Understanding what your current data cannot tell you, and knowing where to begin in developing more rigorous approaches to assessing learner ability, is itself a critical skill. In anticipation of this reality, participants were introduced to the Program for the Analysis of Education Systems of CONFEMEN (PASEC) 2019 dataset from the Conférence des ministres de l'Éducation des États et Gouvernements de la francophonie (CONFEMEN). The PASEC 2019 dataset is a rich and well-structured resource notable for its rigorous use of IRT in producing accurate and comparable assessment results across Francophone Africa. Other participants were guided carefully through the limitations of their own data, learning as much from what went wrong as from what worked.

By the end, participants had gained foundational psychometric knowledge, practical skills in preparing and fitting IRT models, and the ability to interpret and communicate results responsibly to policymakers and practitioners.

A Growing Community of Practice

The heterogeneity of the cohort, which could have been a liability, became an asset. Different lenses on the same problem produced an understanding that no single perspective could. The enthusiasm, curiosity, and commitment this cohort brought to a demanding ten days is exactly what this kind of training is designed to meet. AFLEARN looks forward to welcoming future cohorts into the growing community of African practitioners equipped to engage critically and confidently with IRT.

Access the training materials used for this course via the link below. We would love to hear about your experience with the course materials—connect with us on LinkedIn or email us at datafirst@uct.ac.za if you have any questions or feedback.

Link to training materials: Item Response Theory

Reference

Gustafsson, M. (2026). IRT analysis – a valuable tool, but only when appropriately applied. AFLEARN Hub.