- Title
- Enhancing the use of large-scale assessment data in South Africa: Multidimensional Item Response Theory
- Creator
- Lahoud, Tamlyn Ann
- Subject
- Differential item functioning
- Subject
- Item response theory
- Subject
- Mathematical ability Testing
- Subject
- Educational tests and measurements
- Subject
- Multidimensional scaling
- Date Issued
- 2023-03-29
- Date
- 2023-03-29
- Type
- Academic theses
- Type
- Master's theses
- Type
- text
- Identifier
- http://hdl.handle.net/10962/422389
- Identifier
- vital:71938
- Description
- This research aims to enhance the use of large-scale assessment data in South Africa by evaluating assessment validity by means of multidimensional item response theory and its associated statistical techniques, which have been severely underutilised. Data from the 2014 administration of the grade 6 Mathematics annual national assessment was used in this study and all analyses were conducted using the mirt package in R. A two parameter logistic item response theory model was developed which indicated a clear alignment between the model parameters and difficulty specifications of the test. The test was found to favour learners within the central band on the ability scale. An exploratory five dimensional item response theory model was then developed to investigate the alignment with the test specifications as evidence for construct validity. Significant discrepancies between the factor structure and the specifications of the test were identified. Notably, the results suggest that some items measured an ability that was not purely mathematical, such as reading ability, which would distort the test’s representation of Mathematics ability, disadvantage learners with lower English literacy, and reduce the construct validity of the test. Further validity evidence was obtained by differential item functioning analyses which revealed that fourteen items function differently for learners from different provinces. Although possible reasons for the presence of differential item functioning among provinces were not discussed, its presence provided sufficient evidence against the validity of the test. In conclusion, multidimensional item response theory provided an effective and rigorous approach to establishing the validity of a large-scale assessment. To avoid the pitfalls of the annual national assessments, it is recommended that this multidimensional item and differential item functioning techniques are utilised for the development and evaluation of future national assessment instruments in South Africa.
- Description
- Thesis (MSc) -- Faculty of Science, Statistics, 2023
- Format
- computer
- Format
- online resource
- Format
- application/pdf
- Format
- 1 online resource (157 pages)
- Format
- Publisher
- Rhodes University
- Publisher
- Faculty of Science, Statistics
- Language
- English
- Rights
- Lahoud, Tamlyn Ann
- Rights
- Use of this resource is governed by the terms and conditions of the Creative Commons "Attribution-NonCommercial-ShareAlike" License (http://creativecommons.org/licenses/by-nc-sa/2.0/)
- Hits: 495
- Visitors: 502
- Downloads: 22
Collections
RU Department of Statistics
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | SOURCE1 | LAHOUD-MSC-TR23-74.pdf | 2 MB | Adobe Acrobat PDF | View Details Download |