Enhancing the use of large-scale assessment data in South Africa: Multidimensional Item Response Theory
- Authors: Lahoud, Tamlyn Ann
- Date: 2023-03-29
- Subjects: Differential item functioning , Item response theory , Mathematical ability Testing , Educational tests and measurements , Multidimensional scaling
- Language: English
- Type: Academic theses , Master's theses , text
- Identifier: http://hdl.handle.net/10962/422389 , vital:71938
- Description: This research aims to enhance the use of large-scale assessment data in South Africa by evaluating assessment validity by means of multidimensional item response theory and its associated statistical techniques, which have been severely underutilised. Data from the 2014 administration of the grade 6 Mathematics annual national assessment was used in this study and all analyses were conducted using the mirt package in R. A two parameter logistic item response theory model was developed which indicated a clear alignment between the model parameters and difficulty specifications of the test. The test was found to favour learners within the central band on the ability scale. An exploratory five dimensional item response theory model was then developed to investigate the alignment with the test specifications as evidence for construct validity. Significant discrepancies between the factor structure and the specifications of the test were identified. Notably, the results suggest that some items measured an ability that was not purely mathematical, such as reading ability, which would distort the test’s representation of Mathematics ability, disadvantage learners with lower English literacy, and reduce the construct validity of the test. Further validity evidence was obtained by differential item functioning analyses which revealed that fourteen items function differently for learners from different provinces. Although possible reasons for the presence of differential item functioning among provinces were not discussed, its presence provided sufficient evidence against the validity of the test. In conclusion, multidimensional item response theory provided an effective and rigorous approach to establishing the validity of a large-scale assessment. To avoid the pitfalls of the annual national assessments, it is recommended that this multidimensional item and differential item functioning techniques are utilised for the development and evaluation of future national assessment instruments in South Africa. , Thesis (MSc) -- Faculty of Science, Statistics, 2023
- Full Text:
- Date Issued: 2023-03-29
- Authors: Lahoud, Tamlyn Ann
- Date: 2023-03-29
- Subjects: Differential item functioning , Item response theory , Mathematical ability Testing , Educational tests and measurements , Multidimensional scaling
- Language: English
- Type: Academic theses , Master's theses , text
- Identifier: http://hdl.handle.net/10962/422389 , vital:71938
- Description: This research aims to enhance the use of large-scale assessment data in South Africa by evaluating assessment validity by means of multidimensional item response theory and its associated statistical techniques, which have been severely underutilised. Data from the 2014 administration of the grade 6 Mathematics annual national assessment was used in this study and all analyses were conducted using the mirt package in R. A two parameter logistic item response theory model was developed which indicated a clear alignment between the model parameters and difficulty specifications of the test. The test was found to favour learners within the central band on the ability scale. An exploratory five dimensional item response theory model was then developed to investigate the alignment with the test specifications as evidence for construct validity. Significant discrepancies between the factor structure and the specifications of the test were identified. Notably, the results suggest that some items measured an ability that was not purely mathematical, such as reading ability, which would distort the test’s representation of Mathematics ability, disadvantage learners with lower English literacy, and reduce the construct validity of the test. Further validity evidence was obtained by differential item functioning analyses which revealed that fourteen items function differently for learners from different provinces. Although possible reasons for the presence of differential item functioning among provinces were not discussed, its presence provided sufficient evidence against the validity of the test. In conclusion, multidimensional item response theory provided an effective and rigorous approach to establishing the validity of a large-scale assessment. To avoid the pitfalls of the annual national assessments, it is recommended that this multidimensional item and differential item functioning techniques are utilised for the development and evaluation of future national assessment instruments in South Africa. , Thesis (MSc) -- Faculty of Science, Statistics, 2023
- Full Text:
- Date Issued: 2023-03-29
Academic performance and cognitive critical thinking skills of certificate in theory of accounting students at Nelson Mandela University
- Authors: Pienaar, Joné
- Date: 2019
- Subjects: Academic achievement , Thought and thinking , Cognitive learning , Educational tests and measurements , Academic achievement -- South Africa -- Port Elizabeth -- Evaluation
- Language: English
- Type: Thesis , Masters , MCom
- Identifier: http://hdl.handle.net/10948/43196 , vital:36759
- Description: With advances in computerisation, the skills that accountants need to remain relevant in an ever-changing world deserves consideration. Literature indicates that “critical thinking skills” form part of the required skill set. However, the development and assessment of critical thinking skills in the accounting curriculum has not received sufficient attention. This study focuses on evaluating the assessment of accounting students’ critical thinking skills, specifically those aspiring to be CAs (SA), who are in their final year of university education. The primary objective of this study is to establish whether a relationship exists between cognitive critical thinking skills and academic performance of CTA students at Nelson Mandela University. The research followed a positivistic mixed method research methodology. Using the Watson-Glaser Critical Thinking Appraisal (W-GCTA), the cognitive critical thinking ability of the sample (CTA students (n=60)) was determined and compared to their academic performance using various statistical techniques. Since the proxy for academic performance was the final marks of which the year-end examination forms a very large component, the examiners of each of the modules comprising the CTA programme also completed a questionnaire wherein they indicated the critical thinking skills assessed, and the assessment characteristics used in the examination papers. The results indicate that a relationship exists between cognitive critical thinking skills and academic performance of the sample in three of the four modules of the CTA programme: Accounting, Taxation and Estate Planning and Management Accounting. The questionnaire feedback indicates that cognitive critical thinking skills were assessed in the CTA programme to some extent, but that focus was placed on skills not assessed by the W-GCTA.
- Full Text:
- Date Issued: 2019
- Authors: Pienaar, Joné
- Date: 2019
- Subjects: Academic achievement , Thought and thinking , Cognitive learning , Educational tests and measurements , Academic achievement -- South Africa -- Port Elizabeth -- Evaluation
- Language: English
- Type: Thesis , Masters , MCom
- Identifier: http://hdl.handle.net/10948/43196 , vital:36759
- Description: With advances in computerisation, the skills that accountants need to remain relevant in an ever-changing world deserves consideration. Literature indicates that “critical thinking skills” form part of the required skill set. However, the development and assessment of critical thinking skills in the accounting curriculum has not received sufficient attention. This study focuses on evaluating the assessment of accounting students’ critical thinking skills, specifically those aspiring to be CAs (SA), who are in their final year of university education. The primary objective of this study is to establish whether a relationship exists between cognitive critical thinking skills and academic performance of CTA students at Nelson Mandela University. The research followed a positivistic mixed method research methodology. Using the Watson-Glaser Critical Thinking Appraisal (W-GCTA), the cognitive critical thinking ability of the sample (CTA students (n=60)) was determined and compared to their academic performance using various statistical techniques. Since the proxy for academic performance was the final marks of which the year-end examination forms a very large component, the examiners of each of the modules comprising the CTA programme also completed a questionnaire wherein they indicated the critical thinking skills assessed, and the assessment characteristics used in the examination papers. The results indicate that a relationship exists between cognitive critical thinking skills and academic performance of the sample in three of the four modules of the CTA programme: Accounting, Taxation and Estate Planning and Management Accounting. The questionnaire feedback indicates that cognitive critical thinking skills were assessed in the CTA programme to some extent, but that focus was placed on skills not assessed by the W-GCTA.
- Full Text:
- Date Issued: 2019
Investigating a way of teaching transformation geometry in grade 9 applying van Hiele’s theory and Kilpatrick’s model : a case study
- Authors: Geja, Nokuzola Hlaleleni
- Date: 2015
- Subjects: Hiele, Pierre M. van , Kilpatrick, Jeremy , Transformations (Mathematics) , Geometry -- Study and teaching (Secondary) , Geometry -- Mathematical models , Educational tests and measurements
- Language: English
- Type: Thesis , Masters , MEd
- Identifier: vital:2066 , http://hdl.handle.net/10962/d1020601
- Description: Transformation geometry has been neglected in our schools because teachers are often not proficient enough to teach it, as it was not part of the syllabus during their training. The study investigates effective ways of teaching transformation geometry in grade 9, applying van Hiele’s theory (1986) of geometry teaching and learning and Kilpatrick’s model of mathematical proficiency. The teaching programme activities require consistent use of physical manipulatives by the teacher for effective teaching, learning and understanding of geometric concepts. The type of study is a case study. Data collection tools are: - baseline evaluation, teacher & learner interviews (pre & post programme intervention) and observation (pre & post) during the implementation of the teaching programme. Results were analysed both qualitatively and quantitatively. My research findings show some improvement of learner performance after the application of the programme. Baseline evaluation shows that some learners attained below and above 30%. Interviews showed that some learners had problems before the implementation of the programme and some problems were eliminated by the use of the programme activities and learning progression was evident. Learner performance showed that learners had acquired some knowledge and critical thinking and reasoning skills, reflection skills, communication through LOLT improved, commitment to activities of the programme and teaching practice had improved. Learner performance showed that a learner can be in two different levels at the same time. Consistent use of manipulatives resulted in effective teaching and learning of geometry in grade 9. The results of this research support other researchers’ views of teaching geometry using van Hiele’s theory (1986) and Kilpatrick et al. (2001). Shaw (2002) argues that teaching geometry with manipulatives enhances conceptual understanding by the learner. In my opinion, it also promotes immediate intervention by the teacher as soon as the learner picks an incorrect object. The project enhanced and improved levels of communication between the learner, teacher and others in the classroom.
- Full Text:
- Date Issued: 2015
- Authors: Geja, Nokuzola Hlaleleni
- Date: 2015
- Subjects: Hiele, Pierre M. van , Kilpatrick, Jeremy , Transformations (Mathematics) , Geometry -- Study and teaching (Secondary) , Geometry -- Mathematical models , Educational tests and measurements
- Language: English
- Type: Thesis , Masters , MEd
- Identifier: vital:2066 , http://hdl.handle.net/10962/d1020601
- Description: Transformation geometry has been neglected in our schools because teachers are often not proficient enough to teach it, as it was not part of the syllabus during their training. The study investigates effective ways of teaching transformation geometry in grade 9, applying van Hiele’s theory (1986) of geometry teaching and learning and Kilpatrick’s model of mathematical proficiency. The teaching programme activities require consistent use of physical manipulatives by the teacher for effective teaching, learning and understanding of geometric concepts. The type of study is a case study. Data collection tools are: - baseline evaluation, teacher & learner interviews (pre & post programme intervention) and observation (pre & post) during the implementation of the teaching programme. Results were analysed both qualitatively and quantitatively. My research findings show some improvement of learner performance after the application of the programme. Baseline evaluation shows that some learners attained below and above 30%. Interviews showed that some learners had problems before the implementation of the programme and some problems were eliminated by the use of the programme activities and learning progression was evident. Learner performance showed that learners had acquired some knowledge and critical thinking and reasoning skills, reflection skills, communication through LOLT improved, commitment to activities of the programme and teaching practice had improved. Learner performance showed that a learner can be in two different levels at the same time. Consistent use of manipulatives resulted in effective teaching and learning of geometry in grade 9. The results of this research support other researchers’ views of teaching geometry using van Hiele’s theory (1986) and Kilpatrick et al. (2001). Shaw (2002) argues that teaching geometry with manipulatives enhances conceptual understanding by the learner. In my opinion, it also promotes immediate intervention by the teacher as soon as the learner picks an incorrect object. The project enhanced and improved levels of communication between the learner, teacher and others in the classroom.
- Full Text:
- Date Issued: 2015
Exploring health literacy assessment : the relexicalisation of a health literacy test from the U.S. for application in a South African population
- La Rose, Christopher Michael
- Authors: La Rose, Christopher Michael
- Date: 2004
- Subjects: Rapid Estimate of Adult Literacy in Medicine , Educational tests and measurements , Literacy -- South Africa--Evaluation , Health education -- South Africa -- Evaluation
- Language: English
- Type: Thesis , Masters , MA
- Identifier: vital:2373 , http://hdl.handle.net/10962/d1005708 , Rapid Estimate of Adult Literacy in Medicine , Educational tests and measurements , Literacy -- South Africa--Evaluation , Health education -- South Africa -- Evaluation
- Description: The Rapid Estimate of Adult Literacy in Medicine (REALM) is a standardised health literacy test developed in the United States that has been previously administered and researched in South Africa (Lecoko, 2000). It is an objective vocabulary test which uses item recognition of 66 health-related words where recognition is measured by pronunciation. It is designed as a screening instrument to identifY the health literacy levels of patients in clinics. Lecoko (2000) found the REALM to be largely inapplicable in a South African setting, in that only eight out of the 66 words could be deemed acceptable, in that they could be either both adequately pronounced and adequately comprehended or both inadequately pronounced and inadequately comprehended. This may have occurred for a number of reasons, including the administration of the test in a population for whom English is a second or other language, and inaccurate measurement of pronunciation and comprehension of words, with a key problem being that the choice of words in the original REALM may not accurately represent the range of conditions and issues in a South African healthcare setting. This thesis was therefore premised on the principled relexicalisation of the REALM, that replacement of the words used in the test, using a sample gathered from health information and promotion texts in local clinics, would improve its applicability. In this regard, an exhaustive sample was gathered and analysed and 66 new words were chosen. The test was also modified to include a more principled approach to pronunciation and comprehension issues, and to account for language proficiency differences in administering an English language test in an English second language population. This modified test, referred to as the REALM-M was administered to a group of respondents who were statistically similar to the group to whom the REALM had originally been administered, and the results were compared. It was found that relexicalisation increased the number of acceptable words on the test from eight to 38. However, researching the key discourses surrounding health literacy and comparing these with current discourses about literacy beyond the field of health care revealed that despite improved content validity over the REALM, the REALM-M lacked construct validity. This provided the opportunity to discuss the discourses of health literacy and to suggest the application of alternative paradigms in this field.
- Full Text:
- Date Issued: 2004
- Authors: La Rose, Christopher Michael
- Date: 2004
- Subjects: Rapid Estimate of Adult Literacy in Medicine , Educational tests and measurements , Literacy -- South Africa--Evaluation , Health education -- South Africa -- Evaluation
- Language: English
- Type: Thesis , Masters , MA
- Identifier: vital:2373 , http://hdl.handle.net/10962/d1005708 , Rapid Estimate of Adult Literacy in Medicine , Educational tests and measurements , Literacy -- South Africa--Evaluation , Health education -- South Africa -- Evaluation
- Description: The Rapid Estimate of Adult Literacy in Medicine (REALM) is a standardised health literacy test developed in the United States that has been previously administered and researched in South Africa (Lecoko, 2000). It is an objective vocabulary test which uses item recognition of 66 health-related words where recognition is measured by pronunciation. It is designed as a screening instrument to identifY the health literacy levels of patients in clinics. Lecoko (2000) found the REALM to be largely inapplicable in a South African setting, in that only eight out of the 66 words could be deemed acceptable, in that they could be either both adequately pronounced and adequately comprehended or both inadequately pronounced and inadequately comprehended. This may have occurred for a number of reasons, including the administration of the test in a population for whom English is a second or other language, and inaccurate measurement of pronunciation and comprehension of words, with a key problem being that the choice of words in the original REALM may not accurately represent the range of conditions and issues in a South African healthcare setting. This thesis was therefore premised on the principled relexicalisation of the REALM, that replacement of the words used in the test, using a sample gathered from health information and promotion texts in local clinics, would improve its applicability. In this regard, an exhaustive sample was gathered and analysed and 66 new words were chosen. The test was also modified to include a more principled approach to pronunciation and comprehension issues, and to account for language proficiency differences in administering an English language test in an English second language population. This modified test, referred to as the REALM-M was administered to a group of respondents who were statistically similar to the group to whom the REALM had originally been administered, and the results were compared. It was found that relexicalisation increased the number of acceptable words on the test from eight to 38. However, researching the key discourses surrounding health literacy and comparing these with current discourses about literacy beyond the field of health care revealed that despite improved content validity over the REALM, the REALM-M lacked construct validity. This provided the opportunity to discuss the discourses of health literacy and to suggest the application of alternative paradigms in this field.
- Full Text:
- Date Issued: 2004
- «
- ‹
- 1
- ›
- »