Evaluating the effectiveness of a computerized achievement test using learn smart for psychometric assessment under item response theory
Mimi Ismail
Educational Studies Department, University of Technology and Applied Science, Rustaq College of Education, Oman, and Educational Psychology, Zagazig University, Zagazig, Egypt.
https://orcid.org/0009-0006-8631-2559
Ahmed Al – Badri
Educational Studies Department, University of Technology and Applied Science, Rustaq College of Education, Oman.
https://orcid.org/0009-0006-4765-1259
Said Al – Senaidi
Educational Studies Department, University of Technology and Applied Science, Rustaq College of Education, Oman.
https://orcid.org/0009-0005-5306-5291
DOI: https://doi.org/10.20448/jeelr.v12i1.6676
Keywords: Effectiveness, Electronic test, Measurement accuracy, Psychometric assessment, Item response theory, Learn smart.
Abstract
This study aimed to reveal the differences in individuals’ abilities, their standard errors, and the psychometric properties of the test according to the two methods of applying the test (electronic and paper). The descriptive approach was used to achieve the study’s objectives. The study sample consisted of 74 male and female students at the University of Technology and Applied Sciences in Rustaq. An electronic test was built on the Learn Smart platform supported by artificial intelligence in psychological measurement. The results showed no statistically significant differences in the individuals' average ability estimates and their standard errors between the electronic and paper-based tests. Besides, the vocabulary difficulty estimates in the electronic test ranged between -2.562 and 2.007 and the vocabulary difficulty estimates in the paper-based test ranged between -3.483 and 2.194. All of them are within the acceptable range. The chi-square values for the electronic test items are not statistically significant except for items (8 and 10). On the other hand, all chi-square values for the items on the paper test are not statistically significant except the item numbers (3, 6, 8, 10 and 12) which are statistically significant at the 0.05 and 0.01 levels.