Quality Analysis of Intelligence Structure Test 2000 Revision (IST 2000R) Items in Indonesian

Authors

DOI:

https://doi.org/10.15575/psy.v9i1.14977

Keywords:

Intelligence Structure Test (IST), Item Response Theory, item analysis

Abstract

This study aimed to test the quality of item analysis in the Indonesian IST 2000R using the Item Response Theory (IRT) method and participants comprising 1780 students aged 16-20. IRT 3PL analysis and DIF examination showed that the Indonesian IST 2000R had good item quality, with an improvement in IST 70. The discriminatory power parameter (É‘) of the verbal category items was high. Furthermore, the difficulty level parameter (b) indicated that more than 50% of the items on the IST 2000R are in the average category. In the guessing probability parameter (c), 8 of 9 subtests showed that more than 50% of the items are in a low category. Overall, the Indonesian IST 2000R has a small gender bias because all subtests obtained more than 60% in category A.

References

Abedalaziz, N., Leng, C. H., & Alahmadi, A. (2014). Detecting a gender-related differential item functioning using transformed item difficulty. Malaysian Online Journal of Educational Sciences, 2, 16-22.

Adedoyin, O. O., & Mokobi, T. (2013). Using IRT psychometric analysis to examine the quality of junior certificate mathematics multiple choice test items. International Journal of Asian Social Science, 3(4), 992–1011.

Adinugroho, I. (2016). Pengujian properti psikometrik intelligenz struktur test subtes kemampuan spasial dua dimensi (form auswahl): Studi pada dua SMA swasta di Jakarta. Jurnal Ilmiah Psikologi MANASA, 5(2).

Ayanwale, M. A. (2019). Efficacy of item response theory in the validation and score ranking of dichotomous and polytomous response mathematics achievement tests in Osun State, Nigeria (Doctoral Thesis Unpublished). Institute of Education, University of Ibadan.

Baker, F. B. (2001). The basics of item response theory second edition. ERIC Clearinghouse on Assessment and Evaluation.

Beauducel, A., Brocke, B., & Liepmann, D. (2001). Perspectives on fluid and crystallized intelligence: Facets for verbal, numerical, and figural intelligence. Personality and Individual Differences, 30(6), 977–994. https://doi.org/10.1016/S0191-8869(00)00087-8

Beauducel, A., Liepmann, D., Horn, S., & Brocke, B. (2010). Intelligence structure test. Hogrefe.

Cain, M. K., Zhang, Z., & Yuan, K. H. (2017). Univariate and multivariate skewness and kurtosis for measuring nonnormality: Prevalence, influence, and estimation. Behavior Research Methods, 49(5), 1716–1735. https://doi.org/10.3758/s13428-016-0814-1

Cohen, & Swerdlik. (2010). Psychological testing and assessment: An introduction to tests and. measurement, seventh edition. McGraw-Hill.

Foster, G. C., Min, H., & Zickar, M. J. (2017). Review of item response theory practices in organizational research: Lessons learned and paths forward. Organizational Research Methods, 20(3), 465-486. https://doi.org/10.1177/1094428116689708

Gierl, M. J., Bisanz, J., Bisanz, G. L., Boughton, K. A., & Khaliq, S. N. (2001). Illustrating the utility of differential bundle functioning analyses to identify and interpret group differences on achievement tests. Educational Measurement: Issues and Practice, 20(2), 26-36. https://doi.org/10.1111/j.1745-3992.2001.tb00060.x

Ibrahim, A. (2018). Differential item functioning: The state of the art. Jigawa Journal of Multidisciplinary Studies (JJMS), 1(1), 37-50.

Martín, E. S., Del Pino, G., & De Boeck, P. (2006). IRT models for ability-based guessing. Applied Psychological Measurement, 30(3), 183–203. https://doi.org/10.1177/0146621605282773

McGrory, S., Doherty, J. M., Austin, E. J., Starr, J. M., & Shenkin, S. D. (2014). Item response theory analysis of cognitive tests in people with dementia: A systematic review. BMC Psychiatry. https://doi.org/10.1186/1471-244X-14-47

Meyer, J. P. (2014). Applied measurement with jmetrik. Routledge. https://doi.org/10.4324/9780203115190

Obinne, A. D. E. (2012). Using IRT in determining test items prone to guessing. World Journal of Education, 2(1). https://doi.org/10.5430/wje.v2n1p91

Ogunsakin, I. B., & Shogbesan, Y. O. (2018). Item response theory (IRT): A modern statistical theory for solving measurement problems in the 21st century. International Journal of Scientific Research in Education (IJSRE), 11(3B), 627-635.

Quaigrain, K., & Arhin, A. K. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1), 1–11. https://doi.org/10.1080/2331186X.2017.1301013

Rahmawati, E. (2014). Evaluasi karakteristik psikometri intelligenz struktur test (IST) [Proceeding]. Seminar Nasional Psikometri, UMS, Indonesia.

Reilly, D., & Neumann, D. L. (2013). Gender-role differences in spatial ability: A meta-analytic review. Sex Roles, 68(9–10), 521–535. https://doi.org/10.1007/s11199-013-0269-0

Reilly, D., Neumann, D. L., & Andrews, G. (2022). Gender differences in self-estimated intelligence: Exploring the male hubris, female humility problem. Frontiers in Psychology, 13, 812483. https://doi.org/10.3389/fpsyg.2022.812483

Ring, P., Neyse, L., David-Barett, T., & Schmidt, U. (2016). Gender differences in performance predictions: Evidence from the cognitive reflection test. Frontiers in Psychology, 7, 1680. https://doi.org/10.3389/fpsyg.2016.01680

Sadhu, S., & Laksono, E. W. (2018). Development and validation of an integrated assessment for measuring critical thinking and chemical literacy in chemical equilibrium. International Journal of Instruction, 11(3), 557–572. https://doi.org/10.12973/iji.2018.11338a

Schulze, D., Beauducel, A., & Brocke, B. (2005). Semantically meaningful and abstract figural reasoning in the context of fluid and crystallized intelligence. Intelligence, 33(2), 143–159. https://doi.org/10.1016/j.intell.2004.07.011

Sirodj, D. A. N. (2018). Analisis kualitas item intelligence structure test (IST) melalui metode item response theory (IRT). SCHEMA (Journal of Psychological Research), 4(2), 98–108. https://doi.org/10.29313/schema.v4i2.4420

Tarigan, M., & Fadillah. (2021a). Properti pikometrik intelligenz struktur test subtes kemampuan numerik (rechenaufgaben dan zahlen reihen). Jurnal Psikologi Ilmiah, 13(2), 155–170. https://doi.org/10.15294/intuisi.v13i2.31839

Tarigan, M., & Fadillah, F. (2021b). Properti psikometri struktur inteligensi IST subtes verbal (Satzergaenzung, Wortauswahl, dan Analogien) berbahasa Indonesia. Jurnal Muara Ilmu Sosial, Humaniora, dan Seni, 5(1), 63-72. https://doi.org/10.24912/jmishumsen.v5i1.9623.2021

Yacob, A., Ali, N., Yusoff, M. H., MohdSaman, M. Y., & Hamzah, W. M. (2014). Personalized learning: An analysis using item response theory. World Academy of Science, Engineering and Technology, International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering, 8, 1107-1113.

Downloads

Additional Files

Published

2022-10-27

Issue

Vol. 9 No. 1 (2022): PSYMPATHIC

Section

Articles

Citation Check

License

The author whose published manuscript approved the following provisions:

1. The right of publication of all material published in the journal / published on the E-Journal of Psychology website is held by the editorial board with the knowledge of the author (moral right remains the author of the script).

2. The formal legal provisions for access to digital articles of this electronic journal are subject to the terms of the Creative Commons Attribution-ShareAlike license (CC BY-SA), which means that the Journal of Psychology reserves the right to save, transmit media / format, manage in database form, , maintain, and publish articles without requesting permission from the Author as long as it continues to include the name of the Author as the owner of Copyright.

3. Printed and published print and electronic manuscripts are open access for any purposes. In addition to these objectives, the editorial board shall not be liable for violations of copyright law.

Â