Development of Integrated Assessment Instrument to Measure Critical Thinking Skills and Self-Efficiency in Acid-Base Concept

Authors

  • Nuryuliana Nuryuliana Departement of Chemistry Education, Postgraduate Program, Yogyakarta State University, Jl. Colombo No.1, Karang Malang, Yogyakarta, 55281
  • Anti Kolonial Prodjosantoso Departement of Chemistry Education, Postgraduate Program, Yogyakarta State University, Jl. Colombo No.1, Karang Malang, Yogyakarta, 55281

DOI:

https://doi.org/10.15575/jtk.v6i2.14872

Keywords:

acid-base, integrated assessment, self-efficacy

Abstract

The purpose of this research is to describe the stages of developing and validating a product which includes content validity and constructs validity. The method used is Research and Development (R&D) with define, design, develop, and disseminate (4D) model. The initial product was validated by material and learning evaluation expert lecturer. The suitability of the items evaluated by chemistry teacher based on indicators of competency achievement. The sample selection was determined by stratified random sampling of 170 students of 11th grade senior high school in Yogyakarta City. Content validity was analyzed using Aiken's V formula. The results of data analysis show that all components of the test instrument are valid with Aiken's V value greater than 0.80 with a significance of 0.05. Verification of construct validity was analyzed using exploratory factor analysis with the help of the SPSS version 21.0 program. The results prove that all components in the integrated assessment instrument are suitable to be used to measure the critical thinking skills and self-efficacy of students on acid-base concept.

Author Biographies

Nuryuliana Nuryuliana, Departement of Chemistry Education, Postgraduate Program, Yogyakarta State University, Jl. Colombo No.1, Karang Malang, Yogyakarta, 55281

PROGRAM PASCASARJANA PENDIDIKAN KIMIA

Anti Kolonial Prodjosantoso, Departement of Chemistry Education, Postgraduate Program, Yogyakarta State University, Jl. Colombo No.1, Karang Malang, Yogyakarta, 55281

PROGRAM PASCASARJANA PENDIDIKAN KIMIA

References

Astuti, S. R. D., Suyanta, Lfx, E. W., & Rohaeti, E. (2017). An integrated assessment instrument: Developing and validating instrument for facilitating critical thinking abilities and science process skills on electrolyte and nonelectrolyte solution matter. AIP Conference Proceedings, 1847, 1-6. Retrieved from https://aip.scitation.org/doi/abs/10.1063/1.4983909

Asy’ari, M., Ikhsan, M., & Muhali, M. (2018). Validitas instrumen karakterisasi kemampuan metakognisi mahasiswa calon guru fisika. Prisma Sains : Jurnal Pengkajian Ilmu Dan Pembelajaran Matematika Dan IPA IKIP Mataram, 6(1), 19-26. https://doi.org/10.33394/j-ps.v6i1.955

Bandura, A. (1990). Perceived self-efficacy in the exercise of personal agency. Journal of Applied Sport Psychology, 2(2), 128–163. https://doi.org/10.1080/10413209008406426

Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquivel, S. L. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research and Evaluation, 18(6), 1–13. https://doi.org/10.7275/qv2q-rk76

Chi, M. T. H., De Leeuw, N., Chiu, M. H., & Lavancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18(3), 439–477. https://doi.org/10.1016/0364-0213(94)90016-7

Danczak. S. M., Thompson. C. D., Overton. T. L. (2017). 'What does the term critical thinking mean to you?’ A qualitative analysis of chemistry undergraduate, teaching staff and employers’ views of critical thinking. Chemistry Education Research and Practice, 18, 420-434. https://doi.org/10.1039/C6RP00249H

Danczak, S. M., Thompson, C. D., & Overton, T. L. (2020). Development and validation of an instrument to measure undergraduate chemistry students’ critical thinking skills. Chemistry Education Research and Practice, 21(1), 62-78. https://doi.org/10.1039/C8RP00130H

Dalgety, J., & Coll, R. K. (2006). Exploring first-year science students’ chemistry self-efficacy. Chemistry Education Research and Practice, 1(1), 2-17. Retrieved from https://link.springer.com/article/10.1007/s10763-005-1080-3

Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12, 43–52. https://doi.org/10.1016/j.tsc.2013.12.004

Ennis, R. H., Millman, J., & Tomko, T. N. (1985). Cornell critical thinking tests (3rd ed.). Pacific Grove, CA: Midwest Publications.

Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Millbrae, CA: California Academic Press.

Ferrell, B., Phillips, M. M., & Barbera, J. (2016). Connecting achievement motivation to performance in general chemistry. Chemistry Education Research and Practice, 17(4), 1054-1066. https://doi.org/10.1039/C6RP00148C

Field, A. (2009). Discovering statistics using SPSS (3rd ed). London: Sage Publication Ltd.

Flaherty, A. A. (2020). A review of affective chemistry education research and its implications for future research. Chemistry Education Research and Practice, 21(3), 698–713. Retrieved from https://doi.org/10.1039/C9RP00200F

Frey, R. F., Cahill, M. J., & McDaniel, M. A. (2017). Students’ concept-building approaches: A novel predictor of success in chemistry courses. Journal of Chemical Education, 94(9), 1185-1194. https://doi.org/10.1021/acs.jchemed.7b00059

Galloway, K. R., & Bretz, S. L. (2015). Development of an assessment tool to measure students’ meaningful learning in the undergraduate chemistry laboratory. Journal of Chemical Education, 92(7), 1149–1158. https://doi.org/10.1021/ed500881y

Gerritsen-van Leeuwenkamp, K. J., Joosten-ten Brinke, D., & Kester, L. (2019). Students’ perceptions of assessment quality related to their learning approaches and learning outcomes. Studies in Educational Evaluation, 63(1), 72–82. https://doi.org/10.1016/j.stueduc.2019.07.005

Hambleton, R. K. & Swaminathan, H. (1985). Item response theory. Boston, MA: Kluwer Inc.

Hensen, C., & Barbera, J. (2019). Assessing affective differences between a virtual general chemistry experiment and a similar hands-on experiment. Journal of Chemical Education, 96(10), 2097–2108. https://doi.org/10.1021/acs.jchemed.9b00561

Hoffman, B., & Schraw, G. (2009). The influence of self-efficacy and working memory capacity on problem-solving efficiency. Learning and Individual Differences, 19(1), 91–100. https://doi.org/10.1021/acs.jchemed.9b00561

Kartowagiran, B., Mardapi, D., Purnama, D. N., & Kriswantoro, K. (2019). Parallel tests viewed from the arrangement of item numbers and alternative answers. Research and Evaluation in Education, 5(2), 169-182. http://dx.doi.org/10.21831/reid.v5i2.23721

Kivunja, C. (2014). Innovative pedagogies in higher education to become effective teachers of 21st century skills: Unpacking the learning and innovations skills domain of the new learning paradigm. International Journal of Higher Education, 3(4), 37–48. Retrieved from https://eric.ed.gov/?id=EJ1067585

Kriswantoro, K. Amelia, R. N. & Irwanto (2016). Peningkatan kompetensi calon pendidik kimia melalui item response theory: strategi menghadapi masyarakat ekonomi asean. Prosiding Seminar Nasional Kimia dan Pendidikan Kimia VIII Program Studi Pendidikan FKIP UNS, 64–73. Retrieved from https://osf.io/preprints/inarxiv/nqf79/download

Lewis. R. Aiken. (1985). Three Coefficients For Analyzing The Reliability And Validity Of Ratings. Educational and Psychological Measurement, 45, 131-141. https://doi.org/10.1177%2F0013164485451012

Mahanal, S., Zubaidah, S., Sumiati, I. D., Sari, T. M., & Ismirawati, N. (2019). RICOSRE: A learning model to develop critical thinking skills for students with different academic abilities. International Journal of Instruction, 12(2), 417-434. Retrieved from https://eric.ed.gov/?id=EJ1211048

Pyburn, D. T., Pazicni, S., Benassi, V. A., & Tappin, E. E. (2013). Assessing the relation between language comprehension and performance in general chemistry. Chemistry Education Research and Practice, 14(4), 524-541. https://doi.org/10.1039/C3RP00014A

Rasmawan, R. (2020). Development of SETS-based teaching materials in acid-base accompanied by critical thinking exercises and moral forming. EduChemia (Jurnal Kimia Dan Pendidikan), 5(2), 134. http://dx.doi.org/10.30870/educhemia.v5i2.7934

Redhana, I. W. (2019). Mengembangkan keterampilan abad ke-21 dalam pembelajaran kimia. Jurnal Inovasi Pendidikan Kimia, 13(1), 2239-2253. https://doi.org/10.15294/jipk.v13i1.17824

Retnawati, H. (2014). Teori respon butir dan penerapannya (untuk peneliti, praktisi pengukuran, dan pengujian mahasiswa pascasarjana). Yogyakarta: Nuha Medika.

Reynders, G., Suh, E., Cole, R. S., & Sansom, R. L. (2019). Developing student process skills in a general chemistry laboratory. Journal of Chemical Education, 96(10), 2109-2119. https://doi.org/10.1021/acs.jchemed.9b00441

Sadhu, S., & Laksono, E. W. (2018). Development and validation of an integrated assessment for measuring critical thinking and chemical literacy in chemical equilibrium. International Journal of Instruction, 11(3), 557-572. Retrieved from https://eric.ed.gov/?id=EJ1183420

Salirawati, D. (2011). Pengembangan instrumen pendeteksi miskonsepsi kesetimbangan kimia pada peserta didik SMA . Jurnal Penelitian dan Evaluasi Pendidikan, 15(2), 232-249. http://dx.doi.org/10.21831/pep.v15i2.1095

Saputri, N., Adlim, A., & Inda Rahmayani, R. F. (2018). Pengembangan Instrumen Penilaian Psikomotorik Untuk Praktikum Kimia Dasar. JTK (Jurnal Tadris Kimiya), 3(2), 114-124. Retrieved from https://core.ac.uk/download/pdf/234031096.pdf

Sari, L. P. N., Fajarianingtyas, D. A., & Hidayat, J. N. (2020). Validitas instrumen penilaian kemampuan berpikir kritis melalui model problem based learning menuju pembelajaran IPA abad ke-21. LENSA (Lentera Sains): Jurnal Pendidikan IPA, 10(2), 125–136. https://doi.org/10.24929/lensa.v10i2.121

Sarigoz, O. (2012). Assessment of the high school students’ critical thinking skills. Procedia - Social and Behavioral Sciences, 46, 5315-5319. https://doi.org/10.1016/j.sbspro.2012.06.430

Saxton, E., Belanger, S., & Becker, W. (2012). The critical thinking analytic rubric (CTAR): Investigating intra-rater and inter-rater reliability of a scoring mechanism for critical thinking performance assessments. Assessing Writing, 17(4), 251–270. https://doi.org/10.1016/j.asw.2012.07.002

Shavelson, R. J., Zlatkin-Troitschanskaia, O., Beck, K., Schmidt, S., & Marino, J. P. (2019). Assessment of university students’ critical thinking: next generation performance assessment. International Journal of Testing, 19(4), 337-362. https://doi.org/10.1080/15305058.2018.1543309

Shaw, A., Liu, O. L., Gu, L., Kardonova, E., Chirikov, I., Li, G., Hu, S., Yu, N., Ma, L., Guo, F., Su, Q., Shi, J., Shi, H., & Loyalka, P. (2020). Thinking critically about critical thinking: validating the russian heighten® critical thinking assessment. Studies in Higher Education, 45(9), 1933-1948. https://doi.org/10.1080/03075079.2019.1672640

Sumarni, W., Supardi, K. I., & Widiarti, N. (2018). Development of assessment instrument to measure critical thinking skills. IOP Conference Series: Materials Science and Engineering, 349(1), 1-11. Retrieved from https://iopscience.iop.org/article/10.1088/1757-899X/349/1/012066/meta

Sumarni, Woro, Sudarmin, S., Wiyanto, W., & Supartono, S. (2016). Preliminary analysis of assessment instrument design to reveal science generic skill and chemistry literacy. International Journal of Evaluation and Research in Education (IJERE), 5(4), 331-340. Retrieved from https://eric.ed.gov/?id=EJ1132223

Sternod. L., & French. B. (2015). The Watson–Glaser™ II critical thinking appraisal. Journal of Psychoeducational Assessment Pullman, USA: Washington State University.

Tan, K. S., Goh, N. K., & Chia, L. S. (2006). Bridging the cognitive-affective gap: teaching chemistry while advancing affective objectives: the singapore curricular experience. Journal of Chemical Education, 83(1), 59–63. https://doi.org/10.1021/ed083p59

Utami, B., Saputro, S., Ashadi, A., Masykuri, M., & Widoretno, S. (2017). Critical thinking skills profile of high school students in learning chemistry. International Journal of Science and Applied Science: Conference Series, 1(2), 124-130. http://dx.doi.org/10.20961/ijsascs.v1i2.5134

Utomo, T., & Ruijter, K. (1994). Peningkatan

dan pengembangan pendidikan. Jakarta: Gramedia Pustaka Utama.

Uzuntiryaki-Kondakçi, E., & Çapa-Aydin, Y. (2013). Predicting critical thinking skills of university students through metacognitive self-regulation skills and chemistry self-efficacy. Kuram ve Uygulamada Egitim Bilimleri, 13(1), 666–670. Retrieved from https://eric.ed.gov/?id=EJ1016667

Xu, X., & Raker, J. R. (2016). Self-efficacy and academic performance in first-semester organic chemistry: testing a model of reciprocal causation. Chemistry Education Research and Practice, 1(1), 1-9. https://doi.org/10.1039/C6RP00119J

Zimmerman, B. J. (2000). Self-Efficacy: An Essential Motive to Learn. Contemporary Educational Psychology, 25(1), 82–91.

ŽivkoviĿ, S. (2016). A model of critical thinking as an important attribute for success in the 21st century. Procedia – Social and Behavioral Sciences, 232(4), 102–108. https://doi.org/10.1039/C6RP00119J

Zoller, U. (2001). Alternative assessment as (critical) means of facilitating hocs-promoting teaching and learning in chemistry education. Chemistry Education: Research and Practice in Europe, 2(1), 9–17. https://doi.org/10.1039/B1RP90004H

Downloads

Published

2021-12-31

Issue

Vol. 6 No. 2 (2021)

Section

Articles

License

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a  Creative Commons Attribution-ShareAlike that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).