Chemical Literacy Skill of High School Students on Solution Chemistry

This research aimed to develop a multiple-choice chemical literacy test on solution chemistry as an instrument that specifically focuses on learners' knowledge and competency aspects; applying the test instrument by surveying 12th-grade high school students. The development of the test instrument adapted the R&D model by Borg and Gall. The Cronbach's Alpha reliability of the test was obtained after small-scale and wide-scale trials are 0.793 and 0.833, respectively, with 35 valid items. Data were collected using random cluster sampling with a total of 463 respondents. The result shows that the mean score of chemical literacy for the students in this study is 46.27. This indicated that the students' level of chemical literacy is low, and further improvement is required.


Introduction
Scientific literacy refers to a person's capability to understand laws, theories, phenomena, and scientific things in the surrounding environment in order to make practical decisions in their life (Dragos & Mih, 2015;Celik, 2014;Demir, 2016). This decision is based on the evaluation of information and arguments that presented by scientists and media through the use of evidence and collected data. This helps to properly participate in public policies that have a direct impact on their lives. scientific literacy was defined by Organization for Economic Cooperation and Development (OECD) as someone's ability to understand and engage in discussions on various issues in relation to science and technology. Three competencies were identified including, the explanation of scientific phenomena, designing scientific investigations or inquiries, and scientific interpretation of data. These three competencies require knowledge (content, procedural, and epistemic). Scientific literacy has an important role in making decisions on science issues that occur in society making it the main focus of science education (Bossér et al., 2015;Vieira & Vieira, 2014;Fives et al., 2014). Chemical literacy is a crucial part of scientific literacy, and one of the main objectives of chemistry education (Cigdemoglu et al., 2017).
The chemical literacy framework of Shwartz et al. (2006) matches with the OECD (2016) scientific literacy framework (Rahayu, 2017;Cigdemoglu, et al., 2017), but the framework of OECD is more popular, simpler, and more explicit than the framework of Shwartz et al. (Muntholib, 2020). Several chemical literacy instruments have been developed based on this framework. Shwartz et al. (2006) measured the five levels of chemical literacy of students in grades 10-12 in Israel. These involve scientific illiteracy, nominal scientific literacy, functional scientific literacy, conceptual scientific literacy, and multi-dimensional scientific literacy. The assessment tools developed were multiple-choice questions, open-ended questions, and the use of a Likert scale. According to these authors, it is difficult to assess all aspects and components of chemical literacy, and thus each level of chemical literacy is assessed using several assessment tools. The topics chosen were the properties of matter particles, chemical reactions, student's ability to use chemical laws and theories to explain phenomena, and the application of chemistry in everyday life. Thummathong and Tathong (2018)  Research has indicated that using a multiplechoice test can be one of the most efficient way to identify student conceptions (Wattanakasiwich et al., 2013). This type of testing allows time for researchers to perform data analysis and provides quantitative results that can be compared (Wattanakasiwich et al., 2013). However, weaknesses of multiplechoice test can be found. For example, students can answer fraudulently. To overcome this, the researcher may carry out several trials and an adequate number of participants so that a decent product can be obtained. Jurnal Tadris (2018) discovered concepts of learners in relation to thermochemistry. These studies revealed that multiple-choice tests had long been used, were in great demand, and widely administered. The aspects of knowledge and chemical literacy competencies focus on chemical knowledge and scientific cognitive inquiry abilities (Muntholib, 2020). These two aspects can be assessed using a multiplechoice test format as in the OECD (2015) framework.
Based on PISA survey data, Indonesia's achievement in 2006 was ranked 50 th out of 57 countries, in 2009 was ranked 57 th out of 63 countries, it was 64 th out of 65 countries in 2012, and it was 64 th out of 72 countries in 2015 (Rahayu, 2017). The latest PISA scientific literacy results in 2018, Indonesia's score has decreased compared to the previous year, thus placing Indonesia in the 70 th rank out of 78 countries (OECD, 2018). Such data indicate that the scientific literacy skills of Indonesian students need to be improved. The concept of science education is very broad, and thus specific subjects in science classrooms must contribute to the goal of increasing students' scientific literacy (Celik, 2014). For example, learning chemistry in the classroom contributes to improving chemical literacy in particular and scientific literacy in general. To improve students' literacy of science in Indonesia, inquiry-based learning has been suggested as an ideal scientific approach. Based on regulation number 22 of Kemdikbud (2016a), the stages of inquiry-based learning approach can be through (1) observing, (2) asking, (3) trying, (4) reasoning, and (5) communicating. However, in practice students have difficulty achieving learning goals that lead to improving their chemical literacy, especially designing and evaluating scientific inquiry and interpreting scientific data and evidence. This is in contrast of learning using the lecture method, when the goal is to convey important concepts to the learners (Muntholib, 2018).
One of the main topics of chemistry in Senior High School in Indonesia is chemistry solution (Kemdikbud, 2016a). Chemistry solution topic is spread over several levels (Kemdikbud, 2016b), the topic was taught in class 10 (electrical conductivity in solutions), class 11 (acid-base solutions, buffer solutions, and salt solutions), and class 12 (colligative properties of solutions). There are several applications of the chemical concept of solutions in everyday life, for example, the process of blood dialysis, filling the battery, body pH, food preservation, and gastric medicine. This shows that chemistry is closely related to everyday life. In addition, this topic also contains aspects of scientific literacy so that it can be used as a study for measuring student scientific literacy. This can be a reason to improve students' chemical literacy. Efforts to improve students' chemical literacy require a test instrument, and thus, a chemical literacy test of chemistry solution is developed for this research.
Development of chemical literacy test instrument assessment in the current research combines the OECD's scientific literacy framework (2016) and Shwartz's chemical literacy framework (2006), which can be seen in Table 1. Selection of this framework is due to the fact that OECD's framework is simple, explicit, and the most widely used, while the Shwartz framework is more specific and detailed. In this research, the focus of the instrument is on context, knowledge, and competency aspects, but without attitude aspects, although the assessment of attitude aspects can be done separately (Muntholib, 2018;Thummathong & Tathong, 2018). Jurnal Tadris

Instrument Development
The development of the test instrument in this research combines the framework of OECD (2016) and Shwartz et al. (2006) as can be shown in Table 1. It covered all the aspects that were suggested by the framework except the attitude aspects. Figure 1 highlights the stages of developing the chemical literacy test instrument that was adapted from the R&D model of Borg and Gall (1983).
The preliminary study stage emphasis on reviewing the literature of international journal articles and relevant books on chemistry solution, chemical literacy, scientific literacy, PISA, instrument development, assessment, and instrument validation.

Preliminary Study
Conducting literature and analyzing studies

Planning
Analyzing the chemical literacy aspects of OECD and Shwartz; analyzing chemistry solutions and types of instruments to be developed

Tabel 1. Chemical Literacy Framework of OECD (2016) and Shwartz et al. (2006) Framework OECD Framework Shwartz Context
Individual, local/national, and global issues. This may involve issues that have occurred daily or issues that have already occurred which require an understanding of science and technology

Chemistry in Context
• Knowledge covering the importance of chemistry in explaining phenomena in everyday life • Use of understanding of chemistry in everyday life, such as consumers of new products and technologies, how to make a decision-based chemistry, and the participation in social debates related to existing chemical issues • An understanding of the relationship between chemical innovation and social processes

Knowledge
An understanding of the main facts, concepts, and explanatory theory that builds the foundation of scientific knowledge. Knowledge in the form of knowledge about the universe and technological artifacts (content knowledge), knowledge of how ideas are generated (procedural knowledge), and understanding of the rationale that underlies these procedures and justification of their use (epistemic knowledge)

Scientific and Chemical Content Knowledge
An understanding of scientific ideas in general (chemistry is an experimental discipline, and chemistry is used to explain existing phenomena) and understanding of chemical characteristics/key ideas (explaining macroscopic phenomena in the molecular structure of matter, investigating the dynamics of processes, reactions, and changes in energy during the reaction process, describing life in terms of chemical structures and life system processes)

Competency
The ability to explain scientific phenomena, evaluate and design scientific inquiry, and the ability to interpret data and to provide scientific evidence

Higher-Order Learning Skills
The ability to ask questions, find related information when needed, and be able to analyze the disadvantages/benefits in every debate

Attitudes
A set of attitudes towards science demonstrated with an interest in science and technology, assessing a scientific approach to appropriate inquiry, and perceptions and awareness of environmental issues

Affective aspects
A view of chemistry and its applications which impartial and realistic. Expressions of interest in chemistry issues, especially in informal framework (such as TV programs) Then at the planning stage, an analysis of chemical literacy aspects was carried out based on OECD and Shwartz, an analysis of chemistry solution materials, the context of the problem by the chemistry curriculum of high schools in Indonesia, determining the type of instrument in the form of multiplechoice in Indonesian and the number of question items which will be developed.
The chemical literacy test instrument developed consists of five contexts with several item questions for each context covering aspects of knowledge and competence with distribution according to Table 2. The number of items developed in the initial draft was 40 items.
Initial testing was carried out by expert validators to determine the content validity of the test instruments. This validity includes four aspects of assessment, 1) suitability of indicators, aspects of literacy, and cognitive demand; 2) scientifically correct material; 3) clarity of the question on the question (clarify); 4) the effectiveness of the used language. In this validation, a score was allocated for each of the developed question items. Score 4 is given if all aspects of the assessment were complete. At this stage, the researcher receives suggestions/input from the validator and then makes improvements or revisions which were also discussed with the chemistry lecturer. Jurnal Tadris    Limited-scale testing was carried out for high school students (non-sample) in Sidoarjo Regency involving 63 students. The reliability of Cronbach's alpha obtained was 0.793 and 26 items were valid, 14 items were invalid and corrected by discussing them again with the chemistry lecturers.
Wide-scale testing was carried out involving 167 high school students (non-sample). At this stage, an analysis of the validity, reliability, item discrimination index, and item difficulty index were carried out. The reliability of Cronbach's alpha in this wide-scale trial showed a result of 0.833 with a total number 35 valid items, and five invalid items were discarded, resulting in a final draft of 35 items for the chemical literacy test.

Sampling
Respondents of this study were Senior High School students, Natural Science twelfth graders in Sidoarjo Regency. Collecting data was attained through cluster random sampling. From a total of 12 public of Senior High Schools, three of them were randomly selected in Sidoarjo Regency, then from each school, 4-5 classes were randomly selected, each class had 28-40 students. The total number of respondents was then 463. The respondents were confirmed to have received lessons on chemistry solution according to the indicators developed by direct interviews with the teaching chemistry teacher. Data analysis was carried out by calculating the percentage of respondents' answers, both at the chemical literacy level and the chemical literacy aspects level.

Instrument Development Results
The analysis of instrument development carried out after wide-scale trials, including validity, reliability, item discrimination index, and item difficulty index will be presented below.

Content Validity
The content validity was carried out by two expert validators who provided an assessment of the test instruments. The validation percentage based on the Eq. (1) was 91.56%. = ∑ 100% (1) Information: P = percentage, ∑x = total assessment score, and n = ideal score (highest score for each aspect x in the number of validators). This means that according to the experts' judgment the items were distributed in very high validity criteria. Therefore, these items were suitable to be used to retrieve data. The chemical literacy test instrument that was declared valid by the expert validator was then amended and tested to determine the validity of the items and their reliability. The trial was conducted twice on high school students (non-sample) in Sidoarjo Regency.

Item Validity
The validity of the items was carried out after a wide-scale trial with a total of 167 high Jurnal Tadris  school students (non-sample) from three schools. From the analysis using IBM SPSS Statistics 22, 35 items were valid, and five items were invalid (28, 30, 33, 36, and 40). The invalid items were discarded.

Reliability
An instrument is declared reliable if it shows the same measurement results at different times. If the reliability of an instrument is high, then the instrument can be used as a data collection tool. The reliability of Cronbach's Alpha obtained in the first trial was 0.793 while in the second trial (wide scale) it was 0,833 (very high). The results obtained statistically using SPSS statistics 22 indicate that the mean score of students in the first and second trial groups is not significant with the sig value. (2tailed) > 0.05 on the Independent Samples Test on the Equal Variances Assumed, while the value is significant. Levene's Test for Equality of Variances is sig. > 0.05 which indicates that the data variance between the first and second trials is homogeneous. So, it could be stated that the developed chemical literacy instrument was suitable for use to measure the chemical literacy.

Item Difficulty Index
The difficulty index is used for the difficulty level of an item or a question. If the difficulty index is in the range 0.00-0.30 then the items are classified as difficult; 0.31-0.70 items are classified as moderate; and 0.71-1.00 items are classified as easy (Arikunto, 2012). From the 35 valid item questions, 11 questions are classified as difficult (2, 5, 11, 12, 26, 31, 32, 34, 35, 37, and 39) and 24 medium questions (1, 3, 4, 6, 7, 8, 9, 10, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 27, 29, and 38), while the average item difficulty index value of 35 items question is 0.376. Overall, the average value of this difficulty index indicates that the chemical literacy test instrument is at the moderate criterion, even though this value is close to the difficult score limit of 0.30 based on the criteria suggested by Arikunto (2012).

Chemical Literacy Skills of High School Students on Solution Chemistry
The final revised instrument used to measure the ability of high school students after the second trial (wide scale). It had a reliability of 0.833 and contained 35 item questions. This test was then distributed to twelfth-grade high school students in Sidoarjo Regency using the cluster random sampling technique. The total respondents were 463 students. The results of measuring students' chemical literacy skills in chemistry solution were presented in Table 3. This result shows that the range of students' chemical literacy scores was 5.71 to 91.43 with an average score of 46.27.  Figure 2 is a graph of the students' literacy scores for each aspect of knowledge. The graph shows that the score of students' epistemic knowledge was the lowest between the scores obtained by students in the different aspects of knowledge. Overall, the results reveal that students' chemical literacy is still low based on all aspects of knowledge. These aspects of knowledge in scientific literacy are related to students' understanding Jurnal Tadris   The results of this study were in line with findings of the study conducted by previous researchers (Thummathong & Tathong, 2016). The average score of the first-year student chemical literacy was 43.58.
The following was a graph of the student's literacy score for each aspect of competency.

Figure 3. Students' Chemical Literacy Scores in each Aspect of Competency (EP, ED, and ID)
As shown in Figure 3 the results reveal that the students' scores on interpreting data and scientific evidence were the highest. Meanwhile, the EP and ED scores belong to the low category. The average score in the competency aspect was 48.11 with a small difference this result was almost similar the results of the research of Triyani and Azizah (2020) which found that the chemical literacy competency aspect, the average score of students was 60.00. This difference can be related to the small number of respondents in the previous study, which were 32 students in a science class and the different topic which was chemical equilibrium.
Content knowledge can be defined as the form of knowledge about the universe and technological artifacts (OECD, 2015). In this study, the instrument contained 18 items that stressed the content knowledge. The results indicated that the average score of students' chemical literacy skills in the aspect of content knowledge was 45.89. One of the students' achievements in the aspect of content knowledge can be seen in the following question. From the question in Figure 4, 42.33% of students were able to answer correctly by selecting the D option (80 mmol). The question was ranked at the medium cognitive demand category, but this question has the characteristics difficult of an item and has a good value of discrimination index.
Procedural knowledge is knowledge about how ideas are generated (OECD, 2015). There were seven items on the aspects of procedural knowledge in the developed chemical literacy test instrument. The results of this study indicate that the average score of students' chemical literacy skills in the aspect of procedural knowledge was 46.85. One of the students' achievements in the aspects of procedural knowledge can be seen in the following question.

Figure 5. Example Question of Procedural Knowledge Item
In the example in Figure 5, the analysis showed that 57.45% of students answered correctly by selecting the D option (there was a bright light). The item was at the medium cognitive demand category that was classified as a difficult question and had a good value of discrimination index.
Epistemic knowledge is knowledge about the understanding of the rationale underlying scientific procedures and decisions or considerations for their use (OECD, 2015). There were 10 items on the aspects of epistemic knowledge in the chemical literacy test instrument developed. The results of this study indicate the average score of students' chemical literacy skills in epistemic knowledge aspect was 41.80.
One of the students' achievements in epistemic knowledge can be seen in Figure 6. In this question, 69.98% of students answered correctly by selecting the option C (CO2 gas ionizes). The question is medium cognitive demand category with characteristics of at moderate level and had a good value of discrimination index. In this study, it was concluded that students' epistemic knowledge was the lowest in comparison to content or epistemic knowledge. This shows that students have difficulty in analyzing and relating concepts about how knowledge of an idea was scientifically produced. There were eight items of competency aspect with regard to scientific explanation of a phenomena. The results of this study indicate that the average score of students' chemical literacy skills in competence aspects to explain phenomena scientifically was 46.62.
One of the students' achievements in competency aspect can be seen in Figure 7. In this question, 50.76% of students answered correctly by selecting the option D (ii and iii). The question was the high cognitive demand. Based on the data from the results of the analysis of the question carried out after the second trial on a large scale, this question had the characteristics of the item that was classified as moderate and had a satisfactory value of discrimination index.
There were 17 items on the competency aspect of evaluating and designing scientific inquiry in the chemical literacy test instrument. The results of this study revealed that the This is an open access article under CC-BY-SA license (https://creativecommons.org/licenses/by-sa/4.0/) Chemical Literacy Skill of High School Students on Chemistry Solution average score of students' chemical literacy skills in the competency aspects of evaluating and designing scientific inquiry was 42.71.

Figure 7. Example Question of Explaining Phenomena Scientifically Item
One of the students' achievements in this competency aspect is shown in Figure 8. In this question, 42.98% of students answered correctly in option C (the type of solute used). The question was included in the cognitive demand category: medium.
Based on the data from the results of the analysis of questions carried out after the second trial on a large scale, this question had the characteristics of a question that was classified as moderate and had a good value of discrimination index.
There were ten items on the competency aspect of interpreting data and evidence scientifically in the chemical literacy test instrument that was developed. The results of this study indicate the average score of students' chemical literacy skills in the competency aspect of interpreting scientific data and evidence was 55.00. One of the students' achievements in this competency aspect can be seen in Figure 9. Example question can be seen in item 2. In this question, 28.72% of students answered correctly in option C (increase in CO2 concentration from 1920 to 1940 followed by an increase in earth temperature). The question was included in the cognitive demand category: medium. Based on the analysis after the second trial on a large scale, this question had characteristics of an item that was classified as difficult and had a satisfactory value of discrimination index.

Conclusion
This study discussed chemical literacy, especially in the aspects of knowledge and competence using chemical literacy test instrument on the chemistry solution topic. The test that was developed consisted of 35 valid items with a Cronbach's Alpha reliability of 0.833. This reliability coefficient considered at a very good value. Therefore, this test instrument was implemented to explore the chemical literacy skills of high school students in the Sidoarjo Regency.
The survey showed that the average score of students on this test was 46.27. This value indicated that students' chemical literacy still low and suggested the need for measures to be improved. This can be used as an evaluation of the learning carried out because the questions in the chemical literacy test instrument refer to the KD K-13 indicator. These findings suggest the need for ideas to develop chemical literacy among high school students in Indonesian context. This research has important implications for chemistry education: (1) multiple-choice chemical literacy tests can be used in chemistry education and needs to be developed to involve the attitude aspect.
(2) for the respondents it seems that they were not accustomed to working on scientific literacy questions and also questions in the context of issues that exist in society; (3) students' chemical literacy must be improved by learning that supports students in literacy; (4) a chemical literacy test should be used to assess student achievement so that they were trained to solve daily life problems scientifically. Some solutions to improve students' scientific literacy include applying environmental-based creative and collaborative learning, inquiry-based learning, metacognition training, authentic scientific inquiry, socio-critical learning, and problem solving, learning based on scientific processes, use of scientific texts, learning strategy approaches integrated, and argumentative methods (Karademir & Ulucinar, 2017). The study of Huann-shyang et al. (2012) found that there was a relationship between attitudes and scientific literacy, namely the interest, enthusiasm, and participation of students in learning science to improve their scientific literacy skills. Scientific literacy can be improved by increasing laboratory activities and equipping students with activities that emphasize affective aspects. Besides that, the existence of adequate scientific literacy instruments, both science as a whole and science in specific fields can be increase students' scientific literacy (Muntholib, 2020). Aksara.