The Impact of Flipped Classroom Implementation in Mathematics Learning at Schools: Systematic Literature Review

Salah satu pembelajaran yang berpusat pada siswa dan sesuai dengan perkembangan teknologi adalah flipped classroom . Namun, terjadi inkonsistensi dalam desain dan implementasi flipped classroom dan dampaknya pada pembelajaran matematika. Tujuan dari studi ini menganalisis studi empiris secara sistematis tentang flipped classroom pada matematika sekolah untuk memahami dasar-dasar teoretis yang mengarah pada pendekatan berbeda untuk flipped classroom dan dampak flipped classroom pada pembelajaran siswa di kelas matematika. Metode yang digunakan adalah metode systematic literature review , mengikuti tujuh langkah yang disarankan oleh Cooper. Setelah terpilih 16 studi empiris kemudian dianalisis secara kualitatif dan hasilnya menunjukkan bahwa: (1) ada lebih banyak literatur yang diterbitkan tentang flipped classroom yang


INTRODUCTION
Flipped classroom is one method of implementing blended learning, which combines face-to-face learning with online learning. It says flipped because usually, the teacher actively serves teaching material inside the class and students do exercises at home individually, in flipped classroom time organization and management is flipped (Jeong et al., 2021) .
In its executions, activity in flipped classroom differentiates two phases which are out class and in class. The terms out class and in class are interpreted in a wide manner related to space and time. In out-class activity, students learn the material individually, asynchronously, and independently, before the scheduled learning hours. In this phase, ICT plays a fundamental role, because learning material is received by students through videos, pictures, graphics, or other iconic forms (Trujillo et al., 2019) . In this out-class activity, students interact with the learning material with self-paced learning (Roehl et al., 2013) . For example, students who already understand certain sub-materials can skip the video and look for additional learning materials from the internet. While students who do not understand can learn the content of the material with repetition (Dove & Dove, 2017;Roehl et al., 2013) .
The second phase is implemented during the scheduled learning hours where questions are asked and exercises are done; develop competencies and solve problems, where learning material content studied is used collaboratively and actively. By diverting the presentation of material out-class, more time in-class can be spent explaining difficult concepts or working on problems with guidance (DeLozier & Rhodes, 2017) . More individual guidance and students with special educational needs can be provided (Bishop & Verleger, 2013) .
There are many studies that report that learning with flipped classroom gives benefits. In learning outcomes flipped classroom can improve student learning achievement (Bhagat et al., 2016) ; student engagement (Khanova et al., 2015) ; motivation (Huang & Hong, 2016) ; critical thinking skill improvement (Van Vliet et al., 2015) ; focus on problem solving (Chen et al., 2015) and so on. Other benefits are time efficiency (Davies et al., 2013) students and teachers disposition (Kong, 2014;Lage et al., 2000) , reduce anxiety (Teo et al., 2014) as well as cost effectiveness (Mason et al., 2013) . Then flipped classroom learning can be chosen to be a solution to various problems in learning on the basis of its benefits.
Even though it is use then flipped classroom learning can be chosen to be a solution to various problems in learning on the basis of its benefits ful, there are a large number of students who still prefer traditional learning to flipped classrooms. Lin & Hwang (2019) examined research trends in flipped classrooms for medical courses by reviewing 60 studies from 2008 to 2017. They found that the use of in-class activities was inconsistent across most of the studies. Discussions, doing exercises, problem-based activities, and group project activities are the most popular in-class activities in flipped classrooms in medical courses.
A review was also carried out on a study conducted by Van Alten, Phielix, Janssen, & Kester (2019) which found that there is a positive effect even though it is small on learning outcomes but no effect is found on student satisfaction from using the flipped classroom. Even in a study conducted in the nursing class, it was found that there was no statistically significant difference between groups of students with flipped classroom learning and traditional classes in measured learning outcomes The Impact of Flipped Classroom Implementation in Mathematics Learning at Schools: Systematic Literature Review (Harrington et al., 2015) . This shows that not all conditions benefit from the implementation of the flipped classroom as previously described.
Although many studies report the positive effects of implementing a flipped classroom, the evidence supporting flipped classrooms being able to improve students' academic performance and perceptions are still weak. (Bishop & Verleger, 2013;Lo & Hew, 2017a;Zuber, 2016) This is caused by inconsistent theoretical frameworks, methods, and applied class activities (Lin & Hwang, 2019;Lo & Hew, 2017b;Zuber, 2016) . This inconsistency can be studied in more depth so that it can be seen how the theoretical framework, methods, and class activities in the implementation of the flipped classroom so that the desired benefits are obtained.
Meanwhile, a review according to individual discipline is necessary because the effect of flipped classroom may depend on the nature of the subject (Giannakos et al., 2014) . According to Gafoor and Sarabi (2015) students usually perceive mathematics as a subject that is much more complicated than other subjects. Mathematics usually studies concepts in an abstract way. More effort is needed in mathematics to understand symbols, notations, and concepts in terms of their depth and precision (Gafoor & Sarabi, 2015) . Repeated practice and external support, such as teacher feedback and student-to-student discussions, are necessary for students to organize their reasoning and justify their planning strategies. (Gafoor & Sarabi, 2015;Kosko & Miyazaki, 2012;Lev S Vygotsky, 1987;1978) . The flipped classroom effect on mathematics subject needs to be specifically reviewed.
In learning mathematics, many studies have shown the benefits of using a flipped classroom at all levels of education (Freeman et al., 2014;Hwang & Lai, 2017;Muir & Geiger, 2016) . This is because using a flipped classroom in mathematics learning allows higher levels of Bloom's taxonomy to be done in class, such as analysis, which requires more discussion, making face-to-face classes more beneficial. Following the implementation of this didactic strategy in junior high school mathematics classes, substantial improvements were found in student evaluations and attitudes, verifying improvements in motivation and skills in analysis and graphical representation. (Belmonte et al., 2019) . This is also reinforced by Bhagat et al. (2016) findings which add impact on improving academic performance and student motivation in mathematics.
Among the many study results stating the benefits of the flipped classroom in learning mathematics, there were also those who disproved this. The effect of the flipped classroom on mathematics is still ambiguous in terms of academic performance and student perceptions (Fung et al., 2021) . There is insufficient evidence to conclude that flipped classrooms are better than traditional approaches. Although six of the nine studies reported positive effects, the aim of this study was to systematically analyze empirical studies of flipped classrooms in school mathematics to understand the theoretical basis leading to different approaches to flipped classrooms and the impact of flipped classrooms on student learning in mathematics classes, four of them had no control group. Likewise related to student perceptions that are still unclear. There are studies that state that the use of flipped classrooms in learning mathematics is liked by students (Buch & Warren, 2017;McGivney-Burelle & Xue, 2013 ) also there are those who find that students do not like it (Boevé et al., 2017;Strayer, 2012) . It was found that the inconsistency of the impact of the application of the flipped classroom in the field of mathematics studies could be caused by several factors, namely the differences in activities carried out both in-class and out-class activities led to different results. The existence of inconsistencies regarding research results related to the use of flipped classrooms in learning mathematics needs to be investigated. So this study aims to systematically analyze empirical studies of flipped classrooms in school mathematics to understand the theoretical basis that leads to different approaches to flipped classrooms and the impact of flipped classrooms on student learning in mathematics classes.

METHOD
Based on the ideas stated above, this article is part of the systematic literature review method, which presents a summary of the literature that is summarized and analyzed using objectives, explicit and replica techniques (Cooper, 2015) . To ensure the quality of this study, this systematic review followed the seven steps suggested by Cooper (2015) . Among them are (1) formulating the problem; (2) searching for literature; (3) collecting information from studies; (4) evaluating the quality of studies; (5) analyzing and integrating study results; (6) interpreting the evidence and (7) presenting the results.
The literature search was conducted on the Scopus international database. This database was chosen because of its potential and international reputation, as well as the criteria they use to index articles. In searching for literature, the help of the publish or perish application is used with the keywords: Mathematics learning and title word: Flipped Classroom. The article search was carried out on March 27, 2022, and without limiting the year to the literature search, 125 articles were obtained. Next is filtering. To carry out further screening, a list of inclusion criteria was made as a research sample limit. The following are the criteria: With the criteria in Table 1 above, screening was carried out in order to obtain 19 articles that would be analyzed as literature. The article screening process follows the following flowchart: In analyzing the literature, which are the 16 studies that were submitted for review, the Atlas.ti 8 software was used. Atlas.ti is a user-friendly Windows-based qualitative data analysis software (S. Hwang, 2008) . In data processing, each data in the literature is coded to make it easier for researchers later to recall the data as material for discussion in their research.
The following is a list of literature as selected studies for review.

RESULTS AND DISCUSSION
This review resulted in 16 empirical studies of school mathematics classrooms. The main findings drawn from the analysis of this study are presented in the following sections, which are firstly related to an overview of the selected literature regarding school years and levels. Then, it presents the main findings with respect to each research question by describing similarities in the design and implementation of flipped classrooms in mathematics classes, reporting various theoretical frameworks that support the flipped classroom design approach, and summarizing the effect of flipped classrooms on students' mathematics learning outcomes.
Eligible articles are sorted and grouped by year and school level as shown in Table 3. Regardless of class level, most studies are at the high school level. The sum is equal to the sum of the two levels below it. Studies are at least carried out at the primary school level. This is due to the implementation of the flipped classroom which requires mastery of technology in learning, as well as independent learning from students which is of course a challenge in itself when implemented at the primary school level. Even so, the data shows that studies are carried out at every level, meaning that this flipped classroom model can be implemented at various levels of education.
Based on the year the articles were published, the highest number is in 2019. In that year there were six studies with the implementation of flipped classrooms at both the primary school, middle school, and high school levels.
The Impact of Flipped Classroom Implementation in Mathematics Learning at Schools: Systematic Literature Review In this section, we present various in-class and out-class activities that are used in flipped classrooms. An overview of out-class and in-class activities is presented in Figure 1 and Figure 2.

Figure 2. Various Out-class Activities
Based on Figure 2 above which is the result of analysis using the atlas.ti 8 software, data is obtained that all (N = 16) articles report activities that occur out-class. Almost all articles (N=14) reported that out-class activities were carried out through videos/films. There are two articles that do not mention out-class activities through videos namely (Rahman et al., 2016) which mention out-class activities through manipulation materials and (Rahman et al., 2018) which mention out-class activities as independent learning activities. The two articles did not explain in detail about these activities, only mentioning that the activities were carried out with the help of the internet.
As an activity that is widely chosen, learning using a video lecturer does have advantages including (1) the video can be played back and allows students to repeat the instructor's explanation; (2) Videos can be viewed at the time, location, and environmental conditions of the student's choice; (3) Their portability allows listening and learning without the distractions that often accompany classroom learning (Brecht et al., 2008) . However, learning activities through videos will be effective if the videos are received and used by students. Providing support from parents in the form of the necessary devices and internet access is also an important thing that can be done to maximize the benefits of out-class learning, which in this case is online learning. Furthermore, teacher communication with parents as household supporters about how online learning should be carried out must also be considered to understand their role in helping their children's education. (John et al., 2021) .
Regarding in-class activities, as shown in Figure 3, it was found that only 14 literature reported the activities they carried out in in-class sessions. Activities carried out in in-class sessions are a continuation of out-class activities. Both are a series of interrelated learning. Out-class activities that occur earlier are a means for students to receive the material they will need later for fluency in inclass activities. In-class activities reinforce what has been received as understanding during out-class activities.

Figure 3. Various In-class Activity
There are similarities in class activities reported in several literature. Eight of the literature states that the in-class activities they carry out are through group learning, they learn through interaction The Impact of Flipped Classroom Implementation in Mathematics Learning at Schools: Systematic Literature Review with friends (Lo et al, 2017;Esperanza et al., 2016;Fedistia et al., 2019;Jarrah & Diab, 2019;Ku et al., 2019;Lai & Hwang, 2016;Rahman et al., 2016;Rahman et al., 2018). Two other literature use project based learning as an in-class activity (Mohamed et al., 2019;Ramadhani & Fitri, 2020). The other three focus more on students' practice in problem solving (Ni et al., 2015;Sharkia & Kohen, 2021;Wei et al., 2020). Meanwhile, one article only wrote that in-class activities were carried out as a conclusion to out-class activities, which is video lectures (Cevikbas & Kaiser, 2020).
There are no specific provisions regarding activities that must occur in in-class or out-class sessions. Teachers can design learning activities in a flipped classroom according to the learning outcomes to be achieved. The design of teacher-designed learning may be influenced by the teacher's view of the theoretical framework that supports the flipped classroom.
The main aim of this review is to map the theoretical framework used in the selected studies for the flipped classroom design. Of the 16 literature we examined, only 11 reported a conceptual or theoretical framework, and the other five did not describe the theoretical framework underlying the flipped classroom design used. The description of the theoretical framework of the 11 articles is as follows:

Figure 4. The theoretical framework used
Overall, there are six theoretical frameworks used, namely the theory of first principle of instruction (Lo et al, 2017), constructivism (Ni et al., 2015, social constructivism (Cevikbas & Kaiser, 2020;6, 10, 11, 12), active learning (Fedistia et al., 2019;5;Sharkia & Kohen, 2021), technology in learning (Ramadhani & Fitri, 2020). The literature Ramadhani et al., (2019) state that the underlying theory is Conversation Theory. Conversation Theory supports constructivism theory which facilitates students in collaborating, communicating, and interacting as well as constructing their knowledge to improve learning outcomes and the quality of learning more effectively (Ahmed, 2016;Al-Huneidi & Schreurs, 2012) . The theory is in line with the theory of social constructivism. So we include it in the social constructivism theory group.
Regarding the design used in compiling flipped classroom learning with the underlying theoretical framework, the following is the mapping: In an analysis of the impact of using flipped classrooms in learning mathematics at school, we grouped the data into two groups, namely the impact on student learning outcomes and other impacts. An overview of these impacts is presented in Figure 4 and Figure 5.
Based on Figure 5, all literature (N=16) reports the positive impact of using a flipped classroom on mathematics learning outcomes. In the cognitive learning outcomes it is stated that the flipped classroom has an impact on student achievement, better performance, developing student potential, improving test results, increasing understanding of mathematical concepts as well as improving students' communication skills, critical and creative thinking. In addition to cognitive outcomes, the use of flipped classrooms also has an impact on self-confidence, learning independence, self-efficacy, satisfaction, perception, motivation, involvement, and enthusiasm of students in learning mathematics.
The Impact of Flipped Classroom Implementation in Mathematics Learning at Schools: Systematic Literature Review Based on Figure 5, there are other impacts outside of mathematics learning outcomes from the use of flipped classrooms. These impacts include changes in learning methods, effective learning processes, interactions created to create closer relationships between students, more efficient use of study time, and students enjoying mathematics learning time.
In cognitive learning outcomes, almost all literature (N = 15) reports a positive impact regardless of the underlying theoretical framework in designing the design. However, in terms of the impact of other learning outcomes, the literature with social constructivism theory reports a positive impact on student involvement, motivation, and enthusiasm. Meanwhile, in other impacts, flipped classroom learning with the theoretical framework of social constructivism is learning that is more interactive and creates closeness between students. The literature with an active learning theoretical framework reports the impact on positive student engagement and efficient use of study time.