Business School Student Satisfaction with Emergency Remote Teaching

Maria del Carmen Arrieta Quispe¹ ; Beatrice Avolio Alecchi^2*

^1,2CENTRUM Católica Graduate Business School, Lima, Peru Pontificia Universidad Católica del Perú, Lima, Peru.

Abstract

This study analyzes Peruvian Business School students’ experiences with emergency remote teaching during the COVID-19 pandemic. We used the critical incident technique (CIT) to explore the salient factors on which students based their satisfaction and dissatisfaction with their online education. Data were collected through semi-structured interviews conducted with 27 MBA students. The results comprised 56 incidents – 25 satisfactory and 31 unsatisfactory – in the two major categories of Virtuality and the Teaching-Learning Process. This study contributes a new perspective on online education through the analysis, conducted through CIT, of the Peruvian Business School students’ experiences with online education during the COVID-19 pandemic.  Some of the most relevant positive incidents included the recording of classes to review them later; the existence of Learning Management Systems to support the development of online materials; the possibility of entering online classes instead of in-person classes, as well as the flexibility with regard to location. The study allows us to determine several critical aspects in the two identified categories that should be considered when improving future online or hybrid programs.

Keywords:Emergency remote teaching, COVID-19, Student satisfaction, Critical incident technique, Online learning, E-learning.

Contribution of this paper to the literature: This study contributes to the existing literature by analyzing Peruvian Business School students’ experiences with emergency remote teaching during the COVID-19 pandemic.

1. Introduction

On January 30, 2020, the World Health Organization declared the COVID-19 outbreak a global health emergency, and on March 11, 2020, it declared COVID-19 a global pandemic. As the pandemic spread, specific measures were adopted in different locations. Several countries declared states of emergency, including the adoption of measures such as social isolation and quarantine of the population. In the education sector, the pandemic forced educational institutions around the world to switch to online education as the only option to continue providing educational services (Sun, Tang, & Zuo, 2020). Crawford et al. (2020) studied how universities in twenty countries (both with developed and developing economies) responded to the quarantine necessitated by COVID-19 and found “different responses to a complex challenge” depending on the resources available to the organization and the cohort of students attending the organization. Some institutions decided to conform to the minimum measures demanded by the government, while others closed their in-person activities and switched entirely to online education. The study conducted by Hodges, Moore, Lockee, Trust, and Bond (2020) determined that the educational model implemented by these institutions can be considered "emergency remote teaching", referring to a "temporary shift of instructional delivery to an alternate delivery mode due to crises". According to Hodges et al. (2020) "emergency remote teaching" differs from “online learning” in that the latter is the result of careful instructional design and planning covering several dimensions, such as teaching method, pedagogy, instructor role, student role, synchronous and asynchronous communication, evaluation and feedback, among others.

Since there is no certainty when this pandemic will be brought under control, it is very likely that online education will continue for some time. Although online education is far from new, the COVID-19 crisis has marked the first mass attempt to implement it (Whittle, Tiwari, Yan, & Williams, 2020 ), which is why it is necessary to analyze the experience of students at different educational levels (primary school, secondary school, undergraduate, postgraduate) in the context of emergency online education. While online learning has been widely covered in the literature, the challenges of emergency remote teaching are still understudied (Krüger, Vogel, & Schnaubert, 2020). In addition, to the best of our knowledge, a thorough analysis has not yet been conducted of the experience and satisfaction of students in business schools during the COVID-19 pandemic, particularly in Latin America.

This study analyzes the experiences of students at a Peruvian Business School during the migration from in-person or blended teaching methods to the emergency remote teaching method. The study seeks to answer the following research question: What are the factors that have affected the (dis)satisfaction of Business School students with emergency remote teaching during the COVID-19 pandemic? The study of student satisfaction with emergency online teaching in Latin America is especially relevant because of its peculiarities compared to developed countries: Latin American countries face significant limitations related to their technological infrastructure and also have less well-developed educational systems compared to developed countries (Cortes Abarca, 2021). In addition, this study contributes a new perspective on online teaching by using the critical incident technique (CIT) to understand business school students’ experiences with online education during the COVID-19 pandemic. The results of this study will prove useful for policymakers in business schools to improve online learning services and to turn programs that are currently in-person into future hybrid or online programs.

This paper has five sections. The next section will offer a review of the literature, discussing studies relating to the critical success factors in the implementation of e-learning. The third section presents the research methodology, and the fourth section will describe the results. The fifth and final section presents a discussion of the findings and the conclusions.

2. Literature Review

The adoption of new communications technologies within the educational process, mainly for distance education purposes, has opened new areas of research, regarding not only the systematic process per se, but also specific topics such as students’ emotional response towards its implementation. Since it is considered a combined process with specific objectives and strategies aimed to address learning needs, overall student satisfaction with online learning has been studied since shortly after the introduction of the Internet in the late 20th century (Cravener, 1999; Hiltz, 1986; Schenk & Hoxhaj, 2019) . In the last decades, there has been a significant increase in the implementation of online education (Keržič, Tomaževič, Aristovnik, & Umek, 2019; Palvia et al., 2018). Accordingly, several studies have addressed students’ level of satisfaction with distance learning environments in different levels of education and countries around the world.

The previous literature on the subject categorizes the critical factors for online education into functional dimensions. The main functional dimensions mentioned in the literature are the technological dimension, organizational dimension, pedagogical dimension, e-learning content and general factors. The technological dimension includes the technologies that support pedagogical strategies, learning management systems, tools and technical support. The organizational dimension includes the management of learning resources, experience of project leaders, top management support, and technological and expert support. The pedagogical dimension includes the adoption of learner-centered pedagogical strategies. The e-learning content includes the design and delivery of the courses, the creativeness of the e-learning design and the course content. Finally, the general factors include the involvement of the teaching staff and students, the communication mechanisms and trust in the e-learning environment (Lin, Ma, & Lin, 2011; McPherson & Baptista-Nunes, 2007; Sridharan, Deng, & Corbitt, 2010) . Likewise, other studies have summarized the critical factors of online education in five dimensions: students, instructors, design and content, system and technology and institutional management. The student dimension includes their attitude towards e-learning, motivation, interaction and commitment to online learning. The instructor dimension includes their attitude, digital skills, appropriate timely feedback to students and communication skills. The design and content dimension includes course content and design, tools for learning, the curriculum, interactive learning activities and user-friendly, organized and understandable content. The system and technology dimension includes the learning management system, technical support, technological infrastructure and the reliability of the system. Finally, the institutional management dimension includes institutional support, training and faculty support (Naveed et al., 2020).

Among these studies, that of Rahman, Hussein, and Aluwi (2015) analyzed business school students’ satisfaction with online learning at higher educational institutions in Malaysia and found that it increases if they feel the online learning is useful and beneficial to them. Cidral, Oliveira, Di Felice, and Aparicio (2018) studied the individual impact of e-learning in 24 higher education institutions across Brazil and concluded that collaboration quality, service quality, information quality, system quality, user perceived satisfaction and usage explain the overall individual impact of e-learning. In addition, Geng, Law, and Niu (2019) examined the different effects of self-directed learning, technology readiness and learning motivation on blended and non-blended learning processes carried out by engineering students in Australia. The authors found that a blended learning environment is better than a non-blended learning environment in facilitating learning and that self-directed learning plays a vital role in influencing students’ cognitive presence and learning motivation. Muzammil, Sutawijaya, and Harsasi (2020) conducted a study at an Indonesian university that offers distance-learning programs. They collected data from 4,305 students and found that student engagement has a positive impact on student satisfaction. On the other hand, Omar and Hussein (2017) conducted a research on learning satisfaction with students from a Malaysian university and concluded that perceived usefulness and perceived ease of use have the strongest effect on learning satisfaction. Pham, Limbu, Bui, Nguyen, and Pham (2019) studied online learning methods in Vietnam and found a positive relationship between student satisfaction with online learning, loyalty and overall online learning service quality. However, they noted that some factors need to be carefully addressed at the outset, including accurately displayed information, ease of access and intuitive organization, qualified online learning instructors, and the quality of course materials.

During the COVID-19 pandemic, educational institutions around the world were forced to switch to online education as the only option to continue offering educational services. The literature states that emergency remote teaching is fundamentally different from planned online learning. Those without prior experience in an online learning environment are the least confident, as mentioned by Moser, Wei, and Brenner (2021). Only a few studies have so far analyzed student satisfaction during emergency remote teaching in different countries.

Among them, Rahiem (2020) analyzed the emergency remote teaching experience of 80 students at a university in Indonesia and pointed out that the implementation of online learning has both positive and negative effects. On the negative side, when professors used very limited online tools, this translated into negative effects on students' perceived learning; while on the positive side, flexible learning schedules and extra time for exercising and resting were highly appreciated by most of the students. Additionally, this author recommended preparing students and professors for using blended resources, designing a specific study plan for online learning, and providing access to useful digital reading materials. Shim and Lee (2020) studied the experiences of South Korean college students during emergency remote teaching as a result of the COVID-19 pandemic, using thematic analysis; they identified that network instability, unilateral interactions and reduced concentration were students’ most common complaints about the implementation of online learning. They also reported that students are very interested in learning how to use new virtual learning platforms in order to improve their educational performance. At the same time, they concluded that the overall level of interaction experienced by students could vary depending on the professor’s efforts and the technology used. In that regard, they highlighted the importance of preparing a proper educational environment that facilitates academic achievement by providing better communication tools and quality resources. Rizun and Strzelecki (2020) studied the satisfaction of undergraduate and graduate students in Poland during the COVID-19 pandemic and found that the best predictors of a student's acceptance of the shift from classical education to distance learning are enjoyment and self-efficacy. Likewise, perceived ease of use and perceived usefulness predict students’ intent to use and attitudes to distance learning.

Furthermore, Fatani (2020) found a positive relationship between teaching through videoconferencing and student satisfaction, while Almusharraf and Khahro (2020) found a positive association between user satisfaction and the implementation of popular online learning platforms such as Google Classroom. Aristovnik, Keržič, Ravšelj, Tomaževič, and Umek (2020) highlighted that sub-par computer skills and high workload perception prevented students from perceiving an improvement in their performance. Finally, some studies have studied the effects of gender and generation on student satisfaction with online learning (Shahzad, Hassan, Aremu, Hussain, & Lodhi, 2021; Yawson & Yamoah, 2020) and found similar positive effects; they also, however, identified criteria that need to be taken into consideration for an optimal implementation.

In conclusion, the literature suggests various factors that may affect student satisfaction during their online education, and especially during the emergency remote teaching experience, and it also highlights potential positive and negative effects. Against this background, the current study uses the critical incident technique to analyze the factors that affect the satisfaction and dissatisfaction of Business School students with emergency remote teaching during the COVID-19 pandemic.

3. Methodology

3.1. Critical Incident Technique

The study used content analysis and the critical incident technique to analyze the experience of Business School students and the factors that have affected their satisfaction and dissatisfaction with emergency remote teaching during the COVID-19 pandemic. The Critical Incident Technique (CIT), introduced by Flanagan (1954), is a qualitative method that collects and analyzes content and classifies directly-observed human behavior during a specific event in order to assess the studied phenomenon (Gremler, 2004). The CIT is used to explore the nature of events and the way participants perceive them, but also to interact with and understand the incidents. One of the technique’s main characteristics is that the study takes place in a natural setting, with the researcher as the main instrument for collecting data about the phenomenon to be studied. The CIT offers some decided advantages. It provides a rich source of data, allowing researchers to identify which incidents are most relevant to them in relation to the phenomenon under study. It does not restrict observations to a limited number of variables. It has an inductive approach. It does not require hypotheses but can be used to generate an accurate and in-depth record of events (Gremler, 2004). On the other hand, in order to use the CIT effectively, the researcher needs suitable research skills to manage interviews with participants and make the necessary critical observations of the process; otherwise, important contributions could be excluded. Although it has received some criticism with regard to its reliability and validity, the CIT provides rich and in-depth data and the strengths of its processes outweigh its weaknesses (Gremler, 2004). Likewise, the CIT does not consist of a single rigid set of rules (Hettlage & Steinlin, 2006), it is a flexible set of principles that must be adapted to specific situations. The CIT comprises three stages. The first stage is to describe the phenomenon and objectives of the study. The second stage consists of interviewing people based on their relevance to the studied phenomenon. Once enough data has been gathered, in the third stage, the findings are analyzed and interpreted .

Several previous studies have used the CIT; however, it is essential to note the motivation behind its implementation. After reviewing influential work from different authors for the purposes of this research, we opted to use the CIT for the following reasons. First, the CIT identifies and classifies the situations that are critical for a specific event and allows researchers to study outlier incidents (Breunig & Christoffersen, 2016; Padma & Ahn, 2020). Second, the CIT’s inductive nature allows participants to be flexible in the way they respond, and also enables researchers to develop new concepts based on the recorded incidents (Bianchi & Drennan, 2012). Third, CIT output could be presented to and straightforwardly used by decision makers to improve processes (Harrison & Beatty, 2011). Finally, the CIT enables researchers to study a phenomenon from a multi-group approach, comparing incidents between different groups (Surachartkumtonkun, Grace, & Ross, 2021).

3.2. Background of the Study

The study was carried out in Lima, Peru. On March 16, 2020, the Peruvian government imposed lockdown restrictions and a mandatory fifteen-day social isolation period in response to the COVID-19 pandemic. Said isolation period was extended several times during 2020 and was expected to continue until December 2021. As a result, the country's education sector could not offer in-person activities, and all educational institutions were forced to continue providing educational services remotely.

The study was conducted at CENTRUM PUCP Business School (CPUCP) in Peru. CPUCP is the graduate business school of the Pontificia Universidad Católica del Perú (PUCP). CPUCP is accredited by AMBA, AACSB and EQUIS. During the pandemic, CPUCP managed to migrate in-person doctoral, masters and executive education programs (approximately 3,500 students) to online learning. In Peru, education is a leading industry and plays a vital role in national development. The significant demographic and economic growth in the country, the growth of domestic demand, and the higher proportion of young people who have completed secondary school and have the necessary resources to continue their studies have generated a growing demand for higher education. At the beginning of the year 1990, there were 49 universities, while as of 2015 there are 132 universities providing educational services (42 public and 90 private). Around 83% of the 80 universities founded between 1990 and 2015 are private. The number of students enrolled in institutions of higher education has tripled from 424 thousand in 2000 to more than 1.3 million in 2016 (National Superintendence for Higher Education SUNEDU, 2021).

3.3. Sample and Data Collection

The target population of the study was MBA students at CENTRUM PUCP who began their master’s programs under an in-person teaching method but had to switch to an emergency remote teaching method after March 16, 2020, because of the response to the COVID-19 pandemic. As the study had an exploratory and inductive approach, the sample was composed of 27 MBA students (6 women and 21 men). The idea behind the sample size was to have sufficient participants to identify the critical incidents that affected the studied phenomenon. Instead of a large sample, this study focuses on developing a deep understanding of all incidents affecting the interviewees (Gremler, 2015). For this reason, the sample size was considered acceptable for the analytical generalization in the CIT.

We conducted personal interviews and used open-ended questionnaires to collect the students’ experiences. Participants signed voluntary informed consent forms in which they were informed about the voluntary nature of their participation and the fact that any information they provided would not be personally identifiable, as well as their right to leave the interview at any time if they did not feel like responding. We collected data from August 18, 2020, to January 14, 2021, during which time the students were completing their MBA programs online. The 10 interviews conducted had a duration of between 30 minutes and one hour, with an average of 45 minutes. The interviews included 17 open-ended questionnaires. The students described in detail their experiences with their online MBA education. In order to generate the incidents, interviewees were asked two specific questions: (1) Thinking of the academic aspect, describe a negative experience you have had during live online classes, at any time during the pandemic; (2) Thinking of the academic aspect, describe a positive experience you have had during live online classes, at any time during the pandemic. In order to obtain a deeper understanding, interviewees were asked to further describe these experiences with the following questions: What happened? How did it make you feel? What could have been done to improve this situation? Do you think this experience would affect your decision to recommend this program? All the students provided one or more satisfactory and dissatisfactory experiences. Thus, the data collection phase resulted in 56 incidents. All interviews were audio-recorded with the students' consent and transcribed to create a database for analysis.

3.4. Data Analysis

The objective of this study was to explore the experiences of MBA students in their remote academic programs during the COVID-19 pandemic. The time gap between the critical incidents the participants described varied from being very recent, just a few days before the interview, to six months previous. The incidents were analyzed using content analysis, which includes the identification of the texts indicated by the students, the classification of the incidents and their systematic categorization into groups with similar themes (Krippendorf, 2004).

Once the information had been collected, it was first read several times to become familiar with all the incidents mentioned by the students. Later, each incident was coded as either a satisfying or dissatisfying emergency remote teaching experience. The second phase of the analysis was to search for the critical themes to gather insights from the satisfying and dissatisfying incidents. The process of classification consisted of re-reading the texts, then analyzing the words and content, and finally, classifying the critical incidents. The satisfying and dissatisfying experiences were categorized into two main themes: "Virtuality" and "Teaching-Learning Process". These categories illustrated the experiences of students as follows: (i) emotions resulting from the emergency remote teaching experience; (ii) confidence in the educational model implemented; and (iii) perception of the quality of the educational service. Finally, all the incidents were analyzed again, divided into subcategories, and renamed until reliable conceptual patterns emerged from the data.

We used the following strategies to ensure the validity and reliability of the study. We examined the incidents separately and classified them as satisfactory or dissatisfactory. Later, we verified the contents and the logic of the information analysis, and a chain of evidence was kept in the analysis. Quasi-statistics were carried out to analyze the information relating to each particular conclusion. Finally, it should be noted that a matching pattern strategy was used in the study.

4. Results

The study identified 56 critical incidents related to the online remote teaching, of which 31 were dissatisfying incidents and 25 were satisfying incidents (Table 1). The most frequently cited critical incident subcategory was the Methodology, representing 32.1% of all reported incidents, both satisfying and dissatisfying. This was followed by Connectivity (19.6%), Remote Connection (19.6%), Professor-Student Bond (14.3%), Professor Preparation (10.7%) and Study Environment (3.6%). The results further showed that dissatisfying incidents were slightly more frequently cited (31 incidents, 55%) than satisfying incidents (25 incidents, 45%). The classification of the critical incidents related to Virtuality and those related to the Teaching-Learning Process is explained below.

Table-1. Frequency of student experiences during emergency remote teaching by category.

Category-1: Virtuality

The Virtuality category includes critical incidents related to connectivity, experiences in the remote sessions, as well as the study environment in the virtual classroom. Eight out of 25 incidents described by students as satisfactory were in the Virtuality category and 16 out of 31 dissatisfying incidents. We subdivided the incidents into three sub-categories: Connectivity, Remote Connection and Study Environment (Appendix A).

Connectivity refers to the internet infrastructure required by students to connect to the live online sessions, ensuring that they have enough bandwidth to listen to the professor's explanations, access multimedia material, and participate in group work and other course activities. The reported critical connectivity incidents were all unsatisfactory (11) and accounted for 19.6% of all incidents. Students expressed their dissatisfaction about not having the required information on how to enter the virtual classrooms, experiencing temporary disconnections during live sessions, losing their sessions, or losing internet connectivity while being evaluated or conducting a group presentation, which affected their work groups. Students were also concerned about the level of understanding displayed by professors in cases where students could not complete evaluations and activities due to their lack of internet connection.

Remote Connection refers to the factors related to the remote teaching method through the various virtual platforms. There were 11 critical incidents related to Remote Connection (8 satisfactory and 3 unsatisfactory), and these represented 19.6% of all incidents. The satisfactory experiences that were reported were the following: Easy access to classes, avoiding time-consuming journeys, busy public transport and the stress of arriving on time to attend in-person classes (1). The amount of time saved ("not having to change clothes or travel anywhere") (1). The high degree of flexibility of online education ("you can listen to classes from anywhere in the world") (1). The improved communication and interaction with other students (they felt that the shared situation they were going through made the group more supportive) (1). The greater ease of expressing themselves and participating in classes through virtual media (students said they felt more comfortable if they were not being watched or judged for their contributions) (1). Finally, another satisfactory aspect was the possibility of recording video/audio/chats of live class sessions (3). Specifically, the main advantages of recording the classes were the possibility of reviewing them several times to better understand the topic; the possibility of reviewing the topic in case of “disconnection” during the live session; and the option of downloading and saving the recordings as course materials to review later or at the end of the course. 

The following statements were identified as critical incidents of dissatisfaction related to Remote Connection: The stress generated by spending excessive hours studying in front of a computer ("depending on the program’s teaching method, sometimes up to 12 class hours in one day") (1). The lack of in-person interaction for learning purposes ("when the professor and classmates are next to me, I can see all their gestures") (1). The use of e-books on learning management systems (students prefer physical books or virtual books that are permanently available, which is not possible with e-books) (1).

The Study Environment encompasses the students' knowledge of the platform used for live online classes as well as the minimum requirements that the student’s environment should have to allow use of the virtual classrooms. The critical incidents related to the study environment were all unsatisfactory (2) and accounted for 3.6% of all incidents. Students expressed their dissatisfaction with the complicated way of accessing the online platforms (especially when they have to identify themselves for security reasons). They were also dissatisfied with the fact that some students had to connect to the live online sessions through their cell phones while traveling in their vehicles or doing other activities, affecting the group work dynamics during the sessions.

Category 2: Teaching-Learning Process

The category Teaching-Learning Process includes incidents related to the processes through which students acquire skills and knowledge, and which are bidirectional between the student and the professor. Seventeen out of 25 incidents described by students as satisfactory were in this category and 15 out of 31 dissatisfying incidents. We subdivided the incidents into three sub-categories: Methodology, Professor-Student Bond and Professor Preparation (Appendix B).

Methodology refers to the methods used by professors to achieve the desired learning outcomes. The study identified 18 critical incidents related to the Methodology (9 satisfactory and 9 unsatisfactory), which accounted for 32.1% of all incidents. Two satisfactory incidents related to class participation. The methodology used by the professors was essential to motivate students to participate verbally or through the chat, to share their opinion and to motivate them to prepare for the sessions ("I felt very good, because I felt more confident when answering the questions. I was also motivated to continue reading"). Four satisfactory incidents related to the technological tools used by professors with expertise in remote access platforms, as well as other technological tools to make classes more dynamic, such as virtual whiteboards, Kahoot and group dynamics, among others. Another satisfactory event relating to the completion of the courses according to the program was the management of the live online classes, especially the different didactic strategies used by the professors to maintain the attention and interest of the students in the virtual classrooms. On the other hand, there were nine unsatisfactory incidents related to the teaching methodology: The lack of feedback (and delay in grading the projects) (2). Inadequate management of live online classes ("some professors think we are experts on the subject and the class advances too fast", "The explanations of the professor on screen are flat. In-person classes include hand and body movements") (3). The use of technological tools to support the learning process, such as the online connection platform, sometimes makes verbal participation difficult and generates a lack of order in the live sessions when students do participate, as well as reducing interaction compared to in-person classes (4).

The Professor-Student Bond refers to empathy, feedback, and trust in the development of the Teaching-Learning Process. There were eight critical incidents in this subcategory (5 satisfactory and 3 unsatisfactory), which accounted for 14.3% of all incidents. We found that the professor-student bond acquired great importance in emergency online teaching due to the students' defensive attitude towards the compulsory nature of online learning dictated by the quarantine. Students were skeptical at first of receiving the same level of educational service they received in the in-person classes. Hence, it was very important for professors to create an empathetic and supportive virtual environment to achieve the learning objectives. The following statements described the satisfactory incidents: Better interaction between professors and students through higher levels of student participation on virtual platforms ("it is clear that professors are more open to listening to students during classes") (1). Professors are more willing to answer questions inside and outside online classes, resulting in students receiving more feedback on virtual media ("I feel confident that I can communicate with the professors directly and that they are always willing to answer my questions") (1). Professors are committed to making lectures attractive to maintain students’ attention during the virtual classes ("Despite the technical difficulties, some professors do their best to teach well and make sure that the students all understand the classes") (1). Professors are also committed to improving their contribution to the learning process ("We value the attitude of a professor who wants to improve") (2). Regarding the incidents that led to students’ feelings of dissatisfaction, they were related to the limited opportunities for remote connection, in terms of professor-student interaction, since it is a distant interaction and the experiences are impersonal (2) ("I would suggest trying to arrange more virtual networking meetings"). There was also a feeling of dissatisfaction when professors did not trust students ("some professors do not believe us when we tell them we have a bad internet connection") (1). 

Finally, the subcategory Professor Preparation for online teaching generated six critical incidents (3 satisfactory and 3 unsatisfactory), which together accounted for 10.7% of the total number of incidents. The satisfactory incidents related to the mastery of technological tools to support the learning process (2) and the proper use of information systems to support the learning process ("Now the professor masters many tools, with only a few difficulties. It is gratifying to see the change") (1). In contrast, the unsatisfactory incidents related to the lack of mastery of new virtual teaching platforms ("the problem was the professor’s lack of mastery of the platform") (1); the lack of materials specially prepared for virtual classes ("some professors did not adapt their methodology to a virtual environment") (1); and professors’ insufficient preparation for the remote teaching method (1).

5. Conclusions and Practical Recommendations

This study analyzed the experiences of Business School students in Peru during the migration from an in-person or blended teaching method to emergency online education. The results were 56 incidents, 25 satisfactory and 31 unsatisfactory, across two major categories we have called Virtuality and Teaching-Learning Process. The study allows us to draw conclusions about several critical aspects of the two identified categories that should be considered in order to improve future online or hybrid teaching programs.

For the Virtuality category, the study concludes that the existence of adequate connectivity, as well as protocols in case of contingencies related to connection failures, must be clearly foreseen in online education, for both professors and students. On the other hand, the satisfaction incidents related to virtuality must be promoted and incorporated into hybrid programs. We propose four practical recommendations in this category (see Table 2).

The first recommendation is that educational institutions need to be aware that online teaching requires a reliable internet connection with adequate bandwidth. For this reason, they must ensure high-quality connectivity for all professors, but also communicate the minimum requirements to properly access the programs to their students. During the COVID-19 pandemic in Peru, connectivity issues, both for students and professors, were one of the main challenges, due to deficiencies of the technological infrastructure in many parts of the country, as well as overloaded connections and the deficient systems of the telecommunication companies. However, educational institutions must look for alternatives, such as contracting directly with telecommunications companies to provide their students and professors with routers, in order to ensure their connectivity during online classes. The second recommendation is that educational institutions need to establish protocols to manage different situations in which a temporary loss of internet connection might affect professors and students, particularly on exam days. Furthermore, to eliminate students’ uncertainty and anxiety, those protocols must be communicated to them in a clear and precise manner, so that they know exactly how such a problem will be solved. The third recommendation is that in order to improve the study environment, online programs must establish minimum conditions for participation in virtual classes. The conditions may include the use of laptops and banning the use of cellphones to access the virtual platforms, as well as solitude during the sessions and the collaboration of the student’s family to avoid interruptions during live online classes, among others. These conditions can help to make the most of live sessions and interaction between students and professors. Moreover, online classes should also be scheduled at times that allow for physical and mental breaks. Finally, the satisfactory incidents from online programs should be included in hybrid programs. Some of the most relevant satisfactory incidents were the recording of classes to review later; the use of Learning Management Systems to support the development of online materials; the possibility of entering online classes instead of in-person classes, as well as the flexibility with regard to location. 

Regarding the category of the Teaching-Learning Process, the studied incidents highlight the methodological aspects, the professor-student bond and the professors’ preparation for online pedagogy. We propose four practical recommendations in this category. The first is that incidents relating to the methodology are the most important and demand adequate planning of the educational model, including the use of proper virtual platforms to improve students’ learning process through educational methods based on competencies. This includes the use of virtual tools to make the teaching more dynamic and maintain the students' attention, the mastery of communication strategies (intonation, speech accompanied by images or videos) and pedagogical strategies (practical examples, development of activities between the professor’s presentation and the students' participation). The second recommendation is to provide students with timely and adequate feedback, since this is a fundamental aspect of student satisfaction. The third is that the professor-student bond needs to be strengthened in online teaching. This bond can be positively affected during live online sessions if the professor creates a climate of trust, tranquility and closeness for students. For example, keeping the professor's camera on at all times, conveying calm and positive messages, using proper intonation, and motivating students to participate without being judged, and with confidence that they will be heard and evaluated fairly. The absence of in-person attendance reinforces the need for the professor to create a reliable, safe and student-friendly classroom environment. The second type of bond is the one generated during non-synchronous activities, where professors must maintain empathetic communication, constant feedback, and timely and complete answers, so that the students can maintain a sense of continuous interaction with their professors.  Finally, regarding professors' preparation, educational institutions must ensure that professors are fully prepared, both academically and technologically, for online teaching. The mastery of teaching methods, dynamics, competency models, and the use of platforms and other technological tools is essential.

It is highly probable that after the pandemic has been overcome new teaching methods will include blended, hybrid or fully online programs. Therefore, it is essential to incorporate all satisfactory incidents of emergency online teaching into these programs, as well as to learn from the unsatisfactory incidents to improve the educational process.

Table-2. Practical recommendations - Remote Teaching.

6. Limitations and Further Research

This study has several limitations. First, the qualitative approach does not allow the data to be generalized beyond the context analyzed. While the CIT was appropriate for this study, the technique relies on self-reporting and the categorization of key constructs from students’ incidents, which depend on the analysis of the researcher (Gremler, 2004). The study was based on the perceptions of students, and it is assumed that they were accurate. In addition, the sample size was relatively small and consisted of only one Business School in Peru. For further research, we recommend considering different perspectives of the online remote teaching experience, comparing the perceptions of students with those of professors and program directors.

References

Almusharraf, N., & Khahro, S. (2020). Students’ satisfaction with online learning experiences during the COVID-19 Pandemic. International Journal of Emerging Technologies in Learning, 15(21), 246-267. Available at: https://doi.org/10.3991/ijet.v15i21.15647.

Aristovnik, A., Keržič, D., Ravšelj, D., Tomaževič, N., & Umek, L. (2020). Impacts of the COVID-19 pandemic on life of higher education students: A global perspective. Sustainability, 12(20), 1-34. Available at: https://doi.org/10.3390/su12208438.

Bianchi, C., & Drennan, J. (2012). Drivers of satisfaction and dissatisfaction for overseas service customers: A critical incident technique approach. Australasian Marketing Journal (AMJ), 20(1), 97–107. Available at: https://doi.org/10.1016/j.ausmj.2011.08.004.

Breunig, K., & Christoffersen, L. (2016). If x then why? Comparative analysis using critical incidents technique. Journal of Business Research, 69(11), 5141–5146. Available at: https://doi.org/10.1016/j.jbusres.2016.04.094.

Cidral, W. A., Oliveira, T., Di Felice, M., & Aparicio, M. (2018). E-learning success determinants: Brazilian empirical study. Computers and Education, 122, 273-290. Available at: https://doi.org/10.1016/j.compedu.2017.12.001.

Cortes Abarca, G. (2021). Implementation of emergency remote teaching in chilean schools due to COVID-19. Journal of Education and e-Learning Research, 8(3), 313-323. Available at: https://doi.org/0.20448/journal.509.2021.83.313.323.

Cravener, P. A. (1999). Faculty experiences with providing online courses: Thorns among the roses. Computers in Nursing, 17(1), 42-47.

Crawford, J., Butler-Henderson, K., Rudolph, J., Malkawi, B., Glowatz, M., Burton, R., . . . Lam, S. (2020). COVID-19: 20 countries' higher education intra-period digital pedagogy responses. Journal of Applied Teaching and Learning (JALT), 3(1), 9-28. Available at: https://doi.org/10.37074/jalt.2020.3.1.7.

Fatani, T. H. (2020). Student satisfaction with videoconferencing teaching quality during the COVID-19 pandemic. BMC Medical Education, 20(1), 1-8. Available at: https://doi.org/10.1186/s12909-020-02310-2.

Flanagan, J. (1954). The critical incident technique. Psychological Bulletin, 48(6), 327–358. Available at: https://doi.org/10.1037/h0061470.

Geng, S., Law, K. M. Y., & Niu, B. (2019). Investigating self-directed learning and technology readiness in blending learning environment. International Journal of Educational Technology in Higher Education, 16(1), 1-22. Available at: https://doi.org10.1186/s41239-019-0147-0.

Gremler, D. D. (2004). The critical incident technique in service research. Journal of Service Research, 7(1), 65-89. Available at: https://doi.org/10.1177/1094670504266138.

Gremler, D. D. (2015). The critical incident technique. in wiley encyclopedia of management (eds C.L. Cooper, N. Lee and A.M. Farrell): Wiley Online Library.

Harrison, M. P., & Beatty, S. E. (2011). Anticipating a service experience. Journal of Business Research, 64(6), 579-585. Available at: https://doi.org/10.1016/j.jbusres.2010.06.009.

Hettlage, R., & Steinlin, M. (2006). The critical incident technique in knowledge management-related contexts, ingenious peoples knowledge (pp. 1-18). Zurich: Swiss Association for International Cooperation.

Hiltz, S. R. (1986). The “virtual classroom”: Using computer-mediated communication for university teaching. Journal of Communication, 36(2), 95-104. Available at: https://doi.org/10.1111/j.1460-2466.1986.tb01427.x.

Hodges, C. B., Moore, S., Lockee, B. B., Trust, T., & Bond, M. A. (2020). The difference between emergency remote teaching and online learning. EDUCAUSE Review. Retrieved from: https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning.

Keržič, D., Tomaževič, N., Aristovnik, A., & Umek, L. (2019). Exploring critical factors of the perceived usefulness of blended learning for higher education students. PloS One, 14(11), e0223767. Available at: https://doi.org/10.1371/journal.pone.0223767.

Krippendorf, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage.

Krüger, J. M., Vogel, F., & Schnaubert, L. (2020). Synchronous online lectures in emergency remote teaching: The role of immersion, social scripts and group awareness. International Conference on Computers in Education, Proceedings, 2(23), 29-35.

Lin, C.-C., Ma, Z., & Lin, R. C.-P. (2011). Re-examining the critical success factors of e-learning from the EU perspective. International journal of Management in Education, 5(1), 44-62. Available at: https://doi.org/10.1504/IJMIE.2011.037754.

McPherson, M., & Baptista-Nunes, M. (2007). Negotiating the Path from curriculum design to e-learning course delivery: A study of critical success factors for instructional systems design. In European Conference on Technology Enhanced Learning (pp. 232-246). Berlin, Heidelberg: Springer.

Moser, K. M., Wei, T., & Brenner, D. (2021). Remote teaching during COVID-19: Implications from a national survey of language educators. System, 97, 102431. Available at: https://doi.org/10.1016/j.system.2020.102431.

Muzammil, M., Sutawijaya, A., & Harsasi, M. (2020). Investigating student satisfaction in online learning: The role of student interaction and engagement in distance learning university. Turkish Online Journal of Distance Education, 21(Special Issue-IODL), 88-96. Available at: https://doi.org/10.17718/tojde.770928.

National Superintendence for Higher Education SUNEDU. (2021). Report of public and private universities.

Naveed, Q. N., Qureshi, M. R. N., Tairan, N., Mohammad, A., Shaikh, A., Alsayed, A. O., . . . Alotaibi, F. M. (2020). Evaluating critical success factors in implementing E-learning system using multi-criteria decision-making. PloS one, 15(5), e0231465. Available at: https://doi.org/10.1371/journal.pone.0231465.

Omar, M., & Hussein, N. (2017). Factors influencing E-learning satisfaction among students: A study of a public university in Malaysia. World Applied Sciences Journal, 35(4), 568-573. Available at: https://doi.org/10.5829/idosi.wasj.2017.568.573.

Padma, P., & Ahn, J. (2020). Guest satisfaction & dissatisfaction in luxury hotels: An application of big data. International Journal of Hospitality Management, 84, 102318. Available at: https://doi.org/10.1016/j.ijhm.2019.102318.

Palvia, S., Aeron, P., Gupta, P., Mahapatra, D., Parida, R., Rosner, R., & Sindhi, S. (2018). Online education: worldwide status, challenges, trends, and implications. Journal of Global Information Technology Management, 21(4), 233-241. Available at: https://doi.org/10.1080/1097198X.2018.1542262.

Pham, L., Limbu, Y. B., Bui, T. K., Nguyen, H. T., & Pham, H. T. (2019). Does e-learning service quality influence e-learning student satisfaction and loyalty? Evidence from Vietnam. International Journal of Educational Technology in Higher Education, 16(1), 1-26. Available at: https://doi.org/10.1186/s41239-019-0136-3.

Rahiem, M. D. (2020). The emergency remote learning experience of university students in Indonesia amidst the COVID-19 crisis. International Journal of Learning, Teaching and Educational Research, 19(6), 1-26. Available at: https://doi.org/10.26803/ijlter.19.6.1.

Rahman, N. A. A., Hussein, N., & Aluwi, A. H. (2015). Satisfaction on blended learning in a public higher education institution: What factors matter? Procedia-Social and Behavioral Sciences, 211, 768-775. Available at: https://doi.org/10.1016/j.sbspro.2015.11.107.

Rizun, M., & Strzelecki, A. (2020). Students’ acceptance of the Covid-19 impact on shifting higher education to distance learning in Poland. International Journal of Environmental Research and Public Health, 17(18), 1-19. Available at: https://doi.org/10.3390/ijerph17186468.

Schenk, B., & Hoxhaj, L. (2019). Challenges in switching to blended learning environments: An analysis of students' attitudes and performance. Paper presented at the International Conference on Information Systems, Munich, Germany.

Shahzad, A., Hassan, R., Aremu, A., Hussain, A., & Lodhi, R. (2021). Effects of COVID-19 in E-learning on higher education institution students: The group comparison between male and female. Quality and Quantity, 55(3), 805-826. Available at: https://doi.org/10.1007/s11135-020-01028-z.

Shim, T. E., & Lee, S. Y. (2020). College students’ experience of emergency remote teaching due to COVID-19. Children and Youth Services Review, 119, 105578. Available at: https://doi.org/10.1016/j.childyouth.2020.105578.

Sridharan, B., Deng, H., & Corbitt, B. (2010). Critical success factors in e-learning ecosystems: A qualitative study. Journal of Systems and Information Technology, 12(4), 263-288. Available at: https://doi.org/10.1108/13287261011095798.

Sun, L., Tang, Y., & Zuo, W. (2020). Coronavirus pushes education online. Nature Materials, 19(6), 687-687. Available at: https://doi.org/10.1038/s41563-020-0678-8.

Surachartkumtonkun, J. N., Grace, D., & Ross, M. (2021). Unfair customer reviews: Third-party perceptions and managerial responses. Journal of Business Research, 132, 631-640. Available at: https://doi.org/10.1016/j.jbusres.2020.09.071.

Whittle, C., Tiwari, S., Yan, S., & Williams, J. (2020). Emergency remote teaching environment: A conceptual framework for responsive online teaching in crises. Information and Learning Sciences, 121(5/6), 311-319. Available at: https://doi.org/10.1108/ILS-04-2020-0099.

Yawson, D. E., & Yamoah, F. A. (2020). Understanding satisfaction essentials of E-learning in higher education: A multi-generational cohort perspective. Heliyon, 6(11), e05519. Available at: https://doi.org/10.1016/j.heliyon.2020.e05519.

Appendix-A. Category 1 - Sample Incidents Related to Virtuality.

Appendix-B. Category 2 - Sample Incidents Related to the Teaching-Learning Process.