A New Approach to Modelling Students’ Socio-Emotional Attributes to Predict Their Performance in Intelligent Tutoring Systems
1,2,3,4Laboratoire de Recherche en Informatique et Télécommunication (LARIT), Institut National Polytechnique Félix Houphouët-Boigny (INP-HB), Yamoussoukro, Côte d’Ivoire.
5 Laboratoire d'Études et de Prévention en Psychoéducation (LEPPE-ENS-Abidjan), Université Jean Lorougnon Guédé (UJLoG), Daloa, Côte d'Ivoire.
Abstract
Intelligent Tutoring Systems (ITS) are computer-based learning environments that aim to imitate to the greatest possible extent the behavior of a human tutor in their capacity as a pedagogical and subject expert. One of the major challenges of these systems is to know how to adapt the training both to changing requirements of all kinds and to student knowledge and reactions. The activities recommended by these systems mainly involve active student performance prediction that, nowadays, becomes problematic in the face of the expectations of the present world. In the associated literature, several approaches, using various attributes, have been proposed to solve the problem of performance prediction. However, these approaches have failed to take advantage of the synergistic effect of students' social and emotional factors as better prediction attributes. This paper proposes an approach to predict student performance called SoEmo-WMRMF that exploits not only cognitive abilities, but also group work relationships between students and the impact of their emotions. More precisely, this approach models five types of domain relations through a Weighted Multi-Relational Matrix Factorization (WMRMF) model. An evaluation carried out on a data sample extracted from a survey carried out in a general secondary school showed that the proposed approach gives better performance in terms of reduction of the Root Mean Squared Error (RMSE) compared to other models simulated in this paper.
Keywords:Intelligent tutoring system, Impact of emotions, Social influence, Socio-emotional intelligence, Matrix factorization, Student performance prediction.
Contribution of this paper to the literature:This research investigates the field of predicting student performance in Intelligent Tutoring Systems by proposing an innovative approach based on the concept of socio-emotional intelligence. This approach incorporates, for the first time, the mutual influence of group work relationships and the impact of emotions felt during assessment sessions in form of attributes to enhance capacities such as motivation, attention and student commitment.
1. Introduction
Social and emotional intelligence, or socio-emotional intelligence, is the ability to integrate feelings, intuition and cognition to recognize, understand, manage, apply and express our emotions and social interactions at the right time, for the right purpose, in the right context and with the right person (Devis-Rozental, 2017; Devis-Rozental., 2018
). It has also been defined as the ability to regulate emotions at the individual, group and cross-boundary levels (Peterson, 2015
). It is considered by some researchers (Mupfumira & Nyaruwata, 2021
; Nasir & Masrur, 2010
; Pajibo, Dzikunu, & Asare, 2019
; Piczon & Asis, 2019
; Turi, Ghani, Javid, & Sorooshian, 2020
) to be a predictor of academic performance and a prerequisite for academic achievement. Payton et al. (2008
) describe social and emotional learning as a process by which all individuals (adults and children alike) acquire the skills necessary to, among other things, recognize and manage their emotions, establish and maintain positive relationships, and effectively manage interpersonal situations. As a result, it seems appropriate to take into consideration factors such as group work relationships and the effect of emotional state on a student's cognitive abilities in order to enhance and optimize learning in traditional learning systems in general and in Intelligent Tutorial Systems in particular. The practice of teaching in ITS should reflect our current understanding of the learning process, as marked by teamwork, and teaching methods should consider the affective and emotional state of students as a means of regulating motivation. In other words, tutors should consider the socio-emotional needs of students and adapt the interaction to have a positive impact on their performance. If these factors are taken into account in ITS, it should improve the learning environment and students’ attitudes towards learning, and potentially improve learning outcomes.
This paper proposes a new approach that integrates the mutual influence of group work relationships and the impact of emotions experienced during assessment sessions in the form of socio-emotional attributes in a Weighted Multi Relational Matrix Factorization (WMRMF) model in order to provide increased accuracy in the prediction of student performance. Using these socio-emotional attributes, we aim to provide a measure of socio-emotional intelligence that can enhance abilities such as motivation and attention and engage students in the use of intelligent learning tutors. The approach proposed in this paper makes it possible to predict student performance by taking into account not only their cognitive abilities but also the impact of students' socio-emotional attributes.
1.1. Related Work
The first Matrix Factorization approach to student performance prediction based on the “KDD Challenge 2010” database was proposed by Thai-Nghe, Drumond, Krohn-Grimberghe, and Schmidt-Thieme (2010). They showed that student performance prediction could be consider the prediction of an assessment. In this approach, student, task and performance became user, article and evaluation, respectively, as presented in a classic recommendation system. The results of this study showed that the FM technique performed better compared to traditional regression methods such as Logistic Regression and Linear Regression. However, this basic technique can only be applied to a single relationship, that between a student and a task. Other scholars have adapted the classical FM approach in various ways, including Singular Value Decomposition (SVD) (Jembere, Rawatlal, & Pillay, 2017
), Tensor Matrix Factorization (Thai-Nghe, Drumond, Horváth, Nanopoulos, & Schmidt-Thieme, 2011
), Non-negative Matrix Factorization (NMF) (Hwang & Su, 2015
). However, these approaches fail to take into account the secondary information included in the practical educational data, such as the data on different relationships associated with different domains of learning. Other improvements have been made to this basic technique over time. These include, among others, Multi-Relational Matrix Factorization (MRMF) and Weighted Multi-Relational Matrix Factorization (WMRMF) (Thai-Nghe & Schmidt-Thieme, 2015
). However, these approaches did not incorporate the mutual influences in group work relationships and the emotions that students might experience when performing tasks. Notably, however, researchers report that, regardless of the subject, students who work in groups generally know more about what is being taught and retain the learning for longer than when the same content is presented using other teaching methods (Alhaj, 2015
). In addition, emotion has been shown to have a major impact on an individual's judgment and reasoning (Damasio, 1995). Affective state also influences cognitive abilities such as attention, memory, problem solving and decision making (Isenman, 2018
). Isenman (2018
) argues that emotion has above all a prospective or motivational role. For these two reasons, we recently proposed in {Assielou, 2020 #58}Assielou, Haba, Gooré, Kadjo, and Yao (2020a
), an FM approach we called So-WMRMF, which combines students’ cognitive capacities and the power of group relations in a Multi-Relational framework. Unlike the work carried out by most of the authors of the related literature, in this approach, the characteristic vector of a student is influenced by the weighted average of the characteristic vectors of all the participants in the working group. Specifically, this approach exploits the individual academic performance of each of the students in a work group as a factor of mutual influence. In Assielou, Haba, Kadjo, Yao, and Gooré (2020b
) we further proposed an approach called Emo-WMRMF that models the impact of emotions that arise in a learning context and a framework for integrating these emotions in performance prediction. This approach is based on the control-value theory of achievement emotions proposed by Pekrun (2006
), which is founded on the principle that a student can feel a combination of emotions at any particular moment. The various simulations carried out have shown that these two approaches make it possible to refine the precision of student performance prediction. However, what these two approaches failed to take into account was the synergistic effect of group relations and the emotions experienced either individually or in groups, since some social scientists, such as Willis and Cromby (2019
), have shown that emotion is also relational.
In this paper, we propose a Weighted Multi-Relational Matrix Factorization and Socio-Emotional approach called SoEmo-WMRMF to better predict student performance.
1.2. State-of-the-Art
The approach proposed in this paper exploits the performance of the So-WMRMF and Emo-WMRMF models in the form of socio-emotional attributes. Therefore, this section presents the state-of-the-art of these two different approaches.
1.2.1. Weighted Multi-Relational Matrix Factorization (WMRMF) Approach
In this Equation 4, the value of is equal to 1 for the principal relation, and this value is included in the interval for the other relations, according to the degree of importance of these relations.
1.2.2. Weighted Multi-Relational Matrix Factorization and Social (So-WMRMF) approach
In the So-WMRMF approach proposed by Assielou et al. (2020a), it is assumed that a student’s s performance is affected by their
work group friends in the following way:
1.2.3. Modelling of Emotional Influence
In the emotional impact modelling approach proposed by Assielou et al. (2020b), the authors adopt the principle that the performance of a student s depends not only on their intrinsic abilities but also on the impact of their experienced emotions. In this approach, emotions are assessed using the AEQ (Achievement Emotions Questionnaire) Self-Assessment Questionnaire proposed by Pekrun (2006
). In Assielou et al. (2020b
),
is a Multiple Linear Regression model whose variables
are the intensities of the eight test emotions linked to the AEQ, see Table 1. The different coefficients of this model are the elements of
.
Table-1. Model variables (Assielou et al. 2020b
) description.
2. Materials and Methods
The approach proposed in this paper combines the two models So-WMRMF and as defined in the previous section. This section describes the methodology used.
2.1. Problem Formulation
This approach associates a performance score as well as any emotions felt by the student when performing the task. We also combine the skills learned by the student and establish a link between the tasks to be performed and the various skills in the field. Figure 1 illustrates the five types of domain relations exploited by our approach.
Figure-1. Relationships taken into account in the SoEmo-WMRMF approach
The approach takes into account the fact that a student may experience, at a specific time, not a single emotion but rather a combination of several achievement emotions related to the test. This is the case, for example, when a student feels both anxious and desperate. Therefore, the challenge for us is to create a way for ITS to manage a combination of emotions in order to enhance students’ learning experience.
Table-2. SoEmo-WMRMF Training algorithm
Since the challenge of using models based on the Matrix Factorization technique is to find the optimal parameters , we have adopted the methodology below, comprising four steps:
- Step 1: The first step is to extract from all the learning data the performance matrix, the matrix of student outcomes by skills and the expert’s matrix. These different matrices are taken from the field of study. As part of this contribution, we worked on a dataset described in section 2.3.
2.3. Data Set for Machine Learning
To assess our prediction approach using socio-emotional attributes, we conducted a survey of 250 students enrolled in their final year at a general secondary school in Daloa (Côte d’Ivoire). The sample consisted of 80 girls with an average age of 16 and 170 boys with an average age of 17. The AEQ questionnaire was filled out several times during different evaluations to avoid the problem of "subject emotion", i.e., the student feeling specific emotions depending on the subject in which they were being assessed. In addition to the questionnaire on emotions, we also identified the practical organization of the previously formed study groups. The majority of groups are maded up of two or three students, the numbers of which are shown in Table 3. From this data sample, we extracted five (5) matrices. The first relates to the “student-performs-task” relationship, the second to the “student-has-skills” relationship, the third to the “task-requires-skills” relationship, the fourth to “student-has-friends” and the fifth to the “student-has-emotions” relationship.
Table-3. Composition of groups.
Number of groups |
Number of students |
|
Groups of 2 students |
40 |
140 |
Groups of 3 students |
20 |
|
students not belonging to any group |
110 |
|
Total number of students |
250 |
|
We also present the percentage composition of the different groups in our data set. Of the 250 students, 44% are not part of a working group, see Figure 2.
Figure-2. Distribution of students by work group.
To avoid the problem of overfitting, we used the Cross-Validation (CV) technique. To this end, we divided our sample into two subsets, i.e., 5/6 for machine learning and 1/6 for the test phase. We evaluated this approach under the same conditions as those used in Assielou et al. (2020b), taking into account the same simulation environment. To compare the performance of our approach and that of the alternative state-of-the-art approaches, we used the Root Mean Squared Error (RMSE) metric, provided in Equation 14 below:
For the sake of fairness in the comparison, we initialized the parameters from the same range of random values for all the approaches. The relevant parameters of each model have been carefully adjusted to achieve the best possible performance. Table 4 gives the different values of the parameters used to optimize the different models simulated in the same experimental environment.
Table-4. Optimization parameters.
3. Results and Discussion
In this section, we present the results of our approach, and we discuss these results in parallel with those of the previously proposed models. The results in Figure 3 show that, with regard to our dataset, the SoEmo-WMRMF model improves the RMSE error compared to So-WMRMF and Emo-WMRMF models.
Figure-3. RMSE results on our data set.
Note that the Emo-WMRMF model significantly improves the RMSE error compared to the So-WMRMF model. The choice of emotion as an attribute in the domain for predicting performance thus appears advantageous over the choice of mutual influence factors in the context of friendly group relations. The results obtained from the SoEmo-WMRMF model show that taking into account both these socio-emotional attributes in performance prediction systems offers better prospects for computer learning systems. This result corroborates those of the authors Turi et al. (2020) and Nasir and Masrur (2010
) who have shown, in empirical studies, that socio-emotional intelligence is a predictor of academic performance and a prerequisite for academic achievement.
Table-5. Comparison of different approaches.
RMSE results |
|
Global Average |
0.436 |
Item Average |
0.421 |
User Average |
0.296 |
MF |
0.286 |
MRMF |
0.283 |
WMRMF |
0.273 |
So-WMRMF |
0.269 |
Emo-WMRMF |
0.265 |
SoEmo-WMRMF |
0.257 |
We also compared the SoEmo-WMRMF model to several other approaches, including MRMF, MF, User Average, Item Average and Global Average, by ranking them in order of precision in terms of performance prediction (see Table 5). Since the SoEmo-WMRMF approach uses several parameters, we have simulated in parallel the effects of some of these parameters on its performance. To carry out this evaluation, we investigated the role of each of the parameters by varying each one in turn and recording the results. Figure 4 shows the results of this investigation.
Figure-4. Effect of parameters on SoEmo-WMRMF model performance.
4. Conclusion
In this paper, we have proposed a new approach to the prediction of student performance by taking into account not only students’ cognitive abilities but also the impact of students’ socio-emotional attributes in the learning environment. This approach takes advantage of the performance of the So-WMRMF model and the impact of emotions on student performance to refine the prediction results. To our knowledge, it is the first Matrix Factorization approach that exploits five types of domain relations: the first relates to the “student-performs-task” relationship, the second to the “student-has-skills” relationship, the third to the “task-requires-skills” relationship, the fourth to “student-has-friends” and the fifth to the “student-has-emotions” relationship. The evaluation that was carried out on a sample of data extracted from a survey carried out in a general secondary education institution showed that the SoEmo-WMRMF model provides a better performance in terms of reducing the RMSE error compared to other simulated models.
The performance prediction approach proposed in this paper only exploits positive social relationships between students. However, some students can also sometimes have negative social influences on others. This can induce more negative emotions than positive and have the consequence of demotivating them in the learning process. It would then be important to study the impact of these negative social relationships on the prediction of students' performances.
Citation:Kouamé Abel ASSIELOU; Cissé Théodore HABA; Tanon Lambert KADJO; Bi Tra GOORE; Kouakou Daniel YAO (2021). A New Approach to Modelling Students’ Socio-Emotional Attributes to Predict Their Performance in Intelligent Tutoring Systems. Journal of Education and e-Learning Research, 8(3): 340-348. |
References
Alhaj, A. (2015). The impact of group work in improving EFL medical students' academic performance. Paper presented at the Paper presented at the The IAFOR International Conference on Language Learning, Dubai.
Assielou, K. A., Haba, C. T., Gooré, B. T., Kadjo, T. L., & Yao, K. D. (2020a). Emotional impact for predicting student performance in intelligent tutoring systems (ITS). International Journal of Advanced Computer Science and Applications, 11(7), 219-224. Available at: https://doi.org/10.14569/ijacsa.2020.0110728.
Assielou, K. A., Haba, C. T., Kadjo, T. L., Yao, K. D., & Gooré, B. T. (2020b). Multi-relational and social-influence model for predicting student performance in intelligent tutoring systems (ITS). International Journal of Engineering and Advanced Technology, 9(3), 2058-2066.
Damasio, A. R. (1995). Descartes' error: Emotion, Reason and the Human Brain: Picador.
Devis-Rozental, C. (2017). Developing socio-emotional intelligence in early years scholars. Doctoral Dissertation, Bournemouth University.
Devis-Rozental, C. (2018). Developing socio-emotional Intelligence in Higher education academics: Benefits beyond the classroom." In Developing Socio-emotional intelligence in Higher education scholars (pp. 157-199). Cham: Palgrave Macmillan.
Hwang, C.-S., & Su, Y.-C. (2015). Unified clustering locality preserving matrix factorization for student performance prediction. IAENG International Journal of Computing Science, 42(3), 245-253.
Isenman, L. (2018). Emotion and Motivation Understanding Intuition (pp. 111-132): Elsevier Inc.
Jembere, E., Rawatlal, R., & Pillay, A. W. (2017). Matrix factorisation for predicting student performance Paper presented at the Paper Presented at the 2017 7th World Engineering Education Forum (WEEF).
Mupfumira, I. M., & Nyaruwata, L. T. (2021). Challenges in the teaching and learning of textiles in the primary school curriculum in Masvingo District Zimbabwe. Asian Journal of Contemporary Education, 5(1), 12–22. Available at: https://doi.org/10.18488/journal.137.2021.51.12.22.
Nasir, M., & Masrur, R. (2010). An exploration of emotional intelligence of the students of IIUI in relation to gender, age and academic achievement. Bulletin of Education and Research, 32(1), 37-51.
Pajibo, E., Dzikunu, C. K., & Asare, M. O. (2019). Administrative leadership behaviours of principals of colleges of education and their impact on Tutors’ job performance in the Volta Region, Ghana. American Journal of Education and Learning, 4(1), 70–83. Available at: https://doi.org/10.20448/804.4.1.70.83.
Payton, J., Weissberg, R. P., Durlak, J. A., Dymnicki, A. B., Taylor, R. D., Schellinger, K. B., & Pachan, M. (2008). The positive impact of social and emotional learning for kindergarten to eighth-grade students. Chicago, IL: Collaborative for Academic, Social, and Emotional Learning.
Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315-341. Available at: 10.1007/s10648-006-9029-9.
Peterson, C. H. (2015). Group emotional intelligence and academic performance: Testing the mediational effects of attendance and contribution to group work Emotional intelligence: Current evidence from psychophysiological, educational and organizational perspectives. (pp. 207-222). Hauppauge, NY, US: Nova Science Publishers.
Piczon, V., & Asis, L. A. (2019). Help-seeking and management behavior of the deans and academic chairpersons in the University of Eastern Philippines. International Journal of Emerging Trends in Social Sciences, 5(2), 65-70. Available at: https://doi.org/https://doi.org/10.20448/2001.52.65.70.
Thai-Nghe, N., Drumond, L., Krohn-Grimberghe, A., & Schmidt-Thieme, L. (2010). Recommender system for predicting student performance. Procedia Computer Science, 1(2), 2811-2819.
Thai-Nghe, N., Drumond, L., Horváth, T., Nanopoulos, A., & Schmidt-Thieme, L. (2011). Matrix and tensor factorization for predicting student performance. Paper presented at the Paper presented at the Proceedings of the 3rd International Conference on Computer Supported Education.
Thai-Nghe, N., & Schmidt-Thieme, L. (2015). Multi-relational factorization models for student modeling in intelligent tutoring systems. Paper presented at the Paper presented at the 2015 Seventh International Conference on Knowledge and Systems Engineering (KSE).
Turi, J. A., Ghani, M. F. A., Javid, Y., & Sorooshian, S. (2020). The impact of socio-emotional intelligence on academic performance. Educational Administration Research and Review, 2(2), 18-31. Available at: https://doi.org/10.17509/earr.v2i2.22478.
Willis, M., & Cromby, J. (2019). Bodies, representations, situations, practices: Qualitative research on affect, emotion and feeling. Qualitative Research in Psychology, 17(1), 1-12. Available at: 10.1080/14780887.2019.1656361.
| Asian Online Journal Publishing Group is not responsible or answerable for any loss, damage or liability, etc. caused in relation to/arising out of the use of the content. Any queries should be directed to the corresponding author of the article. |