Index

• Contents
• 1. Introduction
• 2. Methods
• 3. Result and Discussion
• 4. Conclusion
• References

Factors Affecting the Success of E-Learning-Based Training using Learning Management System Platforms: Adaptations of Updated DeLone and McLean Models

Agus Yudiawan^1*; Siti Rokhmah² ; Talabudin Umkabu³ ; Febriani Safitri⁴ ; Arifin⁵

¹Sorong State Islamic Institute, West Papua, Indonesia.
^2,3Fattahul Muluk State Islamic Institute of Papua, Papua, Indonesia.
⁴Cenderawasih University, Papua, Indonesia.
⁵Muhammadiyah University of Kupang, East Nusa Tenggara, Indonesia.

Abstract

The purpose of this study is to determine the factors, both direct and indirect, that affect the success of the learning management systems implemented in the Ministry of Education's “Sekolah Penggerak” training program. The DeLone-McClean IS Success Model was modified to be used as the study's analysis tool. The study employed a quantitative method with 122 samples.  The survey technique   employing Google Forms was used in the data collection process. The Partial Least Square Structural Model was analyzed using the Smart-PLS 3.0 software. As the t-statistic value was more than 1.96 and the p-value was less than 0.05, all hypotheses testing direct influence on success were accepted. Only four of the thirteen hypotheses on indirect influence were rejected. In conclusion, the DeLone-McClean IS Success Model can be used to evaluate the effectiveness of a learning management system in various e-learning activities.

Keywords: E-Learning based training, Learning management system, Update DeLone-McClean IS success model.

Contribution of this paper to the literature This study illustrates that the DeLone-McClean IS Updated Success Model can be used as a barometer for measuring the success in implementing information systems developed by the government or educational and other institutions. This model is very relevant, especially in developing information systems that are more focused and responsive to user needs.

1. Introduction

During the Covid-19 pandemic, technology-based learning media became popular in conducting learning activities. Learning activities that were previously carried out outside the network shifted to online learning technology platforms (e-learning) (Hamidy, Mashur, & Yaqin, 2021; Setiawan, Munzil, & Fitriyah, 2021). With e-learning, the learning process and administration  became faster and more accessible (Sriwindono & Tumiwa, 2016). The ease of updating, storing, distributing and providing instantaneous instructions and information  was conducive  to users' needs (Pamugar, Winarno, & Najib, 2014).   Studies show that e-learning has been proven to maximize the quality of learning operations (Bradley, 2021). This is  the reason why education  service providers, both academic and training institutions, continue to use and optimize e-learning in conducting service transactions (Chatwattana & Nilsook, 2017; Kasim & Khalid, 2016).

The Ministry of Education of the Republic of Indonesia  has been successful  in implementing  various  effective educational programs, one of which is on e-training and learning to apply the technology-based Learning Management System, Rustandi (2020). The Learning Management System (LMS) is a website-based application designed to manage learning  activities, from planning to evaluation (Setiawan et al., 2021; Szabo, 2002). This learning management system provides various facilities for instructors to create, monitor and assess learning outcomes, (Aman et al., 2020; Eunice & Cosmas, 2019; Gilhooly, 2001; Moşteanu, 2021) . In other words, information systems in education management are able to improve the performance of learning systems, (Aldarbesti & Saxena, 2014). Although the LMS facilitates various online training and learning activities to be connected and made easily accessible  to facilitators and participants, the fact remains that improvements are still being made in different aspects to get optimal results, by designing an LMS that is more flexible and user-friendly (Shurygin, Saenko, Zekiy, Klochko, & Kulapov, 2021; Wallenborn et al., 2017) . Therefore, a framework has been established to  study the success  of LMS technology tools, using the updated  IS Information System’s Success Model (DeLone & McLean, 2003). This model has become an essential reference for understanding the factors that predict the successful use of information systems, including LMS, (DeLone & McLean, 2003; Hudin & Riana, 2016; Mardiana, Tjakraatmadja, & Aprianingsih, 2015) . Aspects of usability and convenience of information technology are the main focus of the IS Success Model. Initially, this model was implemented to study marketing technology (e-commerce) and it proved capable of predicting fluctuations and online shopping transactions (DeLone & McLean, 2003; Schaupp, Bélanger, & Fan, 2009; Wang, 2008) .This model was first developed and used to evaluate information systems in government agencies, (Bawono, 2014; Saputro, Budiyanto, & Santoso, 2016; Sudarto, Sufarnap, Ulfah, & Amalia, 2020) and is still used   to this day.  It has now been adopted and has become a trend in measuring the success of e-learning in various educational programs, (Aman et al., 2020; Kasim & Khalid, 2016; Sriwindono & Tumiwa, 2016) . The quality of systems, information, services, the intensity of use and user satisfaction, all of which  lead to net benefits or impact , are essential variables in measuring the success of learning technology platforms using the IS Success Model (DeLone & McLean, 2003). Therefore, the quality of users and organizations is a measure of the success  and use of an LMS (DeLone & McLean, 2003; Urbach & Müller, 2012).

 Several past studies that  applied the IS Success Model  used the old DeLone-McClean model, resulting in different findings (Sriwindono & Tumiwa, 2016). In addition, the relationship between the variables of the updated Delone and McLean model has not been analyzed for the indirect impact between the variables required in the model (Sudarto et al., 2020). Several existing studies have not fully understood how to use the updated DeLone-McClean model to see the direct and indirect impact between the required variables. In addition, not many studies explain the use of the IS Success Model descriptively or partially in the relationship between variables, especially in identifying the success of the LMS.

Therefore, this paper intends to work on the shortcomings of existing studies. In addition to carefully mapping the effectiveness of Learning Management System services, it  also aims to examine the contribution of the IS Success Model variable in the context of the "Sekolah Penggerak" training program by the Ministry of Education, both directly and indirectly, between the variables specified. Various contributions need to be  identified as they have direct implications for the success of the training, both for individuals and organizing parties (DeLone & McLean, 2003). Likewise, knowledge of the factors that contribute significantly to the successful use of LMS, apart from being a valuable experience for service use, also provides positive input in optimizing better LMS services for managers. The success of LMS in various learning and training programs is determined directly or indirectly by the quality of the system, information, services, intensity of use and user happiness,  as identified in this study. Figure 1 shows the study framework using the most recent De Lone-Mc Clean model as the unit of analysis.

Note: adapted from the DeLone and McLean model.

2. Methodology

The research was designed using a quantitative research method. The population in this study comprised the facilitators from the Ministry of Education who are involved in using LMS in the “Sekolah Penggerak” program.  The research sample comprised 122 individuals, obtained through the use of the Slovin formula, with a significance error of 5%. The sample was obtained using a simple random sampling technique with the entire population having the same potential to make up the sample.  All samples were given a questionnaire based on the Likert scale in a google form to be filled out. The collected data were then analyzed for component-based or variant-based Structural Models popular with Partial Least Square. Structural analysis was performed using SmartPLS 3.0 software. In path analysis, three essential steps were carried out, namely statistical requirements testing, model testing and hypothesis testing, (Hussein, 2015; Monecke & Leisch, 2012; Ramayah, Cheah, Chuah, Ting, & Memon, 2018) .

The first step is to perform an outer model analysis to evaluate how indicators and latent variables are related (Monecke & Leisch, 2012). Convergent validity, discriminant validity and unidimensionality are used to examine the relationship between indicators and other factors. The value of the loading factor on the latent variable indicates whether the variable is convergent or not. If the loading factor value is >0.7, (between 0.5 and 0.7), it is considered good, and if the loading factor is less than 0.5, the model is dismissed or excluded. In addition to the loading factor, the AVE value should be more than 0.5, indicating that the model is not converging. If the root value of the AVE is bigger than the correlation construct, discriminant validity is said to be sufficient. Finally, unidimensionality testing verifies that no measurement issues exist. The composite reliability indicator and Cronbach alpha were used to carry out this test. There is no unidimensionality problem in the model if the Composite Reliability or Cronbach's Alpha score is > 0.7 for all constructs.

The second phase involves analyzing the inner model to ensure that the structural model created is accurate and sturdy (Hussein, 2015). The coefficient of determination (R-Square), Predictive Relevance (Q2) and Goodness of Fit Index are all measures that show how accurate the model is (GoF). The better the prediction model for the suggested research model, the greater the coefficient of determination. R-Square 1-0.67 (substantial), 0.66-0.33 (moderate) and 0.19 (weak) are the three R2 values that are required (Chin, Diehl, & Norman, 1988). Furthermore, the GoF value  ranges from 0 to 1, with 0.1 (little), 0.25 (mid) and 0.36 (big) used as interpretations (Hussein, 2015; Monecke & Leisch, 2012). Finally, predictive relevance (Q2) is good if it is close to 1, indicating that the external latent variable is good or appropriate for predicting the endogenous variable. Equations 1 and 2 determine predictive relevance and the Goodness of Fit Index.

The third phase involves putting the hypothesis proposed in the study to the test. The t-statistical value and significance value are indicators of hypothesis testing. If the t-statistic value is > 1.96 or p 0.05, the hypothesis can be accepted (Hussein, 2015).

3. Results and Discussion

3.1. Quality of the Learning Management System

The IS Success Model created by DeLone and McLean is used to assess the quality of the LMS program implemented by the Ministry of Education of the Republic of Indonesia. This model is recognized as a simple model, but researchers consider it to be quite valid (DeLone & McLean, 2003; Hudin & Riana, 2016; Mardiana et al., 2015) . It's fascinating to watch how organizers and users assess the factors in the IS Success Model. The results of a descriptive investigation of the quality of six variables in predicting service-learning management systems using the IS Success Model are summarized below. Table 1 shows the findings of the descriptive investigation.

No	Variable	Mean	Criteria
1	Information Quality (IQ)	3.514	Good
2	System Quality (SyQ)	3.585	Good
3	Service Quality (SeQ)	3.436	Good
4	Intention to Use (IU)	3.653	Good
5	User Satisfaction (US)	3.751	Good
6	Net Benefit (NB)	3.795	Good

The quality of utilizing LMS is shown in Table 1 with 1.00-1.80 (very poor); 1.81-2.60 (poor); 2.61-3.40 (enough); 3.41-4.20 (good) and 4.21-5.00 (very good)  used as the measuring criteria (Umar, 2014). The quality of implementing a learning management system based on the IS Success Model in the training of facilitators of the "Sekolah Penggerak" program is within good standards or defined as successful, according to the results of the descriptive research shown in Table 1. Information quality, service, system, intention to use, user satisfaction and net benefit are all interconnected and support each other.

3.2. IS Success Model Data Analysis on the Use of Learning Management Systems

There are 3 stages in the analysis of the learning management system based on the IS Success Model.  The results of the study of each step are described below.

3.2.1. Outer Model Analysis Results

At this point, the links between indicators and latent variable constructions is investigated. The correlation can be used to determine the validity and reliability of a suggested model. Convergent validity via factor loading and AVE, discriminant validity and construct reliability are used to assess a model's validity and construct reliability (Ghozali, 2008; Hussein, 2015). The model factor loading results are shown in Figure 3.

From Figure 3, it can be seen that all items (IQ, SyQ, SeQ, IU, US and NB) have a loading factor value of 1.00 or above the required limit of 0.7. Therefore, the six criteria items are ideal and meet the requirements of an analytical model. The results of further analysis can be seen in Table 2 and Table 3.

Indicators	Cronbach Alpha	Rho_A	Composite Reability	Root Average Vari-ance Extracted (AVE)
IQ	1.000	1.000	1.000	1.000
SyQ	1.000	1.000	1.000	1.000
SeQ	1.000	1.000	1.000	1.000
IU	1.000	1.000	1.000	1.000
US	1.000	1.000	1.000	1.000
NB	1.000	1.000	1.000	1.000

Indicators	IQ	IU	NB	SeQ	SyQ	US
IQ	1	0.772	0.759	0.693	0.61	0.745
IU	0.772	1	0.888	0.87	0.821	0.937
NB	0.759	0.888	1	0.787	0.702	0.916
SeQ	0.693	0.87	0.787	1	0.674	0.819
SyQ	0.61	0.821	0.702	0.674	1	0.782
US	0.745	0.937	0.916	0.819	0.782	1

Convergent validity is demonstrated by the AVE value in Table 2 in addition to the loading factor value in Figure 3. Table 2 shows that the AVE value for all criteria is 1.00, which is higher than the threshold of 0.5. As a result, it may be inferred that the model under examination has no convergent validity issues. Furthermore, discriminant validity analysis was performed by comparing the value of the square root of AVE (Table 2) with the correlation construct value (Table 3). The square root value of the AVE of all criteria (IQ, IU, NB, SeQ, SyQ and US) is more than the correlation of each construct, as shown in Tables 2 and 3, indicating that the model is sufficient. Finally, the unidimensionality test, as determined by the Composite reliability indicator and Cronbach's alpha (Table 2), has a value of 1.00 or greater than 0.7.  This shows the model has no unidimensionality issues. As a result, the complete outer model stage test is declared to have passed and met the requirements, allowing the inner model stage test to proceed.

3.2.2. Results of Inner Model Stage Analysis

The inner model is tested at this stage to confirm that the structural model built in the study is reliable and accurate (Hussein, 2015). R-Square, Predictive Relevance (Q2) and Goodness of Fit (GoF) Index are used to test at this stage.  The R-Square analysis findings can be seen in Table 4.

Indicators	R Square	Adjusted R Square
IQ	-	-
SyQ	-	-
SeQ	-	-
IU	0.930	0.928
US	0.796	0.791
NB	0.847	0.844

The R-Square values for intention to use (IU), user stratification (US), and Net Benefit (NB) are 0.930, 0.796, and 0.847, respectively, as shown in Table 4. This means that information quality (IQ), system quality (SyQ), and service quality (SeQ) may each explain or impact intention to use (IU) and user stratification (US) by 93.0% and 79.6%, respectively. In addition, 84.7 percent of the net benefit (NB) can be explained by intention to use (IU) and user stratification (US). This indicates that the relationship between variables has a significant impact on the prediction of other variables and is crucial to the route model (Chin et al., 1988). Furthermore, the value of Q2 is seen when testing the Inner model (predictive relevance). The following are the calculations:

Q²     = 1 – (1-R₁²) (1-R₂²) (1-R₃²)
Q²     = 1 – (1- 0.930) (1-0.796) (1-0.847)
Q²     = 1 – (0.07) (0.204) (0.153)
Q²     = 1 – (0.002185)
Q²     = 0.997815

From the calculation of predictive relevance (Q2), a value of 0.998 (rounded up) is obtained. This predictive relevance value is close to 1. This means that the exogenous latent variable is suitable and can predict the endogenous variable. Finally, the Goodness of Fit (GoF) test is carried out in stage two. GoF value is calculated by Equation 2. The GoF value of 0.1 means (small), 0.25 (medium) and 0.38 (large), (Hussein, 2015). To simplify the GoF analysis, the average AVE and R-Square values are mapped in Table 5.

From Table 5, it can be seen that the average AVE value is 1000, and R-square is 0.8577. These two values are then analyzed using the GoF calculation as follows:

Indicators	Average Variance Extracted (AVE)	R Square
IQ	1.000	-
SyQ	1.000	-
SeQ	1.000	-
IU	1.000	0.930
US	1.000	0.796
NB	1.000	0.847
Average	1.000	0.8577

The GoF value was obtained from the calculations for that amount or 0.9261 (92.61%). This GoF value is far bigger than 0.38; this shows a large GoF. This means the model has been substantially appropriate in presenting the results.

From the entire series of second-stage tests (outer model), both testing the R-Square value, predictive relevance and GoF, it can be concluded that the model formed from the path analysis in this study is robust and accurate. In other words, all stages of the second test (outer model) have been met, and it is feasible to continue testing the hypothesis.

3.3. Hypothesis Testing Stage

The hypothesis is tested after the outside and inner models are met. The path coefficient value or the inner model produced from bootstrapping can be used to determine the significance level in hypothesis testing. If the p-value is less than 0.05 or the t-statistic is more than 1.96, the hypothesis is accepted (Hussein, 2015; Ramayah et al., 2018). Calculating the path coefficient value and significance for each path analyzed answers the proposed hypothesis. Table 6 summarizes the conclusions reached on all theories.

Using the updated DeLone-McClean IS Success Model Update, Table 6 demonstrates two hypothetical criteria, namely direct and indirect effects between variables on the success of learning management systems. There are nine hypotheses for direct influence and thirteen hypotheses for indirect influence. All hypotheses on direct influence in route analysis had t-statistics > 1.96 and p-values of 0.05. This means that any hypothesis on direct effect can be accepted. Only the fourth hypothesis on indirect influence was rejected out of the thirteen. As a result, the relationship between Information Quality and Net Benefit cannot be mediated by Intention to Use.

The Updated DeLone-McClean IS Success Model can predict the success of employing a learning management system in training and learning, according to the findings of the study by Kasim and Khalid (2016) and Aman et al. (2020). Furthermore, this study supports prior research findings that the IS Success Model is used to assess the success of key technological platforms (Mardiana et al., 2015; Sriwindono & Tumiwa, 2016; Urbach & Müller, 2012) . As a result, diverse stakeholders can utilize the modified IS Success Model to assess the success of  an established technological platform (DeLone & McLean, 2003). It can be utilized for government information technology platforms (Bawono, 2014; Saputro et al., 2016; Sudarto et al., 2020) the digital economy (Schaupp et al., 2009; Wang, 2008) and other technological  areas, in addition to learning and training platforms.

No	Hypothesis	Original Sample (O)	Sample Mean (M)	Standard Deviation (STDEV)	T Statistics (|O/STDEV|)	P -Values	Information
Direct effect
1	IQ -> IU	0.105	0.105	0.045	2.333	0.020	Accepted
2	IQ -> US	0.242	0.239	0.065	3.754	0.000	Accepted
3	IU -> NB	0.246	0.233	0.106	2.311	0.021	Accepted
4	SeQ -> IU	0.267	0.266	0.047	5.663	0.000	Accepted
5	SeQ -> US	0.409	0.411	0.076	5.349	0.000	Accepted
6	SyQ -> IU	0.197	0.199	0.047	4.221	0.000	Accepted
7	SyQ -> US	0.359	0.360	0.069	5.191	0.000	Accepted
8	US -> IU	0.486	0.486	0.069	7.085	0.000	Accepted
9	US -> NB	0.686	0.698	0.100	6.837	0.000	Accepted
Indirect effect
1	IQ -> US -> IU	0.118	0.116	0.036	3.252	0.001	Accepted
2	SeQ -> US -> IU	0.199	0.200	0.050	3.988	0.000	Accepted
3	SyQ -> US -> IU	0.174	0.174	0.038	4.552	0.000	Accepted
4	IQ -> IU -> NB	0.026	0.027	0.019	1.354	0.176	Rejected
5	SeQ -> IU -> NB	0.066	0.063	0.032	2.029	0.043	Accepted
6	SyQ -> IU -> NB	0.048	0.046	0.023	2.071	0.039	Accepted
7	IQ -> US -> IU -> NB	0.029	0.025	0.012	2.387	0.017	Accepted
8	SeQ -> US -> IU -> NB	0.049	0.046	0.023	2.098	0.036	Accepted
9	US -> IU -> NB	0.120	0.110	0.050	2.390	0.017	Accepted
10	SqQ -> US -> IU -> NB	0.043	0.041	0.021	2.013	0.045	Accepted
11	IQ -> US -> NB	0.166	0.169	0.058	2.889	0.004	Accepted
12	SeQ -> US -> NB	0.281	0.287	0.068	4.115	0.000	Accepted
13	SyQ -> US -> NB	0.246	0.249	0.051	4.844	0.000	Accepted

4. Conclusion

This study shows that the Updated DeLone-McClean IS Success Model can be used as a barometer for measuring the success of implementing a learning management and information system. This feature is very relevant, especially in the development of information systems. There are direct and indirect relationships between the seven factors and the twenty-two hypotheses examined in supporting the efficacy and success of learning management system services. Only the variable of intention to use has failed to mediate between service quality and net benefits. It is clear that if the quality of this information system is determined by the needs of the consumers, it will directly benefit them. Mediation is not necessary.   Despite the study’s positive findings, there are still some limitations. The sample size for this study is moderate. The number of samples should be increased for further research, even though it is sufficient as a prerequisite for path analysis. Furthermore, the ministry of education, as well as colleges, schools, governments and e-commerce firms which employ LMS need additional investigation. As a result, the Updated DeLone-McClean IS Success Model will be more precise in assessing the success of information systems of diverse parties.  Finally, the development of information systems will be more focused and responsive to user needs.

References

Aldarbesti, H., & Saxena, J. (2014). Management information system for education. IOSR Journal of Research & Method in Education, 4(1), 36-44.

Aman, A., Prasojo, L. D., Sofwan, M., Mukminin, A., Habibi, A., & Yaqin, L. N. (2020). Factors affecting Indonesian pre-service teachers’ use of m-LMS: A mix method study. International Journal of Interactive Mobile Technologies, 14(6), 137-147.Available at: https://doi.org/10.3991/ijim.v14i06.12035.

Bawono, T. A. (2014). Management support and strategic scorecard information system succes: Case study of e-performance in ministry of finance of republic of Indonesia. Doctoral Dissertation, Universitas Gadjah Mada.  

Bradley, V. M. (2021). Learning management system (LMS) use with online instruction. International Journal of Technology in Education, 4(1), 68-92.Available at: https://doi.org/10.46328/ijte.36.

Chatwattana, P., & Nilsook, P. (2017). A web-based learning system using project-based learning and imagineering. International Journal of Emerging Technologies in Learning, 12(5), 4-22.Available at: https://doi.org/10.3991/ijet.v12i05.6344.

Chin, J. P., Diehl, V. A., & Norman, L. K. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI.

DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9–30.Available at: https://doi.org/10.1080/07421222.2003.11045748.

Eunice, M.-K. K., & Cosmas, M. (2019). An analysis of factors affecting utilisation of moodle learning management system by open and distance learning students at the University of Eswatini. American Journal of Social Sciences and Humanities, 5(1), 17–32.Available at: https://doi.org/10.20448/801.51.17.32.

Ghozali, I. (2008). Structural equation modeling metode alternatif dengan partial least square. Semarang: Diponegoro University Publishing Agency.

Gilhooly, K. (2001). Making e-learning effective. Computerworld, 35(29), 52-53.

Hamidy, R. R., Mashur, M., & Yaqin, L. N. (2021). Factors affecting online learning through LMS during the covid 19 period. Edumatic: Journal of Informatics Education, 5(2), 288-295.Available at: https://doi.org/10.29408/edumatic.v5i2.4158.

Hudin, J. M., & Riana, D. (2016). Study of Delone & Mclean information system success model for accurate accounting information system users in Sukabumi City. Journal of Information Systems, 12(1), 1-9.

Hussein, A. S. (2015). Business and management research using partial least squares (PLS) with smartPLS 3.0. Universitas Brawijaya: Teaching Module.

Kasim, N. M., & Khalid, F. (2016). Choosing the right learning management system (LMS) for the higher education institution context: A systematic review. International Journal of Emerging Technologies in Learning, 11(6), 55-61.Available at: https://doi.org/10.3991/ijet.v11i06.5644.

Mardiana, S., Tjakraatmadja, J. H., & Aprianingsih, A. (2015). DeLone–McLean information system success model revisited: The separation of intention to use-use and the integration of technology acceptance models. International Journal of Economics and Financial Issues, 5(1), 172-182.

Monecke, A., & Leisch, F. (2012). semPLS: Structural equation modeling using partial least squares. Journal of Statistical Software, 48(3), 1-32.Available at: https://doi.org/10.18637/jss.v048.i03.

Moşteanu, N. R. (2021). Teaching and learning techniques for the online environment. How to maintain students’ attention and achieve learning outcomes in a virtual environment using new technology. International Journal of Innovative Research and Scientific Studies, 4(4), 278–290.Available at: https://doi.org/10.53894/ijirss.v4i4.298.

Pamugar, H., Winarno, W. W., & Najib, W. (2014). Evaluation model of success and acceptance of E-Learning information systems at government education and training institutions. Scientific Journal of Informatics, 1(1), 13–27.Available at: https://doi.org/10.15294/sji.v1i1.3638.

Ramayah, T., Cheah, J., Chuah, F., Ting, H., & Memon, M. A. (2018). Partial least squares structural equation modeling (PLS-SEM) using smartPLS 3.0: An updated guide and practical guide to statistical analysis (2nd ed.). Kuala Lumpur, Malaysia: Pearson.

Rustandi, D. (2020). Learning management system (LMS) testing. Ministry of Education and Culture. Retrieved from https://dikti.kemdikbud.go.id/pengumuman/pengujian-learning-management-system-lms/.

Saputro, P. H., Budiyanto, D., & Santoso, J. (2016). The Delone and Mclean model for measuring the success of E-government in Pekalongan City. Scientific Journal of Informatics, 2(1), 1–8.Available at: https://doi.org/10.15294/sji.v2i1.4523.

Schaupp, L. C., Bélanger, F., & Fan, W. (2009). Examining the success of websites beyond e-commerce: An extension of the IS success model. Journal of Computer Information Systems, 49(4), 42-52.

Setiawan, A. M., Munzil, & Fitriyah, I. J. (2021). Trend of learning management system (LMS) platforms for science education before-after Covid-19 pandemic. AIP Conference Proceedings, 2330(1), 060005.Available at: https://doi.org/10.1063/5.0043196.

Shurygin, V., Saenko, N., Zekiy, A., Klochko, E., & Kulapov, M. (2021). Learning management systems in academic and corporate distance education. International Journal of Emerging Technologies in Learning, 16(11), 121-139.Available at: https://doi.org/10.3991/ijet.v16i11.20701.

Sriwindono, H., & Tumiwa, F. (2016). Learning management system success analysis (Case Study: USD Yogyakarta). Media Techniques, 11(2), 77-82.

Sudarto, S., Sufarnap, E., Ulfah, R., & Amalia, C. (2020). Analysis of information system implementation success using the Delon & McLean model. Paper presented at the Proceedings of the National Seminar on Information Science Research (SENARIS).

Szabo, M. (2002). CMI theory and practice: Historical roots of learning managment systems. Paper presented at the E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education.

Umar, H. (2014). Research methods for thesis and business thesis. Jakarta: National Library: Catalog in Publications (KDT).

Urbach, N., & Müller, B. (2012). The updated DeLone and McLean model of information systems success. In Information systems theory (pp. 1–18). New York: Springer.

Wallenborn, B., Then, M., Hoang, M. D., Vu, D. B., Becker, J., Fuchs, M., & Hemmje, M. (2017). Towards a competence-based course authoring tool supporting learning management systems. International Journal of Emerging Technologies in Learning, 12(10), 127.Available at: https://doi.org/10.3991/ijet.v12i10.7300.

Wang, Y. (2008). Assessing e-commerce systems success: A respecification and validation of the DeLone and McLean model of IS success. Information Systems Journal, 18(5), 529–557.Available at: https://doi.org/10.1111/j.1365-2575.2007.00268.x.

Asian Online Journal Publishing Group is not responsible or answerable for any loss, damage or liability, etc. caused in relation to/arising out of the use of the content. Any queries should be directed to the corresponding author of the article.