ISSN 0798 1015

Vol. 38 (Nº 05) Año 2017. Pág. 11

A review of literature on the reasons that cause the high dropout rates in the MOOCS

Una revisión de la literatura sobre razones que causan la alta tasa de deserción en los Cursos Masivos Abiertos (CMA)

Luis Naito Mendes BEZERRA 1; Márcia Terra da SILVA 2

Recibido: 23/08/16 • Aprobado: 12/10/2016

Content

1. Introduction

2. The dropout process at MOOCS

3. Systematic review of the literature about dropouts at the MOOCS

4. Results and analysis

5. Conclusion

References

1. Introduction

We have also recently seen the advent of a new type of Distance Learning which is known by the acronym Massive Open Online Course (MOOC). MOOCs are open courses, offered fully online without any prior requirements, without any payment of upfront fees, and with the potential to distribute education on a global scale, even allowing students hailing from emerging countries to have access to quality institutions and courses at a low cost (Hyman, 2012; Cooper & Sahami 2013; Ong & Grigoryan, 2014).

Other authors mention the MOOCs as a new business model for higher education institutions, as even though the courses are exempt from fees for enrollment and access to the content, it is possible to charge fees for issuing the certificates. Considering the high number of students who choose this type of course, this initiative could generate a sustainable business model (Hyman, 2012; Dellarocas & Van Alstyne 2013; Alraimi, Zo & Ciganek, 2015).

However, it is also important to consider that the business model for the MOOCs is not yet consolidated, as right now it is only accessible to a small subset of higher education institutions, including: Harvard, Stanford and Massachusetts Institute of Technology (MIT), that, on using their long-standing tradition and excellent academic reputation, have received enough investment to meet the costs for creation of special platforms for production and distribution of content on a large scale, with the end result being the creatiton of companies inspired by the start-up models of Silicon Valley, including Coursera (Stanford University) and Edx (MIT and Harvard) (Sandeen, 2013).

The first MOOC was set up in September 2008 in Canada. However, this format really started to attract attention from students back in 2001, when Sebastian Thrun and Peter Norvig set up the course in Artificial Intelligence at Stanford University, which attracted over 160,000 students in 190 different countries (Rodriguez, 2012; Ong & Grigoryan, 2014). As from 2011, the growth of this type of distance learning has been impressive, first with the appearance of three major platforms for the offering of MOOCS as mentioned before: Coursera, and EdX, and also Udacity (Little, 2013; Mallon, 2013; Sandeen, 2013).

These platforms have managed to distribute the MOOCS on a global scale. By way of example, in May 2015, Coursera had a total of 1,038 courses, having established partnerships with 119 higher education institutitions from different parts of the world (Coursera, 2015). Currently, interest in MOOCS has also been growing in Europe. According to the website of Open Education Europe which brings data on MOOCs in European countries, in September 2014 a total of 770 different courses was offered, and in September 2015 this total had soared to 1771 courses, a figure which suggests a growth rate of 130% (Open Education Europe, 2015).

Even though the global scale has been reached, with a large number of students served and also a considerable growth in the number of courses, some challenges have been presenting themselves to the universe of MOOCs and have  also been the object of several studies. Several authors have researched aspects such as:

• Pedagogical Model for the MOOCs  (Vardi, 2012; Fournier, Kop & Durand, 2014)
• Validation of the certificate  (Hyman, 2012; Cooper & Sahami, 2013);
• Typology of the MOOCs (Welsh & Dragusin, 2013; Sandeen, 2013; Almenara, Cejudo, & Vazquez Martinez, 2014; Rossele, Caron, & Heutte, 2014; Conole, 2014)
• Perception and experience of the students in MOOCs (Zutshi, O´Hre, & Rodafinos, 2013; Cofrin, Barba, Corrin, & Kennedy, 2014; )
• Quality of MOOCs (Read & Rodrigo, 2014; Walker & Lock, 2014)

The research studies that address issues related to perception and experience of the students and the qualities of the MOOCs often address another aspect which should be considered by the people responsible for the management of the MOOCs, which is the issue of the high dropout rate shown by this type of course. This problem has actually been addressed by several authors who have looked into the reasons why many programmes have been doomed to failure (Roval & Downey, 2010; Poy & Gonzalles-Aguilar, 2014), the challenges of learning in this type of course (Hew & Cheung, 2014) and the main difficulties of students in keeping up with the course (Liu, et al., 2014).

The purpose of this work is to identify the reasons for the high dropout rate in the courses offered as MOOCs, based on a systematic review of specialized literature on the following databases: Springer, Science Direct, ERIC database, and ACM Digital Library.

Apart from this introduction, this work has been divided into another five sections. In the first section, we present the process of dropping out at the MOOCs and then present a systematic literature about the reasons that have led to dropouts at MOOCs. Next, we present the results and analysis, conclusions and, finally, the bibliographical references.

2. The dropout process at MOOCs

Within the scope of the MOOCs, several authors have shown evidence that such courses have significant dropout rates, normally around 90%. (Daniel, 2012; Morris, 2013; Sandeen, 2013; Hew & Cheung, 2014; Jordan, 2014; Wilkowski, Deutsch, & Russell, 2014; Alraimi, Zo, & Ciganek, 2015).

The dropout problem can also be presented for better understanding, by means of examples. According to Coffrin, Barba, Corrin, & Kennedy (2014), the MOOC named Principles of Macroeconomics, offered by the University of Melbourne, has attracted 54,217 students, of which 32,598 have effectively participated in the course and only 1,412 having completed the course and received the certificate (4.33%). The same concern with regard to high dropout rates is part of the research conducted by Rosewell & Jansen (2014) who presented the case of the first MOOC of the University of Edinburgh in 2013, in which only 12% of the students completed the course.

Along the same lines, a more ample research presented by Jordan (2014) analysed 91 MOOCs with numbers of students ranging from 4,50 to 226,652 (with an average enrolment of 42,844), with most having a completion rate of less than 10%, with the average completion rate being only 6.5%.

According to Wilkowski, Deutsch & Russell (2014), due to the characteristics of the MOOCs, such as the lack of fees and previous requirements for student intake, this makes it possible for interested parties just to sign up and never return to the course, a fact which, without a shadow of doubt, contributes for a considerable dropout level right at the start of the course. To better understand the profile of the student who decides to enrol on a course of this type, the authors have proposed a system of classification into four categories:

a) No-shows – The student signs up for the course (often before the content is available) and then never logs on;

b) Observers –  These students want to know how an online course operates and the teaching method;

c) Casual learners – People who need to learn one of two new subjects, either out of curiosity or through needs related to learning of professional considerations;

d) Completers – Complete as many requirements set for the course, to complete the projects and also receive the certificate of completion.

It is important to mention that awareness of this classification would enable the managers to better understand the reasons why the students enrol on the courses, and also make possible the development of projects bearing in mind the specific aims of each student. It can also influence how the courses can be offered; for example, it make sit possible to suggest only a subset of the course for certain students based on their declared preferences or previous experience, thereby helping to bring down the dropout rate.

In addition, the work by Clow (2013) presents the degree of participation which a student could achieve on taking a MOOC, a process which allows a better understanding of the dropout issue in this type of course. This process, consisting of four stages, was presented by the author through a model which has been called the “funnel of participation”, as Figure 1 shows.

Figure 1: The funnel of participation –
Adapted  from Clow (2013)

The stages in this model are detailed as follows:

• Stage 1 – Awareness – potential students must be aware that MOOC exists.
• Stage 2 – Registration – Only some of those who are aware shall wish to sign up and shall manage to do so.
• Stage 3 – Activity – Next, some of those who have registered shall continue to be involved in some activity or other, and some of these shall reach the final phase (Progress), where there is significant learning.

The dropout rate is very high in all stages, but is particularly high in stages 1 and 2.

The work by Clow (2013) and that by Wilkowski, Deutsch, & Russell (2014) are imortant to explain the drastic reduction in interest and participation of the students through time and through the different stages of the course. However, they do not present details of the main reasons responsible for the high dropout rates at the MOOCs. Thus, in the next secion we shall perform a systematic review of the literature to identify the reasons which have led the students to drop out of the course.

3. Systematic review of the literature about dropouts at the MOOCs

a. Methodological Procedures

The review of the literature is a key tool to treat diversity of knowledge within a specific academic area. In the specific case of this research, we have used a selection of Works (Kennedy, 2014; Liyanagunawardena, Adams, Rassol, & Williams, 2014; Hew & Cheung, 2014).

To achieve the proposed goal, we have carried out a bibliographical study that includes the period between the offer of the first MOOC in 2008 (Sandeen, 2013) through to the year when the research is being carried out (2016). The criteria for research are listed in Table 1:

Table 1: Criteria for construction of the article database

The key words used for searching the databases were obtained based on the report drawn up by Quinn (2013), which carried out a study Project for the European Committee for Education, with the main goal of looking into the issue of dropouts in higher learning institutions, in distance learning formats.

Based on the criterion defined in Table 1, the following results, shown in Table 2, have been obtained:

Table 2: Results from searches in the databases

Next, the research was based on the reading of titles and abstracts, with the aim of identifying the articles that meet the conditions of the present study. After the due reading and analysis of the titles and the abstracts, 24 articles, which are part of this analysis, have been selected.

4. Results and analysis

Based on the analysis of the research studies about the issue, it was possible to identify 24 reasons for not completing the course, which are listed on Table 3. In this table, we can observe that for each reason we highlight the authors that have been used as a reference, it also being important to mention that the reasons have not been presented in order of importance.

Table 3: Reasons for dropping out of MOOCs

In the following stage, the reasons have been divided into two groups to facilitate analysis, and here it is important to stress that such a division has been inspired by the researches of Clow (2013) and Wilkowski, Deutsch, & Russell (2014), as follows:

a. Reasons inherent to the very characteristics of the MOOCs

As mentioned above, the MOOCs have, as their main characteristics, such aspects as: being open courses, totally online in format, without any prior requirements, without any upfront charging of fees and with the potential to distribute education on a global scale, also allowing the students coming from developing countries to gain access to institutions of high reputation and quality courses.

However, characteristics such as the absence of fees and the courses’ openness mean that a considerable number of students could enrol just out of curiosity or with the aim of getting to know one course, to drop out soon afterwards, thereby contributing to push up the dropout rates.

In relation to the twenty-three reasons mentioned by the authors, twelve of these (50%) are inherent to the very characteristics of the MOOCs, and here it is important to mention that the managers of these courses have very little leeway for action to soothe the effects in relation to high dropout rates.  It must also be mentioned here that these reasons are not, right now, the main aim of analysis in this study. The reasons are presented in Table 4 that follows.

Table 4: Reasons inherent to the very characteristics of the MOOCS

b. Reasons related to the development of the student during the course

For this set of reasons, it is possible to establish a relationship with the model proposed by Clow (2013) in the activity phase, where the students start getting involved with the course, participating in activities, such as, for example, watching videos and responding to exercises. This means that out of the twenty-four reasons mentioned by the authors, twelve of them (50%) are related to the development of the student during the course. Table 5 lists these eleven reasons:

Table 5. Reasons related to the development of the student during the course

It is important to point out that the reasons shown in Table 5, which means those related to the development of the student during the course, are more critical than those inherent to the characteristics of the MOOCs and should be given a greater level of attention on the part of the course managers, as based on the awareness of what causes have had most effect on the dropout rate of a given course, it would be possible to adopt strategies to reduce the dropout rates.

In specialised literature it is possible to find elements that could collaborate to propose solutions to address the causes mentioned in table 5. Initially, for reason 1 (lack of social interaction and cooperative activity between the students and group work), Khalil & Ebner (2014) suggest that, in discussion forums, as well as the answer given by the teacher, in addition, the students should be encouraged to answer each other, thereby increasing the sharing of supplementary resources and also making it possible that the students could feel as part of a virtual learning community and that they can resort to this when they need help.

For reason 4 (lack of sufficient time to follow the course), reason 6 (difficulty in relating concepts with applications) and reason 22 (difficulty in following the course content), it would be possible to use the principles shown in the work by Blanco, Garcia-Penalvo, & Sein-Echaluce (2013), that proposes the use of an adaptive approach by most current courses. The adaptive approach bears in mind the heterogeneous aspect of the students, suggesting individualised learning paths, where a certain activity could be interesting for one individual or group of students, but not for all.

Also according to the same authors, the adaptive system, based on a diagnostic assessment of the students, proposes personalised activities for each student profile, it also being possible to group the participants by affinity (similar context and learninng targets), thereby contributing to the carrying out of collaborative activities.

For reason 8 (assessment model), a research study (Garcia-Penalvo, Hermo, Blanco, & Sein-Echaluce, 2014) mentions that the participants demand other assessment models, different from the tests that are used by most courses. For example, they suggest adopting the method of peer reviewing with more feedback regarding the faults committed during the appraisal process. One example in this direction is the course on Principles of Macroeconomics by Melbourne University, where the students help each other out through forums and social networks. There is also peer appraisal, where a certain student writes a text of 1,500 words which is assessed by three other students (Coffrin, Barba, Corrin, & Kennedy, 2014).

In relation to the quality of the material (reason 16), especially videos and exercises, the system known as Precise Effectiveness Strategy (PES) is proposed as a methodology to measure the effectiveness of the students when interacting with educational activities and resources. The system uses measurements to calculate the effectiveness of students when they use, for example, lectures on video and automatic correction of exercises. The PES establishes that the conclusion of the resource (a video, for example) implies a correct interaction with the activity. Therefore, a resource is completed when a student solves an exercise correctly, but not when the student tries to solve the exercise without success (Muñoz-Merino, Ruipérez-Valiente, Alario-Hoyos, Perez-Sanagustin, & Kloos, 2014).

For reason 17 (lack of maturity of the student), if the learning activities depend on group action, the students who are immature or unprepared may harm the learning of the other students. In this case, it would be important to define realistic procedures for support and guidance (considering the nature of the MOOCs). Typically, the only type of help offered in most MOOCs is peer assistance, where the students help each other, and also the possibility of the appointment of a moderator to collaborate with group discussions.

Reason 10 (limitations on the use of Information Technology (IT)) and reason 19 (high workload), even though at first they could appear to be outside the control of the managers, should also be give due attention, to propose more collaborative activities, which reduce self-study and promote a community feeling among the students. For the issue of difficulty with technology, many students could have problems regarding the use of the available internal tools, for the discussion forums, for example. In this case, it could be possible to make use of external tools, such as Facebook and some solutions as proposed by Google (Google Docs  and Google+) (Zheng, Rosson, Shih, & Carrol, Understanding Student Motivation, Behaviors and Perceptions in MOOCs, 2015).

As one final example, in cases of reason 3 (lack of motivation on the part of the students) and reason 7 (level of course different from initial expectations), the work by Gené, Nunes, & Blanco (2014), shows the application of the gamification concept to raise the level of motivation of the students while bringing down the dropout rates. Gamification is the practice of using elements of digital games in products and services to improve the experience of the users (de-Marcos, Dominguez, & Saenz-de-Navarrete, 2014). This concept could be implemented, for example, during the process of student assessment, when, after the hosting of quizzes in the Moodle environment, the elements of games are used, for example, with the use of “ranking ratings” for the classification of the students after each activity.

It is important to consider that the open and heterogeneous character of the MOOCS allows the participation of students with a wide range of learning profiles and interests. Thus, it would be fundamental for the reduction of the high dropout rates, based on a diagnostic appraisal to become aware of the profile of each student, to propose personalised activities for each student or group of students. In this case, it would be possible to propose activities and select materials that are best suited to the students with similar learning goals and context, in stark contrast to the rigid model which currently prevails, by which all students are treated in the same way.

In this regard, the studies and projects of implementation of adaptive MOOCs are very important, as they allow the establishment of different strategies for individualised learning based on data originated from then students regarding usage of, and interaction with, the environment, as also customised learning strategies for groups of students.

5. Conclusion

The research has the ultimate aim of identifying, in the relevant literature, the reasons that have led the MOOCs to show such high dropout rates. After systematic review of the relevant specialized literature on the databases Springer, Science Direct, ERIC database and ACM Digital Library, there was the identification of 24 possible reasons, which were then divided into two broad groups: one of reasons inherent to the very characteristics of the MOOCs and the other with reasons connected to the development of the student during the course.

The analysis has shown that twelve of the twenty-four reasons are inherent to the characteristics of the MOOCs themselves, due to, for example, the fact that such courses are open and without any upfront payment of fees, which has significantly contributed to high dropout rates, as many students enter only by way of curiosity and soon drop out of the course. For this group of reasons, the managers of these courses have little margin for action to propose solutions for the reduction of the high dropout rates.

It is important to stress that the twelve reasons connected to the development of the student during the course are the most critical and also deserve special attention from the course managers, because the advance knowledge of these reasons could make it possible to adopt measures to implement mechanisms that could reduce the dropout rates in the MOOCs. Based on relevant literature, some possible solutions to tackle these reasons were suggested, including the use of an adaptive approach for the course, the adoption of the process of gamification, the increase in cooperative activities between the students in the discussion forums of the courses, and the system known as PES (Precise Effectiveness Strategy) to measure the effectiveness of the students when they interact with educational resources and activities.

In terms of future work, we suggest the execution of field research with students to see if the reasons listed in the specialised literature are compatible with the difficulties that they have mentioned during the course. In addition, it would be important to have studies about the adoption of computer tools, especially Learning Analytics (LA) and Educational Data Mining (EDM), that would allow the analysis of the data generated by the students in the virtual learning environment, and would also allow the managers to make an advance analysis of the behaviour of the students, in order to predict when the student could stop attending the course and adopt any applicable measures in order to reduce dropouts in the course.

References

Almenara, J., Cejudo, M., & Vazquez Martinez, A. (2014). Las Tipologias de MOOC: Su Diseño e Implicaciones Educativas. Revista de curriculum y formación de profesorado, p. (pp.) 14-26.

Alraimi, K., Zo, H., & Ciganek, A. (2015). Understanding the MOOCs continuance: The role of openness and. Computers & Education, p. (pp.) 28-38.

Belanger, Y., & Thornton, J. (2013). Bioelectricity: A quantitative approach. Duke University First MOOC. Durham, NC.

Blanco, A., Garcia-Penalvo, F., & Sein-Echaluce, M. (2013). A methodology proposal for developing adaptative cMOOC. TEEM 2013 - Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multiculturality (ACM), p.(pp.) 553-558. Salamanca, Espanha.

Borba, M., Malheiros, A., & Amaral, R. (2011). Educação a Distância Online. Belo Horizonte: Autêntica.

Brahimi, T., & Sarirete, A. (2015). Learning outside the classroom through MOOCs. Computers in Human Behavior, 51 - parte B, p. (pp.) 604-609.

Burd, E., Smith, S., & Reisman, S. (2014). Exploring Business Models for MOOCs in Higher Education. Innovative Higher Education, p. (pp.) 1-13.

Burge, J. (2015). Insights into Teaching and Learning: Reflections on MOOC Experiences. SIGCSE '15 Proceedings of the 46th ACM Technical Symposium on Computer Science Education, p.(pp.) 600-603. Kansas City, MO, USA: ACM New York, NY, USA.

Chen, Y. (2014). Investigating MOOCs Through Blog Mining. The International Review of Research in Open and Distance Learning, p. (pp.) 85-106.

Clow, D. (2013). MOOCs and the Funnel of Participation. Proceedings LAK '13, p.(pp.) 186-189. Leuven, Bélgica.

Coetzee, D., Fox, A., Hearst, M., & Hartmann, B. (2014). Should your MOOC Forum use a reputation system? CSCW 2014 - Learning Analytics and Knowledge. Baltimore, Maryland, USA.

Coffrin, C., Barba, P., Corrin, L., & Kennedy, G. (2014). Visuzalizing patterns of student engagement and performance in MOOCs. Proceedings - LAK2014 - Learning Analytics and Knowledge. Indianapolis, USA.

Conole, G. (2014). A new classification schema for Moocs. The International Journal for Innovation and Quality in Learning (INNOQUAL), p. (pp.) 65-77.

Cooper, S., & Sahami, M. (2013). Reflections on Stanford’s MOOCs. New possibilities in online education create new challenges. Communications of the acm, 56(2), p. (pp.) 28-30.

Coursera. (2015). Coursera. Acesso em 15 de maio de 2015, disponível em https://pt.coursera.org/

Daniel, J. (2012). Making Sense of MOOCs: musing in a maze of myth, paradox and possibility. Journal of Interactive Media in Education, p. (pp.) 1-20.

Dellarocas, C., & Van Alstyne, M. (2013). Money Models for MOOCs. Considering new business models for massive open online courses. Communications of the ACM, 56(8), p. (pp.) 25-28.

de-Marcos, L., Dominguez, A., & Saenz-de-Navarrete, J. P. (2014). An empirical study comparing gamification and social networking on e-learning. Computers & Education, p. (pp.) 82-91.

Fini, A. (2009). The Technological Dimension of a Massive Open Online Course: The Case of the CCK08 Course Tools. International Review of Research in Open and Distance Learning, 10(5).

Fournier, H., Kop, R., & Durand, G. (2014). Challenges to Research in MOOCS. Journal of Online Learning and Teaching, 10(1).

Garcia_Penalvo, F., Hermo, V., Blanco, A., & Sein-Echaluce, M. (2014). Applied Educational Innovation MOOC: Learners Experience and Valorization of Strengths and Weaknesses. TEEM 2014 - Proceedings of the Second International Conference on Technological Ecosystem for Enhancing Multiculturality (ACM), p. (pp.) 139-145. Salamanca, Espanha.

Gené, O., Nunes, M., & Blanco, A. (2014). Gamification in MOOC: Challenges, Oportunities and Proposal for Advancing MOOC Model. TEEM 2014 - Proceedings of the Second International Conference on Technological Ecosystem for Enhancing Multiculturality (ACM), p.(pp.) 215-220. Salamanca, Espanha.

Gomes-Zermeno, M., & De La Garza, L. (2016). Research Analysis on Mooc Course Dropout and Retention Rates. Turkish Online Journal of Distance Education-TOJDE, 17(2), p. (pp.) 3-14.

Guo, P., & Reinecke, K. (2014). Demographic Differences in How Students Navigate Through MOOCs. In Proceedings of the first ACM conference on Learning@ scale conference, p. (pp.) 21-30, ACM.

Hew, K., & Cheung, W. (2014). Students and Instructors use of massive open online courses (MOOCs): motivations and challenges. Educacional Research Review, pp. 45-58.

Hyman, P. (2012). In the Year of Disruptive Education. Communications of the ACM, 55(12), p. (pp.) 20-22.

Jordan, K. (2014). Initial Trends in Enrolment and Completion of Massive Open Online Courses. The International Review of Research in Open and Distance Learning, p. (pp.) 133-160.

Kennedy, J. (2014). Characteristics of Massive Open Online Courses (MOOCs): A research review, 2009-2012. Journal of Interactive Online Learning, 13(1), p. (pp.) 1-16.

Khalil, H., & Ebner, M. (2014). MOOCs Completion Rates and Possible Methods to Improve Retention - A Literature Review. Proceeding of World Conference on Educational Multimedia, Hypermidia and Telecommunications, p. (pp.) 1236-1244. Chesapeake, VA.

Kop, R., Fournier, H., & Mak, J. (2011). A Pedagogy of Abundance or a Pedagogy to Support Human Beings? Participant Support on Massive Open Online Courses. The International Review of Research in Open and Distance Learning, p. (pp.) 75-93.

Little, G. (2013). Massively Open? The Journal of Academic Librarianship, p. (pp.) 308-309.

Liu, M., Kang, J., Cao, M., Lim, M., Ko, Y., Myers, R.(2014). Understanding MOOCs as an Emerging Online Learning Too: Perspectives from the Students. American Journal of Distance Education, p. (pp.) 147-159.

Liyanagunawardena, T., Adams, A., Rassol, N., & Williams, S. (2014). Developing government policies for distance education: Lessons learnt from two Sri Lankan Case Studies. International Review of Educational, p. (pp.) 1-19.

Mallon, M. (2013). MOOCs. Public Services Quarterly, 9 (1), p. (pp.) 46-53.

Maringe, F., & Sing, N. (2014). Teaching large classes in increasingly internationalising higher education environment: pedagogical, quality and equity issues. Higher Education, 67(6), p. (pp.) 761-782.

Morris, L. (2013). MOOCs, Emerging Technologies and Quality. Innovative Higher Education, 38(4), p. (pp.) 251-252.

Muñoz-Merino, P., Ruipérez-Valiente, J., Alario-Hoyos, C., Perez-Sanagustin, M., & Kloos, C. (2014). Precise Effectiveness Strategy for Analyzing the Effectiveness of Students. Computer in Human Behavior, 47, p. (pp.) 1-11.

Nawrot, I., & Docet, A. (2014). Building Engagement for MOOC Students. Introducing Support for Time Management on Online Learning Platforms. Proceedings of the 23rd International Conference on World Wide Web (pp. 1077-1082). Seul, Korea: ACM New York, NY, USA.

Ong, B., & Grigoryan, A. (2014). MOOCs and Universities: Competitors or Partners? International Journal of Information and Education Technology, 5(5), p. (pp.) 373-376.

Open Education Europa. (2015). European MOOCs Scoreboard. Acesso em 23 de setembro de 2015, disponível em The European MOOCs Scoreboard: http://openeducationeuropa.eu/sites/default/files/images/scoreboard/Scoreboard_SEPTEMBER_2015.png

Poy, R., & Gonzalles-Aguilar, A. (2014). Factores de éxito de los MOOC: algunas consideraciones críticas. Revista Ibérica de Tecnologia y Sistemas de Información, pp. 105-118.

Quinn, J. (2013). Drop out and Completion in Higher Education in Europe. Acesso em 2015 de julho de 09, disponível em http://www.nesetweb.eu/sites/default/files/HE%20Drop%20out%20AR%20Final.pdf

Read, T., & Rodrigo, C. (2014). Toward a Quality Model for UNED MOOCs. eLearning Papers, n. 37.

Rodriguez, O. (2012). MOOCs and the AI-Stanford like courses: Two successful and distinct course formats for Massive Open Online Courses. The European Journal of Open Distance and E-Learning, 15(2), pp. 1-13.

Rosewell, J., & Jansen, D. (2014). The OpenupEd quality label: Benchmarks for MOOCs. The International Journal for Innovation and Quality in Learning, 2(3), p. (pp.) 88-100.

Rosselle, M., Caron, P., & Heutte, J. (2014). A typology and dimensions of a description framework for MOOCs. Proceedings of the European MOOCs Stakeholoders Summit, p. (pp.) 130-139.

Roval, A., & Downey, J. (2010). Why some distance education programs fail while others succeed in a global environment. Internet and Higher Education, 13(3), p.(pp.) 141-147.

Saadatmand, M., & Kumpulainen, K. (2014). Participants Perceptions of Learning and Networking in Connectivism MOOCs. MERLOT Journal Online Learning and Teatching, pp. 16-30.

Sandeen, C. (2013). Integrating MOOCs into Traditional Higher Education: The emerging "MOOC 3.0" Era. The Magazine of Higher Learning, 45(6), p. (pp.) 34-39.

Vardi, M. (2012). Will MOOCs Destroy Academia? Communications of the acm, 55(11), p. (pp.) 5.

Walker, L., & Lock, B. (2014). Academics Perception on the Quality of Moocs: an empirical study. The International Journal for Innovation and Quality (INNOQUAL), 2(3), pp. 53-63.

Welsh, D., & Dragusin, M. (2013). The New Generation of Massive Open Online Course (MOOCS) and Entrepreneurship Education. Small Business Institute Journal, 9(1), 51-65.

Wilkowski, J., Deutsch, A., & Russell, D. (2014). Student Skill and Goal Achievement in the Mapping with Google MOOC. In Proceedings of L@S 2014 - Student Skills and Behavior, p (pp.) 3-10. Atlanta, Georgia, USA.: ACM.

Zheng, S., Han, K., Rosson, M., & Carrol, J. (2016). The Role of Social Media in MOOCs: How to Use Social Media to Enhance Student Retention. In Proceedings of the Third ACM Conference on Learning@ Scale , p.(pp.) 419-428. Edinburgh, Scotland, UK.: ACM - Association for Computing Machinery.

Zheng, S., Rosson, M., Shih, P., & Carrol, J. (2015). Designing MOOCs as Interactive Places for Collaborative Learning. In Proceedings of the Second ACM Conference on Learning@ Scale, p. (pp.) 343-346. Vancouver, Canada: ACM - Association for Computing Machinery.

Zheng, S., Rosson, M., Shih, P., & Carrol, J. (2015). Understanding Student Motivation, Behaviors and Perceptions in MOOCs. CSCW '15 Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, p. (pp.) 1882-1895. Vancouver, BC, Canada: ACM New York, NY, USA.

Zutshi, S., O´Hre, S., & Rodafinos, A. (2013). Experiences in MOOCs: The Perspective of Students. American Journal of Distance Education, 27(4), p. (pp.) 218-227.

1. Aluno de Doutorado do Programa de Pós Graduação Stricto Sensu em Engenharia de Produção da Universidade Paulista – UNIP. Email: luisnaito@yahoo.com.br

2. Professor do Programa de Pós Graduação Stricto Sensu  em Engenharia de Produção da Universidade Paulista – UNIP. Email: marcia.terra@uol.com.br