ISSN 0798 1015

Vol. 39 (# 21) Year 2018. Page 12

Methods of study of research competence maturity of engineering students

Métodos de estudio de la madurez competitiva de investigación de estudiantes de ingeniería

Oksana Olegovna GORSHKOVA 1

Received: 12/01/2018 • Approved: 10/02/2018

Contents

1. Introduction

2. Methods

3. Results and discussion

4. Conclusion

References

1. Introduction

The quality of engineering personnel is one of the key factors for the competitiveness of the state, the basis for its technological and economic independence, and the driving force of the technological transformation of its society. In this regard, the creation and implementation of a holistic concept of teaching research activities to engineering students, which is aimed at the formation of their research competence, appear to be relevant and topical.

The study of such a component as the research competence brings forth the interdependence between its formation processes and personal and professional development of a future engineer. Indeed, the formation of a competent engineer who is able to solve professional problems in an efficient way involves not only mastering knowledge and experience but also the search for the meaning of the performed research activities and awareness of one’s role in such activities and responsibility for them. From this perspective, it would be right to consider professional training as a process, the common starting point of which is the formation of personality that determines manifestation of the future engineers in their professional activity: awareness of the significant objectives, appearance of the need for research activities, and an ambition to fulfill one’s potential in creative way. The aim of the experiment was to check efficiency of the engineering students’ research training concept that was implemented through the functional model focused on the formation of the students’ research competence. '

2. Methods

Theoretical (the study, analysis, and synthesis of pedagogical, social, engineering and economic study materials related to the examined issue; the analysis of the subject of research; the summary of the research results); empirical (study of statutory documents, observation, interviews, testing, self-assessment, analysis of documents, examination of activity results, instructional design); experimental (pedagogical experiment, methods of mathematical processing of results).

3. Results and discussion

The research competence of engineers is one of the basic components of their professional competence system. The results of our study allow concluding that the research competence is non-algorithmic because when applying it, students use both known and new algorithms. In the course of research activity, students are able to implement the research approach in different areas of activity and to apply it in various situations; this confirms that research competence is marked by versatility, multifunctionality, and superdisciplinarity. The fact that students apply critical, analytical, communication and other skills and personal qualities in the course of research activities illustrates the multidimensionality of the research competence. The research competence is not fixed but rather mobile and variable subject to any issue and any circumstances. Thus, the research competence has the features of the key competence; it serves as the basis for the development of the application-oriented competencies (both cross-cultural and professional ones) as it contributes to successful learning of students and helps them to become competitive and successful in their profession, i.e. it contributes to the development of competent professionals. This is what defines the significance of its formation.

Taking into account that in terms of the competence approach, preparedness is identical to the research competence, the study of different approaches to the definition of "preparedness for research activity" and "research competence" (B.G. Ananiev, A.A. Derkach (2004), E.S. Gorodtsova (2003), A.V. Khutorskoi (Competences in Education: Design Experience, 2007), V.D. Shadrikov (Shadrikov 2004), etc.) has allowed us to adjust this notion; as a result, we define the research competence of engineers as a developing integrative personal feature that ensures active research attitude towards the activity and to oneself as the subject of such activity in the context of development and functioning of innovative engineering technologies and industries. The research competence has the features of the key competence; it serves as the basis for the development of the application-oriented competencies (both cross-cultural and professional ones) as contributes to the development of competent professionals. This is what defines the significance of its formation (Gorshkova 2016).

The performed analysis of the pedagogical research and examination of the essential characteristics of an engineer’s research activity lead to the conclusion that the research competence consists of three main components: cognitive, personal and activity ones, which we use as the criteria of engineering students’ research competence. The components can have different development levels but as the parts of the system, they maintain complex interaction and interdependence relations representing a holistic functional system.

The analysis of the professional standards, the structure of engineering activity, and a range of competences shows that the research nature manifests itself as the leading component of the engineering activity components. The defined labour functions together with the necessary skills have the research component and determine the need to train engineering students for research activity.

The issues of improving the training of engineering students for research activity become especially relevant as one of the main goals of the higher education is to involve students in research activities which would "allow not only saving the world-known Russian scientific schools but also raising a new generation of researchers focused on the needs of the innovative knowledge-driven economy” (Kuzminov & Frumin, 2008).

During our study, we have systematized, defined and developed the existing approaches to addressing the issues of engineering students’ research training that contribute to development of the methodological basis of such training: research activity training through motivation; focusing on scientific knowledge of students in the learning process; use of the student-centered approach based on the inclusion of students in active learning and development of creativity; focusing on self-learning activity with the use of online resources and projects.

Taking into account the systematization and analysis results of the research competence development among engineering students, the practice of students’ research training, the defined requirements of the employers along with the requirements of professional and educational standards and subject to the age, social and pedagogical peculiarities of students, we have concluded that it is necessary to develop a concept of students’ research training for the purposes of competence-oriented engineering education. The concept contains the system of vocational training regulations based on the changes in the education quality management system and development of innovative didactics for the engineering universities based on the specifics of research training in the context of the emerging national qualification system for the purpose of forming competent graduates. We have proved that research activity serves as the basis for the formation of the cross-cultural, professional and a range of special competences subject to the requirements of educational and professional standards.

The following principles are basic for the implementation of the concept: target-oriented management which implies systematic monitoring of students’ readiness for research activity; arrangement of supportive learning environment at universities providing motivation, interaction, and collaboration of all participants united by the space; integration of teaching and research at all levels of the educational process; transformation of the engineering education content, development of practice-oriented techniques, special forms and means of students’ activities; taking into account the requirements of the employers and of the professional standards on the basis of partnership with the major enterprises; involvement of students in the education quality comanagement together with their self-organization in the course of the research training; ability to establish criteria and determine the results.

The essence of the concept is implemented in the functional model of students’ research training, which is the basis for innovative didactics of engineering universities. The specific content of the model is to ensure motivation of students for research activity and to develop their personality; to create an atmosphere of productive activity; to present the content of training in a structured fashion; to implement interdisciplinary integration and to include special courses and workshops into the professional training; to develop special forms and means of extracurricular activities together with the representatives of the major enterprises; to use online educational resources; to develop and use the tasks relevant to the professional engineering activities and targeted teaching of students how to solve such tasks; and to contribute to development of reflection and self-assessment by students.

The functional model of students’ research training was implemented in the form of an experiment carried out during the educational process at the engineering universities. The goal of the experiment was achieved within two periods: 1) a four-year experiment of testing separate elements of the model during the educational process in 2006-2010; 2) the primary experiment of implementing the model of students’ research training in 2010-2016.The total number of people involved in the experimental work was 1,520: 1,390 students and 130 professors. The primary experiment was carried out in three stages: diagnostic, formative, and summative assessments.

The general scheme of the experiment consisted of three basic stages: diagnostic, formative, and summative assessments. In order to ensure its validity, the control groups, who were trained according to the conventional system and tested at the beginning and end of their study at the university, participated in the experiment along with the experimental groups.

The research competence level of the first-year students was assessed at the first stage of diagnostic assessment. It was suggested: 1) to evaluate each component of the students’ research competence; 2) to get combined assessment of the research competence of each examinee; 3) to distribute the scores according to each criterion; 4) to compare the obtained distribution results. The experimental and control groups were selected according to the results.

The second stage of the experiment – formative assessment – included implementation of the functional model at the university; we clarified and improved the model and evaluated the possibility for its implementation in the educational process of the university.

The final stage – summative assessment – was aimed at evaluating the efficiency of the functional model and identification of the conditions for its efficient functioning. We carried out the final testing of the examinees (graduate students) and identified the dynamics of the research competence development; then we systematized, summarized, evaluated and analyzed the results of the experiment.

An important issue was to select the tools that could be used at all stages of the experiment. The measuring tools had to meet a number of requirements. First of all, the measurement methods should complement, check and verify each other. Secondly, the measurement information should clearly express the progress of formation of preparedness for research activity among the future engineers, as well as be objective and adequate. In addition, we had to take into account the time required for measurement of a given component, i.e. the measurement should take little time. In order to determine the level of maturity for research activity, we used the following methods: testing, surveys, ranking, self-assessment, observation, analysis of various students’ paperwork, etc.

The maturity of the motivation component in the students’ research competence was identified by means of surveys, lesson observation, individual interviews, and analysis of the results of the students’ activities. The quality indicators of the students that were controlled empirically were their interest in mastering research methods; self-dependence and perseverance in overcoming difficulties when performing research; commitment to self-development and preparedness to learn more than the training programs could offer. The level of students’ commitment in mastering the research activity was taken as an indicator as well.

With a view of obtaining reliable tools and selecting the examined material, we used K. Zamfir’s method of testing professional activity motivation as modified by A.N. Rean (Raigorodskii, 1998), which was based on the concept of intrinsic and extrinsic motivation. Motivational complex is the correlation of three types of motivation: intrinsic motivation, extrinsic positive motivation, and extrinsic negative motivation. The motivational component of the research competence is presented as a structure in which the following take the lead: interest in mastering research methods; self-dependence; active participation in research activity; willingness to take part in the contests of research works, to speak at the scientific conferences and seminars; perseverance in overcoming difficulties when solving research tasks; commitment to self-development; willingness to learn more than the training programs can offer, etc. The correlation of the motivational complex components and the levels of the motivational component maturity is illustrated in Table 1. 

Table 1
The correlation of the motivational complex components and the levels of the motivational component maturity

We used T. Ehlers’s Motivation to Success test (Raigorodskii 1998). The responses were scored and then the total number of points was calculated. If students got 1 to 10 points, it meant they had a deficient level of motivational component. If students got 11 to 16 points, they had a low level of motivational component, 17 to 20 points – a middle level of motivational component, and more than 21 points – a high level of motivational component.

We studied the maturity degree of the students’ interest in and commitment to development of the research competence and their ability to reflect by means of observation, surveys, and empirical data obtained by applying A.V. Karpov’s methodology for determining individual measure of reflexivity (Karpov 2003). The students were asked to fill in a questionnaire; they had to put a digit corresponding to the variant of the response opposite to the question number. In total, there were 27 statements: 15 of them were direct while the rest 12 statements were converse, which was necessary to take into account when processing the results. In order to normalize the results, we took the sten (standard ten) score scale to translate test points into the normalized indicators. When interpreting the results, we proceeded from dividing the students into four categories. The results of the method that equaled or exceeded 7 sten scores indicated highly developed reflexivity. The results of 4 to 7 sten scores were the indicators of the middle level. The results of 3 to 1 sten scores were the indicators of the low level of reflexivity and 0 sten score meant zero level.

High results suggest that students are inclined to turn to analysis in their activity and identify the causes and consequences of their actions to a great extent. It is typical for such students to think their activity over, carefully plan it and predict its possible consequences. Low results indicate that an examinee is hardly inclined to think over their actions and consequences of such actions.

The cognitive component of the research competence was evaluated according to the selected indicators by offering the students to answer a number of questions in order to identify the knowledge required to perform research activity. The students were tested on the knowledge of the studied disciplines. The tests of knowledge were carried out by both open and closed tests, interviews, and surveys using direct and indirect questions. The use of survey methods allowed to identify the students’ knowledge about what part the research took in the professional activity of an engineer, as well as about the types of research tasks, requirements to their results, methods of implementing research actions, methods of solving research tasks, and terms of their application, about the types of research projects and programs, the requirements to such projects and programs, etc. The awareness of the necessity to conduct research in order to obtain any new knowledge was verified in the course of solving the research tasks by the students, as well as according to the results of its solving and oral justification of the method selected to solve the task. The awareness of the methods for solving research tasks and terms of their application was identified in a similar way.

If students had the necessary knowledge to deal with more than 70% of the offered tasks, it meant that he had a highly developed level of the cognitive component. Such students were acutely aware of importance and significance of research in an engineer’s activity, learned the types of research tasks and requirements to their solution thoroughly and in full, and learned the methods of their solving and terms of application of such methods. If students were able to solve 35-69% of the tasks, they had an average level of cognitive component. Such students had detailed and in-depth knowledge sufficient to solve most types of research tasks and were aware of the importance and significance of research in an engineer’s activity.

If students had the knowledge to be able to solve 15-34% of the offered tasks, they had a low level of the cognitive component. Such students were dimly aware of the importance and significance of research in an engineer’s activity. They did not learn everything required to implement the research activities. The cognitive component was considered as not developed if students did not have sufficient knowledge to be able to solve less than 15% of the offered research tasks. They understood what part research takes in their future professional activity but their knowledge was random and weak.

The activity component of the research competence was tested by asking the students to solve some research tasks. In addition, the ability to formulate research tasks and to identify requirements to the results of their solution was determined by means of special tasks, by monitoring the students when they were doing the research, and by evaluation of the students’ oral (written) justification of their logic how to identify the content, objectives, methods, and timing to implement the research activity. The abilities to plan the research, to define the structure of the research actions, and to select the appropriate research methods were evaluated similarly. Basically, we evaluated a set of skills to be able to identify one’s need in some knowledge and to formulate an image how to acquire such knowledge in the current circumstances. All the examinees were offered a certain number of tasks and were asked to plan how to solve them by defining the requirements to the results of their solving and a range of the required research activities. They had to select the appropriate methods and to evaluate the quality of the offered research projects, programs, etc.

The students with a high level of the activity component (who solved more than 70% of the tasks) were able to formulate all types of research tasks and to plan how to solve them on their own. They showed the ability to select the appropriate research methods and always evaluated the quality of the research programs objectively. The students who had an average level of the activity component (35-69% of the tasks) were able to formulate most types of research tasks and to plan how to solve them on their own but when they dealt with some tough stuff, they experienced difficulties and failed to make an adequate plan. In most cases, they demonstrated the ability to select adequate research methods and in some cases, it was difficult for them to evaluate the research programs.

According to our classification, the students who were able to plan how to solve 15-34% of research tasks had a low maturity level of the activity component. Such students were able to formulate only some types of research tasks and used standard plans for their solving, i.e. they acted unproductively. In cases when it was not possible to plan the research with standard methods, such students experienced serious difficulties and could rarely solve them on their own. The students who were not able to plan the solution of less than 15% of the tasks were considered to have a zero maturity level of activity component.

In order to evaluate the maturity indicators of activity component, we used the following ratios (Stavrinova 2006): complete skill mastering ratio: k= n /N, where n was the number of correctly applied research methods; N was the number of research methods required to solve a research task; sustainable skill mastering ratio: g = k2/k, where k1 was the complete skill mastering ratio estimated at the first check; k2 was the complete skill mastering ratio estimated at the next check.

The students who solved more than 70% of the offered tasks and received 35 to 50 points were considered to have a high maturity level of the activity component. Such students were able to apply all the basic methods required to solve the engineering research tasks under the existing terms and conditions. The students who solved more than 35-69% of the research tasks and received 20 to 35 points were considered to have an average maturity level of the activity component: they were able to apply a part of the basic research methods under the existing terms and conditions. The students who solved more than 15-34% of the offered tasks and received 5 to 20 points were considered to have a low maturity level of the activity component. Such students were able to apply some basic research methods under the existing terms and conditions. The students who received 0 to 5 points were considered to have zero maturity level of the activity component.

Table 2 presents the indicators in respect of each component of the research competence. The proposed percentage of the tasks correlates with the projected maturity level of the components of the students’ research competence. 

Table 2
Maturity indicators of the research competence components

In the course of the experiment, we realized that the maturity level of each research competence criterion could vary and the presented levels could not be always identified with the same values. Therefore, we have developed the research competence combined assessment scale, which takes into account a mixed gradation that combines two adjacent levels. It allowed taking into account the maturity variation of each criterion. Table 3 shows the combined assessment scale of engineering students’ research competence. 

Table 3
Research competence combined assessment scale

The results of the evaluation of students’ research competence maturity were summarized by means of diagnostic assessment checklists per each student and the group as a whole. In the course of the experiment, we were able to test the components of the students’ research competence at various stages, which helped to prove the efficiency of the functional model.

During the diagnostic assessment, we evaluated the maturity of the research competence of first-year students (2010) and graduate students (2010). The results turned out to be rather low (see Table 4) and reflected the outcome of the conventional training process at the engineering university.

Table 4
Research competence of graduate students

The comparison of the data shows that the students with a low maturity level of the research competence dominate: they are yet ready to solve engineering research problems on their own.

We have concluded that at the start of the experiment, the first-year students have not yet developed their research competence; the results are shown in Table 5. 

Table 5
Evaluation of the maturity of first-year students’ research competence, %

The similarities of the results of the first-year students from both universities are obvious. The students are not interested in research activity; they are not aware of the research content or methods of its implementation in an engineer’s activity and not able to realize the necessity of commitment to professional self-development, which leads to passive attitude towards learning.

By analyzing the data contained in Table 6 (Gorcshkova 2017), it is possible to trace the dynamics of the students’ research competence maturity at different stages of the experiment and to assess the efficiency of implementing the functional model. 

Table 6
Dynamics of research competence maturity, %

The changes in the experimental groups are stable across all components. The positive developments seen in the control groups mean that in the framework of conventional education, the level of students’ preparedness for research activity increases progressively as they gain experience of learning and research work but this process is rather slow.

The results of the experiment were analyzed at each stage and presented in the format of analytical tables and diagrams that allowed tracing the dynamics of the changes that had occurred as a result of purposeful influence on the research training process and evaluating the efficiency of the model. The final state of the research competence of students from the experimental and control groups is shown in Figure 1.

Figure 1
Final state of the students’ research competence

The comparison of the results at the start of the experiment with the final results according to the combined assessment of the research competence showed statistically significant differences in the experimental groups (see Table 7), while in the control groups, the changes were not so significant.

Table 7
Combined assessment of the research competence, %

The authenticity of the results was confirmed by the test based on statistical analysis with the use of mathematical statistics criteria: Pearson's chi-squared test. As the result of the implementation of the functional model, the students from the experimental groups showed statistically significant changes in the maturity levels of all components of the research competence; moreover, these differences were statistically significant in comparison with the results of the control groups.

4. Conclusion

The analysis of the experiment results and the comparison of the indicators of the experimental and control groups demonstrated the efficiency of the developed concept implemented by means of the functional model of student research training for the purposes of competence-oriented engineering education. Our assumption on the efficiency of the identified conditions that affected the process and results of the formation of preparedness for research activity was experimentally substantiated. 

References

Derkach, A.A. (2004). Akmeologicheskie osnovy razvitiya professionala [Acmeological Professional Development Framework]. Moscow, Voronezh. (p. 752).

Gorcshkova, O.O. (2017). Network Educational Resources in the Research Student Training Revista. Revista ESPACIOS, 38(25).

Gorodtsova, E.S. (2003). Organizatsiya issledovatelskoi raboty studentov i issledovatelskaya kompetentnost [Organization of Students’ Research Activity and Research Competence]. Vestnik Instituta psikhologii i pedagogiki, 1, 121-128.

Gorshkova, O.O. (2016). Podgotovka studentov k issledovatelskoi deyatelnosti v kontekste kompetentnostno-orientirovannogo inzhenernogo obrazovaniya: Dis. ... d- ra ped. nauk [Training of Students for Research Activity in the Context of Competence-oriented Engineering Education (Doctoral Thesis Abstract)]. Moscow. (p. 394).

Karpov, A.V. (2003). Refleksivnost kak psikhicheskoe svoistvo i metodika ee diagnostiki [Reflexivity as a Psychological Attribute and its Diagnostics Methods]. Psikhologicheskii zhurnal, 24(5), 45-57.

Kompetentsii v obrazovanii: opyt proektirovaniya [Competences in Education: Design Experience]. Edited volume under the editorship of A.V. Khutorskoi. (2007). Moscow: INEK. (p. 527).

Kuzminov, Ya., & Frumin, I. (Eds.). (2008). Rossiiskoe obrazovanie – 2020: model obrazovaniya dlya ekonomiki, osnovannoi na znaniyakh [Education in Russia – 2020: Educational Model Based on Knowledge]. Moscow: The Higher School of Economics Publishing House. (p. 39).

Raigorodskii, D.Ya. (Ed.). (1998). Prakticheskaya psikhodiagnostika. Metodiki i testy. Uchebnoe posobie [Practical Psychodiagnosis: Methods and Tests. Tutorial]. Samara. (p. 672).

Shadrikov, V.D. (2004). Novaya model spetsialista: innovatsionnaya podgotovka i kompetentnostnyi podkhod [New Specialist Model: Innovative Training and Competence Approach]. Vysshee obrazovanie segodnya, 8, 27-33.

Stavrinova, N.N. (2006). Sistema formirovaniya gotovnosti budushchikh pedagogov k issledovatelskoi deyatelnosti: Dis. ... d-ra ped. nauk [Formation of Willingness of Future Teachers to Research Activity (Doctoral Thesis Abstract)]. Surgut. (p. 356).

1. Federal State Budget Educational Institution of Higher Education “Industrial University of Tyumen” 38 Volodarskogo St., Tyumen, Tyumen Region, Ural Federal District, 625000, Russia, E-mail: gorchkovaoksana@mail.ru