Evaluation of the Level of Prognostic Competence with Fuzzy Logic Systems

Abstract

The chapter deals with main components are part of the prognostic competence. The chapter discusses possibility of using intelligent systems to model the predictive competence of a civil engineer. The analysis of the study of the problem was carried out using content analysis A model of the development of prognostic competence of a civil engineer using the Matlab software package with the Fuzzy Logic Toolbox module has been developed. A step-by-step forecasting technology, as well as possible errors in creating a forecast by an expert and ways of solving them, are proposed. In conclusion the chapter says most of all the level of formation of predictive competence depends on activity component.

1. Introduction

There are only the last two major catastrophes, which allegedly occurred due to the fault of civil engineers and other specialists from related industries: fire in the mall “Zimniya Vishnia” on March 25, 2018 on Kemerovo and the collapse of the Morandi automobile bridge on August 14, 2018. But this list could be continued with a huge number of examples. That’s why the formation of prognostic competence of construction university students as future civil engineers is very important. Predicting a future catastrophe is easier than eliminating its consequences. Every specialist, especially representatives of engineering professions, must possess prognostic skills [1]. Pedagogical forecasting, in contrast to other branches of science about the future state of the object under study, is fraught with a number of difficulties. This is due to the complexity of the predicted object: the changeable personality of the student, the development of which depends on a number of factors, both external and internal. Formalization simplifies the object and does not allow to include all indicators, factors and relationships in the model.

2. Analysis of the problem field

To prove the relevance of the chosen direction of research, the analysis of publication activity for a given keyword “pedagogical forecasting” in the world literature was carried out; found the number of articles published in a certain period of time, the result is plotted in Figure 1.

Figure 1.

The number of publications on the practice of using pedagogical forecasting (according to the Google Scholar database).

We see the growing interest of scientists in the use of pedagogical forecasting tools in 2008, 2013 and 2019, which is associated with technological breakthroughs in the field of computer technology. The introduction of computer technology in the educational system is a priority of modern public policy. Intelligent systems are widely used. This technology is application promising in the field of pedagogical forecasting and the formation of the prognostic competence of a specialist. The theory of forecasting is considered in the works of I.V. Bestuzheva-Lada et al. [2], K. Beecher, D. Berry, J. Maliekal, O. Yasar, R.S.J.D. Baker [3], A. Dutt [4], B. Oancea, C. Romero, S. Ventura, M. Pechenizkiy et al. [5] are engaged in research in the application of Educational Data Mining technology for modeling educational processes. However, only a small amount of work is devoted to the formation of the prognostic competencies of a civil engineer.

3. Task

To build models of the civil engineer predictive competence formation with using the Matlab R2014a software package with the Fuzzy Logic Toolbox module depending on different factors and components.

4. Development of methodology

4.1 Prognostic competence

Forecasting is a cognitive activity aimed at revealing the features and characteristics of the personality development processes of a student of a construction university and the consequences expected from them; an indication of the path and conditions for the implementation of foresight. Predictive competence [6] is the result of education including the level of professional preparedness of a student, knowledge, skills, forecasting skills allow to possess the knowledge and skills of the forecasting process, selection and logical processing of necessary information, analysis and determination of trends have the skills of planning, goal setting, programming, design; develop the ability to understand opportunities, abilities, ways to improve.

4.2 Component composition of professional competencies

Table 1 shows the characteristics of each of the components of the development of competence [7].

Levels	Component composition of professional competencies
	Cognitive	Activity	Motivational	Communicative
C	the student is ready to reproduce the studied algorithms for solving professional problems	the student is ready to solve professional problems according to a given algorithm	the student is ready to modify the existing sample of tasks for a given algorithm	the student is ready to adequately analyze and evaluate their actions and the actions of members of the educational team
B	the student is ready to offer similar algorithms studied in solving standard problems	the student is ready to solve standard professional problems by analogy with the studied algorithm	the student is ready to upgrade professional tasks according to original algorithms	the student is ready to plan, organize and adjust their activities and the activities of the members of the educational team
A	the student is ready to develop his original algorithms for solving professional problems	he student is ready to apply original algorithms in the course of professional activity	the student is ready to simulate non-standard situations containing functional professional novelty	the student is ready to be responsible for the results of professional activity, to model and predict their further professional development

Table 1.

Characteristics of the levels of development of professional competencies of future engineers.

A-level of prognostic competence and its components will be estimated at 100 points, B-level – at 75 points, C-level – 60 points, formal level below 60 points (F, we estimate in 0 points).

Analysis of the scientific literature on the relationship of knowledge, skills and personality traits in the overall effective assessment of training allowed us to identify the following weights of the components of competence: the cognitive component is.2; activity component is.5; motivational component is.15, communicative component is.15 [8].

4.3 Rules and methodology of modeling of dependence of prognostic competence general level depend on the development of its components

We will use the methodology is described in [9] and theoretical principles is described in [10].

Matlab R2014a software package with Fuzzy Logic Toolbox module will be the main tool of our research. We will use the Mamdani algorithm mode. We will need 4 input variables to describe the level of development of the prognostic competence component and 1 outgoing variable to describe the general level. The type of the membership function is gaussmf, since most pedagogical phenomena are subject to the normal Gauss distribution [10].

To formulate the rules of fuzzy logic we will use the MS Excel 2007. We have formulated 256 fuzzy logic rules (4^4), but in article we will present only a few examples:

• If Cognitive component is in B-level AND Activity component is in A-level AND Motivational component is in B-level AND Communicative component is in B-level THEN General level of prognostic competence formation is in B-level;

• If Cognitive component is in B-level AND Activity component is in B-level AND Motivational component is in A-level AND Communicative component is in formal level THEN General level of predictive competence formation is in C-level;

• If Cognitive component is in A-level AND Activity component is in B-level AND Motivational component is in B-level AND Communicative component is in C-level THEN General level of prognostic competency formation is in C-level;

• If Cognitive component is in A-level AND Activity component is in C-level l AND Motivational component is in B-level AND Communicative component is in formal level THEN General level of prognostic competence development is in C-level.

4.4 The most factors influent on prognostic competence

The authors [11] concluded that pedagogical factors influence the formation of prognostic competence by 31%, psychological group of factors – by 28%, social and economic conditions: 33%.

4.5 Rules and methodology of modeling of dependence of prognostic competence general level depend on the development of the most factors

We used the Mamdani algorithm mode. We needed 3 input variables, each of which corresponded to a factor influencing the formation of predictive competence, and 1 outgoing – the level of competence formation. The type of membership function for predictive competence is gaussmf, since most pedagogical phenomena are subject to the normal Gauss distribution [10]. For factors that determine the development of prognostic competence – the type of membership function – trimf.

The evaluation interval for the influence of each of the factors is 0–100, which corresponds to conditional estimates: “low level” (F), “below average” (D), “average level of efficiency” (C), “above average” (B), “High level” (A). The creative level of the development of predictive competence will be estimated at 100 points (A), productive – 75 points (B), reproductive – 60 points (D), formal level – below 60 points, but above 20 (F).

To formulate the rules of fuzzy logic we will use the MS Excel 2007. We have formulated 125 fuzzy logic rules (5^3), but in article we will present only a few examples:

• If pedagogical factors is developed on A-level AND psychological factors is developed on A-level AND social factors is developed on A-level THEN prognostic competence is developed on A-level

• If pedagogical factors is developed on F-level AND psychological factors is developed on B-level AND social factors is developed on A-level THEN prognostic competence is developed on D-level

4.6 Rules and methodology of modeling of dependence of prognostic competence general level depend on selected teaching model and frequency of application of prognostic activity

Every person at least once a day makes plans for the next day. Therefore, as a very low frequency of predictive activity, we will take 7 times. If on each pair 2–3 problems of predictive nature are solved, then about 50 prediction problems will be solved. This frequency is taken as a very high level of the frequency of predictive activity. Conventionally, we distinguish the following frequency levels (once a week): very low (FX), low (F), below average (E), medium (D), above average (C), high (B), very high (A). The method is trimf.

We will designate prediction learning models as follows: lack of special prediction learning – FX, non-systemic prediction learning – F, semiotic educational model – D, simulation educational model – B, social educational model – A. Method – trimf. The range is from 0 to 100 conventional units.

The creative level of the development of predictive competence will be estimated at 100 points (A), productive – 75 points (B), reproductive – 60 points (D), formal level – below 60 points, but above 20 (F). The type of membership function is gaussmf.

We have made 16 rules of fuzzy logic. Some of them are presented below:

• IF the prediction activity frequency is in A-level AND the training B-model is applied to THEN the predictive competence is developed on A-level;

• IF the frequency of prediction activity is in E-level AND the training B-model is applied to THEN predictive competence formed on D-level.

4.7 Forecasting technology

Forecasting is a stepwise process. The developed system of organizational and pedagogical conditions for each stage of forecasting is presented in the table format (Table 2).

Stages		Organizational and pedagogical conditions	Possible sources of forecast errors	Ways to troubleshoot errors	Application time
Flashback stage
1	preliminary stage: obtaining the task and forecasting constraints; choice of subject and object	study of state standards and other regulatory documents; Web Mining; Text mining	the customer’s idea of ​the predicted object can be vague, which leads to an unclear formulation of the task	are eliminated by an expert at the next stage	before the start of the educational process
2	predictive justification: the stage of setting goals and objectives of pedagogical forecasting; tracking the results of preparatory information; formation of expert groups and assessment of the competence of expert analysts	training experts in forecasting and methods of computer pedagogy, specifics of forecasting in the professional training of students in construction areas of training; Web Mining; Text mining	research of a new object for an expert can lead to incorrect formulation of goals and objectives, which can lead to unexpected results (or their absence); incorrect understanding by the expert of the tasks set by the customer of the forecast	clarification of goals after conducting pre-predictive orientation; discussion of goals and objectives with the customer	before the start of the educational process
3	pre-predictive orientation stage; tracking the results of preparatory information; formation of expert groups and assessment of the competence of expert analysts	conducting a test to determine the level of formation of an expert’s predictive skills; development of a fund of predictive and evaluation tools; non-formalized methods, clustering, Visual Mining (“Chernov’s faces”), the Delphi method (to assess the competence of an expert); brainstorming; Organizational EDM; maps of the future and SWOT analysis; informal methods	the application of the algorithm by different teachers can lead to different results; the existence of two or more interpretations (sometimes contradicting each other) of the same phenomenon; abrupt changes in the construction industry and in the training trends of future civil engineers (irregular mistakes)	development of a standard procedure for conducting pre-forecast orientation; expert assessment of the research object; exploring different approaches; study of new literature at all stages of forecasting	after the introductory campaign; after 1 session; upon completion of 1 year of study
4	the stage of systematization of the collected information about the educational process in a construction university; analysis of constraints when creating a forecast	building maps of the future by students, teachers and representatives of construction companies; non-formalized methods, clustering, Visual Mining (“Chernov’s faces”), the Delphi method (to assess the competence of an expert); brainstorm	the most complete list of characteristics, covering all manifestations without exception, practically cannot be realized	Highlighting important characteristics	after the introductory campaign
5	stage of development of the research program; study of possible risks	precise formulation of organizational and pedagogical conditions, appointment of those responsible for implementation and deadlines; Organizational EDM; maps of the future and SWOT analysis	teaching work is carried out in conditions of incomplete information, which are difficult to predict and take into account. Sources of uncertainty can be random factors, the mood of students and the teacher, sometimes these are external factors; pedagogical processes are probabilistic in nature, which causes difficulties in forecasting	standardization of procedures for formalizing pedagogical phenomena	after 1 session
Stage of analysis of diagnostic information
1	the stage of compiling a list of model indicators based on data from various sources	questioning of teachers, students, representatives of construction companies in order to identify the main external factors; study of literary sources; non-formalized methods; “The logical square of the competences of the future civil engineer”; “Competency profile”; descriptive analysis; classification; Visual Mining; fuzzy logic systems	forecasting OR requires an analysis of many factors, conditions and circumstances; what you are looking for is not clearly formulated.; direction of training 08. 03. 01 “Construction” includes a lot of specializations. Students from different groups study different special courses, which complicates the collection of statistical information. Differences in predicting the RR of master’s and bachelor’s students; full-time and part-time students	reducing the number of factors, creating object restrictions; creating different models for each training profile	2 course
2	stage of collecting forecast background data (economic; sociological; socio-cultural; political and international; legal; pedagogical; demographic; natural; scientific and technical; organizational)	maximum digitalization of information: statistics on student attendance at classes, intermediate control data in all disciplines, information on extracurricular activities (for example, volunteer); maintaining a portfolio of a student and an academic group; questionnaire survey of teachers, students, representatives of construction companies in order to identify the main external factors; study of literary sources about the forecast background for each region; postulation using literary sources; system analysis; Real - Time EDM; Visual DM; factor analysis; “Forecast background profile”; fuzzy logic systems	in the literature, indicators of the forecast background for a civil engineering university have not been sufficiently studied; underestimation by an expert of the degree of background influence; different background indicators affect different students differently; the indicators of the forecast background for students of different universities differ; indicators of the forecast background are difficult to formalize	study of economic, psychological and other non-educational literature.; strict adherence to forecasting technology procedures; studying the personal affairs of students, obtaining information from the curators of the groups, creating an individual forecasting background; cluster division of students into conditional groups experiencing the maximum influence of the same types of forecast background	2 course
3	The stage of taking into account factors that negatively affect the achievement of educational results	Educational conversations with students who have academic arrears	“Survivor ‘s Bug”	About asking expelled students about the reasons that may have influenced their failure to achieve educational results
4	the stage of generalizing the preliminary list of indicators to a type that will help to present the best type of pedagogical forecast	filling out the “form for assessing the key competencies of students in construction areas of training” (see Table B. 3, Appendix B); building a “Competency Calculator”; Real - Time EDM; Visual DM; OLAP; individual IP competency profile	educational outcomes for full-time and distance learning differ	correction of the forecast when students switch to distance learning	2 course
5	diagnostic information analysis stage	self-analysis of the formation of prognostic competencies; conducting a test to determine the level of formation of students’ prognostic skills; solving prognostic problems; Delphi method; dividing students into clusters (conditional groups); Real - Time EDM; Visual DM; OLAP; dynamic individual IP competency profile; clustering	it is impossible to prove the consistency of a formal system by means of this system itself (Kurt Gödel).; in a real-life problem, none of the functions is known for sure - approximate or expected values ​​are given.; in the course of forecasting OR, it is necessary to take into account both the uncertainty of the data, the system, and simulate the potential contributions to the predictive uncertainty; in the case of long-term forecasts, it is impossible to reliably state when the system will cease to function; Arrow’s paradox about the impossibility of an absolutely objective assessment by equivalent experts	Application of means of computer pedagogy; In order to get rid of the uncertainty, it is necessary to fix the functions, while losing the accuracy of the description of the problem.; carry out operational dynamic tracking of the state of the predicted object; m Selecting a person responsible for forecasting (“dictator”)	after 2 course
Stage of model building
1	stage of building a model of educational outcomes of future civil engineers and selection, suitable forecasting methods	filling in the “prognostic portfolio of students of construction areas of training” individual for each student pedagogical SWOT analysis; multiple regression; creation of a system of equations; SWOT analysis	statistical methods are well developed for one-dimensional random variables. in multivariate statistics, for lack of something better, poorly grounded heuristic methods are often used; if there is a simple dependence, then its form is not known in advance.	comparison of results obtained in different models or by several experts	before the winter session in the third year
2	Stage of completion of the basic model
Prospect stage
1	the stage of constructing time series of forecasting for each indicator of the base model (extrapolation)	an expert must adhere to the basic principles of pedagogical forecasting: adequacy, alternativeness, variability, verifiability, continuity, consistency, consistency, efficiency; checking the periods of foundation and lead time; construction of time series; Real - Time EDM; OLAP; Visual EDM	You can meet the “Oedipus effect” of forecasting: people tend to believe in predictions, and strive to fulfill the created forecast at any cost (the so-called “self-fulfilling forecast”)	We do not recommend without unnecessary need to inform students about low predicted educational results, so as not to underestimate their stress resistance, but positive predictions can be voiced, since this can increase the motivational component of competencies.	Before the start of the 4th course
2	the stage of determining the absolute optimum with conditional abstraction from the limitation of the forecast background				before the winter session in the fourth year
3	stage of pedagogical forecast verification and examination; analysis of sources of forecast errors; assessment of the consequences of applying the forecast and compliance with all forecasting principles	verification of the reliability, accuracy and forecast error, according to the real level of competence formation in a bachelor’s degree graduate; dichotomy (including multidimensional); expert assessments; statistical methods; Visual EDM	“The illusion of predictability: several experts come to the same wrong conclusion, because the mathematical model does not fully reveal the essence of the phenomenon under study, due to uncertainty, the emergence of the dependent variable;	make a “blind” forecast: experts should not be familiar with the prediction of colleagues	After the students graduate from the experimental groups of bachelor’s degree
4	forecasting model adjustment stage	re-evaluation of expert forecasting knowledge; clarification of the organizational and pedagogical conditions responsible for their implementation, etc. conducting an expert assessment of the forecast; neural networks; EDM
5	the stage of developing recommendations; synthesis of forecasts	creation of a regulatory framework in the field of forecasting on the basis of the university; informal methods			During the defense of this dissertation
Decision-making stage
1	the stage of making decisions based on the results of forecasting; introduction of technology into the practice of construction universities	application of the whole complex of the above conditions; application of the whole complex of the above methods and means			after defending the thesis

Table 2.

Organizational and pedagogical conditions for predicting the educational results of students in construction areas of training.

5. Results

The result of modeling in the format of 3D projections of multidimensional dependence of general level of prognostic competence are presented in Figure 2. The projections onto the 2D plane are shown in Figures 3–6.

Figure 2.

The dependence of the final value of the level of formation of predictive competence on the level of formation of motivational and activity components.

Figure 3.

The dependence of the final value of the level of development of prognostic competence on the level of formation of the cognitive component.

Figure 4.

The dependence of the final value of the level of formation of predictive competence on the level of formation of the activity component.

Figure 5.

The dependence of the final value of the level of formation of predictive competence on the level of formation of the motivational component.

Figure 6.

The dependence of the final value of the level of formation of prognostic competence on the level of formation of the communicative component.

We see the complex nature of the dependence at all intervals. Competence is an integrative quality of a person, therefore it is more correct to interpret it according to the totality of all known factors, and not to look for the dependence of only some indicators on others.

The dependence of the formation of competence on the cognitive component has a stepwise nature, i.e. on a number of intervals, it practically does not depend on external conditions and efforts of both students and teachers, but in the middle of the range, the level of competence is extremely dependent on cognitive efforts, quantitative characteristics dialectically turn into qualitative ones, so teachers should be as attentive as possible.

The dependence of the formation of competence on the activity component (Figure 4) is of the most complex nature, and advice can be given at each section of the level of formation. So, in the section from 30 to 50%, there is a maximum increase in qualitative changes with an increase in the number of solved prognostic problems, but then the slope angle decreases, although in any case the more we do prognostics, the greater the positive effect is observed. Every effort must be made to go from 90–100%.

Motivation for forecasting also has a “stepwise” effect: a low level of motivation is identical to its absence, it is impossible to distinguish between the level of motivation “above average” and “high”. There is a dichotomy: the student is motivated - he learns to forecast, not motivated - he is not taught.

As in the previous graphs, we see a stepwise level of dependence of futurological skills on sociability. The student is able to form the skill of forecasting only in communication: with the teacher and classmates.

The result of modeling of development of prognostic competence depend on the most factors is presented on Figures 7–11.

Figure 7.

The dependence of the formation of prognostic competence of pedagogical factors.

Figure 8.

The dependence of the formation of prognostic competence of psychological factors.

Figure 9.

The dependence of the formation of prognostic competence of socio-economic factors.

Figure 10.

The dependence of the formation of prognostic competence on the frequency of application of prognostic activity.

Figure 11.

Dependence of the formation of the prognostic competence of the technique used.

Pedagogical factors have the most important influence on the level of competence formation. Transitions from reproductive to productive, and from productive to creative are not possible without a mentor, although within these levels the teacher can have only minimal influence. Since there is a “plateau” on the graph.

Forecasting can be taught to almost any student, but for a qualitative transition to a new level of proficiency in futurology, you need to create a special psychological climate, so without a mentor it is difficult to independently learn predictions.

A low level of socio-economic security means a poor skill in the formation of predictive competence, and any minor improvements in the economic component significantly improve futurological skills. However, with an average level of social security, there is an insignificant plateau, at which an increase in the economic well-being has practically no effect.

As indicated in the previous modeling, in Figure 4, the activity has a significant improvement in predictive skill at all intervals. Thus, the two different models confirm each other.

Up to the achievement of a productive level of prognostic competence by a student, all teaching methods are equally effective, but in further training, the selection of the correct individual trajectory for teaching foresight skills is extremely important, because any minor changes in the teacher’s behavior model can lead to synergistic changes (“bifurcation point”) in learning outcomes. One should “sensitively” feel all the changes in the student’s achievements, because the dependence has a rather complex non-linear character.

6. Discussion

Most of the level of development of prognostic competence depends on the formation of its activity component. This is consistent with Selivanov’s statement [12] the main means of education is the activity of the one being educated. The nature of dependence on each component is non-linear. We conditionally accepted that the formal level of development of predictive competence is less than 60%, but also more than 20%. This is due to the fact that for all students, the ability to predict is formed to one degree or another, but it cannot be stated that a student has a sufficient level of competence for engineering activity with a forecast accuracy of less than 60%. But each of the components formed at a sufficiently high level with the underdevelopment of other components of the competence is not a reason to assume that the prognostic competence is developed at least at the reproductive level. The same can be said about the situation when one of the components is not developed absolutely. In this case, the general level of development of predictive competence is not zero.

At a number of intervals in Figures 7–10 various factors have virtually no effect on the formation of prognostic competence. We can influence only pedagogical factors, and the teacher cannot influence other factors directly, only indirectly. The greatest tangent of the angle of inclination of the dependence of the formation of prognostic competence on pedagogical factors is observed in the interval of the effectiveness of their use, which corresponds to the “above average” estimate.

The level of development of prognostic competence depends both on the frequency of application of prognostic activity and on the applied teaching methods, but nonlinearly. If you do not engage in prediction skills with students, prognostic competence will not be formed. With a very frequent prediction exercise, but without system lessons using pedagogical technologies, only a reproductive level of competence can be achieved. At a number of intervals, both the frequency of application of prognostic activities and the change in the effectiveness of the methodology have practically no effect on the formation of competence.

7. Conclusion

Prognostic competence is an integrative skill that is associated with many factors. Most of all, the level of development of prognostic competence depends on the formation of its activity component. Intellectual systems are a powerful tool for modeling the development of the prognostic competence of a civil engineer. The teacher can have the greatest influence on students who have achieved average achievements in the field of forecasting, but its influence on the most and least successful students is less effective.

We’ve discussed some of the problems have been reviewed at our last papers, for example [13, 14, 15, 16, 17] and others.