Perspective Chapter: Problems of Improving the Quality Management System of a University in the Era of Big Data (Experience of Professional Training of Railway Engineers)

Abstract

The application of the analytical approach to quality management in higher education bodies is particularly relevant. The problems of formation, implementation, and practical use of quality management system in higher education have not yet been sufficiently reflected in the studies of foreign and domestic specialists, although there are a large number of publications on the formation and implementation of quality management systems in industrial production conditions. The publication intends to consider the main triad of education quality components: conditions, process, and result of educational activity. The quality of education is presented as a system functionally related to all parameters and a measurable characteristic of the functioning of an educational organization. The quality of such functioning is represented as a degree of implementation of the main goal, which consists of achieving a given (normative) level of students’ preparedness and providing the basis for sustainable development. As an illustration to the theoretical provisions, the experience of implementation and certification of quality management system in the leading technical university of Russia is considered, recommendations on the development of strategic and operational management documents in the implementation and practical use of quality management system in the university are given.

1. Introduction

Global processes taking place in the society are inevitably reflected in the state of education. Economic, social, and environmental challenges in modern society, the formation of innovative orientation of the economy, integration into the world set new priorities and tasks for the system of higher education, among which the most urgent is the task of ensuring the quality of training specialists. At present, education is in the first place among the factors of human development. The importance of knowledge in the economic development of the countries of the world is increasing rapidly, ahead of the importance of the means of production and natural resources. When calculating the rating of countries according to the Human Capital Index, an important factor is investment in people through quality health care, education, skills, and jobs. The health and education components included in the Index are used in combination, which, judging by the data of empirical studies conducted at the microeconomic level, reflects their contribution to the level of productivity and accumulation of public goods. The quality of education plays a key role. The report states that “the governments of many states allocate a significant share of their budgets to education and health, but often public services fail to form human capital because of their poor quality, as the bureaucratic apparatus proves unable or not motivated to convert sound policies into effective programs” [1].

Thus, to a large extent, the economic success of the state is determined by its educational system and the education of its citizens. This circumstance has led to an awareness in developed countries of the role of education in society, the need for its priority development, and the development of new methods and tools for quality management.

The publication deals with quality management in higher education in the era of Big Data. Big Data is not just a large amount of information. We are talking about unstructured data. It is a huge and chaotic flow of information from different sources, which raises the problem of processing and ordering information. Thanks to data analysis, it is possible to predict the behavior of large groups of people. Big data allows not only to know in advance what representatives of a particular audience will choose, but also to predict how this choice will change over time. Big Data is, on the one hand, a set of technologies, tools, methods, and approaches designed to solve the problem of processing large volumes of data, and on the other hand, it is a volume of data that cannot be processed conventionally, that is, by traditional methods. The strongest factor in expanding the range of applications of Big Data is the Internet.

The usual field of application of Big Data is marketing. By analyzing the data, companies study what principles guide the consumer’s choice of product or service. As a result, marketers model the behavior of the potential consumer and launch an appropriate advertising campaign. For education, an important feature of Big Data is the ability to analyze different parameters and modeling.

“Big data has been a popular research problem across different academic disciplines. Although this problem has been treated mainly for advancing and innovating technological development [2], organizations and business communities are continuously exploring different aspects, perspectives and contextual specifics to find or explore benefits and value adding for improving practices” [3]. According to statistics, the introduction and use of analytics and work with big data are not just effective, its use can decide the outcome of competition in the market. In September 2014, Accenture published the results of a major study called “Big Success from Big Data” [4]. Correspondents surveyed 1,000 company directors from seven different industries. Ninety-two percent of respondents expressed satisfaction with the end results of Big Data implementation and its impact on their business, 89% called activity analytics a very important component in setting up business processes. A GE (General Electric) study titled “Industrial Internet Insights Report” [5] was conducted by the company in 2015. Ninety percent of respondents from various industries assured: Big Data is in the top 3 leading areas for their businesses. Eighty-four percent of respondents believe the use of analytics has the potential to displace competing businesses from the marketplace within 1–3 years. According to Proficient Market Insights, “the global Industrial Internet of Things (IIoT) market size is projected to reach US$ 78400 million by 2028, from US$ 57040 million in 2021, at a CAGR of 4.2% during 2022-2028” [6].

Sources of Big Data in education are, first of all, message streams from social communities, statistics sites. Application of Big Data in higher education is possible primarily in the analysis of documents and modeling of educational processes. Big Data technologies imply working with huge arrays of information. There is no universal method of Big Data processing, but there is a possibility of using various methods, so it is important to use specific tools for specific strategic decisions.

According to the provisions of ISO 9000 standards, the quality management system (QMS) must be thoroughly documented. Documentation makes the system “visible” not only to its developers, but also to users and reviewers. It is only possible to prove that the QMS complies with the established requirements when the system is presented in a documented form. Otherwise, it can be argued that there is no quality system. ISO 9001 establishes the criteria for a quality management system and is the only standard in its series that can be certified to (although it is not a mandatory requirement). It can be used by any organization, regardless of its size and field of activity. Accordingly, this research is based on the analysis of documentation and the method of modeling. In the quality management system the main features of the engineering component of the methodology of organizational design are well traced, in particular, its concepts of the system environment, management, and organizational changes. At industrial enterprises, the introduction and implementation of the process approach are seen as a way to improve the effectiveness of the quality management system.

The problem to be solved in improving the quality management system in the educational organization is the partial elaboration of documents. Part of the QMS documents is stipulated in the standard, another part is implied. Therefore, the structure of the quality management system has a “constant” component defined by the standard and a “variable” component, which depends on the specific organization.

2. QMS in the system of higher education in Russia

In modern education system, quality is understood as compliance with the standards. The requirements and recommendations of international ISO 9000 Series of Quality Standards are widely used as a basis for the formation and implementation of quality management systems (QMS). The international standards of ISO 9000 series include requirements and recommendations that provide methods for monitoring, measuring, analyzing, and improving all QMS processes. The implementation and subsequent continuous improvement of the QMS require not only understanding and vision of its development prospects, but also the use of objective methods of measurement (including statistical) to assess the effectiveness and efficiency of both the QMS of the university as a whole and its individual processes. The potential stakeholders in the ISO standards are consumers, society, suppliers, owners (shareholders), and personnel of the organization, which corresponds to the quality management principles formulated in the ISO 9000 Standards.

Problems of formation, implementation, certification, and practical use of QMS, its subsystems and mechanisms in universities, have not yet found a sufficiently complete reflection in the studies of foreign and domestic experts. Most publications are devoted to the problems of quality in the QMS of industrial production organizations.

Ph. Crosby, one of the well-known experts in the field of quality, studying the issues of value assessment of quality, expressed the famous aphorism: “Quality is still free” [7]. It follows that the manufacturer does not have to pay for quality, but for its presence, which must be the subject of constant monitoring and analysis. E. Deming wrote: “The consumer is the most important link in the production line. Quality must be aimed at satisfying his needs, present and future” [8]. Eventually transformed into the process of satisfying the needs of existing and potential customers, at the current stage of business development quality is naturally formalized in the principles of Total Quality Management – TQM and acts in its new understanding as a measure of balancing the expectations of all stakeholders. There is a difficulty in direct application of TQM principles in the formation of QMS of higher education institution.

Currently, educational organizations use the following basic principles of the well-known system of total quality management (TQM) [9]:

• active quality management by the administration of the educational organization;

• organization of the activities based on the requests of employers and reactions of society;

• selection of strategies and policies in the field of quality, aimed at continuous improvement and achievement of results, providing stable operation of the educational organization;

• creation of quality systems, taking into account the recommendations of international ISO 9000 Series of Quality Standards, supplemented by the experience of the best educational organizations;

• continuous professional development of the staff and its involvement in the work to improve the quality of educational services provided;

• provision of necessary resources with minimal stocks and rational use of them;

• information and analytical support of the work in the field of quality;

• effective management of all processes taking place in the university;

• implementing of certification of educational services and quality systems, tracking and compliance with current legislation in the field of quality.

The participation of university units in the implementation of macro processes of quality management system is as follows (Figure 1):

Figure 1.

Macro processes of quality management system at the university.

The university receives information, capital, human resources, and materials from the environment. These components are called inputs. In the transformation process, the educational institution processes these inputs, converting them into products or services, which are outputs of the organization that it brings out into the environment.

In Russian higher education, the participation of management in the development of quality management system should be emphasized. The rector of the university decides on the expediency of the preparation, implementation, and subsequent certification of the quality management system. The rector appoints a quality management representative, who will be the project manager of the quality management system preparation for certification. The quality management representative, top management, and consultants jointly determine the list of subdivisions involved in the work, and by the Rector’s order, form the Quality Council, which includes (ex officio) the heads of administration all departments involved in the preparation of the quality management system departments involved in preparing the quality management system for certification. The Quality Council is chaired by the Rector.

3. The problem of assessing the quality of education

One of the main tasks of higher education is to ensure the quality of specialists. The problem of the quality of education has always existed. Now it has become extremely acute not only in Russia, but all over the world. At the same time, there are no clear criteria for the concept of “quality of education.”

The education system in the Russian Federation is considered as a service that meets the educational needs of the population. In accordance with the Russian Federation Law “On Education” (Article 2), “education is a single purposeful process of education and training, which is a socially important benefit and is implemented in the interests of the individual, family, society and the state, as well as the totality of the acquired knowledge, abilities, skills, values, activity experience and competence of a certain volume and complexity for the purposes of the intellectual, spiritual, moral, creative, physical and/or professional development of an individual, meeting his educational needs, and, as a result, the quality of education” [10]. In Russian higher education, the quality of specialists training is connected mainly with the implementation of State Educational Standards of higher professional education, state regulation of higher education institutions. An educational standard is a set of mandatory requirements for higher education.

Educational programs of a particular cycle of subjects determine the content of education of the corresponding level and orientation and are divided into two major groups by their intended purpose: general education and professional. Basic educational programs are designed to ensure the achievement of such a level, which corresponds to the state educational standard. State Educational Standards of higher professional education determine the educational minimum content of the main educational programs, the maximum educational load of students, the requirements for the level of training of graduates. These standards are established by the state authorities (management) and act as a basis for objective assessment of the level of education and qualification of graduates, regardless of the form of education. Orientation of some universities only on State Educational Standards of higher professional education as a guarantee of quality assurance has two drawbacks: the false idea that quality can be achieved by inspection; neglecting the needs of the educational market (standards do not keep pace with changes in employers’ requirements).

In the State Educational Standards, the model of training specialists is personally oriented. The quality of the result of training programs is determined by two blocks of parameters that characterize the knowledge accumulated in a particular academic area and acquired competencies, including the personal development of students (critical thinking, general development).

The education system objectively operates on the following markets: the segment of the market of educational services, providing satisfaction of the needs of citizens in education and training; the segment of the labor market, providing satisfaction of the needs of employers and professionals; the segment of the market of intellectual goods, providing satisfaction of consumers (customers) in new knowledge, technologies, and knowledge-intensive products. The results of the activity of higher education institutions are manifested in several types, namely: services of educational character; scientific and technical products; integrated products based on scientific and technical products and educational services; educational and methodological developments.

3.1 Approaches to assessing the quality of educational services

There are two approaches to assessing the quality of educational services. Assessment of consumer properties of services (quality as value) is determined by active involvement of all stakeholders (students, faculty, corporate partners, etc.) in quality assurance processes at all levels of its management (the USA, Germany, Taiwan, Philippines). At the same time, quality control and assessment are either the priority of state structures or government-funded structures (Germany, the United Kingdom), or are based on internal process of control and self-regulation and external, expert quality assessment (the USA, Taiwan, the Philippines) [11].

The second approach is characterized by a wide range of powers of the state in ensuring the quality of educational services. The composition and quality of educational programs are determined and regulated by the state. The control and evaluation of education quality are centralized (France, Russia, Kazakhstan). In accordance with this, the quality of education in the context of the quality of the result of the educational process is considered as “compliance of the level of knowledge of students and graduates to the standards,” on the one hand, and the needs of the market, on the other hand. The parity of state and market interests in the field of quality is provided by the state standards through the system of elective courses, the composition and content of which are determined by the university independently.

The system of higher education in Russia has a more developed external quality assessment, focused on standards and performance indicators. The main elements of this system are standardization and procedures for licensing, attestation, and accreditation, as well as a comprehensive assessment of educational institutions as a whole and individual specialties based on the rating system. All of these procedures involve an internal review. One of the undesirable consequences of external control in education is the tendency to block information that lowers the grade. This leads to the loss of credibility of the entire system of management information.

The starting point of quality management system formation is to build a “model” of a university graduate as a set of certain personal professional qualities, the development of which should be aimed at the educational process: its content, teaching methods, forms of organization, methods of monitoring and evaluation of students’ knowledge. The quality management system in higher education institution as a whole consists of the quality management of each type of university activities (educational, scientific, educational-methodological, educational, administrative-economic, etc.).

The analysis of different models of quality management systems has shown that the technological solutions of quality management in an educational organization should be focused on:

• elaboration of the mission, policy, and development strategy of the educational organization;

• creation of a new dynamic organizational management structure, including the definition of the components of this structure and their position in relation to each other, the establishment of interconnection of components, and ensuring the implementation of “development” strategy and interaction;

• transition from the subordinate principle of management organization to the dominance of horizontal coordination type organization, uniting equal and independent components on the basis of self-organization and self-development;

• the optimal combination of functional and linear quality management structures with a situational approach to management;

• ensuring the integrity of management functions within the management cycle;

• targeting of control actions, definition of objects and subjects of management, clear delimitation of powers, rights, duties, and responsibilities of the subjects of management;

• strengthening elements of monitoring, analysis, and evaluation of the results of the educational process, the construction of sound criteria and evaluation indicators;

• combination of different types of material and social motivation, as well as volitional management, organizational impact;

• economic, educational, organizational, and administrative, psychological and pedagogical methods of management.

3.2 Development of quality systems in higher education organizations

There are two main ways to create quality systems in higher education institutions. The first is the development of a unique model of quality management system on the example of a particular institution, partly universal and applicable to other organizations. The second way is the use of universal principles of modern quality management systems used in different spheres of human activity. The third way is the integrated process of building a unique quality management system in education using TQM principles and the requirements of international ISO standards.

The top management should bring to the attention of the staff the following provision: in order to manage the quality of the process, it is necessary to be able to measure its effectiveness and efficiency. In addition, the management of the university should ensure that the management representative prepares a report on the functioning of the QMS and the need to improve it. This report should be used as input for management’s analysis. It usually includes the following information [12]: the status of actions based on previous management reviews; changes in external and internal factors; information on the QMS performance and results, including various trends (customer satisfaction; feedback from stakeholders; extent to which quality objectives have been achieved; nonconformance and corrective actions; monitoring and measurement results; audit results; external supplier results); adequacy of resources; effectiveness of actions taken in relation to risks and opportunities; opportunities for improvement.

Methods should be developed to measure (evaluate) the performance of each process. Senior management should ensure that the results of analysis are used to assess:

• the conformity of products and services (results of processes of educational services and other categories of university products);

• the degree of customer satisfaction; c) performance and effectiveness of the QMS; d) the success of planning;

• the effectiveness of actions taken in relation to risks and opportunities;

• results of external suppliers (including the expected number of high school graduates and secondary vocational school graduates);

• needs for improvement of the QMS.

The quality of the results of the activity of higher educational institutions (HEIs) should be provided through quality management of the main working processes of HEIs. It is reasonable to distinguish three groups of processes: basic processes, management processes, and supporting processes. Taking into account these requirements, it is possible to recommend to distinguish the formation of educational program of professional education and organization of educational process among the main processes of educational activity of university. Each of the abovementioned processes should include subprocesses, which should take into account the general requirements for the formation and organization of educational process, as well as the specifics of the university. In turn, these processes can also be decomposed.

Quality management processes at universities (HEIs) should include, first of all, those processes that are regulated by mandatory procedures in accordance with ISO 9001:2015 “Quality management system. Requirements” [12]. These include: QMS documentation management, quality data management (records), internal audit, management of nonconforming products, corrective and preventive actions. In addition, it is also necessary to provide for a number of other processes, such as development and updating of quality policy and objectives, quality planning, quality analysis by management, etc. (Figure 2).

Figure 2.

Quality management processes at universities that should be under analysis.

It is claimed in some research that all of the dimensions of e-services quality have had an impact on student satisfaction for ease of use [3].

To characterize the quality of education, certain indicators are introduced, which can be divided into three enlarged groups: indicators of investment in education, indicators of the quality of processes, and indicators of the quality of results. The indicators of the first group include information on financial, material, and technical, personnel, information and methodological support, etc. The indicators of the second group—process quality indicators—are identified on the basis of information

In order to be able to use the information obtained in the process of improving the quality of education, it is necessary to combine all the indicators that characterize the educational system “at the input,” “in the process,” and “at the output,” to establish the nature of the observed relationships and then, based on the analysis done, to outline a strategy to improve the quality of the output results.

There are indicators of the third group, they include the results of the assessment of the quality of training and value orientation of students, as well as information about their career development after the completion of a certain educational stage.

To assess the effectiveness of the educational institution, the indicators affecting the learning outcomes can be divided into:

• indicators of the capacity of the organization or the area of activity under study; static indicators of performance and effectiveness achieved to date.

Processing of the results of education quality assessment can be carried out using special statistical methods of regression and correlation analysis, specific techniques recommended by ISO 9000 Series of Quality Standards. The method of multilevel models is also used to assess the effectiveness and efficiency of the organization or a separate direction.

After determining the indicators that characterize the quality of the educational process, it is necessary to design the process of quality analysis (monitoring, information processing, and evaluation). The process of self-analysis and self-assessment is extremely useful, which is not so simple and requires considerable expense. It is extremely important to conduct self-analysis of an educational organization and be able to compare its results with the opinion of independent external experts.

The organization must identify the processes required for the quality management system and establish the sequence of these processes and their interaction.

With the introduction of big data analysis in the quality management system of universities, it is effective to consider the concepts of “risks” and “opportunities” as two independent and independent indicators. In this case, it is characteristic to tie “risks” to unfavorable events, and “opportunities” to favorable ones. The indicator “opportunities” should be considered separately from the indicator “risks.”

The system of Hazard Analysis and Critical Control Points (HACCP) provides for the application of risk assessments using a combination of two indicators, the first of which is called “probabilities of hazard realization” and the second – “severity of consequences from the realization of the hazard” [13].

By analogy with HACCP, it is proposed to use the following approach: when assessing the indicator “possibility” (P) using a combination of two indicators:

1. Experts make a point estimate of the first indicator – “probability of realization of potential improvement” (PI) based on four possible assessment options in the form of points: 1 – practically equal to zero; 2 – low (probability of realization not more than 40%); 3 – medium (probability of realization 40...85%); 4 – high (probability of realization 85...100%);

2. Experts carry out point estimation of the second indicator – “significance of positive consequences of the supposed improvement” (SI). Here also four variants of this consequence estimation are possible (in the form of points given below): 1 – very small (the effect of improvement is insignificant); 2 – small (costs are recouped in 3 – 7 years); 3 – significant (costs are recouped in 1 – 3 years); 4 – large (costs are recouped faster than 1 year);

3. The boundary of permissible values of indicator “possibilities” on the qualitative diagram coordinates in the form of “probability of realization of potential improvement” (PI) and “significance of positive consequences of supposed improvement” (SI) as it is indicated in Figure 3;

4. For each considered case of using the indicator “possibility” (improvement), a point is plotted on the diagram with the coordinates of PI and SI, the values of which were evaluated as indicated above. If the point lies on the border or above – the assessed option of improvement of activity in QMS is characterized by a high value of the “possibility” and should be considered by the management of the division or organization for the use of the available opportunity. If the point lies below the border, then the value of the indicator of “possibility” is small in magnitude, and the considered option of improvement of activity is unpromising for planning and implementation of the project aimed at use of the available situation. Thus, the indicator “possibility” (improvement) is represented by a point (vector) on the plane with coordinates PI and SI. Possibility = F(PI, SI), where PI – point estimate of “probability of potential improvement”; SI – point estimate of “significance of positive consequences of the expected improvement”; F – designation of the function, which puts a point on Figure 3 in accordance with the coordinates of PI and SI.

Figure 3.

Sphere of opportunity distribution.

In the figure, the boundary of the desired values of the indicator “possibility” is plotted with the coordinates of PI and SI. Going to or beyond this boundary means that the situation is promising for the project to use the available opportunity of improvement. There are cases if a point lies in the area of low values of the indicator “possibility” or if a point lies in the area of high (desirable) values of this indicator. If a point lies on the border, it is also corresponding to high (desirable) values of the “possibility” indicator. The two-dimensional evaluation of the “possibility” indicator seems to be the most convenient and universal. Interpretation of this indicator (“possibility”) as a set of two indicators of PI and SI allows to enter the following definition: indicator of “possibility” of improvement is “probability of realization of potential improvement” (PI) taking into account “significance of positive consequences of supposed improvement (SI).”

4. Conclusion

The use of big data seems obvious and inevitable: “the industry 4.0 revolution was entirely based on the new technologies, mainly in computer and information technologies. With the introduction and the use of Internet of things, cloud services, big data, artificial intelligence, various algorithms result in integrated systems providing excellent customer service for the organization activities” [14]. However, the analysis and application of processing results require resources and should be built into the quality management system of the university.

In all educational systems, foreign and national, the object of education quality assessment is the quality of educational services, which is traditionally evaluated by the results, which are understood as learning outcomes (experience accumulated in the learning process). The content of the concept “quality” is largely determined by the system of quality assurance and control adopted in the sphere of education in this or that state and, primarily, by the government’s powers in solving these problems.

The HEI quality management system can be built in accordance with the requirements and recommendations of international ISO 9000 Series of Quality Standards, the principles of Total Quality Management (TQM), or based on the model of the European Foundation for Quality Management (EFQM).

Universities need to find their own methods, techniques, procedures of self-assessment, and self-analysis of activities, to create associations of single-profile educational organizations and enterprises—consumers of specialists for an adequate and effective quality audit. In modern conditions practically every HEI is interested in creating a system that would constantly monitor, evaluate, forecast, plan, and manage all the processes in the HEI. The need for this system is justified, and its reasonable application will allow the institution to successfully compete in the labor market and educational services.

One of the main difficulties that exist today for the implementation of big data in university management practices is the difficulty of translating data sets into simple indicators. The expensiveness of the use of big data is obvious, while the efficiency is recognized insofar as we are living in the era of another technological revolution. In addition, the processing of big data requires software and specialists. In practice, universities engage professional marketers and market specialists to solve their problems. It is necessary to provide a mechanism for translating big data into a few simple indicators that reflect management efficiency.

The use of big data in higher education is necessary. First of all, it is the analysis of attendance at the university website and university profiles in social networks. This analysis involves serious analytics and statistical processing that will never be of deep interest to middle and upper management because of its complexity. This is another reason for converting the data into simple metrics. Big data analysis should be implemented, first of all, in monitoring the results of educational services.

To sum up, the mission, vision, goals, development strategy, and quality policy of the HEI together form an important strategic management tool, which expresses the spirit of the organization, states what the organization stands for, what its purpose is, what its goals are, where it is going, how it is going to achieve all this, and what important points everyone has to focus on. They shape the collective ambition of the organization, have an important influence on the connection of employees and employees to the organization and to the quality of their work. A successfully articulated collective ambition reveals to people how their activities contribute to the common cause, how they work together to achieve goals that contribute to a higher quality of performance in the organization.

After developing strategic plans, determining critical success factors and performance metrics, defining policies, and setting quality goals for the organization, there appear: the strategic plan of the organization as a whole; operational plans for the departments (services) of the organization; specific quality improvement projects. After approval of these strategic and operational plans, they proceed to implement them. The specific actions are usually defined in operational plans for the various business functions, usually developed over a period of time for various business functions, usually developed for a period of up to one year. The central issues of such operational plans are:

• what market research needs to be done?

• what new specialties need to be opened?

• what academic disciplines will be introduced in the new academic year?

• what new educational laboratories should be created?

• what needs to be financed?

• what human resources will be needed?

• what will have to be purchased?

• what research and development

• what repairs and (or) maintenance will have to be performed?

• what services (third-party organizations, partners) will be required?

Only after the answering the above questions it will be possible to begin to achieve the goals set by implementing the strategic and operational plans of the organization. These questions are useful in that they capture those areas of activity where the use of big data is appropriate.

Conflict of interest

All authors declare that they have no conflicts of interest.

Appendices and nomenclature

EFQM	European Foundation for Quality Management
HEIs	higher educational institutions
IIoT	Industrial Internet of Things
PI	probability of realization of potential improvement
QMS	quality management systems
SI	significance of positive consequences of the supposed improvement
TQM	Total Quality Management