The analysis of the current situation in the wireless communication market shows an increase in the workload, which leads to an increase in the need in additional resources. However, the uneven loading of the infrastructure nodes leads to their loss of use; so, there is a need in introducing technologies that both do not lead to downtime of equipment and ensure the quality of load service during the day. An overview of the NFV virtualization technology has shown that it is appropriate to build wireless networks, since it provides the necessary flexibility and scalability. The method for determining the location and capacity of reserved computer resources of virtual network functions in the data centers of the mobile communication operator, method for determining the size of computing resources constant configuration time interval, and distributed method of local reconfiguration of the virtual network computing resources in the case of a failure or overload are proposed. Thus, configuration, operation, and reconfiguration processes in mobile core network with virtualized functions are described.
- network function virtualization
- evolved packet core
- resource allocation
- mobile network
In the mobile cellular network, the rapid development has been observed. Modern telecommunication systems are being constructed as complex networks that involve various types of devices united into a single complex, operating in conditions of large load flows and large number of connections . They can offer higher data transfer rates, with the integration of more services and guarantee of high quality of experience. Nevertheless, this development also means that the amount of data that is transferred in the mobile network is increasing and the volume of signaling traffic is increasing, respectively. According to , it is expected that total mobile data traffic will have increased to 77 exabytes per month by 2022, almost seven times more compared to 2017. Mobile data traffic will grow at an average annual growth rate (CAGR) equal to 46% from 2017 to 2022.
According to Shimojo et al. , vehicles, houses, personal devices, robots, sensors, etc. will be connected wirelessly. It means that an automatic and intelligent control system will be achieved. An increase in the number of devices will affect the IoT market, which is estimated to be $19 trillion , and is expected to reach 50 billion . In addition, rich content services, such as real-time streaming movies that require high resolution and tele-surgery requiring small delay must be provided (Figure 1).
In addition, the average signaling requirement per subscriber is up to 42% higher in LTE compared to the standard of the past generation communication .
Furthermore, market competition requires faster deployment of services and elasticity of changing service criteria as well as the ability to cope with higher service requirements. Therefore, there is a need to manage the signaling traffic in order to provide the necessary quality of service to end users and the proper use of resources of the network operator.
In such circumstances, operators are forced to build up the network infrastructure to ensure the process of service of telecommunication services at a given level of quality. During the day, the load differs, and according to , up to 80% of the computing capacity of the base stations and up to half of the capacity of the core network are unused. This leads to a low usage of resources as well as a high level of energy consumption, which reduce the cost-effectiveness of the network for mobile operators.
The emergence of the concept of network functions virtualization opens up new opportunities for the world of telecommunication systems. At the same time, there is a need for new approaches, models, and methods for organizing service handling. The use of virtual servers to solve the tasks of the mobile core network can greatly simplify the process of organizing resources on the service server and ensure its scalability and fault tolerance.
The principle of network function virtualization (NFV)  is aimed at transforming network architectures by deploying network functions into software that can run on a standard hardware platform. According to the ETSI , the network function is a functional block within a network infrastructure that has defined external interfaces and a defined functional behavior. Network functions are components of the LTE evolved packet core (EPC) network, such as MME, HSS, PGW, and SGW, which for the NFV case will be deployed on the basis of data center system, with the use of leased computing resources (CPU core, memory, disk space, and network interface card), which can be allocated and reallocated in the process of operation depending on actual load requirements.
Thus, the features of NFV can be characterized as follows :
Separation of software from hardware. Since the network element is no longer an aggregate of integrated hardware and software entities, the evolution of both is independent of each other. This allows having separate terms of development and maintenance of software and hardware.
Flexible deployment of network functions. The separation of software from hardware helps to reallocate and share infrastructure resources; thus, together hardware and software can perform various functions at different times. It helps network operators to deploy new network services faster on the same physical platform. Consequently, the components can be created in any NFV-compliant device on the network and their connections can be installed on a flexible basis.
Dynamic scaling. Dividing the functionality of the network function into created software components provides greater flexibility in scaling the actual performance of the virtual network function (VNF) more dynamically and with greater details, for example, according to the actual traffic for which the network operator should provide capacity.
At present, numbers of problems remain unresolved. You need to consider hybridity of the service environment, where flexible, well-scalable, virtual servicing entities located in rented cloud-based databases operate along with specialized hardware with limited features. Therefore, the task to organize the computing resources of service nodes and flows between them in a hybrid environment, which consists of hardware telecommunications and virtual computing entities, is important.
Unlike the existing static architecture of the LTE EPC network, a system (Figure 2) in which service flows are processed by hardware, and in the case of expected overload, the redistribution of flows happens and takes into account the expansion of the service network by adding virtual service facilities located in the leased clouds of the data centers is proposed (Figure 3). After organizing a hybrid service environment, there is a need to adapt the computing resources of the system in the process of operation to ensure a high-quality service, and also it is necessary to consider the features of the reconfiguration process and the costs associated with it. So far, there has not been any comprehensive solution to the task of controlling the computing resources of the hybrid telecommunication environment. The peculiarities of the load distribution of resources of network elements, hardware or virtual ones, have it been considered yet either.
Thus, the chapter proposes a structured approach to the management of resources of network functions through sequential control of the following stages: monitoring, forecasting, controlling the sufficiency of resources, and controlling the given level of quality of telecommunication services.
Having analyzed the research and development processes of telecommunication networks of the next generation, we may argue that the existence of powerful data centers greatly expands the possibilities of organizing the process of providing services. One of the key aspects of network virtualization is the allocation of physical resources to virtual network functions. This involves mapping virtual networks on physical networks, as well as managing dedicated resources throughout the life cycle of a virtual network. The optimality and flexibility of resource allocation are key factors for successful network virtualization.
Most of the existing methods for solving the tasks of organizing hardware and virtual resources offer a static distribution of resources, in which, when computing and telecommunication environment is organized, the reallocation of resources does not occur throughout its life cycle. As the network traffic is not static, this may result in improper use of shared computing resources. It is important to organize monitoring of virtual nodes and provide the resources on the basis of their real needs.
2. Method for determining the location and required capacity of virtual reserved computing resources in case of an overload of the physical network
The method is based on the shared embedding concept  of the individual virtualized services of the core network on the physical network. We suppose that the virtual network functions of the mobile core network have the same functionality and interfaces as the network components of the 3GPP LTE EPC architecture.
The number of service chains must be determined in advance. The extreme case would be consideration of one service chain for the mobile phone/eNodeB. Since realistic scenarios for mobile networks are up to 10,000 eNodeBs, the resulting optimization model will be enormous and quite long computation time is required to solve it. Therefore, we accept reasonably large clusters of eNodeBs and assume that each of these eNodeB clusters refers to a single service chain of the core network.
Consider the situation when the provider of telecommunication services already has an existing topology of base stations. You need to determine a subset of the network nodes where the load aggregation blocks will be placed which will generate the requests to the same virtualized EPC service. After that, for each base station site, we assign a node of aggregation (traffic aggregation point – TAP).
Objective function (1) aims to minimize network latency. Objective function (2) represents the total cost of placing aggregation nodes and the cost of establishing channels between base stations and the respective TAPs. The objective function (3) aims to leave more free bandwidth on each physical channel. The residual bandwidth of all channels is maximized, since high-downloaded channels can lead to network overload; so, it is advisable to get a solution where more free channels are left.
These optimization goals can be useful for network operators to plan the best deployment strategy.
It is possible to use the linear combination (4) of Eqs. (1)–(3) with weights
Restriction (7) ensures that the maximum TAP does not exceed budget
Below we describe the method for solving the problem of placement and the capacity of reserved computing resources of virtual network functions.
Physical network is given in the form of graph
A virtual mobile core network is represented by a set of services (one service per TAP) which are embedded in the physical network.
The requirements to the bandwidth between two functions,
The goal of optimization is to find the location of the virtualized services of the core network (i.e., the allocation of network functions and the allocation of resources, as well as definition of the ways to transfer traffic between them), so as to minimize the cost of the occupied resources of channels and nodes in the physical network, while satisfying the load requirements
The following Eqs. (11)–(20) represent the formulation of the optimization problem of mixed integer nonlinear programming. The Boolean variables
Similarly, Boolean variables indicate whether the physical channel
Eq. (12) ensures that for each TAP/service, only one network function of each type is placed. Eq. (13) ensures that the allocation of resources is carried out on physical nodes, which have an administrative opportunity to locate the corresponding network functions. Eqs. (14)–(16) represent restriction for the available resources of physical nodes and channels. Eq. (17) represents a restriction for flow conservation of all paths in the physical network. Eq. (18) ensures that the variables in the task of locating network functions and displaying a path are Boolean.
In order to limit the delays on channels, the delay limit shown in Eq. (19) is also added. And to take into account the necessary performance of the virtual network function, the restrictions for the value of the processing time of the request determined in Eq. (20) are necessary.
It is supposed to solve the problem (11)–(20) in the offline mode at the initial stage. According to the solution, each network function reserves a certain number of resources of the virtual network function based on the assessment of its greatest resource requirements. The instantaneous needs of different network functions are dynamically satisfied by activating the necessary configuration of virtual machines during execution in such a way as to satisfy the guarantees provided for each network function.
3. Method for determining the size of the time interval of the constant configuration of computing resources
The decision when to provide resources depends on the dynamics of traffic loads. Telecommunication loads undergo long-term changes, such as hourly effects or seasonal effects, as well as short-term fluctuations such as unexpected crowds. While long-term fluctuations can be predicted in advance, observing changes in the past, short-term fluctuations are less predictable, and in some cases, unpredictable. The proposed method uses two different approaches for working in conditions of changes that are observed at different time scales. Proactive resource management is used to assess the load and corresponding management, as well as reactive resource management is used to correct long-term errors or to respond to unforeseen overload.
We propose to apply a mechanism which implies dynamic change in the duration of the constant configuration of the resources of the virtual network function, depending on the difference between the maximum load value at a certain base interval and the minimum one. Eq. (21) describes the principle:
To do this, you need to define the base interval. The goal is to present a daily load pattern, sampling its requests into successive, non-overlapping time intervals with a single representative value in each interval. Load discretization: having a time series
The solution for time series discretization (Eq. 22) is given as follows:
Eq. (22) is an objective function which has to be minimized, where
At the same time, it is proposed to continuously monitor the values of the request arrival rate and use the predicted values if the load does not exceed the threshold; otherwise, current trends are evaluated and resources are scaled on the basis of the new forecast.
Load forecasting for the next time interval is carried out by taking into account long-term statistics and adjusting it according to the model of exponential smoothing, where errors of more recent past periods have a greater importance factor:
4. Method of local reconfiguration of network computing resources in case of failure or overload
There might be situations when the resources available on the nodes will be insufficient or if the node fails. Potential failures can be physical nodes failures, failures of servers that have higher failure rates than telecommunication hardware or the infrastructure provider will perform node maintenance tasks and this will require the migration of nodes.
For this case, the methods of reconfiguration are used which seek to find the places for migration of network functions from the affected nodes, minimizing the cost of recovering the node after failure and maintaining a high level of physical performance of the network. The proposed improved recovery methods differ from existing ones by taking into account the cost of resources on the nodes and the final quality of service, as well as the case of node overload. In addition, in previous research, the problem of locating management nodes, which are coordinators of the movement of virtual network functions, remained unsolved.
The main purpose of the optimal placement of managers is to minimize delays between nodes and managers in the network. However, considering only delays is not enough. The placement of managers should also take into account certain restrictions of stability. Figure 4 shows different issues that need to be considered when evaluating the stability of the placement. Below we will briefly explain these issues and what is needed to be sustainable in relation to them. Figure 4 shows normalized delays between nodes and arrival rate at nodes.
Let us presume that the nodes are assigned to their closest manager, using as the metric of delay, i.e., the shortest path
It is obvious that one manager is not enough to achieve network resilience. On the other hand, when multiple managers are hosted in the network, the logic of network management is distributed across multiple managers, and these managers must be synchronized to maintain a consistent global state. Depending on the frequency of synchronization between managers, the delay between individual managers plays an important role.
Based on the
UAlatency(p) = max(ddcn),
We consider not the average, but the maximum delay value, since the average hides the values of the worst case which are important when resiliency needs to be improved.
Depending on the situation, it may be desirable to have an approximately equal load for all managers, so that no manager is overloaded, while others have little work. Next, we consider the balanced distribution of nodes between managers. As a formal metric, we introduce the balance of placement, or rather, the imbalance,
As the last aspect of a resilient placement of managers, let us consider how the delay between managers can be taken into account when choosing managers. Formally, the delay between managers
In general, placement with a delay between managers’ considerations tends to place all managers closer to each other. This increases the maximum delay from nodes to managers.
Thus, the target optimization function is given by:
The recovery algorithm is based on prototype described in  but considers modified problem formulation and expands the solution on node overload case.
The physical network is given in the form of a graph
The process of moving the nodes of the virtual network hosted on the failed node,
There is also a probability of the node failure due to overload. To perform a recovery in an overloaded network, the reconfiguration procedure is performed to migrate the virtual nodes hosted on the overloaded physical node.
The recovery process begins with sorting all the virtual nodes located on the overloaded physical node. The criterion (CRT in Figure 7) used to sort these nodes in a virtual network is the capacity of the virtual nodes. Then, the recovery procedure is performed on the first sorted virtual network node, which has a capacity equal to the overloaded, to migrate to the new node of the physical network.
When the load or resources change, some virtual network functions (VNFs) may have to be moved. There is a probability that finding a new node candidate for a node of a virtual network hosted on a failed site will not be possible. In this case, the reconfiguration procedure is performed to migrate one or more virtual nodes. Let us consider the problem of migration as an optimization problem, which is aimed at minimizing the general migration costs with the limits of permissible delay and computational resources.
The goal of optimization is to find the location of virtual network functions (i.e., the location of network functions and resource allocation as well as channels to transfer traffic between them), so as to minimize the cost of the occupied resources of channels and nodes in the physical network, while satisfying the requirements of traffic. Let us give the objective function (26) in the form of a linear combination (with weighted coefficients
Let us determine the binary variable
Then, we enter the binary variable
Thus, we can use the
In order to consider the resources that are consumed while migrating, we introduce the following equations:
denotes the use of resource
The objective function will be calculated as follows:
Taking everything into account, we formulate the problem as follows.
Hence, the objective function (26) is a linear combination of four equations which aims to minimize: the cost of starting and using a server, using server resources, communication channels, and resources for migration. Eq. (27) ensures the one-time allocation of network functions, and Eq. (28) is the administrative possibility of placement on the node. Eqs. (29) and (30) represent a limit for the resources of physical nodes and channels, i.e., they ensure that the amount of resources involved in a node does not exceed the amount of available resources. Eq. (31) represents a flow conservation limit, i.e., the input stream at the node is equal to the output stream. Eq. (32) ensures that the variables in the problem are Boolean. Eqs. (33) and (34) represent a limit for the time of transmission by telecommunication channels and time of processing by service nodes, respectively, and ensure compliance with the specified time requirements for the service.
5. Operating scheme of the resource management system
Thus, before operation starting, it is necessary to have statistics on the requests arrival rate for the network function and the probability characteristics of the request servicing. According to the allocation method, the binding of each network function of the traditional network to the data center and the amount of resources that should be reserved for the corresponding virtualized network function is determined. Next, it is necessary to divide the lifecycle of the network function into intervals during which its configuration will remain unchanged and a certain amount of resources will be activated in accordance with the method of determining the size of the resources constant configuration time interval, while taking into account the expected load. When a mobile network operates, a physical node may not be able to continue to handle an incoming load due to lack of resources or due to its failure, and in this case, a distributed local reconfiguration of resources that re-distributes virtual nodes is triggered.
The general resource management system is shown in Figure 8.
The monitoring system tracks traffic and counts the number of requests. The monitoring system sets the threshold for the number of requests and sends a message to the coordinator if detecting an overload. When the coordinator accepts an overload message from the monitoring system, the resource allocation unit calculates the required amount of resources to process the applications properly and dynamically distributes the estimated volume. Then, the coordinator redirects the requests and the overload is eliminated.
The coordinator is launched periodically. To predict the base load, one can take the average value of historic daily load. The coordinator sends an incoming load to a data center, which maintains excessive workload, and also exchanges data with a resource allocation unit to provide information about the predicted input load.
The resource distribution module is responsible for distributing the appropriate amount of resources needed to handle the load with the specified quality indicators. During the direct operation of the system, this module is started when the actual load exceeds the base predicted value of the load in order to provide additional resources for excessive load. Since the resource distribution module and coordinator do not start when the actual load is lower than the predicted one, the resource reconfiguration procedure creates minimal additional costs associated with this process.
The general operating scheme of the resource management system is shown in Figure 9.
Quantitative and qualitative analysis (Figure 10) of the proposed methods showed a reduction of the cost associated with reserved resources up to 15%, which contributes to increasing the efficiency of load processing, saving computing resources.
The examples of representation of time series values, i.e., loads that illustrate the accuracy of representation, depending on the selected interval of the constant configuration, are presented in Figure 11, where the representation error for the case of intervals in 10 minutes is 7%, and for the case of 60 minutes—19%.
The results of simulation of the method of determining the size of the resources constant configuration time interval (Figure 12) showed that the difference between representational value and actual one can be 9%. If you do not apply a dynamic adjustment system to the value of the constant configuration interval, then the deviation will be 18%, i.e., 9% more, and the resources will be spent more.
In order to assess the proposed approach, the average amount of free resources per day was determined as the difference between fixed allocations, i.e., when 100% of resources were always allocated during the day, and dynamically allocated resources by using NFV. According to the results of simulation, the volume of resources allocated dynamically on average is 42% less than in the case of using the traditional distribution approach. Figure 13 depicts the result of the dynamic distribution of resources in the virtualized EPC of the mobile network in a graphical form. A gray line illustrates the fixed allocations for the worst case scenario. The black curve shows the amount of resources distributed dynamically according to the proposed method.
According to the simulation results (Figure 14), the proposed local reconfiguration method showed up to 27% lower costs compared to a strategy aimed at minimizing delay, the delay being within the permissible limits but by 20% greater.
The main result of the study has become the development of the method for reconfiguring resources of the core network by means of virtualization technology. As a result of the research, the following basic scientific results have been obtained.
The analysis of the current situation in the wireless communication market shows an increase in the workload, which leads to an increase in the need in additional resources. However, the uneven loading of the infrastructure nodes leads to their loss of use; so, there is a need in introducing technologies that both do not lead to downtime of equipment and ensure the quality of load service during the day.
An overview of the NFV virtualization technology has shown that it is appropriate to build wireless networks, since it provides the necessary flexibility and scalability.
We have developed the method for determining the location and capacity of reserved computer resources of virtual network functions in the data centers of the mobile communication operator, which guarantees the quality of providing telecommunication services with the minimum necessary resources by determining their sufficient configuration in a heterogeneous environment of available resources. This allows reducing costs by 13% compared to the randomly selected monocloud and by 47% compared with the traditional approach to deploying the network.
In addition, we have developed the method for determining the size of computing resources constant configuration time interval, which involves its changing and the consideration of both the cost of reconfiguration and the use of resources, as well as provides a flexible use of resources in the virtualized environment, which reduces the percentage of free resources by 42% compared to the dedicated equipment and by 9% compared to existing analogs and reducing the workload on the network.
Furthermore, we have improved the distributed method of local reconfiguration of the virtual network computing resources in the case of a failure or overload, which uses decentralized management and considers migration costs, that redistributes virtual network functions in normal and emergency modes while providing rational resource usage and reducing costs on average by 21%.