Combining Supervisory Control and Data Acquisition (SCADA) with Artificial Intelligence (AI) as a Video Management System

Abstract

The latest Video management systems (VMS) software relies on CCTV surveillance systems that can monitor a larger number of cameras and sites more efficiently. In this paper, we study the utilization of SCADA to control a network of surveillance IP cameras. Therefore, the video data are acquired from IP cameras, stored and processed, and then transmitted and remotely controlled via SCADA. Such SCADA application will be very useful in VMS in general and in large integrated security networks in particular. In fact, modern VMS are progressively doped with artificial intelligence (AI) and machine learning (ML) algorithms, to improve their performance and detestability in a wide range of control and security applications. In this chapter, we have discussed the utilization of existing SCADA cores, to implement highly efficient VMS systems, with minimum development time. We have shown that such SCADA-based VMS programs can easily incubate AI and deep ML algorithms. We have also shown that the harmonic utilization of neural networks algorithms (NNA) in the software core will lead to an unprecedented performance in terms of motion detection speed and other smart analytics as well as system availability.

1. Introduction

The Supervisory Control and Data Acquisition (SCADA) is a software overlay application, which is used on top of intelligent control networks. The control nodes are traditionally smart sensors and programmable logic controllers (PLCs) [1, 2, 3]. In fact, the SCADA industry started due to the need for a user-friendly front-end to control systems containing smart devices, devices, and PLCs. However, SCADA systems evolved rapidly and are now penetrating the reliable operation of modern infrastructures [2, 3] of smart cities. SCADA systems have made substantial progress over the recent years to increase their functionality, scalability, performance, and openness. As shown in Figure 1, the main components of a SCADA system are as follows:

• Multiple Remote Terminal Units (RTUs) or Smart sensors or PLCs.

• Master Station and Human machube interface (HMI) Computer(s).

• Communication infrastructure

Indeed, it is possible to purchase a SCADA system from a single supplier or tailor a SCADA system from different manufacturers, such as Siemens and Allen-Bradley PLCs. This chapter presents a method based on neural networks (NN) for monitoring and operating video surveillance systems (VMS), like those in traffic control networks and electronic plaza sites. The method suggests that the thresholds used for generating alarms can be adapted to each surveillance device (e.g., IP Camera). The intelligent SCADA method was actually utilized in other application fields, for example, in electrical power control and renewable energy systems [3].

In this chapter, we show how to exploit such existing SCADA programs to implement a wide-area video management systems (VMS), which incorporate state-of-the-art AI technologies, such as access control, intrusion detection, face recognition, license plate recognition, crowd detection, and city surveillance. These technologies have been implemented in our emerging VMS, Xanado [4], which is expected to have a unique value in identifying criminals and terrorists, patrolling highways, and in aiding forensics.

Figure 1.

Conventional Architecture of a SCADA system, The Master Station refers to the servers and software responsible for communicating with Remote Terminal Units (RTUs), such as PLCs.

Such SCADA solutions are multi-tasking and are based upon a real-time database that is located on dedicated servers of the system. Such SCADA servers are responsible for data acquisition and handling (e.g., data polling, alarm checking, logging verification, and data archiving) on the basis of a set of chosen parameters.

2. SCADA-based video surveillance solutions

One of the main advantages of SCADA systems is that it allows operators to visualize, in real time, what is happening in any particular industrial process, react to alarms, control processes, change configurations, and track information in real time. However, SCADA systems differ from distributed control systems (DCSs) that are generally found in industrial plant sites. While a DCS covers a plant site, a SCADA system covers much larger areas. Similarly, wide-area video monitoring requires large-scale monitoring systems, like those of SCADA networks. For operations that span several sites, it is important to have a central monitoring station that acts as eyes and ears across all sites. Central monitoring stations use diverse types of cameras and sets of technology to monitor and protect people and property, especially when personnel cannot be on site. It is important that these technologies work together to create a holistic monitoring system. Fortunately, SCADA architecture supports TCP/IP (Internet Protocol), UDP, or other IP-based communications protocols, which makes it ideal for video surveillance control with a network of IP cameras. In fact, SCADA systems have traditionally used combinations of direct serial buses or Ethernet or Wi-Fi connections to meet communication requirements, as well as IP over SONET (Synchronous Optical Network) at large sites. Figure 2 depicts the architecture of SCADA-based VMS programs, which are employing neural network algorithms (NNA). The employed NNA aims to improve the control of the system by using an iterative supervised process. The objective is to determine and optimize the SCADA-VMS control parameters, for specific sites with specific surveillance devices. The chosen parameters, such as the favorite angles of PTZ cameras, the detection speed (of motion anomalies), and false arm causes, will help to increase the surveillance performance and system availability. In addition, the optimized system will minimize false alarms in a continuous adaptive manner, according to each site-specific equipment. Actually, the NNA is based on finding differences in the behavior of the surveillance system over time. The iterative process starts with a database of the stored SCADA-VMS database, as shown in Figure 2.

Figure 2.

Block diagram of SCADA-based VMS, with neural network algorithms (NNA).

The SCADA-based VMS programs, with NNA, can help in this context and can easily incorporate the following features and intelligent analytics:

1. Object detection and tracking in surveillance

2. Face recognition systems for access control

3. Automatic Number Plate Recognition (ANPR)

4. Incident management to identify, analyze and correct dangerous situations.

2.1 Object detection and tracking

The process of identifying objects in an image and finding their position is known as object detection. Figure 3depicts the object detection and identification tasks. This activity benefitted a lot from the field of computer vision assisted by AI. As shown, the trained model using deep learning must be evaluated for its performance on some data called as test dataset’ [5, 6, 7, 8, 9].

Figure 3.

Block diagram of object detection and identification y artificial intelligence.

2.2 Face recognition

Face identifiers offer advantages in access control, safety, security, retail stores, and traffic control. Facie recognition is actually an analytic program that identifies persons from their facial features in an image or video surveillance.

Face recognition programs are usually utilizing AI to quickly identify and interpret complex figures. When comparing a face image to a database of previously stored images of known faces, the AI algorithm can then determine the best match. In fact, the face recognition and analysis algorithms have enabled security systems to capture many wanted criminals and stop many crimes.

As facial recognition increases in efficacy, the number of other applications will also increase (e.g., in banking, retail stores, and means of transportation). Thanks to deep learning-based AI algorithms, face analysis is not only able to identify with high accuracy, but also provides extraordinary analytic capabilities. For instance, it can now detect the criminal behavior, from his/her mood and gests. Thanks to 3D/4D digital signal processing (DSP), with AI, face recognition technology will be expanding in identifying terrorists and criminals’ actions, by incorporating motion detection algorithms.

2.3 Automatic number plate recognition (ANPR)

The ANPR is analytic software that reads vehicle plates and automatically matches them to recognized vehicle license plates, without the need for human intervention. Therefore, ANPR offers accurate identification and safety of vehicle access and traffic control. ANBR is usually implemented using optical character recognition (OCR) and convolutional neural networks (CNN). Actually, CNN is a widely used neural network architecture for computer vision tasks. The CNN automatically extracts important features on images. More details about CNN are provided below in Section 4.

3. Artificial intelligence in video surveillance and video analytics

Artificial intelligence (AI) is nowadays revolutionizing video management systems (VMS) and the ways of securing smart premises and smart cities by video surveillance and control. In fact, AI-based technology can promote security and surveillance equipment by enhancing object detection and motion interpretation as well as providing analytics with increasingly reliable data. For instance, the reduction of false alarms in security systems is one of the major benefits of AI-based tools and algorithms. For instance, AI-CCTV cameras are networked IP cameras that deliver advanced analytical functions, such as face recognition, vehicle classification, car counting, license plate recognition (LPR), and other traffic analytics. Advanced video analytics software is built into the camera and recorder, which then enables artificial intelligence functions. Some AI algorithms are rule-based and others are self-learning. Like typical CCTV cameras, AI-CCTV stores information so any incidents can be reviewed. However, AI CCTV can detect and send alerts in real time. In particular, SCADA legacy can help a lot in large-sites and wide-area VMS systems.

The artificial intelligence (AI) tools are heavily dependent on neural networks (NN) and computer vision [5]. As shown in Figure 4, a neural network (NN) is a system of software or hardware, which mimics the operation of human brain neurons. Therefore, an NN is simply a group of interconnected layers of a perceptron. Note that an NN has multiple hidden layers, and each layer has multiple nodes. The neural network takes the training data in the input layer and forwards it through hidden layers, on the basis of specific weights at each node [10]. Therefore, it returns an output value to the output layer. The inputs to nodes in a single layer have adaptable weights that affect the final output prediction.

Figure 4.

Schematic diagram of a neural network (NN) and how it works.

There are a lot of different kinds of neural networks that are used in machine learning projects. There are recurrent neural networks, feed-forward neural networks, and convolutional neural networks (CNNs). It can take some time to properly tune a neural network to get consistent, reliable results. Testing and training your NN is important before deciding which parameters (input features of a face image) are important in your recognition model.

4. Deep learning-based video surveillance solutions

In deep learning, a convolutional neural network (CNN) is a sort of NNs; which is particularly useful in video surveillance projects. The advantage of deep learning-(DL)-based algorithms with respect to legacy computer vision algorithms is that DL systems can be continuously trained and improved with updated datasets.

In DL, a convolutional neural network (CNN) is a type of NN, commonly used in image recognition and processing, with emphasis on machine vision of images and video. As shown in Figure 5, the layers of a CNN consist of an input layer, an output layer, and a hidden layer that includes multiple convolutional layers, pooling layers, fully connected layers, and normalization layers.

Figure 5.

Schematic diagram of the layers of a convolutional neural network (CNN) showing its classification sequence of a handwritten character (in ASCII).

Deep learning systems have shown a remarkable ability to detect undefined or unexpected events. This feature has the true potential of significantly reducing false alarm events that happen in many security video analytics systems. Many applications have shown that deep learning systems can “learn” to achieve 99.9% accuracy for certain tasks, in contrast to rigid computer algorithms where it is very difficult to improve a system past 95% accuracy [4].

5. Xanado program

Xanado is a video management system (VMS) software, designed for large-scale and high-security installations. It is built as a client-server DCS to ensure end-to-end protection of video integrity and boost the overall performance of existing hardware. In addition to central management of all data servers, IP cameras, and users in a multi-site setup, Xanado includes an integrated video wall size for operators demanding overall awareness of any event. The software supports failover recording servers making it the perfect choice for mission-critical installations that require continued access to video recording in case of a server failure. Xanado is ideal for installations with 24/7 operation requirements and can run on high-speed recording engines (NVR), making it suitable for monitoring airports, banks, traffic control, as well as smart city surveillance.

The general system architecture is shown in Figure 6. As shown in Figure, the management data server lies in the center of the VMS. It holds the main application and handles the system configuration. Note that the recording server is responsible for all communication, recording, and event handling related to devices, such as cameras and I/O modules. The system stores the video in a customized database. The management server, event server, and log server use an SQL server to store configuration, alarms, and log events. As the VMS is designed for a large-scale operation, the Management Client may run locally or remotely, for centralized administration.

Figure 6.

Xanado general system architecture.

The smart client nodes (SCN) are working as follows. SCN connects to the management server and attempts to log in. The management server tries to authenticate the user and the user-specific configuration is retrieved from the SQL database. Therefore, the login is granted and the configuration is sent to the SCN. Live video streams are then retrieved from cameras by the recording server. The recording server sends a multicast stream to the multicast-enabled network. This requires that all switches handling the data traffic between the SCN and the recording server must be configured for multicast.

We adopt the ONVIF standard [11] for full video interoperability in multi-vendor installations to ensure information exchange by a common protocol. The ONVIF protocol profiles are collections of specifications for interoperability between ONVIF compliant devices, such as cameras and NVR.

Of course, Xanado VMS has many add-on modules and can be tailored to several specific applications. In the following subsection, we describe an application of our VMS to the case of Electronic Toll Plaza control, which is utilized nationwide in highway traffic control.

6. Case study: toll plaza control

Toll plazas are utilized on traffic highways to collect fees from passing vehicles. The toll plaza consists of six zones—an approach zone, queue area, toll lanes, the toll island, departure zone, a bailout lane, and, in some cases, a terminal supervisor. The so-called Electronic Toll Collection (ETC) enables toll collection without delay or total stop of vehicles. This section deals with the equipment and software to be installed on the roadside of toll plaza networks. We shortly depict the basic concepts for installing the system of electronic toll collection. Figure 7 is a schematic of a single booth and a lane of an electronic toll plaza.

Figure 7.

Schematic of a booth in an electronic toll office, which communicates with a passing vehicle.

6.1 Main specifications of a smart toll-plaza control system

A smart toll collection system has the following four components. The first three components are usually installed at the toll booths. The later backend is installed in the control room and is usually connected with manages the complete toll collection process [12].

• Vehicle Tracking System: This is a set of surveillance cameras, vehicle identification, and hardware sensors (e.g., RFID readers and loop detectors).

• Vehicle Enforcement Devices, such as barriers.

• Billing Terminals (POS), for manual fees collection of passing vehicles.

• Toll Management System: This system is responsible for processing the authentication and billing data of passing vehicles and dispatching this data to the headquarter office.

6.2 Toll plaza components equipment

The toll system comprises of Lane System and Plaza System, integrated into an architecture that facilitates easy and accurate toll collection. The following Figures 8 and 9 depict the lane and plaza equipment. The OHLS (Over Head Lane Signal) is often required. The so-called AVCC (auto vehicle classification system ) is needed to determine the different fares of different vehicles if lanes have no signage with the type of passing vehicle. AVCC Systems may be Treadle systems that use a combination of vehicle magnetic loop, height sensors, and piezo sensors. Alternatively, AVCC may be IR-based. Also, the PBX telephone, which may be required inside each booth, is not shown. The WIM (Weigh-In-Motion) platform, which senses and records the vehicle weight, is dedicated to truck traffic control. Additional cameras should be installed inside each booth. In fact, a major challenge faced by any concessionaire in operating toll roads is the prevention of revenue leakage that goes very high like 20% of the daily collection in the situation of un-monitored systems.

Figure 8.

Schematic of toll-plaza lane and booth equipment.

Figure 9.

Schematic of plaza equipment.

6.2.1 Booth & Lane equipment list

a-Booth Equipment: 1. Toll Lane Controller; 2. Operator Terminal Screen (POS); 3. Receipt Printer; 4. Barcode Reader; 5. Barrier Controller; 6. Alarm; 7. Document Viewer; 8. WIM Indicator.

b-Lane Equipment: 9. OHLS; 10. Fare Display; 11. Incident Capture Outdoor Camera; 12. Barrier Gate; 13. Traffic Light; 14. AVCC; 15. Vehicle Entry Loop; 16. Vehicle Exit Loop; 17. Outdoor ANPR Camera; 18. Smart Card or RFID Reader; 19. WIM Platform.

6.2.2 Plaza equipments

The toll Plaza equipment consists of:1. Database Server; 2. CCTV Display Screen; 3. POS Workstation; 4. CCTV (extra security) Cameras; 5. Report Printer; 6. IP Phone; 7. Network Switches; 8. IP phone Master (PABX) Unit; 9. UPS Power Supply.

6.3 Plaza network installation procedure

The first step, before the installation of any security system, is to ensure the presence of detailed drawings and documentation, a bill of materials with their specification. The installation of the ETC system uses many devices like vehicle-mounted electronic tags, toll point of sale(POS), RFID readers, and switches.

Actually, there exist several scenarios to connect the plaza network using Ethernet copper cables and/or optical fibers. The network topology may have several choices. For instance, you can use a bus topology, rings, μ-rings, or a hybrid bus/ring topology. One of these scenarios is depicted below in Figure 10.

Figure 10.

Connection scenario #1 of a toll plaza using a ring fiber between booths and plaza control room.

If we have a sufficiently long multi-core optical fiber cable, you can also install a large ring between all switches in the main booth and the control room.

Note that the design of the toll plaza data network should be secured and isolated from direct exposure to other internet users. In particular, the sensitive video signal (from IP cameras) should be routed indirectly to the internet, via NVR, followed by a separate NIC card to the plaza server. The plaza server can be then connected to the internet by another NIC card. This is illustrated in the following Figure 11. In all cases, a virtual private network (VPN) should be installed before routing the plaza signals, to any external WAN, such as the internet.

Figure 11.

Illustrating example showing the internet.

In all cases, the data switches are dividing the core network into small subnets; called VLANs. For instance, by sorting node devices by functions or positions (camera, POS, etc., VLANs are often associated with IP subnets. Hence, networks with different VLANs will not be visible to each other. On the physical layer, the network remains the same as shown in Figure 12.

Figure 12.

Example of a VLAN configuration of a toll plaza network.

6.4 IP planning

The only way for someone to access the CCTV system is to know the IP address, username, and password (Table 1).

Camera No	Camera name	IP address	Location	Username	Password
1	Front PTZ	192.168.1.81	Front	assigned by NVR	assigned by NVR
2	Back PTZ	192.168.1.82	Back	assigned by NVR	assigned by NVR
3	Booth Camera	192.168.1.83	Booth	assigned by NVR	assigned by NVR
:	:	:	:	:	:

Table 1.

Example of IP planning of the overall data network.

6.5 Port forwarding and accessing the internet

To remotely view a security CCTV system, you had to allow it to communicate to the internet. To allow access to your system from the internet, you have to configure the firewall inside the router to flow through the NVR. This process is called port forwarding and requires some advanced knowledge. There exist many guides on port forwarding if you feel confident configuring remote viewing. Each router will have its own method for port forwarding and I recommend checking PortForward.com for your specific model.

6.6 Remote viewing using a smartphone

There exist Android and iPhone network operating systems (NoS) Apps that work with CCTV systems, such as IPTecno Pegaso [12]. This can connect to the system and display live video feeds. It also allows for one-way and two-way audio interaction, PTZ camera control, and motorized camera control. To learn how to connect your system, please follow any guide on how to view security cameras from iPhone or Android, such as this: https://www.cctvcameraworld.com/how-to-view-security-cameras-from-phone/.

6.7 Plaza database server and toll management system

The Smart Toll Collection System can greatly expedite the time taken by each vehicle to pay the toll fees. There are existing solutions that are deployed and are practically well-suited. A distributed database architecture platform should be installed to enable operations of ticket issues and validation. In the control system, the failure of the LAN connectivity should not impact the lane operation.

6.8 ETC management system

The following Figure 13 depicts the modular architecture of Xanado, with emphasis on traffic surveillance control at toll plazas. Such software provides comprehensive capabilities to manage toll collection operations (24/7). The Central Administration Module facilitates the entire operation of monitoring and collection across toll plazas as one centralized unit. The Plaza Module can configure, manage all plazas toll collection operations, report toll collection in an audited manner. The Lanes Module is running under the plaza module, lanes module facilitates accurate toll collection for the vehicle passing from the toll plaza.

Figure 13.

Modular architecture of Xanado VMS, with application to traffic surveillance and traffic control at a toll plaza network.

7. Conclusions

Public concern over security in recent years has driven the demand for video surveillance. Security and video surveillance need continuous change with time, to cope with the new hardware capabilities of IP cameras and video storage equipment. In addition, the increasing need for video analytics is required in modern VMS software, to perform their job of monitoring activities and protecting humans and their properties. Governments and police departments worldwide are constantly looking for new CCTV surveillance features that will help prevent crime. The latest VMS software relies on CCTV surveillance systems that can monitor a larger number of cameras and sites more efficiently. Therefore, combining SCADA features with VMS is significant.

This chapter presents a method based on neural networks (NN) for monitoring and operating video surveillance systems (VMS), like those in traffic control networks of electronic plaza sites. The method suggests that the thresholds used for generating alarms can be adapted to each surveillance device (e.g., IP Camera).

The industry needs to do more research on hybrid systems that combine the best of SCADA and AI algorithms together with VMS software.