Open access
1. Introduction
Sensordata fusion is the process of combining error-prone, heterogenous, incomplete, and ambiguous data to gather a higher level of situational awareness. In principle, all living creatures are fusing information from their complementary senses in order to coordinate their actions and to detect and localize danger. In sensor data fusion, this process is transferred to electronic systems, which rely on some “awareness” of what is happening in certain areas of interest, and since the sensors involved most often are confronted with a dynamical world, the state of interest underlies an evolution process in time, which has to be reflected within the data processing. By means of probability theory and statistics, it is possible to model the relationship between the state space and the sensor data. Kinematic laws and stochastic processes further provide the basis for evolution models in object tracking. The number of ingredients of the resulting Kalman filter is limited, but its applications are not.
Invented many decades ago—Kalman’s initial paper was published in 1960, and it is well known that similar solutions to the tracking problem were found even earlier—the Kalman filter is an algorithm with an extraordinary career. In the days it was invented, the Kalman filter was designed for tracking airplanes in the sky based on surveillance radars. As a consequence, the problems of measurement error and object dynamics were the key points, which the Kalman filter can cope with. Since then, sensor technology has evolved enormously. In recent decades, sensor technology has become increasingly important for numerous civilian and military applications, and it is obvious that this trend will continue in the future. High performance sensors have conquered many novel applications, and existing applications have been brought to a much higher level of technical complexity. This technological trend is accompanied with an evolution in the field of sensor data processing algorithms. Besides the family of Kalman filters, other solutions to the Bayesian approach to information processing have been developed—based on grids or Monte Carlo simulation for instance. However, the most often used approach for practical tracking system still is the Kalman filter, at least in one of its numerous variants. While the classical assumptions including linear models for sensor and dynamics, perfect data association, and known track existence are mostly handled in the academic and educational field, many extensions have been developed to bring the Kalman filter into practical and industrial systems. It is literally impossible to list all methods that have been developed based on the Kalman filter, a few of the most important variations possibly would be:
Extended and unscented Kalman filter (EKF/UKF) for nonlinear models.
Multiple hypothesis tracking (MHT) for ambiguous data association.
Interacting multiple model (IMM) for Markov Chain motion model systems.
Distributed Kalman filter (DKF) and information filter for multisensor fusion.
Random matrix model and hypersurface model for extended target tracking.
Self-localization and mapping (SLAM) for autonomous navigation in unknown environments.
In the past decade, challenges in target tracking applications have changed again. Today, often heterogenous sensors such as radar, LIDAR, E/O and I/R cameras, and others are combined to achieve robustness and a high level of perception. Also, Kalman filters are applied to a great variety of applications where, for instance, intelligence data from social media or computer program logs for intrusion detection systems are combined to achieve situational awareness. And, of course, we may not omit the uprising self-driving cars, which might be the most famous application these days. It is a particularly interesting application, since the challenges involved are enormous: bias- and error-prone point clouds are obtained from partially scattering points in the environment, the quality highly depends on weather and daylight conditions, the environment might be unknown, and other participants might not be aware and have to be protected. And all this is safety critical. And the processing unit is supposed to be lightweight for economic reasons.
There are many more examples to demonstrate for further development, trends, and improvements of Kalman filter based algorithms, and I am happy that we have gathered some interest of practical relevance in this book.
2. Kalman filter in the shed light of artificial intelligence
Artificial intelligence (AI) comprises a wide range of technologies and methodologies such as machine learning (ML), support vector machines (SVM), logistic regression (LR), game theory, logic reasoning, and many more. The term was introduced once to highlight the increasing complexity of machines in combination with numerical algorithms for robotic perception for instance. In particular the success in image and video processing based on massive training data, well-tuned models, and high-performance hardware has created a boost in research and expectations for new AI applications and results in the past years. ML methods show good performances when it comes to perception task such as object classification and image segmentation; however, it will be hard to match the expectations on AI if models created and tested by engineers together with logic reasoning and the Bayes formalism are kept out of complex systems with decision-making and strategic situational awareness on a higher level. This is due to the fact that ML encodes correlations between input data and the resulting class very efficiently but does not have a concept to infer causality. Thus, a deeper understanding beyond a rough comparison of shallow features is not possible. But often, this is not enough. In particular when it comes to safety critical applications or even military use, a mere performance argument is not sufficient to justify the usage of algorithms, which still suffer from the black box property with respect to the transparency of computed results. Moreover, one cannot overemphasize the dependency of ML methods on huge training sequences, which must be labeled for most useful applications. Image recognition essentially is solved due to the fact that there are tons of labeled pictures in the Internet. However, this is not the case for complex multisensor systems, which must adapt appropriately in unseen environments. Moreover, it is obvious that the feature space grows exponentially when it comes to complex systems and higher-order reasoning. As a consequence, the amount of data which would be necessary to train a neural network for such systems is a natural show stopper. This is where we will find logic reasoning and Bayes methods such as the Kalman filters (KF) increasingly. The human experience and ability to infer chains of causality can be transferred into AI algorithms by means of sensor models, motion models, and time evolution models. The KF with its low weight consumption when it comes to numerical costs is an important backbone in advanced driver assistance systems already, and this trend is about to continue heavily.