Typical sensors in an android smartphone.
Nowadays, various sensors are embedded in smartphone, making it a great candidate for localization applications. In this chapter, we explored and listed the localization sensors in smartphone, their characteristics, platforms, coordinate system and how they can be used in BAN. These sensors can be roughly divided into three types: physical IMU sensors (accelerometer, gyroscope and magnetometer), virtual IMU (gravity, step counter and electronic compass) and the environmental sensors (barometer, proximity and other miscellaneous). By applying different mathematical methods, the location of the target or the users can be calculated and used for further use, such as navigation, healthcare or military purpose.
- sensors fusion
Nowadays, smartphones are open and programmable and come with a growing number of powerful embedded sensors, such as an accelerometer, digital compass, gyroscope, GPS, and camera, many of which, can be used for localization. In a typical BAN, the localization sensors measure certain characteristics related to the location of the phone to position it on a map . These sensors either provide us information on the distance from reference points to help us calculate the absolute location, such as Received Signal Strength (RSS), Time-of-Arrival (TOA), Angle-of-Arrival (AOA) of RF signals or they provide us with the velocity and direction of movement from the motion sensors for calculation of displacement or the relative location of the object. One of the fundamental differences between the two methods is that the absolute localization needs an infrastructure of reference points but relative localization position the object relative to its previous location. Figure 1 shows two examples of the results of localization using absolute and relative positioning systems  on a square route in a typical office building. Figure 1b shows the results of absolute Wi-Fi localization. The estimated locations (dots) are randomly distributed around the path of movement (solid line) of the object. Figure 1b shows the results of localization using an Inertia Motion Unit (IMU) device measuring the speed and direction of the movement. The estimated path of movement follow a straight line but, because of errors in angle and speed of movements, the location estimate gradually drifts from the actual path. Hybrid algorithms can combine the results of the two classes of sensors to achieve a more reliable navigation. These algorithms use the absolute localization to reduce the drift of IMU and use the IMU result to smooth the location estimates of the absolute location.
The most popular absolute positioning systems, such as Global Positioning System (GPS) and Wi-Fi Positioning System (WPS), sense features of the RF signals to measure the distance between the object and the reference points. The 26 satellites used as the reference points for GPS infrastructure are deployed for the purpose of positioning, while WPS opportunistically uses the Wi-Fi and cell phone signals for localization. Since absolute RF localization is the most fundamental and the most complex element of positioning, it is crucial to have a deep understanding of the behavior of RSS- and TOA-based ranging in different environments, which is well discussed in a great number of existing literature .
On the other hand, a typical application for relative positioning is Inertial Navigation Systems (INS). An INS uses a computer, motion sensors (accelerometers), rotation sensors (gyroscopes), and occasionally magnetic sensors (magnetometers) to continuously calculate by dead reckoning the position, the orientation, and the velocity (direction and speed of movement) of a moving object without the need for external references. With the development of technology, other methods can also be used for relative positioning. The speed of movement can be measured by number of rotations of the wheel (speedometer), Doppler spectrum of a received RF signal, the number of steps that a pedestrian walks (step counter), or similarities between the consecutive pictures taken by a camera.
Recently, Internet-of-Things (IOT) becomes a new fashion and opens up a new door to BAN localization. Large amount of IoT sensors have been deployed inside buildings, each of which has the ability to communicate and share location information to one other. Since the smartphone is able to communicate with both WiFi- and Bluetooth-based devices, its role in BAN localization becomes more and more important.
In this chapter, we present the way smartphone system visualize the location. We will also illustrate all the localization sensors in smartphone platform and discuss the characteristics of each. Examples are provided to show how these sensors are applied in the localization system and their accuracy is investigated as well.
2. Coordinate systems to visualize the location
In order to position a location on a map we need a coordinate system to visualize the location information. For global positing of a location, we need to refer to the global 3D coordinates of the earth. The most commonly 3D coordinates of the earth (Figure 2) are the geographical polar coordinates commonly referred to as: longitude, latitude and altitude . Longitude expresses angular deviation from east to west on the surface of the earth in degrees and latitude is the angular deviation from north to south in degrees. Elevation shows the distance from the center of the earth and it is usually given with respect of the sea level height. In the aerial vehicle applications elevation is referred to as altitude.
Ground navigation systems for outdoor and indoor areas track movements of the vehicles or robots on 2D maps such as layout of a floor of a building or 2D map of streets of an urban area in area for which altitude does not change substantially. Even in multi-floor 3D applications we use a 2D map with the floor number (a representative of altitude). Cartesian coordinates provide a better tool for visual tracking for most of popular applications in indoor and urban areas. Therefore, we usually convert longitude, latitude, and altitude to a Cartesian form using:
In absolute positioning, a GPS or WPS location fix in polar or in Cartesian coordinate, represent a point in the 3D world frame coordinate. In relative positioning we represent the velocity and direction as vectors at each location. Accelerometers and gyroscopes are mechanical devices used for measurements of differential vectors for acceleration and rotation. We integrate the differential acceleration to determine the instantaneous velocity vector and we integrate the rotation vectors to determine the direction of motion.
In the universal Cartesian reference coordinate system, shown in Figure 3a, an acceleration or velocity vector in X-axis defines a vector tangential to the ground at the measurement sensor’s current location pointing to the East. The vector in Y-axis is tangential to the ground at the measurement sensor’s current location and pointing towards magnetic north. A vector in Z-axis points towards the sky and perpendicular to the ground at the location of the measurement sensor. However, as shown in Figure 3b, measurement sensors are MEM devices installed in a device, such as a smart phone, and their coordinates are defined relative to the device’s screen and not the universal earth coordinate. When a device is held in its default orientation, the X-axis is horizontal and points to the right side of the device, the Y-axis is vertical and points up to the top of the device, and the Z-axis points towards the outside of the screen face. In this system, coordinates behind the screen have negative Z-values.
3. Sensors and platforms for opportunistic localization
Opportunistic localization is based on the location sensor payload of the platform we want to localize. Typical smart phones carry RF based GPS, Wi-Fi, Cell Phone, and iBeacon (Bluetooth Low Energy) chip sets, which support a variety of opportunistic RF localization capabilities for different environments and precision requirement. In addition, they carry a number of other sensors which can be used to enhance positioning of the device using traditional RF signals. Table 1 shows typical sensors available in an Android device and their functionality if they are used for localization. In addition to the traditional sensors such as gyroscope, magnetometer, accelerometer and barometer, there are sensors for ambient light, temperature, and humidity as well as force of gravity levels, proximity and step counts. The sensors can be beneficial for localization applications to enhance precision of RF localization. We can use the data collected by these sensors for 3D localization, calculation of speed and direction of movements and specifying the environment of operation. Positioning algorithms use RF location and motion information obtained from a variety of sensors. These sensors perform differently in different environments. For example, GPS which provides very reliable information for outdoor navigation does not operate deep in indoor areas. Therefore, in addition to the algorithm to determine the location, we need to have an algorithm to detect the environment to distinguish between the indoor and the outdoor environments. The data from all sensors is available as a resource to a positioning engineer to design different algorithms to achieve the precision requirements in the environment of operation.
The detailed description of these localization sensors is presented in the following sections as well as how accurate they can achieve in locating objects.
3.1. Accelerometer sensor
Accelerometer sensor measuring the acceleration (differential rate of variations of velocity) applied to a device in meter per square second in the local Cartesian device coordinates (Figure 3b). Figure 4a shows the basic concept inside an accelerometer. A weight is hanging between two springs in a box and the displacement of the weight due to external forces moving the box is measured to calculate the acceleration in the direction of movement. The right side of Figure 4c shows how we can measure 3D acceleration of a plate carrying three accelerometer boxes placed in orthogonal directions.
Conceptually, an accelerometer measures the forces applied to the sensor in different directions and using Newton’s second equation calculates the acceleration by dividing the force by mass of the sensor. When an IMU device rests in a place on a table in parallel to the earth surface, the accelerometer of the device should read a magnitude of , the gravity of the earth. When the IMU device is in free-fall and therefore accelerating towards the ground at , its accelerometer should read a magnitude of .
In order to measure the real acceleration of the device, also referred to as linear-acceleration, the contribution of the force of gravity must be eliminated. Applying a high-pass filter to the results of accelerometer readings eliminates the slow changes of the readings caused by gravity and detects the fast changes introduced by movements of the device. Conversely, using a low-pass filter isolates the force of gravity and eliminates the fast changes in the acceleration caused by the movements of the device.
3.2. Gyroscope sensor
Gyroscope sensor measure the differential rotation momentum around the local x, y, z axis of the device Cartesian coordinates in radians/second. Figure 4b shows the basic concept behind a gyroscope. A cylindrical mass is hanging around a central shaft like a wheel, as we change the direct wheel tits and we measure the angular displacement of the wheel. The coordinate system is the same as the one is used for the acceleration sensor. The right side of Figure 4c shows how we can measure the rotation in 3D around each of three axis of the device by placing three gyroscopes in orthogonal directions. The output of the gyroscope is integrated over time to calculate a rotation describing the change of angles over time.
Rotation is positive in the counter-clockwise direction. That is, an observer looking from some positive location on the x, y or z axis at a device positioned on the origin would report positive rotation if the device appeared to be rotating counter clockwise. If we refer to angular speed around the x-axis as angular speed around the y-axis as and angular speed around the z-axis as (Figure 5a). The Euler transformation maps the accelerometer readings, , from the device coordinates to the universal coordinate:
Figure 5b shows a typical IMU device with both the device and the earth-fix coordinates. To find the location from the above equation we need to take a double integral. The first integration provides the velocity in each direction and the second integral provides the distance traveled by the device in each direction. Since smartphones are equipped with all the sensors mentioned above, we can consider them as a kind of smart IMU, and movement can be determined in the same way.
In practice, the gyroscope noise and offset will introduce some errors which need to be compensated for. This is usually done using the information from other sensors. The gyroscope cannot be emulated based on magnetometers and accelerometers, as this would cause it to have reduced local consistency and responsiveness. It must be based on a usual gyroscope chip. Most recent systems are of the strap down type, where the IMU sensor outputs are taken directly and processed to compute the orientation and displacement of the vehicle.
A simple laboratory experiment moving the IMU device shown in Figure 6a forward and then turning to the right. The IMU device is a relatively expensive device purchased in mid-2000 for approximately [4, 5]. Figure 6b and c shows the three dimensional readings of the gyroscope and accelerometer of the device. The gyroscope reads the pitch, yaw and roll in radian/second. The only rotation with this scenario of motion is around the z-axis for the yaw angle and pitch and roll remain at zero. Acceleration only has reading in x and y coordinates because we have no vertical motion. These values change ups and downs. Figure 7a and b shows the results first and second integral of the results after applying Euler’s equation combining the results of accelerometer and the gyroscope (Eq. 2). The two figures represent the speed and traveled distance recorded by the laptop. The linear speed that is the square root of the sum of squares of the speeds on the x and y axes should remain relatively constant representing the speed of movement. Figure 7c shows the trajectory of the movement of the device as predicted from the results of IMU sensors readings. Results are similar to those of Figure 1b, which was obtained from another IMU unit.
Magnetometer is a magnetic field sensor measuring the ambient magnetic field along the 3 sensor device local x, y and z axis in micro-Tesla (). Modern MEMS magnetometers measure the current induced in a coil due to changes in the magnetic field in the surrounding environment. Figure 8a shows the basic concept behind modern magnetometer devices. The intensity of the induced current in the coil is used as a measure for the strength of the magnetic field. Figure 8b shows how the 3D magnetic field can be measured with a device. Since magnetometers, measure strength and direction of magnetic field, they are also useful for measurement of the direction for localization and tracking. In the following sections we explain how we can design an electronic compass from readings of a MEMS magnetometer inside a smartphone.
3.4. Gravity sensor
Rather than high pass filtering, we low pass filter the measurements from an accelerometer we measure the force of gravity in the device coordinate. The magnitude of this vector is and its 3D (x, y, z) values reflect the effects of gravity on each of the three axis. These are values that we should subtract from the acceleration vector components to determine the linear acceleration. The value of the gravity vector in device coordinate is used for turning the smartphone and pads screens to be in parallel to the eye of the person reading the screen. Considering device coordinates shown in Figure 3b, if the gravity force along y-axis is the highest component, device is in up position and screen with smaller top is used. When the x-axis is the highest device is in the side position and the screen with long top is used. When device is resting on the floor z-axis is the largest value and the screen can be turned off. Results of gyroscope and magnetometer measurements are often used to complement the result of gravity force measurement obtained from the accelerometer. When the device is at rest, the output of the gravity sensor should be identical to that of the accelerometer.
3.5. Step counter sensor
Step counter counts the number of steps taken by the pedestrian carrying a smart device such as a smartphone or a health monitoring band. The value is in integer and it reset to zero on a system reboot. The timestamp of the event is set to the time when the last step for that event was taken. The obvious application of this sensor is in fitness tracking applications but it is also useful for indoor geolocation using smart devices.
We can implement a step counter by using the measurements from an accelerometer. Figure 9 shows hundred samples of magnitude of the accelerometer measurements for an Android phone. Considering Eq. 7.2, the magnitude of the acceleration is . Figure 2a shows the pattern of changes in acceleration between steps of a pedestrian carrying a device with accelerometer with hands. In each step first the pedestrian increase accretion of the body torso (carrying the hands and the device) using the landing foot and then she/he reduce the speed by applying negative acceleration to land on the other foot. As a result, variations of the magnitude of the accelerometer measurements follows a zigzag pattern on each step. We can detect the number of steps by counting the zigzags and that can be done either by peak detection or by detecting the zero crossing. With the peak detection algorithm we detect all peaks above a threshold and we select the highest in a window of time associated to the time for one step (Figure 9a). For zero crossing we find the intersect between the line representing the average acceleration and rising (or falling) parts of the plot (Figure 9b).
Using the step counter and the time stamps associated with each step, the average speed and the average step size can be determined and with that we can estimate the linear distance traveled by a pedestrian carrying a device with an accelerometer. To position the location of a pedestrian in a 2D map indoors or outdoors we also need to measure the rotations. Rotation can be measured with the results of gyroscope or the compass.
Figure 10 shows the results of an example relative localization system using the step counter and gyroscope of a typical smartphone at the third floor of the Atwater Kent Laboratory at the Worcester Polytechnic Institute. The schematic of corridors of this floor of building and layout of the exterior of the building in shown on the right figure. A user carrying the smartphone enters the floor from right to and walks straight and then around the central corridors before turning back to the original location. User holds the smartphone in front of the torso in parallel to the ground. In this posture the z-axis of the smartphone is towards the ceiling and the only rotation of the device is around this axis. Figure 10a shows the results of gyroscope readings in the three dimensions. There are four jumps in rotations around the z-axis (Azimuth or yaw) associated with four 900 turns in the route of movement of the device. The red line in Figure 10b illustrates the estimated path of movement when the results of step counts is complemented with the gyroscope readings. Typical increasing pattern of the drift of the location estimate away from the actual path of movement is observed. This type of error pattern is caused from using speed and direction of movement for relative localization and it is similar to the error pattern [6, 7, 8].
3.6. Electronic compass
Compass is one of the oldest mechanical magnetometer historically used for navigation. If we lay a compass in parallel to the earth surface the heading of the compass magnet points to the magnetic north. As a results, by rotating the compass to point towards a specific direction we measure the angle of that direction with respect to the magnetic north. This angle provides us with direction for relative navigation. A compass takes advantage of the earth electromagnetic field to determine the rotation angle of the device. Earth acts like a large dipole magnet with magnetic field lines originating from magnetic south pole (near the geographic south pole) and terminating in magnetic north pole located near the geographic north pole (Figure 11a). A compass has a small magnet in the center aligning with the magnetic north pole in all locations on the surface of the earth. The angle between the compass heading (the magnetic north) and the geographic north is called the magnetic declination angle (Figure 11b). The value of the decantation angle changes in location as well as in time. If we know the direction of geographical north in a location we can align the compass to that and measure the declination angle as the angle between the heading of the compass magnet and direction of the geometric north. In the east coast of United States inclination angle can go as low as (W) and on the west coast it can be as high as (E).
The National Oceanic and Atmospheric Administration (NOAA) has a public web calculator for calculation of magnetic declination angle around the Globe. Using this web site we read W on December 22, 2017 in Newton, MA at a location with 130-ft elevation and longitude, latitude of (N, W). Results indicate that the accuracy of values are within and the estimated value declines E each year.
All popular smartphones in the market carry an electronically emulated compass application. These applications typically emulate the compass using the results of magnetometer, commonly available as a part of the IMU chip of the smartphone. Figure 11a shows a typical electronic compass implementation on an iPhone when the device is facing upward (z-axis of the device and the earth fix coordinates are aligned). The scaled round shape on top of the figure allows visualization of the angle between the north axis (y-axis) of the earth and the device to emulate the appearance of a compass. The Hx and Hy measurements of the magnetometer (Figure 12b) define the direction of compass heading in degrees using:
As we turn the device around its z-axis, while laying the device on a table, the reading of the angle changes. For () reading, shown in Figure 12a, the device heading points to the geographic north in Newton, MA which has a declination angle of approximately .
Barometer is a pressure sensor measuring the atmospheric pressure in hPa (millibar). Barometer is used to estimate elevation changes and consequently floor number of operation inside a multi-floor building. The relation between air pressure and height is given by :
Using Eq. (4) and parameters in Table 2, we can calculate altitude from air pressure. Figure 13 shows the results of a smartphone barometer measurements in a typical 3-story office building. Figure 13a is produced from the results obtained in an elevator elevating three floors of the building and Figure 13b shows the results for climbing one floor of the building using stairs. Both figures demonstrate gradual change of the barometer reading during climbing up and down among the floors regardless of the speed and the pattern of motion. Range of variation of measurements in the elevator experiment for climbing three floors are approximately three times larger than that of the measurements for obtained from climbing one floor on the stairs. These observations demonstrate that barometer can be utilized to measure the change in altitude inside the buildings. Figure 14 shows results of barometer measurements from a smartphone when a user carries the device in four floors of a typical office building in two different time of the same day. Pattern of measurement in different floors are clearly different allowing us to distinguish the floor of operation. However, results of barometer are corrupted by additive Gaussian noise, bias, and it is sensitive to the time of measurement [10, 11]. Methods to mitigate these effects and have a better estimate of the floor number in an office building is discussed in .
|Sensor name||Description||Type||Common uses|
|Barometer||Measures the atmospheric pressure in hPa (millibar)||Environment||Elevation for floor numbering|
|Ambient temperature||Measures ambient room temperature in degree Celsius||Environment||Detecting environment changes|
|Ambient light||Measures ambient light level in SI lux.||Environment||Detecting environment changes|
|Relative humidity||Measures relative ambient air humidity in percent||Environment||Detecting environment changes|
|Accelerometer||Measures the acceleration force in m/s2 that is applied to a device on all three physical axes (x, y, and z), including the force of gravity||Mechanical||Motion detection (shake, tilt, etc.)|
|Gyroscope||Measures a device’s rate of rotation in rad/s around each of the three physical axes (x, y, and z)||Mechanical||Rotation detection (spin, turn, etc.)|
|Step counters||Counts the number of steps taken by the person carrying the device||Mechanical||Measuring traveled distance|
|Magnetometer||Measures ambient geomagnetic field strength along the x,y,z axes in micro-Tesla (μT)||Electromagnetic||Creating a compass|
|Gravity||Measures the force of gravity in m/s2 that is applied to a device on all three physical axes (x, y, z)||Electromagnetic||Motion detection (shake, tilt, etc.)|
|Proximity||Measures distance from the sensor to the closest visible surface measured in centimeters||Electromagnetic||Mapping the environment|
|Standard atmospheric pressure||101,325 Pa|
|Universal gas constant||8.31447 J/(mol·K)|
|Sea level standard temperature||288.15 K|
|Molar mass of dry air||0.0289644 kg/mol|