Open access peer-reviewed chapter

Assistive Systems for the Visually Impaired Based on Image Processing

Written By

Hotaka Takizawa and Mayumi Aoyagi

Submitted: 09 November 2016 Reviewed: 23 August 2017 Published: 20 December 2017

DOI: 10.5772/intechopen.70679

From the Edited Volume

Causes and Coping with Visual Impairment and Blindness

Edited by Shimon Rumelt

Chapter metrics overview

1,474 Chapter Downloads

View Full Metrics

Abstract

In this chapter, we proposed three assistive systems for visually impaired individuals based on image processing: Kinect cane system, Kinect goggle system, and light checking system. The Kinect cane system can detect obstacles of various sizes and also recognize objects such as seats. A visually impaired user is notified of the results of detection and recognition by means of vibration feedback. The Kinect goggle system is another type of wearable system, and can make user’s hands free. The light checking system is implemented as an application for a smartphone, and can tell a visually impaired user the ON/OFF states of room lights and elevator button lights. The experimental results demonstrate that the proposed systems are effective in helping visually impaired individuals in everyday environments.

Keywords

  • assistive system
  • Kinect
  • cane
  • goggle
  • smartphone
  • camera
  • image processing
  • obstacle detection
  • object recognition
  • light checking

1. Introduction

The world health organization estimated the number of visually impaired individuals to be approximately 285 million in 2014 [1]. Many of them use white canes to detect obstacles around them. However, the detectable ranges of white canes are very short. Guide dogs are also used to navigate visually impaired individuals to their destinations while avoiding obstacles. However, it is difficult to provide the sufficient number of guide dogs due to long-time periods and expenses to train them. In order to overcome these problems, extensive research has been dedicated to creating assistive systems for the visually impaired [2].

Obstacle detection is one of the representative research themes. Many research groups have proposed obstacle detection systems based on laser sensors [3, 4, 5, 6, 7, 8, 9, 10, 11], single charge-coupled devices cameras [12, 13, 14, 15, 16], ultrasonic sensors [17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29], stereoscopic cameras [30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41], or RGB-D cameras [42, 43, 44, 45, 46]. These assistive systems are built on the basis of the concept of the electronic travel aid (ETA) [47, 48], which aims to assist visually impaired users in walking while avoiding obstacles. Therefore, these systems can notify the users about obstacles but cannot tell them what kind of objects they are.

Here, let us consider a situation where there is a seat (bench) in front of a visually impaired user as shown in Figure 1, and the user wants to sit on the seat to take a rest. In this situation, the seat is not just any obstacle, but a useful equipment. If the user uses one of the obstacle detection systems mentioned earlier (see Figure 1(a)), he or she is required to confirm the obstacle by himself or herself. However, if the user uses an object recognition system, which can determine the object to be a seat (Figure 1(b)), the user can obtain a benefit. It is necessary to build an assistive system to recognize objects around a visually impaired user.

Figure 1.

Obstacle detection system (a) versus object recognition system for the visually impaired (b).

Several research groups have proposed object recognition systems. Drug packages [49], classroom doors [50, 51, 52, 53], podiums [50], and pathways [54, 55] are recognized by using barcodes [49, 56], augmented reality markers [50, 52, 53, 54], radio frequency identification tags [23, 51, 57, 58], Bluetooth devices [59], wireless network devices [55], or visible light communication devices [60, 61]. These physical devices are useful, but it is difficult to deploy such devices in everyday environment.

Other research groups have also proposed assistive systems to notify visually impaired users about tables [62], color blocks [63], and staircases [64, 65, 66, 67] by means of laser range sensors [65, 66, 67] and Kinect sensors [62, 63]. These systems are useful, but are not sufficient yet. Other types of object should be recognized to help visually impaired individuals more.

This chapter proposes our assistive systems not only to detect obstacles of various sizes but also to recognize objects of various types by use of image processing technique.

Advertisement

2. Kinect cane system

Figure 2 shows our Kinect cane system composed of a white cane and a backpack [68]. A Kinect sensor, a numeric keypad, and a tactile device are attached to the white cane. Kinect is an infrared-based range sensor for a consumer game machine, that is, Microsoft Xbox, and the white cane is also a commercial product for the visually impaired. (The sensor and the cane are approximately 300 and 100 USD, respectively.) The Kinect sensor is set at 75 cm from the floor. These devices are connected by wires with a portable personal computer and a UPS battery in the backpack. The computer and the UPS battery are used for device controlling and power supply, respectively.

Figure 2.

Our Kinect cane system.

In this system, the X, Y, and Z axes of the world coordinate system are defined to be the horizontal vector oriented from left to right, the vertical vector oriented from top to bottom, and the horizontal vector extending from a Kinect sensor into the environment, respectively.

The Kinect cane system can detect obstacles and recognize several objects, such as seats, by use of our special computer programs implemented for the following methods.

2.1. Obstacle detection

The Kinect cane system can detect obstacles which would prevent a visually impaired user to walk safely [45]. We provide two detection methods of small and large obstacles considering the property of a Kinect sensor. These two methods are simultaneously executed in the obstacle detection mode of the Kinect cane system, and if one of these methods detects obstacles, the tactile device returns vibration feedback to the user.

2.1.1. Detection of small obstacles

Figure 3(a) shows an image of a corridor scene including a small obstacle (i.e., a box with a height of 7 cm) on a floor. Figure 3(b) shows the depth data of the corridor scene. The scene image and depth data were obtained by the Kinect sensor. In the depth data, the distances from the Kinect sensor to points on object surfaces are coded. Black pixels represent positions where the sensor cannot measure the distances. The colored pixels (except black ones) are called edges in this chapter.

Figure 3.

A small obstacle on a floor in a corridor scene. (a) Color image; (b) depth data; (c) profile.

In order to detect a small obstacle on a floor, the method uses a measurement line, which is a vertical line at the center of depth data as shown in Figure 3(b). The measurement line is projected onto the floor plane, and the projected line is called a floor line. Figure 3(c) shows the profile of the depth data along the floor line. Black dots represent edges on the Z-Y plane.

The floor line is formulated by

y=az+b,E1

where a and b are coefficients. Let (zi, yi) denote the coordinate of the i-th edge (i = {1, 2, ⋯, I}). The line is fitted to the edges by minimizing the following sum of the squared distances:

S=i=1Idi2,E2

where

di=azi+byi.E3

The optimal values of a and b are obtained by using a robust estimation [69] as follows:

ab=wizi2wiziwiziwiwiziyiwiyi,E4

where weight values are defined as

wi=di1+12di2.E5

The equation

y=az+bE6

represents the optimal floor line. The edges above the optimal floor line are determined to be the obstacle.

2.1.2. Detection of large obstacles

The detectable range of a Kinect sensor is from approximately 40 to 600 cm. In many cases, the upper limitation, 600 cm, is sufficient for obstacle detection. However, the lower limitation, 40 cm, may cause problems that visually impaired individuals collide with obstacles in front of them. In this section, we propose a detection method of large obstacles not only in the detectable ranges but also nearer than the lower limitation.

In the method, three small circular windows, called obstacle measurement (OM) spots, are set on depth data as shown in Figure 4. They are arranged horizontally with a certain interval. The positions and interval are determined considering the height and width of the body of a user. Another small circular window, called a floor measurement (FM) spot, is set on the bottom area. The OM and FM spots are represented as SO1, SO2, SO3, and SF, respectively.

Figure 4.

A large obstacle (pillar) in a building. (a) Color image; (b) depth data.

If each measurement spot includes enough number of edges, the spot is defined to be detected, and the mean depth value is calculated from the depth values of the edges in the detected spot.

The system determines the distance between a Kinect sensor and a large obstacle on the basis of their relation as follows (see Figure 5):

Figure 5.

Relation between a Kinect sensor and a large obstacle.

Case 1: If the obstacle is nearer than 40 cm, all the spots would not be detected. In this case, the system determines the distance to be less than 40 cm.

Case 2: If the obstacle is between 40 and 600 cm, at least one of the OM spots would be detected, and the FM spot would be detected as well. The system outputs the minimum value among the mean depth values of the detected OM spots.

Case 3: If the obstacle is further than 600 cm, all the OM spots would not be detected, whereas the FM spot would be detected. The system determines the distance to be more than 600 cm.

The pillar was successfully detected in Figure 4.

2.2. Object recognition

The Kinect cane system can recognize several objects from depth data. The recognition methods and results are described below.

2.2.1. Planes

Artificial environments are generally composed of many planes, such as floors and walls, and therefore, planes can be effective clues to recognize the environments. Figure 6(a) and (b) shows an example scene and its depth data, respectively. Planes are recognized [70] by using the following method based on random sample consensus (RANSAC) algorithm [71]:

  1. Three edges are randomly chosen from edges in depth data, and then a plane is fit to the chosen edges by use of the least-square method. The three pink points in Figure 6(c) and (d) are the randomly-chosen edges, and the blue regions in Figure 6(e) and (f) are the planes.

  2. The method determines edges whose distances to the plane are nearer than a threshold. These edges are called inliers.

  3. Steps (1) and (2) are iterated a certain number of times.

  4. The plane with the most inliers is selected. Figure 6(g) and (h) shows the selected plane, which corresponds to the floor. The inlier edges are eliminated.

  5. Steps from (1) to (4) are iterated until the number of the remaining edges is less than a threshold.

Figure 6.

Processes of recognition of planes (a)-(h).

In Figure 7, the floor and the two walls were recognized correctly. Black pixels are the remaining edges, which are used for seat recognition described below.

Figure 7.

Recognition result of planes.

2.2.2. Seats

The sitting surfaces of seats (such as chairs, stools, and benches) are considered to be the most essential parts of the seats and are recognized as regions that satisfy the following conditions:

  1. Candidate regions are composed of the remaining edges which are between 30 and 50 cm from the floor.

  2. The areas of the candidate regions are more than 1200 cm 2.

Figure 8 shows the recognition result of the seat in Figure 6(a). Red pixels represent the sitting surface of the seat. Figure 9 shows the color image, depth data, and recognition result of other seats. All the seats were recognized correctly.

Figure 8.

Recognition result of a seat.

Figure 9.

Recognition result of other seats. (a) Color image; (b) depth data; (c) recognition result.

The seat recognition method was applied to 62 sample scenes including seats, and 88% of the scenes were recognized correctly.

It is difficult for this method to recognize seats that have nonparallel sitting surfaces, but there would be not so many such seats in general environment.

2.2.3. Other objects

The Kinect cane system can also recognize upward staircases, downward staircases, and elevators on the basis of the recognition results of planes. The recognition methods are described in detail, for example, in [68, 70], and, in this section, the recognition results are shown in Figures 1012.

Figure 10.

Recognition result of upward staircases. (a) Color image; (b) recognition result.

Figure 11.

Recognition result of downward staircases. (a) Color image; (b) recognition result.

Figure 12.

Recognition result of an elevator. (a) Color image; (b) recognition result.

It is difficult to recognize upward staircases composed of only one or two steps, slopes, and elevators with sealed doors. The Kinect cane system is not designed to detect holes. A user can detect holes by using the system as a conventional white cane.

2.3. User interaction

Ordinarily, a visually impaired user can use the Kinect cane system as a conventional white cane. Figure 13(a) shows an example situation where a user walks in an elevator hall. The user has been here several times, and therefore the user knows there is a bench in the hall, but does not know (or forgot) its accurate location. The user stops walking for safety and then instructs the system to find the bench (seat) as shown in Figure 13(b). The user executes the seat recognition program by pushing the corresponding key on the numeric keypad. The user makes the system search for the bench (Figure 13(c)). If the sensor finds the bench, the tactile device returns vibration feedback to the user (Figure 13(d)). The user walks toward the bench (Figure 13(e)), and then confirms it (Figure 13(f)). Finally, the user can sit on the bench (Figure 13(g)).

Figure 13.

A visually impaired user wants to sit on a bench to take a rest in an elevator hall. (a) A visually impaired user comes out of an elevator. (b) The user instructs the system to find a bench. (c) The user pans the Kinect sensor. (d) The system finds the bench. (e) The user walks toward the bench. (f) The user confirms the bench. (g) The user can sit on the bench.

Advertisement

3. Kinect goggle system

This section introduces another type of Kinect-based wearable assistive system, a Kinect goggle system [72] (Figure 14). A Kinect sensor is attached to a goggle on the face of a visually impaired user. A notebook computer, a numeric keypad, a tactile device, and a UPS battery are set in a shoulder bag. These devices are connected with wires for device controlling and power supply. The Kinect goggle system does not require a visually impaired user to hold a heavy Kinect sensor, and therefore the user can make his or her hands free.

Figure 14.

Our Kinect goggle system.

The current system can detect obstacles and recognize seats by use of the software which is almost the same as those of the Kinect cane system. Figure 15 shows an example scene including a bench. The red region in Figure 15(c) represents the sitting surface of the bench.

Figure 15.

Recognition result of a seat by the Kinect goggle system. (a) A bench; (b) depth data; (c) recognition result.

Advertisement

4. Smartphone-based light checking system

Currently, many visually impaired individuals become to use smartphones as well as sighted people, and therefore it is useful to build assistive systems based on smartphones. In this section, we propose a notification system of the ON/OFF states of room lights and elevator button lights by use of a smartphone camera [73].

4.1. Room light

It is said that many visually impaired individuals often use room lights for the prevention of crime, and so on. They cannot confirm (or have difficulties in confirming) the ON/OFF states by seeing the lights, but can change the states by using the switches of the lights.

There are two types of room light switch. Ones are switches that change their shapes according to the ON/OFF states. The others are switches that do not change their shapes. Figures 16 and 17 show examples of the former and latter switches, respectively.

Figure 16.

Seesaw-type switch. (a) ON; (b) OFF.

Figure 17.

Push-type switch. (a) ON; (b) OFF.

Visually impaired individuals can confirm the ON/OFF states if the switches are the former types, but cannot if the latter. It is necessary to help visually impaired individuals know whether room lights are ON or OFF. This section proposes a notification system of the ON/OFF states of room lights based on interaction between a visually impaired user, a smartphone, a room light, and its switch.

First, a visually impaired user turns a smartphone camera toward a room light and then changes the switch of the light as shown in Figure 18. The system determines the luminosity change of the light by analyzing the intensity histogram and the exchangeable image file format (EXIF) information of an image obtained by the camera. If the light becomes brighter and then darker, the smartphone system will tell the user that the room light turns ON and OFF, respectively. If the user fails to set the smartphone camera, the system returns nothing. Therefore, the user can be aware of the failure and do again after resetting the camera.

Figure 18.

A visually impaired user tries to confirm the ON/OFF state of a room light.

In a lecture room shown in Figure 19, a user used the proposed system and was able to correctly determine that the room lights were turned off.

Figure 19.

Recognition result of the state of room lights in a lecture room. (a) ON; (b) OFF.

4.2. Elevator button light

The notification system can be also used to confirm the arrival floor of an elevator cage. The light of a floor button on a control panel in an elevator cage often turns off when the cage arrives at the corresponding floor. By using the system, a visually impaired user can know whether the cage arrives at a desired floor as described below.

First, a visually impaired user sets a smartphone camera toward the button of a desired floor as shown in Figure 20 and executes the light notification program. When the cage arrives at the floor, the light of the button will be turned off, and the system tells the user that the light is turned off. By hearing the message, the user can know the current floor.

Figure 20.

A visually impaired user tries to confirm the current floor.

In Figure 21, a user can correctly determine that he arrived at the ninth floor.

Figure 21.

Recognition result of the state of an elevator button light. (a) ON; (b) OFF.

Advertisement

5. Conclusion

In this chapter, we proposed three assistive systems for visually impaired individuals based on image processing. The Kinect cane system can detect obstacles of various sizes and also recognize objects such as floors, walls, seats, upward staircases, downward staircases, and elevators. The detection and recognition results are notified to a visually impaired user by means of vibration feedback. The Kinect goggle system is another type of wearable system and can make user’s hands free. The system can also detect obstacles and recognize objects. The smartphone-based light checking system can inform a visually impaired user about the ON/OFF states of room lights and elevator button lights. The user can confirm light states and arrival floors. The experimental results demonstrate that the proposed systems are effective in helping visually impaired individuals in everyday environments.

Advertisement

Acknowledgments

The authors wish to acknowledge Dr. Nobuo Ezaki, Professor of National Institute of Technology, Toba College and Dr. Shinji Mizuno, Professor of Aichi Institute of Technology, for advice on design of this study.

This work was supported in part by the JSPS KAKENHI Grant Number 16K01536.

References

  1. 1. WHO. World health organization, media centre, visual impairment and blindness, fact sheet no. 282. 2014. Available from: http://www.who.int/mediacentre/factsheets/fs282/en/ [Online; Accessed: August 1, 2014]
  2. 2. Dakopoulos D, Bourbakis NG. Wearable obstacle avoidance electronic travel aids for blind: A survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews. Jan 2010;40(1):25-35
  3. 3. Benjamin JM, Ali NA, Schepis AF. A laser cane for the blind. In: . Proceedings of the San Diego Biomedical Symposium. Vol. 12; 1973. p. 53-57
  4. 4. Lin Q, Han Y. A context-aware-based audio guidance system for blind people using a multimodal profile model. Sensors. 2014;14(10):18670-18700
  5. 5. Benjamin Jr JM, M.S.E.E. The laser cane. Journal of Rehabilitation Research & Development. 1974;BPR 10-22:443-450
  6. 6. Saegusa S, Yasuda Y, Uratani Y, Tanaka E, Makino T, Chang J-Y. Development of a guide-dog robot: Human-robot interface considering walking conditions for a visually handicapped person. Microsystem Technologies. 2010;17(5–7):1169-1174
  7. 7. Bolgiano DR, Meeks Jr. ED. A laser cane for the blind. IEEE Journal of Quantum Electronics. 1967;3(6):268
  8. 8. Imadu A, Kawai T, Takada Y, Tajiri T. Walking guide interface mechanism and navigation system for the visually impaired. In: . Proceedings of the 4th International Conference on Human System Interactions, Keio University Yokohama, Japan; 2011. p. 34-39
  9. 9. Gomez JV, Sandnes FE. Roboguidedog: Guiding blind users through physical environments with laser range scanners. Procedia Computer Science. 2012;14:218-225
  10. 10. Vera P, Zenteno D, Salas J. A smartphone-based virtual white cane. Pattern Analysis and Applications. 2014;17(3):623-632
  11. 11. Engelbrektsson P, Marianne Karlsson IC, Gallagher B, Hunter H, Petrie H, O’Neill A-M. Developing a navigation aid for the frail and visually impaired. Universal Access in the Information Society. 2004;3(3):194-201
  12. 12. Peng E, Peursum P, Li L, Venkatesh S. A smartphone-based obstacle sensor for the visually impaired. In: Yu Z, Liscano R, Chen G, Zhang D, Zhou X, editors. Ubiquitous Intelligence and Computing, Lecture Notes in Computer Science. Vol. 6406. Berlin Heidelberg: Springer; 2010. p. 590-604
  13. 13. Takahisa Kishino, Sun Zhe, and Micheletto Ruggero. A fast and precise hog-adaboost based visual support system capable to recognize pedestrian and estimate their distance. In: ICIAP Workshops, pp. 20-29, 2013
  14. 14. Tapu R, Mocanu B, Bursuc A, Zaharia T. A smartphone-based obstacle detection and classification system for assisting visually impaired people. In: . IEEE International Conference on Computer Vision Workshops (ICCVW); 2013. p. 444-451
  15. 15. Praveen RG, Paily RP. Blind navigation assistance for visually impaired based on local depth hypothesis from a single image. In: . Procedia Engineering, International Conference on Design and Manufacturing (IConDM2013). IIITDM Kancheepuram, Chennai, India; Elsevier. Vol. 64; 2013. p. 351-360
  16. 16. Caldini A, Fanfani M, Colombo C. Smartphone-based obstacle detection for the visually impaired. In: . Image Analysis and Processing — ICIAP 18th International Conference, Genoa, Italy, September 7–11, 2015, Proceedings, Part I. Cham: Springer International Publishing; 2015. p. 480, 2015-488
  17. 17. Pressey N. Mowat sensor. Focus. 1977;11(3):35-39
  18. 18. Dambhare S, Sakhare A. Smart stick for blind: Obstacle detection, artificial vision and real-time assistance via gps. In: . IJCA Proceedings on 2nd National Conference on Information and Communication Technology, NCICT. Vol. 6. New York, USA: Foundation of Computer Science; Nov 2011. p. 31-33
  19. 19. Shoval S, Borenstein J, Koren Y. The navbelt – A computerized travel aid for the blind based on mobile robotics technology. IEEE Transactions on Biomedical Engineering. 1998;45(11):1376-1386
  20. 20. Ulrich I, Borenstein J. The guidecane - applying mobile robot technologies to assist the visually impaired. IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans. 2001;31:131-136
  21. 21. Cardin S, Thalmann D, Vexo F. A wearable system for mobility improvement of visually impaired people. The Visual Computer. 2007;23(2):109-118
  22. 22. Okayasu M. Newly developed walking apparatus for identification of obstructions by visually impaired people. Journal of Mechanical Science and Technology. 2010;24(6):1261-1264
  23. 23. Rohan P, Ankush G, Vaibhav S, Dheeraj M, Balakrishnan M, Kolin P, Dipendra M. Smart cane for the visually impaired: Technological solutions for detecting knee-above obstacles and accessing public buses. In: . 11th International Conference on Mobility and Transport for Elderly and Disabled Persons (TRANSED 2007), Montreal, Canada; 2007
  24. 24. Wahab MHA, Talib AA, Kadir HA, Johari A, Noraziah A, Sidek RM, Mutalib AA. Smart cane: Assistive cane for visually-impaired people. International Journal of Computer Science Issues. 2011;8(4–2):21-27
  25. 25. Bharathi S, Ramesh A, Vivek S, Vinoth Kumar J. Effective navigation for visually impaired by wearable obstacle avoidance system. International Journal of Power, Control Signal Computation. 2012;3(1):51-53
  26. 26. Morrissette DL, Goodrich GL, Hennessey JJ. A follow-up study of the mowat sensor’s applications, frequency of use, and maintenance reliability. Journal of Visual Impairment and Blindness. 1981;75:244-247
  27. 27. Bahadir SK, Koncar V, Kalaoglu F. Wearable obstacle detection system fully integrated to textile structures for visually impaired people. Sensors and Actuators A: Physical. 2012;179:297-311
  28. 28. Mahmud MH, Saha R, Islam S. Smart walking stick – An electronic approach to assist visually disabled persons. International Journal of Scientific & Engineering Research. 2013;4(10):111-114
  29. 29. Geoff C. Mowat, Wormald International Sensory Aids Ltd., Wormald International Sensory Aids, 6140 Horseshoe Bar Rd., Loomis, CA 95650, USA.1973
  30. 30. Molton N, Se S, Brady JM, Lee D, Probert P. A stereo vision-based aid for the visually impaired. Image and Vision Computing. 1998;16:251-263
  31. 31. Saito T, Takizawa H, Yamamoto S. A display system of obstacle positions for visible disabled persons. In: . Proceedings of the IEICE General Conference. The Institute of Electronics, Information and Communication Engineers, Waseda University, Japan. Vol. 2; 2002. p. 316
  32. 32. Zelek J, Audette R, Balthazaar J, Dunk C. A stereo-vision system for the visually impaired. Technical report. University of Guelph; 2000
  33. 33. Ikarashi M, Yokote H, Takizawa H, Yamamoto S. Walking support system using stereo data for blind person. In: . Proceedings of the IEICE General Conference. The Institute of Electronics, Information and Communication Engineers, Hiroshima University, Japan. Vol. 2; 2000. p. 337
  34. 34. Kawai Y, Tomita F. A supporting system for visually impaired persons to understand three-dimensional visual information using acoustic interface. In: . Proceedings of the 16th International Conference on Pattern Recognition. The International Association for Pattern Recognition, Quebec, Canada. Vol. 3; 2002. p. 974-977
  35. 35. Balakrishnan G, Sainarayanan G, Nagarajan R, Yaacob S. Wearable real-time stereo vision for the visually impaired. Engineering Letters. 2007;14(2):1-9
  36. 36. Velzquez R, Maingreaud F, Pissaloux EE. Intelligent glasses: A new man-machine interface concept integrating computer vision and human tactile perception. In: . Proceedings of EuroHaptics 2003. The Interaction Design Foundation, Dublin, Ireland; 2003. p. 456-460
  37. 37. Balakrishnan G, Sainarayanan G, Nagarajan R, Sazali Yaacob A. stereo image processing system for visually impaired. World Academy of Science, Engineering and Technology. 2006;20:206-215
  38. 38. Kim D, Kim K, Lee S. Stereo camera based virtual cane system with identifiable distance tactile feedback for the blind. Sensors (Basel). 2014;14(6):10412-10431
  39. 39. Meers S, Ward K. Substitute three-dimensional perception using depth and colour sensors. In: . The 2007 Australasian Conference on Robotics and Automation. The Australian Robotics and Automation Association, Brisbane, Australia; 2007. p. 1-5
  40. 40. Dunai L, Fajarnes GP, Praderas VS, Garcia BD, Lengua IL. Real-time assistance prototype – A new navigation aid for blind people. In: . IECON 2010 – 36th Annual Conference on IEEE Industrial Electronics Society. IEEE, Renaissance Hotel & Spa Glendale, AZ, USA; 2010. p. 1173-1178
  41. 41. Kitagawa A, Takizawa H, Aoyagi M, Ezaki N, Shinji M. A preliminary study on detection of obstacles and recognition of chairs and tables by use of a stereocamera cane system. In: . IEICE Technical Report (WIT2014-121). Vol. 114; 2015. p. 203-208
  42. 42. Salerno M, Re M, Cristini A, Susi G, Bertola M, Daddario E, Capobianco F. Audinect: An aid for the autonomous navigation of visually impaired people based on virtual interface. International Journal of Human Computer Interaction. 2013;4(1):25-33
  43. 43. Bernabei D, Ganovelli F, Di Benedetto M, Dellepiane M, Scopigno R. A low-cost time-critical obstacle avoidance system for the visually impaired. In: . International Conference on Indoor Positioning and Indoor Navigation. IEEE, Centro Cultural Vila Flor Guimaraes, Portugal; 2011. p. 21-23
  44. 44. Lee YH, Medioni G. Rgb-d camera based navigation for the visually impaired. In: . RSS 2011 RGB-D: Advanced Reasoning with Depth Camera Workshop; 2011. p. 1-6
  45. 45. Orita K, Takizawa H, Aoyagi M, Ezaki N, Shinji M. Obstacle detection by the Kinect cane system for the visually impaired. In: . Proceedings of the 2013 IEEE/SICE International Symposium on System Integration. IEEE, Kobe International Conference Center, Kobe, Japan. Vol. 1; 2013. p. 115-118
  46. 46. Khan A, Moideen F, Lopez J, Khoo WL, Zhu Z. Kindectect: Kinect detecting objects. In: . 13th International Conference on Computers Helping People with Special Needs, LNCS 7383. Springer, University of Linz, Austria. Vol. II; 2012. p. 588-595
  47. 47. Working Group on Mobility Aids for the Visually Impaired and Blind, Committee on Vision. Electronic Travel Aids: New Directions for Research. The National Academies Press. The National Academies Press. Washington, D.C. 20001; 1986
  48. 48. Hersh MA, Johnson MA. Assistive Technology for Visually Impaired and Blind People. Springer; 2008
  49. 49. Lee HP, Sheu T-F. Building a portable talking medicine reminder for visually impaired persons. In: . The Sixth International Conference on Future Computational Technologies and Applications. IARIA, Venice, Italy; 2014. p. 13-14
  50. 50. Manduchi R, Coughlan J, Ivanchenko V. Search strategies of visually impaired persons using a camera phone wayfinding system. In: . Computers Helping People with Special Needs, Lecture Notes in Computer Science Volume 5105. Springer, Linz, Austria; 2008. p. 1135-1140
  51. 51. Kulyukin V, Gharpure C, Nicholson J. Rfid in robot-assisted indoor navigation for the visually impaired. In: . Proceedings of the 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, Sendai, Japan; 2004. p. 1979-1984
  52. 52. Michael Zöllner, Stephan Huber, Hans-Christian Jetter, and Harald Reiterer. Navi – A proof-of-concept of a mobile navigational aid for visually impaired based on the microsoft Kinect. In INTERACT 2011, 13th IFIP TC13 Conference on Human-Computer Interaction, Vol. IV Lecture Notes in Computer Science Volume 6949. Springer, Lisbon, Portugal, pp. 584–587, 2011
  53. 53. Halabi O, Al-Ansari M, Halwani Y, Al-Mesaifri F, Al-Shaabi R. Navigation aid for blind people using depth information and augmented reality technology. In: . The proceedings of NICOGRAPH International 2012. The Society for Art and Science, Hotel Santika Premiere Beach Resort Bali, Indonesia; 2012. p. 120-125
  54. 54. Fernandes H, Costa P, Filipe VM, Hadjileontiadis L, Barroso J. Stereo vision in blind navigation assistance. In: . World Automation Congress (WAC). IEEE, Kobe Convention Center Kobe, Japan; 2010
  55. 55. Anthea Wain Sy A, Feng C, Valaee S, Reyes S, Sorour S, Markowitz SN, Gold D, Gordon K, Eizenman M. Indoor tracking and navigation using received signal strength and compressive sensing on a mobile device. IEEE Transactions on Mobile Computing. 2013;12(10):2050-2062
  56. 56. Kulyukin V, Nicholson J, Coster D. Shoptalk: Toward independent shopping by people with visual impairments. In: . Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility, Assets ’08. New York, NY, USA: ACM; 2008. p. 241-242
  57. 57. Debnath N, Hailani ZA, Jamaludin S, Aljunid SAK. An electronically guided walking stick for the blind. In: . Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, Istanbul, Turkey. Vol. 2; 2001. p. 1377-1379
  58. 58. Tatsumi H, Murai Y, Miyakawa M. Rfid for aiding the visually impaired recognize surroundings. In: . IEEE International Conference on Systems, Man and Cybernetics. IEEE, Delta Centre-ville Hotel Montreal, QC, Canada; Oct 2007. p. 3719-3724
  59. 59. Markiewicz M, Skomorowski M. Public transport information system for visually impaired and blind people. In: Mikulski J, editor. Transport Systems Telematics, Communications in Computer and Information Science. Vol. 104. Berlin Heidelberg: Springer; 2010. p. 271-277
  60. 60. Nakajima M, Haruyama S. New indoor navigation system for visually impaired people using visible light communication. EURASIP Journal on Wireless Communications and Networking. 2013;2013(1):1-10
  61. 61. Nakazawa Y, Makino H, Nishimori K, Wakatsuki D, Komagata H. Led-tracking and id-estimation for indoor positioning using visible light communication. In: . Proceedings of Fifth International Conference on Indoor Positioning and Indoor Navigation, BEXCO, Busan, Korea; 2014. p. 1-8
  62. 62. Wang Z, Liu H, Wang X, Qian Y. Segment and Label Indoor Scene Based on RGB-D for the Visually Impaired. Cham: Springer International Publishing; 2014. p. 449-460
  63. 63. Gomez JD, Mohammed S, Bologna G, Pun T. Toward 3d scene understanding via audio-description: Kinect-ipad fusion for the visually impaired. In: . The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility, ASSETS ’11. New York, NY, USA: ACM; 2011. p. 293-294
  64. 64. Filipe V, Fernandes F, Fernandes H, Sousa A, Paredes H, Barroso J. Blind navigation support system based on microsoft Kinect. In: . Proceedings of the 4th International Conference on Software Development for Enhancing Accessibility and Fighting Info-exclusion (DSAI 2012), Douro Region, Portuga; 2012. p. 94-101
  65. 65. Yasumuro Y, Murakami M, Imura M, Kuroda T, Manabe Y, Chihara K. E-cane with situation presumption for the visually impaired. In: . Proceedings of the User interfaces for all 7th international conference on Universal access: theoretical perspectives, practice, and experience, ERCIM’02. Berlin, Heidelberg: Springer-Verlag; 2003. p. 409-421
  66. 66. Ueda T, Kawata H, Tomizawa T, Ohya A, Yuta S. Visual information assist system using 3d sokuiki sensor for blind people, system concept and object detecting experiments. In: . 32nd Annual Conference on IEEE Industrial Electronics, IECON 2006. IEEE, Conservatoire National des Arts & Metiers Paris, France; Nov 2006. p. 3058-3063
  67. 67. Ishiwata K, Sekiguchi M, Fuchida M, Nakamura A. Basic study on step detection system for the visually impaired. In: . 2013 IEEE International Conference on Mechatronics and Automation (ICMA). IEEE, Takamatsu, Kagawa, Japan; Aug 2013. p. 1332-1337
  68. 68. Takizawa H, Yamaguchi S, Aoyagi M, Ezaki N, Mizuno S. Kinect cane: An assistive system for the visually impaired based on the concept of object recognition aid. Personal and Ubiquitous Computing. 2015;19(5–6):955-965
  69. 69. Press WH, Flannery BP, Teukolsky SA, Vetterling WT. Numerical Recipes in C. Cambridge University Press; 1988
  70. 70. Kuramochi Y, Takizawa H, Aoyagi M, Ezaki N, Shinji M. Recognition of elevators with the Kinect cane system for the visually impaired. In: . Proceeding of 2014 IEEE/SICE International Symposium on System Integration. IEEE, Korakuen Campus, Chuo University, Tokyo, Japan. Vol. 1; 2014. p. 128-131
  71. 71. Fischler MA, Bolles RC. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM. Jun 1981;24(6):381-395
  72. 72. China LH, Takizawa H. A preliminary study on object recognition and obstacle detection using a Kinect goggle system for the visually impaired. In: . Visual / Media Computing Conference 2016. The Institute of Image Electronics Engineers of Japan, Waseda University, Tokyo, Japan. Vol. 1; 2016. p. 1-4
  73. 73. Nakamura D, Takizawa H, Aoyagi M, Ezaki N, Mizuno S. A preliminary study on image-based recognition of on/off of room lights and elevator buttons for the visually impaired. In: . Interdisciplinary Workshop on Science and Patents (IWP) 2016. Vol. 1; 2016. p. 1-4

Written By

Hotaka Takizawa and Mayumi Aoyagi

Submitted: 09 November 2016 Reviewed: 23 August 2017 Published: 20 December 2017