Smart Brain Interaction Systems for Office Access and Control in Smart City Context

Over the past decade, the term “smart cities” has been worldwide priority for city plan‐ ning by governments. Planning smart cities implies identifying key drivers for trans‐ forming into more convenient, comfortable, and safer life. This requires equipping the cities with appropriate smart technologies and infrastructure. Smart infrastructure is a key component in planning smart cities: smart places, transportation, health and educa‐ tion systems. Smart offices present the concept of workplaces that respond to user’s needs and allow less commitment to routine tasks. Smart offices solutions enable employees to change status of the surrounding environment upon the change of user’s preferences using the changes in the user’s biometrics measures. Meanwhile, smart office access and control through brain signals is quite recent concept. Hence, smart offices provide access and services availability at each moment using smart personal identification (PI) interfaces that responds only to the personal thoughts/preferences issued by the office employee not any other person. Hence, authentication and control systems could benefit from the biometrics. Yet these systems are facing efficiency and accessibility challenges in terms of unimodality. This chapter addresses those problems and proposes a prototype for multimodal biometric person identification control system for smart office access and control as a solution.


Introduction
Building smart office systems implies building a system that recognizes an employee and interacts with employees through reading their brain signals and interpreting their brain activities and signals patterns to control their offices. The control involves controlling the light brightness off/low intensity/high intensity and temperature increase/decrease, chair height or back angle, curtains up/down, and doors lock/unlock status. Concerning the infrastructure required for building smart offices, it is worthy as it enhances accommodating important category of people with disabilities and improves their employability. For ordinary people, it will provide flexible working environment by adding comfort and fun to the workspace [1].
The smart office perceives intentions and responds to their intended needs by actuating the environment [2]. Hence, designing smart offices involves sensory data from reading their brain signals, temperatures, etc., and hence auto responds to their need in terms of controlling office light brightness off/on and temperature increase/decrease, chair height or back angle, and curtains up/down status. This employee-office interaction will save them some time and will increase the work efficiency and effectiveness as well as adding a strong helpful tool to those who have struggles doing such a thing. For ordinary people, it will also add some fun and make offices happy zones by acquiring employee's thought signals, and they will have a flexible working environment. This smart human-office interaction requires up-to-date sensory devices as well as capturing devices to be used for collecting employee's intentions and hence send it to interpretation system for identifying the commands to be executed for access or controlling particular office item.

Literature review
Designing smart offices and smart environment has been reported in the recent research as biometric technologies that deploy human-computer interaction. These technologies fall into either the personal identification or command-based systems. Personal identification systems imply identity recognition such as finger print, eye print, voice print, and palm vein data for personal identification systems. Command-based systems use eye gaze, voice commands, etc. In addition, a most recent research has reported the use of more advanced interaction levels such as the use of brain signals and emotion recognition systems for both personal identification and controlling office devices.
An attempt to develop an intelligent emotion stress recognition system (ESR) using brain signals (EEG) is published in the field of biomedical engineering and sciences for diagnosis of verbal communication problems and treatment of disability in speech and bodies. Eye tracking was also used by disabled to communicate with the outside world [3]. This research investigates the possibility of how to recognize employee's stress emotions using signal processing of electroencéphalographie. ESR suggested new system recognition for émotionnel stress, using multimodal bio-signals using electroencephalogram (EEG) as the main signals, since its use is spread widely in clinical diagnosis and biomedical research. A cognitive model is then used to extract the brain signals from the appropriate EEG channels that represent emotional stress relevant data [4].
Generally speaking, any EEG-based system would go through the following units: signals capturing unit, signals preprocessing/processing unit, classifier, and decision-making unit that translates the classified signal, Figure 1.

Current EEG applications
One of the leading projects in building smart environment was a Smart Environment for offices at University of Stuttgart (Sens-R-Us) application [5]. Sens-R-Us project focused on using graphical interface (GUI) and Mica2 motes sensors that capture the real-world data of their employees. These sensors are static sensors and personal sensors. The base sensors are installed in all rooms such as office and meeting rooms, and they send location beacons with room ID constantly. Personal sensors are carried around by the employees, and they receive location beacons and then select the highest signal base stations. Personal sensors can also send signals to update their information which is used in a constant detection of meeting occurrence. The developed Sens-R-Us application advantages are the use of lower power consumption, small size, possibility to be used off offices. The GUI is used to acquire employee's information and status, room temperature, and available switched on devices in the office. Another application reported by the literature is WSU "Smart Home." WSU is an assistive technology that aims to assist elderly people in performing daily routine tasks home. The "smart home in a box" is about 30 sensors application which detects motion, temperature, and power sensors that are easy to install. The system provides functionalities such as monitoring and learning the elderly routines, recording changes when arise and remind elderly if they forgot something [6].

EEG authentication applications
Over the past decade, unimodal authentication systems occupied the top place in many fields, for example, finger prints in students/employees attendance system, eye print/face  recognition in the airports, etc. Most of the demonstrated problems in unimodal biometric systems are noisy data such as scars in the skin of fingerprints, defects in the capturing sensor, limited number of degrees of freedom that result in feature similarities with large population, recording the voice and use it to get access to voice recognition systems. Also, they justified using multimodal biometric system as a solution since the main cause of efficiency problems imposed by unimodal biometric systems is the reliance on the evidence of one source of information.
Ross and Jain [7] introduced smart office access system based on multimodal biometric technologies. They developed multimodal biometric system taking into consideration appropriate fusion for output of different modalities, strategies to integrate the models [7]. There are some examples for the development of biometric multimodal systems such as face recognition and fingerprint multimodal biometric authentication system by Rahal et al. [8]; Face recognition and speech-based multimodal biometric authentication system by Soltane et al. [9], Al-Hudhud et al. [10]; and speech, signature, and handwriting features authentication system by Eshwarappa [11]. Other research is concerned with a new biometric model known as electroencephalography or EEG, and it is a type of wave signals produced by the brain, mostly used in applications related to brain health/research. Researchers have suggested that the EEG has a potential as a powerful authentication model since some features that are extracted from the EEG signals are unique from one person to another [12][13][14][15]. A survey conducted by Khalifa et al. [12] presents several methods used in EEG authentication, please refer to Table 1.
Unlike "what the user have" authentication such as iris and fingerprint or "what the user knows" such as password variants of authentication, Mohanchandra et al. [16] introduced a "what the user is" authentication type as an application that uses real-time EEG signals for locking/unlocking the computer screen. It matches mental task encoded features (MTEF) of the EEG through Euclidean distance measurement with MTEF of current EEG user status. The result has been shown that the system is a reliable system of authentication. Additionally, the results presented a good classification accuracy that, however, needs some improvements [16].
Building EEG-based mobile biometric authentication systems was initiated by Klonovs and Petersen [17]. They proposed a system that uses EEG and NFC tags. Users would choose their  [12] presented the following classification accuracy rate when deploying EEG in authentication including task measure in terms of true acceptance rate (TAR) and false acceptance rate (FAR).
own personal password as an image in the enrolment phase; EEG data will be obtained from the headset from four EEG sensor locations: P7, P8, O1, and O2. In the access time, this image would be shown to the user when authenticating in a five seconds period of time. The authors chose zero crossing rate technique [26,27] and wavelet analysis [34] for feature classification and latencies measurement of visual-evoked potentials, respectively. Potentials, respectively. The result of their work is that they have found that the most significant features can be extracted from the visual parietal-occipital cortex of the brain, and thus, their implementation of presenting the image method can be seen beneficial.

Signals capturing types
In BCI, there are three different methods to get the signals: invasive, partially invasive, and noninvasive. In the invasive method, the BCIs are implemented directly into the gray matter of the brain (usually used to help paralyzed people). However, although this method gives high-quality signals, it still presents some risk to human health. On the other hand, in the partially invasive BCIs, only part of the BCI is implemented inside the skull but not within the brain. However, the signal captured has lower resolution than the invasive method. In addition, it presents lower health risk on the patient. For the noninvasive method, sensors are placed on the scalp and no implanting needed. This method does not present any risk to human health, and it is convenient and easy to use. Additionally, it provides good signal readings [18,19].

Signals acquiring techniques
There are different methods to obtain brain signals. One of them is by electrical means such as electroencephalogram (EEG) where sensors called electrodes are used to acquire the signals. This method has low set up cost and ease of use. However, it is susceptible to noise and requires intensive training before using it. Other methods to acquire the signals are by nonelectric means such as measuring by the magnetic and metabolic changes or even from the pupil size oscillation as developed lately in Ref. [20], all these techniques can be used in a noninvasive manner. The functional magnetic resonance imaging (fMRI) technique uses magnetic to capture the brain activity, and it focuses on measuring the blood oxygenation and flow that is increased in the area of the brain which involves mental activity. Therefore, it requires large devices with a large magnetic field scanner. The functional near-infrared spectroscopy (fNIRS) uses infrared waves to measure the blood oxygenation and flow. However, most of the techniques that depend on measuring the metabolic changes suffer from long latency compared to the EEG technique [18,19].

BCI device types
The term headset is used to describe the capture devices and may include shapes of cap, tiara, headband, helmet, or even loose electrodes. Many commercial headsets have been released to the market with an attractive design and low cost. NeuroSky [21,22] and Emotive EPOC [23] are examples of these devices. Most of these applications improve their performance by using brain signals as an input along with other parameters such as body temperature or pupil size, and it also combines the BCI technology with other technologies such as the virtual reality [18,19].

Thought identification
There are different ways to identify thoughts or mental activities that resulting action such as motor imagery, bio/neurofeedback for passive BCI designs, and visual-evoked potential (VEP). In motor imagery, imagining moving any parts of the body results in sensorimotor cortex activation, which modulates sensorimotor oscillations in the EEG [24]. The second type is the bio/neurofeedback for passive BCI designs, where the relaxation and attention were measured using some bodily parameters along with mental concentration that is measured by monitoring the alpha and beta waves of the brain [18,24]. On the other hand, the VEP captures the brain response to visual stimulus such as certain flashing graphic elements or sound stimulus such as special sound pattern [18,24].

Feedback type
Zander and Kothe [35] introduced the categories of BCI approaches as follows: • Active BCI: independent of external events, useful for controlling an application [25].
• Reactive BCI: arising in reaction to external stimulation. It is indirectly modulated by the user for controlling an application [25,33].
• Passive BCI: It refers to the brain activities that are integrated to produce an input. The integrated input applies mental state BCI in which the user does not try to control his brain activity [23].
In addition, there are other classifications for BCI approach that depend on the processing and rhythm types. There are two types of BCI processing: online which happens while the user utilizes the BCI and offline which occurs after experiment. On the other hand, the rhythm's classification is divided into two types: synchronous where the commands are processed after every certain amount of time and asynchronous where processing the commands will be upon the request [18].

Application area
The development of the BCI technology makes it no longer used only at laboratories but anywhere (home, offices, etc.) since it became a portable device. Therefore, its applications are also growing and becoming more different and advanced to many areas such as communication and control, motor substitution, entertainment, motor recovery, and mental state monitoring [19].

Group of BCI Beneficiaries
The basic group of BCI beneficiaries is the disabled patients to help them in their remembering & managing daily tasks and expressing themselves. In addition, BCI has been used in rehabilitation of disorders such as stroke, addiction, autism, ADHD, and emotional disorders [19].
Dues to the development of new applications in fields other than medical field, new groups are emerging recently as BCI technologies main users groups. Among these new emerging fields are authentications & security systems, health applications, controlling games, biometrics, and controlling of smart and virtual environments [19].

Proposed technical solution
The proposed solution for accessing and controlling smart offices system includes two main modules.
First module tackles a multimodal biometric accessibility system that includes electroencephalography (EEG) and face recognition part in addition to a nonbiometric part, known as SMS token. This part describes the feature extraction from the cloud storage of the biometric data and the best multimodal fusion technique for the biometric and nonbiometric combination [31,32].
The other module is the smart office control. This module includes controlling office devices through brain signals. The office devices control would be highly demanded in very busy schedule for workers in terms of saving time to walk away from the desk in order to increase/ decrease the light brightness or the temperature of the office. In addition, it is important to embed infrastructure for cases of people who have major disabilities that prevent easy movements and actions. A mental control for smart workspace module is introduced in this chapter that is based on acquiring brain signals that represent the workers thoughts regarding their feelings of temperature and lighting in their offices. Hence, the brain signals are processed and filtered [28,29] in order to analyze and interpret the feeling in terms of frustration and willing to increase/decrease any of the surrounding status. Mental control system would require a smart working environment that is equipped with sensors and actuators. All these components together with the data collected from brain signals are used to anticipate any need.

The proposed authentication subsystem process units
The core functionalities for the proposed system implementation and overall processing are as follows: (a) Data acquisition unit: EEG signals through an EEG headset and face images through web camera.
(b) Signal & Image preprocessing and filtering unit: for ALL signals and images being acquired.
(c) Feature extraction unit for each model, brain signal, and face images.

Data acquisition unit
Brain signal data are acquired during both enrolment and login phases for each biometric modality. The system administrator defines authorized users and provides them with passwords. Once an authorized user enters his name, the EEG enrolment phase starts by displaying the images representing the password. The user chooses the password image from a photo gallery and then confirms. Hence, the EEG signal capturing instructions is displayed for the user which indicates that the signal will be captured for 5 seconds. During that period, the chosen image will appear and the user should focus on it without blinking and in a relaxed condition for 5 seconds.
The user will repeat the same procedure done in the enrolment phase without the first step (choosing the password image). The following steps take place at the login phase: (a) The user enters his name.
(b) The user is forwarded to the EEG log in page.
(c) Brain signals will be captured only twice; the total time of each recording is 5 seconds.
(d) The system will select the brain signals channels AF3, F3, and F7 located at standard positions of the international 10-20 system.
(e) The raw data representing the captured signals are then written to a CSV file.
(f) The other modality capturing; face recognition, starts. The system will forward the user to face image capturing page, and instructions appear to inform the user to look at the camera.

Preprocessing and filtering
The captured EEG signals are written in CSV, named the raw data. Raw data are noisy, that is, it contains a lot of irrelevant information. A preprocessing step is needed to extract the relevant features. Spatial EEG data are prepared by zero amplification to a power of two in order to be Fourier transform to frequency domain. The next step was to remove the baseline activity from each channel and then calculate the mean of each channel. Hence, the mean of each channel was subtracted from the original values of the channel. The transformed data is filtered with a 5th order sinc filter and band pass filter with frequency range between 0.5 Hz and 60 Hz to notch out 50 Hz and 60 Hz. The sampling rate is 128 Hz. Hence, the system calculates the mean of each channel, and then, the mean of each channel is subtracted from each original value of the channels. Finally, we applied the inverse Fourier transform.

Feature extraction unit
The system initiates EEG feature vectors that are saved temporarily in the runtime memory during the enrolment phase, to extract the following features: • Signals speed (frequency), • Power spectral density, • Magnitude, signal-to-noise ratio, • Variance, mean, and standard deviation for each channel, • Zero crossing rate from each channel.
The features vector is then stored using either local server or a cloud storage. Hence, the following processes and calculations take place: (c) The low pass filter is applied to reject frequency higher than 40 Hz.
(d) The data are then filtered and processed by removing the baseline activity from each channel.
(e) The extracted features are standard deviations (SD) from each channel. The training pattern was five from each subject, and the SD average was calculated and saved as stored features. This was implemented by the authors previously and published [10,30,31,32].

Face recognition modality
During enrolment and login phases, the system prompts the face recognition modality interface that captures the face image. The face image is then converted into gray scale. Haar Cascade classifier is applied to the grayscale image for face recognition. Cropping and resizing (to 1010 × 100) step is done to the face images. The resulted image is encrypted with AS encryption technique and lastly saved in the file system as a training set. By this the will be fully enrolled in the system.
At the login phase, the applies the same steps being performed during the enrolment procedure. The system applies face recognition using principal component analysis (PCA). First, the system will: 1. Retrieve the enrolment features vector.
2. Decrypt the enrolment features vector using AS algorithm.
3. Calculate combination of some component or face basis called the "Eigenface" from the features vector.
4. Face space is calculated by projecting the face images.
5. Euclidean distance between the detected face and each image is calculated from "Eigenface." Smart Brain Interaction Systems for Office Access and Control in Smart City Context http://dx.doi.org/10.5772/65902 6. Recognize if the distance is above the distance threshold. The system temporarily saves the face recognition matching decision.

Classification and decision-making unit
The classification is performed using Euclidean distance, the classifier would perform in a way that the interclass (distance between the groups) was maximized, and the intraclass (distance within the same group) was minimized. The Euclidean distances for three patterns were computed for each subject to result in three thresholds for that subject. Both thresholds are saved for each subject in the enrolment phase. The classification is performed for the brain signals and for the face image using the following classifiers: The classifier computes the threshold for each subject for the input modality by comparing the enrolment and login feature vectors. The average threshold is computed from the five enrolment trials and saved as stored threshold. Authentication is done during the login phase using five patterns from the subject, and the new resulted threshold is averaged from the five patterns. Then, the new average threshold is subtracted from the stored threshold.
Personal identification decision score is produced in the matching process for each input modality. The decision can be either: Accept the subject if the classifiers average thresholds from two classifiers are less than 0.100, or Reject the subject if the classifiers average thresholds from two classifiers are less than 0.100.

Fusion unit
In this stage, the system gets the decision scores of both modalities the brain signals (EEG) and the face recognition. The system then will: 1. Reject access if ALL decision scores are "Reject" 2. Grant access if ALL decision scores are "Accept." However, if the system gets one Accept from one modality with Reject from another modality, then the system uses the SMS token. The SMS token works by sending a system-generated one-use password in the form of SMS to the registered mobile number. The is then redirected to a page where the received password can be entered password to verify the. If the password is correct then the grant access and if not access will be denied; please refer to Figure 2.

Controlling the office devices using EEG signals
This section describes the subsystem that is used for smart offices control through recording the brain signals during the brain activity when thinking of increasing/decreasing the temperature and increasing/decreasing the light intensity. For each activity, brain sensory data are passed to the system so that it can be encoded into a command. The system stores these commands in the form of vector feature associated with the. This subsystem integrates the following: a simulator which will be designed using 3D modeling tools to model the offices, sensors, and devices to be controlled through the brain thoughts, Emotive Headset to read brain signals, and then interfacing tools to integrate and produce the interface (see Figure 3).
In order to build a smart office that allowing employees to control their offices temperature and brightness, this subsystem will integrate physical devices, brain signals coming from the Emotive Headset, and computing entities in offices with the interfacing tools needed to   produce the interface. In addition, interpretation of the thoughts will be translated into a command that will be passed to the actuator of the specified device, Table 2.

Experimentation setup and results
A total of 30 people volunteered to participate in our study. The subject was instructed to put on the Emotive EPOC EEG headset and is asked to follow these steps: 1. The subject will be seated on a normal chair, relaxed arms resting on their legs, and in noise-controlled room.
2. The will be exposed to the GUI of the system.
3. The chooses an image from an image gallery.

4.
The mental task that is to focus on the particular image of a celebrity for 5 s, during which the signals will be captured; a task during which the should concentrate and not moving the body nor blinking.
5. The brain signals are recorded and forwarded to the next step.
6. The subject will be looking at the camera in order to allow the system to capture the face image.

7.
The system then will process both input modalities: EEG and face image in order to produce the feature vectors.
8. The features vector is then compared to the stored feature vector for each participant, and experimentation results are presented in Table 3.
The experiments are designed such that the user will be wearing an Emotive EPOC EEG headset and will be provided with instructions for completing the session. The first instruction

Hardware Software
The 3D models in the offices for the physical appliances (sensors, fan, and light bulb) Arduino set Software to control these physical appliances  Table 3. System performance and summery experimentation results.   will be to ask to do a mental task that is to focus on increasing the temperature for 6 s, during which the signals will be captured. The EEG data were recorded, filtered, and processed as the same way described in the previous section (see Figures 4-7).
The results are interpreted in terms of the false matching rate and true matching rate. False matching rate (FMR) is defined as the percentage of matching false user's thought with the  correct action. The true matching rate (TMR) is defined as the percentage of correct match between the users thought with the correct action. Based on the results being collected, it is found that 26% FMR was reported and 100% TMR for the brain commands was obtained. Both rates are considered excellent but due to the high number of patterns needed from each user (five patterns) each time they use the system, Table 4.

Conclusion
The work presented here investigated two main terms: first, brain signal and what is the perfect way to read the signal and translate it into real action and second, smart offices and its use in real time.
The work focuses primarily on smart access to the office and smart control of the office devices. Hence, a model was proposed in the chapter for reading the thought in the form of brain signal, translating the thought into password for accessing the system and hence creating other control actions in the office. This requires many sensors in the work environment to receive the translated action and apply it.
Regarding the smart accessibility of the office, the work investigated the use of three authentication modalities as a multimodal authentication system to overcome the limitations of unimodal biometric authentication systems. However, according to the experimentation results, the multimodal system has proven to overcome the efficiency, accessibly problems, fusion mechanism for the multiple models, and the immaturity of EEG model in the field of biometric authentication.
Regarding the smart office access, the model focuses primarily on using EEG as an authentication biometric and secondly, on face recognition. In addition, the proposed solution also investigates the multimodal fusion technique that combines all system models (electroencephalography, face recognition, and SMS token). The authors also referred to reported research results to decide on the most suitable channels from the extracted brain signals using the EEG and multimodal.
The major contributions done through this work are the findings of the best features, classifiers, and methods that are suitable for EEG in authentication and control. For this kind of multimodal system, the findings have been shown the best fusion level to present a powerful and efficient multimodal authentication system with accuracy rates of TAR = 90% and FAR = 0%.
The future improvements suggested are as follows: (a) flexibility regarding the EEG signals acquiring device, (b) improving the classifier and thresholding technique to count for the different concentration levels for the same user, and (c) achieving more accuracy in terms of TAR and FAR.
In conclusion, this prototype opens the gate wide in front of new era of internet of things toward a smarter society needs and requirements. Hence, the research could be the milestone for newer inventions and researches and a helpful contribution in the great field of brain computing interaction for authentication systems. Table 5 shows a comparison between Sens-R-Us and the brain signal smart office control functionalities.