Affect Interpretation in Metaphorical and Simile Phenomena and Multithreading Dialogue Context

The detection of complex emotions and value judgments from open-ended text-based multithreaded dialogue and diverse figurative expressions is a challenging but inspiring research topic. In order to explore this line of research, previously we have developed an affect inspired AI agent embedded in an improvisational virtual environment interacting with human users. The human players are encouraged to be creative at their role-play under the improvisation of loose scenarios. The AI agent is capable of detecting 25 affective states from users’ open-ended improvisational input and proposing appropriate responses to stimulate the improvisation.


Introduction
The detection of complex emotions and value judgments from open-ended text-based multithreaded dialogue and diverse figurative expressions is a challenging but inspiring research topic.In order to explore this line of research, previously we have developed an affect inspired AI agent embedded in an improvisational virtual environment interacting with human users.The human players are encouraged to be creative at their role-play under the improvisation of loose scenarios.The AI agent is capable of detecting 25 affective states from users' open-ended improvisational input and proposing appropriate responses to stimulate the improvisation.
We notice in the collected transcripts, metaphors and similes are used extensively to convey emotions such as "mum rocks", "u r an old waiter with a smelly attitude", "I was flamed on a message board", "u stink like rotten meat", "a teenage acts like a 4 year old" etc.Such figurative expressions describe emotions vividly.Fainsilber and Ortony (1987) commented that "an important function of metaphorical language is to permit the expression of that which is difficult to express using literal language alone".There is also study on general linguistic cues on affect implication in figurative expressions as theoretical inspiration to our research (Kövecses, 1998;Barnden, 2007;Zhang et al., 2009).Thus affect detection from metaphorical and simile phenomena draws our research attention.In this chapter, we particularly focus on affect interpretation of a few metaphors including cooking and sensory metaphors, and simile expressions with the usage of comparative 'like' prepositional phrases.
Moreover, our previous affect sensing is conducted purely based on the analysis of each turn-taking input itself without using any contextual inference.However, most relevant contextual information may produce a shared cognitive environment between speakers and audience to help inference affect embedded in emotionally ambiguous input and facilitate effective communication.As Sperber & Wilson (1995) stated in Relevance theory "communication aims at maximizing relevance and speakers presume that their communicative acts are indeed relevant".Such relevant contextual profiles have also been employed in our present work to model cognitive aspect of personal and social emotion and

Related work
Much research has been done on creating affective virtual characters.Indeed, emotion theories, particularly that of Ortony et al. (1988) (OCC), are used widely in such research.Egges et al. (2003) provided virtual characters with conversational emotional responsiveness.Aylett et al. (2006) also focused on the development of affective behaviour planning for their synthetic characters.
Text-based affect detection becomes a rising research branch recently (Shaikh et al., 2007;Zhang, 2010;Liu & Singh, 2004).Façade (Mateas, 2002) included shallow natural language processing for characters' open-ended input.But the detection of major emotions, rudeness and value judgements was not mentioned.Zhe and Boucouvalas (2002) demonstrated an emotion extraction module embedded in an Internet chatting environment.However the emotion detection focused only on emotional adjectives, and did not address deep issues such as figurative expression of emotion.Also, the concentration purely on first-person emotions is narrow.Context-sensitive research is also employed to detect affect.Ptaszynski et al. (2009) developed an affect detection component with the integration of a web-mining technique to detect affect from users' input and verify the contextual appropriateness of the detected emotions.However, their system targeted conversations only between an AI agent and one human user in non-role-playing situations, which greatly reduced the complexity of the modeling of the interaction context.Moreover, the metaphorical description of emotional states is common in literature and has been extensively studied (Fussell and Moss, 1998), for example, "he nearly exploded" and "joy ran through me," where anger and joy are being viewed in vivid physical terms.In the work of Zhang & Barnden (2010), a few other metaphorical affective expressions (such as animal metaphor ("X is a rat") and food metaphor ("X is walking meat")) were intensively studied and affect was derived from such simple metaphorical expressions.
There are also well-known cognitive theories on emotion modeling.The OCC model provided cognitive appraisal theories for 22 emotional states, while Gratch and Marsella (2004) also presented an integrated emotion model of appraisal and coping, in order to reason about emotions and to provide emotional responses, facial expressions and potential social intelligence for virtual agents.However, there is very limited research on contextual 1 It is mainly about the bully, Mayid, is picking on a new schoolmate, Lisa.Elise and Dave (Lisa's friends) and Mrs Parton (the school teacher) are trying to stop the bullying. 2Peter has Crohn's disease and has the option to undergo a life-changing but dangerous surgery.He needs to discuss the pros and cons with friends and family.Janet (Mum) wants Peter to have the operation.Matthew (younger brother) is against it.Arnold (Dad) is not able to face the situation.Dave (the best friend) mediates the discussion.

The original affect detection processing and the system architecture
As mentioned earlier, our original system has been developed for secondary school students to engage in role-play situations in virtual social environments.Without pre-defined constrained scripts, the human users could be creative in their role-play within the highly emotionally charged scenarios.After an inspection of the recorded improvisational transcripts, we noticed that the language used for improvisation is complex and idiosyncratic, e.g.often ungrammatical and full of abbreviations, mis-spellings, etc.Several pre-processing procedures have been developed in our application previously to deal with misspellings, abbreviations, letter repetitions, interjections and onomatopoeia etc.Moreover, the language contains a large number of weak cues to the affect that is being expressed.These cues may be contradictory or they may work together to enable a stronger interpretation of the affective state.In order to build a reliable and robust analyser of affect it is necessary to undertake several diverse forms of analysis and to enable these to work together to build stronger interpretations.Also our previous affect detection has been performed solely based on the analysis of individual turn-taking user input without any contextual inference.Overall, we have adopted rule-based reasoning, robust parsing, pattern matching, semantic and sentimental profiles (e.g.WordNet (Fellbaum, 1998) and WordNet-Affect (Strapparava and Valitutti, 2004)) for affect detection analysis.Jess, the rule engine for Java platform, has been used to implement the rule-based reasoning while Java has been used to implement other algorithms and processing with the integration of the offthe-shelf language processing tools, such as WordNet.
In our turn-taking based affect interpretation, we have considered the following affective expressions.We found that one useful pointer to affect was the use of imperative mood, especially when used without softeners such as 'please' or 'would you'.Strong emotions and/or rude attitudes were often expressed in this case.Expression of the imperative mood in English is surprisingly various and ambiguity-prone.We have used the syntactic output from the Rasp parser (Briscoe and Carroll, 2002) and a semantic resource (Esuli and Sebastiani, 2006) to deal with certain types of imperatives.In an initial stage of our work, affect detection was based purely on textual pattern-matching rules that looked for simple grammatical patterns or templates partially involving specific words or sets of specific alternative words.As mentioned above, Jess is used to implement the pattern/templatematching rules in the AI agent allowing the system to cope with more general wording and ungrammatical fragmented sentences.The rules conjectured the character's emotions, evaluation dimension (negative or positive), politeness (rude or polite) and what response the automated actor should make.However, it lacked other types of generality and could be fooled when the phrases were suitably embedded as subcomponents of other grammatical structures.
In order to go beyond certain such limitations, sentence type information obtained from Rasp was also adopted in the rule sets.Such information not only helped the agent to detect affective states from the input (such as the detection of imperatives), and to decide if the detected affective states should be counted (e.g.affects detected in conditional sentences were not valued), but also contributed to proposing appropriate responses.
The results of this affective analysis were then used to (see Figure 1): 1. Control the automated actor (EMMA) that operates a character in the improvisation.I.e.
the detected affective states enable the AI agent to make appropriate responses to stimulate the improvisation.2. Additionally, drive the animations of the avatars in the user interface so that they react bodily in ways that is consistent with the affect that they are expressing, for instance by changing posture or facial expressions.We have also developed responding regimes for the AI actor.EMMA normally responds to, on average, every Nth speech by another character in one improvisational session, where N is a changeable parameter (currently usually set to 3).However, it also responds when EMMA's character's name is mentioned, and makes no response if it cannot detect anything useful in the utterance it is responding to.The one-in-N average is achieved by sampling a random variable every time another character says something.We also have N dynamically adjustable according to how confident EMMA is about what it has discerned in the utterance at hand so that it is less likely to respond if it has less confidence.EMMA makes a random response from several stored response candidates that are suitable for the affective quality it has discerned in the utterance it is responding to.Then the AI agent also searches its knowledge base to provide a suitable response to the human players using the detected affective states.We have particularly created the AI agent's responses in a way which could stimulate the improvisation by generating sensitive topics of the storyline.
Then an XML stream composed of the detected affective state from one user input and the AI agent's response is dynamically created and broadcasted to all other clients by the server.The users' clients parse the XML stream to obtain the information of the previous "speaker's" emotional state and the current AI character's response.An animation engine has embedded in each user client which updates the user avatars' emotional facial and gesture animation on each user's terminal.Therefore, if the previous human-controlled character expresses 'anger' affective state by saying "r u messing with me!!!", the animation engine in each user client updates emotional animation of that character on each terminal using cross behavior via simple facial and gesture animation (see Figure 2).In each session, up to five characters are engaged in.
room is a cesspit": affect (such as 'disgust') associated with a source item (cesspit) gets carried over to the corresponding target item (the room).There are also cooking metaphor examples implying emotions implicitly, such as "he is grilled by the teacher", "he knew he was going to be toast when he got home".In these examples, the suffering agents have been figuratively conceptualized as food.They bear the results of intensive or slow cooking.Thus, these agents who suffer from such cooking actions tend to feel pain and sadness, while the cooking performing agents may take advantage of such actions to achieve their intentions, such as persuasion, punishment or even enjoyment.We detected affect from such metaphorical expressions previously and used the AI agent as a useful application of theoretical inspiration for figurative language processing generally.
Especially we notice sensory and another type of cooking metaphors not only implying emotions but also sharing similar linguistic syntactical cues.The sensory metaphor we are interested in includes temperature, smell, taste, and light metaphors.We gather the following examples for the study of the semantic and syntactical structures of such metaphorical expressions, including cooking metaphor: "the news inflamed her temper", "he dishes out more criticism than one can take", "she was burned by a shady deal"; light metaphor: "you lighted up my life"; temperature metaphor: "they are kindling a new romance"; taste metaphor: "bittersweet memories" and smell metaphor: "love stinks", "the stench of failure" etc.
In ).In another word, some cognitive abstract entities could be un-cooked ("a raw talent"), tasty ("bittersweet memories") or have temperature ("heated debate", "burning love") or smell ("the stench of failure").We use such semantic preference violations to sense these metaphor phenomena and their affective states.
First, we use Rasp (Briscoe & Carroll, 2002) to indentify each subject, verb phrase and object in each sentence.Then we particularly send the main terms in these three components to WordNet (Fellbaum, 1998) to recover their hypernyms.We also focus on the analysis of phrases with a structure of 'adjective + noun' by deriving the synonyms or related nouns for the adjective and hypernyms for the noun term using WordNet.If the inputs indicate structures of 'abstract subject entities + actions', 'physical actions + abstract object entities' or 'temperature/smell/taste/cooking adjectives + abstract entities', then the inputs are recognized as metaphorical expressions.The detailed processing is also shown in Figure 3.For example, the AI agent carries out the following processing to sense the metaphorical expression "they are kindling a new romance".
1. Rasp: the input -> 'subject PPHS2 (they) + VBR (are) + VVG (kindle + ing) + AT1 (a) + JJ (new) + object NN1 (romance)' 2. WordNet: 'kindle' -> hypernym: FLARE UP, LIGHT.'Romance' -> relationship -> abstract entity; 3.An evaluation profile (Esuli & Sebastiani, 2006) determines: LIGHT-> positive; 'romance' -> positive.4. The input indicates -> 'third person subject performs a 'positive' action towards an abstract entity (romance) -> it is recognised as a metaphorical input. 5.The third person human subject (they) may experience a positive emotion by boosting up a 'positive' relationship abstract entity.Although the above metaphor recognition is at its initial stage, the system is capable of performing affect sensing and metaphor recognition more robustly and flexibly.It can also recognize other metaphorical input such as "a warm reception", "she is burnt by a shady deal", "deep, dark thoughts", "he stirred up all kinds of emotion" etc.
In the Crohn's disease scenario, metaphorical expressions have also been used to indicate battles between family members and Peter's stress towards his life changing operation.An example interaction taken from a recorded transcript is as follows: In the first input from Peter, 'thoughts' have been regarded as physical solid objects that can occupy physical space such as a plate or head.With the contextual inference, in Peter's second input, plate has also been metaphorically used to refer to one's head.Moreover, we can hardly consider the last input as a metaphorical expression if without any contextual inference.This also indicates directions of our future work on metaphor interpretation.

Affect sensing from simile expressions
In the collected transcripts from the school bullying scenario, we also notice similes are used to convey strong emotions by linking and comparing dissimilar concepts via prepositions such as 'like'.An example interaction is demonstrated as follows.
Indicated by italics in the above example, Mayid used a simile to indicate 'insulting/angry' by comparing Lisa's behavior of calling the schoolteacher for help with that of a 4 year old, while Lisa also employed a similar simile expression to imply Mayid's inappropriate behavior such as threatening or beating other characters as if he were a heavyweight boxer.There are also other similar simile expressions implying emotions such as "you dance like an angel", "Tom ate like a pig", "she sings like a bird" etc.In our processing, we particularly focus on such similes with a syntactical structure of 'subject + verb + preposition (like) + object'.The subjective evaluation profile and WordNet are then used to further derive the affect attached with the simile input.For example, we use the following processing to interpret the simile expression "you dance like an angel": 1. Rasp: 'PPY (you) + VV0 (dance) + II (like) + AT1 (an) + NN1 (angel)' 2. WordNet: 'angel' -> good person -> physical entity.
3. The evaluation profile shows: 'angel' -> positive; 4. The input indicates: 'the performance of a second person subject is compared with that of another person with 'positive' implication'.5.The input implies 'affectionate'.
For the example, "a teenager acts like a heavyweight boxer", the processing taken is in the following.
However, purely based on the analysis of the input "a teenage acts like a 4 year old" itself, the AI agent recognizes the simile expression but fails to determine the affect conveyed in it due to the following processing: 1. Rasp: 'subject NN1 (teenager) + VVZ (acts) + II (like) + AT1 (a) + MC (4) + NNT1 (year) + JJ (old)' 2. WordNet: 'old' -> age; the evaluation profile: age -> objective (i.e.non-emotional); 3. The input becomes: 'the subject's action is compared with that of a 'neutral' object' -> recognized as a simile.
Since interaction context plays important roles in discovering affect conveyed in emotionally ambiguous input, it is resorted to to further justify the affect detected from the analysis of individual turn-taking input.I.e.context-based affect detection is employed to justify the neutral expression drawn from the analysis of the input itself, such as affect justification for the above neutral simile expression.As Schnall (2005) stated that the intention of communication is to achieve the greatest possible cognitive outcome with the smallest possible processing effort, i.e. "to communicate only what is relevant".Thus in the following section, we discuss context-based affect detection and emotion modeling in personal and social interaction context to justify affect interpretation in literal and figurative expressions.Lopez et al. (2008) suggested that context profiles for affect detection included social, environmental and personal contexts.In our study, personal context may be regarded as one's own emotion inclination or improvisational mood in communication context and the social context may refer to other characters' emotional influence to the current speaker.We believe that one's own emotional states have a chain effect, i.e. the previous emotional status may influence later emotional experience.We make attempts to include such effects into emotion modelling.Bayesian networks are used to simulate such personal causal emotion context.E.g. we regard the first, second and third emotion experienced by a particular user respectively as A, B and C. We assume that the affect B is dependent on the first emotional state A. Further, we assume that the third emotion C, is dependent on both the first and second emotions, A and B. In our application, given two or more most recent emotional states a user experiences, we may predict the most probable emotion this user implies in the current input using a Bayesian network.

Context-based affect detection
Briefly, a Bayesian network employs a probabilistic graphical model to represent causality relationship and conditional (in)dependencies between domain variables.It allows combining prior knowledge about (in)dependencies among variables with observed training data via a directed acyclic graph.It has a set of directed arcs linking pairs of nodes: an arc from a node X to a node Y means that X (parent emotion) has a direct influence on Y (successive child emotion).Such causal modelling between variables reflects the chain effect of emotional experience.It uses the conditional probabilities (e.g.P[B|A], P[C|A,B]) to reflect such influence between prior emotional experiences to successive emotional expressions.
In our application, any combination of the 10 most commonly used emotional states could be used as prior emotional experience of the user.Also each conditional probability for each potential emotional state given two or more prior emotional experiences (such as P[approval|A,B] etc) will be calculated.The emotional state with the highest conditional probability is selected as the most probable emotion the user conveys in the current turntaking.Moreover, it is beneficial that the Bayesian network allows us to use the emotional states experienced by a particular character throughout one improvisation as the prior input to the network so that our system may learn about this user's emotional trend gradually for future prediction.In detail, at the training stage, two human judges (not involved in any development) marked up 3 example transcripts of the school bullying scenario, which consisted of approximately 470 turn-taking inputs.For each character, we extract three sequences of emotions from the improvisation of the 3 example transcripts to produce prior conditional probabilities.We take a frequency approach to determine the conditional probabilities for each Bayesian network.When an affect is annotated for a turn-taking input, we increment a counter for that expressed emotion given the two preceding emotions.For each character, a conditional probability In the above table, the predicted emotional state C could be any of the most frequently used 10 emotions.At the training stage, the frequencies of emotion combinations in a 10 * 10 * 10 ((A*B)*C) matrix are produced dynamically.This matrix represents counters (N CAB ) for all outcomes of C given all the combinations of A and B. A one-dimensional array is also needed to store counters (N AB ) for all the combinations of two prior emotions, A and B. Such a conditional probability matrix is constructed at run-time for each human-controlled character in the school bullying scenario based on the training emotional sequences.
For the prediction of an emotion state mostly likely implied in the current input by a particular character at the testing stage, the two prior recent emotional states are used to determine which row to consider in the conditional probability matrix, and select the column with the highest conditional probability as the final output.The emotional sequences used for testing expressed by each character have also been used to further update and enrich the training samples so that these testing emotional states may also help the system to cope with any new emotional inclination because of each character's creative improvisation.
An example algorithm of the Bayesian affect sensing is provided in the following.For the initial run of the algorithm, A, B and C are initialized with the most recent affects detected for each character purely based on the analysis of individual input.

Pseudo-code for affect prediction using a Bayesian network
Function Bayesian_Affect_Prediction { 1. Verify the contextual appropriateness of the affect C predicted by the Bayesian reasoning; 2. Produce the row index, i, for any given combination of the two preceding emotional states A & B in the matrix; 3. Indicate the column index, j, for the recommended affect C; 4. Increment counters: N AB [i] and N CAB [i][j]; 5. Update two preceding emotions by: Emotion A = Emotion B; Emotion B = The newly recommended affect C; 6. Produce the new row index, k, for any given combination of the updated two preceding emotional states A & B; 7. Calculate probabilities (i.e.P[C|A,B] = N CAB [k][column]/N AB [k]) for the predicted emotional state C being any of the 10 emotions; 8. Select and return the affect with the highest probability as the predicted affect C; } At the testing stage, when an affect is predicted for a user's input using the Bayesian network, the contextual appropriateness of the detected affect will be further justified.The verification processing using neural network-based reasoning, which will be introduced at a later stage, results in a final recommended affect.Then the conditional probability table obtained from the training stage is updated with the newly recommended affect and its two preceding emotions.The above processing is iterative to predict affect throughout an improvisation for a particular character based on his/her personal emotional profiles.
Moreover social emotional context also has great potential to affect the emotional experience of the current speaking character.E.g., a recent threatening input contributed by Mayid may cause Lisa and her friends to be 'angry'.A neural network algorithm, backpropagation, is used to model such an effect, which accepts two most recent emotions contributed by two other characters as input.The neural network implementation has three layers and 2 nodes in the input layer & 10 nodes respectively in the hidden and output layers indicating 'neutral' and the most commonly used 9 emotions in our application.Since it is a supervised learning algorithm, we use emotional context gathered from transcripts across scenarios as training data.This neural network-based reasoning may discover the emotional influence of other characters towards the current speaker as output.
At the training stage, we have used 5 transcripts of the school bullying scenario collected in our previous user testing to generate the training data of the emotional contexts.Two

Evaluations and conclusions
As mentioned previously, the detected affective states from users' open-ended text input have also been used to produce emotional animation for human players' avatars.The emotional animation mainly includes expressive gestures and social attention (such as eye gazing).Thus, our processing has employed emotions embedded in the scenarios, dialogue and characters for expressive social animation generation without distracting users from the learning context.We also carried out user testing with 220 secondary school students from Birmingham and Darlington schools for the improvisation of school bullying and Crohn's disease scenarios.Generally, our previous statistical results based on the collected questionnaires indicate that the involvement of the AI character has not made any statistically significant difference to users' engagement and enjoyment with the emphasis of users' notice of the AI character's contribution throughout.Briefly, the methodology of the testing is that we had each testing subject have an experience of bot h scenarios, o ne including the AI minor character only and the other including the human-controlled minor character only.After the testing sessions, we obtained users' feedback via questionnaires and group debriefings.Improvisational transcripts were automatically recorded during the testing so that it allows further evaluation of the performance of the affect detection component.
We also produce a new set of results for the evaluation of the updated affect detection component with contextual and metaphorical affect interpretation based on the analysis of some recorded transcripts of the school bullying scenario.Generally two human judges marked up the affect of 400 turn-taking user input from the recorded 4 transcripts of this scenario (different from those used for the training of Bayesian and neural networks).In order to verify the efficiency of the new developments, we provide Cohen's Kappa interagreements for the AI agent's performance with and without the new developments for the detection of the most commonly used 10 affective states.The agreement for human judge A/B is 0.57.The inter-agreements between human judge A/B and the AI agent with the new developments are respectively 0.48 and 0.43, while the results between judge A/B and the agent without the new developments are only respectively 0.39 and 0.34.
Although future work is needed, the new developments on contextual affect sensing using both Bayesian and neural network based reasoning have improved the AI agent's performance comparing with the previous version.We have also provided evaluation results of the improvisational mood modeling using the Bayesian networks for the 3 leading characters in the school bullying scenario based on the analysis of the 4 testing transcripts.We have converted the recognized affective states into binary evaluation values and obtained the following accuracy rates shown in Table 2  Table 2. Accuracy rates of improvisational mood modeling using Bayesian networks.
Generally negative emotions are well detected across testing subjects.Since in the school bullying scenario, the big bully tends to make other characters suffer, the improvisation tends to be filled with negative emotional expressions such as threatening, angry and fear.
Although positive and neutral expressions are recognized less well, the percentages of the inputs indicating positive and neutral expressions based on the human judges' interpretation are respectively approximate 30% and 25%.Thus although there is room for further improvements, the performances of affect sensing from positive and neutral expressions are acceptable.
Moreover, we also provide accuracy rates for the performance of the affect sensing in social interaction context using neural networks.Approximate 100 interaction contexts taken from the selected 4 example transcripts of the school bullying scenario are used for testing.We have also converted the recognized affective states into binary evaluation values and obtained 69% accuracy rate for positive emotions and 88% for negative emotions by comparing with the annotation of one human judge.The results indicate that other characters' emotional influence to the speaking character embedded in the interaction context is well recovered in our application using neural net based inference.
From the inspection of the evaluation results, although contextual affect detection based on both personal and social interaction context is provided, there are still some cases: when the two human judges both believed that user inputs carried negative or positive affective states, the AI agent regarded them as neutral.One most obvious reason is that sometimes Bayesian networks failed to predict some of the positive affective states (e.g.grateful) due to their low frequencies presented in the training data.Also affect sensing based on the analysis of individual turn-taking input sometimes failed to uncover the affect embedded in emotionally ambiguous input due to characters' creative improvisation which may affect the performance of contextual affect sensing.We also aim to extend the evaluation of the context-based affect detection using transcripts from other scenarios.Using a metaphorical resource (http://knowgramming.com), our approach for disease, cooking and sensory metaphor recognition obtains 50% average accuracy rate among the 80 With a limited sample size of 40 simile examples extracted from the transcripts of school bullying and Crohn's disease scenarios, our approach for simile detection achieves 63% accuracy rate.The simile interpretation will also be further developed to accommodate more complex phenomena in future work.
Figure 4 also shows some evaluation results from a 'within-subjects' analysis looking at the difference made PER SUBJECT by having EMMA IN (= playing Dave, in either scenario) or OUT.When EMMA is out, the overall boredom is 31%.When EMMA is in, it changes to 34%.The results of 'human Dave and EMMA Dave said strange things' respectively are 40% and 44%.When EMMA changes from in to out of an improvisation, the results of 'improvisation kept moving' are respectively 54% to 58% and the results of 'the eagerness to make own character speak' are respectively 71% to 72%.Although the measures were 'worsened' by having EMMA in, in all cases the worsening was numerically fairly small and not statistically significant.
We have exploited emotion evolvement and prediction in personal and social context using the Bayesian reasoning and a supervised neural network.The conversational intelligent agent has also been equipped with the capabilities of metaphor & simile recognition and interpretation.Although the proposed approaches indicate initial exploration on contextbased affect analysis and metaphor & simile inference, the implementation has enabled the AI agent to perform more effectively in affect detection tasks.In future work, we intend to employ emotion research of Hareli and Rafaeli (2008) and use Hidden Markov Models to further study and model how emotions evolve within individuals and in social communication context given various stimuli.

Appendix
Inspection of the transcripts collected indicates that EMMA usefully pushed the improvisation forward on various occasions.The following example transcript collected from the previous user testing shows how EMMA contributed to the drama improvisation in the Crohn's disease scenario.In the following interactions, Dave was played by EMMA.

Fig. 1 .
Fig. 1.Affect detection and the control of characters.

Fig. 2 .
Fig.2.An example of real-time interaction.Moreover, our system employs a client/server architecture for implementation.The conversational AI agent and other human-controlled characters consist of clients.The server broadcasts messages sent by one client to all the other clients.Thus user's text input from normal user client is sent to the AI agent client via the server.Then the AI agent, who plays a minor role in the improvisation with other human-controlled characters, analyzes the user's text input and derives the affective implication out of the text.Then the AI agent also searches its knowledge base to provide a suitable response to the human players using the detected affective states.We have particularly created the AI agent's responses in a way which could stimulate the improvisation by generating sensitive topics of the storyline.Then an XML stream composed of the detected affective state from one user input and the AI agent's response is dynamically created and broadcasted to all other clients by the server.The users' clients parse the XML stream to obtain the information of the previous "speaker's" emotional state and the current AI character's response.An animation engine has embedded in each user client which updates the user avatars' emotional facial and gesture animation on each user's terminal.Therefore, if the previous human-controlled character expresses 'anger' affective state by saying "r u messing with me!!!", the animation engine in each user client updates emotional animation of that character on each terminal using cross behavior via simple facial and gesture animation (see Figure2).In each session, up to five characters are engaged in.
www.intechopen.comAffect Interpretation in Metaphorical and Simile Phenomena and Multithreading Dialogue Context 57

Fig. 4 .
Fig. 4. Statistical results for 'boredom', 'Dave said strange things', 'improvisation kept moving' and 'eager to make own character speak' when EMMA is OUT (blue) or IN (purple) an improvisation.
Dave: what are your other options peter Peter: im trying very hard but theres too much stuff blocking my head up Peter: my plate is already too full.... there aint otha options dave

Table 1 .
table is produced based on the training data.An example conditional probability table is presented in Table 1.An example conditional probability table for emotions expressed by one character by comparing with the annotation of one human judge.
Affect Interpretation in Metaphorical and Simile Phenomena and Multithreading Dialogue Context 65 testing examples.Also, we intend to use other resources (e.g.Wallstreet Journal and other metaphorical databases (e.g.ATT-Meta)) to further evaluate the metaphorical affect sensing. www.intechopen.com