We aim to show in this chapter some meaningful insights regarding a Virtual Collaborative Decision Environment (VCDE) prototype. The main goal of VCDE is to provide an environment for enterprise collaborative decisions using a DSS-like approach. The simulation is DSS-like because it provides the user with all the necessary information and tools that ensure a documented decision regarding all major aspects of a virtual company. The characteristics of the multi-agent systems as autonomy, local views and decentralization are also present in the VCDE. Therefore, the user is provided with knowledge over the marketplace; his organization’s business capabilities and business processes; and, at a certain extent, over the business knowledge. He is required to use that explicit knowledge in conjunction with his implicit knowledge in order to make decisions over his virtual enterprise. Then, considering the actions of the actors involved in the virtual environment we will automatically mine mainly for the business strategy view, the social networks and also, at a certain extent, for the implicit business knowledge.
This system is developed primarily in order to simulate decision situations as a part of the academic training of students. The second goal of the system is to provide us with user activity logs that will be used in the process of decision pattern mining and decision modeling. This is an important step in achieving our final goal of transforming implicit knowledge of decision makers in explicit knowledge that can be shared. In order to reach our two goals, we first need to model the decisions that will be made by the managers and then, by following every action of each user, mine the activity logs in order to gain insights into the decision making style of each participant.
In the second section of this chapter we will introduce the reader into the particularities of the problem domain to be presented and we will show how we intend to connect to previous research.
In the third part, the proposed architecture of the system will be introduced. After giving the reader the overview of the system we elaborate on several key aspects of the system like some of the decisions that need to be made, the decision models, the variables involved and the relations between them.
In the fourth section of this chapter, we provide evidence regarding the possibility of: mining decision models from user activity logs; comparing different decision making strategies of users; and building decision reference models. In this section we will also discuss the methods we implemented for user activity logging, the format of the logs and the proposed mining methods.
In the last section we state our findings so far, as well as the future flow of the project. We also argue the strong points of our research and we show the aspects that still need to be improved.
2. The problem domain and previous research
As stated in the first section we are developing a Virtual Collaborative Decision Environment using a DSS-like approach. There has been done a lot of research in decision support systems, multi agent environments as well as in business simulations. Since it is out of scope to discuss decision support systems definitions and history we will resume ourselves to underlining the most important features of such a system that will be implemented in VCDE. A DSS must provide the decision maker, in an interactive user friendly interface, with all the necessary data and information that allows him to identify a decisional situation and choose one alternative, provide data analysis tools (such as what-if analyzes and simulations). The reason why we consider we have a DSS-like approach over VCDE is that in the software engineering process we mainly focus on decision model development. What we intend to build is a system that acts and looks like a DSS from the user point of view.
Multi agent systems have already been used for modeling and simulating organizational decision making (Sun & Naveh, 2004). We are aiming to create a virtual environment in which students will be represented as intelligent agents managing a company and will interact with each other through the environment. The characteristics of multi agent systems will be present since each company will be autonomous, each company will have access only to a limited view of the entire environment and there will be no super entity controlling the agents. Each agent will exercise free will by responding in its own time and manner to the changes in the environment. This means that behind the user interface the system needs to enable the interaction between the virtual companies.
One branch of computer based simulations is real business simulations employed for teaching company management. Students gain a lot of knowledge by applying theoretical knowledge in a simulated virtual company since students “rarely take economics as a free elective – especially beyond principles.” (Allgood et al., 2004). Students involved in management simulations show great improvements in education and knowledge (Lean et al., 2006). We argue that in order to actually create a virtual business simulation the best approach can be DSS theory and practice since there are two activities implied: it allows student to theoretically model decisions and it also provides them with facts, data and information that require model implementation.
Because we are developing a simulation for academic training purposes we need to use existent knowledge in human-computer interaction science since the students will interact with the virtual environment using computers. The main concern in this direction must be a compromise between an interface easy to use and an appropriate amount of data and information needed for describing a decisional situation. The user must also be provided with an interface that will allow model building and a high degree of personalization. Each decision maker may consider some data to be relevant and some not. Therefore, the system should allow the user to add figures, tables, charts to a fully customizable decision making workspace. By following and analyzing each customized workspace we will be able to compare the foundation of the decision making process for each decision maker.
The second goal is to develop means for logging the user’s actions and, by mining those logs, automatically build an individual model of decision making. In achieving this goal we must rely on previous research regarding workflows as well as process mining. A workflow was defined as a depiction of the sequence of activities performed by an individual (van der Aalst & van Hee, 2002). There is also an extended research in process mining domain. Process mining aims to use event logs produced by different systems involved in the enterprise’s operations in order to create a model of the processes and workflows that actually take place in the company (van der Aalst et al., 2004); (Ingvaldsen & Gulla, 2006). Another goal of process mining is to compare a prescribed model with the model obtained after mining (Dongen et al., 2005). Also, using process mining algorithms the analysis of decision points can be performed (Rozinat & van der Aalst, 2006a). A common and well known definition of a decision is that it is a cognitive process that starts with the discovery of the need for a decision and ends with the choice of one alternative (Holsapple & Whinston, 1996). Considering what we stated above we argue that “a decision workflow represents the depiction of the sequential activities performed by the decision maker starting with the discovery of the need for a decision and ending with the execution of the chosen alternative” (Petrusel, 2009). We also argue decision mining is the activity that, based on the activity and event logs obtained from a software tool (usually a DSS), extracts and creates a model of the decision making process depicted as a decision workflow (decision model).
The decision mining problem was approached before in (Rozinat & van der Aalst, 2006b). The mining algorithms are implemented in a plug-in called Decision Miner that is implemented in ProM Framework. This approach uses a derivation of C4.5 algorithm to build decision trees that allow analysis of choices in the decision points of a workflow. Rozinat proposes the use of Petri Nets in order to determine the points in which a choice was made and one or other of the branches were followed. After the decision point is identified the problem is turned into a classification problem that tries to determine if the cases with certain properties follow specific routes. However, this approach is different from our approach. We do not try to determine the points where one of the branches in a decision tree is selected and to determine the case data that influences the choice. We try to follow the actions of the decision maker and to identify the overall strategy employed in decision making. We try to determine what is the data and information used in the decision making process, how is this data used and what is the knowledge employed by the decision maker in choosing one alternative. The model that we create is very different from a decision tree because it is linear and basically does not have a decision point. The tree like structure can be obtained only if a large number of models are aggregated. The aggregation method is actually more similar to creating reference models. The approach proposed by Rozinat can be applied only to the reference model.
Once each individual decision model is mined and automatically created we aim to create reference models for each category of decisions implemented in VCDE. This requires us to rely on previous work regarding the creation of reference models (Fettke & Loos, 2006). A decision reference model aims to embody the necessary actions to be taken in order to make a certain kind of decision. We argue that the decision reference model can be depicted as a decision workflow improved with the data and information needed for decision making. We also need to compare decision workflow models in order to determine the differences between them or between the reference model and one actual model. In achieving this aim we can rely on conformance checker which is a tool already developed for process model comparison and verification.
Regarding the methodologies, techniques and tools we must employ in developing our project we need to cover certain aspects. First we need to decide the representation of decision models. Another aspect that needs to be addressed is the means for decision model validation. Then, based on the conceptual architecture of the system, we need to decide the actual development environment that will suit our needs. Then we must decide over the tools and algorithms used for decision mining.
In our approach over the software architecture we consider the Business Architecture as presented by OMG (BAWG, 2008), also present in the Zachman framework and subsequent derivations. The key views of Zachman framework are: business strategies; business capabilities; business processes; business knowledge; and organizational overview. What is different from the usual approach is that we will fully specify the business capabilities; business processes and the organizational overview but we will determine the business strategies and the business knowledge once the software is implemented by mining the activity logs of the decision makers within the VCDE. This is why the software engineering process’s goal is different than the one in the classical approaches. We do not try to specify the requirements of the user and then develop the software around those needs as in most of the other software processes. Instead, we aim to create software that will force the user to reason and decide on several pre-defined decisional situations. Our goal is not to fulfill the needs of a user but to use the software as a discovery tool. This needs to be done while logging all his actions and to force him to transform each mental activity in one explicit action within the software. What we do is very similar to modeling a business process and then creating the software that will support that process (Ouyang et al., 2009).
3. The virtual collaborative decision environment
In this section we will underline some of the most important aspects of VCDE. In the first sub-section we will give an overview of the general concepts and the architecture of the system. In the second sub-section we will show some of the most important decision models we included in the VCDE, the implementation and the means we used for model validation.
3.1. The architecture
The general concept is that of a multi-agent system. This means that the system is intended to be used by students that will act as intelligent agents in the environment. Each agent will act on its own free will and needs to compete with all other agents over limited resources.
We use the classic client-server architecture. All data regarding the environment is stored on the server while the users will connect to it using a client computer that has only the role of presenting data and information.
The VCDE will be populated by enterprises either in the same business field or in several related fields. The first alternative is more appropriate for the academic training since it allows us to compare and grade the evolution of each student. Each enterprise interacts with the other enterprises through the “marketplace”. This consists in data and information that is available to all enterprises as well as the software modules that facilitate interaction (such as an auction system, news bulletins, a financial market and a loaning agent for credits and leasing).
Each enterprise is divided into six departments that interact with each other and require specific decisions from the managers. Besides the connection with other departments there will be a permanent data exchange with the marketplace. The composition of the enterprise and the internal and external relationships are depicted in Fig. 1.
There can be one or several decision makers. There can be different configurations of decision responsibilities within each enterprise. The common set-up is with one decision maker that will manage all departments. Alternative setups can include:
several decision makers that are in charge with all the departments and a decision requires either consensus or 51% of the votes;
several decision makers that are in charge with different departments;
a hierarchy of decision makers in which the lower level reviews the decisional situation and presents to the higher level decision makers a condensed view and a couple of recommended decision alternatives.
The different possible decision setups of an enterprise are presented in Fig. 2.
The next important point in the enterprise is the actual activities performed by the decision maker which were depicted as the general use-case diagram of the system (Fig. 3.).
There are two main actors in the environment: the decision makers of each enterprise and the system/mediator of the environment. We aim to create an environment that will be entirely autonomous but in the early stages of deployment some of the actions that are intended for automatic processing will be conducted by the instructor which will thus asume the role of a mediator.
The first set of activities the decision maker performs regards information gathering for the next decision to be made. The manager first needs to review available data and information and then needs to develop some kind of strategy towards creating decision alternatives and selecting one of those alternatives as the final decision. This is the process that interests us the most. Therefore, besides providing the necessary figures, VCDE allows the user to start from existing data and build his own what-if analyses and scenarios using Excel-like tables, formulas and functions. Each action of the user will be logged for later usage.
The second set of activities that require the manager’s involvement is the collaboration process in order to reach a decision. This is important because often a decision of one individual decision maker is different then the decision made by a group. We are also extremely interested in logging each action in the process of collaboration since it is the best source of information regarding what actually happens. The VCDE enables collaborative decision making through several tools as file sharing, instant messaging and voting.
The activity that ends the decision process is the choice of one alternative and the communication of that alternative to the system (the marketplace). This is done by sending updates to the server regarding the action that is implemented by the company. The instance of this action can be, for example, setting a figure for advertisment expenses, hiring a new employee, etc. This particular moment of the decision making process is extremely important for us because we consider that the decision process starts with the recognition of the need for a decision and ends with the choice of one alternative. Therefore, all the activities logged between the timestamp of the “start decision research” action and the timestamp of the “decision submit” action belong to that particular decision process. This is the base of the decision mining theory that we will introduce in the fourth section.
The system interacts with all the enterprises as the marketplace in the real economy. In VCDE this means that the system is accepting updates regarding the decisions of each enterprise, consolidates the actions of all enterprises and returns the new situation on the marketplace as updates for each enterprise. However, each particular enterprise will only have access to a pesonalized view of the marketplace for every department. This means, for example, that in the personnel department a company will have access to the labour fource only based on the advertising expenses for the available positions in the company. If the company invests in local advertising (small sums of money) it will be provided with a view containing less persons with low qualification available for hire, compared to the available emplyees generated by national advertising (larger sums of money required). If the company does not advertise open positions it will have no access to the labour market and will be able to hire only untrained new employees.
3.2. Decisions in VCDE
Each user of VCDE will be the manager (or one of the managers) of one virtual enterprise. The game takes place over a limited number of years divided into months. A decision once made will be effective starting next months and will continue its influence until another decision is made. Each company starts with the same fixed sum of money and must build the business from the beginning. The overall goal is to distribute the largest dividends to the owners over each fiscal period (a year starting January and ending December). The winner is the company that holds the lowest overall rank determined as the sum of ranks in each year.
The VCDE is modeled as an environment of virtual companies from several different related business domains. However, for the prototype, all companies will compete in the restaurant business because it is a business regarding to which all students should have some knowledge.
The decisions that managers need to make were divided into two major groups: decisions at the start of the company and decisions for each period.
Decisions that need to be made at the start of the company are:
choice of the type of company to be established;
choice of the location where the business will function;
amounts invested in the restaurant’s environment;
choices over the staff to be hired,
choices over the raw materials to be used
choice over the advertising of the initial grand opening.
Since VCDE is designed to be a tool used in academic training, all those decisions will be first explored by students in a business plan developed before the actual simulation will start. Each decision will also be discussed in class and each student will be required to present motivation and a coherent strategy behind the selected alternative. In the next paragraphs we will discuss shortly the framework for each decision, as implemented in VCDE.
First decision will be to choose what kind of company the manager starts and is the most important decision since it affects the future of the enterprise. Alternatives are: gourmet, traditional and fast food. A gourmet restaurant aims an elite customer base. Some of the requirements that need to be considered by the decision maker are: location (not important), environment (critical), employee number (important), employee quality (critical), raw materials used (critical), customer satisfaction (critical), customer rate of return (critical - must be kept very high), meal quality (critical), meal innovation (important), meal price (very high), return per meal (very high). A traditional restaurant aims an average customer base. The requirements are: location (important), environment (medium), employee number (medium), employee quality (medium), raw materials (medium), customer satisfaction (critical), customer rate of return (important - must be kept high), meal quality (very important), meal innovation (unimportant), meal price (average), return per meal (average). A fast-food restaurant aims an extended customer base. The requirements are: location (very important), environment (not important), employee number (important), employee quality (medium), raw materials (medium), customer satisfaction (critical), customer rate of return (important - must be kept average), meal quality (important), meal innovation (unimportant), meal price (low), return per meal (low). Overall value of the business is calculated as %location + %environment + %employee quality + %customer satisfaction + %profit + %dividends paid. Variation of overall value influences the growth percentage of the future incomes.
The second decision will be to bid for one of the available commercial spaces (location) using the VCDE auction system. The general environment is similar to a mall’s food court: there are central spaces, medium spaces and peripheral spaces. The commercial space is characterized by variables as: the quality, number of new visitors, rent and purchase price. It influences variables as the overall value of the business and the number of new visitors.
Third decision will be to invest in interior design of the restaurant. Depending on the type of restaurant the investment needs to be bigger or lower. Investment is a value set by the decision maker.
Fourth decision will be to find raw materials suppliers. The raw materials are important for the overall quality of food. There are five alternatives for food quality: low, poor, average, good and excellent, each with a different price tag.
Fifth decision will be to hire employees. The marketplace will supply only a limited number of specialists (like cooks, waiters) and an unlimited number of untrained employees. According to the advertising expenses the manager will have a complete or partial view of available resources. If several managers have access to the same view and decide to hire more persons than the maximum allowed an auction will start. Every year a new set of specialists will become available. Over specific periods of time each type of employee will automatically gain training levels if they are used in the same specialty. The managers can decide to invest in employee training in order to speed level gaining. This will be done at a cost that increases with each training level. The training levels vary from 1 (untrained) to 10 (expert). If the company lacks employees in one specialty and has excess employees in other specialties, employees will be shifted automatically and the needed positions will be filled at training level 1.
Sixth decision will be to advertise the grand opening. There are three levels of advertising: local, regional and national. There are four means of advertising: flyers, newspapers, radio and TV. Advertising can be done for: location, food or hiring. Each kind of advertising will have different impact on the new customers or available staff.
Some of the decisions of the managers will result in a competition over limited resources (for example several companies want to hire cooks but on the market there is only one highly qualified person). In such a situation there is an auction system which is available to all users. There can be closed auctions and open auctions. Closed auctions are visible only to the bidding enterprises. Open auctions are visible to all the companies on the market. For example bidding for hiring a level 10 chef is visible only to the companies that invested enough in advertising to discover the employee while bidding for shares on the stock exchange is visible to all companies. Auctions allow multiple bids, each bid must be higher than the previous (in fixed increments). All participants are informed of each bid and the winner is the last bidder. When using the auction system it is also very important to log all the actions of the manager in order to create a personalized decision model and to determine what are the key figures used by the decision maker and what is the strategy employed during the auction.
The decisions for each period are divided according to the time frame affected and to the frequency a new similar decision needs to be made into three categories: operational, tactical and strategic decisions.
Some of the operational decisions are:
Decide on monthly expenses. This means reviewing all the previous financial data, analyzing data, building personal decision model then setting the necessary decision variables to the selected figures. The main data sources are: cash-flow (CF), profit and loss (P&L) and several key indicators calculated within VCDE. Each decision maker is able personalize the decision workspace by extracting and analyzing only the desired data (by building charts, what-if analyzes and scenarios). In order to mine for a personalized decision pattern each action performed by the decision maker is logged. The alternative source of information for the managers that provides external data will be reports from commissioned work like market research, head hunting companies, etc. For example, by employing a marketing company the decision maker will receive a customer satisfaction survey report that will show the food quality, service quality, environment, crowding, service time, and menu suggestions. If a larger amount of money is invested, the survey will also show some data regarding the competing companies. Another source of information will be the monthly news report from the marketplace. This will state extraordinary events, will show yearly hierarchies of company values, etc.
Decide on employee policy. This includes decisions regarding wages, new hires, advertising available jobs.
Decide on investments to be made like amounts spent on current repairs, improvements to the location.
Decide amounts invested in advertising the company’s business.
Some of the tactical decisions are:
Training employees. Can be done by investing money in one or several employees in order to improve scores for overall employee quality.
Insuring the company’s location. At a certain point in time each of the companies will suffer one disaster (like fire, earthquake). Logging this moment is very important because we can show the manager’s attitude and strategies while dealing with crisis situations.
Selling own shares. Each company is present on the stock exchange and starts by owning 80% of its own shares. One way of getting cash is to sell own shares on the stock market. This is done in packages of 10%. Each package price is calculated according to the company’s overall value. Selling is done by using the auction system each time one company sells shares. Each competitor can buy shares from any other company. Each shareholder is entitled to a percentage of the dividends distributed at the end of each year by the company. The sums distributed as dividends are the major criteria for victory.
Paying dividends to the shareholders. Since the sums distributed as dividends are the major criteria for victory, each company should consider paying large dividends to the shareholders. The sum is distributed among the shareholders. The money will be withdrawn from the accounts when decided by the manager but no later than end of June of the following year. If there is not enough money available, an overdraft on the bank account will be automatically established for a limited period at a high interest rate. If the overdraw is not paid, shares of the company will be automatically sold in order to cover the debt. If one company owns shares from another company it will get the dividends amount as a cashing in the bank account.
Getting a loan or a leasing. In order to cover current expenses each company will get a small overdraft on the bank account for which it will have to pay interest each month. However, larger loans can be granted by the marketplace if the company has enough fixed assets to be mortgaged. In order to finance some major improvements of the company or to buy out a competitor the loan is the only source of extra cash. The company can purchase equipment (e.g. kitchen equipment) by leasing if enough cash is not available. This way the equipment is already mortgaged and becomes the property of the company only when paid in full.
One of the most important strategic decisions is to expand the business. In VCDE this can be done by buying out a competitor by using the stock exchange. This means adding: a new location to the company, new employees, new type of business, etc.
One of the goals of VCDE is to provide the students with a structured, rational approach over decision making. The first step is to avoid poorly documented decisions based on instincts. The decision modeling is done using the Criterium Decision Plus (CDP) software tool created by InfoHarvest. This tool offers support from the early stages of modeling (like the initial brainstorming) to the last stages (like validation of the model). The reason for which we selected this particular tool is that the modeling effort is well supported and is easy to understand for students. The easiness of use is also aided by the fact that the modeling can be done using ratings instead of complex mathematical equations. This also improves understanding of the model and keeps the focus of the students on the model itself instead of the formalism used. Before VCDE simulation starts, the students are presented with some details of the decisional situations in which they will be later involved and are required to model them using CDP. After the VCDE simulation is ended, the students are again required to model the same decisional situations, this time with the benefit of the experience gained while using VCDE. The initial and the final models can be compared and this way we can evaluate and grade the progress done by each student. The students are also graded after the second VCDE simulation run. The winner is graded with top marks. All other students are graded with decreasing marks (according to the results of their virtual companies) up to the student that manages the worst company who gets a failing grade.
The main activities performed when creating a decision model are:
define the goal of the problem,
define the decision group,
identify factors or criteria important to achieving the goal,
expand each criteria further until a criteria that can be easily evaluated is found,
assign weights to each criteria,
evaluate decision alternatives against each criteria,
There are three main phases in creating a model using CDP: the brainstorming when alternatives and criteria are defined; the creation of the hierarchy, weight assignment and rules; and the fine tuning of the model and final validation.
Before the VCDE simulation will start, the students are required to decide the type of virtual restaurant to be opened. Each student is required to use CDP in order to build a decisional model based on the previous class discussions that underline the criteria most relevant for the decision. One possible initial brainstorming model is presented in Fig. 4:
The goal is to select the restaurant type by choosing one of the three available alternatives: fast-food, traditional and gourmet. There are six primary criterions each with several sub-criterions.
The second phase is weighting each criterion and sub-criterion and then creating a hierarchy. CDP implements analytical hierarchy process (AHP) that was also associated with strategic decision making by (Bhushan & Rai, 2004) and Simple Multiattribute Utility Technique (SMART). AHP divides the weight of each criterion to the sum of all weights for the same level of criterions. This way each criterion gets a score between 0 and 1. SMART uses a function (linear, exponential or user-defined) to convert a weight for one criterion to an internal score of the criterion also in the 0 to 1 range. The weighting for each criterion can be done by direct comparison or by pair-wise comparison. Direct comparison means that a weight is set by the decision maker by comparing the criterion with other criterions on the same level, according to previous experience or by using third party information. Pair-wise comparison means comparing every possible pair of criterions and assigning the weights for each pair at a time. Rules can also be defined at this moment. There can be created either simple rules (e.g. expenses cannot be greater than 1000) or if/then rules (e.g. if raw material price is 100 then meal price must be greater than 100). Each student must create weights and rules for his own model. Since each person assigns different weights to criterions and sub-criterions the alternative ranking produced by each model will be different. The model can be further enhanced by taking into account uncertainty. Each criterion may be assigned with different degrees of uncertainty for the values of the weights.
After successfully creating the model the fine tuning and validation of the model will be performed. This means making sure that weights are correctly distributed and that there is no criterion that will fully decide the best alternative. One of the important analyzes is the contribution of each criteria to the final score of each alternative. The validation is done in CDP also by sensitivity analysis of the preferred alternative to changes in weights or rating values.
Each model created by the students, besides the validation of weights and rules needs to be mapped to the “reality” in VCDE. For example, in the model presented in Fig. 4. the weights and rules are set so that CDP determines the best alternative is a traditional restaurant. But if a rule limits the monthly rent to a certain amount, and in the auction that amount is surpassed CDP could determine that the new best alternative is a gourmet restaurant. In this case, the whole model needs to be changed in order to include the new situation.
Since we created also a numerical business simulation many of the business models are implemented in VCDE using variables and constants. There are three categories of variables: decisional, environment and calculated. Decisional variables (DV) are set by the decision maker. Some examples are: dividends to be paid, number of employees, employee salary, product price, etc. Environment variables (EV) are set within the system and cannot be changed or influenced by the actions of the companies. Some examples are: maximum number of customers, minimum number of employees needed, employee minimum salary/specialty, employee training cost/level, etc. Calculated variables (CV) can be based on any of the previous two and, therefore, can only be influenced by the decision maker but not altered directly. Some examples are: overall value of company, number of returning visitors, company’s yearly rank, overall employee quality, etc. A constant is a fixed value that does not change over time. Some examples are: the score of location, the interest for credits, the interest for deposits, etc.
The variables and constants are used to create business models for VCDE. One example of a variable calculation is:
In Fig. 5. we show the last four steps involved in the calculation of the number of customers the company will have next month. The customers next month variable is numeric and is obtained by adding the numerical variables: returning customers and new customers. Returning customers are calculated as the number of customers in the previous month multiplied with the variable customer rate of return which is expressed as a percentage. The customer rate of return depends on the variables: environment of the restaurant, employee quality and customer satisfaction with the meals served. Each variable is expressed as a percentage and customer rate of return is calculated as an average rate of the three. The last three variables are also calculated based on others but, in order to keep the example short, the dependencies were not depicted.
The mathematical equation that summarizes the model and is implemented in VCDE for the partial model depicted above is:
For each calculated variable in VCDE we created a similar model. The models are then validated using scenarios in order to balance each variable in the model and the influence on others.
We create decision models in order to determine the alternatives that will be available to the decision maker and to determine how choosing one alternative regarding a variable affects the other variables in VCDE.
Is obvious that the models created for one calculated variable are related to other models implemented in VCDE. There are a lot of cases in which the same variable (decision, calculated or environment) is present in several models. In order to show a partial model of interaction between the variables we created the following diagram:
In this model advertising expenses is a decisional variable set by the manager for each month. Corroborated with the advertising price (environment variable) it influences the advertising units for the company which, in turn, will further influence the product units sold. The sales of the each company will be added up so that the total market is calculated. Each company will therefore hold a market share of the total market. This is an extremely important indicator for the decision making process regarding the price strategy of the company. But the sold units also influence the revenues of the company. Based on the cash revenues from sales and all the expenses, the financing need variable is calculated. It will influence the decision making process regarding the loaning strategy (if sales are not big enough) or the dividends distributed to the shareholders (which is the major criteria for winning).
4. Decision mining and decision models
As we argued in the second section of the paper, a decision process consists of a series of cognitive actions undertaken by the decision maker from the moment the need for a decision arises until the choice of one alternative. This is why the decision process actually resembles a workflow. The challenge is to create means of transposing cognitive processes to physical actions. Since Business Process Modeling Notation (BPMN) is the de facto standard in depicting workflows we chose this notation to model the prescribed collaborative decision process that is implemented in VCDE:
The process starts when a decision situation requires the intervention of the managers of the virtual enterprises. The focus in this model is the capture of the activities performed and also the enactment of implicit knowledge of the decision makers. Each activity in the model is mapped within the system to several objects. Each artifact of the model is created in order to facilitate the mining of the decision workflow. VCDE is designed so that the decision makers use the software for all communication and no face to face meetings are permitted. This is why in VCDE there is a sub-system designed for voting (Fig. 8.) seen as the only mean of deciding over one alternative or another. The only choice available to the users is whether the vote requires majority or consensus in order to choose one alternative.
Since VCDE will only be used in class starting the spring semester of 2010, in the next part we will show the results we already obtained by using a DSS designed for enterprise financial decision making (Petrusel, 2008). Those preliminary results are very similar to those that will be available after the VCDE simulation. What we try to do is provide a view of our completed first experiment (Petrusel, 2009) as an example of what we expect to accomplish by using VCDE. This experiment aimed to prove a priori that decision mining is feasible and that the decision models created this way can improve our understanding over the whole decision process (Petrusel, 2009). The principles that we already used and the ones employed in VCDE are the same; the only difference will be that the logs available for mining will be considerably larger.
When mining for the decision model there are several goals we try to accomplish. First goal is to determine the control-flow perspective over the decision process. This mainly means to establish dependencies among tasks. In order to do that we have to answer several questions: which activity precedes which, are there any activities that imply others, are there concurrent activities (we observed that in decision processes concurrent activities usually means reviewing information from two or more sources) and if there are any loops (in decision processes we observed that loops appear mainly when what-if analyses and scenarios are reviewed). Another important piece of information is whether a path is more frequent than the others. If there is not a high frequency for one path it means that the user does not have a routine but searches for information in random places. This was found mainly in unstructured decisions that appear rarely (like strategic decisions and sometimes tactical ones). In operational decisions, the path is almost always the same. Another goal is to mine for the social networks when the decision is collaborative. This means answering to one critical question “how are the communications between actors performed and what are the dependencies between the decision makers?”. Another important question is whether a decision maker influences the others.
Once the model is created, ProM Framework allows the validation of the model by using conformance checker. It allows us to see how much the model matches existing execution data and to highlight discrepancies. Conformance checker is also useful when a prescriptive model exists and we need to check if the real execution data follows the model. ProM Framework allows decision point analysis that aims to detect data dependencies that affect the routing of a case (Rozinat & van der Aalst, 2006a). By analyzing decision points we can determine the probability for a certain action to follow another action, thus providing us with the possibility of also mining for business rules.
As we already suggested, each action of the user within the system is logged for further analysis by decision mining algorithms. The entity-relationship diagram for the tables implemented in VCDE for logging purposes is similar to the one recommended by ProM Import Tool and is presented in Fig. 9.
Data is added in the four tables at every action of every user. The result we obtained for our financial DSS are presented in Fig. 10. The four tables were converted by using ProM Import Tool to the MXML format required by ProM Framework. Cleaning the logs is required once the import is finished. Cleaning the logs aims to remove incomplete processes that can affect the quality of the model and is mainly done manually. Incomplete processes are the ones that either do not start with “decision start” activity or do not end with “communicate decision” or “drop decision” events. Incomplete processes usually show up when the decision maker starts researching one decisional situation and then aborts the process without choosing one alternative. After the cleaning is finished the decisional model needs to be created. This involves several activities: ordering the activities, assigning activities to the correct processes (there can be the case when several decisions are researched or made at the same time by the same decision makers) and building several views of the process model in order to ensure a better analyze. We created decision models by using three logically different process mining algorithms: alpha++, heuristic miner and fuzzy miner. Because in this first experiment the decision situation and environment was simulated, the logs were almost noiseless and all three algorithms produced about the same results. However, we expect to obtain much noisier logs after the VCDE simulation is run. This is why we plan to evaluate the performance of each algorithm on longer, noisier logs at a later time.
In the preliminary experiment we created only three companies and nine decisional situations. Starting from the logging tables (on the left side of Fig. 10.) we mined the following models (on the right side of Fig. 10.):
In the mined company there are two decision makers (D1 and D2). In the scenario presented in Fig. 10. the decision makers had to decide over the financing sources to be used when buying a car. It can be seen that D1 is more analytical and relies on more simulations and what-if analyses then D2. By feeding new data into the system he changes the initial values and tries to broaden his perspective over the decisional situation. D2 relies only on simulations based on accounting data and jumps to the decision without careful consideration. It also can be seen that D1 initiates the debate over the right decision and sends an Excel file to support his option. By following the two decision workflows we can argue that D1 has carefully considered all the alternatives and his choice is based on an analysis. Meanwhile, D2 briefly reviewed available data and jumped to the decision (possibly relying on experience). Even though the decision needs consensus, it can be argued that D1 influenced the final decision since he initiated a debate and sent a file to D2 in order to back up his choice. Because there is no significant difference between the two decision makers from the point of view of former experience and studies (both have worked around eight years in similar positions and have graduated an economics faculty) it can be said that D1 is more involved in decision making and usually influences the other decision maker. When we disclosed our findings regarding their decision profiles, both decision makers agreed that, in the majority of cases, the alternative suggested by D1 is the one chosen. This is why we can argue that our decisional patterns are close to reality. However, if we take into consideration the prescribed decisional model presented in Fig. 7. we can state that it is not followed entirely. Since VCDE simulation will be run twice for the students in one semester, we plan to use conformance checker in order to compare the models mined after each simulation. We expect to see better decision models by observing less noise (less incomplete processes) and by observing less random activities (the decision makers should be using the first experience in developing some decision strategies regarding specific decisions).
5. Conclusion and future work
We focus our research on two main objectives. The first one is creating an academic DSS-like virtual enterprise simulation. The second one is creating decision models by mining user activity logs obtained after the simulation. In order to reach each objective we must rely on previous research from various fields like simulations, multi-agent environments, decision theory, and process mining.
In the third section of the paper we introduced the general setup of the virtual environment, the possible internal setups of enterprises and the user interaction with the system. In order to present the VCDE we shortly described some of decisions that the managers of the virtual enterprises must face. Because the interaction with the system is DSS-like the decision maker will be able to document the decisional situations, build decision strategies and analyze data in a fully customizable decision workspace by using what-if analyzes and scenarios. Moreover, in order to reach our second goal, the software is developed so that all communication between the users takes place only through the specific tools of the system (as file sharing, instant messaging, etc). All the actions of the decision makers while using the systems will be logged, thus creating a very important source of knowledge that can be exploited in order to better understand the decision processes. This section is intended to argue the point that in the Decision Support Systems course the usage of a system as VCDE can be invaluable. The students will be able to learn how to research a decisional situation, how to structure the decision making process, how to build decision models that will help them evaluate alternatives, how to actually implement the chosen alternative and how to follow up the results of the decision and make necessary adjustments if the results are not the ones predicted.
In the fourth section of the paper we show the interaction model implemented in VCDE for reaching a collaborative decision. Then we show how logging is actually implemented and an initial experiment that presents a partial log and the resulting decision model. The aim of this section is to show that logging the actions of the users in a DSS is possible and should be done. Such a log can be mined by using either existent or new algorithms in order to create decision models and patterns. New insights are available on the decision making process as the control flow perspective, the organizational perspective and the social networks of collaborative decisions. Those models show how the decision is made, how the decision maker researches a situation, what are the strategies employed, how is the collaboration between decision makers, how is one of the alternatives actually chosen. Creating a model for the decision workflow also offers the possibility of comparing either different decision makers or the actual decision pattern with a prescribed model. The advantages are great, considering that this way we can indicate the process that provided a good alternative and compare it with a decision workflow that led to the choice of the wrong alternative. This way we expect to discover enacted implicit knowledge of the decision maker.
While keeping in mind that the system will first function in the spring semester of 2010 we can identify some shortcomings that still need to be improved. The most important one is that we still have not developed a specific decision mining algorithm and are still using process mining algorithms. On our initial experiment we concluded that process algorithms build fair decision models but based on noiseless, short logs. However, since we expect long logs with multiple incomplete processes we plan to build a specific decision mining algorithm. It is also necessary to develop either an algorithm that will not require log cleaning or a separate script for log cleaning.
The main point we want to argue is our new approach over decision mining. This is a new research direction that aims to explain the decision process based on what the decision maker is actually doing when using the DSS software. What we want to achieve is a large number of logged decision behaviors in the simulated environment that can be exploited by mining for decision patterns and models. If enough models are mined and if the patterns are similar, then we can create reference models for that decisional situation. The models can later be used in conjunction with an evaluation of decision effectiveness.
One other higher purpose of this research is to promote our beliefs that using decision mining and a carefully designed system we can turn decision maker’s implicit knowledge into explicit knowledge that can be captured, reviewed, explained and compared.
This research was founded through Grant type PN2 no. 91-049 / 2007 “Intelligent Systems for Business Decision Support (SIDE)”.