Performance of Varying Genetic Algorithm Techniques in Online Auction

This work describes the agents’ strategies based on different methodologies that employ incompatible rules in collecting information and reproduction. This work used the information collected from the evolutionary framework for economic studies as many researches have attempted to use evolutionary frameworks for economics studies (Nelson, 1995; Epstein & Axtell, 1996; Roth, 2002; Tesfatsion, 2002). This evolutionary approach allows the strategies to be heterogeneous rather than homogenous since only a particular evolutionary approach is applied. This work has shown that the heterogeneous strategies evolved from this framework can be used as a useful research data.


Introduction
Genetic algorithm is one of the successful optimization algorithm used in computing to find exact or approximate solutions for certain complex problems.This novel algorithm was first introduced by John Holland in 1975 (Holland, 1975).Besides Holland, many other researchers have also contributed to genetic algorithm (Davis, 1987;Davis, 1991;Grefenstte, 1986;Goldberg, 1989;Michalewicz, 1992).This is an algorithm that imitates the evolutionary process concept based on the Darwinian Theory which emphasizes on the law of "the survival of the fittest".This algorithm used techniques which are inspired from evolution biology such as inheritance, selection, crossover and mutation (Engelbrecht, 2002).
There are several important components in genetic algorithm which includes representation, fitness function, and selection operators (parent selection and survivor selection, crossover operator and mutation operator).Genetic algorithm starts by generating an initial population of individuals randomly.The individuals are represented as a set of parameter which is the solution to the problem domain.Normally, individuals are fixed length binary string.The individuals are then evaluated using fitness functions.The evaluation will give a fitness score to individuals indicating how well the solutions perform in the problem domain.The individuals that have been evaluated using the fitness function will be selected to be parents to produce offspring through the crossover and mutation operators.The genetic algorithms will repeat the above process except for the population initialization until the termination criteria is met.Fig. 1 shows the structure of a genetic algorithm.
GAs have been applied successfully in many applications including job shop scheduling (Uckun et al. 1993), the automated design of fuzzy logic controllers and systems (Karr 1991;Lee & Takagi, 1993), hardware-software co-design and VLSI design (Catania et al. 1997;Chandrasekharam et al. 1993).In this chapter, variations of genetic algorithms are applied in optimizing the bidding strategies for a dynamic online auctions environment.
Auction is defined as a bidding mechanism and is expressed by a set of auction rules that specify how the winner is determined and how much he or she has to pay (Wolfstetter, 2002).Jansen defines an online auction as an Internet-based version of a traditional auction (Jansen, 2003).In today's e-commerce market, online auction has acted as an important tool in the services for procuring goods and items either for commercialize purposed or for personal used.Online auctions have been reported as one of the most popular and effective ways of trading goods over the Internet (Bapna et al. 2001).Electronic devices, books, computer software, and hardware are among the thousands items sold in the online auctions every day.To date, there are 2557 auction houses that conduct online auctions as listed on the Internet (Internet Auction List, 2011).These auction houses conduct different types of auctions according to a variety of rules and protocols.eBay, as one of the largest auction house alone has more than 94 million registered users and had transacted more than USD 92 billion worth of goods during 2010 (eBay, 2010).These figures clearly show the importance of online auctions as an essential method for procuring goods in today's ecommerce market.
Begin Generation = 0 Randomly Initialize Population While termination criteria are not met Evaluate Population Fitness Crossover Process Mutation Process Select new population Generation = Generation + 1 End Fig. 1.The structure of a Genetic Algorithm The auction environment is highly dynamic in nature.Since there are a large number of online auction sites that can be readily accessed, bidders are not constrained to participate in only one auction; they can bid across several alternative auctions for the same good simultaneously.As the number of auction increases, difficulties such as monitoring the process of auction, tracking bid and bidding in multiple auctions arise when the number of auctions increases.The user needs to monitor many auctions sites, pick the right auction to participate, and make the right bid in order to have the desired item.All of these tasks are somewhat complex and time c o n s u m i n g .T h e t a s k g e t s e v e n m o r e complicated when there are different start and end times and when the auctions employ different protocols.For this reasons, a variety of software support tools are provided either by the online auction hosts or by third parties that can be used to assist consumers when bidding in online auctions.
The software tools include automated bidding software, bid sniping software, and auction search engines.Automated bidding software or proxy bidders act on the bidder's behalf and place bids according to a strict set of rules and predefined parameters.Bid sniping software, on the other hand, is a practice of placing of bid a few minutes or seconds before an auction closes.These kinds of software, however, have some shortcomings.Firstly, they are only available for an auction with a particular protocol.Secondly, they can only remain in the same auction site and will not move to other auction sites.Lastly, they still need the intervention of the user, that is, the user still needs to make decision on the starting bid (initially) and the bid increments.
To address the shortcomings mentioned above, an autonomous agent was developed that can participate in multiple heterogeneous auctions.It is empowered with trading capabilities and it is able to make purchases autonomously (Anthony, 2003;Anthony & Jennings, 2003b).Two primary values that heavily influenced the bidding strategies of this agent are the k and β.These two values correspond to the polynomial function of the four bidding constraints, namely the remaining time left, the remaining auction left, the user's desire for bargain and the user's level of desperateness.Further details on the strategies will be discussed in Section 3. The k value ranges from 0 to 1 while the β value is from 0.005 to 1000.The possible combinations between these two values are endless and thus, the search space for the solution strategies is very large.Hence, genetic algorithms were used to find the nearly optimal bidding strategy for a given auction environment.
This work is an extension of the solution above, which has been successfully employed to evolve effective bidding strategies for particular classes of environment.This work is to improve the existing bidding strategy through the optimization techniques.Three different variations of genetic algorithm techniques are used to evolve the bidding strategies in order to search for the nearly optimal bidding solution.The three techniques are parameter tuning, deterministic dynamic adaptation, and self-adaptation.Each of this method will be detailed in Section 4, 5 and 6.The remainder of the chapter is organized as follow.Section 2 discusses related work.The bidding strategy framework is discussed in Section 3. The parameter tuning experiment is discussed in Section 4. Section 5 and 6 discussed the deterministic adaptive experiment and self-adaptive experiment.A comparison between all the schemes is discussed in Section 7. Finally, the conclusion is discussed in Section 8.

Related work
Genetic algorithm has shown to perform well in the complex system by which the old search algorithm has been solved.This is due to the nature of the algorithms that is able to discover optimal areas in a large search space with little priori information.Many researches in auctions have used genetic algorithm to design or enhance the auction's bidding strategies.The following section discusses works related to evolving bidding strategies.
An evolutionary approach was proposed by Babanov (2003) to study the interaction of strategic agents with the electronic marketplace.This work describes the agents' strategies based on different methodologies that employ incompatible rules in collecting information and reproduction.This work used the information collected from the evolutionary framework for economic studies as many researches have attempted to use evolutionary frameworks for economics studies (Nelson, 1995;Epstein & Axtell, 1996;Roth, 2002;Tesfatsion, 2002).This evolutionary approach allows the strategies to be heterogeneous rather than homogenous since only a particular evolutionary approach is applied.This work has shown that the heterogeneous strategies evolved from this framework can be used as a useful research data.ZIP, introduced by Cliff, is an artificial trading agent that uses simple machine learning to adapt and operate as buyers or sellers in online open-outcry auction market environments (Cliff, 1997).The market environments are similar to those used in Smith's (Smith, 1962) experimental economics studies of the CDA and other auction mechanisms.The aim of each zip agent is to maximize the profit generated by trading in the market.A standard genetic algorithm is then applied to optimize the values of the eight parameters governing the behavior of the ZIP traders which previously must be set manually.The result showed that GA-optimized traders performed better than those populated by ZIP traders with manually set parameter values (Cliff, 1998a;Cliff, 1998b).This work is then extended to 60 parameters to be set correctly.The experiment showed promising result when compared to the ZIP traders with eight parameters (Cliff, 2006).Genetic algorithm is also used to optimize the auction market parameters setting.Many tests have been conducted on ZIP to improve the agent traders and the auction market mechanism using genetic algorithm (Cliff, 2002a;Cliff, 2002b).Thus, ZIP was able to demonstrate that genetic algorithm can perform well in evolving the parameters of bidding agents and the strategies.
In another investigation, a UDA (utility-based double auction) mechanism is presented (Choi et, al. 2008).In UDA, a flexible synchronous double auction is implemented where the auctioneer maximizes all traders' diverse and complex utility functions through optimization modeling based on genetic algorithm.It is a double auction mechanism based on dynamic utility function integrating the notion of utility function and genetic algorithm.The GA-optimizer is used to maximize total utility function, composed of all participants' dynamic utility functions, and matches the buyers and sellers.Based on the experimental result, it performance is better than a conventional double auction.

The bidding strategy framework
As mentioned, this work is an extension of Anthony's work (Anthony, 2003) to tackle the problem of bidding in multiple auctions that employ varying auctions protocols.This section details the electronic marketplace simulation, the bidding strategies and the genetic algorithm implemented in the previous work.

The electronic market place simulation
The market simulation employed three different auction protocols, English, Vickrey and Dutch that run simultaneously in order to simulate the real auction environment.The market simulation is used in this work to evaluate the performance of the evolved bidding strategies.The following section explains how the market simulation works.
The marketplace simulator shown in Fig. 2 consists of concurrent running auctions that employ different protocols.These protocols are English, Dutch and Vickrey.All of these auctions have a known starting time and only English and Vickrey auctions have a known ending time.The bidding agent is given a deadline (t max ) by when it must obtain the desired item and it is told about the consumer's private valuation (p r ) for this item.The agent must only buy an instance of the desired item.
The marketplace announces the current bid values and the current highest bids for English auctions and the current offers for Dutch auctions at each time step.At the end of a given auction, it determines the winning bid and announces the winner.This set of information is used by the agent when deciding in which auction to participate, at what value to bid and in which time to bid.

Bidding strategy
The bidding algorithm for this framework is shown in Fig. 3. Let Item_NA be a boolean flag to indicate whether the target item has already been purchased by the agent.Assume that the value of p r is based on the current reliable market prices observed from past auctions and that the marketplace is offering the item which the agent is interested in.While the bidder agent has not obtained the desired item, the bidder agent needs to build an active auctions list in order to keep track of the current active auction.Active auction is defined as auction that is ongoing or just started but has not reach the ending time yet.In order to build the active auction list, the bidder agent follows the algorithm as shown in Fig. 4. S i (t) is a boolean flag representing the status of auction i at time t, such that i є A and S i (t) є (ongoing; completed).Each auction i є A, has a starting time σ i , and its own ending time η i .The active auction list is built by taking all the auctions that are currently running at time t.In English and Vickrey auctions, any auction that has started but has not reached its ending time is considered as active.S i (t) is used in Dutch auctions since the ending time of this type of auction is not fixed.
After the bidder agent builds the active auctions list, the bidder agent will start calculating the current maximum bid based on the agent strategy.The current maximum bid is defined as the amount of the agent willing to bid at the current time that is lesser than or equal to the agent's private valuation.Four bidding constraints are used to determine the current maximum bid namely the remaining time left, the remaining auction left, the desire for bargain and the level of desperateness.
The remaining time tactic considers the amount of bidding time the bidder agent has to obtain the desire item.This tactic determines the bid value based on the bidding time left.Assuming that the bidding time t is between 0 and t max (0 ≤ t ≤ t max ), the current bid value is calculated based on the following expression: () where () α is a polynomial function of the form: () ( ) This function is a time dependent polynomial function where the main consideration is the time left from the maximum time allocated.k rt is a constant that determines the value of the starting bid of the agent in any auction multiplied by the size of the interval.This time dependent functions can be defined as those that start bidding near p r rapidly to those only bid near p r right at the end along with all the possibilities in between with variation of the value () . Different shapes of curve can be obtained by varying the values of β by using the equation defined above.There are unlimited numbers of possible tactics for each value of β.In this tactic, β value is defined between 0.005 ≤ β ≤ 1000.It is possible to have two different behaviors for β.When β < 1, the tactic will bid with a low value until the deadline is almost reached, whereby this tactic concedes by suggesting the private valuation as the recommended bid value.When β > 1, the tactic starts with a bid value close to the private valuation and quickly reaches the private valuation long before the deadline is reached.Fig. 5 shows the different shape of the curves with varying β values.Fig. 5.The curve with varying β value.(Anthony, 2003) www.intechopen.com The remaining auction left tactic, on the other hand, considers the number of remaining auctions that the bidder agent is able to participate in order to obtain the item.This tactic bids closer to p r as the number of the remaining auctions decreases when the bidder agent is running out of opportunities to obtain the desired item.The current bid value is calculated based on the following expression: The polynomial function ra α is quite similar to the terms use in rt α , whereby the only difference between the two function is the c(t).c(t) is a list of auctions that have been completed between time 0 and t.The β value for this tactic is identical to the remaining time tactic between 0.005 ≤ β ≤ 1000.
The desire for a bargain tactic is the bidder agent that is interested in getting a bargain for obtaining the desired item.In this scenario, the bidder agent needs to take into account all the ongoing auctions and the time left to obtain the item.The current bid value is calculated based on the following expression: In the expression above, the variable () t ω takes into account all the ongoing auctions along with the current bid value.The Dutch and English are considered solely in this expression as only these two auctions have current bid value.As a consequence, the minimum bid value is calculated based on the current bid value and also the proportion of the time left in the auction.These values are summed and averaged with respect to the number of active auctions at that particular time.The expression for () t ω is calculated based on the formula as below: where v i is the current highest bid value in an auction I at time t, and I є L(t); σ i , and η i is the start and end time of auction i The expression for () ba t α is defined as: () ( ) www.intechopen.com The valid range for the constant k ba is 0.1 ≤ k ba ≤ 0.3 and the β value is 0.005 ≤ β ≤ 0.5.The β value is lower than 1 as bidder agent that is looking for bargain will never bid with the behavior of β >1.The β value is, therefore, constantly lower than 1 in order to maintain a low bid until the closes to the end time.Hence, the value of β < 0.5 is used.
The level of desperateness tactic is the bidder agent's desperateness to obtain the target item within a given period and thus, the bidder agent who possesses this behavior tend to bid aggressively.This tactic utilizes the same minimum bid value and the polynomial function as the desire for bargain tactic but with a minor variation to the β and k de value.The valid range for the constant k de for this tactic is 0.7 ≤ k de ≤ 0.9 while the β value is 1.67 ≤ β ≤ 1000.The β value is higher than 1 in this case as the bidder agent that is looking for bargain will never bid with the behavior of β <1.As a result, the β value is always higher than 0.7 since the bidder agent will bid close to the private valuation.
There is a weight associated to each of this tactic and this weight is to emphasize which combination of tactics that will be used to bid in the online auction.The final current maximum bid is based on the combination of the four tactics by making use of the weight.

Representation
Floating point encoding is applied in this particular work as floating point encoding has shown to produce faster, more consistent and more accurate results (Janikow & Michalewicz, 1991).The floating encoding is, therefore, represented using an array of structure.The individuals that are represented in a floating point array structure are shown in Table 1.Relative weight for the remaining auction tactic w ba Relative weight for the desire for a bargain tactic w de Relative weight for the desperateness tactic fitness Fitness score for the individual Table 1.Bidding strategies representation

Representation
Fitness function is an objective function that quantifies the optimality of a solution in a genetic algorithm so that the particular chromosome may be ranked against all the other chromosomes.The main focus of the strategies evaluation in this work is the success rate and average utility of the strategies.Three fitness equations are used to evaluate the performance of the strategies namely the success rate, the agent's utility function and agent's utility with penalty.The success rate is the rate in obtaining the desired item and the second fitness function is the agent's utility () where v represents the winning bid and c is an arbitrary constant 0.001 to ensure that the agent receives some value when the winning bid is equivalent to its private valuation.The third fitness equation involves a variation of the agent utility.If the agent fails to get the item, a penalty that ranges from 0.01 to 0.05 is incurred.Basically, Fitness Equation 1 is used if the delivery of the item is of utmost importance to the user.Fitness Equation 2 is used when the agent is looking for a bargain.Fitness Equation 3 is used when both the delivery of the item and looking for a bargain are equally important.The fitness score is then computed by taking the average utility from a total of 2000 runs.

Selection operators
Elitism is an operator used to retain some number of the best individuals in each generation to the next generation in order to ensure that the fittest individual is not lost during the evolution process (Obitko, 1998).Elitism is applied in this work to retain ten percent of the best individuals to the new population and to ensure that a significant number of the fitter individuals will make it to the next generation.Tournament selection is applied in the genetic algorithm for selecting the individuals to the mating pools for the remaining ninety percent of the population (Blickle & Thiele, 2001).Tournament selection technique was chosen because it is known to perform well in allowing a diverse range of fitter individuals to populate the mating pool (Blickle & Thiele, 1995).By implementing the tournament selection, fitter individuals can contribute to the next generation genetic construction and the best individual will not dominate in the reproduction process compared to the proportional selection.

Crossover process
The extension operator floating point crossover operator is used this work (Beasley et al. 1993b).This operator works by taking the differences between the two values, adding it to the higher value (giving the maximum range), and subtracting it from the lower value (giving the minimum range).The new values for the genes are then generated between the minimum and the maximum range that were derived using this operator (Anthony & Jennings, 2002).

Mutation process
Since the encoding is a floating point, the mutation operator used in this work must be a non-binary mutation operator.Beasley has suggested a few non-binary mutation operators such as random replacement, creep operator and geometric creep (Beasley et al. 1993b) that can be used.The creep operator which adds or subtracts a small randomly generated amount from selected gene is used to allow a small constant of 0.05 to be added or subtracted from the selected gene depending on the range limitation of the parameter (Anthony & Jennings, 2002).

Stopping criteria
The genetic algorithm will repeat the process until the termination criteria are met.In this work, the evolution stops after 50 iterations.An extensive experiment was conducted to determine the point at which the population converges.It was decided to choose 50 as the stopping criterion since it is was observed that the population will always converge before or at the end of the 50 iterations.
Anthony's work has some shortcoming where the crossover and mutation rate used in the work is based on literature review recommended values.However, researches have shown that the crossover rate and mutation rate applied in the application are application dependent, thus, simulation need to be conducted in order to find the suitable crossover and mutation rate.Besides that, other variations of genetic algorithm have proven to perform better that traditional genetic algorithm which is worthwhile to be investigated.

Parameter tuning
Many researchers such De Jong, Grefenstte, Schaffer and others have contributed considerable efforts into finding the parameters values which are good for a number of numerical test problems.The evolution of the bidding strategies by Anthony and Jennings (Anthony & Jennings, 2002) employed a fixed crossover and mutation probability based on the literatures.However, these recommended values may not perform at its best in the genetic algorithm as it has been proven that the parameter values are dependent on the nature of problems to be solved (Engelbrecht, 2002).In this experiment, the crossover and mutation rates are fine tuned with different combination of probabilities in order to discover the best combination of genetic operators' probabilities.Thus, the main objective of this experiment is to improve the effectiveness of the bidding strategies by "hand tuning" the values of the crossover rate and mutation rate to allow a new combination of static crossover and mutation rates to be discovered.By improving the algorithm, more effective bidding strategies can be found during the exploration of the solution.
The experiment is subdivided to two parts.The first one varies the crossover rate and the second one varies the mutation rate.At the end of this experiment, the combination rate discovered is compared and empirically evaluated with the bidding strategies evolved in Anthony's work (Anthony, 2003).

Experimental setup
Table 2 and 3 show the evolutionary and parameter setting for the genetic algorithm.The parameters setting in the simulated environment for the empirical evaluations are shown in Table 4.These parameters include the agent's reservation price; the agent's bidding time and the number of active auctions.The agent's reservation price is the maximum amount that the agent is willing to pay for the item while the bidding time is the time allocated for the agent to obtain the user's required item.The active auctions are the list of auctions that is ongoing before time t max.Fig. 7 shows the pseudocode of the genetic algorithm.

Experimental evaluation
The performance of the evolved strategies is evaluated based on three measurements.Firstly, the average fitness is the fitness of the population at each generation over 50 generations.The average fitness shows how well the strategy converges over time to find the best solution.
Secondly, success rate is the percentage of time that an agent succeeds in acquiring the item by the given time at any price less than or equal to its private valuation.This measure will determine the efficiency of the agent in terms of guaranteeing the delivery of the requested item.Individual will be selected from each of the data set to compete in the simulated marketplace for 200 times.The success is calculated based on the number of time the agent is able to win the item over 200 runs.The formula below is used to calculate the success rate.where p r is the agent's private valuation, n is the number of runs, v i is the winning bid value for auction i.This value is then divided by the agent's private valuation, summed and average over the number of runs.The agent's payoff is 0 if it is not successful in obtaining the item.
A series of experiments was conducted using the set of crossover and mutation rate described in Table 2.It was found that 0.4 crossover rate and 0.02 mutation rate performed better than the other combinations (Gan et al, 2008a, Gan et al, 2008b).An experiment was conducted with the newly discovered crossover rate p c = 0.4 and mutation rate p m = 0.02.The result was then compared with the original combination of the genetic operators' (p c = 0.6 and p m = 0.02).Figures 8, 9 and 10 shows the comparison between the strategies evolved using a combination of crossover rate 0.4 and a mutation rate of 0.02 and the combination of crossover rate 0.6 with a mutation rate of 0.02.The new strategies evolved from the combination of the crossover rate of 0.4 and mutation rate of 0.02 produced better result in terms of the average fitness, the success rate and the average payoff.It can be observed that the mutation rate of 0.02 evolved better strategies when compared to other mutation rates as well (0.2 and 0.002).This rate is similar to the research outcome by Cervantes (Cervantes & Stephen, 2006) in which a mutation rate below the 1/N and error threshold is recommended.Besides, the results of the comparison showed that the combination of 0.4 crossover rate and 0.02 mutation rate can achieve better balance in the exploration and exploitation in evolving the bidding strategies as well.T-test is performed to show the significant improvement of this newly discovered combination of genetic operator probabilities.The symbol of ⊕ in Table 5 indicates that the P-value is less than 0.05 and has significant improvement.

P Value Average Fitness
⊕ Success Rate ⊕ Average Payoff ⊕ Table 5. P value of the t-test statistical analysis for comparison between newly discovered genetic operator probabilities with the old set of genetic operator probabilities  This section investigated the performance of various combinations of predetermined sets of genetic operators' rates in genetic algorithm on a flexible and configurable heuristic decision making framework that is capable to tackle the problem of bidding across multiple auctions that applied different protocols (English, Vickrey and Dutch).As mentioned earlier, the optimal combinations of operators' probabilities of applying these operators are problem dependent.Thus, experiments have to be conducted in order to discover a new operator of combinations genetic operator probability which can improve the effectiveness of the bidding strategy.This experiment has proven that the crossover rate and mutation rate which were applied in the previous work are not the best value to be used in this framework.With this new combination of genetic operators, the experimental evaluation has also shown that the strategies evolved performed better than the other strategies evolved from the other combinations in terms of success rate and average payoff when bidding in the online auction marketplace.By discovering a better combination of genetic operator's probabilities, the improved performance of the bidding strategies as shown in Fig. 8, 9, and 10 are achieved.From this parameter tuning experiment, it can be confirmed that the parameters are problem dependent.However, trying out all of the different combinations systematically is practically impossible as hand tuning the parameter is very time consuming.Therefore, in the second stage of the experiment, deterministic dynamic adaptation is applied to genetic algorithm to evolve the bidding strategies in order to overcome the manual tuning problem.

Deterministic dynamic adaptation
Many researchers have applied deterministic dynamic adaptation in evolutionary algorithms as a method to improve the limitation in the performance of evolutionary algorithms.This type of adaptation alters the value of strategy parameter by using some deterministic rule (Fogarty, 1989;Hinterding et al. 1997).The value of the strategy parameter is modified by the deterministic rule which is normally a time-varying schedule.It is different from the standard genetic algorithm since GA applies a fixed mutation rate over the evolutionary process.Most of the practical applications often favor larger or nonconstant settings of the genetic operators' probabilities.(Back & Schutz, 1996).Some of the studies have proved the usefulness and effectiveness of larger, varying mutation rates (Back, 1992;Muhlenbein, 1992).
In this work, a time-variant dependent control rule is applied to change the control parameters over time without taking into account any present information by the evolutionary process itself (Eiben et al. 1999;Hinterding et al. 1997).Several studies have shown that a time dependent schedule is able to perform better than a fixed constant control parameter (Fogarty, 1989;Hesser & Manner, 1990;Hesser & Manner, 1992;Back & Schutz, 1996).The control rule is used to change the control parameter over the generation of the evolutionary process.The newly discovered crossover and mutation rates from the first experiment will be used in this particular schedule to serve as the midpoint in the time schedule.The parameter step size will change equally over the generation of the evolutionary process as well.This experiment is intended to discover the best deterministic dynamic adaptation by varying the genetic operators' probability scheme in exploring the bidding strategies.
The deterministic increasing and decreasing schemes for the crossover and mutation are different due to the changing scale of the values.The newly discovered crossover rates obtained from Section 3 is used as the midpoint for the time variant schedule because the convergence period of the evolution occur around the 25 th generation.Consequently, the deterministic increasing scheme for the crossover rate will change progressively from p c = 0.2 to p c = 0.6 over the generation whereas the decrease scheme for the crossover rate is vice versa.The mutation rate obtained from the previous experiment is used as the midpoint of the time variant schedule for the increasing and decreasing schemes.The deterministic increasing scheme for the mutation rate, in contrast, will change progressively from p m = 0.002 to p m = 0.2 over the generation and vice versa for the deterministic decreasing schemes.The changing scale during each generation is decided by taking the difference between ranges of the rate divided by the total number of generation.

Experimental setup
Table 6 shows the parameter setting for the deterministic dynamic adaptation genetic algorithm.The evolutionary setting and parameter setting in the simulated environment is the same as Tables 2 and 4. Fig. 11 shows the pseudocode of the deterministic dynamic adaptive genetic algorithm.

Experimental evaluation
The performance of the evolved bidding strategies is evaluated based on three measurements discussed in Section 4.2.As before, the average fitness of the each population is calculated over 50 generations.The success rate of the agent's strategy and the average payoff is observed over 200 runs in the market simulation.
A series of experiments were conducted with the deterministic dynamic adaptation using the testing sets in Table 7. From the experiments, CFMD and CDMI performed better than the other combinations (Gan et al, 2008a, Gan et al, 2008b).Fig. 12 shows that the population evolved with deterministic dynamic adaptation is able to perform a lot better than the fixed constant crossover and mutation rates.This result is similar to the ones observed by other researches where non-constant control parameter performed better than fixed constant control parameter (Back 1992;Back 1993;Back & Schutz 1996;Fogarty 1989;Hesser & Manner, 1991;Hesser & Manner, 1992).Even though, the point of convergence for the different dynamic deterministic scheme is similar, the population with CDMI achieved a higher average fitness when compared to the populations with CFMD.The CDMI scheme with the increase mutation rate is able to maintain exploration velocity in the search space till the end of the run with the decreasing crossover rate achieving a balance between exploitation with the exploration in the search space and also to achieve a balance between exploration and exploitation in this setting.Based on Fig. 13 and Fig. 14 CDMI outperformed CFMF and CFMD in both the success rate and the average payoff.This shows that the strategy evolved by using the CDMI does not only generate a better average fitness but also evolves better effective strategies compared to the strategy evolved for the other deterministic schemes and they are able to gain a higher profit when procuring the item at the end of the auction.It achieved a higher average fitness function during the evolution process as well.This experiment has proven that non-constant genetic probabilities are more favorable than constant genetic probabilities.However, the deterministic dynamic adaptation may change the control parameter without taking into account the current evolutionary process as it does not take feedback from the current state evolutionary process whether the genetic operators' probabilities performed best at that current state of evolutionary process.The third stage of the experiment applies another adaptation method known as selfadaptation.The self-adaptation method is different from the deterministic dynamic adaptation where the self-adaptation evolves the parameter based on the current status of the evolutionary process.The self-adaptation method incorporates the control parameters into the chromosomes, thereby, subjecting them to evolution.In the last stage of the experiment, the self-adaptation is applied to genetic algorithm in order to evolve the bidding strategies.

Self-adaptation
The idea of self-adaptation is based upon the evolving of evolution.Self-adaptation has been used as one of the method to regulate the control parameter.As the name implies, the algorithm controls the adjustment of the parameters itself.This is done by encoding the parameter into the individual genomes by undergoing mutation and recombination.The control parameters can be any of the strategy parameters in evolutionary algorithm such as mutation rate, crossover rate, population size, selection operators and others (Back et al. 1997).However, the encoded parameters do not affect the fitness of the individuals directly, but rather, "better" values will lead to "better" individuals and these individuals will be more likely to survive and produce offspring and hence, proliferating these "better" parameter values.The goal of the self-adaptation is not only to find the suitable adjustment but also to execute it efficiently.The task is further complicated when the optimizer faced by a dynamic problem is taken into account since a parameter setting that was optimal at the beginning of an EA-run might become unsuitable during the evolution process.This scenario has been shown in some of the researches that different values of parameters might be optimal at different stages of the evolutionary process (Back, 1992a;Back, 1992b;Back, 1993;Davis, 1987;Hesser & Manner, 1991).Self-adaptation aims at biasing the distribution towards appropriate regions of search space and maintains sufficient diversity among individuals in order to enable further evolvability (Angeline, 1995;Meyer-Nieberg & Beyer, 2006).
The self-adaptation method has been commonly used in evolutionary programming (Fogel, 1962;Fogel, 1966) and evolutionary strategies (Rechenberg, 1973;Schwefel, 1977) but it is rarely used in genetic algorithms (Holland, 1975).This work applies self-adaptation in genetic algorithm which aims to adjust the crossover rate and mutation rate.The optimal rate for different phases of the evolution is obtained when different self-adaptation is capable in improving the algorithm by adjusting the crossover rate and mutation rate based on the current phase of the algorithm.Researchers have shown that the self-adaptation is able to improve the crossover in genetic algorithm (Schaffer & Morishima, 1987;Spears, 1995).In addition, studies also showed that the self-adaptive mutation rate does perform better than fixed constant mutation rate by incorporating the mutation rate into the individual genomes (Back, 1992a;Back, 1992b).In this section, three different self-adaptation schemes will be tested to discover the best self-adaptation scheme from this testing set.The self-adaptation requires the crossover and mutation rates to be encoded into the individual's genomes.Thus, some modification the encoding representation needs to be performed.The crossover and mutation rate become part of the genomes which will go through the crossover and mutation processes similar to the other alleles.

Experimental setup
Table 8 shows the parameter setting for the self-adaptive genetic algorithm.The evolutionary setting and parameter setting in the simulated environment is same as Table 2 and 4. Fig. 15 shows the pseudocode of the deterministic dynamic adaptive genetic algorithm.Fig. 16 shows the different encoding representation of the individual genome that will be used in the experiment.The crossover and mutation rate are encoded into the representation in order to go through the evolution process.

Experimental evaluation
The performance of the evolved bidding strategies is also evaluated based on the three measurements discussed in Section 4.2.As before, the average fitness of the each population is calculated over 50 generations.The success rate of the agent's strategy and the average payoff is observed over 200 runs in the market simulation.
A series of experiments were conducted with the self-adaptive testing sets described in Table 10.From the experiments, self-adapting both crossover and mutation rates performed better than the other combinations (Gan et al, 2009).The population with self adaptive crossover and mutation (SACM) achieved a higher average fitness compared to the population of self-adaptive crossover (SAC) and self -adaptive mutation schemes (SAM) as shown in Fig. 17.This scenario implies that the population with self adaptive crossover and mutation perform at its best among other populations and this is due to the self-adaptation crossover and mutation scheme which has combined the advantageous of the self-adaptive crossover and self-adaptive mutation scheme together.By having the two parameters to self-adapt, the control parameter can be adjusted to find the solution in different stages with the best control parameter which have been shown in the previous study indicating that different evolution stages will possess different optimal parameter values (Eiben et al. 1999).All of the individuals generated a 4% increase in success rate and average payoff after employing the self adaptive crossover and mutation schemes as shown in Fig. 18 and Fig. 19 This has proven that the strategy evolved by using the self adaptive crossover and mutation does not only generate a better average fitness and success rate but also evolves better effective strategies compared to the strategy evolved for other self adaptive schemes.

Comparison between variations of genetic algorithm
In order to determine which of the three approaches perform the best in improving the effectiveness of the bidding strategies, the best result of each experiment is compared.The comparison is made by choosing the best performing schemes from the parameter tuning, deterministic dynamic adaptation and self-adaptation experiments.The main objective of this work is to improve the effectiveness of the existing bidding strategies by using different disciplines of the genetic algorithm.20 shows the average fitness for the evolving bidding strategy with different disciplines of the genetic algorithm.It can be seen clearly that there is an obvious differences between the convergence points in the different genetic algorithm disciplines.Self-adaptation achieves a higher average fitness compared to benchmark, the newly discovered static rate and deterministic dynamic adaptation.Although average fitness of the self-adaptation and deterministic dynamic adaption is similar, self-adaptation achieves a higher average fitness when compared to deterministic dynamic adaptation.The individuals evolved from the self adaptive genetic algorithm outperformed the other individuals from the other disciplines by delivering a more promising success rate.The strategy evolved is 1% higher than the strategies evolved from the deterministic dynamic adaptation.When compared to the benchmark value, an increase of 4% in the success rate is generated by the strategy which that employed the self-adaptation method.As a result, the strategy evolved from the self adaptive genetic algorithm can evolve better strategies and deliver higher success rate when bidding in online auctions which will eventually, improve the GA in searching for better bidding strategies.
All of the strategies evolved from the self adaptive genetic algorithm outperformed the rest with 2% higher average payoff when compared to the strategies which applied deterministic dynamic adaptation and 4% higher when compared to the strategies from the benchmark.This result obtained indicates that the strategy evolved by using the self adaptive genetic algorithm does not only produce a better average fitness and success rate but also evolves  10 indicates that the P-value is less than 0.05 and has significant improvement.The result of P value in the t-test in Table 10 shows the improvement generated by the self-adaptation is more significant compared to the other disciplines.
Hence, it can be confirmed that self-adaptation is the best discipline in improving the effectiveness of the bidding strategies.

Conclusion
Based on the results of the experiments, the strategies evolved with self adaptive genetic algorithm achieved the most ideal result in terms of success rate and average payoff in an online auction environment setting.The strategies have also achieved a higher average fitness function during the evolution process.
The result in Figure 20, 21, 22 and Table 10 confirmed this conclusion by empirically proving that self adaptive genetic algorithm can evolve better bidding strategies compared to the other genetic algorithm disciplines.Among these different methods, the self-adaptation outperformed all of the other methods due to the nature of the method.In order to achieve better bidding strategies, the self-adaptation crossover and mutation scheme can be used to ensure better bidding strategies which in turn produces higher success rate, average fitness and average payoff.
Further investigation can be conducted by evolving the bidding strategies with two other evolution methods which are the evolution strategies and evolution programming.Evolving the bidding strategies with the evolution programming and evolution strategies may generate interesting result which different from genetic algorithm.A comparison between performances the evolutions strategies, evolution programming and genetic algorithm may produce interesting results.

Fig. 4 .
Fig. 4. Building active auction list algorithms Fig. 6 shows various combinations of the bidding constraints based on the different weight associated to the bidding tactics.It can also be seen that different bidding patterns are generated by varying the value of weights of the bidding constraints.

Fig. 6 .
Fig. 6.Various combinations of the bidding constraints p r Agent's private valuation t max Deadline given to the agent to obtain the desired item k rt k for the remaining time tactic β rt β for the remaining time tactic k ra k for the remaining auction tactic β ra β for the remaining auction tactic k ba k for the desire for a bargain tactic β ba β for the desire for a bargain tactic k de k for the desperateness tactic β de β for the desperateness tactic w rt Relative weight for the remaining time tactic w ra

Fig. 8 .
Fig. 8.Comparison of Average Fitness between the benchmark and the newly discovered rate.

Fig. 9 .
Fig. 9. Success rate for strategies evolved with the benchmark and the newly discovered rate Fig. 12. Comparisons between the average fitness of CFMF, CFMD, and CDMI

Fig. 19 .
Fig. 19.Average payoff for strategies evolved from different self-adaptation schemes

Fig. 20 .
Fig. 20.Average fitness population with different genetic algorithm disciplines Fig.20shows the average fitness for the evolving bidding strategy with different disciplines of the genetic algorithm.It can be seen clearly that there is an obvious differences between the convergence points in the different genetic algorithm disciplines.Self-adaptation

Fig. 21 .
Fig. 21.Success rate for strategies evolved from different genetic algorithm disciplines better effective strategies compared to the other strategies evolved for other disciplines and they have gained higher profit when procuring the item.Table10.P value for the comparison between different disciplines in term of success rate and average payoff The symbol ⊕ in Table

. The bidding agent's algorithm for all i є A if
((t ≥ σ i ) and (t ≤ η i ) or (S i (t) = ongoing) add i to L(t) endif endfor

Table 2 .
Genetic algorithm evolutionary setting

Table 3
. Genetic algorithm parameter setting Agent reservation price 73 ≤ p r ≤ 79 Bidding time for each auction 21 ≤ t max ≤ 50 Number of active auction 20 ≤ L(t) ≤ 45

Table 4 .
Configurable parameters for the simulated marketplace

Table 6 .
Deterministic dynamic adaptation parameter setting

Table 7 .
The Deterministic Dynamic Adaptation testing sets www.intechopen.com

Table 9 .
Fig. 15.The self adaptation algorithm both genetic operators k rt β rt k ra β ra k ba β ba k de β de w rt w ra w ba w de p c p Self-adaptation testing sets m Fig. 16.Encoding of a bidding strategy for self-adaptation crossover and mutation rate www.intechopen.com