Model-Based Evolutionary Operation Design for Batch and Fed-Batch Antibiotic Production Bioprocesses Model-Based Evolutionary Operation Design for Batch and Fed-Batch Antibiotic Production Bioprocesses

The control policy determination for batch and fed-batch antibiotic production bioprocesses is an important practical issue due to the high added value of these bioproducts. Since it is highly desirable to optimize the antibiotic production, several methods have been proposed aimed at this objective. Once having a mathematical model for the bioprocess, the optimization problem can be formulated within the framework of Pontryagin's maximum principle and of the optimal control theory to determinate the best control trajectory for certain key manipulated variables, such as temperature, pH, and substrate feed rate. In this chapter, applications of these model-based techniques to optimize and control antibiotics production bioprocesses are reviewed and new aspects are emphasized. The cases analyzed included the optimization of the substrate feed rate in a fed-batch reactor and of the temperature in a batch reactor during penicillin fermentations. The main contributions of this study were: (i) the proposition of a different procedure for calculating the second switching time of substrate feed rate, (ii) the application of simpler numerical methods to solve the two-point boundary-value problem associated with the temperature profile optimization, and (iii) the demonstration that the non-isothermal operation is more productive in antibiotic than the operation under constant temperature.


Introduction
Improvement in the productivity of many submerged fermentation processes is carried out by manipulating nutritional and physical parameters such as medium composition, agitation speed, aeration rate, pH, and temperature [1,2]. Although the attainment of optimal conditions for a multivariable fermentation process is often tedious, it is possible to undertake a rational procedure by using statistical experimental designs [2].
Experimental designs can be divided in two distinct groups [3][4][5][6]: (i) model-based experimental designs and (ii) statistical experimental designs. In model-based experimental designs, predictions of a mathematical model are used to determine how an experiment or process should be performed, whereas, with statistical experimental designs, these model predictions are not explicitly required.
The optimization and operation of fermentation processes play a key role in the biotechnology industry due to heavy competition among companies. Secondary metabolites, such as antibiotics and other pharmaceutical products, represent an important added value; therefore, improvements in the production of these bioproducts are of great interest to industries. To achieve high-performance operations, the optimization of manipulated variables that affect the fermentation process becomes a significant task.
In general, optimization problems can be classified in two categories: set-point and profile optimizations [7]. Set-point optimization problems involve finding the best set of values of manipulated variables that lead to the maximization of performance indexes [7]. Profile optimization consists of determining temporal or spatial functions (profiles), rather than a point in n-dimensional space, which lend an optimal value to the performance index [7].
In antibiotic fermentation, it is well known that the temperature and pH for the maximum rate of antibiotic production are different from those for the maximum rate of cell growth [8]. In this sense, the implementation of temperature and pH profiles plays an important role in significant improvements in antibiotic production bioprocesses [8].
Since the primary goal of a fermentation process is the cost-effective production of bioproducts, it is important to select the more appropriate operating mode that allows the production of the desired product at a high concentration with a high productivity and yield [9]. Fed-batch bioprocesses have been widely employed for the production of various bioproducts, including primary and secondary metabolites [9]. In the particular case of secondary metabolites, such as antibiotics, the interaction between growth metabolism and product biosynthesis is critically affected by growth-limiting nutrient concentrations. Since both the underfeeding and the overfeeding of nutrients are detrimental to cell growth and product formation, due to the occurrence of phenomena such as cell starvation and catabolite repression, establishing a suitable feeding strategy is crucial in fed-batch bioprocesses [9,10].
A particular time sequence of control variables may be required in order to conduct the bioprocess over time in a trajectory that provides the greatest productivity. This can lead to complex optimal time profiles for the control variables, which are sometimes impossible to be determined purely experimentally. Thus, appropriate mathematical and numerical methods can be applied for the determination of these profiles in order to reduce the experimental effort and the required time for optimization.
The search for the optimal pH, temperature, and substrate feed-rate profiles in batch and fedbatch antibiotic fermentation is a typical problem of optimization and evolutionary operations for which the use of kinetic models and powerful mathematical techniques is essential for their solution [7,10,11]. According to Rani and Rao [12], several approaches for the determination of optimal time profiles for control variables have been reported in the literature [13][14][15]. In these reports, the optimization problem is generally formulated on the basis of Pontryagin's maximum principle, taking as a starting point a phenomenological mathematical model of the bioprocess. For simple mathematical models, the problem can be solved analytically, from the Hamiltonian of the system, by applying an iterative scheme on the control variable to determine the optimal control profile [8,[16][17][18][19].
In this chapter, two studies on the optimization and the evolutionary operation of antibiotic production bioprocesses are revisited, and new results are obtained and highlighted. Such studies report mathematical models of bioprocesses, in conjunction with Pontryagin's maximum principle, to optimize the substrate feed-rate profile for a fed-batch bioreactor and the temperature profile for a batch fermentation in order to maximize the production of antibiotic. The fundamentals of Pontryagin's maximum principle, when applied to the cases analyzed, are also presented.
The aim is to provide a theoretical basis for the application of a model-based methodology that can be used for the optimization and control of bioprocesses from other antibiotics and secondary metabolites with a broad structural diversity and therapeutic activity, including antibacterial, antifungal, antiviral, antitumor, immunosuppressive, antihypertensive, and antihypercholesterolemic compounds.

Case studies: batch and fed-batch antibiotic production bioprocesses
During batch and fed-batch bioprocesses, the state variables (cell, substrate, oxygen and product concentrations, temperature, and pH) change significantly, from initial to final values. This dynamic behavior motivates the development of optimization methods to find the optimal time trajectories for the control variables in order to improve the performance of these bioprocesses.
Two case studies on the optimization of control variables in batch and fed-batch antibiotic production bioprocesses are revisited, and additional results are obtained and presented. The cases studied are those reported by Costa [20] and Constantinides and Mostouffi [17], concerning the optimization of the substrate feed rate in a fed-batch reactor and the temperature in a batch reactor, respectively. These cases are presented and detailed in the following sections.
2.1. Case study #1: determination of the optimal substrate feed-rate profile in fed-batch bioreactor for penicillin production In this case study, the bioprocess of penicillin production by Penicillium chrysogenum is described by the mathematical model presented by Costa [20], which is based on the classical model proposed by Bajpai-Reuss for penicillin fermentation. In this model, the specific growth rate (μ) takes into account diffusional limitations that occur in the filamentous fungal biomass, as described by the Contois model. The specific rate of product formation (π) considers that penicillin production is repressed by high substrate concentrations (catabolic repression), being modeled by the Andrews equation. Penicillin degradation by hydrolysis is also considered, assuming first-order kinetics for this reaction. The specific rate of substrate consumption (σ) is represented by the Herbert-Pirt generalized model, whereby the substrate is consumed for cell growth and maintenance and for product formation. The equations of the full mathematical model are as follows (note that if the dilution rate is used as the control variable, the total mass-balance equation is not required for the optimization problem formulation, thus reducing the equation system dimension): where • X, S, and P denote the concentrations of cell, substrate, and product, respectively; • μ, σ, and π are the specific rates of cell growth, substrate consumption, and product formation; • μ m , B, Y X/S , Y P/S , m, π m , k m , and k i are the parameters of the mathematical model, including kinetic and yield parameters; In matrix notation: The constraints imposed on the control variable u are as follows: where u min = 0 and u max = F max / V . Another constraint concerns the maximum volume of culture (final volume), i.e., V(t f ) = V max = V f , where t f is the final processing time.
The initial conditions are given by The objective of the optimization/control problem is to determine the optimal time profile for the control variable that maximizes the antibiotic concentration at the end of the bioprocess.
According to the Pontryagin's maximum principle, the optimal profile must maximize the Hamiltonian, given by Since f ðXÞ ¼ 3 5 , the following equations are obtained: For optimal control, it is established that • If φðtÞ < 0, then u ¼ u min .
In the singular interval : Since λ ¼ λðtÞ, the following equations can be developed: Substituting Eqs. (21) and (24) into Eq. (22) gives By developing the matrices indicated in the previous equation, one obtains where In the singular interval: From Eq. (64): Substituting Eq. (66) into Eq. (65) results in the following equation: By introducing the expression of λ 2 into the expression of λ 1 , one obtains Substituting λ 1 =αλ 3 and λ 2 =βλ 3 into the expression of dϕ=dt in the singular interval provides The equation QðXÞ ¼ 0 is the expression of the singular arc, which is independent of the adjoint variables λ 1 , λ 2 , λ 3 . In the expression of QðXÞ, the indicated derivatives are given by For the determination of the singular dilution rate (u sin ), one starts from the first derivative of ϕ as follows: where Thus: Substituting the expression of dλ 3 /dt into the expression of the second derivative of φ gives When u = u sin , the second derivative of ϕ is zero, i.e., d 2 ϕ=dt 2 ¼ 0. Thus, The expression of u sin determines how the dilution rate must be manipulated (varied) during the singular interval, being a function only of the state variables. When developed, the final expression obtained for u sin is quite complex and extensive, which was implemented in computational programming language to evaluate the value of this variable over the singular interval.
The condition for stopping the integration of mass-balance equations during the period following the singular interval, which is conducted in batch mode (u = 0), is determined from the fact that for free final time (t f ) problems, how is the case, H(t f ) = 0. Thus As the final conditions of the adjoint variables are λ 1 = 0, λ 2 = 0, and λ 3 = 1, the stopping condition at t = t f is dP/dt = 0.
Thus, the problem-solving algorithm consisted of the following steps: 1. Integrate the mass-balance equations with u = 0 (batch operation) until the values of the state variables satisfy QðXÞ ¼ 0, being the instant that this occurs, the first switching time t 1 ; 2. From instant t 1 , make u = u sin until the reactor volume reaches its maximum value, which corresponds to the second switching time t 2 ; 3. Starting from time t 2 , return the operation of the reactor with u = 0 until the stop condition is reached in t f .
From the data reported by Costa [20] and summarized in Table 1, the mass-balance equations were numerically integrated to determinate t 1 , i.e., the instant at which QðXÞ ¼ 0. According to the data presented in Table 2, this time instant can be determined as t 1 = 28.74 h, since QðXÞ changes its signal around this time value.
Using the computational program developed for the calculation of u sin , it was verified that this control variable was practically constant during the singular interval, being 0.00213 h -1 a value quite representative.
With respect to the second switching time (t 2 ), this was determined from u sin and the initial and final (maximum) volumes as follows: Thus, the singular interval duration is of 97.71 h and the second switching time (t 2 ) is t 2 = t 1 + Δt = 28.74 + 97.71 = 126.45 h. The mass-balance equations were then integrated until this time, and it was verified that, at the end of the integration, the stop condition (dP/dt=0) had been satisfied, dispensing with the complementary period of batch operation and reaching a final product concentration (P) of 6.35 g/L.

Simulation of the penicillin production bioprocess in fed-batch reactor under optimized conditions
The simulation of the bioprocess under optimized conditions was performed using a computer program in FORTRAN language. For the numerical integration of the ordinary differential equations corresponding to the mass balances, the variable-step fourth-order Runge-Kutta-Gill method was used [17]. The full profiles of the state variables during the bioprocess are shown in Figures 1 and 2.
Due to the decoupling between biomass growth and product synthesis, this type of fermentation behaves as a biphasic process. Therefore, characteristic profiles of penicillin fermentation were obtained for the state variables as shown in Figure 1, i.e., a first phase of accumulation of the cell is observed in which the substrate is almost entirely consumed for this purpose, without associated product formation (trophophase). After this growth phase, the fed substrate is practically used for penicillin production since there is no further catabolic repression of the antibiotic synthesis due to the low levels of substrate concentration established in the Kinetic parameters/operating variables Values μ m (h -1 ); B (g-S/g-X) 1.1Â10 -1 ; 6Â10 -3 π m (g g -1 h -1 ); k m (g/L); k i (g/L); k h (h -1 ) 4.0Â10 -3 ; 1.0Â10 -4 ; 1.0Â10 -1 ; 1.0Â10 -2 Y X/S (g/g); Y P/S (g/g); m(g g -1 h -1 ) 0.47; 1.2; 2.9Â10 -2 X 0 (g/L); S 0 (g/L); P 0 (g/L); S f (g/L) 1.3; 69.0; 0.0; 500.0 V 0 (L); V f (L) 8.121; 10.0 Table 1. Values of kinetic parameters and operating variables used in the case study of the optimization of penicillin production in fed-batch reactor (source: [20]).  Table 2. Data of the numerical integration of the mass-balance equations used to determine the first switching time (t 1 ) during the penicillin production in fed-batch reactor.
reactor in this second phase (idiophase). In addition, the kinetic pattern observed is in agreement with that expected for a secondary metabolite, i.e., the production occurs mostly after cell growth.  Regarding the fermentation medium volume in the bioreactor, the behavior of this variable shown in Figure 2 was already expected since, during the batch operation, this volume is constant because there is no addition or removal of fermentation medium to or from the bioreactor. In the fed-batch operation with continuous feed of unfermented medium to the bioreactor at a constant flow rate, the volume increases linearly over time, as shown in Figure 2. The temporal profile exhibited by the control variable u (dilution rate) in a step format is due to the change in the bioreactor operation, from batch (u = 0) to fed-batch (u 6 ¼ 0) mode.

Case study #2
: determination of the optimal temperature profile in a batch bioreactor for penicillin production In this case study, the fungal growth is described by the logistic law, a substrate-independent model for microorganism population dynamics. In addition, the production of penicillin is also modeled considering that the formation of antibiotic is not associated with cell growth, and that the product is degraded by hydrolysis according to a first-order kinetics. The mathematical model, comprising two ordinary differential equations corresponding to the mass balances of cell and product in a batch bioreactor, and containing four parameters, is represented by (more information about this model can be found in Ref. [21]): where t is the time, X is the cell concentration, P is the antibiotic concentration, r X is the cell growth rate, r P is the antibiotic production rate, r h is the product's hydrolysis rate, and μ m , X m , β, and k h are the parameters of the model, according to the following meanings: μ m is the maximum specific growth rate, X m is the maximum possible cell concentration to be achieved, β is the constant of product formation not associated with growth, and k h is the rate constant for the antibiotic hydrolysis reaction.
For the application of the Pontryagin's maximum principle, the model variables were dimensionless and expressions describing the kinetic parameters (b i ) as a function of temperature (θ) were incorporated in order to extend the validity range of the model to non-isothermal conditions. These functions have shapes typical of those found in microbial or enzyme-catalyzed reactions (concave down parabolas). The hydrolysis of the antibiotic is neglected in this dimensionless version of the model, with this version given by the following equations [17]: where • y 1 = dimensionless concentration of cell (-); y 2 = dimensionless concentration of product (-); τ = dimensionless time, 0 ≤ τ ≤ 1 (-) • w 1 ¼ 13:1; w 2 ¼ 0:005; w 3 ¼ 30 C • w 4 ¼ 0:94; w 5 ¼ 1:71; w 6 ¼ 20 C As in this case, gðXÞ ¼ 0 and u = 0, due to the reactor being operated in a batch mode, the massbalances equations are simplified to As previously established in the first case study, the Hamiltonian is given by The temporal variation rates of the adjoint variables λ 1 and λ 2 are formulated as From the previous equation, the following equations can be derived The necessary condition for the optimization of the bioprocess is From the expressions b i = b i (θ), the following derivatives can be obtained By inserting the derivatives of the parameters with respect to the temperature into the expression of ∂H/∂θ = 0, the expression of the optimal temperature profile (θ opt ) is obtained as follows: As previously demonstrated, when the objective is to maximize the antibiotic concentration at the end of the bioprocess, it is necessary that λ 1 (1) = 0 and λ 2 (1) = 1. Since dλ 2 /dt = 0, the second condition requires that λ 2 be constant and equal to 1.0 in the entire time domain, i.e., λ 2 = 1.0 for 0 ≤ τ ≤ 1.
Several numerical methods have been developed to solve this two-point boundary-value problem arising from the application of the maximum principle of Pontryagin to a batch penicillin production bioprocess. Constantinides and Mostoufi [17] used the orthogonal collocation method to solve this problem, justifying that this method is more accurate than the finite difference method. The problem was solved here using a much simpler numerical method than that of the orthogonal collocation to integrate the differential equations, which is the variable-step fourth-order Runge-Kutta-Gill method [17]. Thus, the algorithm for solving the problem consisted of the following steps: 1. Assignment of an initial value for λ 1 (0); 2. Integration of the system of ODEs from τ = 0 to τ = 1 and verification if λ 1 (1) = 0. If not, assign a new value to λ 1 (0) until the final condition is satisfied.
In order to make the computational algorithm autonomous for the determination of λ 1 (0), the Newton-Raphson method [17] was coupled to the numerical integration method, solving the following non-linear algebraic equation:  strict agreement with those reported by Constantinides and Mostoufi [17] when they used the orthogonal collocation method to solve this problem. The value determined for λ 1 at τ = 0 was λ 1 (0) = 3.61210035.
The cell concentration profile shown in Figure 3 depicts the main phases involved in a typical microbial growth curve, i.e., the exponential, stationary, and decline phases. The decline phase is attributed to the negative effects of low temperatures on cell growth. In Figure 4, concerning the penicillin production dynamics, an initial short lag phase can be observed, followed by a transition phase in which penicillin production is initiated, until a final linear production phase is achieved.
According to the presented formulation (bioprocess model and Pontryagin's maximum principle), the optimum temperature profile varies between 20 and 30 C following the curve (a) shown in Figure 5. This profile suggests a variable operating temperature during the growth and penicillin production phases, contradicting the standard industrial operating procedure of maintaining a constant temperature throughout the bioprocess. Particularly during the penicillin production phase, the profile prescribes a decrease in the operating temperature so that a high concentration of antibiotic is reached at the end of the bioprocess. The temperature profile (a) shown in Figure 5 may bring some practical difficulty to its programming/execution. In this context, an approximate profile derived from the exact profile, such as that represented by the curve (b) in Figure 5, may make the temperature programming strategy more feasible. This proposal is in agreement with that reported by Bailey and Ollis [22], i.e., the temperature schedule predicted by these calculations can be closely approximated in industrial practice with little added cost. The approximate profile was built from the following equations, which were based on the analysis of the temperature data generated by the exact profile:  Figure 6. Dimensionless concentration profiles of cell and product during isothermal penicillin fermentations at different temperatures.

Simulation of the penicillin production bioprocess in batch reactor under non-optimized isothermal conditions
A pertinent simulation to be performed is one under isothermal conditions to verify whether this thermal operation mode is, in fact, less productive in penicillin than non-isothermal mode following an optimal temperature profile. For this purpose, simulations were performed for constant temperatures of 20, 25, and 30 C and the results were compared with those obtained with the optimized temperature profile ( Figure 6). Figure 6(a) illustrates the well-known fact that high temperatures (30 C) favor the growth of the fungus, while low temperatures (20 C) favor the synthesis of the antibiotic since relative to the amount of penicillin produced, more and less biomass was accumulated at these respective temperature levels [22]. It is observed in Figure 6(b) that the isothermal operation at an intermediate temperature to those investigated (θ = 25 C) presents a performance very similar to the operation with optimized temperature profile, becoming a viable alternative if the variable temperature strategy cannot be implemented, although with lower productivity, according to the data presented in Table 3.
Despite the fact that operation at a fixed temperature between 24 and 25 C predominates in current industrial practice, the benefits of temperature programming during batch antibiotic fermentations are clear.

Conclusions
In this chapter, the usefulness of the Pontryagin's maximum principle has been demonstrated for the optimization and operation of complex antibiotic production bioprocesses such as those conducted in batch and fed-batch reactors under isothermal/non-isothermal conditions. By applying this principle, it was possible to determine the optimal profile of temperature in batch reactors and substrate feed rate in fed-batch reactors that maximize the antibiotic concentration at the end of the bioprocess. Although having a rather complex mathematical formulation, the Pontryagin's maximum principle can be classified as a powerful and suitable  Table 3. Final values of the dimensionless concentration of cells (y 1 ) and product (y 2 ) in isothermal and non-isothermal penicillin fermentations.
tool for the optimization, control, and model-driven operation of bioprocesses aiming at maximum productivity of bioproducts. However, for the application of this principle, it is necessary to dispose a mathematical model, preferably phenomenological and representative of the bioprocess, in order to evaluate whether or not the solution found for a given problem is feasible. In the present study, two classical phenomenological models of penicillin production bioprocesses were used, together with the Pontryagin's maximum principle, aiming to determine the optimal operating conditions for the production of antibiotic, and the solutions found are considered feasible and can be implemented in real cases. However, a more complete mathematical model, incorporating the medium oxygenation state, could provide better bioprocess control, since the productivity in penicillin fermentations is highly dependent upon dissolved oxygen concentration, with its critical level being around 30% of saturation. In the models used here, the dissolved oxygen concentration was implicitly assumed to be nonlimiting of the bioprocess, making this a rather restrictive hypothesis.