Overview of neural networks applying to sludge bed reactor modeling.

## Abstract

Anaerobic granule sludge is a self-forming biofilm. This biofilm can be developed without the presence of bio-carriers. Anaerobic granule sludge-based technologies are dominant technologies in the field of anaerobic wastewater treatment. Although they are successful technologies, many efforts are still needed for a better understanding of the granules and granule-based reactors because reactor failure can occur. Here, reactor modeling is highly helpful in understanding the performance of anaerobic bioreactors. A model that can accurately model bioprocesses in a sludge bed reactor and predict concentrations of effluent components is valuable. This is because the model can provide insights into the reactor and be useful in reactor control. Current models of granules are models on bioprocesses in a single granule sludge or on the hydrodynamics and biokinetics in a sludge bed reactor. Here, we review advances in reactor model and its applications as well as limitations and further improvements in the models.

### Keywords

- anaerobic
- granule
- model
- wastewater
- sludge
- modeling

## 1. Introduction

The phenomenon of anaerobic sludge granulation was first observed in the 1990s. Extensive experimental works have been implemented since then. The culture conditions for forming the anaerobic granular sludge are well understood [1]. A high upflow velocity (usually >1 m/h) is usually required. The diameters of anaerobic granules can be up to 0.15–4 mm, which results in high free sedimentation velocities, that is, 15–50 m/h. The high sedimentation velocities can make a large amount of highly active granular sludge retained in a bioreactor in a highly efficient way. By 2007, the market share for anaerobic granule sludge-based technologies in the field of anaerobic wastewater treatment was 89%. Anaerobic granular sludge-based technologies have been extensively applied to treat wastewater from different industries, including agriculture, food, beverage, alcohol distillery, pulp, and papermaking.

Bioreactors involved in wastewater treatment are complex systems, and many nonlinear biokinetics occur in the bioreactors. A model that can successfully model bioprocesses in the bioreactors is effective in understanding the bioreactors and their manipulation. Versus aerobic wastewater treatment, modeling an anaerobic wastewater treatment is much more difficult. This chapter summarizes different model strategies for a granular sludge bed reactor. These strategies are beneficial for further model development and applications.

## 2. Bioparticle model

The distribution of microorganisms in an anaerobic granule has big impacts on modeling the bioactivity of this granule. Different microbial structures for granules are identified. A layered and a cluster granular sludge structures are observed [2]. Here, three layers are proposed. The outermost layer includes acidogens and hydrogen-consuming organisms. In the middle layer, hydrogen-producing organisms as well as hydrogen-consuming organisms both exist, while *Methanosaeta* locate in the core layer. In this clustered structure, *Methanosaeta* clusters and zones with syntrophic eubacteria and hydrogenotrophic methanogens scatter in the granule.

A granular sludge bed consists of numeral sludge granules. Modeling substrate degradation in a single sludge granule has other applications. Indeed, understanding bioreactions in a single granule can explain the operation of an entire bioreactor. Two strategies are used to model substrate degradation in a single granule. Modeling strategies are both termed bioparticle models in this study. The bioparticle models are discussed below.

### 2.1 Diffusion-reaction model

A diffusion-reaction model couples mass transfer and substrate degradation kinetics in a single granule. Some assumptions need to be made to establish a diffusion-reaction model. The shape of real granules in reactors is irregular and nonuniform. In addition, the biogas that results from bioprocesses contributes to the formation of pores in the inner space of a granule. Water and biomass are different materials and constitute a granule. Therefore, substrate diffusion in the inner space of a granule is different at different locations. Nevertheless, some assumptions are adopted for building a typical diffusion-reaction model to simplify the difficulty in modeling and ensure model accuracy. The assumptions are listed here: (1) the granules are spherical and uniform; (2) only radial diffusion transport is considered and is described by Fick’s law; (3) the diffusion coefficient is constant; and (4) there are no active biomass gradients in the granules at time zero [3].

A representative granule is assumed in a diffusion-reaction model [3, 4]. A typical diffusion-reaction model is characterized by the following equations:

with two boundary conditions:

where *Si* is the substrate concentration of component i in the granule, *Si,sur* is the substrate concentration of component *i* in the granule surface, *ri* is the volumetric substrate conversion rate in the granule, and *Di* is the diffusion coefficient of substrate *I*; *r* is the distance from the granule center.

The diffusion-reaction model was successfully applied in an anaerobic ammonium oxidation (ANAMMOX) granule [3]. However, the above diffusion-reaction model must be revised accordingly, while other sludge granules are modeled. The ANAMMOX reaction is a simple and single reaction that involves simple substrates. If a complex substrate is involved in a diffusion-reaction model, then a hydrolysis process as well as other downstream processes are involved, and it is hard to calibrate the kinetic parameters for each process. In addition, assumption (4) for a diffusion-reaction model may not be true for complex substrates such as carbohydrates. If a carbohydrate is used as the substrate, then a multilayer model could be a better alternative. In the multilayer model, a granule is divided into three layers: H_{2}, producing acetogens; H_{2}, consuming organisms (*Methanothrix*); and acidogens. Furthermore, the boundary conditions (Eq. (2)) should be revised accordingly, while substrate concentrations at the core of a representative granule are not zero or there is a pore at the core of the representative granule.

### 2.2 Individual-based model

In the other model, substrate degradation can be coupled with the dynamic growth of a sludge granule. In the dynamic growth process, the sludge granule consists of many bacteria, and the granular surface growth and detachment are involved. The model is called an individual-based model (IBM) because the model is based on each single individual bacterium.

The IBM significantly differs from the above diffusion-reaction model. The size and shape of a single granule are not constant in the IBM. Bacteria grow and can be sheared off in the model, which mimics the natural growth of a single granule. The model has clear and active biomass gradients because the growth of different bacterial species interacts with substrate degradation. The IBM can be one-dimensional, two-dimensional, or three-dimensional.

Figure 1 shows the model strategy of the IBM model. The IBM model was applied to model the biofilm development in a reverse osmosis module. This data verified the validity of the IBM model [5, 6]. In principal, any kind of microorganisms can be applied in this model strategy.

The implementation of an IBM model requires a big computational workload because a modeling domain must be divided into numerous micro grids. Hence, the implementation of an IBM model at a reactor scale would require huge computational workload and appear to be impossible.

## 3. Reactor model

### 3.1 Integration of hydrodynamic and biokinetics

#### 3.1.1 Applied hydrodynamic models

The modeling of wastewater treatment at the reactor scale usually requires a hydrodynamic model. The hydrodynamic model tries to explain water flow in a bioreactor. There are two major strategies for constructing a hydrodynamic model. On one hand, a reactor can be treated as a connection of continuous stirred reactors (CSTRs) and/or plug-flow (PF) reactors. This is termed the reactor compartmentalization (RC) strategy. On the other hand, computational fluid dynamics (CFD) can be applied instead of reactor compartmentalization. This is termed the CFD strategy.

#### 3.1.1.1 RC strategy

Many different flow schemes have been applied to model hydrodynamics in granular sludge bed reactor. In each of these models, CSTRs are widely applied to model a sludge bed and a blanket, while a PF reactor is usually applied to model a settler in a reactor. The flow schemes do not have to fit the real physical flow conditions. A flow scheme is considered acceptable if the resulting tracer concentrations fit the tracer concentrations measured at the outlet of reactors [7].

Figure 2 shows that four major flow schemes have been applied to model the hydrodynamics in granular-based reactors [8, 9, 11]. A sludge bed can be modeled by using a combination of a CSTR and a dead volume. The sludge blanket can be modeled via the other CSTR. A bypass flow always starts from the inlet of a sludge bed but ends at different compartments in different flow schemes. The settler can be modeled as a plug-flow reactor. Other flow schemes are also applied but with less applications. An upflow anaerobic sludge bed (UASB) reactor was treated as the connection of several CSTRs, and there was a good agreement between experimental and simulated results. This shows that this variation is acceptable [12].

A CSTR is often applied when a blanket zone is modeled because biomass transport and rising bubbles are two important factors that lead to turbulence in this compartment. In a settler, a degree of mixing can be expected due to movement of rising gas bubbles. Therefore, this zone is modeled as a dispersed plug-flow reactor. The choice of a plug-flow reactor or a CSTR should depend on the flow conditions in the sludge bed. Although many researchers use a CSTR to represent a sludge bed, a plug-flow reactor can be an alternative [13]. Similarly, a sludge bed in a reactor is often modeled as a CSTR when a high recirculation rate is applied [4, 7].

#### 3.1.1.2 CFD strategy

If the characteristics of each sludge granule can be obtained by applying basic equations such as Navier-Stokes equations, then the exact hydrodynamic modeling of a granular sludge bed can be obtained. However, it is impossible to obtain details of each sludge particle; therefore, an exact model cannot be obtained and is not even necessary.

The hydrodynamic modeling of a granular sludge bed by CFD requires certain techniques. On one hand, a sludge bed can be a liquid that is different from water. The sludge bed and the blanket over the sludge bed can then be modeled separately but simultaneously using different approaches [14]. The sludge concentration has a big impact on sludge hydraulic characteristics such as viscosity [15], and the concentration of the sludge decreases along the reactor height even at lab scales [16]. However, Wang et al. overlooked this difference. In contrast, the sludge bed can be treated as a porous bed. Influent penetrates through the porous bed and flows into a blanket over the porous bed [16]. The permeability and porosity of the sludge bed can be measured and then included in the second method that treats the sludge bed as a porous bed.

The CFD strategy has a few advantages over the RC strategy. First, an optimum flow scheme must be selected from many different flow schemes with the RC strategy; however, there is no need to manually divide a real reactor into virtual reactors while applying a CFD strategy. An RC strategy is difficult to establish with a very complex reactor structure, that is, internal circulation reactor. Second, the CFD strategy can provide many more details about water flow. Complex hydraulic calculations were applied to obtain head loss in a granular bed anaerobic baffled reactors [17]. Such reactor details can be simply extracted from a CFD model, which is more efficient than a complicated manual calculation. Third, the RC strategy cannot provide flow details for each part of a reactor; however, CFD can be applied to obtain the details [18, 19, 20].

#### 3.1.2 Applied biokinetics

The biokinetics regarding wastewater treatment are nicely represented by a series of mathematical equations. Either the RC strategy or the CFD strategy is ready to be combined with the equations to model bioprocesses in a bioreactor.

#### 3.1.2.1 Reactor modeling with anaerobic digestion model 1 (ADM1)

The combination of CFD and a simple bioprocess was used to model an expanded granular sludge bed (EGSB) reactor [21]. The biokinetics usually interact with each other. ADM1 is the widely applied model for modeling bioprocesses in anaerobic wastewater treatment. When calibrated, ADM1 can be integrated with hydrodynamic models to obtain an integrated model for reactor modeling. An integrated model can be obtained when a granular sludge bed reactor is treated as a connection of virtual CSTRs and PF reactors and by applying the ADM1 model to each of these virtual reactors. An integrated model was used to model an UASB reactor treating traditional Chinese medicine wastewater [22]. In the integrated model, values for nonsensitive parameters were adopted from public reports, while sensitive parameters were calibrated. Similarly, sensitive parameters were calibrated while modeling a UASB reactor treating wastewater from a molasses-based ethanol distillery [23]. In these two studies, the original form of ADM1 was maintained. However, the ADM1 can be extended to be more practicable. The ADM1 can be extended by extending the number of microbial species [7, 24] or by including new soluble fermentable substrates [25].

#### 3.1.2.2 Reactor modeling with a bioparticle model

Strategies for reactor modeling based on a bioparticle model are reported. However, this reactor modeling strategy is case-specific, and relevant road maps of each strategy are not clearly stated [3, 4, 12]. By cross-checking these models, a general model strategy is summarized below:

First, a representative granular size is assumed and applied to all granules in the model.

Second, a RC strategy is applied to divides a real reactor into a single or a series of virtual reactors, that is, CSTRs and/or PF reactors.

Then, the number of representative granules can be obtained in each virtual reactor in the model by measuring the total sludge mass in a real reactor and calculating the mass of the representative granule.

Fourth, the substrate degradation rates in each virtual reactor are obtained by adding substrate degradation rates of all representative granules in each virtual reactor.

Finally, the substrate degradation rates in each virtual reactor can be added together to obtain a reactor model that models the operation of a real granular sludge bed.

The bioparticle model applied here is a diffusion-reaction model rather than an IBM because the implementation of an IBM will encounter a huge computational workload. In addition, this strategy can be enriched by including other sub-models such as sludge concentration distribution along reactor height and sludge bed expansion at different upflow velocities [4]. Furthermore, the size of a representative granule is manually but carefully selected while applying a bioparticle model. The weakness of reactor modeling with a bioparticle model is that a reactor model cannot be obtained when a bioparticle model cannot be obtained. A bioparticle model has not been convincingly established for complex substrates. Therefore, reactor modeling with a bioparticle model for complex substrate is still difficult.

### 3.2 Neural network modeling

Although a anaerobic digestion model can be applied, model calibration is difficult and laborious, while errors between model results and measured results still cannot be ignored [26, 27]. The nonlinear regression method shows an empirical relationship between effluent chemical oxygen demand (COD) and operation parameters such as organic load and hydraulic retention time (HRT) [28]. This empirical relation can be treated as a weak artificial neural network (ANN). ANN can effectively model nonlinear systems such as bioreactors for wastewater treatment. ANN is a very powerful computational technique for modeling complex nonlinear relationships [29].

There are a few types of ANNs. Table 1 shows that the most popular type for granular sludge bed modeling is back propagation (BP). The main difference between various types of ANNs is the network structure and the method for determining the weights and functions for inputs and neurons [29]. Figure 3 shows the schematic structure of a BP model. In the BP model structure, there is an input layer that is applied for inputting measured data for model training (calibration). An output layer is also required for model results. The selected reactor operation parameters for the input layer and output layer are case-specific. Generally, influent COD and effluent COD are usually applied in an input layer and an output layer to model a wastewater treatment reactor, respectively; pH is not always included. Table 1 shows the selected parameters for the two layers. In addition, a few layers or a hidden layer is applied to bridge the input layer and output layer. The number of hidden layers in an ANN model is usually determined automatically by a trial and error method, while a single hidden layer network is commonly sufficient for most of the problems [29]. Therefore, three layers are generally applied (Table 1). In the structure of an ANN, each layer consists of a few neurons that are shown as circles in Figure 3. The connections between neurons in each two nearby layers are usually determined while training the system [29]. The modeled results in the output layer are different from measured data, and the weights are recalculated until the model results that fit the measured results.

Types | Reactors and wastewater | No. of layers | Inputs | Outputs | Ref |
---|---|---|---|---|---|

BP | UASB, domestic | 3 | TSS, VSS, COD, alkalinity and VFA concentration, T, pH in the influent | COD of effluent | [30] |

BP | UASB, domestic | 3 | BOD, COD, NH_{4}–N, TKN | Effluent BOD and COD | [29] |

BP | EGSB, denitrifying sulfide removal | 4 | NO_{3}^{−}, NO_{2}^{−}, S^{2−}, pH, HRT | Nitrate, nitrite, sulfide acetate | [31] |

BP | UASB, denitrifying sulfide removal | 3 | Influent sulfide, nitrate concentration, S/N mole ratio, pH, and HRT | Sulfide, nitrate removal percentage, sulfate and nitrogen production percentage | [32] |

BP | UASB, pharmaceutical | 3 | Influent COD, HRT, pH, COD loading rate | Effluent COD | [33] |

BP | UASB, cotton textile | 3 | HRT, influent COD, pH, T, alkalinity, VFA, dilution rate, organic load, TSS | COD removal | [34] |

NARX | UASB, bagasse wash | 4 | Influent, flow rate, inlet and outlet COD | Biogas production rate | [35] |

ANFIS | EGSB, corn processing | 5 | Influent COD, Q , TKN, effluent VFA and bicarbonate | Effluent COD | [27] |

BP | UASB, molasses | 3 | OLR, VFA of effluent, influent–effluent alkalinity and pH, T | Biogas production | [36] |

AMIMO | UASB, molasses | — | OLR, TCOD removal rate, influent alkalinity and pH, effluent pH | Biogas and methane production rates | [37] |

Not clear | UASB, molasses | 3 | OLR, influent and effluent pH, T, alkalinity effluent COD and VFA concentrations | Biogas and methane production rates | [38] |

Table 1 shows that BP has been effectively applied to model granular sludge-based reactors. Although these reactors treat different kinds of wastewater, the model results can accurately fit the measured results. However, while reactors are treating the same kind of wastewater, model inputs and output can vary significantly [29, 30, 32]. As a result, different ANN models can be applied to UASB reactors to treat the same kind of wastewater [36, 37, 38].

## 4. Challenges and discussions

A bioparticle model can explain biokinetics in a sludge granule. Although a bioparticle model can theoretically be applied for reactor modeling, ANN and models integrating reactor hydrodynamics and ADM1 are much more suitable.

A model was established for modeling ANAMMOX process in a single granule based on a few assumptions. However, neither the layered structure nor the cluster structure of a single granule can model complex substrate degradation in a single granule. The IBM model successfully modeled VFA degradation in a granule and shows a clustered structure. The IBM model is based on accurate relevant anaerobic kinetics and can hopefully model complex substrate degradation in a single granule.

The influent COD should be nicely characterized when applying the ADM1 for reactor modeling. The ADM1 requires a detailed characterization of influent organic matter. Nevertheless, such a detailed characterization is generally very difficult [25, 26].

Furthermore, there are 86 parameters in the ADM1. While nonsensitive parameter values can be adopted from the literature, sensitive parameters—which vary significantly—must be calibrated, which is extremely time-consuming and laborious [27]. In addition, the mass of microbial species in bioreactors are not measurable, which challenges the implementation of ADM1 [30].

Calibrating ANN models is easier than ADM1. When the measured variables begin to show differences in the response of ANN models, the model can be re-trained using the newer data employed for cross-checking [27]. Numerous applications of ANNs have been successfully utilized in wastewater treatment modeling [38, 39, 40]. This is because of the reliable and robust characteristics of ANNs in capturing the nonlinear relationships between variables (multi-input and output) in complex systems.

The other benefit of applying an ANN model for reactor modeling is that an ANN model does not need well-established biokinetics. Currently, the production of extracellular polymeric substances (EPS) is not well understood—a modified ADM1 still cannot effectively model the production of EPS [41]. Therefore, the ANN can be hopefully applied to model the EPS production in an anaerobic sludge bed to provide better effluent quality modeling. Nevertheless, although an ANN model is convenient and reliable in reactor modeling, an applied ANN model cannot explain reactor operation failure because it always treats a bioreactor as a black box.

ANNs are better and more convenient tools for reactor modeling than the integration of hydrodynamics and ADM1. Nevertheless, a calibrated ADM1 can provide more details regarding reactor operation. A calibrated ADM1 can nicely control reactor operation. An algorithm could be developed for ADM1 calibration considering the difficulty in manual ADM1 calibration and efficient calibration of ANN. These have been successfully achieved [42]. This makes applying ADM1 much easier because parameter calibration is not as difficult as it used to be.

## 5. Conclusions

A bioparticle model is beneficial for providing insights into reactions in the inner space of a granule. Anaerobic ammonia oxidation processes are a simple process and have been modeled in an ANAMMOX granule. However, when complex substrates are involved, a model including relevant bioprocesses in a single granule has not been available. This calls for further research in this field.

The RC and CFD strategies can both be applied to obtain a reactor hydraulic model that can be further integrated with a kinetic model for modeling effluent quality. The RC strategy manually divides a sludge bed reactor into several virtual reactors. The division does not have to fit the real flow conditions in the reactor. Alternatively, the CFD strategy can provide more details for reactor understanding and manipulation while being integrated with a kinetic model.

Parameter calibration for ADM1 is required before being integrated with a hydraulic model—this is a difficult task. Alternatively, most applied BP neural networks can accurately model concentrations of components in effluent, although the involved reactor is still a black box because the BP neural network completely ignores all bioprocesses in the reactor. An algorithm could be programed for ADM1 calibration by applying the high calibrating capacity of the ANN.

## Acknowledgments

This chapter was supported by the Youth Innovation Promotion Association (NO.2019375) and Key Research Project from Chongqing City (cstc2018jszx-zdyfxmX0013).

## Appendices and nomenclature

Reactor modeling with anaerobic digestion model 1

Multiple inputs and multiple outputs

An anaerobic ammonium oxidation

Adaptive neuro-fuzzy inference system

Artificial neural network

Biological oxygen demand

Back propagation

Computational fluid dynamics

Chemical oxygen demand

Continuous stirred reactors

The diffusion coefficient of substrate I

Extracellular polymeric substances

Hydraulic retention time

Individual-based model

Volumetric organic loading rate

Organic loading rate

Plug-flow reactors

Reactor flow rate

The distance from the granule center

Reactor compartmentalization

_{i}

The volumetric substrate conversion rate in the granule

_{i}

The substrate concentration of component i in the granule

_{i,sur}

The substrate concentration of component i in the granule surface

Temperature

Volumetric total chemical oxygen demand

Total Kjeldahl nitrogen

Total suspended solids

Volatile fatty acid

Volatile suspended solids