Perspective Chapter: Cyclic Generation of Box-Behnken Designs and New Second-Order Designs

Abstract

Box-Behnken designs (BBDs) are three-level second-order spherical designs with all points lying on a sphere, introduced by Box and Behnken, for fitting the second-order response surface models. They are available for 3–12 and 16 factors. Together with the central composite designs for the second-order model, BBDs are very popular response surface designs, especially for 3–7 factors. This chapter introduces an algorithm to produce cyclic generators for BBDs and similar designs, which we call cyclic BBDs (CBBDs). The new CBBDs offer more flexibility in choosing the designs for a specified number of factors. Comparisons between some BBDs and the new CBBDs indicate the superiority of the new CBBDs with respect to multiple design quality measures and graphical tools assessing prediction variance properties. A catalog of 24 new CBBDs, which includes orthogonally blocked CBBDs for 11, 13, and 14 factors, will be given.

1. Introduction

Box-Behnken designs (BBDs) are three-level response surface designs (RSDs), introduced by Box and Behnken [1, 2], to fit a second-order response surface model

$$\mathbf{y}=\mathbf{X}\beta +\epsilon$$E1

For $m$ factors in $n$ runs. Here, ${\mathbf{y}}_{n\times 1}$ is a response vector; ${\mathbf{X}}_{n\times p}$ the model matrix having an intercept term, $m$ main effect (ME) terms, $m$ quadratic effect (QE) terms, and $\left(\begin{array}{c}m\\ 2\end{array}\right)$ 2-factor interaction (2FI) terms; vector ${\beta }_{p\times 1}$ of $p=1+2m+\left(\begin{array}{c}m\\ 2\end{array}\right)$ parameters; and error vector ${\epsilon }_{n\times 1}$ with zero mean and covariance matrix $\mathbf{I}{\sigma }^2$. BBDs are currently available for 3–12 and 16 factors [3]. Except for 11 factors, BBDs can be constructed by superimposing the two-level factorial design onto treatments in each block of a balanced incomplete block design (IBD) or partially balanced IBD. BBDs have the following properties:

1. Each factor has the same number of runs at high (+1) and low ($-1$) levels;

2. All points lie on a sphere of the radius $\rho$ or at the center of the design space;

3. They are rotatable for 4 and 7 factors. Otherwise, they are near-rotatable;

4. They can be orthogonally blocked except for 3 and 11 factors;

5. Let design $D$ be a $n\times m$ design matrix $D$ with $m$ factors $x_1,\dots ,x_m$. Let the row $u$ of the model matrix $\mathbf{X}$ be written as $(1,x_{u1}^2,\dots ,x_{\mathit{um}}^2,$$x_{u1},\dots ,x_{\mathit{um}},x_{u1}{x}_{u2},\dots ,$$x_{u\left(m-1\right)}{x}_{\mathit{um}})$, where $x_{\mathit{ui}}$ is the element in row $u$ and column $i$ of $D$. The information matrix $\mathbf{M}={\mathbf{X}}^{\prime }\mathbf{X}$ (and its inverse) has the following form:

E2

where ${\mathbf{M}}_{11}$ is a square matrix of order $1+m$, and ${\mathbf{M}}_{22}$ is a square matrix of order $m+\left({}_2^m\right)$. For a BBD, ${\mathbf{M}}_{21}=\mathbf{0}$, ${\mathbf{M}}_{12}={\mathbf{0}}^{\prime }$ and ${\mathbf{M}}_{22}=\mathbf{D}$, where $\mathbf{D}$ is a diagonal matrix. Matrix $\mathbf{M}$ in (2) reduces to:

E3

As an example, we construct a 6-factor BBD. Consider an IBD of size $\left(v,k,r\right)$= (6, 3, 4) for six varieties, arranged in blocks of size three, each with three replications per variety. Superimposing a $2^3$ factorial onto the corresponding varieties of this IBD will result in the following 6-factor BBD without center points:

$$\left(\begin{array}{rrrrrr}\pm 1& \pm 1& 0& \pm 1& 0& 0\\ 0& \pm 1& \pm 1& 0& \pm 1& 0\\ 0& 0& \pm 1& \pm 1& 0& \pm 1\\ \pm 1& 0& \pm 1& \pm 1& 0& 0\\ 0& \pm 1& 0& 0& \pm 1& \pm 1\\ \pm 1& 0& \pm 1& 0& 0& \pm 1\end{array}\right).$$

In each row, $\left(\pm 1\pm 1\pm 1\right)$ represents the eight points of a $2^3$ design and $0$ is a column vector of eight 0’s. Czyrski and Sznura [4] applied the 6-factor BBD in the optimization of HPLC separation of fluoroquinolones.

Next, we examine a foldover design in 48 runs (with no center points) generated by four cyclic generators: (−1, 0, 0, −1, 1, 0), (0, 1, 0, 0, 1, 1), (0, 0, 1, −1, 0, −1), and (0, 0, −1, −1, 0, 1). The first generator, for example, cyclically generates six design points:

$$\left(\begin{array}{rrrrrr}-1& 0& 0& -1& 1& 0\\ 0& -1& 0& 0& -1& 1\\ 1& 0& -1& 0& 0& -1\\ -1& 1& 0& -1& 0& 0\\ 0& -1& 1& 0& -1& 0\\ 0& 0& -1& 1& 0& -1\end{array}\right).$$

The four cyclic generators produce 24 runs. The next 24 runs are obtained by folding over the first 24 runs (i.e., changing the signs of the factor levels). All points lie on a sphere of radius $\rho =\sqrt{3}$. It can be shown that these design points are also points in the 6-factor BBD. In this chapter, we call this type of design a cyclic BBD or CBBD.

Each factor of this BBD has half of its runs at the 0-level and the remaining at $\pm 1$ levels. Now assume that the researchers are looking for an alternative spherical design with fewer 0-levels and more $\pm 1$ levels for each factor. This allows the experimenter to increase the volume of the spherical design region by increasing the radius associated with CBBD points. This chapter introduces an algorithm that can generate CBBDs of varying radii. Designs with the same number of factors and runs but with different radii are compared with respect to D-criterion values (or $d$-values), variances of the parameter estimates, and the correlation among the main (ME), quadratic (QE), and interaction (2FI) effects. Concepts, such as rotatability, orthogonal blocking, and spherical designs, are well-described in Box and Behnken [2] and textbooks on response surface methodology, such as Myers et al. [5] or Box and Draper [6].

2. Calculating the elements of $\mathbf{M}$ of a CBBD

The design matrix $D$ of a CBBD has the form ${\left(C_1^{\prime }\dots C_r^{\prime }{0}^{\prime }\right)}^{\prime }$ where $C_1,\dots ,C_r$ are the circulant matrices of order $m$ generated by $r$ generating vectors $c_1,\dots ,c_r$ and $0$ is a matrix containing center points. For the information matrix $\mathbf{M}$ to have the form in (3), the elements $D$ must satisfy the following conditions:

$$\sum _{u=1}^n{x}_{\mathit{ui}}=0\left(\forall i\right)$$E4

$$\sum _{u=1}^n{x}_{\mathit{ui}}{x}_{\mathit{uj}}=0\left(i\ne j\right)$$E5

$$\sum _{u=1}^n{x}_{\mathit{ui}}^2{x}_{\mathit{uj}}=0\left(i\ne j\right)$$E6

$$\sum _{u=1}^n{x}_{\mathit{ui}}^2{x}_{\mathit{uj}}{x}_{\mathit{uk}}=0\left(i\ne j\ne k\right)$$E7

$$\sum _{u=1}^n{x}_{\mathit{ui}}{x}_{\mathit{uj}}{x}_{\mathit{uk}}=0\left(i\ne j\ne k\right)$$E8

$$\sum _{u=1}^n{x}_{\mathit{ui}}{x}_{\mathit{uj}}{x}_{\mathit{uk}}{x}_{\mathit{ul}}=0\left(i\ne j\ne k\ne l\right)$$E9

where $x_{\mathit{ui}}$ is the level of the factor $i$ for run $u$ (Cf. Appendix A of [2]). The condition in (4) implies that $D$ is a balanced design; that is, each column of $D$ has the same number of $+1$ and $-1$. To make $D$ balanced, we just have to restrict the sum of the elements of the generating vectors $c_1,\dots ,c_r$ to 0. As $D$ is constructed from the circulant matrices, conditions (5)–(9) can be written as:

$$\sum _{t=1}^r\sum _{i=1}^{m-1}{c}_{\mathit{ti}}{c}_{t\left(i+j\right)modm}=0\left(1\le j<m\right)$$E10

$$\sum _{t=1}^r\sum _{i=1}^{m-1}{c}_{\mathit{ti}}^2{c}_{t\left(i+j\right)modm}=0\left(1\le j<m\right)$$E11

$$\sum _{t=1}^r\sum _{i=1}^{m-1}{c}_{\mathit{ti}}^2{c}_{t\left(i+j\right)modm}{c}_{t\left(i+k\right)modm}=0\left(1\le j<k<m\right)$$E12

$$\sum _{t=1}^r\sum _{i=1}^{m-1}{c}_{\mathit{ti}}{c}_{t\left(i+j\right)modm}{c}_{t\left(i+k\right)modm}=0\left(1\le j<k<m\right)$$E13

$$\sum _{t=1}^r\sum _{i=1}^{m-1}{c}_{\mathit{ti}}{c}_{t\left(i+j\right)modm}{c}_{t\left(i+k\right)modm}{c}_{t\left(i+l\right)modm}=0\left(1\le j<k<l<m\right)$$E14

where $c_{\mathit{ti}}$ is the value of the factor $i$ on the generating vector $t$. It can be seen that there are $m-1$ summations in (10) and (11), $\left({}_2^{m-1}\right)$ in (12) and (13), and $\left({}_3^{m-1}\right)$ in (14). This explains why the lengths of the vectors $J_q$ and $J$ in Section 3 are $2\left(m-1\right)+2\left({}_2^{m-1}\right)+\left({}_3^{m-1}\right)$.

3. The CBBD algorithm

Our CBBD algorithm is the generalization of the algorithm in Nguyen et al. [7] and Pham et al. [8]. Using the results in Section 2, we present the steps of the algorithm for generating a CBBD for $m$ factors in $n=2\mathit{rm}+n_c$ runs (where $n_c$ is the number of center points) with points on a sphere of radius $\rho$, and $\left(\frac{1}{3}m\le {\rho }^2<m\right)$.

1. Form a matrix $C$ of size $r\times m$. Set $\frac{1}{2}r{\rho }^2$ elements of $C$ to 1, $\frac{1}{2}r{\rho }^2$ to $-1$, and the remaining elements to 0. For each row vector $c_q$ of $C$, form a vector $J_q$ with a length $2\left(m-1\right)+2\left({}_2^{m-1}\right)+\left({}_3^{m-1}\right)$ containing the sums in (10) to (14). Set $J=\sum J_q$. Calculate $f$, the sum of the squares of the elements of $J$.

2. Search for a pair of entries in $C$ such that swapping their positions results in the biggest reduction in $f$. If the search is successful, update $f$ and $C$. This step is repeated until $f=0$, or $f$ cannot be reduced further.

Remarks

1. These two steps make up one trial. Among all trials with $f=0$, we select the CBBD with the highest $D$-criterion value, which is defined as:

   $$d‐\text{value}=\frac{1}{n}{\left|\mathbf{M}\right|}^{1/p}$$E15

   for the information matrix $\mathbf{M}$ and the number of parameters $p$ for the second-order model.

2. There are situations, where there is no CBBD with $f=0$ for particular values of $m,{\rho }^2$ and $r$. In this case, we compute two values $f_1$ and $f_2$, set $f_1$ equal to the sum of squares of the first $2\left(m-1\right)+2\left({}_2^{m-1}\right)$ elements of $J$ (or the first $2\left(m-1\right)+\left({}_2^{m-1}\right)$ elements of $J$) and $f_2$ the sum of squares of the remaining elements. A design is selected if $f_1=0$, $f_2$ is minimum and the $d$-value in (15) is maximum.

3. If $D$ is a foldover design, the sums in Eqs. (11) and (13) will be 0, and the length of the vector $J_q$ and $J$ is shortened to $\left(m-1\right)+\left({}_2^{m-1}\right)+\left({}_3^{m-1}\right)$.

4. BBDs and new CBBDs

Table 1 displays the quality measures of BBDs whose run sizes (excluding the two center runs) are multiples of the number of factors $m$ and 24 CBBDs. Table 1 does not include two BBDs for $\left(m,{\rho }^2\right)$ = (9, 3) and (16,4) due to their over-abundance of 0-factor levels. This table includes $m$ (the number of factors), ${\rho }^2$ (the square of the radius), $n$ (the run size of each BBD which includes two center points), and the quality measures of the designs. These measures are the $d$-value in (15), $v_{\mathrm{Q}},v_{\mathrm{M}}$_, and $v_{\mathrm{I}}$ (the maximum scaled variances of the QEs, MEs, and 2FIs, respectively), $r_{\mathrm{QQ}},r_{\mathrm{QI}},r_{\mathrm{MI}}$_, and $r_{\mathrm{II}}$ (the maximum of the absolute values of the correlations between two QEs, between a QE and a 2-FI, between a ME and a 2FI, and between 2FIs, respectively). Note that $r_{\mathrm{QM}}$ (the correlation between a QE and a ME) and $r_{\mathrm{MM}}$ (the correlation between two MEs) for all designs in Table 1 are always zero.

Table 1.

Quality measures of BBDs, CBBDs, and CBBD*s,†.

^†

Each design run size $n$ includes two center runs. All BBDs can be orthogonally blocked except BBDs for $m=3,11$ factors (3a and 11a). CBBDs require $r=\left(n-2\right)/2m$ cyclic generators. CBBD*s require $r=\left(n-2\right)/m$ cyclic generators.

^‡

The two BBDs for $m=8,9$ (8a and 9a) appear in Box and Behnken [1].

Out of 24 CBBDs in Table 1, there are 15 CBBDs with $f=0$ using the foldover technique with the first half-fraction being balanced with factors having the same number of $\pm 1$’s. The first half-fraction of the CBBDs for 3–7 and for 8–14 factors in this table require four and eight cyclic generators, respectively. Like BBDs, these CBBDs have $r_{\mathrm{QI}}=r_{\mathrm{MI}}=r_{\mathrm{II}}=0$. Also, like BBDs, they can be orthogonally blocked, with each half-fraction forming a block. The four CBBDs that are identical to BBDs in terms of quality measures are the ones for 5, 6, 7, and 12 factors. Note that for 3 and 4 factors, the CBBDs have more runs than the corresponding BBDs, and, hence, provide more error degrees of freedom. Also, the 8-factor BBD requires many more runs (nearly 200) than the CBBD. The BBD for 11 factors cannot be orthogonally blocked, and BBDs for 13 and 14 factors are not available. It is necessary to mention that the designs in Nguyen and Borkowski [9] are not the foldover CBBDs in Table 1, and as such cannot be blocked in the same way.

There are nine CBBDs for 3–8 factors that are constructed without applying the foldover technique to the first half-fraction. We denote these CBBDs as CBBD*s. The CBBD* for three factors requires four cyclic generators, while all others require eight. CBBD*s for 5–8 factors have $f_1=0$ (see Remark 2 of Section 3). These designs cannot be blocked in the same way as the CBBDs in Table 1. They can, however, be nearly orthogonally blocked using suitable software (see [10]).

These CBBDs and CBBD*s offer additional design choices to an experimenter. Comparisons of CBBDs and CCBD*s to the BBDs for the same number of factors and runs indicate that they, in general, have higher $d$-values, smaller variances of the estimates, and smaller $r_{\mathrm{QQ}}$ (the correlation between two different quadratic effects). Figure 1 displays the color cell plots (CCPs) of BBDs for 5–8 factors, that is, 5a, 6a, 7a, and 8a, and the corresponding CBBD*s with ${\rho }^2=m-1$, that is, 5c, 6b, 7d, and 8f. CCPs, proposed by Jones and Nachtsheim [11], display the magnitude of the correlation between the columns of the model matrix $\mathbf{X}$ (in terms of the absolute values). The color of each cell ranges from white (no or near-zero correlation) to dark (one or near-one correlation). It can be seen from these CCPs that the information matrices $\mathbf{M}$ of the mentioned CBBD*s do not have the form in (3), but all QEs are orthogonal to all MEs and 2FIs. Note that the BBD for 8 factors has 194 runs, while the corresponding CBBD* has only 66 runs.

Figure 1.

CCPs for BBDs and CBBD*s with ${\rho }^2=m-1$ ($m=\mathrm{5,6,7,8}$).

Appendices A and B display the cyclic generators of the CBBDs and CBBD*s respectively, in Table 1.

5. FDS plot and VDG comparisons

When assessing the prediction properties of an RSD, we want a design that will produce predicted values ${\widehat{Y}}_{\left(x_1,\dots ,x_m\right)}$ with low variance for points $\left(x_1,\dots ,x_m\right)$ in the design space. The prediction variance at $\left(x_1,\dots ,x_m\right)$ is $var\left({\widehat{Y}}_{\left(x_1,\dots ,x_m\right)}\right)={\sigma }^2\mathbf{x}{\left({\mathbf{X}}^{\text{'}}\mathbf{X}\right)}^{-1}{\mathbf{x}}^{\text{'}}$_, where ${\sigma }^2$ is the error variance and $\mathbf{x}$ is $\left(x_1,\dots ,x_m\right)$ expanded to contain the $m_2$ second-order model terms. Re-scaling by $n/{\sigma }^2$ yields the scaled prediction variance $V\left(x_1,\dots ,x_m\right)=n\mathbf{x}{\left({\mathbf{X}}^{\text{'}}\mathbf{X}\right)}^{-1}{\mathbf{x}}^{\text{'}}$.

Although a design efficiency measure (such as the $d$-value) may provide useful information regarding the overall quality of prediction, it does not provide information regarding the distribution of the prediction variance throughout the design region. This is addressed by studying a design’s spherical prediction variance (SPV) properties.

$V_{\rho }$ is defined to be the average of the scaled prediction variance function taken over $S_{\rho }$, the sphere of radius $\rho$. (See [12]) Thus,

$$V_{\rho }=\frac{1}{{\omega }_{\rho }}{\int }_{S_{\rho }}V\left(x_1,\dots ,x_m\right){\mathit{dx}}_1\dots {\mathit{dx}}_m$$E16

where ${\omega }_{\rho }$ is the surface area of $S_{\rho }$. Also of interest are the minimum and maximum scaled prediction variances defined as:

$${\mathit{\text{VMIN}}}_{\rho }=\underset{\left(x_1,\dots ,x_m\right)\in S_{\rho }}{min}V\left(x_1,\dots ,x_m\right)$$E17

$${\mathit{\text{VMAX}}}_{\rho }=\underset{\left(x_1,\dots ,x_m\right)\in S_{\rho }}{min}V\left(x_1,\dots ,x_m\right)$$E18

Fraction of design space (FDS) plots and variance dispersion graphs (VDGs) will be utilized to assess the prediction variance properties of designs in Table 1. Giovannitti-Jensen and Myers [13] introduced the VDG, which superimposes plots of ${\mathit{\text{VMAX}}}_{\rho }$, ${\mathit{\text{VMIN}}}_{\rho }$_, and $V_{\rho }$ against the radius $\rho$ within a spherical design space. Modified VDGs that also include the SPV values of $V\left(x_1,\dots ,x_m\right)$ for a large set of random points in the spherical region [9] will be presented. Note that the proportion of the volume of the design region is small for values of $\rho$ near-zero but increases rapidly with increase $\rho$. Thus, a large proportion of the design space is associated with a relatively small interval $\rho$ near the design space boundary. To address this issue, Zahran et al. [14] introduced the FDS plot of the quantiles of $V\left(x_1,\dots ,x_m\right)$ against the fraction (or proportion) of the volume of the design region. Unlike single-valued design efficiency measures, both VDGs and FDS plots allow a more thorough assessment throughout the design region. For a summary of graphical methods for assessing the prediction variance properties of RSDs, see Borkowski [15].

Before a comparison of designs using these graphical tools can be made, a critical issue involving factor scaling needs to be addressed. A major difficulty in comparing a BBD to a CBBD or CBBD* with the same design size $n$ is that the design spaces are not the same. For example, consider the BBD with $\left(m,{\rho }^2\right)=$ (5 , 2), that is, 5a. Calculation of $v_Q$, $v_M$_, and $v_I$ is based on the assumption that the design region includes points within the 5-dimensional hypersphere of radius $\sqrt{2}$. However, for the CBBD* with $\left(m,{\rho }^2\right)=$ (5, 4), that is, 5c, the calculation of $v_Q$, $v_M$_, and $v_I$ are based on the assumption that the design region includes points within the 5-dimensional hypersphere of radius $\sqrt{4}$.

Consider the following five-factor experiment presented in Myers et al. [5]. The response to be analyzed is rayon whiteness (RW), which is associated with fabric quality. The experimenters believed that RW can be affected by process variables, which include acid bath temperature in °C (temp1), percent acid concentration (conc1), water temperature in °C (temp2), sulfide concentration (conc2), and amount of chlorine bleach in lb./min (bleach). The experimental levels and the coded levels $x_1,x_2,x_3,x_4,x_5$ for the five variables are as follows:

Coded	Experimental levels
Levels	temp1 (°C)	conc1 (%)	temp2 (°C)	conc2 (%)	bleach (lb/min)
$-1$	35	.3	82	.20	.3
$0$	45	.5	85	.25	.4
$1$	55	.7	88	.30	.5

Table 2 shows the 42 design points for the BBD with ${\rho }^2=2$, the CBBD with ${\rho }^2=3$, and the CBBD* with ${\rho }^2=4$ (designs 5a, 5b, and 5c, respectively). If any 0-factor level is replaced with a value $>0$ or $<0$ in any of these designs, then that point is outside that experiment’s design space. There is an important implicit assumption that the fitted model will be appropriate when extrapolating outside the design space. This can be dangerous because it can not only result in predictions with increased bias but also result in larger prediction variances. Whether or not bias is introduced when extrapolating, increasing variances will occur and can be seen in the comparison of VDGs.

Table 2.

Design points for the five-factor BBD, CBBD, and CBBD*.

The first point (row) in each circulant block of five points generates the other four points cyclically. For the BBD and CBBD, the first 20 points are folded over to form the second 20 points. Each design has two center points to form these 42-point designs.

Therefore, to make comparisons between designs 5a, 5b, and 5c when choosing a design, it is reasonable to assume that the coded factor levels of $-\mathrm{1,0,1}$ representing the same levels when uncoded. This will be true for all design comparisons made for $m=3,\dots ,11$ factors in Table 1.

We begin our comparisons between designs 5a, 5b, and 5c by generating FDS plots and VDGs over the maximum ${\rho }^2$, which are seen in Figure 2. For $m=5$, that would be ${\rho }^2=4$. In the VDGs, vertical reference lines are placed at $\rho =\sqrt{2}$ and $\rho =\sqrt{3}$, which represent the maximum $\rho$ for points in designs 5a and 5b, respectively. The FDS plots are based on the distribution of the SPV values for 10,000 randomly selected points in a sphere of radius $\sqrt{4}$. The 10,000 ($m=\mathrm{4,5,6,7}$) or 20,000 ($m=8$) SPV values are also plotted in the VDGs (as suggested in [9]).

Figure 2.

FDS plots and VDGs for designs with 5 factors ($n=42$). FDS lines: blue for BBD, green for CBBD, and red for CBBD*. VDGs include solid black lines for the minimum, average, and maximum SPV. Vertical reference lines are plotted at $\sqrt{2}$ and $\sqrt{3}$.

To compare the five-factor designs, the VDGs in Figure 2 should be examined over three disjoint intervals for the radius: (i) $\left[0,\sqrt{2}\right]$, (ii) $\left[\sqrt{2},\sqrt{3}\right]$, and (iii) $\left[\sqrt{3},\sqrt{4}\right]$. For (i), the maximum and average SPV is best for the BBD followed by the CBBD* 5c and CBBD 5b. This should not be surprising because every BBD design point is within $\sqrt{2}$ of the origin. However, for (ii) and (iii), it is clear that the CBBD* is best for having smaller maximum, average, and minimum SPV values over $\rho \in \left[\sqrt{2},\sqrt{4}\right]$. These plots indicate that the BBD is best only if the experimenter does not plan to predict the mean response at points with $\rho >\sqrt{2}$ (such as at $\left(\pm 1,\pm 1,\pm \mathrm{1,0,0}\right)$ or $\left(\pm 1,\pm 1,\pm 1,\pm 1,0\right)$). This seems unrealistic. As stated earlier, if any 0-factor level is changed, then the negative consequences of extrapolation must be acknowledged.

Predictions based on the BBD at such points are extrapolations leading to larger SPV values. This is reflected in $\left(v_Q,v_M,v_I\right)=\left(.198,.063,.250\right)$ for the BBD and $\left(.031,.068,.050\right)$ for the CBBD*. These values indicate that the estimated parameter variances associated with the CBBD* are smaller than those for the BBD for $\rho \in \left[\sqrt{2},\sqrt{4}\right]$. Thus, we would expect better predictions with the CBBD*. This is supported by the VDGs and the CBBD* having the largest $d$-value. The CBBD is the least desirable of the $m=5$ factor designs primarily due to the large $v_Q=.208$ value.

Using the comparison approach applied to the five-factor designs, we now summarize the comparison of equal-sized designs for $m=\mathrm{4,6,7}$_, and 8 factors.

For the four-factor designs with $n=34$, the FDS plot and VDG in column 1 of Figure 3 for the CBBD with ${\rho }^2=3$, that is, 4c, are superior to the BBD with ${\rho }^2=2$, that is, 4b, especially over the interval $\left[\sqrt{2},\sqrt{3}\right]$_, where it has the smaller maximum, average, and minimum SPV values. This is expected because no extrapolation occurs over this interval for 4c, while it does for 4b. These plots indicate that design 4b is best only if the experimenter does not plan to predict the mean response at points with $\rho >\sqrt{2}$. This is reflected in the larger $d$-value and smaller $\left(v_Q,v_M,v_I\right)$ for design 4b.

Figure 3.

FDS plots and VDGs for designs with 4 and 6 factors. FDS lines for $m=4$: blue for ${\rho }^2=2$ CBBD and red for ${\rho }^2=3$ CBBD. FDS lines for $m=6$: blue for ${\rho }^2=3$ BBD and red for ${\rho }^2=5$ CBBD*. VDGs include solid black lines for the minimum, average, and maximum SPV. A vertical reference line is plotted at $\sqrt{2}$ for $m=6$ and at $\sqrt{3}$ for $m=6$.

For the six-factor designs with $n=50$, the FDS plot and VDG in column 2 of Figure 3 for the CBBD* with ${\rho }^2=5$ (design 6b) are superior to the BBD/CBBD with ${\rho }^2=3$ (design 6a) for most of the design space. The only exception is for a small fraction of the design space, where ${\rho }^2$ is close to $\sqrt{5}$ and maximum SPV values are larger for design 6b. Despite this small subregion, design 6b has the smaller average and minimum SPV values over the interval $\left[\sqrt{3},\sqrt{5}\right]$_, which comprises most of the spherical volume. Design 6b also has a larger $d$-value and smaller $\left(v_Q,v_M,v_I\right)$.

For seven factors, there are four designs with $n=58$. The FDS plots in Figure 4 indicate that the CBBD* with ${\rho }^2$ = 4, that is, 7b, is the best design over a spherical design space of radius $\sqrt{6}$. The VDGs also indicate that this design has the smallest maximum, average, and minimum SPV values for $\rho >\sqrt{3}$, and based on the concentration of SPV values near the maximum for any $\rho$, the distribution of SPV values is highly skewed-left. The experimenter, however, must realize that beyond $\rho >\sqrt{4}$, extrapolation occurs for 7b and the experimenter is ignoring the possibility that increased bias may exist with predictions when using the fitted model that results from the experimental data. The VDG and FDS plot for 7c indicates that the geometry of the design points in the design space is poor despite having ${\rho }^2=5$. This indicates that in certain cases, a design with a larger ${\rho }^2$ value does not necessarily guarantee a better design. It is important to note, however, that this case is a rare exception. The BBD/CBBD 7a is rotatable. Therefore, the minimum, maximum, and average SPVs are all equal for a given radius. This is reflected in the single curve in its VDG. The VDG for the ${\rho }^2=5$ CBBD* is truncated at SPV = 240 for scale clarity when making VDG comparisons.

Figure 4.

FDS plots and VDGs for designs with 7 factors ($n=58$). FDS lines: blue for BBD/CBBD, green, magenta, and red for CBBD*s with ${\rho }^2=\mathrm{3,4,5,6}$, respectively. VDGs include solid black lines for the minimum, average, and maximum SPV. Vertical reference lines are plotted at $\sqrt{3}$, $\sqrt{4}$ and $\sqrt{5}$.

For eight factors, there are three CBBB* designs with $n=66$. The FDS plots in Figure 5 suggest that the CBBD* with ${\rho }^2$ = 4 (design 8e) is the best design over a spherical design space of radius $\sqrt{7}$. The VDGs also indicate that this design has the smallest maximum and average SPV values for $\rho >\sqrt{3}$. It is important to remind the experimenter that between $\rho =\sqrt{4}$ and $\sqrt{7}$, extrapolation is occurring for 8e. Thus, although 8e appears better than 8f, there may be increased bias with any prediction associated with using a fitted model for 8e in comparison with 8f over this interval. Note that although the minimum SPV curve for ${\rho }^2=7$ CBBD* (design 8f) is the lowest for $\rho >\sqrt{3}$, it is associated with only a small fraction of the design space as evidenced by the sparsity of points near the minimum. The VDG for the ${\rho }^2=3$ CBBD* is truncated at SPV = 375 for scale clarity when making VDG comparisons.

Figure 5.

FDS plots and VDGs for designs with 8 factors ($n=66$). FDS lines: blue, green, and red for CBBD*s with ${\rho }^2=\mathrm{3,4,7}$, respectively. VDGs include solid black lines for the minimum, average, and maximum SPV. Vertical reference lines are plotted at $\sqrt{3}$ and $\sqrt{4}$.

Based on the comparisons for $m=7$and 8 factors, the design with the largest $d$-value is not necessarily the best design when using FDS plots and VDGs as criteria. A larger $d$-value does not ensure a good distribution of SPV values throughout the design space. It should be noted that the best design based on the FDS plots and VDGs always had the smallest $v_Q$ value. That is, those designs are associated with the smallest estimated variances for the quadratic effects (QEs).

What is clear in these comparisons is that there exists a CBBD or a CBBD* that is superior to every BBD of the same size based on $d$-values, FDS plots, and VDGs. This is most likely due to the over-abundance of 0-factor levels in BBDs leading to poor prediction for larger radii.

6. Conclusions

This chapter offers the cyclic-generating approach to create new designs (CBBDs and CBBD*s) as alternatives to existing BBDs. Our new designs offer a compromise between the definitive screening designs [11] (where each factor has just three 0’s) and BBDs (where the number of 0’s for each factor is more than the number of $\pm 1$’s). In addition to quality measures, FDS plots and VDGs were generated to assess the prediction variance properties in $\left(m-1\right)$-dimensional spherical regions. These were used to compare designs of equal size but with varying ${\rho }^2$. The comparisons indicate that for each number of design factors $m$, there exists a CBBD or CBBD* that is superior to a BBD based on these quality measures and graphical methods. Because of extrapolation concerns related to points extending beyond the maximum value of $\rho$ associated with a design, it is stressed that comparisons of BBDs to CBBDs or CBBD*s should take into account for the differences in the spherical design regions based on differing ${\rho }^2$ values. Once implemented, experimental data resulting from a CBBD or CBBD* can be analyzed analogously to a data analysis for a BBD using currently available statistical software. A catalog of the RSDs in Table 1, which includes 15 CBBDs and nine CBBD*s is given at the link https://designcomputing.net/cbbd/.