Variable Selection in Nonlinear Principal Component Analysis

Abstract

Principal components analysis (PCA) is a popular dimension reduction method and is applied to analyze quantitative data. For PCA to qualitative data, nonlinear PCA can be applied, where the data are quantified by using optimal scaling that nonlinearly transforms qualitative data into quantitative data. Then nonlinear PCA reveals nonlinear relationships among variables with different measurement levels. Using this quantification, we can consider variable selection in the context of PCA for qualitative data. In PCA for quantitative data, modified PCA (M.PCA) of Tanaka and Mori derives principal components which are computed as a linear combination of a subset of variables but can reproduce all the variables very well. This means that M.PCA can select a reasonable subset of variables with different measurement levels if it is extended so as to deal with qualitative data by using the idea of nonlinear PCA. A nonlinear M.PCA is therefore proposed for variable selection in nonlinear PCA. The method, in this chapter, is based on the idea in “Nonlinear Principal Component Analysis and its Applications” by Mori et al. (Springer). The performance of the method is evaluated in a numerical example.

1. Introduction

Principal components analysis (PCA) is a popular dimension reduction method and is applied to analyze quantitative data. For PCA to qualitative data, the data are quantified by using optimal scaling that nonlinearly transforms qualitative data into quantitative data. The PCA with optimal scaling is called nonlinear PCA. Nonlinear PCA reveals all qualitative variables uniformly as numerical variables by using optimal scaling quantifiers in the analysis, that is, it can deal with nonlinear relationships among variables with different measurement levels.

Using this quantification, we can consider variable selection in the context of PCA for qualitative data. In PCA for quantitative data, Tanaka and Mori discussed a method called modified PCA (M.PCA) that can be used to compute principal components (PCs) using only a selected subset of variables that represents all of the variables, including those not selected [1]. Since M.PCA includes variable selection procedures in the analysis, if we quantify all the qualitative variables by using the optimal scaling and then apply M.PCA to the quantified data, we can select a reasonable subset of variables from the qualitative data.

In this chapter, we refer to Mori et al. [2] to revisit a variable selection problem in PCA for qualitative data. The proposed method here (we call it nonlinear M.PCA or NL.M.PCA) is an extension of M.PCA so as to deal with a mixture of quantitative and qualitative data. In Section 2 we provide the overview of NL.M.PCA (optimization, the original M.PCA and NL.M.PCA for qualitative data) based on studies by Mori et al. [2], and in Section 3, we apply this method to the customer engagement data [3] to show how it works in the real data and how you use the output from the method for variable selection, and to evaluate the performance of the method.

2. Modified PCA for mixed measurement level data

2.1 Quantification of qualitative data

We must use a suitable quantification method in the context of PCA because we here wish to consider a variable selection problem in PCA. One of the best methods is the optimal scaling in nonlinear PCA. Nonlinear PCA is a method to deal with qualitative data, which estimates the parameters of PCA and quantifies qualitative variables simultaneously by alternating between estimation and quantification. PRINCIPALS of Young et al. [4] and PRINCIPALS of Gifi [5] are algorithms for nonlinear PCA. Here we use PRINCIPALS.

PRINCIPALS is an algorithm using the alternating least squares (ALS) algorithm as follows: Let $\mathbf{Y}$ = (${\mathbf{y}}_1{\mathbf{y}}_2\dots {\mathbf{y}}_p$) be a data matrix of $n$ objects by $p$ categorical variables and let ${\mathbf{y}}_j$ of $\mathbf{Y}$ be a qualitative vector with $K_j$ categories labeled $1,\dots ,K_j$. PRINCIPALS minimizes the loss function

$${\sigma }_L\left(\mathbf{Z},\mathbf{A},{\mathbf{Y}}^{\ast }\right)=\mathrm{tr}\left({\mathbf{Y}}^{\ast }-\widehat{\mathbf{Y}}\right){}^{\top }\left({\mathbf{Y}}^{\ast }-\widehat{\mathbf{Y}}\right)=\mathrm{tr}{\left({\mathbf{Y}}^{\ast }-{\mathbf{ZA}}^{\top }\right)}^{\top }\left({\mathbf{Y}}^{\ast }-{\mathbf{ZA}}^{\top }\right),$$E1

where ${\mathbf{Y}}^{\ast }$ is an optimally scaled matrix form $\mathbf{Y}$, $\mathbf{Z}$ is an $n\times r$ matrix of $n$ component scores on $r$$\left(1\le r\le p\right)$ components, and $\mathbf{A}=\left\{{\mathbf{a}}_1{\mathbf{a}}_2\dots {\mathbf{a}}_r\right\}$ is a $p\times r$ weight matrix that gives the coefficients of the linear combinations. PRINCIPALS alternately makes two estimations: the model parameters $\mathbf{Z}$ and $\mathbf{A}$ for ordinary PCA, and the data parameter for optimally scaled data ${\mathbf{Y}}^{\ast }$.

In the computation of PRINCIPALS, ${\mathbf{Y}}^{\ast }$ are standardized for each variable such as to satisfy restrictions ${\mathbf{Y}}^{\ast \top }{\mathbf{1}}_n={\mathbf{0}}_p$ and $diag\left[\frac{{\mathbf{Y}}^{\ast \top }{\mathbf{Y}}^{\ast }}{n}\right]={\mathbf{I}}_p$. We denote the value $\theta$ estimated the $t$-th iteration by ${\theta }^{\left(t\right)}$. Given the initial data ${\mathbf{Y}}^{\ast \left(0\right)}$ (the observed data $\mathbf{Y}$ may be used as ${\mathbf{Y}}^{\ast \left(0\right)}$ after the above standardization), PRINCIPALS iterates the following two steps:

• Model estimation step: By solving the eigenvalue problem (EVP) of the covariance matrix of ${\mathbf{Y}}^{\ast \left(t\right)}$ ($=\mathbf{S}$)

$$\left[\mathbf{S}-\lambda \mathbf{I}\right]\mathbf{a}=0,$$E2

where $\lambda$ is the eigenvalues, obtain ${\mathbf{A}}^{\left(t+1\right)}$ and compute ${\mathbf{Z}}^{\left(t+1\right)}={\mathbf{Y}}^{\ast \left(t\right)}{\mathbf{A}}^{\left(t+1\right)}$. Update ${\widehat{\mathbf{Y}}}^{\left(t+1\right)}={\mathbf{Z}}^{\left(t+1\right)}{\mathbf{A}}^{\left(t+1\right)\top }$.

• Optimal scaling step: Obtain ${\mathbf{Y}}^{\ast \left(t+1\right)}$ such that

$${\mathbf{Y}}^{\ast \left(t+1\right)}=arg\underset{{\mathbf{Y}}^{\ast \left(t\right)}}{min}\mathrm{tr}{\left({\mathbf{Y}}^{\ast \left(t\right)}-{\widehat{\mathbf{Y}}}^{\left(t+1\right)}\right)}^{\top }\left({\mathbf{Y}}^{\ast \left(t\right)}-{\widehat{\mathbf{Y}}}^{\left(t+1\right)}\right)$$E3

for fixed ${\widehat{\mathbf{Y}}}^{\left(t+1\right)}$ by separately estimating ${\mathbf{y}}_j^{\ast }$ for each variable $j$ under the measurement restrictions on each of the variables. That is, compute ${\mathbf{q}}_j^{\left(t+1\right)}$ for nominal variables as

$${\mathbf{q}}_j^{\left(t+1\right)}={\left({\mathbf{G}}_j^{\top }{\mathbf{G}}_j\right)}^{-1}{\mathbf{G}}_j^{\top }{\widehat{\mathbf{y}}}_j^{\left(t+1\right)},$$E4

where ${\mathbf{q}}_j$ is a $K_j\times 1$ category score vector for ${\mathbf{y}}_j^{\ast }$ and ${\mathbf{G}}_j$ is an $n\times K_j$ indicator matrix

$${\mathbf{G}}_j=\left(g_{\mathit{jik}}\right)=\left(\begin{array}{ccc}{g}_{j11}& \dots & g_{j1K_j}\\ ⋮& ⋮& ⋮\\ g_{\mathit{jn}1}& \dots & g_{{\mathit{jnK}}_j}\end{array}\right)=\left({\mathbf{g}}_{j1}\dots {\mathbf{g}}_{{\mathit{jK}}_j}\right),$$E5

where

$$g_{\mathit{jik}}=\left\{\begin{array}{ll}1& \text{if}\text{object}i\text{belongs}\text{to}\text{category}k\\ 0& \text{if}\text{object}i\text{belongs}\text{to}\text{some}\text{other}\text{category}{k}^{\prime }\left(\ne k\right),\end{array}\right.$$E6

and then the optimally scaled vector ${\mathbf{y}}_j^{\ast }$ is obtained by ${\mathbf{y}}_j^{\ast }$=${\mathbf{G}}_j{\mathbf{q}}_j$.

Re-compute ${\mathbf{q}}_j^{\left(t+1\right)}$ for ordinal variables using the monotone regression [6]. For nominal and ordinal variables, update ${\mathbf{y}}_j^{\ast \left(t+1\right)}={\mathbf{G}}_j{\mathbf{q}}_j^{\left(t+1\right)}$ and standardize ${\mathbf{y}}_j^{\ast \left(t+1\right)}$. For numerical variables, standardize the observed vector ${\mathbf{y}}_j$ and set ${\mathbf{y}}_j^{\ast \left(t+1\right)}={\mathbf{y}}_j$.

These two steps alternately iterate until convergence, and ${\mathbf{y}}_j^{\ast }$ obtained at convergence is the quantified variable while $\mathbf{A}$ and $\mathbf{Z}$ are the solutions of PCA for qualitative data.

2.2 Modified PCA

M.PCA of Tanaka and Mori [1] derives PCs that are computed using only a selected subset but represent all of the variables, including those not selected. This means that M.PCA naturally includes variable selection procedures in its estimation process. Although there are several variable selection methods in PCA, we use M.PCA, because a subset of variables selected by M.PCA can represent all the variables very well and it is easy to incorporate the quantification method in Section 2.1 into M.PCA, which will be described in Section 2.3.

Suppose we obtain an $n\times p$ data matrix $\mathbf{Y}$ that consists of numerical variables or optimally quantified variables. Let $\mathbf{Y}$ be decomposed into an $n\times q$ submatrix ${\mathbf{Y}}_1$ and an $n\times \left(p-q\right)$ submatrix ${\mathbf{Y}}_2$$\left(1\le q\le p\right)$. $\mathbf{Y}$ is represented by $r$ PCs, which is a linear combination of a submatrix ${\mathbf{Y}}_1$, that is, $\mathbf{Z}={\mathbf{Y}}_1\mathbf{A}$, where $r$ is the number of PCs $\left(1\le r\le q\right)$. To derive $\mathbf{A}=\left({\mathbf{a}}_1\dots {\mathbf{a}}_r\right)$, the following Criterion 1 based on Rao [7] and Criterion 2 based on Robert and Escoufier [8] can be used:

(Criterion 1) The prediction efficiency $\mathbf{Y}$ is maximized using a linear predictor in terms of $\mathbf{Z}$.

(Criterion 2) The closeness of configurations between $\mathbf{Y}$ and $\mathbf{Z}$ is maximized using the $\mathit{RV}$-coefficient.

We denote the covariance matrix of $\mathbf{Y}=\left({\mathbf{Y}}_1,{\mathbf{Y}}_2\right)$ as $\mathbf{S}=\left(\begin{array}{ll}{\mathbf{S}}_{11}& {\mathbf{S}}_{12}\\ {\mathbf{S}}_{21}& {\mathbf{S}}_{22}\end{array}\right),$ where the subscript $i$ of $\mathbf{S}$ corresponds to ${\mathbf{Y}}_i$. The maximization criteria for the above Criterion 1 and Criterion 2 are given by the proportion $P$

$$P=\sum _{j=1}^r{\lambda }_j/\mathrm{tr}\left(\mathbf{S}\right),$$E7

and the $\mathit{RV}$-coefficient

$$\mathit{RV}={\left\{\sum _{j=1}^r{\lambda }_j^2/\mathrm{tr}\left({\mathbf{S}}^2\right)\right\}}^{1/2},$$E8

respectively, where ${\lambda }_j$ is the $j$-th eigenvalue with the order of magnitude of the EVP

$$\left[\left({\mathbf{S}}_{11}^2+{\mathbf{S}}_{12}{\mathbf{S}}_{21}\right)-\lambda {\mathbf{S}}_{11}\right]\mathbf{a}=0.$$E9

The solution is obtained as a matrix $\mathbf{A}$, the columns of which consist of the eigenvectors associated with the largest $r$ eigenvalues of EVP (9), and ${\mathbf{Y}}_1$ that provides the largest value of $P$ or $\mathit{RV}$ is the best subset of $q$ variables among all possible subsets of size $q$. Thus, to obtain a reasonable subset of variables with size $q$ in PCA, you apply M.PCA to the data and find the subset of size $q$, ${\mathbf{Y}}_1$, that has the largest $P$ or $\mathit{RV}$. The selected subset ${\mathbf{Y}}_1$ is reasonable in the sense of PCA because it contains information that includes not only the selected variables ${\mathbf{Y}}_1$ but also the deleted ones ${\mathbf{Y}}_2$.

2.3 Modified PCA for mixed measurement level data

M.PCA is a good method to find a reasonable subset of numerical variables as described in the previous section. To select variables from mixed measurement level data by using a criterion in M.PCA, qualitative/categorical variables in the data should be quantified in an appropriate manner. Based on the original idea in ref. [9], considering PRINCIPALS in Section 2.1 and M.PCA in Section 2.2, it is easy to incorporate the quantification (PRINCIPALS) into M.PCA, because we can formulate M.PCA for qualitative data only by replacing the EVP (2)) in the Model estimation step of PRINCIPALS by the EVP (9) to get the model parameters $\mathbf{A}$ and $\mathbf{Z}$ for M.PCA. Thus, M.PCA and optimal scaling are alternately executed until ${\theta }^{\ast }=\mathrm{tr}{\left({\mathbf{Y}}^{\ast }-\widehat{\mathbf{Y}}\right)}^{\top }\left({\mathbf{Y}}^{\ast }-\widehat{\mathbf{Y}}\right)=\mathrm{tr}{\left({\mathbf{Y}}^{\ast }-{\mathbf{ZA}}^{\top }\right)}^{\top }\left({\mathbf{Y}}^{\ast }-{\mathbf{ZA}}^{\top }\right)$ is minimized. This is nonlinear M.PCA or NL.M.PCA.

Here, we rewrite the ALS algorithm of PRINCIPALS as follows—for given initial data ${\mathbf{Y}}^{\ast \left(0\right)}=\left({\mathbf{Y}}_1^{\ast \left(0\right)},{\mathbf{Y}}_2^{\ast \left(0\right)}\right)$ from the original data $\mathbf{Y}$, the following two steps are iterated until convergence:

• Model estimation step: From ${\mathbf{Y}}^{\ast \left(t\right)}=\left({\mathbf{Y}}_1^{\ast \left(t\right)},{\mathbf{Y}}_2^{\ast \left(t\right)}\right)$, obtained ${\mathbf{A}}^{\left(t\right)}$ by solving the EVP (9).

Compute ${\mathbf{Z}}^{\left(t\right)}$ from ${\mathbf{Z}}^{\left(t\right)}={\mathbf{Y}}_1^{\ast \left(t\right)}{\mathbf{A}}^{\left(t\right)}$. Update ${\widehat{\mathbf{Y}}}^{\left(t+1\right)}={\mathbf{Z}}^{\left(t\right)}{\mathbf{A}}^{\left(t\right)\top }$.

• Optimal scaling step: Obtain ${\mathbf{Y}}^{\ast \left(t+1\right)}$ for fixed ${\widehat{\mathbf{Y}}}^{\left(t+1\right)}$ by separately estimating ${\mathbf{y}}_j^{\ast }$ (=${\mathbf{G}}_j{\mathbf{q}}_j)$ for each variable $j$ under the measurement restrictions. Re-compute ${\mathbf{Y}}_j^{\ast \left(t+1\right)}$ by an additional transformation to keep the monotonicity restriction for ordinal variables and skip this computation for numerical variables.

${\mathbf{Y}}^{\ast }=\left({\mathbf{Y}}_1^{\ast },{\mathbf{Y}}_2^{\ast }\right)$ obtained after convergence is an optimally scaled (quantified) matrix of $\mathbf{Y}$, and ${\mathbf{Y}}_1$ corresponding to ${\mathbf{Y}}_1^{\ast }$ is a subset to be selected and ${\mathbf{Y}}_2$ to ${\mathbf{Y}}_2^{\ast }$ is one to be deleted.

NL.M.PCA procedure for fixed $q$ is as described above, but since the variable selection performs M.PCA calculation for $q=p,\dots ,r$ and ${}_p{C}_q$ times to find the best ${\mathbf{Y}}_1$, there are three possible types of selection according to where the quantification is implemented in the computation flow (see Fig. 4.1 in [2]).

The first type (Type 1) is that the quantification is performed only once at first, that is, nonlinear PCA is applied to the data $\mathbf{Y}$ to obtain the quantified data ${\mathbf{Y}}^{\ast }$_, and ordinary M.PCA selection is applied to ${\mathbf{Y}}^{\ast }$. No more quantification is carried out in the selection stage. The second type (Type 2) is that the quantification is carried out every time after the best subset of size $q$ is found in the selection stage. That is, the quantified $\left({\mathbf{Y}}_1^{\ast },{\mathbf{Y}}_2^{\ast }\right)$ based on the best subset of the size $q$ found in the previous selection is used to find the best subset of size $q-1$ or $q+1$ in the next selection. The third type (Type 3) is that the quantification is carried out for every temporary $\left({\mathbf{Y}}_1,{\mathbf{Y}}_2\right)$ in the section stage, that is, NL.M.PCA is performed whenever temporary $\left({\mathbf{Y}}_1,{\mathbf{Y}}_2\right)$ is given to compute its criterion value.

A reasonable subset of size $q$ is given as ${\mathbf{Y}}_1$ corresponding to the best subset ${\mathbf{Y}}_1^{\ast }$ which is finally found at $q$ when the selection procedure is terminated.

3. A numerical example

3.1 Data

The data we analyze here was gathered in the survey about the relationship among customer engagement on “fashion,” “brand,” and “shop staff” [3]. The questions (variables) are divided into three groups based on the purposes for consumption: “Involvement” (16 variables), “Expectations” (35 variables), and “Values” (34 variables). The total number of questions is 85 on a five-level scale and 825 responses are obtained. Ohyabu et al. [3] analyzed this data to find the structure of the customer consciousness, but we use this data simply as sample data for variable selection in PCA without considering the original purpose in ref. [3]. Here we apply NL.M.PCA to the second question group “Expectation” (35 variables) to show the performance of the proposed method. The questions asked in the survey are indicated in the “Question” column of Table 1 and answers (responses) are shown in Table 2.

Group	Item	Question	q = 25
About the fashion	Q	We would like to ask you about your thoughts and behaviors about fashion.
	Q1	I think about fashion by putting on clothes or choosing the clothes.	×
	Q2	I think about fashion by putting on clothes or choosing the clothes.	×
	Q3	I want to know about fashion by putting on clothes or choosing clothes.	×
	Q4	I’m enthusiastic when I think about fashion.	×
	Q5	I’m happy with thinking about fashion.	×
	Q6	I feel relaxed when I think about fashion.	×
	Q7	I’m proud of my fashions when I think about fashion.	×
	Q8	I spend a lot of one’s time when I think about the fashions.
	Q9	I talk about fashion with my friends.
	Q10	I’m checking about SNS or writing comment for fashion.	×
	Q11	I’m posting about a fashion to SNS.	×
About the brand	Q	We would like to ask you about your thoughts and behaviors about fashion brands.
	Q12	I think about the brand by putting on clothes or choosing the clothes.
	Q13	I think about the brand by putting on clothes or choosing the clothes.	×
	Q14	I want to know about the brand by putting on clothes or choosing the clothes.	×
	Q15	I’m enthusiastic when I think about the brand.	×
	Q16	I’m happy when I think about the brand.
	Q17	I feel relaxed good when I think about the brand.	×
	Q18	I’m proud when I think about the brand.	×
	Q19	I spend a lot of one’s time when I think about the brand.	×
	Q20	I always use a specific brand when I wear or choose clothes.
	Q21	I always use the brand when I clothes or choice of clothes.	×
	Q22	I’m checking about SNS or writing comment for the brand.	×
	Q23	I’m posting about a brand to an SNS of mine.	×
About the shop staff	Q	We would like to ask you about your thoughts and behaviors about the staff member
	Q24	I think about the staff member by talking to other staff	×
	Q25	I think about staff members when I speak to other shop staff.
	Q26	I want to know more about shop staff by speaking.	×
	Q27	I’m enthusiastic when I’m talking with staff members.
	Q28	I’m happy when I’m talking with staff members.	×
	Q29	I feel relaxed when I’m talking with staff members.	×
	Q30	I’m proud when I’m talking with shop staff.
	Q31	I spend a lot of time talking with shop staff.	×
	Q32	I always talk to the specific staff member when choosing clothes or putting on clothes.
	Q33	I always talk the specific staff member.	×
	Q34	I’m checking about the specific staff member of SNS or writing comment for the brand.
	Q35	I’m posting about the specific staff member to my SNS.	×

Table 1.

35 questions in “expectation” and 25 selected ones (marked by $\times$ in the right column).

Table 2.

Expectation data (825 responses on 35 variables).

3.2 Output from NL.M.PCA

Table 3 shows the output of NL.M.PCA when NL.M.PCA is applied to Expectation data with $r=5$ of the number of PCs, proportion $P$ as a criterion, and forward-backward stepwise selection and $\mathit{\text{type}}$ 3 quantifications as selection procedures. The number $q$ is the number of selected variables and the value $P$ is the criterion value. ${\mathbf{Y}}_1$|${\mathbf{Y}}_2$ shows that the left side of each row is the question numbers to be selected $\left({\mathbf{Y}}_1\right)$ and the right side to be deleted $\left({\mathbf{Y}}_2\right)$. If you have a specific number $q$ for variables to be used, such as 20, 10, or 2/3 = 24, 1/2 = 18, you can use variables whose numbers are displayed in ${\mathbf{Y}}_1$ at that $q$. If the number of variables to be used is not determined, the proportion $P$ can be used. For example, since the proportion $P$ is 66.95% with all 35 variables, if you want to keep $P$ up to 65%, looking at the row of $P=0.6512$ (i.e., $q=20$), you can use 20 variables in ${\mathbf{Y}}_1$. Alternatively, if the difference between the proportion with all 35 variables and that with selected variables should be less than 1%, 25 variables can be used because $0.6695-0.01=0.6595$, which is the $P$ value at $q=25$. Figure 1 shows the change of $P$ for every $q$. This graph can be used to obtain guidance on the determination of the number of variables. Looking at this graph, if there is a large drop in $P$, the number of variables just before that point can be used (for this data, no particular drop is observed).

Table 3.

Selection results (expectations, $r=5$, proportion $P$, forward-backward stepwise selection, $\mathit{\text{Type}}$ 3).

Figure 1.

Change of the proportion $P$ for every $q$ (from 35 to 5).

When using $\mathit{RV}$, the same considerations are applied, and scatter plots are also considered to see how close the configurations are.

3.3 Results of variable selection

Here we select a subset of variables from 35 variables of Expectation data focusing on the loss of proportion $P$. Suppose we want to keep it under 1%, $q=25$ which is assigned from Table 3 and Figure 1. The selected variables are marked by $\times$ in the right column of Table 1. As far as looking at the variables deleted from each block, two variables {8, 9} from 11 variables in “fashion” block, three variables {12, 16, 20} from 12 variables in “brand” block, and five variables {25, 27, 30, 32, 34} from 12 variables in “shop staff” block are deleted. That is, nine variables are selected from the first two blocks and seven from the third block. It can be stated that the proposed method selects a reasonable subset of variables. Comparing the number of deleted variables in the three blocks, a slightly larger number of variables are removed from the third block, so it is thought that questions on “shop staff” have little information rather than those in the other two blocks and some of them have less significance on the prediction efficiency. From this point of view, we can evaluate the usefulness of each question in the questionnaire.

To evaluate the significance of variables, we observe how many times each variable is selected through the selection for $q=35,\dots ,5$. Extracting the variables selected over $2/3$ times (24 or more), for example, in the “fashion” block, variables {1, 6, 10, 11} were selected. Given the fact that the close-up questions are located close to each other (1 to 3, recognition on fashion, 4 to 7—consciousness on fashion, 8 to 11—activity on fashion), it is generally clear that NL.M.PCA using the proportion $P$ selects variables well-balanced from the close-up questions. Similarly, if the most frequently selected variables (such as the above four items) are considered as the most important questions, they should be involved in future surveys. If variables are selected a few times, they should not be involved in the future. In such a way, there is a possibility to use the selection results to evaluate the questionnaire itself.

4. Concluding remarks

We reconsider a variable selection problem in PCA for qualitative data based on the idea of Mori et al. [2]. For the problem of how to deal with qualitative data, we apply optimal scaling with the ALS algorithm [4] to the qualitative data. For the variable selection in PCA, we use the criteria in M.PCA of Tanaka and Mori [1] for optimally quantified data. That is, the proposed method is an extension of M.PCA by implementing optimal scaling into M.PCA so as to select a subset of qualitative variables. Using this method, since the quantification is done separately for each variable, we can select a subset of variables from mixed measurement level data.

We apply this method to real data from a customer engagement study [3] to select a subset of qualitative variables by using a criterion that maximizes the prediction efficiency. For a case where there is no preassigned number of variables to be selected, it can be suggested to specify the number in such a way that the maximum loss of the efficiency is not over a certain percentage.

As a result, variables are selected in a well-balanced manner from questions asking similar contents, and the selected subset, therefore, provides as much information as possible. It is expected that the nonlinear M.PCA works well for any mixed measurement level data.