Open access peer-reviewed chapter

Surveying Sensitive Topics with Indirect Questioning

Written By

Evrim Oral

Reviewed: 17 January 2019 Published: 25 February 2019

DOI: 10.5772/intechopen.84524

From the Edited Volume

Statistical Methodologies

Edited by Jan Peter Hessling

Chapter metrics overview

1,336 Chapter Downloads

View Full Metrics


Data reliability is a common concern especially when asking about sensitive topics such as sexual misconduct, domestic violence, or drug and alcohol abuse. Sensitive topics might cause refusals in surveys due to privacy concerns of the subjects. Unit nonresponse occurs when sampled subjects fail to participate in a study; item nonresponse occurs when sampled subjects do not respond to certain survey questions. Unit nonresponse reduces sample size and study power; it might also increase bias. Respondents, on the other hand, might answer the sensitive questions in a manner that will be viewed favorably by others instead of answering truthfully. Social desirability bias (SDB) has long been recognized as a serious problem in surveying sensitive topics. Various indirect questioning methods have been developed to reduce SDB and increase data reliability, one of them being the randomized response technique (RRT). In this chapter, we will review some of the important indirect questioning techniques proposed for binary responses, with a special focus on RRTs. We will discuss the advantages and disadvantages of some of the indirect questioning techniques and describe some of the recent novel methods.


  • social desirability bias
  • unmatched count technique
  • network scale-up technique
  • nonrandomized response technique
  • randomized response technique

1. Introduction: surveying sensitive topics

Data reliability is a common concern across all studies that use surveys, but more so while asking sensitive questions. Sensitive questions include sacred, private, or potentially exposing information that could be incriminating or discriminating for a respondent, or for the social group that is represented by the respondent [1]. For example, in studies which evaluate exposure to HIV infection, respondents are often asked sensitive questions regarding their opposite- or same-sex sexual practices. As another example, in studies which aim to assess substance use and abuse, respondents might suppress disclosure of their drug and alcohol misuse to avoid embarrassment or potentially harmful/unwanted consequences. Estimating the prevalence of such sensitive attributes is particularly important for health care researchers to build scientific knowledge, create necessary public health interventions, and develop political strategies.

Two problems typically arise while studying sensitive topics, (1) nonresponse rate increase and (2) social desirability bias (SDB), which is defined as the tendency of answering questions in a socially acceptable fashion rather than answering truthfully, occurs. Nonresponse rates can be reduced by utilizing some strategies such as using advance letters, offering incentives, using more experienced interviewers, and making the topic salient to respondents [2, 3, 4, 5, 6]. However, if these strategies are more effective for some particular subpopulations compared to others, a reduction in nonresponse can in reality increase nonresponse bias. Statistical techniques can also be used to minimize the effects of unit nonresponse after the data is collected [7]; yet, none of these approaches can prevent SDB.

Many researchers have suggested using self-administered modes such as mail, web, computer-assisted self-interviewing (CASI), audio computer-assisted self-interviewing (ACASI), telephone audio computer-assisted self-interviewing (T-ACASI), or touchtone data entry (TDE) in order to reduce SDB [8, 9, 10, 11]. However, self-administered modes have their own drawbacks. For example, all self-administered surveys are known to be susceptible to produce low-quality data since they lack the interviewer feedback to help clarify the questions when respondents do not understand them. As another example, it is known that computer-based surveys, such as CASI or ACASI, are mostly completed by younger, more computer savvy respondents, which potentially could introduce bias to estimates. T-ACASI, on the other hand, mainly suffers from high break-offs. There are additional issues about the feasibility of utilizing T-ACASI in survey tools with the elderly [12]. In addition, when surveying disadvantaged populations, self-administered surveys might not be a viable option. In fact, illiteracy, poor vision, respondent preference, or other reasons can cause self-administration not to occur: in a self-administered component of a computer-assisted personal survey, only 79% of CASI cases were actually fully self-administered [13]. For an extensive review of advantages and disadvantages of some of the common survey modes, see Smith and Kim [14].

An effective, alternative way to improve response rates and prevent SDB simultaneously is to increase the perceived privacy of the respondents. If the respondents’ privacy can be guaranteed, then their tendency to refuse to participate and/or provide untruthful answers would decrease. All indirect questioning techniques aim to achieve this goal via different approaches. In the next section, we review some of the indirect questioning techniques that have been developed to increase the perceived privacy of the respondents, where the characteristic under study (the outcome) is binary in nature. Some of these techniques explained here have been extended to the cases where the characteristic under study is quantitative or polychotomous in nature; here we will only focus on the binary outcomes, such as in yes/no type of questions. Note that all indirect questioning techniques have produced extensive research areas: they all have been modified and/or extended since they have been first introduced; here we briefly summarize the most important ones and present their main aspects for conciseness.


2. Indirect questioning techniques

Several indirect surveying methodologies have been developed to increase respondents’ confidentiality when the characteristic under study has a sensitive nature. Among them, the most commonly used ones for binary outcomes are, namely, unmatched count technique (UCT), network scale-up technique (NST), nonrandomized response technique (NRRT), and randomized response technique (RRT).

UCT, which is also called the item count technique, was first introduced by Raghavarao and Federer [15] with the name “block total response procedure,” but it was formally developed by Miller [16] in the form that we use today. Since then, UCT has been applied by many researchers such as Miller et al. [17], LaBrie and Earleywine [18], Biemer and Brown [19], Wolter and Laier [20], Gervais and Najle [21], and so forth. UCT provides privacy by embedding a sensitive behavior (which is of interest) within several nonsensitive behaviors. All nonsensitive behaviors and sensitive behavior should be binary outcomes (yes/no). In applying the technique, survey participants are randomly divided into two groups. Individuals in one group are provided with a list of nonsensitive behaviors (say, k items), whereas individuals in the other group are provided with the same list of nonsensitive behaviors plus one sensitive behavior that is under study (in total k + 1 items). Participants in both groups are asked to report the total number of activities that they have engaged in. The prevalence of the sensitive behavior is then estimated by calculating the difference between the two independent group means. Several researchers have modified and/or extended UCT in recent years such as Tsuchiya [22], Chaudhuri and Christofides [23], Hussain et al. [24], Ibrahim [25], and so on. As an example on how to apply UCT, consider the study of Zimmerman and Langer [26] in which the technique was utilized to improve the prevalence estimates of marihuana use and ever having had sex with a person of the same gender among the tenth grade students in the Miami-Dade County Public Schools. In their study, the sample (1524 students) was randomly assigned to receive one of three lists on the survey: 33% of the respondents received a list with four nonsensitive items only; 33% received a list with the four nonsensitive items plus the sensitive item about sex with someone of the same gender; 33% received a list with the four nonsensitive items plus the sensitive item about ever using marihuana. Their UCT estimate of prevalence of having sex with someone of the same gender was 80% higher than the prevalence from direct questioning (DQ) (15.9 vs. 8.7%); similarly, their UCT estimate of marihuana use was 21% higher than the prevalence from DQ (20.6 vs. 16.7%). Assuming that for socially undesirable behaviors, higher prevalence estimates are more accurate than the lower prevalence estimates; Zimmerman and Langer [26] concluded that their results demonstrate the superiority of UCT compared to DQ.

The NST was first proposed by Bernard et al. [27] in order to estimate the size of a population at risk and was first used to get an estimate of the number of victims killed in the 1985 Mexico City earthquake [28]. The method was later refined and used to estimate the HIV-seropositive persons in the USA [29]. NST basically involves two steps: (1) the personal network size of the members of a random sample of a population is estimated and (2) an estimate of the number of members of the hidden subpopulation is obtained using the information from step 1. The method heavily relies on the assumption that people’s social networks on average are representative of the general population; for example, if respondents report knowing 500 people on average, five of whom are sex workers, we can estimate that 1% of the general population is sex workers. The estimated prevalence is then combined with known information about the size of the general population, say the population of the USA, to produce an estimate for the number of people in the USA who are sex workers (see Russell et al. [30] for more details on NST and its limitations).

NRRT was first introduced by Swensson [31] and later modified by Takahasi and Sakasegawa [32]. The main idea behind Swensson’s NRRT was to combine a nonsensitive behavior with a sensitive behavior in the same question so that it would not be possible for the interviewer to know which behavior is being responded with a “yes” answer; and therefore, respondents are provided with some level of privacy. Swensson’s NRRT requires two independent samples to calculate the estimate of the sensitive characteristic; for this purpose, survey participants can be randomly divided into two groups. Let U be the nonsensitive behavior, such as being married, and let A be the sensitive behavior under study, such as abusing prescription drugs. Let us demonstrate the combined question using Table 1 given below.

A (Do you use prescription pain relievers without a doctor’s prescription?) U (Are you married?)
Yes No
Yes a b
No c d

Table 1.

Demonstration of Swensson’s NRRT.

Respondents in the first group receive the question “Do you belong to one of the groups a, b, or c in the table?” whereas respondents in the second group receive the question “Do you belong to one of the groups a, b, or d in the table?” Realize that the probability of belonging to groups a, b, c, or d is P (A = 1, U = 1), P(A = 1, U = 0), P(A = 0, U = 1), or P(A = 0, U = 0), respectively. If we denote the unknown proportion of the population members who have the sensitive characteristic A, i.e., the prevalence of the sensitive characteristic, by π A , and let

X ij = 1 , if the respondent i replies with a   '' yes '' in sample j 0 , if the respondent i replies with a   '' no '' in sample j

where 0 π A 1 , i = 1 , 2 , n j , and j = 1 , 2 , then the probability of getting a “yes” response from sample 1 becomes

P X i 1 = 1   =   P A = 1 U = 1   +   P A = 1 U = 0   +   P A = 0 U = 1 E1

and the probability of getting a “yes” response from sample 2 becomes

P X i 2 = 1 = P A = 1 U = 1   +   P A = 1 U = 0   +   P A = 0 U = 0 . E2


P A = 1 U = 1   +   P A = 1 U = 0 = P A = 1 ,


P A = 0 U = 1   +   P A = 0 U = 0 = P A = 0 ,

using the Eqs. (1) and (2), we can write

P X i 1 = 1   +   P X i 2 = 1 = 2 P A = 1 + P A = 0 ,


P A = 1 = P X i 1 = 1   +   P X i 2 = 1 1 . E3

From Eq. (3), π A can be estimated using the proportions of “yes” responses, which are calculated from the samples as:

π ̂ AS = π ̂ 1 + π ̂ 2 1 E4

where π ̂ j = i = 1 n j X ij / n j , j = 1 , 2 . One can easily show that the estimator given by (4) is an unbiased estimator of π A :

E π ̂ AS = 1 n 1 E i = 1 n 1 X i 1 + 1 n 2 E i = 1 n 2 X i 2 1 = 1 n 1 i = 1 n 1 P X i 1 = 1 + 1 n 2 i = 1 n 2 P X i 2 = 1 1 = 2 P A = 1 + P A = 0 1 = π A .

The variance of the estimator given in Eq. (4) is derived in a few steps:

Var π ̂ AS = 1 n 1 2 Var i = 1 n 1 X i 1 + 1 n 2 2 Var i = 1 n 2 X i 2 = 1 n 1 2 i = 1 n 1 P X i 1 = 1 1 P X i 1 = 1 + 1 n 2 2 i = 1 n 2 P X i 2 = 1 1 P X i 2 = 1 = P X i 1 = 1 1 P X i 1 = 1 n 1 + P X i 2 = 1 1 P X i 2 = 1 n 2 ,

which can be estimated by

Var ̂ π ̂ AS = π ̂ 1 1 π ̂ 1 n 1 + π ̂ 2 1 π ̂ 2 n 2 . E5

Swensson’s NRRT was later modified by Takahasi and Sakasegawa [32] as follows: In the first stage, all respondents are asked a nonsensitive binary question, such as “If you have to choose between adopting a cat or a dog, which would you prefer?” but directed not to report their answers to the interviewer (replies are silent). In the second stage, the sensitive behavior is combined with the previous nonsensitive question and asked the respondents in the format given below:

  • If you are a dog person, and use prescription pain relievers without a doctor?s prescription, say 0.

  • If you are a dog person, and do not use prescription pain relievers without a doctor?s prescription, say 1.

  • If you are a cat person, and use prescription pain relievers without a doctor?s prescription, say 1.

  • If you are a cat person, and do not use prescription pain relievers without a doctor?s prescription, say 0.

In this NRRT, to be able to obtain the estimates, the nonsensitive and sensitive behaviors need to be independent (U and A should not be related). As we explain in the next section, if the nonsensitive binary question’s prevalence in the population is known, then Takahasi and Sakasegawa’s NRRT is equivalent to Warner’s model. If the nonsensitive binary question’s prevalence in the population is not known, then this method requires the use of two independent samples. By relaxing the assumption of independence of the nonsensitive and sensitive behaviors, Takahasi and Sakasegawa [32] proposed an additional NRRT as well, where the nonsensitive behavior has three outcomes instead of two; as a result, in this NRRT, three independent samples are required instead of two independent samples. For a comprehensive review of NRRTs, interested readers are referred to Tian and Tang [33].

Different than the previous indirect questioning techniques, RRTs provide confidentiality by utilizing a randomization device, such as a deck of playing cards, a pair of dice, or a spinning game wheel while asking the sensitive question. The interviewer does not observe which playing card is chosen or what numbers the dice show, thus a respondent’s privacy is absolutely guaranteed by the RRT process because the interviewer cannot know which statement the participant is responding to. In the next section, we review some of the main RRTs that have been proposed as well as some of the recently developed techniques.


3. Randomized response techniques

The first RRT was introduced by Warner [34]. Warner’s RRT asks the sensitive question by providing respondents a randomization device with two statements on it that appear with known probabilities θ and 1−θ. For example, consider a spinning game wheel which divided into two sectors with areas θ and 1−θ. The first statement on the device possesses the sensitive characteristic, and the other statement is simply the complementary of the first one. Let us suppose that the sensitive characteristic is being HIV+. The two statements on the wheel would read “I am HIV+” and “I am HIV-” (see Figure 1). The respondents spin the wheel in private and answer truthfully with a “yes” or “no” to the statement on which the arrowhead lands. Since the spinning process is not observed by the interviewer, this model assures privacy protection for the respondents [34].

Figure 1.

Demonstration of the randomization device in the Warner’s model.

Let the unknown proportion of the population members who have the sensitive characteristic A be denoted by π A , and let

X i = 1 , if the respondent i replies with a   '' yes '' 0 , if the respondent i replies with a   '' no '' E6

where 0 π A 1 and i = 1 , 2 , n ; then the probability of getting a “yes” response can be written as

P X i = 1 = π A θ + 1 π A 1 θ , E7

and, from Eq. (7), an unbiased estimator of π A can be derived as

π ̂ AW = π ̂ + θ 1 2 θ 1 ,

where θ 0.5 and π ̂ = i = 1 n X i / n . The variance of Warner’s estimator can be derived as [34]

Var π ̂ AW = π A 1 π A n + θ 1 θ n 2 θ 1 2 . E8

Now, let us compare Swensson’s NRRT with Warner’s model. For simplicity, let us assume that n 1 = n 2 = n / 2 and the prevalence of U (say, the proportion of the married people in the population) is known from previous studies. Let us also assume that U and A are not related behaviors. In Swensson’s NRRT, if we denote the prevalence of U by p, the probability of getting a “yes” response from samples 1 and 2 can be written as

P X i 1 = 1 = π A + P A = 0 P U = 1 = π A + 1 π A p


P X i 1 = 2 = π A + P A = 0 P U = 0 = π A + 1 π A 1 p ,

respectively, assuming that the nonsensitive and sensitive behaviors are independent. Since we have

P X i 1 = 1 1 P X i 1 = 1 = π A 1 π A 1 2 p + p 1 π A 1 p 1 π A


P X i 2 = 1 1 P X i 2 = 1 = p 1 π A 1 p 1 π A ,

the variance of π ̂ AS can be simplified as

Var π ̂ AS = 2 n P X i 1 = 1 1 P X i 1 = 1 + P X i 2 = 1 1 P X i 2 = 1 = 2 n π A 1 π A p 2 + 1 p 2 + 2 1 π A p 1 p .                               ( 9 )

In order to compare Warner’s RRT with Swensson’s NRRT, we calculated theoretical relative efficiencies (REs)

RE θ , p = Var π ̂ AW Var π ̂ AS

from Eqs. (8) and (9) for various combinations of π A and θ = p values using a FORTRAN code and providing our results in Table 2. The FORTRAN code used is given in the Appendix.

πA θ = p
0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45
0.1 0.445 0.489 0.590 0.765 1.067 1.630 2.854 6.360 25.298
0.2 0.495 0.546 0.650 0.828 1.138 1.717 2.978 6.592 26.122
0.3 0.524 0.588 0.702 0.892 1.219 1.831 3.162 6.974 27.577
0.4 0.545 0.624 0.754 0.963 1.320 1.984 3.427 7.558 29.878
0.5 0.564 0.662 0.813 1.052 1.455 2.201 3.818 8.446 33.445
0.6 0.585 0.708 0.891 1.175 1.650 2.527 4.424 9.848 39.145
0.7 0.615 0.775 1.009 1.370 1.969 3.072 5.455 12.263 49.028
0.8 0.667 0.899 1.234 1.749 2.600 4.164 7.541 17.188 69.271
0.9 0.817 1.256 1.892 2.867 4.480 7.444 13.843 32.120 130.806

Table 2.

Relative efficiencies of the estimators from Warner’s model and Swensson’s NRRT for various π A and θ = p values. The shaded cells show the cases where Warner’s model is more efficient than Swensson’s model.

One can conclude from Table 2 that Warner’s model is more efficient, i.e., has smaller variance, than Swensson’s NRRT for small θ = p values. When θ = p 0.25 , however, Warner’s estimator π ̂ AW loses its efficiency. The reason for this efficiency loss is due to the fact that when θ 0.5 , the variance in Eq. (8) . However, keep in mind that small θ values are not preferable in Warner’s model since they do not provide sufficient privacy protection [33, 35]. In order to further investigate how Warner’s RRT compete with Swensson’s NRRT, we also assumed the case where θ p and calculated the theoretical REs as given above, for the case where π A values were set to 0.1, 0.2, and 0.3. The results are provided in Table 3. Realize that we only present the RE values for 0.05 p 0.5 and 0.05 θ 0.45 since

RE θ , p = RE 1 θ , p = RE θ , 1 p = RE 1 θ , 1 p .
p θ
0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45
πA = 0.1
0.05 0.445 0.691 1.049 1.601 2.516 4.200 7.840 18.239 74.394
0.10 0.315 0.489 0.743 1.133 1.781 2.974 5.551 12.913 52.672
0.15 0.251 0.389 0.590 0.901 1.416 2.365 4.414 10.268 41.882
0.20 0.213 0.330 0.501 0.765 1.203 2.008 3.748 8.720 35.567
0.25 0.189 0.293 0.445 0.679 1.067 1.781 3.324 7.733 31.543
0.30 0.173 0.268 0.407 0.621 0.976 1.630 3.043 7.078 28.870
0.35 0.162 0.251 0.382 0.583 0.916 1.529 2.854 6.640 27.085
0.40 0.155 0.241 0.366 0.558 0.877 1.465 2.734 6.360 25.940
0.45 0.151 0.235 0.357 0.544 0.855 1.428 2.666 6.202 25.298
0.50 0.150 0.233 0.354 0.540 0.848 1.417 2.644 6.152 25.091
πA = 0.2
0.05 0.495 0.681 0.952 1.369 2.061 3.334 6.086 13.949 56.409
0.10 0.397 0.546 0.763 1.098 1.653 2.675 4.883 11.192 45.258
0.15 0.338 0.465 0.650 0.935 1.408 2.278 4.158 9.530 38.537
0.20 0.300 0.412 0.576 0.828 1.247 2.018 3.684 8.443 34.142
0.25 0.273 0.376 0.525 0.756 1.138 1.841 3.360 7.700 31.138
0.30 0.255 0.351 0.490 0.705 1.061 1.717 3.134 7.183 29.046
0.35 0.242 0.333 0.466 0.670 1.008 1.632 2.978 6.826 27.604
0.40 0.234 0.322 0.450 0.647 0.974 1.576 2.876 6.592 26.659
0.45 0.229 0.315 0.441 0.634 0.954 1.544 2.819 6.460 26.122
0.50 0.228 0.313 0.438 0.630 0.948 1.534 2.800 6.417 25.948
πA = 0.3
0.05 0.524 0.683 0.916 1.275 1.871 2.967 5.336 12.103 48.646
0.10 0.450 0.588 0.788 1.097 1.610 2.553 4.591 10.412 41.851
0.15 0.401 0.523 0.702 0.977 1.433 2.273 4.087 9.270 37.259
0.20 0.366 0.478 0.641 0.892 1.309 2.075 3.732 8.465 34.024
0.25 0.341 0.445 0.597 0.831 1.219 1.933 3.477 7.886 31.695
0.30 0.323 0.422 0.565 0.787 1.154 1.831 3.292 7.468 30.014
0.35 0.310 0.405 0.543 0.756 1.109 1.758 3.162 7.172 28.826
0.40 0.302 0.394 0.528 0.735 1.078 1.710 3.075 6.974 28.032
0.45 0.297 0.387 0.520 0.723 1.061 1.682 3.025 6.861 27.577
0.50 0.295 0.385 0.517 0.719 1.055 1.673 3.009 6.824 27.429

Table 3.

Relative efficiencies of the estimators from Warner’s model and Swensson’s NRRT for various θ and p values when π A = 0.1, 0.2, 0.3. The shaded cells show the cases where Warner’s model is more efficient than Swensson’s model.

The FORTRAN code used for Table 3 can be obtained from the author upon request.

One can observe from Table 3 that Warner’s model is more efficient than Swensson’s NRRT only for small θ values. One can also observe that REs become higher when π A values increase, except for some special cases where p 0.25 and θ 0.10 . Note that in both Tables 2 and 3, REs when θ = 0.5 .

When the nonsensitive binary question’s prevalence in the population is known (say, the prevalence of being a dog person is 1 θ , and the prevalence of being a cat person is θ ), Takahasi and Sakasegawa’s NRRT becomes equivalent to Warner’s model. To see this, let the unknown proportion of the population members who have the sensitive characteristic A be denoted by π A , where 0 π A 1 , and let X i ( i = 1 , 2 , n ) be defined as in Eq. (6). If θ is known, the probability of getting a “1” response from Takahasi and Sakasegawa’s NRRT can be written as

P X i = 1 = 1 π A 1 θ + π A θ , E10

which is equal to the probability of getting a “yes” response from Warner’s model, i.e., Eq. (7) and thus, provides the same estimator when solved for π A .

There have been many different RRTs developed since Warner’s original method, such as in Greenberg et al. [36, 37], Gupta [38], Gupta et al. [39, 40, 41], Yu et al. [42], Sihm et al. [43], Gupta and Shabbir [44], and so on. Efforts specifically have been made to improve the efficiency of the technique by reducing the variance and thus the confidence intervals, because, the primary disadvantage of the Warner’s model, and in fact of all RRTs, is that the variances of the estimators are higher than the ones that could be obtained from DQ [35, 45]; for a comprehensive review of RRTs, interested readers are referred to Chaudhuri and Mukerjee [35], Chaudhuri [46], and Chaudhuri and Christofides [47]. In the next subsection, we will review some real-life applications of RRTs and their comparisons with other surveying techniques.

3.1 Applications of RRTs

Two meta-analyses of 6 validation and 32 comparative studies which utilized RRTs showed that in various settings, the RRT results are superior to those from DQ and become more valid as the sensitivity of a topic increases [48].

The benefits of the RRTs have also been demonstrated by many statistical methodology researchers via theorems and simulation studies; however, their use in large or national surveys has been somewhat limited. In fact, to our knowledge, the only study which applied an RRT on a national level was done by Kirtadze et al. [49] in the country of Georgia. Kirtadze et al. [49] used a multistage cluster sampling and surveyed 4805 respondents to assess under-reporting of drug abuse in the Republic of Georgia. They utilized the unrelated question RRT to ask questions such as “During the last 12 months, have you taken hashish or marihuana?” They found that all RRT estimates for prevalence of controlled substance use were higher than the DQ estimates, which indicates under-reporting with DQ. For example, lifetime cannabis use estimate was 88.24% higher from RRT than from DQ. Kirtadze et al. [49], however, did not use a gold standard such as urinalysis and thus did not know the “true” value of prevalence of illegal drug use in the Republic of Georgia’s study population.

Although not nationwide, there have been other researchers who incorporated RRTs in their surveys. Fisher et al. [50], for example, used forced RRT to estimate the prevalence of substance use and sexual activity of high school students who enrolled to their clinic. They compared their results with the ones from a non-anonymous questionnaire which was completed by the same students earlier the same academic year. While RRT provided higher rates for substance use-related questions with respect to the DQ, it provided similar rates for sexual activity-related questions (36% from DQ vs. 31% from RRT). Fisher et al. [50] concluded that admitting to sexual activity in the school setting might carry less stigma and perceived risk than admitting to marihuana or cocaine use [50]. We suggest, however, that this result might indicate that the high school students who participated in the study overestimated their sexual behavior when asked directly to live up to peer acceptance; in other words, we suggest that forced RRT corrected the over-reporting.

In another study, Srivastava et al. [51] used Warner’s RRT to assess the extent of sexual abuse among children in several districts of Uttar Pradesh state of India. They found that the estimates from RRT were higher than the national estimates obtained from Ministry of Women and Child Development, Government of India, which is an indicator of potential under-reporting with DQ.

In a more recent study, Chhabra et al. [52] used convenience sampling and asked the question “Have you ever been a victim of sexual abuse by a friend or family member?” to 585 students in a college in Delhi, India. They divided their sample into three equal randomly selected groups and asked the sexual abuse-related question using (1) DQ, (2) the RRT proposed by Sihm et al. [43], and (3) the confidential method, which was their gold standard to compare their results. The prevalence of sexual abuse was 14% with the gold standard, 8% with the DQ method, and 12% with the RRT. Note that their confidential method, however, was also a surveying technique in which the participants wrote down their answers and put them into a closed box.

3.2 Inflated variance

Although all RRTs lead to unbiased (i.e., accurate) estimates of the sensitive characteristic of interest, their variances are larger than the ones from the DQ technique. Thus, the price for using an RRT instead of the DQ is the inflated variance, which is due to the randomization process. Consequently, if the question of interest is not considered to be really sensitive by most of the respondents in a specific population, using an RRT instead of the DQ inflates the variance of the estimates unnecessarily. Besides, it is known that there are cultural or social differences in the extent to which topics are perceived as sensitive. For example, smoking marihuana is a less threatening topic in the Netherlands than in the USA, or questions regarding education are considered to be sensitive in Sweden [1]. Similarly, questions regarding HIV status or sexual practices might not be considered to be sensitive by patients who visit an HIV clinic for treatment. Unfortunately, once an RRT is incorporated within a survey, even if the question of interest turns out to be not sensitive after collecting the data, one has to proceed with the inefficient estimate obtained from the RRT. Motivated by the fact that if researchers do not have a priori knowledge about the sensitivity level of a question in a specific population, then they should be able to select between the estimators from an RRT and the DQ; Ardah and Oral [45] proposed a novel two-stage RRT for a binary response where they utilized Warner’s RRT in the second stage. Their model provides unbiased estimators of both prevalence of the sensitive characteristic and the proportion of cheating in the population simultaneously and allows one to obtain better estimates by avoiding the unnecessary penalty, i.e., the inflated variance, if the question of interest turns out to be significantly not sensitive. Although there are some other RRTs which can estimate the sensitivity level of a question, such as in Gupta [38], none of these RRTs have the ability to avoid the inflated variance if the question’s sensitivity level turns out to be low.

Ardah and Oral [45] denoted the unknown true proportion of population members that have the sensitive characteristic A by π A , where 0 π A 1 . They showed that, in a self-protective no-saying model (i.e., when a respondent cheats, s/he always answers in favor of the least stigmatizing category (see [53])) if one uses the DQ, the proportion of respondents who have the sensitive characteristic A becomes

π A = π D + π C , E11

where π D is the proportion of respondents who reply with a “yes” to the sensitive question when asked directly and π C is the proportion of respondents who cheat (i.e., the proportion of respondents who reply with a “no” to the sensitive question even though they have the characteristic A). Realize that in Eq. (11), π C is the SDB, and if all respondents answer truthfully in the DQ, i.e., if π C = 0 , then π A = π D . Unfortunately, the SDB, i.e., the proportion of cheaters, cannot be measured using the DQ. Ardah and Oral’s [45] proposed model, however, allows one to both estimate it and select the unbiased estimator with the minimum variance depending on the sensitivity level of the question (or proportion of cheating in the population). Ardah and Oral [45] utilized unrelated question RRT in the second stage of their model as well; in fact, their model can easily be extended to any desired RRT [45].


4. Conclusion

Numerous indirect questioning techniques have been developed to ask survey questions on sensitive topics or stigmatizing characteristics. All indirect questioning techniques have their own limitations: In applying the UCT, only one of the group members provide the information on the sensitive characteristic of interest [25]. NST is known to suffer from recall bias, barrier effects, transmission error, and response bias [54]. NRRTs can be vulnerable to cheating due to distrust as much as the RRTs [55]. RRTs have some disadvantages as well: integrating a randomization device into a survey tool might not be practical in some situations, such as when researchers plan to use venue sampling to reach LGBT community members in gay bars or clubs. RRTs are also known not to work well if the respondents do not understand the process and/or do not follow the instructions properly. Besides, there might be cultural, social, or personal differences in the extent to which topics are perceived to be sensitive. Thus, we suggest that researchers should select the optimal surveying technique by considering various aspects of their study, such as the target population, sensitivity level of the question, available resources, and practicality of integrating a specific technique, at once. As in Erdmann [55], we also suggest that researchers should not rely on the more-is-better assumption, which is assuming that the higher prevalence estimates are more accurate than the lower prevalence estimates, in comparing different techniques; instead, we suggest to use a valid gold standard (such as urinalysis or a lie detector) for comparisons, whenever possible, perhaps using a small subsample.



! This code was written to calculate the relative efficiencies in Table 2


INCLUDE 'link_fnl_static.h'


REAL(8) R(21,21)


OPEN (3,FILE="C:\Users\Oral\Documents\Fortran results\Table2.txt")

WRITE(3,*) ""

WRITE(3,*) ""

WRITE(3,50) "Calculated Relative Efficiencies for various p=theta and Population Proportion (Pi_A) combinations"


WRITE(3,*)"*****RE MATRIX*****"


WRITE(3,*)"p=Theta value "

DO K=0.0,1.05,0.05

WRITE(3, 100,advance='no') K


WRITE(3, *) "

DO J=1,148

WRITE(3, 200,advance='no') '_'


WRITE(3, *) "



DO I=1,11


DO J=1,21

WRITE (*,*) 'theta=p=', THETA, 'Pi_A=', PIA






WRITE(3,100,advance='no') R(I,J);


WRITE (*,*)"

WRITE(3, *) "



100 FORMAT (F15.5,1X)

200 FORMAT (A1)




  1. 1. Lensvelt-Mulders G. Surveying sensitive topics. In: de Leeuw ED, Hox JJ, Dillman DA, editors. International Handbook of Survey Methodology. New York: LEA, Taylor & Francis; 2008. pp. 1-17
  2. 2. DeLeeuw E, Callegaro M, Hox J, Korendijk E, Lensvelt-Mulders G. The influence of advanced letters on response in telephone surveys: A meta-analysis. Public Opinion Quarterly. 2007;71(3):413-443
  3. 3. Groves RM, Fowler FJ, Couper MP, Lepkowski JM, Singer E, Tourangeau R. Survey Methodology. 2nd ed. Hoboken, NJ: Wiley; 2009
  4. 4. Link M, Mokdad A, Town M, Weiner J, Roe D. Improving Response Rates for the BRFSS: Use of Lead Letters and Answering Machine Messages. Paper presented at the annual conference of the American Association for Public Opinion Research, Nashville, TN; 2003
  5. 5. Singer E, Groves RM, Dillman DA, Eltinger JL, Little RJA, editors. The Use of Incentives to Reduce Nonresponse in Household Surveys. Wiley-Interscience; 2002. pp. 163-177
  6. 6. Spiers S, Oral E, Fontham E, Peters ES, Mohler JL, Bensen JT, et al. Modelling attrition and nonparticipation in a longitudinal study of prostate cancer. BMC Medical Research Methodology. 2018;18:60. DOI: 10.1186/s 12874-018-0518-6
  7. 7. Oral E, Simonsen N, Brennan C, Berken J, Su LJ, Mohler JL, et al. Unit nonresponse in a population-based study of prostate cancer. PLoS One. 2016;11(12):e0168364. DOI: 10.1371/journal.pone.0168364
  8. 8. Lessler JT, O’Reilly JM. Mode of interview and reporting of sensitive issues: Design and implementation of audio computer assisted self interviewing. NIDA Research Monograph. 1997;167:366-382
  9. 9. van Griensven F, Naorat S, Kilmarx PH, et al. Palmtop-assisted self-interviewing for the collection of sensitive behavioral data: Randomized trial with drug use urine testing. American Journal of Epidemiology. 2006;163(3):271-278
  10. 10. Lind LH, Schober MF, Conrad FG, Reichert H. Why do survey respondents disclose more when computers ask the questions? Public Opinion Quarterly. 2013;77:888-935
  11. 11. Schober MF, Conrad FG, Antoun C, Ehlen P, Fail S, Hupp AL, et al. Precision and disclosure in text and voice interviews on smartphones. PLoS One. 2015;10(6):e0128337
  12. 12. Beach SR, Schulz R, Degenholtz HB, Castle NG, Rosen J, Fox AR, et al. Using audio computer-assisted self-interviewing and interactive voice response to measure elder mistreatment in older adults: Feasibility and effects on prevalence estimates. Journal of Official Statistics. 2010;26(3):507-533
  13. 13. Couper MP, Rowe B. Evaluation of a computer-assisted self-interview component in a computer-assisted personal interview survey. Public Opinion Quarterly. 1996;60:89-105
  14. 14. Smith TW, Kim J. A review of survey data collection modes: With a focus on computerizations. Sociological Theory and Methods. 2015;20(2):185-200
  15. 15. Raghavarao D, Federer WT. Block total response as an alternative to the randomized response method in surveys. Journal of the Royal Statistical Society: Series B: Methodological. 1979;41:40-45
  16. 16. Miller JD. A new survey technique for studying deviant behavior [PhD thesis]. The George Washington University; 1984
  17. 17. Miller J, Cisin I, Harrell A. A new technique for surveying deviant behavior: Item-count estimates of marijuana, cocaine, and heroin. Paper presented at the Annual Meeting of the American Association for Public Opinion Research. St. Petersburg, FL; 1986
  18. 18. LaBrie JW, Earleywine M. Sexual risk behaviors and alcohol: Higher base rates revealed using the unmatched-count technique. The Journal of Sex Research. 2000;37(4):321-326
  19. 19. Biemer P, Brown G. Model-based estimation of drug use prevalence using item count data. Journal of Official Statistics. 2005;21(2):287-308
  20. 20. Wolter F, Laier B. The effectiveness of the item count technique in eliciting valid answers. To sensitive questions: An evaluation in the context of self-reported delinquency. Survey Research Methods. 2014;8(3):153-168
  21. 21. Gervais WM, Najle MB. How many atheists are there? Social Psychological and Personality Science. 2018;9(1):3-10. DOI: 10.1177/1948550617707015
  22. 22. Tsuchiya T. Domain estimators for the item count technique. Survey Methodology. 2005;31(1):41-51
  23. 23. Chaudhuri A, Christofides TC. Item count technique in estimating the proportion of people with a sensitive feature. Journal of Statistical Planning and Inference. 2007;137:589-593
  24. 24. Hussain Z, Ali Shah E, Shabir J. An alternative item count technique in sensitive surveys. Revista Colombiana Estadística. 2012;35:39-54
  25. 25. Ibrahim F. An alternative modified item count technique in sampling survey. International Journal of Statistics and Applications. 2016;6:177-187
  26. 26. Zimmerman RS, Langer LM. Improving estimates of prevalence rates of sensitive behaviors: The randomized lists technique and consideration of self-reported honesty. The Journal of Sex Research. 1995;32(2):107-117
  27. 27. Bernard HR, Johnsen EC, Killworth PD, Robinson S. Estimating the size of an average personal network and of an event subpopulation. In: Kochen M, editor. The Small World. Norwood, NJ: Albex Pub. Corp.; 1989. pp. 159-175
  28. 28. Bernard HR, Johnsen EC, Killworth PD. Estimating the size of an average personal network and of an event subpopulation: Some empirical results. Social Science Research. 1991;20:109-121
  29. 29. Killworth PD, Johnsen EC, McCarty C, Shelley GA, Bernard HR. A social network approach to estimating seroprevalence in the United States. Social Networks. 1998;20:23-50
  30. 30. Russell HB, Hallett T, Iovita A, Johnsen EC, Lyerla R, McCarty C, et al. Counting hard-to-count populations: The network scale-up method for public health sexually transmitted infections. 2010;86(Supp. 2):ii11-ii15
  31. 31. Swensson B. Combined questions: A new survey technique for eliminating evasive answer bias (I)—Basic theory. Report No. 70 of the Errors in Surveys Research Project. Institute of Statistics, University of Stockholm; 1974
  32. 32. Takahasi K, Sakasegawa H. A randomized response technique without making use of any randomizing device. Annals of the Institute of Statistical Mathematics. 1977;29(1):1-8
  33. 33. Tian GL, Tang M-L. Incomplete Categorical Data Design: Non-Randomized Response Techniques for Sensitive Questions in Surveys. Boca Raton, FL: Chapman & Hall/CRC; 2014
  34. 34. Warner SL. Randomized response: A survey technique for eliminating evasive answer bias. Journal of the American Statistical Association. 1965;60:63-69
  35. 35. Chaudhuri A, Mukerjee R. Randomized Response: Theory and Techniques. Statistics: Textbooks and Monographs. Vol. 85. New York: Marcel Dekker, Inc.; 1988
  36. 36. Greenberg RG, Abul-Ela ALA, Simmons WR, Horvitz DG. The unrelated question randomized response model: Theoretical framework. Journal of the American Statistical Association. 1969;64(326):520-539
  37. 37. Greenberg RG, Keubler RT, Abernathy JR, Horvitz DG. Application of randomized response technique in obtaining quantitative data. Journal of the American Statistical Association. 1971;66:243-250
  38. 38. Gupta SN. Qualifying the sensitivity level of binary response personal interview survey questions. Journal of Combinatorics, Information & System Sciences. 2001;26:101-109
  39. 39. Gupta SN, Gupta RC, Singh S. Estimation of sensitivity level of personal interview survey questions. Journal of Statistical Planning and Inference. 2002;100:239-247
  40. 40. Gupta SN, Thornton B, Shabbir J, Singhal S. A comparison of multiplicative and additive optional RRT models. Journal of Statistical Theory and Applications. 2006;5:226-239
  41. 41. Gupta SN, Shabbir J, Sehra S. On the estimation of population mean and sensitivity in a two-stage optional randomized response model. Journal of the Indian Society of Agricultural Statistics. 2010;61:164-168
  42. 42. Yu B, Jin Z, Tian J, Gao G. Estimation of sensitive proportion by randomized response data in successive sampling. Computational and Mathematical Methods in Medicine. 2015;18(2015):172918
  43. 43. Sihm JS, Chhabra A, Gupta S. An optional unrelated question RRT model. Involve: A Journal of Mathematics. 2016;2(9):195-209
  44. 44. Gupta SN, Shabbir J. Sensitivity estimation for personal interview survey questions. Statistica. 2004;64:643-653
  45. 45. Ardah IH, Oral E. Model selection in randomized response techniques for binary responses. Communication in Statistics-Theory and Methods. 2018;47(14):3305-3323
  46. 46. Chaudhuri A. Randomized Response and Indirect Questioning Techniques in Surveys. Boca Raton, FL: Chapman and Hall/CRC Taylor and Francis Group. 2011
  47. 47. Chaudhuri A, Christofides TC. Indirect Questioning in Sample Surveys. Berlin, Heidelberg: Springer-Verlag; 2013. DOI:
  48. 48. Lensvelt-Mulders G, Hox JJ, van der Heijden P, Maas C. Meta-analysis of randomized response: 35 years of validation studies. Sociological Methods & Research. 2005;33:319-348
  49. 49. Kirtadze I, Otiashvili D, Tabatadze M, Vardanashvili I, Stutua L, Zabransky T, et al. Republic of Georgia estimates for prevalence of drug use: Randomized response techniques suggest under-estimation. Drug and Alcohol Dependence. 2018;187:300-304
  50. 50. Fisher M, Kupferman LB, Lesser M. Substance use in a school-based clinic population use of the randomized response technique to estimate prevalence. The Journal of Adolescent Health. 1992;13:281-285
  51. 51. Srivastava R, Nigam AK, Singh N. Application of randomized response techniques in estimation of prevalence of child sexual abuse. Statistics and Applications. 2015;13:37-45
  52. 52. Chhabra A, Dass BK, Gupta S. Estimating prevalence of sexual abuse by an acquaintance with an optional unrelated question RRT model. The North Carolina Journal of Mathematics and Statistics. 2016;2:1-9
  53. 53. van den Hout A, Bockenholt U, Van der Heijden PGM. Estimating the prevalence of sensitive behavior and cheating with a dual design for direct questioning and randomized response. Applied Statistics. 2010;59:723-736
  54. 54. Jing L, Lu Q, Cui Y, Yu H, Wang T. Combining the randomized response technique and the network scale-up method to estimate the female sex worker population size: An exploratory study. Public Health. 2018;160:81-86
  55. 55. Erdmann A. Non-randomized response models: An experimental application of the triangular model as an indirect questioning method for sensitive topics. Methods, Data Analyses. 2018;13(1):139-167. DOI: 10.12758/mda.2018.07

Written By

Evrim Oral

Reviewed: 17 January 2019 Published: 25 February 2019