Online Ads Annoyance Factors: A Survey of Computer Science Students

Abstract

Despite the technological sophistication of current advertising techniques, the way online ads are delivered is in many cases intrusive, distracts users from their task, and annoys them. To delve into the reasons that make an online ad annoying, we investigate eight specific ad functional features and their effect on perceived annoyance: ability and ease of closing the ad, coverage of page contents, coverage of browser window, ad expansion, automatic ad activation, video/animation, sound, and ad targeting based on the recent browsing history. The study addressed 132 computer science students, both undergraduate and MSc, who were asked to select and document three to six online ads they consider annoying, resulting in a total of 462 recorded ads. The majority of collected annoying ads were automatically activated ads, which in most cases could be closed or stopped easily. Furthermore, ads that covered most of the browser window scored the highest percentage of perceived annoyance. Ads with video or animation and automatically triggered sound, even if displayed in a fixed size area, seem to significantly affect users’ perceived intrusiveness and attention distraction. Also, most ads reported as very annoying were not personalized and were automatically activated, while student level makes no statistically significant difference.

1. Introduction

The issue of ad annoyance is significant, as two seemingly opposing forces strive to serve their purpose: on the one hand, marketers and advertised firms try to assure a higher number of ad impressions (and hopefully higher conversion rates), and on the other hand, web users want easy access to the content they look for without being targeted (and retargeted) by ads every step of the way. Between these two, but much closer to the users’ side, stand design and usability experts warning of the significant negative effects of advertising techniques on users [1, 2, 3, 4, 5]. Users point out that modern tactics and mechanisms of internet advertising often distract them from their goal and prevent them from easily accessing the content they are interested in [6, 7, 8, 9, 10, 11]. Moreover, in many cases, ads take up deceiving forms and lure users into clicking them only to realize short after that they have been tricked [5, 12, 13, 14, 15]. For these reasons, users have developed defense mechanisms against ads that are imposed on them; either they disable ads using ad-blocking software [16, 17, 18, 19, 20], or when they recognize them, they avoid (close or stop them) or ignore them (the banner blindness phenomenon is powerful and known for decades [21, 22, 23, 24]).

Over time, the situation becomes worse as advertising takes advantage of increasingly sophisticated techniques to be imposed on users, to attract their attention, to ‘watch’ them as they navigate to serve them personalized ad content, even to deceive them. Modern digital marketing uses sophisticated techniques to gather data from a variety of sources and build up-to-date user profiles and applies data analysis techniques so that it can make informed decisions about how to target users more efficiently [25, 26, 27].

The purpose of this article is to investigate the functional features of online ads that contribute to their perceived annoyance. More specifically, the study investigates: (1) the ability and ease of closing the ad, (2) the coverage of page contents, (3) the coverage of browser window, (4) ad expansion, (5) automatic ad activation, (6) video/animation, (7) sound, and (8) ad targeting based on recent browsing history. To this end, the study addresses savvy web users (computer science students at undergraduate and postgraduate levels) who were asked to find and document ads they consider annoying due to the way they are displayed or behave and regardless of their actual content. Such users can find the way to close an ad or can spot and avoid click-bait. Moreover, they are more likely to have selected and documented highly annoying ads more objectively than the average or less experienced web users.

The rest of the chapter is structured as follows: Section 2 describes representative related literature; Section 3 discusses the methodological approach adapted in the study (participants, assigned task, and designed questionnaire). Section 4 presents the collected data and the main analyses their correlations. Section 5 discusses the findings of the study, and Section 6 concludes the chapter.

2. Related literature

Research on the topic of online ad techniques and how they are perceived by the users has been rich during the past decades. Rejer and Jankowski in [7] provide an insight into how users’ brain activity patterns are affected by intrusive online advertisements. They conducted an experiment with six participants tasked to read a series of texts on the web browser and answer some questions to confirm that they have understood them. While proceeding from one text to another, a total of 10 advertisements were displayed randomly for 3 seconds to draw the users’ visual attention. They recorded the total completion time, users’ web activity, and EEG data to investigate brain and cognitive patterns. Major decrease of beta activity indicated that users lost their concentration when ads were displayed, although the authors note that users’ motivations may affect their approach to the intrusive ad, which was suggested by the changes in frontal and prefrontal asymmetry index.

Eye-tracking tools have increased in popularity in recent years, particularly when used in cognitive, behavioral, and psychological studies. Lee and Ahn in [28] investigate how cognitive processing and consumer memory recall ability are influenced by visual attention when exposed to online advertisement messages, based on eye-tracking data. In the experiment, 118 participants were tasked to navigate through 20 new websites where different types of banner ads (static, animated), with two speed modes (slow, fast) and different product content each, were implemented and randomly assigned. Fixation duration and fixation frequency were used as metrics of attention, while memory recognition and attitude toward brands were examined via a post-test questionnaire. The results indicated that ad avoidance was not affected by the advertisement speed, while consumers avoided animated banner ads. As expected, longer exposure and focus on a banner ad affect positively the recall ability as shown by total fixation duration metrics. While animated banners are meant to produce a more striking visual stimuli and grab attention, the attention metrics used in this study showed insignificant changes to cognitive processing and limited capacity to retain details and being able to recognize certain aspects of the ads.

Eye-tracking was also deployed by Hong et al. [10] to study online ads. More specifically, they identified three key animation features (i.e., motion, lagging, and looming) based on three attention theories and investigated both their direct and combinatorial effects on capturing online consumers’ attention in different online tasks using eye-tracking data. While motion and looming are effective for capturing online consumers’ attention, lagging is not as effective. Also, they found a positive relationship between online consumers’ attention and their recall performance. Furthermore, the effects of these animation features are stronger when browsing (than searching for a particular target object) and when the webpage is simple (rather than complex). Overall, they conclude that the effects of animation are contingent on multiple factors, including the specific animation feature, task condition, and complexity of the webpage.

A factor that advertisers and advertised companies seem to ignore or underestimate is that aggressive (intrusive, annoying, or deceiving) marketing techniques may have significant negative effects on the advertised product and the brand as a whole. Zha and Wu [9] investigated online ads and how the elements of intrusiveness can affect a website’s credibility. To test their hypotheses, they set up different versions of a news website. Each version had different types of disruptive ads, with the first version not including any ads (it was used as the control group by 61 participants). The second version included ads of irrelevant content (65 participants), and the third version included ads with relevant content (56 participants). The total sample of 182 participants was given 15 minutes to navigate the website and tasked to read 4 news stories with ads appearing randomly every 3 minutes. A post-questionnaire evaluated the validity of the new stories and the site’s credibility, which also included questions to examine the negative attitudes toward the site. Although users found the ads intrusive and distracting while performing their task, they did not affect the cognitive process or behavioral attention of users due to the goal-driven character of the assigned task. Although the relevance of the ads did not affect the website’s perceived credibility, when the content of the ads was related to the content, it caused users’ suspicion about the credibility of the news story.

Animated banners hold a prominent role in digital marketing strategies and especially in websites. To better understand the degree of their influence, Thota et al. [29] constructed a conceptual model to identify the reasons behind revisit intentions and brand advertisement of the animated banner ads. The researchers initially combined a think-out-loud protocol and exploratory interviews to find and categorize the major accrued themes based on content analysis from 93 participants. Ads may have a negative effect and result in skepticism toward the website, while the results indicated a negative correlation between brand evaluation and banner ads. Based on the emerged themes (banned ad evaluations, host website evaluations, and brand evaluations) and the individual’s need for cognition, the researchers formed a structured questionnaire. The second phase of the study involved a sample of 37 participants who visited a specific website that included banner ads. The results indicate a strong negative correlation between residual skepticism and attitude toward the website and revisit intentions. Individuals’ cognition and loyalty toward the website have a moderating effect on attitudes toward the website and intention to revisit, while animated banners lead consumer skepticism and a negative correlation between the consumers’ attitudes and the advertised brand.

McCoy et al. [30] studied the impact of website familiarity on customers’ future intentions to revisit and the effect of advertisements’ presence on a website’s perceived quality. In their experimental process, 76 undergraduate students were randomly exposed to two already existing websites. The used websites were chosen based on familiarity (familiar/unfamiliar), while six ads (slogans) in the form of pop-ups were created and displayed to half of the participants (the rest were not exposed to ads). A structured questionnaire was created to examine the website quality dimensions and user intentions to revisit it. The results indicated that the familiarity of the website affects positively the perceived quality in cases where the familiar website includes ads. The intrusiveness of pop-up ads has a significant impact on the perceived website quality in both cases of familiar and unfamiliar websites. The analysis showed that regardless of the type of familiarity, those without ads did not achieve higher quality scores than those with ads.

The mechanisms (in terms of interaction design) used by an ad to be presented to the target user and the degree of freedom to stop, close, or ignore it in many cases determine the effect an ad has on its recipients and how much they get annoyed. In other words, there are ads that are annoying regardless of their content, due to the way they are delivered. Fessenden in 2017 [1] conducted a survey about the most hated online advertising techniques, where she confirms the main findings of the initial survey done by Nielsen in 2004 [31]. In the new survey, they involved 452 adult respondents who were shown 23 wireframes corresponding to the different types of advertisements and rated how much they disliked them on a scale of 1 to 7. On desktop, the most hated ad types were modal ads, auto play video ads, intra-content ads that shuffle page content as they load, and deceptive links that look like content but are ads. On mobile, the hierarchy of ad dislike is more complicated. The most disliked ads were modal and intra-content ads combined with content reorganization. The survey concluded that modal windows are still disliked as much as they were over a decade ago (when they had the form of pop-ups), and the same stands for automatically playing audio. Moreover, the following ad characteristics remained just as annoying for participants as they were in the early 2000s: pop-ups, slow loading time, covering page contents, content that moves around occupying most of the page, and automatically playing sound.

Smith in [8] investigated the preferred digital marketing strategies that lead to online purchases. An online survey with a sample of 571 participants was conducted to explore how intrusive advertisements affect their cognitive behavior and preferences (side-panel or pop-up ads), the influence of personalized advertisement strategies regarding attention grabbing and website revisit, and finally, whether peer reviews provide positive feedback. The results indicate that pop-up ads can irritate and deter from future revisits, while incentives such as discount coupons, promotions, and competitive prices attract new consumer audiences and lead to more peer-written reviews and interaction with the website. Another influential factor is the website design as it significantly affects the visual and cognitive behavior of the consumer.

The content of an ad is another crucial factor that determines how it is received by users. Based on the observation that problematic content in modern web ads can be more subtle than flashing banner ads or techniques of user deception (such as click-bait, advertorials or endorsements without proper disclosure practices, low-quality content farms, etc.), Zeng et al. [5] examined how users perceive online ads depending on the ad content. The study investigates which attributes of an ad’s content contribute to negative user reactions and poses two research questions: (a) what the different types of negative (and positive) reactions of people to online ads are and (b) what specific kinds of content and tactics in online ads cause negative reactions. The methodology comprised two phases: first, a taxonomy of 15 positive and negative user reactions to online advertising was created from a survey of 60 participants. Then, a set of 500 ads crawled from popular websites was labeled by 1000 participants, who used the constructed taxonomy to characterize the ads. The study concluded that large fractions of ads in the random sample used resulted in concrete negative reactions on the part of the participants.

Personalized ads are another interesting topic of investigation, as such ads aim to serve tailored ad content to each specific user. O’Donnell and Cramer [32] emphasized the importance of personalized ads and their effect on consumer attitudes. To achieve an overall view on the subject’s study, they constructed an online questionnaire to measure internet advertising behavior, attitudes toward personalized ads, and preferable used devices with a sample of 296 participants. They performed a semi-structured, open-ended interview, and the results indicated that there was a positive correlation between the individual’s attention and ad relevancy based on the different used devices (desktop/mobile), while the authors signify the importance of understanding target audiences, especially in mobile devices, when implementing personalized ads. The analysis of the interviews revealed that personalized ads and relevance positively affect the users’ ability to recall ads, while themed ads that promote life awareness, useful information, and personalized brand advertisement can lead to positive interactions.

As mentioned above, web users are quite familiar with most types of online ads, and this allows them to recognize (even with peripheral vision) and ignore them. Kim and Seo in [33] provide a multidimensional insight into ad avoidance, both traditional and digital, while attempting to explore and identify other influential factors based on consumer attitudes toward advertising. An online survey was administrated to 253 students, which adapted measurement scales of advertising beliefs, attitudes toward advertising, advertising avoidance, and consumer exposure to various forms of advertising media. YouTube ads lead to higher ad avoidance in both desktop and mobile devices, with consumer attitude toward advertising being strongly correlated to ad avoidance. Once again goal-oriented users despite the disruptive messages (i.e., pop-ups) avoid ads without retaining information or experiencing cognitive overload.

The study in [34] investigates whether alternative e-lifestyles and ad avoidance are interdependent and how different styles of ad avoidance affect users’ attitudes toward internet advertising. The first stage of the study included structural equation modeling on adapted measurement scales to explore the respondents’ e-lifestyles (need-driven, internet-driven, entertainment-driven, sociability-driven, importance-driven, uninterested or concern-driven, and novelty-driven) and types of ad avoidance (cognitive, affective, and behavioral) from 412 participants who averaged 2–3 hours of internet usage. Since the results showed no direct effect of e-lifestyle on ad avoidance, the researchers introduced an open-ended question to measure daily internet usage. The results from the modified data indicated and confirmed the direct effect of e-lifestyle on ad avoidance with need-driven e-lifestyle to be strongly correlated with behavioral ad avoidance. Also, ad avoidance was not affected by entertainment-driven individuals as they showed lower cognitive and behavioral tendencies. Individuals who are goal-oriented, seek information, or are highly motivated, novel-driven users exhibit high behavioral ad avoidance.

Social networks form a communication channel and provide the opportunity to implement new digital business strategies aimed at a larger consumer. In [35], authors investigate the factors leading to ad avoidance on Facebook’s social media platform. In the between-subject experiment, 253 participants were randomly assigned to a searching or surfing task on a Facebook newsfeed, which contained different, randomly assigned ad placements. The researchers adapted and constructed measurement scales to explain ad avoidance intent and the influence of Facebook usage frequency and experience and the moderating effect of product involvement and task conditions. Their findings suggest that ad placement and ad avoidance were strongly correlated and confirm that low involvement with the product and users’ motives can lead to higher ad avoidance.

A discussion on current marketing techniques and user annoyance could not leave out the topic of dark patterns. Traditional retail has a long history with dark patterns even before its online version came into play, and online advertising has also adopted dark patterns to amplify its messages and increase its audience. Dark patterns are tricky user interfaces designed in a way that users are deceived into making decisions that are not in their best interests, and research both from the marketing and the computer science domain has significantly studied this topic [36, 37, 38, 39]. Dark patterns exploit main observations from behavioral research to serve the purposes of advertised firms and offer higher profits and the surveillance economy as Narayanan et al. [40] explain. They also claim that dark patterns resulted from three distinct trends (each one with a long story of its own): (a) deceptive web services practices (coming from the retail world), (b) nudging (stemming from research and public policy), and (c) growth hacking (from the design community). Disguised ads are typical examples of dark patterns where, for instance, an ad banner looks like useful content that the user is looking for and falsely clicks on to realize later that they assumed wrong. These ads are also known as native ads, defined as paid ads that match the look, feel, and function of the media format in which they appear. As Taylor [12] claims, “the fundamental problem with native advertising is that, in the absence of a clear disclosure indicating that the message is an ad, the media outlet and advertiser are, in effect, blocking the advertiser’s persuasion knowledge.” Native ads are also considered extremely annoying since users realize they have been deceived soon after they click on the ad. Some alarming indications that dark patterns will persist in the next years is the emergence of companies that offer dark patterns as a service as well as personalized dark patterns that incorporate rich data about users and apply advanced analytic processing to tailor appealing messages to each user [41]. Nowadays, there is a strong demand for incorporating ethics into the design process (an issue that should concern designers and marketers) and also imposing specific regulatory restrictions to the format of acceptable (and lawful) ads and web content more generally (Narayanan et al., 2020). To defend against manipulative dark patterns, I Bongard-Blanchy et al. in [37] suggest bright patterns, design frictions, training games, and applications to expedite legal enforcement.

3. Materials and methods

The aim of the study was to investigate the features of online ads that contribute to their perceived annoyance. More specifically, the study investigates: (1) the ability and ease of closing the ad, (2) coverage of page contents, (3) coverage of browser window, (4) ad expansion, (5) automatic ad activation, (6) video/animation, (7) sound, and (8) ad targeting based on recent browsing history. Thus, the research question was formed as follows:

RQ: Which of the investigated ad features (1–8) affect the perceived degree of ad annoyance?

Moreover, we studied the potential effect of user group on perceived ad annoyance, even though the group of undergraduate students outnumbers the group of MSc students, and this most probably will not allow for significant findings.

3.1 Participants and methods

The sample used in this research included two groups of participants (Table 1) based on the level of their education (which also affects their age group): (1) undergraduate students and (2) MSc students. Group 1 consisted of 109 students who attended either the 8th or the 10th semester of the Computer Engineering and Informatics Department at the University of Patras, Greece. Group 2 consisted of 23 MSc students who attended the postgraduate program “Technologies and Services of Smart Information Systems and Communications” at the Department of Electrical and Computer Engineering of the University of Peloponnese, Greece. Ages of Group 1 members ranged from 21 to 25 years and those of Group 2 from 25 to 45 years. Both groups are considered highly experienced web users, and they have all attended a course on e-commerce and online advertising from a computer science perspective.

	Male	%	Female	%	Total
Group 1 (undergraduate)	89	81.7%	20	18.3%	109
Group 2 (MSc)	16	69.6%	7	30.4%	23
Total	105	79.5%	27	20.5%	132

Table 1.

Participants by education level and gender.

Students were asked to find online ads they consider annoying, record their interaction with these ads when attempting to ‘close’ them (if this was possible), and fill in a questionnaire where they provided behavioral features of each ad and assessed its annoyance level. Each student selected and recorded from 3 to 6 such ads, and they were free to choose ads from any kind of website either in Greek or in English.

3.2 Design of the instrument

For each selected ad, users were asked to fill in the questionnaire designed for this purpose, which was composed of nine questions as depicted in Table 2. Eight questions regarded functional features of the ad, and one question asked respondents to assess how annoying the ad was according to their perception on a scale from 1 (not at all) to 5 (extremely).

Question	%
Q1. Was the ad automatically activated or it was triggered by you?
Automatically (on page load, after scrolling or after a short time)	80.1%
I triggered it	19.9%
Q2. Were you able to close the ad at any time?
No, I could not close it (or it was a moving or a non-moving banner that could not be closed), or I had to wait until it finished	24.7%
No, it was on the background of the page (on the left and the right side)	0.6%
Yes, but only after a certain time	18.4%
Yes, but only after a certain time (but then it moved to the top of the page)	0.2%
Yes, but it was difficult to find the way to close it	10.8%
Yes, and it was easy to close it	44.8%
Yes, and it was easy to close it, but it reopened itself	0.4%
Q3. Did the ad include motion?
Yes, animated graphics	22.9%
Yes, video	34.0%
No	43.1%
Q4. Did the ad include sound?
Yes, the sound started to play automatically	27.9%
Yes, the sound started after an action I did (e.g., mouse over)	5.2%
No	66.9%
Q5. Did the ad expand?
Yes, the ad expanded automatically, but I could shrink it to its original size	8.7%
Yes, the ad expanded automatically, and I could not ‘shrink’ it back	10.6%
No, the ad had fixed dimensions	80.7%
Q6. Did the ad prevent you from seeing the page contents?
Yes, the ad covered the complete browser window	24.5%
Yes, the ad covered part of the browser window	49.4%
No	26.2%
Q7. What percentage of the browser window did the ad cover?
It did not cover any part of the browser window	14.3%
Less than 5%	7.1%
Between 5 and 25%	29.4%
Between 25 and 50%	21.0%
More than 50%	28.1%
Q8. How annoying was the ad (due to the way it was presented, not its content)?
1 (not at all)	2.8%
2	15.4%
3	24.0%
4	27.5%
5 (extremely)	30.3%
Q9. Do you think that the ad was selected for you (based on your browsing history)?
Yes	19.0%
Could be, I am not sure	21.6%
No	59.3%

Table 2.

Questionnaire design and responses.

Cronbach’s alpha was calculated to determine the instrument’s internal consistency.

4. Results

4.1 Collected data

As already mentioned, students were asked to select and document three to six online ads that they consider annoying, which resulted in a total of 462 recorded ads. According to the data depicted in Table 2, users recorded as annoying ads that were triggered automatically (80.1%, as opposed to 19.9% ads that were triggered by the users); in most reported cases, ads were easy to close/stop (44.8%), but also there were ads that could not be closed, or the user had to wait before they could close it (43.1%). Regarding the type of content of the reported ads, most of them were static (43.1%) or video (43%), and 66.9% of reported ads did not include sound. Users also reported 27.9% of ads that had sound that started to play automatically.

80.7% of ads reported as annoying did not expand but had fixed dimensions and a mere total of 19.3% expanded (10.6% of ads did not allow users to shrink them back to their original dimensions). 49.4% of ads reported as annoying overlaid and covered part of the browser window (with the rest 24.5% covering the complete window, and 26.2% being placed inside the webpage without covering the contents). With regard to the area of the visible page covered, in most cases, ads covered from 5 to 25% (29.4% of ads) and more than 50% (28.1% of ads). Even though users were specifically asked to report annoying ads, when asked to say the degree of annoyance for each ad, the recorded answers span the complete range from 1 (not at all) to 5 (extremely annoying). In most cases, users assessed the ads with scores from 3 to 5. When asked whether the ad was targeted (selected for them based their current browsing history), in most cases, the answer was negative (59.3%), while 19% of ads were considered targeted, and 21.6% could be targeted but not for certain.

4.2 Ad functional features and perceived degree of ad annoyance

The following figures (line charts and stacked bar charts with summaries for groups of cases) are used to analyze and present the characteristics of the ads in relation to their perceived degree of annoyance. Figure 1 displays per group the answers to whether it was possible to close the ad. In both groups, 207 (44.8%) of the reported ads could be closed, and this was done easily, but still students reported these ‘politically correct’ ads as annoying due to other functional characteristics they possess.

Figure 1.

Perceived ability and ease to close the ad for undergraduate and MSc student groups.

As depicted in Figure 2, most ads (45.8%) had covered part of the browser window and specifically covered between 5% and 50%, and they had fixed dimensions (did not expand). From these ads, 16% were considered extremely annoying, 27% very annoying, 18.8% relatively annoying, and 9% slightly annoying. The ads that covered most (more than 50%) or the complete browser window were the most annoying (56.4%), regardless of the ad expansion variations.

Figure 2.

Browser window coverage and ad expansion vs. perceived annoyance.

In Figure 3, we analyze the characteristics of the ads (expansion, sound, and motion) in terms of the perceived degree of annoyance. As shown in the chart, most ads had no sound and motion or had animated graphics. Moreover, most ads (217 or 46.9%) had fixed dimensions. 23% of students considered these ads as slightly annoying, 27.6% of students as moderately annoying, 27.1% as very annoying, and 17.9% as extremely annoying.

Figure 3.

Ad expansion, animation/video, and sound vs. perceived annoyance.

The most annoying ads were considered those that included video; their sound started to play automatically and had fixed dimensions (24.5% of all reported ads). Specifically, 37.1% of students claimed that these ads were extremely annoying, 26.7% very annoying, 17.6% moderately annoying, and only 2% slightly annoying.

As shown in Figure 4, 57.4% of all reported ads were automatically activated, and of them 18.9% were perceived as moderately (4.8%), very (5%), and extremely (9.1%) annoying ads that could not be closed (either easily or at all). 24% of highly annoying ads -considered as moderately (9.7%), very (8.9%), or extremely annoying (5.4%)- were ads that could be closed. Concerning the rest of the ads that were triggered by the user (42.6%), the most annoying were those that could be closed easily (8.7%) with 2.2% moderately annoying, 2.8% very annoying, and 3.7% extremely annoying. For the ads that were triggered by the user and could not be closed (3.4%), 1.5% were considered moderately annoying, 0.2% very annoying, and 1.1% extremely annoying.

Figure 4.

Automatic activation and perceived ability and ease of ad closing vs. perceived annoyance.

In total, annoying ads (i.e., those that were assigned scores from 3 to 5 on the 5-level annoyance scale) were 84.8% of all identified ads. In Figure 5, we observe that the most annoying ads (i.e., with assigned scores from 3 to 5) were those that were not related to the browsing history of the individual user (50.8%), most of which (40.9%) were automatically activated, and a mere 9.9% being trigged by the user. For 34% of the annoying ads, respondents believe that they were selected based on their recent browsing history (15.8%) and that for the rest 18.2% of ads respondents were not sure if they were based on their browsing history.

Figure 5.

Targeted ad and automatic activation vs. perceived annoyance.

4.3 Student group and perceived degree of ad annoyance

Normality (Table 3) tests are used to determine if a data set is well-modeled by a normal distribution. The data in the groups are normally contributed (sig = 0.877, sig = 0.878).

Tests of Normality
	Annoying	Kolmogorov-Smirnov^a			Shapiro–Wilk
		Statistic	df	Sig.	Statistic	df	Sig.
Q8. How annoying was the ad (due to the way it was presented, not its content)?	Undergraduate	.192	344	.000	.877	344	.000
	MSc	.189	118	.000	.878	118	.000

Table 3.

Tests of normality for undergraduate and MSc student groups.

a. Lilliefors Significance Correction

An independent t-test (Table 4) was applied to compare the means of the two independent groups to determine whether there is statistical evidence that the associated population means are significantly different. Considering the Levene’s test for equality of variances sig = 0.806 > α = 0.05, it is suggested that the variances of the two groups are equal, and thus, the null hypothesis (H0) (that the variances are equal) applies (sig = 0.765 > α = 0.05), meaning that there is no difference in perceived annoyance between undergraduate and MSc students at the 5% significance level.

Independent Sample Test
		Levene’s Test for Equality of Variances		t-test for Equality of Means
		F	Sig.	t	df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	95% Confidence Interval of the Difference
									Lower	Upper
How annoying was the ad (due to the way it was presented, not its content)?	Equal variances assumed	.061	.806	.296	460	.767	.036	.122	−.204	.276
	Equal variances not assumed			.295	201.395	.768	.036	.123	−.205	.278

Table 4.

Independent sample test for undergraduate and MSc student groups.

5. Discussion

According to the analysis that preceded a feature that most reported ads share is that they are automatically triggered, with most of them allowing to be closed in a straightforward manner and without having to wait. When considering how perceived annoyance is affected by ad targeting, most ads reported as very annoying were not personalized (i.e., they were not related to the recent browsing history of the user), and they were also automatically activated. Previous research on the issue of personalized ads provides split results with users either being positive about having related and meaningful interactions or being worried about their invaded privacy and personal information collected [32, 42, 43, 44]. The fact that in this study users have not reported targeted ads as highly annoying is in fact a strong indication that users accept it or have a positive attitude. It rather conveys that users do not necessarily corelate targeted ads with annoyance. Further research is required to investigate if this non-negative attitude results from the fact that users like ads that are tailored to them personally and have a positive perception of targeted advertising advantages and value.

Another cluster of reported ads that scored high on annoyance included ads that incorporate video and sound, with sound being triggered automatically but with fixed dimensions. The last feature seems odd as one would probably expect ads that include video, audio, and expansion to be even more annoying. The most probable explanation that this is not supported by our study is that students were asked to document annoying ads, and hopefully the web today does not contain so many video ads with automatically triggered audio that also expand. If we were to show users specific ads to compare in terms of annoyance, the situation would most likely be different.

An interesting observation is that users did not report any native ads, which is probably because most of them do not recognize them as ads, or due to the non-intrusive nature of these ads that do not cause annoyance.

As previously mentioned, modals (pop-up ads) were and still are one of the most annoying types of ads [1], but this has not been confirmed as a major feature of documented annoying ads by the current study. To clarify the issue, authors went back to respondents and found out that 73.5% of them use pop-up blocking software, a fact that justifies the low percentage of documented modal ads.

6. Conclusion

The study investigated the effect of functional features of online ads on the perceived degree of annoyance. More specifically, the considered functional features comprised: (1) the ability and ease of closing the ad, (2) coverage of page contents, (3) coverage of browser window, (4) ad expansion, (5) automatic ad activation, (6) video/animation, (7) sound, and (8) ad targeting based on recent browsing history. To this end, the research question was which of the investigated ad features affect the perceived degree of ad annoyance. The analysis revealed that the percentage of the browser window occupied by an ad has a significant effect on whether the ad is considered annoying, with ads covering more than 50% being considered the most annoying. Although existing research and common sense indicate that dynamically expanding ads are very annoying as they interrupt and distract users, in this study the ads that were recorded as very annoying had fixed limits but covered a significant percentage of the browser visible window.

We also investigated the potential effect of user group on perceived ad annoyance. The analysis showed that there is no difference in perceived annoyance between undergraduate and MSc students, meaning that the age difference of these groups has no significant effect on how annoying they consider the ads they have documented. This was expected based on the user sample, even though age is a factor that could make a difference if the sample was expanded to include more diverse age groups. This is a limitation of the study as the number of MSc students aged over 30 is very limited.

Another limitation is that both groups comprise highly skilled and computer and internet savvy people who attend higher education institutions. This, for instance, may have affected their responses since they are experienced web users and can easily find the way to close an ad or can spot and avoid click-bait. Nevertheless, and due to their expertise, these people have a much clearer idea about how an ad behaves or how to ‘handle’ it, and we can assume that they have selected and documented highly annoying ads objectively. One might also argue that a sample of 132 participants (even if they all share a quite specific profile) is not large enough, but one should keep in mind that our main objective and research question addresses the 462 recorded ads and their functional features. This sample size is significant and large enough to allow to identify dominant features of ads considered annoying.

Finally, this study has not addressed the case of multiple ads appearing concurrently on the same page. It would be interesting to investigate whether the degree of annoyance is affected by the total number of displayed ads on a pageview and if, in this case, there are ads that are considered more annoying when compared with other ads on the same pageview.

Acknowledgments

Authors would like to thank the participants of this study, whose contribution is valuable.

The present work was partially financially supported by the “Andreas Mentzelopoulos Foundation”. The study was conducted in accordance with the Declaration of Helsinki and approved by the Research Ethics Committee (REC) of the University of Patras (application no. 12852, date of approval 23.03.2022) for studies involving humans. The Committee reviewed the research protocol and concluded that it does not contravene the applicable legislation and complies with the standard acceptable rules of ethics in research and of research integrity as to the content and mode of conduct of the research.