Eye Movements during Barking at Print

Abstract

In order for educational software coupled with eye-tracking capability to respond with pedagogical appropriateness to a reader’s eye movements, reading metrics must be validly interpreted. These metrics and the types of reading they diagnose, for example, scanning, skimming and reading for meaning, come largely from reading fiction texts in a home language. The use of existing classification systems for disadvantaged South African learners did not yield consistent and credible classification of these learners’ reading. This could be attributed to learners barking at print, that is, decoding the text without comprehending what they were reading beyond the word level. Eye movements of barkers and non-barkers were analysed and no statistically significant differences were found. Barking at print was found to be distinct from mindless reading and mind-wandering, as well as other reading types for both first and second languages. Barking is characterised by slow reading with few regressions, average fixation durations typical of second language reading, and variability in eye-movements between lines of text. This work is significant in that it establishes that eye-movement during barking at print is distinct from other categories of reading. However, further research is needed before valid applications can be made from this work.

1. Introduction

Eye-tracking has the potential to enhance learning, for example through incorporation in intelligent learning technologies which make use of artificial intelligence which acts on data gathered from a user to alter the user experience in a personalised manner to optimise learning [1]. Although incorporation of eye tracking technology into such systems is still in its infancy, such technology has already been shown to, for example: improve attention during engagement with intelligent tutoring systems for learning Biology [2], Geography [3] and Computer Programming [4]; Respond to mind-wandering during reading [5]; Detect engagement in metacognitive processing [6]; Predict affect [7]. Particularly if future uses of such technology are aimed at reading-improvement, the software creators must be able to interpret the correspondence between eye-movement metrics and the type of reading the user is undergoing, validly. A well-developed categorisation system does exist for this purpose for a variety of reading types, such as skimming, scanning and reading for meaning at various grade levels, for English home language readers reading fiction, which will be discussed in detail in a subsequent section. Data we collected from a group of poor South African learners reading a science text in English, which is not their home language, however, could not credibly be categorised using these existing systems, exposing a gap in the literature addressed in this article. This chapter will proceed by giving background on the problem under investigation as well as related studies. This will be followed by a brief discussion on the methodology and an in-depth discussion of the data analysis. The paper will conclude by summarising the significance and limitations of the study.

2. Problem statement

The poorest 80% of South African learners possess, on average, reading skills which rank among the worst in the world [8]. For example, 60% of South African grade 6 learners are unable to read with comprehension in any language [9]. Pretorius and Spaull [10] identify inability to decode text accurately as the primary problem, with barking at print being an additional problem among many of those relatively stronger learners who are at least able to undergo text-decoding. Barking at print refers to engaging in decoding with little to no comprehension of what the text means on a global level, although the meaning of individual words or even groups of words may be comprehended [11]. Such a reading style is consistent with the engagement in superficial textual strategies that strongly characterises poor South African learners’ multiple-choice answering patterns [12], for example choosing options containing terms common to or with superficial similarity to the question or to terms in an associated comprehension passage. Barking at print is not unique to South African learners, with the term having been coined by Samuels [13] in the United States, and reports on barking at print even including presence among relatively high achieving learners in what could be considered good schools in affluent areas (see, e.g. [14]). However, given the high prevalence of barking at print among poor South African learners, whom we have easy access to due to our engagement in various intervention programmes for such learners, we are well situated to investigate this reading phenomenon.

Despite the firm establishment of barking at print in education literature, the nearest correspondence in eye-tracking literature is mindless reading, researched by observing participants reading nonsense-text, i.e. text having no meaning in any language [15], as well as reading during mind-wandering (e.g. [5]). Both mindless reading of nonsense text and reading during mind-wandering differ in a number of ways from non-mind-wandering barking at print written in a language which the reader does understand, at least to some extent. These differences include motivation, perceived purpose and prior exposure and expectations to perform each of these activities. Therefore, the findings of mindless reading and mind-wandering research may not correspond to barking at print, and if this is found to be the case, then obviously the usefulness of the existing literature, at least for mindless reading, to applications such as intelligent learning technologies, is limited and a new and more useful set of metrics associated with decoding without comprehension is needed. Further, the eye-tracking metric guidelines resulting from research related to mindless reading and mind-wandering are restricted to gaze length, so that even should barking at print prove to be similar to mindless reading or mind-wandering, there is a gap in the literature about other eye-movement metrics during such reading.

In this study the eye-movement characteristics of 67 grade 8 and 9 South African learners from financially and educationally impoverished backgrounds were examined during silent reading of science text in English, their second language. Based on their comprehension scores, these participants are divided into three groups: barkers (n = 23), poor readers (n = 25) and moderate readers (n = 19). Statistical analyses were then performed on the eye-movement characteristics of the barkers relative to that of the other groups of participants, as well as descriptive analyses of the differences between barking and existing literature about other types of reading. For this purpose the following research questions are applicable:

1. How do the reading eye-movements of those participants who were barking at print compare to those of their peers of two levels of reading proficiency?

2. What are the eye movement characteristics of those participants who were barking at print?

3. How do the eye-movement characteristics of barking at print compare to those published for skimming, scanning and reading for meaning at various levels of proficiency?

3. Background

3.1 Reading

If one is able to read, it means one can look at a word and process its meaning [16]. Rauding, derived from reading and auding, means the ability of a person to understand most of the thoughts contained within the material they are reading [16]. During rauding the eyes move across the lines of words allowing consecutive words to be perceived without needing to concentrate on where the eyes will move next. There are 5 basic reading processes, referred to as gears 1–5, where rauding or gear 3 is the process used most often. Readers can control the rate of input [17] thus the different gears are characterised by different reading speeds, averaging from 138 wpm to 600+ wpm for college students [16]. The goal of the reader determines the gear they use to process the material [16] which changes the reading behaviour [17]. A person’s rauding rate is the fastest speed at which they able to successfully process relatively easily text [16]. The average rauding rate for grade 8 and 9 learners is 205 and 219 wpm respectively [16].

While fluency does not guarantee comprehension, it is essential to be able to comprehend [18]. The four levels of reading which are still applicable today were introduced in 1946 and are as follows: (1) the independent level, (2) the instructional level, (3) the frustration level and (4) the probable capacity level [19]. The fourth level is based on material which is read to a student but the first three are based on the decoding and comprehending ability of the student when reading a text [19], and are therefore focussed on in this article. The word-reading accuracy and comprehension of the first 3 levels are given in Table 1. Readers who are able to read at the independent and instructional levels are likely to be able to self-direct their learning through reading, although those at the instructional level would do so sub-optimally unless provided with explicit help. Readers operating at the frustration level are unlikely to engage in voluntary reading activity, given the large amount of effort required.

	Decoding accuracy (%)	Comprehension (%)
Independent level (level 1)	99	90
Instructional level (level 2)	95–99	75–89
Frustration level (level 3)	≤90	≤50

Table 1.

Reading classification according to Halladay [19].

3.2 Eye movements during reading

The basic eye movements relevant to reading and visual search are fixations and saccades. Fixations are periods during which the eye is held relatively still in order to focus on an object [20]. Fixations typically last between 200 and 300 ms but the duration is dependent on the task [20]. For example, when reading in English, the mean fixation duration is 225–250 ms [21]. Fixation duration refers to the time, in milliseconds, that the eyes dwell on an object. Between fixations, saccades are used to move the eyes to an object of interest. Saccades are high velocity ballistic movements during which visual acuity is suppressed [20]. Saccadic amplitude is a measurement of the length of the saccades and can be measured in terms of visual angle, and refers to the eye span which can be deduced from the jumps (saccades) made by the eyes across the text. When reading text, the saccade length is generally measured in terms of character spaces ([22] as cited in [21]). Average saccade length when reading English is 7–9 characters [20] or 8–9 characters as reported in a later study [21].

Regressions refer to eye movements in the opposite direction of the reading movement. A regression can be a correction when a saccade overshoots the desired text or it can be used to re-read text. Good readers are skilled at using regressions to reposition their eyes where they would like to in order to reinforce something or to gain clarity if they lack understanding, but poor readers are inclined to struggle to use regressions accurately and efficiently and will trace indiscriminately backwards through the text [23]. While the difficulty of the text does influence the number of regressions, it is typical for L1 readers to perform regressions for 10–15% of the time while reading fiction [21].

3.3 Reading behaviour

Reading can be measured in fixation progress by determining how many characters the reader advances with each saccade [24, 25]. On average, the fixation will fall just left of the word centre [26, 27]—this is referred to as the preferred viewing location [26]. Some studies have found that readers do not fixate on every word while others have found that readers do indeed fixate on almost every word, with readers tending to skip short words [17]. When speed reading or skimming more words are skipped as a natural process [17]. When reading linearly, readers may skip to the next piece of text if they find that the current piece they are reading is no longer giving enough information [28].

Literature suggests that eye movements are useful in detecting reading difficulty and analysing reading behaviour. As readers progress from beginner to skilled (adult) levels, their reading speed [29] and mean saccadic amplitude [30] increase while the number of fixations their eyes make per 100 words, their mean fixation duration and the frequency of their regressions [20] decrease. Reading difficulty is characterised by longer fixation durations, more regressions and shorter saccades [31]. The length of the regression can also highlight whether the reader is experiencing difficulty or not, namely short regressions within a word show lack of understanding of that particular word. Longer regressions show a lack of understanding of the text. Regression percentage is the number of regressions divided by the number of fixations made, expressed as a percentage. It also appears that the number of words per fixation differs according to the difficulty of the text presented. When reading a passage with a difficulty appropriate to the age of the reader, there is an average of 1.2 words per fixation. If the text is easier than the age level of the reader, then the number of words per fixation has a higher average [17].

During reading studies, measurements such as first fixation duration, single fixation duration, and gaze duration are often used instead of average fixation duration since readers do not generally only fixate on a word once and words are often skipped during the course of reading the text [21]. It is possible that skipped words may be perceived in the prior fixation and words that are fixated on more than once are likely done so in order to process their meaning [21]. Perceptual span can also be considered a key feature to take into account since this indicates how much the reader can “see” when pausing over a word [21]. English first language (L1) readers appear to be able to perceive a range starting 3–4 characters to the left of the fixation and ending 14–15 characters to the right of the fixation [21]. Vocabulary size has a significant effect on the total time spent on words but not on the initial processing of the word [32].

The amount of time spent on a word is affected by the ease or difficulty with which the word is processed and other variables [21]. Fixation durations are strongly influenced by the frequency of the word (high frequency words have shorter durations), the predictability of the word being read, the number of meanings the word has, when the meaning of the word was acquired, semantic relations between the word and preceding words and how familiar the word is to the reader (multiple sources as cited in [21]). The length of a word correlates with the likelihood that the reader will fixate on the word again and the likelihood that the reader will skip the word [33]. Words with high frequency are more likely to be skipped than words with low frequency [34] and low frequency words receive more initial processing time than high frequency words [35]. Predictable words are also more likely to be skipped than unpredictable words [36, 37] and they also have a shorter fixation duration [37]. More attention and cognitive effort are required for unfamiliar words than familiar words [32] and while familiar words in L1 and L2 (second language) require similar processing times, unfamiliar words in L1 increase the cognitive load [32]. Unfamiliar words are read slower than familiar words in terms of fixation durations and they are read “more times”—in other words regressions to unfamiliar words are more common than to familiar words [35]. Similarly, when reading in L2, unfamiliar words have a higher fixation duration than familiar words and are visited more than familiar words [38]. However, in this instance the initial processing time between familiar and unfamiliar words is not significantly different [38].

Eye movement measurements can also be evaluated on the first and second pass—the first pass being the first time the word is read and the second pass being the subsequent time the word/piece is read if the reader regresses to that word/piece [39]. First fixation duration, single fixation duration and the likelihood that the word will be refixated are indicators of the difficulty of the word experienced during the initial reading while gaze duration is an indicator of the difficulty experienced in identifying the word [40]. Second pass duration indicates late measures of word difficulty and total fixation time can be used to measure comprehension difficulty [40].

3.4 Behaviour in different types of reading

The measurements discussed in the previous section were presumably measured for readers reading in English, which in all likelihood was their first language. Reading in a second or third language may be characterised by different behaviour and there are different types of reading which can be conducted.

For instance, since reading behaviour is coupled with cognitive processing, it stands to reason that eye movements can indicate when attention is low. Fixation durations are indicative of the amount of processing which is occurring, with longer fixations on words that require more processing [17]. Words which require more processing are infrequent words, while longer fixations are seen when making inferences at the end of a sentence and when integrating information from important clauses [17]. As a means to detect mindless wandering, or low attention while reading, eye movement behaviour at the end of phrases or sentences can be used—the natural slowing down in reading which occurs in order to integrate the words does not occur when attention is low [41]. Additionally, the variability in fixation durations caused by word length and frequency is lower when the reader is not paying attention [41]. Hence, when the mind of the reader wanders, there are short fixations on low frequency words and long fixations on high frequency words [41]. Fixations are also fewer and longer and eye behaviour is more erratic when the mind wanders [42]. First fixation durations, total gaze duration and total viewing time are shorter for normal reading than for wandering [42]. Additionally, when the mind wanders, readers are less likely to make fixations and regressions on text and more likely to fixate on areas other than the text [42]. The number of saccades and fixations drops when the mind wanders and there are less and shorter within-word regressions [43].

When scanning transformed text as opposed to reading, fixations are longer and saccades are shorter [15]. In this instance, transformed text refers to the practice of replacing all alphabetic characters with the letter z but preserving casing and punctuation among other characteristics [15]. Participants were requested to pretend to read the transformed text [15] in order to simulate scanning. In contrast, skimming normal text for proofreading is characterised by shorter fixations, and longer saccades than when reading text for understanding [21]. Readers also tend to read the start of the text more thoroughly than the second half and readers first skim the entire text before reading it [28]. When reading in a second language (L2), reading times are longer and readers exhibit more fixations, shorter saccades and less word skipping than when they read in their first language (L1) [44]. Furthermore, L2 readers of Afrikaans text required an average of one fixation per syllable [45]. Fixation durations for L2 readers were longer, averaging 313 and 331 ms for an easy and difficult text respectively [45]. The increased fixation times in L2 could be attributed to the fact that L2 processing requires more cognitive load than L1 [32]. One of the purposes of this research is to understand whether the reading ability of South African township learners can meaningfully be classified using the guidelines developed in publications such as those referred to above.

In a comprehensive study eye movement behaviour was investigated for the different types of reading. This study will be discussed in greater depth for the purposes of this chapter. Regular reading, or reading for comprehension, is defined as reading a piece of text as one would normally read [46] as cited [47]. Thorough reading is reading to learn and is used to read text in a manner which will allow them to learn the content, perhaps in order to write a test about the content [48] as cited in [47]. Skimming, also known as reading for gist, means the reader must read the text as quickly as possible while still trying to understand the content [46] as cited in [47] and spell checking is the type of reading which is conducted in order to detect spelling errors in a text [49].

Thorough reading exhibits longer reading times and more rereading which results in higher comprehension scores [47]. When skimming, participants exhibit longer saccades, short fixations and skip more words [47]. Additionally, the total reading time is shorter and comprehension is lower [47]. When reading in order to achieve spell checking, saccades are short, fixation durations are long and fewer words are skipped [47]. Comprehension scores are lower and total reading times are longer [47]. Overall thorough reading was less uniform than regular reading, skimming was faster and more uniform and spell checking was slower and more uniform [47].

3.5 Summary of reading behaviour indicators

In summary, reading difficulty is characterised by long fixations, higher incidence of regressions and shorter saccades. Similarly, scanning transformed text results in longer fixations and shorter saccades but skimming results in shorter fixations and longer saccades. When reading in a second language, readers exhibit more fixations, shorter saccades and less word skipping and unfamiliar words cause higher fixation durations and are visited more. Unfamiliar words in L1 require more cognitive processing and hence they are read slower and cause more regressions.

Thorough reading has more visits on words and more vertical saccades and has a pattern similar to regular reading. When skimming, fixation durations are much shorter, while saccades are much larger and there are fewer regressions. In contrast, spell checking shows an increase in fixation duration and smaller saccades and an increase in first pass metrics.

4. Methodology

4.1 Sample and data collection

The 67 grade 8 and 9 learners who form the sample for this study attended two schools in densely populated areas of extreme poverty and high unemployment, 50 km from the nearest town, Bloemfontein, in South Africa. The sample was relatively academically strong for the context, since 50 of the learners had been identified, by their teachers, as being among the strongest in mathematics and natural sciences in their class, with the others being randomly chosen to increase the ability range. Each participant read a comprehension text about lighting, with a Flesch-Kincaid reading difficulty level [50] of grade 9, and answered four multiple choice questions about this text, individually, on a computer fitted with a Tobii TX300 eye-tracker. Tobii Studio 3.4.5, installed on the computer, was used for data extraction, including the generation of a screen-capture video showing eye-movements and mouse clicks, for each learner. The data collected for this article were obtained as the learners engaged with two screens. Screen 1 consisted of 5 lines of text about lighting, with an illustrative diagram below the text. Screen 2 was divided into two with the left-hand half being a repeat of the first screen and the right-hand half displaying the multiple choice questions one at a time. The learner progressed through the four questions once he/she had answered a question correctly.

Consistent with Pretorius and Spaull’s [10] statistics about poor South African learners’ reading abilities, 2 of the original 69 learners in this relatively strong sample did not even show evidence of being able to decode the text they were given to read since they moved their eyes randomly around the screen for a while before claiming they had finished reading. These 2 learners were therefore excluded from this study. The remaining 67 learners’ eyes did track the text systematically, at least for parts of the text, suggesting, to the extent to which this is possible from eye tracking data, that they were engaging in text decoding, and so were admitted into the sample for this study.

These learners were divided into the categories shown in Table 2, which is based on the learners’ reading behaviour and comprehension scores, informed both by a qualitative analysis of the learners’ eye movements during the question answering process and by the scores the software displayed in response to their choices. Guessing was inferred if the learner did not read sufficient text in the question or chosen distractor to be able to answer the question with comprehension. Use of superficial text matching was inferred if the learner chose the distractor designed to have superficial ties to the question and comprehension text. For example, for the question “Why do you know that conditions are right for lightning if you feel your hair standing up in a storm?”, the distractor “This means that you have a tingling feeling.” has superficial correspondence to the text: “If you ever feel your hair standing up or get a tingling feeling during a storm it could mean charges are moving onto you and you may be in danger of being hit by lightning!”. The comprehension score was derived as the average of the four scores obtained for the four-question multiple choice test, where each of these four scores was obtained as follows: If the learner engaged in guessing, inferred as described above, he/she was assigned 0% for that question. Otherwise, the learner was assigned 100% if he/she answered the question correctly on his/her first attempt, with 25% being subtracted for each successive incorrect attempt at the answer.

Category		n		Criteria
Barkers	Jumping	10	23	Learner either guessed or used superficial text-matching for all 4 questions on screen 2	Eyes tracked some text and jumped over other text on screen 1
	Regular	13			Eyes faithfully tracked the entire text on screen 1
Poor readers		25		Comprehension score was below 75%. Learner may have guessed or used superficial text-matching for some, but not for all, of the 4 questions on screen 2
Moderate readers		19		Comprehension score was 75% or higher for screen 2

Table 2.

Division of the sample into reading categories.

This classification system groups those who could be described as reading at the instructional or independent levels, according to Halladay’s [19] comprehension criterion, together in the group ‘Moderate readers’. The 23 barkers did not answer any of the 4 multiple choice questions correctly on the first attempt, except for a few cases of lucky guessing, i.e. happening to choose the correct option despite not having read the text of the option chosen or, frequently, even the question itself. From this we can deduce their comprehension of the text was minimal, well below the 50% minimum for inclusion in Halladay’s [19] frustration level. The poor readers showed at least some evidence of trying to answer the questions from comprehension, rather than guessing or superficial text-matching, but were clearly poor comprehenders. They fall into the upper end of Halladay’s [19] frustration level as well as in the unnamed category between the frustration and instructional levels. It should be noted that it is impossible, from the data at our disposal, to apply the decoding accuracy section of Halladay’s [19] classification system. We assume that evidence of the eyes systematically tracking the text is indicative of engagement in a high degree of decoding accuracy. Particularly for the jumping barkers, whose eyes only tracked some of the text systematically, this assumption may not be valid.

4.2 Metrics

The following metrics were extracted from the eye-tracking data gathered as the learners read the 5-line text about lighting on screen 1:

• Reading speed: similar to the method in [47] reading speed was calculated as milliseconds per character calculated using the total time the participant read the passage.

• Fixation duration: this indicates the length of fixations during the reading and is measured in milliseconds. The fixation duration was calculated as the mean fixation duration for each participant. For this measure, both those who were jumping and using regular barking were included.

• Fixation count: the total number of fixations captured during the reading process. Additionally the number of fixations per word was also calculated. This gives an indication of the distribution of the fixations over the piece and, on average, how many fixations were required per word in the piece. For this measure, those who were jumping were removed as they would naturally have less fixations as a result of their behaviour. Consequently, they would have much fewer fixations and this would not be due to them experienced no difficulty in reading, but rather in skipping large areas of the text without attempting to read it.

• Saccadic amplitude: as previously mentioned, the saccadic amplitude measures the distance between successive fixations. The mean saccadic amplitude per participant was calculated over the whole reading piece. Saccades which were longer than 8 degrees were considered to be a line sweep and were discarded. However regressive saccades were not discarded from this measurement. In this instance, participants who were jumping were also removed from the analysis, as they have very large saccades in order to facilitate their skipping behaviour. Even though the saccades were shorter than the length estimated for a line sweep, they would be fairly large and may unnecessarily skew the data.

• Number of regressions: the total number of regressions was counted manually for each participant and defined as any fixation that has an upward and left movement in order to fixate on a piece of text that was previously read.

• First fixation duration: the mean first fixation duration was calculated for each participant and each word as an indication of the length that was required per word on the first pass read.

• Visit count: the mean number of visits to each word was calculated for each participant as an indication of how many times, on average, each word is looked at. A visit is defined as distinct viewings of the word, in other words, separate fixations on each word within a single reading of the word constitutes a single visit. In order for another visit to be registered, the participant must read another word and return to a previously read word. As with fixations, the jumping barkers were removed for this analysis.

4.3 Data analysis

As briefly mentioned above, for the analysis of all duration metrics, reading speed and regressions, the regular barkers and the jumping barkers were classed as a single category collectively referred to as “Barkers”. Since the jumping barkers exhibited large saccades and few fixations as a result of their inherent reading behaviour which differs from regular barking they were excluded from the remaining metric analyses and only the barkers and non-barkers were analysed. Thus for fixation and visit count and saccadic amplitude, the jumping barkers were excluded. The non-barkers were then further subdivided based on their comprehension scores as per the reading levels of Halladay [19] as frustration readers—comprehension scores lower than 75%—and instructional and independent readers were grouped together as moderate readers. To answer the first research question, regarding the comparison of the barkers’ eye movements during reading with that of their peers, Kruskal-Wallis statistical tests were performed between the three groups for each reading metric. A Friedman ANOVA was also applied to the per-line reading metrics of each group to determine whether reading behaviour varied significantly between the lines. For both these analyses, p < 0.05 is taken as showing statistical significance. Descriptive analyses were performed to answer the remaining research questions, with gaze plots and comparisons between average measurements and those found in the literature drawn on to guide such descriptions.

5. Results

5.1 Qualitative discussion

The images below are gaze plots and serve to qualitatively illustrate the different reading behaviours which were evident in the sample.

Figure 1 is a gaze plot of the strongest reader in the group. From this it can clearly be seen that the reader fixated on most words. Durations, as reflected by the size of the fixation points, fluctuate within acceptable ranges. Some fixations are slightly offset but it can clearly be seen that reading is occurring at a steady pace with regular reading behaviour.

Figure 1.

The gaze plot of the strongest of the learners in the ‘moderate reader’ category.

Figure 2 is an example of one of the jumping barkers who is clearly not reading but rather exhibiting clusters of fixations interspersed with large saccades—or jumps. Fixations are more erratic and spread wildly over the body of the text and no regular pattern is discernible except in short spurts hence they are not habitual barkers but instead tend to skip large parts of the text. Durations also remain fairly constant for this jumper.

Figure 2.

The gaze plot of one of the ‘jumping barkers’.

Figure 3 is one of the regular barkers, but here it can be seen that the fixations closely relate to regular reading. There is a regular pattern, fixations are spread over the whole text and on each word the durations fluctuate. This type of pattern is representative of the majority of the barkers and it can be deduced that the behaviour very closely mimics proper reading.

Figure 3.

The gaze plot of one of the ‘regular barkers’.

5.2 Reading metrics for whole text

This section discusses the metrics which were analysed for the whole text while the participants read screen 1, thus no distinction was made on the word level and the entire text piece was treated as a single AOI. Table 3 shows these metrics, together with the range of expected values that could be obtained from previous literature. The ranges were read from graphs that were reported and as such are approximate values which will be used for reference to aid the comparison.

	N	Reading speed (ms per character)	Mean fixation duration (ms)	Regressions (n)	Regression %
Barkers *	23	$\overline{x}$  = 88.9; sd = 29.1	$\overline{x}$  = 296.8; sd = 63.2	$\overline{x}$  = 14.9; sd = 7.8	$\overline{x}$  = 9.0; sd = 3.5
Poor readers	25	$\overline{x}$  = 85.7; sd = 19.6	$\overline{x}$  = 310.9 sd = 42.8	$\overline{x}$  = 17.9; sd = 11.0	$\overline{x}$  = 10.5; sd = 5.3
Moderate readers	19	$\overline{x}$  = 88.1; sd = 33.1	$\overline{x}$  = 309.0; sd = 66.0	$\overline{x}$  = 21.2; sd = 10.8	$\overline{x}$  = 12.3; sd = 5.8
Kruskal-Wallis		H(2) = 0.3, p > 0.05	H(3) = 0, p > 0.05	H(3) = 0, p > 0.05	H(3) = 0, p > 0.05
Expected values		Thorough reading 56 ms/c; Skimming 26 ms/c; Spell checking 62 ms/c; Regular reading 45 ms/c	Thorough reading 196 ms; Skimming 192 ms; Spell checking 221 ms; Regular reading 197 ms; English L1 200–250 ms; Afrikaans L2 300+ ms		English L1 10–15%

Table 3.

A summary of the analysis for the metrics calculated over the whole piece of text for all participants.

^*

Barkers includes jumping and regular barkers due to the nature of these metrics.

Since the reading speed is measured as milliseconds per characters, higher values actually indicate a slower reading speed in this instance. From the mean values in the table above, it can be seen that the reading speed is slightly lower for the barkers than the readers, but not significantly so. Surprisingly, the poor readers had the fastest speed. Fixation durations are lower for the barkers than the other groups and they have fewer regressions and a lower regression percentage. For English first language silent reading it is accepted that the average fixation is between 200 and 250 ms. When experiencing reading difficulty, fixations will be longer, hence this could be indicative of the nature of the text and the attempt to process and understand the text. However, for these participants, English is not their first language but it is their language of instruction. As such, the fixation durations are closer to what would be expected from a second language reader as evidenced in [45]. Interestingly the barkers had the lowest fixation duration perhaps indicating the lower cognitive processing that was occurring. Regression percentage of the moderate readers is in line with what one would expect of English L1 reading. Given the nature of the text, one might expect more regressions as readers attempt to make sense of the scientific content. Poor readers and barkers have fewer regressions, which is contrary to what is expected when experiencing reading difficulty. Once again, the contrary findings for the barkers could be indicative of the lack or lower cognitive processing which is occurring.

As shown in Table 4, barkers have fewer fixations and longer saccades than the poor and moderate readers, although not significantly so. For L2 reading it is expected that saccades will be shorter and there will be more fixations. Skimming exhibits larger saccades while spell checking has shorter saccades. Therefore, the saccadic amplitude is contradictory to previous findings and veers towards skimming behaviour. The fact that regressive saccades were included in the data could account for the larger saccadic amplitude if the regressions were large. However, barkers had fewer saccades and therefore the longer saccades perhaps indicates the tendency not to concentrate on words in order to assimilate them but instead to mimic the behaviour of reading and thus not always to exhibit the saccade amplitude required to process and understand words. The actual nature of the saccades should be investigated in more depth in order to determine whether the cause is large backwards or forwards regressions. What should be kept in consideration is that, on average, the difference is very small and could thus not be on a scale that makes a difference to the number of fixations per word which will be analysed next. Without a reference to the number of fixations per word it is difficult to determine whether the barkers have fixations similar to any other type of reading. However, it can clearly be seen that, on average, barkers have fewer fixations than the poor and moderate readers which once again could indicate the lack of cognitive processing that is occurring.

	N	Fixation count (n)	Saccadic amplitude (visual angle)
Regular barkers *	13	$\overline{x}$ = 163.2,; sd = 25.0	$\overline{x}$ = 1.9,; sd = 0.1
Poor readers	25	$\overline{x}$ = 165.9,; sd = 37.1	$\overline{x}$ = 1.8,; sd = 0.1
Moderate readers	19	$\overline{x}$ = 175.9,; sd = 44.1	$\overline{x}$ = 1.8,; sd = 0.1
Kruskal-Wallis		H(2) = 1.2,; p > 0.05	H(2) = 1.8,; p > 0.05
Expected values			English L1 7–9 characters

Table 4.

Fixation count and saccadic amplitude over the whole text excluding jumping barkers.

^*

Due to the nature of the metrics jumping barkers are excluded from this analysis.

In summary, while none of the differences are significant it is noticeable that the behaviour of the barkers mimics that of very good readers (apart from the speed), even giving the impression that they are experiencing less difficulty with the text than the readers.

5.3 First pass reading of text and per word analysis

The mean first fixation duration for all words over the whole text was calculated for each participant as a measurement of a first pass at the text. Furthermore, the average fixation duration, the total fixation duration and fixation count was calculated as a function of the words—that is, each word was treated as a separate AOI and the mean values were calculated as such. This will give an indication of the behaviour on a per word level showing how long, on average, each word required and how many revisits or refixations each word required. The summary of the metrics is given in Table 5.

	N	First fixation duration (ms)	Mean fixation duration (ms)	Total fixation duration (ms)	Mean visit duration (ms)
Barkers *	23	$\overline{x}$ = 300.4; sd = 64.7	$\overline{x}$ = 300.4; sd = 61.6	$\overline{x}$ = 495.2; sd = 93.3	$\overline{x}$ = 335.7; sd = 75.4
Poor readers	25	$\overline{x}$ = 318.8; sd = 46.8	$\overline{x}$ = 309.2; sd = 46.2	$\overline{x}$ = 485.2.4; sd = 96.4	$\overline{x}$ = 347.6; sd = 54.4
Moderate readers	19	$\overline{x}$ = 307.9; sd = 74.0	$\overline{x}$ = 306.3; sd = 66.3	$\overline{x}$ = 502.6; sd = 151.9	$\overline{x}$ = 345.3; sd = 77.7
Kruskal-Wallis		H(2) = 2.8, p > 0.05	H(2) = 0.7, p > 0.05	H(2) = 0.3, p > 0.05	H(2) = 0.7, p > 0.05

Table 5.

A summary of the metrics for first pass reading of text.

^*

Barkers includes jumping and regular barkers due to the nature of these metrics.

Similar to the fixation duration over the whole text, barkers have the shortest first fixation duration. This metric is indicative of the cognitive processing which is required to process the word on the first pass reading. In this instance, the poor readers required the longest initial processing time which confirms the fact they could be experiencing difficulty on the first pass which is not unexpected. The barkers are clearly not spending more time processing the words. When comparing the first fixation duration to the overall fixation duration, the values for the barkers is very similar, only differing by 4 ms. The poor readers have, on average, first fixations which are approximately 8 ms longer but the moderate readers have lower first fixations than overall fixations. However, when inspecting mean fixation durations per word in the Table 5, it is only the barkers who remain unchanged while both the poor and moderate readers have lower mean fixation durations, showing an increase in processing when first encountering the word. The difference between these and the values in the previous section could be attributed to the settings of the AOIs, hence there are some fixations which are outside the bounds of the individual words but within the body of the text.

	N	Fixation count (n)	Visit count (n)
Regular barkers *	13	$\overline{x}$ = 1.6; sd = 0.2	$\overline{x}$ = 1.5; sd = 0.2
Poor readers	25	$\overline{x}$ = 1.6; sd = 0.2	$\overline{x}$ = 1.4; sd = 0.2
Moderate readers	19	$\overline{x}$ = 1.6; sd = .2	$\overline{x}$ = 1.5; sd = 0.2
Kruskal-Wallis		H(2) = 1.7, p > 0.05	H(2) = 1.7, p > 0.05

Table 6.

The average number of fixations and visits per word excluding the jumping barkers.

^*

Due to the nature of the metrics jumping barkers are excluded from this analysis.

	Line 2	Line 3	Line 4	Line 5
Line 1		B
		P	P	P
		M	M	M
Line 2		B	B	B
		P	P	P
		M	M	M
Line 3			B
			P
Line 4
				P

Table 7.

Summary of significant difference in reading speeds between lines.

The number of fixations and visits per word are very similar between the groups indicating that on this level the reading behaviour closely resembles one another (Table 6). This confirms that the minor difference in saccadic amplitude is not of the order that barkers fixate on individual words less. The number of fixations is much higher than the average of 1.2, which is used as a measurement of age appropriate difficulty, suggesting that in all instances the participants were perhaps experiencing some difficulty. Of course, the fact that they were reading a scientific text as opposed to a piece of fiction could naturally change their reading behaviour.

5.4 Reading behaviour spread over text (uniformity)

In order to investigate whether the reading behaviour was uniform over the whole text, some metrics were calculated for each line. These metrics were average fixation duration per line, reading speed (milliseconds per character) and mean number of fixations per word.

The metrics were compared for barkers and non-barkers separately as a repeated measure to determine if their behaviour changed significantly as they read the text. A Friedman ANOVA was used for this purpose.

Additionally, some graphs are given in order to illustrate the distribution of behaviour over the text, in some instances the metrics are not analysed statistically.

5.5 Fixation duration

There was a significant difference in the line reading for barkers (χ² = 23.4, p < 0.05, p < 0.01), poor readers (χ² = 24.8, p < 0.05, p < 0.01) but not for moderate readers (χ² = 0.9, p > 0.05) for the mean fixation duration. For barkers, the significant difference could be attributed to a difference between lines 1 and 2, 4, and 5. For poor readers, line 1 differed significantly from lines 2, 3, 4 and 5.

From the graph below, it can clearly be seen that the mean fixation durations for line 1 are lower than for the other lines from Figure 4.

Figure 4.

Mean fixation durations per line for each participant group.

Figure 5.

Mean fixation duration per word, in word order for each participant group.

The number of words per line in increasing order is lines 1, 3, 5, 4, 2. Interestingly the fixation duration of the barkers imitates this order in ascending order with the lowest average fixation duration on line 1 and the highest average fixation duration on line 2. This is not true for the poor and moderate readers who have, in ascending order of mean fixation duration, lines 1, 2, 4, 5, 3 and lines 1, 5, 2, 3, 4 respectively. In this respect, it appears the barkers adjust their behaviour according to the length of the line they are currently reading.

In terms of the difficulty, the two longest lines, namely lines 2 and 4 are also the lines which contain the most scientific content and concepts. Line 1 can be considered the easiest as it contains only everyday language and line 3 contains easier words and shorter sentences than lines 2 and 4. Hence in terms of length lines 1 and 3 are the shortest and in terms of difficulty also the easiest while lines 2 and 4 are both the longest and most difficult. Line 5 contains some words which may be unfamiliar to the participants but contains no scientific concepts and can thus be considered to be easier than lines 2 and 4. Therefore, the difficulty of the lines coincidentally mimics the length of the lines. Therefore, further investigation is required in order to determine whether the difficulty of the words in the line impacted the behaviour of the barkers or not.

Figure 5 gives an indication of mean time spent on words in the order the words appeared in the text. The vertical dashed lines indicate the lines of text. Clearly the first line has the lowest fixation durations for all groups but there are clear spikes and dips in the durations for each of the groups. An in-depth analysis of the length and difficulty of the word, together with the surrounding words, or concept, will possibly shed more light on the behaviour difference detected. However, that is beyond the scope of this chapter and will be analysed in a further study.

5.6 First fixation duration

Figure 6 shows the first fixation duration per word, in order of words over the text. From the graph it can be seen that for all groups the first fixation durations fluctuate across the text. The values do not appear to plateau based on the position of the word nor do they hold steady as one might expect for mindless reading. While the patterns are similar in some instances, i.e. all the groups decrease or increase for some words, it can be said that in some other instances there are different patterns where some groups increase and others decrease. Similar to the mean fixation durations, an individual word analysis which may provide more insight into the cause of the reading behaviour is beyond the scope of this chapter. For interest sake, some of the words and their associated behaviour will be discussed in an anecdotal manner, leaving in-depth analysis for a further study. For example, the word “potential” caused an increase in first fixation duration. The word potential preceded the word “difference” as the scientific concept of potential difference was under discussion. However, the word “difference” did not cause an increase in first fixation duration, nor did it have the same magnitude duration as “potential” suggesting that the participants were perhaps treating the text on a word-by-word basis and not processing concepts created by successive words. The poor readers had much higher instances of first fixation durations for many words in line 2, which was the most difficult line. Furthermore, for poor readers the words “collide” and “crystals” caused the highest and second highest first fixation duration on line 2. These could be considered to be more difficult words, hence the increase in first pass processing for this particular group.

Figure 6.

First fixation duration per word for each participant group.

5.7 Total fixation duration

Figure 7 shows the total fixation duration per word in the order the words appeared in the text. Similar to previous per word metrics, these also fluctuate. As one would expect there are some larger spikes on line 2 which was classified as the most difficult line. For example, looking at the larger duration on word number 25, the word is “enormous” and refers to an “enormous electrical field”. The word “enormous” might have been a particularly difficult word for the participants. The first fixation duration on this word was also high for all groups. Barkers also had, on average, an increased duration on the word “potential” for the concept “potential difference”, as with the first fixation duration. The increase for poor and moderate readers at word number 38 and 39 was caused by the words “ice crystals” at the start of the sentence “ice crystals inside the clouds…”.

Figure 7.

Total fixation duration per word for each participant group.

Comparing this graph to the graph of mean fixation durations for line 1, it appears that the participants spent a longer time in total at the start of the line, perhaps as they were getting into the reading pattern and settling down.

5.8 Reading speed

Reading speed was calculated for each line as milliseconds per character. There was no significant difference between the lines for the barkers (χ² = 9.3, p = 0.05) at an alpha-level of 0.05 but it can be considered significant at a level of 0.1. There was a significant difference for the poor readers (χ² = 45.9, p < 0.05) and moderate readers (χ² = 22.3, p < 0.05).

Significant differences are plotted in the Table 7, where B denotes barker, P denotes poor readers and M denotes moderate readers.

The table clearly shows that for the majority of the cases the same lines account for significant differences in each of the groups. Inspecting Figure 8 shows that the reading speed (in milliseconds per character) is significantly faster for lines 1 and 2. Line 3 is the shortest lines in terms of the number of words and it appears that participants slowed down when the line was shorter in this instance which is contrary to what one would expect given the mean fixation durations. Mean fixation durations for the barkers corresponded to the length of the line. Considering this, together with the reading speed, it can be posited that they had many short fixations on line 3.

Figure 8.

Reading speed (milliseconds per character) for each line.

Figure 9.

Fixations per word for each line.

5.9 Number of fixations

The number of fixations was calculated for each line and then spread over the number of words, hence the measurement is mean number of fixations per word for each line.

There was no significant difference between barkers (χ² = 3.04, p > 0.5), but there was a significant difference between poor readers (χ² = 22.0, p < 0.05) and moderate readers (χ² = 11.8, p < 0.05).

For poor readers, line 1 differed significantly from lines 3, 4, and 5. Line 2 differed significantly from lines 3, 4, and 5 and line 3 differed from lines 4 and 5. For the moderate readers, lines 1 and 2 differed significantly from line 3.

Inspection of Figure 9 shows that line 3 had the most fixations per word for all groups. This confirms the supposition that barkers on line 3 had many short fixations, accounting for the slower speed. To reiterate, line 3 was the shortest and did not contain any difficult words or concepts and, in particular, the barkers and the poor readers may have realised that the text in line 3 was understandable and hence they tried to read with more comprehension and cognitive processing, thus causing an increase in the number of fixations per word. An increase in cognitive processing also corresponds to the increase in fixation duration for poor readers. However, the same phenomenon is not seen with the barkers, who had lower fixation duration here, hence they seem to be processing the words by fixating on them more with short fixations, perhaps on a per syllable basis or with many refixations in order to understand the text. As mentioned previously, a word-by-word analysis will provide more details on this but is beyond the scope of this chapter.

6. Discussion

The following research questions were posed at the start of this chapter.

1. How do the reading eye-movements of those participants who were barking at print compare to those of their peers of two levels of reading proficiency?

2. What are the eye movement characteristics of those participants who were barking at print?

3. How do the eye-movement characteristics of barking at print compare to those published for skimming, scanning and reading for meaning at various levels of proficiency?

The first two research questions will be answered together. There were no significant differences detected between barkers and non-barkers for any of the standard reading metrics. However, barkers do exhibit lower fixation durations, although durations are of a higher magnitude typical of second language reading, fewer regressions, coupled with a lower regression percentage. They also have fewer fixations and longer saccades although only marginally so. During first pass reading, barkers have shorter first fixation durations and shorter visits per word but the number of fixations and visits per word are similar to non-barkers. Barkers tend to adjust their mean fixation durations to the length and/or difficulty of the line currently being read as they read slower on easier and shorter sentences as a result of many short fixations, suggesting either more regressions or more fixations per word on the first pass reading.

In terms of fixation durations for skimming, scanning and thorough reading, barkers have shorter durations such as with skimming. Similarly, the longer saccades and fewer regressions [47] are similar to skimming. An in-depth comparison of these types of reading with our barkers will determine if there are significant differences or not but for now an anecdotal description can be provided. When mind wandering, first fixations and fixations are longer and number of regressions [42] and fixations are lower and the length of the saccades are shorter [43]. Hence, apart from the number of regressions, barking is not comparable to mind wandering on this level.

7. Significance, limitations, further investigation

Although this pioneering research into eye-movements during barking at print may have raised more questions than it has resolved, its significance particularly lies in pointing out that eye movements during barking at print are distinct from kinds of reading which exist in the eye-tracking literature. Findings from research into mindless reading and mind-wandering during reading, which are both associated with decoding with negligible comprehension, differ from the findings presented here in manners which show that comprehension, at least at the word level, for at least some of the words, does indeed affect eye-movement metrics. Except for the ‘jumping barkers’, eye movement was not found to be erratic in barking at print, whereas eye movement during mind-wandering is. Fixation durations of the barkers were shorter than those of the readers in this study and similar to second language reading, whereas mind-wandering and mindless reading are known to be associated with longer fixation durations than normal reading. Significant variation in eye-movement metrics between the lines of text, during barking at print, suggest changes in cognitive activity in response to textual features. In contrast, mind-wandering (cf. [5, 42, 43]) and mindless reading [15] are both associated with considerable uniformity in these metrics across lines. The low presence of regressions, however, is a point of similarity to both mind-wandering and mindless reading.

This chapter has provided a general description, with tentative metric ranges, for barking at print, at least by learners in this context. However, detection of these metrics does not necessarily diagnose barking, as evidenced by the lack of statistical significance between the barkers and the non-barkers in this sample. Therefore, detection of such metrics should be seen as indicating a high likelihood of barking, rather than as necessary detection of barking, with additional research required to enhance the validity of the diagnosis on the basis of eye-movements. Two limitations in this research are considered to have contributed to this lack of precision: (1) the assumption that regular eye-movement across text, at least for part of the text, indicates a high enough degree of decoding proficiency for a reader to potentially engage in barking at print (recall that in order to bark a reader must be able to decode); (2) the possibility that some of the learners classified as readers may also have been barking at print while reading screen 1, obscuring differences between the groups for the analysis performed here. Barking was deduced from the eye-movement behaviour and comprehension scores obtained as the learners answered questions on screen 2, which had screen 1’s text repeated on one half of the screen, allowing the participants to re-read the text. It is possible that participants mitigated barking on screen 1 by undergoing reading with comprehension as they referred back to the text on screen 2, thus being able to answer the questions with reasonable comprehension, and therefore being categorised in one of the two non-barking groups despite their metrics actually displaying barking. This could explain the insignificant difference between the groups. If this is the case, it would mean that barking could be seen as an additional reading gear which at least some people can move into or out of depending on expectations, such as whether the reader realises that he/she is expected to answer questions about the text or not.

Future research could address these limitations by: (1) testing decoding accuracy explicitly using a fluency test while learners read text out loud (2) requiring the participants to answer questions immediately after reading, without being given the opportunity to refer back to the text. The limitation caused by the assumption that decoding proficiency is sufficient for barking to even be a possibility, is reduced by the fact that these were grade 8 and 9 learners reading in the language of learning and teaching (LoLT) which they had been schooled in for at least 5 years, and that the majority had been identified by their teachers as being academically strong. Therefore, we can claim with considerable confidence that at least those learners who were classified as barking at print were indeed doing so. Our findings about the characteristics of these learners’ eye movements during barking are therefore not negated by the possibility that some of the learners classified in other groups were actually also barking in the analysed data. We predict that the methodological changes suggested above would result in the same trends being found between the groups as we have reported here, but that these differences would then be statistically significant. Future research could also explore the variations in eye-movement metrics between the lines and, where appropriate, between the words, of the text, in an attempt to understand the cognitive processing the barkers are undergoing as they bark at print.

8. Conclusion

Given the pivotal role the ability to read with comprehension plays in cognitive development and academic achievement, it is vital that we enhance our understanding of reading difficulties, such as barking at print. This is done with the view of eventually being able to inform application of this knowledge to providing effective interventions. Eye-tracking technology is particularly powerful as a research method since it exposes the otherwise invisible and individual process which readers undergo. It is also a potentially powerful tool for intervention, once a phenomenon is understood sufficiently for valid application. This research has begun the investigation into understanding of a reading difficulty which is highly prevalent among the poor and marginalised, while also being in no way absent from more developed and affluent communities. Once a definitive classification can be proposed for readers in disadvantaged areas in South Africa, diagnostic and intervention programmes can be developed. Learners who are struggling can be evaluated using the diagnostic tools in order to determine whether their reading behaviour could be the reason for poor performance in an academic setting. Once each learner has been classified, the intervention programmes designed for their particular group can be applied. In this way, these learners can be identified and assisted and this could eventually lead to improved performance for these learners.