Introduction
Rationale
Experiences mediated by technology (e.g. printed books, screens, and virtual reality) are studied across a variety of disciplines, often with little cooperation. Different theorizations, models, and empirical tools have been developed, resulting in a fuzzy agglomerate of related and overlapping concepts, like presence (Lombard et al., 2015), flow (Csikszentmihalyi, 1990; Harmat et al., 2016), and narrative absorption (Hakemulder et al., 2017). A scoping review is a suitable method to identify and summarize the core aspects of these various concepts, since they are currently obscured by the heterogeneity of disciplines investigating them. The questionnaires most used in empirical research regarding this kind of psychological phenomena have been surveyed and the items in each questionnaire have been categorized based on their wordings, thus independently from the conceptual models within which they have been developed. Overlapping concepts have been formulated in different fields according to specific disciplinary interests and based on knowledge within each field, this review focuses on how language is actually used in questionnaire items, rather than on how concepts are formulated top-down and arbitrarily associated with corresponding linguistic expressions that become items of a questionnaire.
Objectives
The goal is to highlight similarities and overlaps between questionnaires’ items in order to identify which are the most relevant aspects of the psychological phenomena labelled as “presence,” “flow,” and “narrative absorption.” Based on this categorization, the domains to which each group of items can be ascribed (e.g. space, realism, agency, etc.) will be suggested and they will be associated to the respective psychological phenomena for which they are more frequently used (e.g. presence, social presence, narrative absorption, etc.).
Methods
Eligibility criteria
The sources considered are questionnaires available in English, no year limit has been used. To be included in the review, questionnaires need to have been developed or used for research about one of the following media: VR, video game, film, book. Only questionnaires measuring psychological states have been included, not those measuring personality traits or broader psychological concepts (e.g. empathy). Validation and statistical reliability were not necessary criteria.
Search
The queries used in Google Scholar are: “presence questionnaire,” “immersion questionnaire,” “flow questionnaire,” “narrative questionnaire,” “narrative engagement,” “narrative absorption,” “narrative transportation.”
Selection of sources of evidence
Information about questionnaires has been obtained directly from published articles and also from reviews included in Master theses or PhD dissertations. The criterion used to consider a questionnaire eligible as a source of evidence is its application in recent years: once a questionnaire has been identified, its use in research starting from the year 2000 has been checked. This selection has been made with the help of a review of the questionnaires most used in VR research in the years 2016–17 (Hein et al., 2018).
Data charting process
When multiple versions of a questionnaire were available, only the most recent or shortest version has been considered, since this is likely to be an improvement over previous or longer versions. Each item of the data was recorded in a spreadsheet following manual annotation.
Data items
Being a data-driven bottom-up review, no specific variables have been defined a priori. Rather, all questionnaires’ items have been analyzed. Among the total items in all the questionnaires studied, I only grouped and categorized the items for which I found close similarities and overlap of wordings.
Critical appraisal of individual sources of evidence
From a preliminary screening, I found that some items inquire about more than one aspect of the target experience. During the analysis, I identified all such items and excluded them from the synthesis of results, in order to avoid confusion with respect to the aspect covered by each type of item.
Synthesis of results
Items of the selected questionnaires have been compared and grouped according to similarities in the wordings used. For instance, the narrative absorption item “When I was finished with reading the story it felt like I had taken a trip to the world of the story” (Kuijpers et al., 2014) strongly resembles the spatial presence item “After my experience of the displayed environment, I had a sense that I had returned from a journey” (Lessiter et al., 2001). Once various clusters of items have been identified, each group has been labeled and linked to the most relevant psychological phenomenon. When items where already grouped in subdimension of the broader psychological construct, the subdimensions have been used as guidance for the classification.
Results
Selection of sources of evidence
The process of selection is outlined in Figure 1.
Figure 1. Flow diagram of the selection of sources process (adapted from Moher et al., 2009).
Characteristics of sources of evidence
The questionnaires analyzed are listed in Table 1. Out of the 23 questionnaires included in the analysis, 8 have been developed to measure presence, 3 for flow, 6 for game immersion/engagement, and 6 for narrative phenomena (absorption, engagement, transportation, immersion, identification with characters, and empathy with characters).
Table 1. Questionnaires analyzed and categorized.
Total number of items, n= 484.
| Questionnaire | Type | Total number of
items | Number of selected
items |
|---|
| 1 | Temple Presence Inventory (TPI) (Lombard et al., 2000) | Presence | 42 | 18 |
| 2 | Slater, Usoh and Steed (SUS) (Usoh et al., 2000) | Presence | 6 | 3 |
| 3 | Sense of Presence Inventory (ITC-SOPI) (Lessiter et al., 2001) | Presence | 44 | 23 |
| 4 | Igroup Presence Questionnaire (IPQ) (Schubert, 2003) | Presence | 14 | 10 |
| 5 | Networked Minds Social Presence Inventory (NMSPI) (Harms & Biocca, 2004) | Presence | 36 | 33 |
| 6 | Presence Questionnaire, version 3 (PQ) (Witmer et al., 2005) | Presence | 29 | 10 |
| 7 | Spatial Presence Experience Scale (SPES) (Hartmann et al., 2016) | Presence | 8 | 8 |
| 8 | Multimodal Presence Scale (MPS) (Makransky et al., 2017) | Presence | 15 | 12 |
| 9 | Flow Short Scale (FSS) (Rheinberg, 2008) | Flow | 13 | 8 |
| 10 | EduFlow Scale (EFS) (Heutte et al., 2014) | Flow | 12 | 8 |
| 11 | Reading Flow Short Scale (RFSS) (Thissen et al., 2018) | Flow | 8 | 6 |
| 12 | EGameFlow (EGF) (Fu et al., 2009) | Game/Flow | 42 | 16 |
| 13 | Immersion in the Narrative Game Questionnaire (INGQ) (Qin et al., 2009) | Game | 27 | 18 |
| 14 | Game Engagement Questionnaire (GEQ) (Brockmyer et al., 2009) | Game | 19 | 12 |
| 15 | User Engagement Scale (UES) (O’Brien & Toms, 2013; Wiebe et al., 2014) | Game | 28 | 15 |
| 16 | Game Experience Questionnaire (GExQ) (IJsselsteijn et al., 2013) | Game | 40 | 26 |
| 17 | Game Immersion Questionnaire (GIQ) (Cheng et al., 2015) | Game | 14 | 7 |
| 18 | Transportation Scale (Green & Brock, 2000) | Narrative | 11 | 10 |
| 19 | Identification Scale (Cohen, 2001) | Narrative | 10 | 9 |
| 20 | Narrative Engagement Scale (NES) (Busselle & Bilandzic, 2009) | Narrative | 12 | 12 |
| 21 | State Empathy Scale (SES) (Shen, 2010) | Narrative | 12 | 9 |
| 22 | Story World Absorption Scale (SWAS) (Kuijpers et al., 2014) | Narrative | 18 | 17 |
| 23 | Film Immersion Questionnaire (FIQ) (Jennett et al., 2008; Rigby et al., 2019) | Narrative | 24 | 18 |
Critical appraisal within sources of evidence
Some items present a combination of more than one aspect and they have been excluded from the synthesis of the results in order to avoid confusions within each group of items. For instance, the item “I lose perceptions of time and the real world surrounding me, as if everything just stops” (Game Immersion Questionnaire, Cheng et al., 2015) asks about perception of both time and space. Other items have been excluded even though they inquire about some of the aspects identified, because they have peculiar wordings that do not overlap with other items. Out of the total 484 items, 308 have close similarities and overlapping of wordings.
Results of individual sources of evidence
Table 1 reports the number of items selected in each questionnaire.
Synthesis of the results
The complete categorization of the questionnaire items can be found in the underlying data. A summary of the most frequent categories is reported in Table 2. Attention is undoubtedly the most relevant term for all the constructs considered, conceived as disregard for both thoughts and perceptions that are not part of the activity eliciting presence, flow, or absorption. Similarly, a distorted perception of time is in many cases considered to be a sign of the occurrence of all the considered phenomena. With respect to categories specific to each concept, spatial presence is characterized by items related to space, agency, and a comparison with reality non mediated by technology. Social presence is characterized by the same categories that are relevant for presence (space and agency) but in relation to the existence of other agents; additionally, some kind of cognitive attention to the other and emotional arousal elicited by them are also frequent. Flow is specifically characterized by the perception of a sense of challenge. Narrative absorption is characterized by a comparison with non-mediated reality (in terms of vividness of imagery), by an easy comprehension of content, and by emotions and thoughts anticipating possible outcomes (suspense). Lastly, there are two groups of items explicitly asking about the user’s perception of involvement/engagement or absorption/immersion.
Table 2. Categorization of items (n = 308) from presence, flow, game, and narrative questionnaires.
Total
items | Scales
with item | Item type | Category | Main psychological
phenomenon |
|---|
| 23 | 11 | Attention (no external thoughts) | Attention | Attention |
| 17 | 9 | Attention (no external perceptions) |
| 18 | 11 | Time distortion | Time | – |
| 17 | 9 | “Being there” (feelings and perceptions, not
thoughts) | Space | Spatial presence |
| 8 | 5 | Realities overlapping |
| 6 | 3 | Closeness of story world |
| 7 | 6 | Return to reality |
| 5 | 5 | Being part of the action (also partly overlaps with
"being there") |
| 10 | 5 | Possibility of action in space | Agency |
| 6 | 4 | Control of content |
| 5 | 3 | Control of medium |
| 9 | 6 | Naturalness/fluency of medium use |
| 14 | 6 | Perceived realism | Comparison |
| 5 | 2 | Attention to another agent | Attention | Social presence |
| 5 | 4 | Co-location with another agent | Space |
| 24 | 4 | Mind reading | Cognition |
| 5 | 2 | Behavioural response to another agent | Agency |
| 13 | 7 | Matching of another agent 's emotions | Emotion |
| 4 | 3 | Feelings for another agent |
| 6 | 5 | Connection with another agent | Emotion/
Cognition |
| 16 | 6 | Understanding of another agent (perspective
taking, cognitive empathy) | Cognition |
| 12 | 7 | Challenge | Cognition | Flow |
| 8 | 4 | Vividness of imagery | Comparison | Narrative absorption |
| 14 | 7 | Comprehension of content | Comprehension |
| 9 | 6 | Suspense/anticipation | Emotion/
Cognition |
| 18 | 10 | Emotional response to medium/content | Emotion | Emotional impact |
| 14 | 7 | Explicit use of involvement/engagement terms | Metaphor | – |
| 10 | 9 | Explicit use of absorption/immersion terms |
Discussion
Summary of evidence
In all questionnaires, the most frequently recurring items concern attention and the sense of time. The isolation from external thoughts and perceptions is the main characteristic of presence-related phenomena, and such disconnection from stimuli unrelated to the undergoing experience probably leads to an alteration of the sense of time. Despite the evolution towards broad psychological conceptions of presence (Baños et al., 2000; Lee, 2004; Riva et al., 2015), a review (Hein et al., 2018) of the psychometric questionnaires used in VR research in the years 2016–17 found that the most used one is the Presence Questionnaire (Witmer & Singer, 1998), which heavily focuses on visual realism and naturalness of interaction. However, the broadest and most protracted collective effort aimed at clarifying how to measure presence (Hartmann et al., 2016; Vorderer et al., 2004) has excluded realism from the subdimensions of presence, keeping only “self-location” and “possible action” as core dimensions. Indeed, these two categories seem to be the two really specific to presence, since a comparison with non-mediated reality is also relevant for the “imagery” category, which concerns items related to narrative absorption. Inquiring about the vividness of imagery or about the realism of a VR scene is a way to check how similar the imagined/mediated experience is to a non-mediated one. Both realism and vivid imagery are outcomes that can be associated to presence, but they are not particularly helpful to explain the underlying psychological processes that bring the emergence of a sense of presence.
Many questionnaires also take into account the possibility that perceiving the existence of other agents can affect our sense of presence or, more broadly, that we can have intense experiences when interacting with others or following their actions. With a growing degree of complexity, such perception goes from merely noticing the existence of others, to interacting with them, to emotional and cognitive ways of responding to and understanding others’ mental states. These groups of items, that I have associated to the concept social presence, often occur together with presence items and seem to entail it as the basis on top of which they can emerge. Indeed, they are all different expressions of a self-other relationship and can be conceptualized as forms of presence in co-participation. Analogously, questionnaires about flow experiences include items that I have here associated to presence – and in some cases also items related to social presence – plus a specific group of questions regarding the perception of an experience as challenging. Similar wordings can be also found in items of narrative and game questionnaires.
Items that I specifically associated to the concept of narrative absorption regard imagery, the feeling of suspense triggered by the narrated events, and the comprehension of the content of the story, an aspect which can be connected to the sense of challenge of flow experiences, since the right match between the complexity of a story and the cognitive skills of the reader is relevant for narrative absorption. It is worth noting that questionnaires investigating narrative absorption include these three groups but also items related to presence and social presence (with characters of a story), which can be considered subdimensions of narrative absorption. Given their metaphorical nature, items explicitly asking whether an experience elicited involvement, engagement, immersion, or absorption are not particularly useful for describing the psychological processes activated during the experiences they aim at qualifying. Moreover, “immersive” is used in VR research as a technical attribute of the medium – consistently with Sheridan seminal definition (Sheridan, 1992) – whereas in game and narrative studies it is a quality of the player or reader’s experience (Jennett et al., 2008; Ryan, 2015; Stockwell, 2019).
Another popular but quite heterogeneous group of questions concerns the emotional impact of mediated experiences. Ten questionnaires investigate this aspect in slightly different ways, so it is hard to say whether emotional impact is a component of any of the presence-related phenomena or a secondary effect elicited by them.
Based on the recognition presented, a cross-disciplinary systematization of concepts is possible. To sum up, attention and time distortion are common to all the considered phenomena, and presence (space and agency) is the phenomenon with the narrowest scope, the core. Social presence and narrative absorption are phenomena of increasingly broader scope, each of them including the listed phenomena of narrower scope. Flow is a concept transversal to the other three, being more related to the balance between a person’s skills and the complexity of the stimulus, rather than to a specific psychological dimension.
Limitations
Categorizing only 308 items, out of the total 484 found in the sampled questionnaires, this scoping review may have missed some aspects of presence and related concepts that are important to grasp the nuances of the phenomenal experience that may be specific to certain media. However, by focusing on items showing a recurring intersubjective agreement between researchers and disciplines, I think I have successfully identified and summarized the core aspects of the surveyed phenomena.
Conclusions
The categorization proposed here can be used to further refine existing questionnaires and possibly encourage a convergence of different disciplines towards a use of the same items, so that insight coming from different fields could be used for the advancement of knowledge in specific areas. For instance, empirical research on narrative could benefit from using existing items for presence and social presence, without “reinventing the wheel” and focusing rather on refining how to measure dimensions like suspense and imagery. Moreover, a shared agreement on basic items will enable better and more informative meta-analyses, as well as comparative media studies, a kind of research that is strongly relevant for all the disciplines that I mentioned here, since only a comparison between experiences with different media can help to account for the specificity of presence and related phenomena.
Following the above-mentioned strategy, in Table 3 and Table 4 I present my suggestion for a selection of items to be used to measure presence, social presence, and narrative absorption, with the aim of achieving a more solid epistemic comparability among research on these themes. In order to benefit from previous statistical validations, in case of similarities, I gave preference to items coming from the same questionnaire. Depending on the task/content with which the participants are engaging, only a part of these items may be relevant.
Table 3. Selection of questionnaire subdimensions recommended to achieve a more solid epistemic comparability among research on presence, social presence, and narrative absorption.
| Item type | Category | Recommended questionnaire
subdimension | Main
psychological
phenomenon |
|---|
| Attention (no external thoughts) | Attention | NES by Busselle & Bilandzic (2009)
– “Attentional focus” | Attention |
Attention (no external
perceptions) | PQ v.3 by Witmer et al. (2005)
– “Adaptation/Immersion” /
FIQ by Rigby et al. (2019) –
“
Real-world Dissociation” |
| Time distortion | Time | Various | – |
“Being there” (feelings and
perceptions, not thoughts) | Space | SPES by Hartmann et al. (2016) – “Self-
location” | Spatial presence |
| Realities overlapping |
| Closeness of story world |
| Return to reality |
Being part of the action (also
partly overlaps with "being there") |
| Possibility of action in space | Agency | SPES by Hartmann et al. (2016) – “Possible
action” |
| Control of content |
| Control of medium |
Naturalness/fluency of medium
use |
| Attention to another agent | Attention | NMSPI by Harms & Biocca (2004) –
“Perceived Attentional Engagement” | Social presence |
| Co-location with another agent | Space | MPS by Makransky et al. (2017) – “Social
presence” |
| Mind reading | Cognition |
Behavioural response to another
agent | Agency | NMSPI by Harms & Biocca (2004) –
“Perceived Behavioural Interdependence” |
Matching of another agent 's
emotions | Emotion | NMSPI by Harms & Biocca (2004)
– “Perceived Emotional Contagion” / SES by
Shen (2010) – “Affective empathy” |
| Feelings for another agent |
| Connection with another agent | Emotion/Cognition |
Understanding of another agent
(perspective taking, cognitive
empathy) | Cognition | NMSPI by Harms & Biocca (2004) –
“Perceived Comprehension” / SES by Shen (2010) – “Cognitive empathy” |
| Challenge | Cognition | RFSS by Thissen et al. (2018) – “Absorption” | Flow |
| Vividness of imagery | Comparison | SWAS by Kuijpers et al. (2014) – “Mental
imagery” | Narrative
absorption |
| Comprehension of content | Comprehension | NES by Busselle & Bilandzic (2009)
– “Narrative understanding” |
| Suspense/anticipation | Emotion/Cognition | Transportation Scale by Green & Brock (2000) – “Transportation” |
Table 4. Selection of questionnaire items (with minimal adaptation) recommended to achieve a more solid epistemic comparability among research on presence, social presence, and narrative absorption.
(R = reverse scored).
| Item | Item type | Recommended questionnaire
subdimension | Main
psychological
phenomenon |
|---|
| 1 | While [task/content] I found
myself thinking about other
things. [R] | Attention (no external
thoughts) | NES by Busselle & Bilandzic (2009)
– “Attentional focus” | Attention |
| 2 | I had a hard time keeping my
mind on the [task/content]. [R] |
| 3 | I was able to concentrate very
well on [task/content] rather
than on the mechanisms used
to [perform/represent] that
[task/content]. | Attention (no external
perceptions) | PQ v.3 by Witmer et al. (2005)
– “Adaptation/Immersion” |
| 4 | I didn’t notice events taking
place around me. | FIQ by Rigby et al. (2019) – “Real-
world Dissociation” |
| 5 | I lost track of time. | Time distortion | Various | – |
| 6 | I felt like I was actually there
in the environment of the
presentation. | Self-location | SPES by Hartmann et al. (2016)
– “Self-location” | Spatial presence |
| 7 | It seemed as though I actually
took part in the action of the
presentation. |
| 8 | It was as though my true
location had shifted into
the environment in the
presentation. |
| 9 | I felt as though I was physically
present in the environment of
the presentation. |
| 10 | The objects in the presentation
gave me the feeling that I could
do things with them. | Possible action | SPES by Hartmann et al. (2016)
– “Possible action” |
| 11 | I had the impression
that I could be active in
the environment of the
presentation. |
| 12 | I felt like I could move around
among the objects in the
presentation. |
| 13 | It seemed to me that I
could do whatever I wanted
in the environment of the
presentation. |
| 14 | I paid close attention to [other
agent/s]. | Attention to another agent | NMSPI by Harms & Biocca (2004) – “Perceived Attentional
Engagement” | Social presence |
| 15 | I was easily distracted from
[other agent/s] when other
things were going on. [R] |
| 16 | I felt like I was in the presence
of someone else while [task/
content]. | Co-location with another
agent | MPS by Makransky et al. (2017)
– “Social presence” |
| 17 | I felt that the [other agent/s]
in [place] were aware of my
presence. | Mind reading |
| 18 | The [other agent/s] in [place]
appeared to be sentient
(conscious and alive) to me. |
| 19 | My actions were often
dependent on [other agent/s’] actions. | Perceived Behavioural
Interdependence | NMSPI by Harms & Biocca (2004) – “Perceived Behavioural
Interdependence” |
| 20 | My behavior was often in direct
response to [other agent/s’] behavior. |
| 21 | What [other agent/s] did often
affected what I did. |
| 22 | I was sometimes influenced by
[other agent/s’] moods. | Affective empathy | NMSPI by Harms & Biocca (2004)
– “Perceived Emotional Contagion” |
| 23 | I experienced the same
emotions as the [other agent/s]
while [task/content]. | SES by Shen (2010) – “Affective
empathy” |
| 24 | I could feel the [other agent/s’]
emotions. |
| 25 | I was able to understand what
[other agent/s’] meant. | Understanding of another
agent (perspective taking,
cognitive empathy) | NMSPI by Harms & Biocca (2004)
– “Perceived Comprehension” |
| 26 | I can see the [other agent/s’]
point of view. | SES by Shen (2010) – “Cognitive
empathy” |
| 27 | I can understand what the
[other agent/s’] was going
through. |
| 28 | I felt optimally challenged while
[task/content]. | Challenge | RFSS by Thissen et al. (2018)
– “Absorption” | Flow |
| 29 | When I was reading the story,
I had an image of the main
character in mind. | Vividness of imagery | SWAS by Kuijpers et al. (2014)
– “Mental imagery” | Narrative
absorption |
| 30 | When I was reading the story,
I could see the situations
happening in the story being
played out before my eyes. |
| 31 | I could imagine what the world
in which the story took place
looked like. |
| 32 | At points, I had a hard time
making sense of what was
going on in the story. [R] | Comprehension of content | NES by Busselle & Bilandzic (2009)
– “Narrative understanding” |
| 33 | I wanted to learn how the story
ended. | Suspense/anticipation | Transportation Scale by Green & Brock (2000) – “Transportation” |
Data availability
Underlying data
OSF: Presence, flow, and narrative absorption questionnaires: a scoping review
https://doi.org/10.17605/OSF.IO/RBZ8G (Pianzola, 2021)
This project contains the following underlying data:
scoping_review_data_2021-02-26.xlsx (Human-readable version containing the 23 selected questionnaires with color coding of the items and summary model)
scoping_review_data_2021-02-26.csv (Machine-readable version containing the 23 selected questionnaires with the respective annotations for each item)
Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).
