Introduction
In contemporary theories of emotions, the term affect refers to a neurophysiological state characterized by either positive or negative valence (Lindquist et al., 2016; Barrett, 2004; Diener et al., 1999; Feldman Barrett & Russell, 1998; Russell & Carroll, 1999; Watson et al., 1999). This state is considered a core component of emotions, forming the basis upon which individuals interpret and label their emotional experiences (Russell & Pratt, 1980). Affect plays a central role in shaping subjective emotional experiences, influencing cognitive processes, mental health and well-being (Dejonckheere et al., 2018).
The PANAS - Positive Affect and Negative Affect Schedule (Watson et al., 1988) is the most widely translated and adapted scale for assessing affect worldwide. Rigorous instrument adaptations are necessary for developing affect scales because emotion semantics exhibit great cultural variability (Jackson et al., 2019). Significant differences in the networks of emotion concepts were observed across languages, which are influenced by the geographic proximity of language families (Jackson et al., 2019). Moreover, emotion concepts vary across cultures, are shaped by contextual factors, and arise from the interplay between brain processes and cultural experiences, highlighting the need for culturally sensitive measurement tools (Lindquist et al., 2016; Russell & Pratt, 1980; Barrett, 2004).
The original PANAS was developed in English using a method that has been shown to provide good indices of reliability and validity (Seib-Pfeifer et al., 2017; Flores-Kanter et al., 2021; Medvedev et al., 2023). However, few studies have developed culturally sensitive instruments for measuring affect in other languages and cultures (Torres et al., 2023), with most instruments relying on direct translations of the original PANAS. The exceptions to date include a Japanese (Watson et al., 1984), a European Portuguese (Galinha & Pais-Ribeiro, 2012) and a Estonian version (Alllk & Realo, 1997) of the affective scale.
Besides methodological issues, some underlying theoretical assumptions of the PANAS might not be universally valid. The factor structure of the original scale comprises two independent factors (positive and negative) of a single construct (affective valence; Feldman Barrett & Russell, 1998; Watson & Tellegen, 1999; Diener, 1999; Russell & Carroll, 1999). However, some recent studies using factor analysis have found that models with three or more factors provide better fits for PANAS in different cultural contexts (see Torres, et al., 2023 for details). Moreover, there is compelling evidence suggesting that the affective variables are correlated (DeVellis, 2016; Damásio, 2012), mainly because affective and emotional constructs tend to overlap. Nonetheless, most PANAS studies have used orthogonal models (uncorrelated factors) 2-factor structures, with positive affect loading on the first factor and negative affect loading on the second factor (Watson et al., 1988; Terracciano et al., 2003; Gaudreau et al., 2006; Pandey & Srivastava, 2008; Mihic et al., 2014; López-Gómez et al., 2015; DeVellis, 2016).
Given these current debates in affective sciences and the lack of a properly constructed affective and emotional scale tailored for the Brazilian population, we set out to develop and test the reliability and validity of such an instrument. The development of the EAPB- Escala de Afeto do Português Brasileiro is described in detail in Torres et al. (2022) https://paperpile.com/c/HAX1ni/XBVH). Briefly, the three main classical stages of scale construction were followed: conception, development, and evaluation (Dawis, 1987). In the conception phase, we grounded the theoretical framework on the importance of taking contextual variations on affect and emotion into account, ensuring cultural and linguistic specificity for the Brazilian population. Hence, we developed the new instrument directly in the language of the participants, which is more appropriate than the adaptation/translation of scales that already exist in other languages (Watson et al., 1984). This development procedure avoids any difficulty in the translation of words that do not exist in other languages https://paperpile.com/c/HAX1ni/jKbp(International Test Commission, 2018
https://paperpile.com/c/HAX1ni/jKbp). Specifically, since cross-cultural studies have also shown structures with more than two factors (Boudhiba & Kridis, 2019; Mihic et al., 2014), we did not a priori assume a two-factor structure in the Brazilian sample.
Importantly, the EAPB was developed using the same methodology used to develop the original PANAS and agreement and advice was obtained for the authors of the original scale. The development of the EAPB began with the creation of an affective lexicon consisting of 404 words, which was subsequently reduced using psycholinguistic criteria (e.g., familiarity, semantics). A reduced list of 139 words was selected for an online synonym grouping task, which was administered to a country-wide Brazilian sample (N = 1000). An exploratory factor analysis with oblimin rotation was conducted and the words with the highest factor loadings were chosen to compose the 26 items (13 synonyms pairs) of the EAPB used here.
In the present study, we aimed to evaluate the reliability, construct and content validities of the newly developed EAPB in a large and independent sample. For this purpose, we collected data on the initial 26 items of the EAPB in a representative sample of the Brazilian population (N = 3479 participants recruited). Standard metrics for reliability and consistency are reported and exploratory factor analysis, followed by confirmatory factor analysis were used to investigate the internal validity of the EAPB.
Methods
Ethics information and open science information
This study received approval from the Research Ethics Committee of ABC Federal University on April 23, 2019 (CAAE: 1247318.9.0000.5594; Opinion number: 3.752.357). Informed consent was obtained from all participants prior to the beginning of the study. Written consent was obtained on the data collection app after the presentation of participation term with a copy also sent to participants by email. The study was preregistered on the Open Science Framework (OSF; https://doi.org/10.17605/OSF.IO/DMQVG). The fully reproducible code and necessary dataset are hosted on GitHub and archived on Zenodo (https://doi.org/10.5281/zenodo.22033554).
Participants
A total of 3479 participants (female = 52.66%, male = 47.34%) were recruited from all regions of Brazil (North = 8.7%, Northeast = 26.6%, Midwest = 8.4%, Southeast = 41.28% and South = 14.93%) proportionally to the most recent national census data (IBGE Instituto Brasileiro de Geografia e Estatística, 2022). From the recruited participants, 2986 consented to participate and completed the first wave of data collection (first week). A total of 1883 participants remained in the subsequent wave (second week), resulting in an attrition rate of 36.96% (data on the second week was used only for calculating test-retest in the present study). From the 2986 participants in the first week, 1493 were randomly selected for exploratory factorial analysis and the remaining half of the sample was selected for confirmatory factorial analysis. Importantly, the exploratory and confirmatory subsamples had no overlap. The age of the participants ranged from 18 to 83 years (M = 37.96, SD = 12.01). The socio-economic status varied from class A (6.9%), B1 (7.9%), B2 (13.3%), C1 (37.7%), C2 (24.9%) and DE (9.3%) according to the Brazilian Economic Classification Criterion (CCEB).
Measures
Demographic information, including age, gender, socioeconomic status and city of residency were acquired. The first version of the EAPB scale comprising 26 items representing emotion words was collected. All participants self-reported, using an online platform (Pinion app), how much of each of the following 26 emotions they were feeling at the moment from 0 (not at all) to 100 (Extremely) on a Visual Analog Scale: “Raivoso” (enraged), “Bravo” (angry), “Triste” (sad), “Infeliz” (unhappy), “Determinado” (determined), “Disposto” (motivated), “Desrespeitado” (disrespected), “Humilhado” (humiliated), “Amedrontado” (frightened), “Aterrorizado” (terrified), “Feliz” (happy), “Alegre” (joyful), “Encantado” (delighted), “Fascinado” (fascinated), “Relaxado” (relaxed), “Aliviado” (relieved), “Ansioso” (anxious), “Inquieto” (restless), “Exausto” (exhausted), “Cansado” (tired), “Abandonado” (abandoned), “Sozinho” (lonely), “Chocado” (shocked), “Pasmo” (astonished), “Duvidoso” (doubtful), and “Desconfiado” (suspicious).
Data collection
The recruitment process was managed by Pinion, an engagement platform designed to mobilize users for mobile data collection. The data was collected in September and October 2022. Participants completed the EAPB for 7 consecutive days (2022/09/12 to 2022/09/18), once a day, at different hours each day, followed by a 14-day break. After the break, participants answered again for 7 consecutive days (2022/10/03 to 2022/10/09). On the last day of each week, participants answered an additional questionnaire asking about unexpected events, sleep quality and medication use. When the participants skipped more than two consecutive days, they were excluded. For the analyses reported here, the first day of the first week of data collection was used for reliability and validity analyses. For the test-retest reliability, data from the second day of both weeks were used (then collected 21 days apart).
Statistical analysis
Structural Validity
The total of 2986 participants were divided in 2 randomized sub-samples with 1493 participants each. Each group included a sufficient number of participants to meet the recommended ratio of at least 5 participants per item, totalling 57 participants per item (MacCallum et al., 1999; Worthington & Whittaker, 2006; Cattell, 2012).
The EFAs were conducted as a first approach to investigate the instrument’s structure. We used both orthogonal and oblique rotations. Structural modifications, guided by modification indices. We tested solutions for 2, 3,4,5, 6, 7, 8 and 10 extracted factors. Based on the results of EFA and systematic index modification analyses, we used Confirmatory Factor Analyses (CFA) in an independent sample for testing an 8 factors solution for the 21-items scale version and a 2 factors solution for a short version (9 items). Analyses are fully detailed in the Supplementary Material file and data and code are open and each step can be fully reproduced.
Reliability analyses
To assess the EAPB’s reliability, we conducted internal consistency calculations using McDonald’s Omega (DeVellis, 2016; Goodboy & Martin, 2020; Kalkbrenner, 2023), both at global and factor levels. Though commonly used as an estimate of reliability in the literature, Crombach alpha can inflate reliability if covariances between items happen due to random error (Kalkbrenner, 2023). In this case, the Omega coefficient is the option of choice for investigating scale reliability (Cortina et al., 2020; Kalkbrenner, 2023).
We also conducted a global test-retest analysis to examine the temporal stability of the scale, using data of the second day on both waves. The results were compared to assess whether the scale produces consistent responses over time.
Results
Structural validity
Exploratory factorial analysis
The KMO measure for the data was 0.94 and Bartlett’s test was significant, χ2(325) = 21487.29, p < .001, indicating that the correlation matrix was factorable. Parallel analysis (Horn, 1965) and scree test (Cattell, 1966) indicated that a 5-factor solution was the most appropriate representation of the data, while EKC suggested a 4-factor and the Lower Bound of RMSEA supported a 8-factor solution. We thus investigated 2 and 3 factors (based on the original PANAS and cultural adaptations), 4, 5, 6, 7 and 8 factors (based on parallel analysis, scree test and results on other PANAS versions) and a 10 factors solution (based on metrics supporting more than 10 factors). Solutions beyond 10 factors were not pursued due to anticipated identification concerns. Detailed comparative fit indices for these models are in Supplementary Material.
Modification indices analyses
Before confirmatory factor analysis, we performed item-level diagnostics and calculated modification indices. After item-level diagnoses, 5 items were excluded: “Pasmo”, “Chocado”, “Duvidoso”, “Encantado” e “Fascinado”. A final 21 items were retained for the long version while a short 9-item version composed by the most representative remained synonymous from the task used in the scale construction phase.
For each version, we explored model modifications to identify the most suitable structures based on the best possible CFA’s fit indices for all models. We implemented modifications based on the suggestions provided by the ‘modindices’ command in the lavaan R package.
Confirmatory factorial analysis
After modifications, a final 8-factor model for the long version was retained for Confirmatory Factorial Analysis in the independent subsample. In the exploratory phase, this model outperformed all other alternatives with a CFI of .96, a TLI of.94 and a RMSEA of 0.055. Model fit values for the CFA in an independent subsample were similar (CFI: .96; RMSEA: .055; and SRMR: .036). For the short version of the scale, the best fit indices at the exploratory phase comprised a 3-factor solution. However, statistical over-extraction was identified and a 2-factor structure was retained for the independent Confirmatory Factorial Analysis. Fit indices for this 2-factor model were also adequate (CFI: .96; RMSEA: .07; and SRMR: .041).
Hence, the short version follows a 2-factor structure (Positive Affect and Negative Affect), while the long version comprises 8 affective dimensions (Sadness, Joy, Fear, Anxiety, Rage, Humiliation, Tiredness and Energy). The results of Confirmatory Factor Analyses for both final versions are described in
Tables 1 and 2.
Table 1. CFA Results: Loadings for the EAPB long version.
| Items | Sadness | Joy | Fear | Tiredness | Anxiety | Rage | Humiliation | Energy |
|---|
| Triste (Sad) | .88 | | | | | | | |
| Sozinho (Lonely) | .72 | | | | | | | |
| Abandonado (Abandoned) | .62 | | .18 | | | | | |
| Infeliz (Unhappy) | .32 | | .25 | | | | | |
| Feliz (Happy) | | .90 | | | | | | |
| Alegre (Joyful) | | .84 | | | | | | |
| Relaxado (Relaxed) | | .47 | | | | | | |
| Aterrorizado (Terrified) | | | .84 | | | | | |
| Amedrontado (Frightened) | | | .85 | | | | | |
| Exausto (Exhausted) | | | | 0.89 | | | | |
| Cansado (Tired) | | | | 0.86 | | | | |
| Inquieto (Restless) | | | | | .77 | | | |
| Ansioso (Anxious) | | | | | .71 | | | |
| Desconfiado (Suspicious) | | | | | .38 | | .43 | |
| Raivoso (Enraged) | | | | | | .88 | | |
| Bravo (Angry) | | | | | | .80 | | |
| Humilhado (Humiliated) | | | | | | | .84 | |
| Desrespeitado (Disrespected) | | | | | | | .83 | |
| Aliviado (Relieved) | | | | | | | | .77 |
| Disposto (Motivated) | | | | | | | | .76 |
| Determinado (Determined) | | | | | | | | .62 |
Table 2. CFA Results: Loadings for the EAPB short version.
| Items | Positive Affect |
Negative Affect |
|---|
| Feliz | .92 | |
| Relaxado | .43 | |
| Disposto | .57 | |
| Bravo | | .64 |
| Triste | | .88 |
| Ansioso | | .47 |
| Aterrorizado | | .59 |
| Humilhado | | .73 |
| Sozinho | | .69 |
In the long version, terms such as “Feliz”, “Triste”,
“Amedrontado”, “Exausto”, “Raivoso”, and “Humilhado” presented high communalities, indicating that a large proportion of their variance was explained by the latent factor. The items “Sozinho”, “Alegre”, “Inquieto”, and “Disposto” showed moderate communalities, suggesting a reasonable level of representation by the factor. In contrast, items “Infeliz”, “Relaxado”, and “Desconfiado” showed low communalities.
In the short version, the items “Feliz” and “Triste” presented the highest communality, indicating that most of their variance was explained by the latent factor. The items “Humilhado” and “Sozinho” showed moderate communalities, suggesting a reasonable level of representation by the factor. And items “Relaxado”, “Ansioso”, “Aterrorizado”, “Disposto”, and “Bravo” had low communities.
Reliability analyses
The global omega coefficients were.86 for the short version 2-factor structure and.96 for the long version 8-factor structure. In the factors level, Positive Affect in the short version had an omega of .67 while Negative Affect had an omega of .83. Omega for all eight factors of the long version were in the range of.76 to.87.
The test-retest reliability for both scales was moderate with a Pearson correlation of .66 for the long version (ICC = .65, N = 2240) and of.61 for the short version (ICC = .61).
Discussion
Two validated versions of the Brazilian Portuguese Affect Scale (EAPB - Escala de Afeto do Português-Brasileiro) are presented here: a 2-factor scale with 9 items comprising Positive and Negative Affect and a 8-factor scale with 21 items comprising the dimensions of Joy, Energy, Tiredness, Sadness, Fear, Anxiety, Rage and Humiliation.
Importantly, these scales were specifically constructed and validated for the Brazilian population. When considering all the items, the 2-factor solution observed in the original PANAS was not reproduced. However, such a 2-factor model fitted our data well when dimensionality reduction based on the EAPB construction (choosing only one of the 2 synonyms words) was implemented. Reducing the number of items limits the representation of the emotional diversity when the intention is to capture the full spectrum of affective experience (Clark & Watson, 1995; DeVellis, 2016). But the short version proposed here can be useful when affective valence, divided in Positive and Negative affect, is the construct of interest and aligns with theoretical assumptions of the circumplex models. The reliability index for the two-factor short form solution is also adequate, indicating strong internal consistency among the items.
The difference in structural findings in EAPB when compared with PANAS may indicate cultural differences in how affect is structured and expressed in Brazilian Portuguese when compared to the original context of the PANAS. Indeed, different structures of affective scales are observed across languages and cultures (Torres et al., 2023). Some features of the original PANAS such as the inclusion of words that are not purely affective (e.g., Alert, Active, Strong and Attentive,) and the lack of direct antonyms may have also influenced the disparate findings in affect structure. In the EAPB, three direct opposites were found (“Triste - Alegre” (Sad - Happy), “Feliz - Infeliz” (Happy - Unhappy) and “Cansado - Disposto” (Tired - Motivated). The presence of these pairs of opposites suggests that certain emotions are represented in a more dichotomous or bidimensional manner, which may facilitate emotional discrimination.
The test-retest results showed low temporal stability, which was expected for an affect scale, as affect tends to fluctuate considerably within participants over time. Communality analysis revealed that, while some items were well represented in both the 2-
and 8-factor structures, others exhibited weak loadings. This suggests that some items were not strongly representative of their respective factors, which is also expected in emotional measurement.
According to Costello and Osborne (2005), solid factors should have at least three items, and preferably five or more, with loadings above 0.50. Damásio (2012) emphasizes that factors with four or more items and high loadings (>0.60) are strongly overdetermined. Factors with only three items can be considered stable when they exhibit high loadings. For the EAPB 2-factor model, Positive Affect is composed of 3 items with high loadings, and Negative Affect is composed of 6 items with only one item below 0.50, indicating that both factors are stable and well-defined. For the 8-factor model, Sadness, Joy, and Anxiety are composed of more than 3 items, indicating a moderate to strong overdetermination, while Fear, Tiredness, Rage, Ressentiment, and Energy have only 2 items per factor (all items with more than 0.60), which indicates that these factors may be less stable.
Though the dimensions of the 8-factor solution resemble classic basic emotions (Sadness, Joy, Fear, and Rage), it is important to emphasize that our results are based on self-report measurements and can not provide evidence of biological constructs. The concept of basic emotions has been extensively debated, with critics arguing that universality often neglects cultural, linguistic, and contextual variability (Fugate, Gendron, Nakashima & Barrett, 2018; Barrett, 2006; Lindquist et al., 2016). By carefully constructing the EAPB, with attention to semantic and cultural nuances, this study offers a more precise and context-sensitive assessment of emotional experiences that can be further applied for test hypotheses based on different theories, including basic emotions, circumplex models or psychological constructionism.
Whether positive affect (PA) and negative affect (NA) should be viewed as opposite poles of a single continuum or as distinct dimensions is still under debate. Although early perspectives favored a bipolar interpretation, psychological research has increasingly challenged this view, presenting evidence that positive and negative feelings can coexist. In fact, individuals may experience happiness and unhappiness simultaneously (Russell & Carroll, 1999). Building on this perspective, if the factors are indeed independent, orthogonal rotations would be expected to provide the most accurate representation of the data (Matos & Rodrigues, 2019). The EAPB exploratory factorial analysis suggests that both correlated and uncorrelated structures produced similar results. However, orthogonal structure showed less cross-loadings. Moreover, psychological constructs are often correlated (DeVellis, 2016; Damásio, 2012; Everitt & Hothorn, 2011), so using oblimim rotation in exploratory factor analysis should be an appropriate practice when assessing affective instruments.
In a recent systematic review (Torres et al. ,2023) we identified only 3 other studies that properly developed affective scales for specific populations besides the original PANAS. In the study by Watson, Clark, and Tellegen (1984), aimed at developing an affect scale in Japanese, the authors applied the same method but arrived at a final scale consisting of 19 factors and 57 items. The authors administered the 57 items to a group of 18 students over 90 consecutive days, and the analyses were conducted using factor analysis with the P-technique, which evaluates data over time and separately for each participant. As a result, the authors found that a two-factor structure emerged in most of the individual analyses (14 participants), while a three-factor structure emerged for three participants, and a single general factor (G) emerged for one participant. In the study by Allik and Realo (1997), the authors used a list of 210 Estonian affect terms, which was divided into two parts and administered to a sample of 184 participants. After item reduction, the authors obtained a set of 92 terms that was subjected to factor analysis, which identified a 7-factors structure. For a 2-factor solution, the authors used the two dimensions that explained the largest portion of the variance. These previous studies used PCA rather than factor analysis, highlighting a significant methodological difference. Especially because PCA reduces variables to components rather than factors, the best approach is to use factor analysis.
As we can see, even studies that developed affect scales show substantial variation in their analyses and results. Consequently, it is expected that culturally diverse samples will present distinct factorial structures, reinforcing the importance of conducting studies specifically designed for each cultural group. For proper transcultural comparisons, further studies developing affect measures are needed, including the target populations simultaneously. Compared to previous PANAS translations in Brazil, besides proper construction rather than translation, we used a fairly representative sample of Brazilian Portuguese speakers.
It is important to highlight some limitations of the current study. We did not collect the PANAS together with the EAPB precluding direct comparisons between our new and the original instrument. Moreover, how this measure applies to specific clinical populations remains to be determined. Finally, the validity evidence for the EAPB should be extended in the future with studies of its convergent, discriminant and predictive validity.
Conclusion
The EAPB is a properly constructed and validated Brazilian affect scale comprising a short (9 items) and a long (21 items) version. The short version exhibits a 2-factor structure (Positive Affect and Negative Affect) while the long version shows an 8-factor structure (Sadness, Joy, Fear, Tiredness, Anxiety, Rage, Humiliation, and Energy). We have found an excellent level of internal consistency and structural validity for both versions of the EAPB.. Establishing instruments that are culturally and linguistically grounded is essential for advancing the understanding of emotional experience across different populations.
Ethics and consent
This study received approval from the Research Ethics Committee of ABC Federal University on April 23, 2019 (CAAE: 1247318.9.0000.5594; Opinion number: 3.752.357). Informed consent was obtained from all participants prior to the beginning of the study. Written consent was obtained on the data collection app after the presentation of participation term with a copy also sent to participants by email.
