Introduction
This report summarises key findings and insights from the pilot testing process of the initially prepared survey design for the GlobalSCAPE science communicator diary survey. Pilot testing has been employed in recognition of the fact that a key risk in any survey research is measurement error rooted in poor question wording and questionnaire design (e.g., see Jensen, 2020). We have conducted the testing to ensure the GlobalSCAPE survey is not subject to issues relating to these factors and exceeds the normal standards for market research.
Pilot testing is a key element of robust survey question and instrument development, covering a set of techniques to help reveal the cognitive processes of respondents who are similar to those who will undertake the real survey (Jensen & Laurie, 2016). Pilot testing is designed to expose survey questions and answer potential issues with clarity, accessibility, relevance, reliability, validity or measurement structure (e.g., Smith & Jensen, 2016). Such issues are addressed based on pilot testing findings, including identifying which items should be retained, removed, or revised.
Methods
The empirical research underpinning this paper was collected following formal ethical approval for the GlobalSCAPE project’s planned studies by the Institutional Review Board of Trinity College Dublin (the coordinating institution for the project) on the 16th February 2021. This section describes the procedures used to gather pilot testing data, and the approach to analysing responses. This pilot study used digital methods and structured feedback measures with a sample of respondents in Europe, America, New Zealand, and Japan. These countries were selected due to project partners’ connection to wide pools of potential participants within the target population of professional science communicators in these areas. This approach has the advantage of closely replicating aspects of the respondent experience with the (future) live GlobalSCAPE questionnaire. Respondents took 29 minutes on average (5% trimmed mean) to complete the pilot testing process.
A key factor in prioritising questions for testing was whether they were newly developed for this project or had been previously validated for a different survey. Any newly developed questions for this survey were automatically included for cognitive testing. These survey questions yielded primarily quantitative data alongside two sets of qualitative responses collected via open-ended thought-listing questions. In addition, optional open-ended text boxes were provided after two quantitative measures for participants to expand on and clarify their answers.
We used a purposive sampling strategy, as is typical for pilot testing research. This strategy included distributing the pilot survey enrolment link across project partners’ networks of science communicators, and identifying sampling criteria to ensure geographically diverse recruitment of professional science communicators from different parts of the world. Respondents recruited for the pilot were not incentivised but were briefed on the objectives of this testing. This recruitment resulted in a final sample of N = 23 respondents (completed responses).
To replicate the experience of completing the GlobalSCAPE survey as closely as possible, survey questions were presented on a digital survey platform, which was the format intended for the final survey. After completion of each question block, three semantic differential scales were presented for the respondent to rate the relevance, clarity, and ease of the questions presented. Responses were collected with a seven-point semantic differential scale, ranging from Relevant-Irrelevant, Clear-Confusing, and Easy-Difficult, respectively1 (see Figure 1).
Figure 1. Semantic differentials for rating survey blocks.
Respondents’ perception of the relevance of a certain question is an important measure, as low perceptions of relevance may negatively affect motivation to complete that question, induce a negative perception of the survey as a whole, as well as potentially contributing to survey fatigue due to confusion about the purpose of some of its aspects. Respondents’ views about the clarity of the question are crucial to assess the accessibility of questions for all social groups and understand if there are any changes which need to be made to instructions, terminology or phrasing of questions / response options. Furthermore, it is important to understand if any questions are seen as difficult to answer, as this would indicate that the question leads to a high demand on respondents’ cognitive resources. Such perceptions of difficulty could affect completion rates by increasing the likelihood of survey fatigue.
If respondents gave a negative score for any of the semantic scales, an additional box per scale automatically appeared to gather further explanatory information as to why they found the questions irrelevant, confusing, or difficult (Figure 2). This qualitative information was used to guide assessment of any necessary changes to questions explicitly identified as irrelevant, confusing or difficult for at least 10% of respondents.
Figure 2. Qualitative feedback box for negative responses.
Results
This section presents the results from the assessment of how clear/confusing, easy/difficult and relevant/irrelevant each key item in the survey design was perceived to be. Items in need of modification are also indicated.
Self-perception of science communication identity
This set of questions focused on how respondents saw themselves in terms of their professional identity as a science communicator, as well their professional experiences in science or science communication (Figure 3).
Figure 3. Self-perception of science communication identity question block.
Figure 4 below shows the distribution of respondents in terms of how easy, clear, and relevant they perceived questions about their science communication identity.
Figure 4. Responses to question, ‘The set of survey questions I completed were…,’ for items on respondents’ science communication identity.
The majority of respondents to this question had no negative perceptions about the set of questions, with the modal category for each semantic scale being the most strongly positive option, amounting to 100% (n = 23) for easy, 79% (n = 19) for relevant, and 100% (n = 24) for clear. However, Figure 4 also shows that 21% (n = 5) of the respondents found the items to be confusing rather than clear - exceeding the 10% threshold. Therefore, further discussion as to why this may be the case and potential solutions can be found in the section below discussing Changes to the survey design.
Professional science communication thoughts (open-ended)
These questions asked respondents to list their thoughts about the term ‘professional science communication’ and about being a ‘professional science communicator’ in free-text, open-ended response boxes (Figure 5).
Figure 5. Open-ended ‘Professional Science Communication’ and ‘Professional Science Communicator’ thought-listing questions block.
Figure 6 below shows mixed results in terms of how respondents rated these open-ended items in terms of ease, clarity and relevance.
Figure 6. Responses to question, ‘The set of survey questions I completed were…,’ for open-ended items on thoughts about professional science communication.
While the majority of respondents found these items relevant (91%, n = 20), clear (76%, n = 16) and easy (75%, n = 15), there were notable proportions who did not. Specifically, 25% (n = 5) found it difficult to respond to the questions, and 24% (n = 5) found the questions confusing. Therefore, analysis of the explanations for these negative responses and potential modifications to the item can be found in the section below discussing Changes to the survey design.
Personal science communication situation
Figure 7 below shows the set of statements about respondents’ personal science communication working lives, and the corresponding Likert-type agreement scale.
Figure 7. Personal science communication situation statements survey block.
Figure 8 below shows strongly positive views about this survey block.
Figure 8. Responses to question, ‘The set of survey questions I completed were…,’ for items on respondents’ personal science communication situation.
Almost all respondents found this set of statements easy (95%, n = 21), clear (96%, n = 22), and relevant (96%, n = 22), and most also expressed this with the most strongly positive response option (73%, n = 16; 65%, n = 15; and 65%, n = 15; respectively). Only 5% (n = 1) of respondents found the items difficult, and 4% (n = 1) found them confusing, and irrelevant, therefore not meeting the threshold for review. Therefore, no changes were required for this item.
Household income
This item asked respondents about their difficulty in meeting basic needs with their monthly household income (Figure 9).
Figure 9. Household income and meeting basic needs question block.
The chart below (Figure 10) shows that some respondents had issues with this question, particularly around clarity and relevance.
Figure 10. Responses to question, ‘The set of survey questions I completed were…,’ for the item on household income.
Most respondents found this question easy to answer (95%, n = 21), and only 5% (n = 1) found it difficult. While majorities also found it clear (87%, n = 20) and relevant (75%, n = 15), considerable proportions found the question confusing (13%, n = 3) and irrelevant (25%, n = 5). Therefore, this item is reviewed for modification in the section below discussing Changes to the survey design.
Science communication training
To contextualise the upcoming set of questions about science communication qualifications and training, respondents were asked whether they had received any training inside or outside of a university setting (Figure 11).
Figure 11. Type of science communication training question block.
Science communication qualifications
If respondents indicated that they had attended science communication training within a university degree or diploma programme, they were asked about what formal qualifications they had completed (Figure 12).
Figure 12. Science communication qualification question block.
Figure 13. Responses to pilot feedback question, ‘The set of survey questions I completed were…,’ for the items on science communication qualifications.
Figure 13 above shows that all respondents found this question block relevant (100%, n = 11), and a satisfactory majority found it clear (92%, n = 11). However, 17% (n = 2) indicated that they found the questions difficult to answer, meaning a review for modification is necessary. This can be found in the section below discussing Changes to the survey design.
Non-university-based science communication training
If respondents indicated that they had attended science communication training external to a university degree or diploma programme, they were asked about what type of training this was (Figure 14).
Figure 14. Type of science communication training outside of a university degree or diploma programme.
Figure 15 below shows very positive views about this question block.
Figure 15. Responses to question, ‘The set of survey questions I completed were…,’ for the item on science communication training.
Respondents overwhelmingly found this question block easy (100%, n = 20), clear (95%, n = 19) and relevant (95%, n = 19), meaning this question block needs no amendments.
Science communication semantic differentials
The next item involved asking respondents to rate how they felt about ‘doing science communication’ by selecting a point between pairs of opposing adjectives, known as a semantic differential (Figure 16).
Figure 16. ‘Doing science communication’ semantic differentials block.
Figure 17 below shows that respondents were not entirely satisfied by the clarity or ease of answering this question block.
Figure 17. Responses to question, ‘The set of survey questions I completed were…,’ for the items on respondents’ science communication experience.
The results show that the vast majority of respondents found this question block relevant (96%, n = 22), easy to answer (91%, n = 20) and clear (87%, n = 20). However, there were enough negative responses indicating respondents found this question block confusing (13%, n = 3) to mean a review of this question and negative feedback is necessary. This can be found below in the section below discussing Changes to the survey design.
Views about doing science communication
Respondents were also asked to rate their level of agreement with different statements about the practice of doing science communication (Figure 18).
Figure 18. Views about doing science communication survey block.
Figure 19 above shows that 100% (n = 23) of respondents found this question block relevant, 95% (n = 21) found it easy to answer and 91% (n = 20) found it clear. This means that less than 10% of respondents found the question block difficult, confusing or irrelevant, meaning a review for amendments is not necessary.
Figure 19. Responses to question, ‘The set of survey questions I completed were…,’ for the items on respondents’ views on doing science communication.
Science communication activity
The question shown below (Figure 20) marked the beginning of a set of questions exploring respondents' experiences completing science communication activities in the last 7 days.
Figure 20. Completion of science communication activities in the last 7 days question item.
Science communication activity semantic differentials
If respondents answered ‘yes’ to having completed any science communication activities in the past 7 days, they were asked to give their perceptions about how it went across a range of semantic differentials (Figure 21).
Figure 21. Science communication activity semantic differentials survey block.
Figure 22 above shows respondents unanimously agreed that the semantic differentials were easy (100%, n = 12) and clear (100%, n = 12). The majority also agreed that they were relevant (92%, n = 11), meaning no changes are required.
Figure 22. Responses to question, ‘The set of survey questions I completed were…,’ for the items on respondents’ assessment of their science communication activities.
Science communication activity barriers
The questions below (Figure 23) focused on whether respondents had faced any barriers to achieving the objectives of their science communication activity completed in the last 7 days.
Figure 23. Science communication activity barriers question block.
All respondents found this set of questions relevant. The majority also found them easy to answer (82%, n = 9) and clear (83%, n = 10). However, more than 10% found the question block confusing (17%, n = 2) and difficult (18%, n = 2) (Figure 24). Review was needed to understand the reasons and potential modifications to the question block, and can be found in the section below discussing Changes to the survey design.
Figure 24. Responses to question, ‘The set of survey questions I completed were…,’ for the items on barriers to respondents’ science communication activities.
Assessment of science communication activity delivery
The question block below (Figure 25) shows four statements about the delivery of the recent science communication activity respondents completed, along with a Likert-type agreement scale. They were asked to rate their level of agreement with each statement.
Figure 25. Assessment of science communication activity delivery question block.
The chart above (Figure 26) shows most of the respondents rated these statements as relevant, clear and easy (92%, n = 11, for each). Therefore, no amendments to this block are necessary.
Figure 26. Responses to question, ‘The set of survey questions I completed were…,’ for the items on respondents’ assessment of science communication activity delivery.
Changes to the survey design
Self-perception of science communication identity
Recap of questions assessed:
In what way(s) do you see yourself?
‘Science writer or journalist’, ‘Citizen scientist’, ‘Science museum or centre professional’, ‘Science communication researcher’, ‘Scientist or researcher’, ‘Science teacher or educator’, ‘Science performer’, ‘Volunteer in science communication’, ‘Other (please specify)’, ‘None of the above’
Are you involved in communicating about science or research with people who are not scientists or researchers?
Do you consider yourself a science communication professional?
Do you consider yourself a science communication professional?
As indicated earlier in this report, 21% (n = 5) of respondents found the questions about their self-perception of their professional identity as a science communicator confusing. Table 1 below shows the feedback from those who gave these negative responses.
Table 1. Responses to 'Please explain why you found this set of questions Confusing:' for Self-perception of science communication identity.
| ‘Confusing’ | Vague definitions, such as what is a 'science museum professional?' Is it a permanent position in a
science museum or getting paid to curate one exhibit? |
| What is encompassed in the definition of science here? |
It is a bit confusing for me as I see myself more as a science engagement professional than a science
communication pro. My role is more about making links in the background, creating resources and opportunities
for science communicators. |
| There are no definitions of what you consider science communication. |
I don't understand the question "Are you involved in communicating science or research". Seems a bit redundant
to the "In what ways do you see yourself" |
One response was rooted in confusion about being asked about science communication due to not identifying as a science communicator, and was therefore more of an explanation behind their response. The final respondent perceived one part of the question block as redundant. As this was only the perception of one person, no amendments based on this factor is necessary.
However, all other feedback revolved around the lack of definitions given for several terms included in the question block. This included the list of different kinds of science professionals, and the general terms ‘science’ and ‘science communication’. With regards to the lack of definition of the term ‘science’ and ‘science communication’, this is intentional, as part of the survey’s purpose is to uncover differing perceptions about what it means to be a science communicator. However, some clarity about this could be beneficial to assure respondents that their response should be based on their own definitions of ‘science communication’. Therefore, the following disclaimer text was added alongside the survey item, in order to explain why no definition of science communication is given: ‘We understand that people will define the categories below in different ways. Please use your perception of what these terms mean when answering this question.’
Professional science communication thoughts (open-ended)
Recap of questions assessed:
In general, what comes to mind when you think of 'Professional Science Communication'?
What comes to mind when you think of being a 'Professional Science Communicator'?
These questions received negative responses both in terms of difficulty of answering the questions, and in terms of the questions being confusing. Table 2 below shows the feedback from those who gave these negative responses.
Table 2. Responses to 'Please explain why you found this set of questions Difficult, Confusing, or Irrelevant:' for Professional science communication thoughts (open-ended).
| ‘Difficult’ | Defining things are difficult for me as there's so much nuance and complexity, and too many things
come to my mind to be shared in a short open ended question. |
| I don't understand the second question - do you mean being a prof science communicator myself or in general? |
I gave quite similar answers because 'Professional Science Communication' to me is extremely closely linked to
being a professional in science communication. |
I was not sure how much to write. They say 'what comes to mind', but I didn't know if it should be words or
phrases like in a word cloud, or if it should be sentences. I didn't know if it meant the idealised version of
"professional science communication" or the reality of it, as there may be a disconnect between what we think
it should be and what it actually is in one's own practice. I found the second question easier, the word "being"
guided me more towards descriptive words or phrases to describe personal qualities or values. But again, not
sure if it was looking for the idealised "professional science communicator" or the reality - I could just have easily
have written things like "stressed, looking for funding, misunderstood" but I had to make a call so I chose to write
more aspirational values/traits! More parameters might have made it easier to answer. |
How general should the "in general" be? I can give top of my head answer here easily but I don't know if that’s
what you need. |
| ‘Confusing’ | It wasn't clear to me what type of response it was expected. |
Distinction not so easy; I started filling out the first and then rephrased, copy-pasted when turning to the second
question. Wondering what the surveyers want with the distinction... is the second one about how being is FOR
ME or also in GENERAL, but on a personal level? |
I do understand the merit of open questions and phrasings like 'what comes to mind' but this also makes the
question difficult to interpret. What comes to mind can be anything. I could have answered: 'I get a nice feeling',
or 'I think of talking about mathematics', or 'I think of the book 'Darwins hofvijver', or 'The term itself is nice' or 'I
am proud to be a science communicator' or 'I like the variety of work'... |
| When I think of being a professional I think mostly about invoices ;) This question is very vague. |
I wasn't quite sure what the distinction is the second question was looking for with 'being', is it after a personal
reflection, or is it asking for my thoughts on the concept of 'being' a science communication professional in
general |
The main themes running through these responses were to do with lack of specificity in the question wording, and perceived lack of distinction between thinking about ‘professional science communication’ and thinking about being a ‘professional science communicator’. In the case of the last response for confusing, these issues were linked.
In terms of there being a lack of distinction between these two objects of thought, this is a valid research finding, rather than an issue with the survey question itself. Therefore, no modifications based on this factor are necessary.
Some responses (n=6) expressed general uncertainty about what ‘type’ of response was expected. While this may at first seem to indicate an issue with this item, this was expected because of the broad nature of this open-ended question. The nature of thought-listing measures is an intentional lack of prescription given in terms of the format of the response given, to elicit genuine immediate cognitive associations. Our view is that the insights to be gained by asking this question will be worth the mental effort that some respondents find in answering.
However, several other respondents (n=4) expressed confusion explicitly about whether the question ‘What comes to mind when you think of being a 'Professional Science Communicator'?’ meant their own experience and the ‘reality’ of being a science communicator, or the abstract, idealised concept of being a science communication professional in general. The type of responses given based on each interpretation could be vastly different, and indicates a lack of consistency in interpretations of the question. Therefore, clarification in the question wording on this could increase the clarity of the question and ease of answering it, and improve the validity of results.
To achieve this, the following disclaimer text (Figure 27) was added again alongside the survey item, in order to explain why no definition of science communication is given: ‘We understand that people will define the categories below in different ways. The following questions should be answered based on your perception of what these terms mean when answering this question’. In addition, the wording for the second question was amended as follows: ‘What comes to mind when you think of the day-to-day experience of being a 'Science Communicator'?’ (Figure 28).
Figure 27. Disclaimer text added.
Figure 28. Disclaimer text added and amended question wording.
Household income
Recap of question assessed:
The negative open-ended feedback for the measure asking about household income (Table 3) was driven primarily by the perception that it is not relevant to the rest of the survey, as it does not ask specifically about income relating to science communication work. Furthermore, some responses highlighted that the question is not clear regarding whether other streams of income should be included, such as student loans or partners’ incomes. Therefore, two changes were made (Figure 29). Firstly, the question text was narrowed to focus on the respondent’s work income by making the following amendment: ‘Thinking about your work income, meeting basic needs is:’
Table 3. Responses to 'Please explain why you found this set of questions Confusing or Irrelevant:' for Household income.
| ‘Confusing’ | Without further context, household income is a strange concept - depends on (if applicable) partner
income, family status, etc. |
No question about my employment status. As a postgrad student, my answer would be different from someone
with a fixed salary |
Do you mean about my personal income, or combined with a partner, do you mean on the basis of the income
without taking into account other factors like previous jobs and capital etc? |
| ‘Irrelevant’ | My household income is too different from my own current salary and working conditions. If you're trying to
"compare the working conditions of science communicators around the world", this question doesn't
seem to help with this task, as my partner income distorts the answer. |
| Curious about the correlation between science communication income and meeting basic needs |
| I don't see the link with communicating science. |
what is this aiming at? is the profession properly paid? depends on the level, the institution and many more
factors - without asking more context, this sounds irrelevant/hard to interpret |
| I am not sure how this is related to science communication. |
Figure 29. Revised question and response option text.
In addition, a new question was added to address comments regarding the appropriateness of household income questions vs people’s salaries, and to provide an indicator of ‘professional’ roles: “I am always paid for my science communication work.”.
Science communication qualification
Recap of question assessed:
The feedback given about items exploring respondents’ science communication qualifications (Table 4) revealed that the original question wordings and response options did not provide an easy respondent experience. This was because the original set of response options did not sufficiently cover nor effectively categorise the divergent ways in which people engage in science communication training. Therefore, several changes were made (Figure 30). Firstly, a piece of introductory text was added before the questions to shift focus to content of the programme(s) that respondents participated in, rather than the official format. Furthermore, response options were modified for the question asking about ‘Training outside of a university degree’, including a newly added option for ‘Self-led skill development undertaken independently’. In addition, to increase clarity, response options for different types of science communication university degrees were removed, leaving the item ‘Have you been awarded a university degree in a topic relating to science communication?’ as a simple, binary ‘Yes’ or ‘No’ question.
Table 4. Responses to 'Please explain why you found this set of questions Difficult or Confusing:' for Science communication education.
| ‘Difficult’ | I acquired scicomm skills in a one-week summer school and a single course on research in science
communication which were at university but were more like smaller trainings than full Bachelor
or Master programs. Therefore I wonder whether in the previous question I should have actually
answered yes to whether I completed any scicomm trainings at university. |
I answered difficult to be able to say something about the previous page. The question about training at/outside
University was confusing to me. After all, when you write for your personal pleasure, you also learn a lot, but it is
not regarded as real training. Moreover, I chose a science communication master because I had already interest
in this creative direction and had already personal experience with designing/writing etc, without official training.
The problem here is that it is assumed that you learn science communication by doing a master or something
like that. In my case, of course I learned a lot, but with some things it was more like the master polished talents
and experiences that I already had. I think this often is the case with science communication, where creativity is
very important and of course you can develop such skills just as much outside of official training. Should I then
have answered 'outside university'? but it is not something that you 'attended'...Note: the question about when
you finished your master (year) does not work - I could not select 2011. |
| ‘Confusing’ | Could ask for more information on the qualification programme. My undergrad minor was in Museum Studies,
which briefly encompassed science museums. Would this count as a science communication programme? |
Figure 30. Revised question and response option text.
Science communication semantic differentials
Recap of question assessed:
In order to address the feedback in Table 5 which highlighted a lack of symmetry in the semantic differentials provided, the word pairs were adjusted towards unidimensional comparisons (Figure 31). The revised word pairs are: ‘Enjoyable – Unenjoyable’, ‘Relaxing – Stressful’, and ‘Energizing – Exhausting’.
Table 5. Responses to 'Please explain why you found this set of questions Difficult or Confusing:' for Science communication semantic differentials.
| ‘Difficult’ | These are not true dichotomies! I consider scicomm as both enjoyable AND stressful, as frustrating
AND motivating, etc. I would rather answer those terms separately - then they become relevant in
comparison to each other. Otherwise, I would resort to 0 - with no relevance for the survey. |
| I had to look up the word Invigorating |
| ‘Confusing’ | "Doing science communication" is a quite general category so I am not sure what I was actually rating... I think I
mostly had in mind actually working with audiences. |
| see above |
I think overall these questions are fine although I wonder if people might be expecting the words on each end
of the scale to be antonyms. It's probably fine but Inspiring-to-Uninspiring might seem more of natural fit than
"disappointing." Similarly some of the others don't seem to match, like Enjoyable/Stressful... |
Figure 31. Revised question and response option text.
Science communication activity barriers
Recap of question assessed:
Have you experienced any barriers to achieving your objectives for this science communication activity?
What science communication barriers have you experienced?
‘Instability in broad context (e.g. crime, flooding, political control, etc)’, ‘Insufficient financial resources’, ‘Inadequate transportation’, ‘Unreliable equipment or technology’, ‘Lack of know-how’, ‘Lack of education or training’, ‘Lack of feedback from audiences’, ‘Insufficient infrastructure (country, institution etc)', ‘Insufficient human resources support (volunteers, staff, etc)’, ‘Inadequate facilities’, ‘Lack of support from the institution or management’, 'Lack of a system or strategies’, 'Lack of interest from audiences’, ‘Other (please specify)’.
Have you successfully resolved all of these barriers?
The feedback given for this section in Table 6 shows respondents found these items confusing because of a lack of clarity about what science communication definitions or activities should underpin their answers. This problem was eradicated through an earlier amendment to the survey detailing why no definition of science communication was given and how to answer questions as a result. Therefore, no amendments were necessary on this basis. To address the feedback about answering questions on the basis of science communication research, clarification text was added if people indicate they are researchers. Finally, a colour contrast check was conducted in response to the statement regarding colour choices on the user interface. Colour selections were then updated on the survey platform to ensure accessibility.
Table 6. Responses to 'Please explain why you found this set of questions Difficult or Confusing:' for Science communication activity barriers.
| ‘Difficult’ | Needed more thinking time to answer |
The white text on bright orange background is very difficult to read at length — like when there is a long list of
checkbox options |
| ‘Confusing’ | Is science communication always to external audiences outside academia? I suppose I needed a definition of
science communication before I started this |
With specific scicomm activity I had a research activity in mind, but the questions seemed aimed at practising
science communication instead of researching it. Therefore, I'm not sure whether I correctly filled in the
questions. |
Discussion
This paper has reported findings from a pilot study investigating the quality of survey experience and clarity of survey questions and response options for the GlobalSCAPE science communicator diary study. This study is seeking to break new ground in its global reach and in the extended timeframe for data collection (up to 12 months per individual participant). Such an ambitious research approach requires due consideration of methodological challenges, including the difficulty of arriving at a globally relevant and sufficiently clear survey design. Despite having the limitation of a relatively small sample size, we are confident the sampling criteria employed has delivered the level of geographic representation required to ensure global suitability, in addition to a depth of qualitative insight more difficult to attain with larger sample sizes.
Here, we have presented findings suggesting that the initially prepared survey design was largely satisfactory already. Results pointed to specific changes that would further improve the survey design, reducing ambiguities and improving the validity and reliability of the data that will be collected. Such changes will be implemented in the final version of the survey design, thereby achieving the immediate goals of the pilot research. However, it is our intention that others who are undertaking research on professionals operating in fields similar to science communication can benefit from these findings, identifying lessons that will aid in their own survey designs. Likewise, this paper is intended to contribute to the methodological literature on using diary survey methods to investigate professional practices as they unfold over time.
Ethical approval and consent
The empirical research underpinning this paper was collected following formal ethical approval for the GlobalSCAPE project’s planned studies by the Institutional Review Board of Trinity College Dublin (the coordinating institution for the project). This formal approval was given via a document sent to the GlobalSCAPE project coordinator on the 16th February 2021.
Informed consent was attained from respondents via the text and response option in the box below.
Thank you for participating in the GlobalSCAPE research study!
This study invites Science Communicators across the globe to capture a wide range of views and help clarify the Science Communication landscape of tomorrow. This research focuses on how Science Communicators handle personal and professional challenges while engaging in communication activities.
Who should participate in this study?
People who are actively involved in communicating scientific or research findings with non-academic, public audiences. This may involve professional commitments, key decisions or shaping communication content.
How are you being asked to help?
Currently, we are looking for your help to answer some of the questions for the full study. We appreciate your interest to help us with it. This questionnaire should take you about 10-15 minutes to complete. There are just a few things we would like to point out before you continue:
• You must be at least 18 years of age or older.
• Any personal information provided about yourself is kept confidential.
• The data you submit will be fully anonymized and used for research, evaluation, and future publication purposes of the GlobalSCAPE research project.
|
Please indicate whether you understand and agree with the statements above, and are willing to participate in this survey:
Yes, I understand, agree, and am willing to participate in this survey.
Qualia Analytics (qualiaanalytics.org) is a data processor that is fully compliant with the EU General Data Protection Regulation (GDPR). You may view the Qualia Analytics privacy policy here.
The GlobalSCAPE project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101006436. |
Acknowledgements
The authors would like to thank the representatives of other partners in the GlobalSCAPE consortium for their feedback and support with this research, and in particular, for circulating the data collection survey request to gather respondents. The GlobalSCAPE project partners are: Trinity College Dublin, Leiden University, Qualia Analytics, SciDevNet, European Network of Science Engagement Organisations (Ecsite) and Springer Nature.
Faculty Opinions recommendedReferences
- Jensen E, Laurie C:
Doing real research: A practical guide to social research. SAGE: London. 2016. Reference Source
- Jensen EA:
Why impact evaluation matters in science communication: Or, advancing the science of science communication. In P. Weingart, et al., Science Communication in South Africa: Reflections on Current Issues (African Minds). 2020.
- Jensen EA, Jensen AM, Noles S, et al.:
GlobalSCAPE Diary Survey Pilot [Data set]. Zenodo.
2021. http://www.doi.org/10.5281/zenodo.5706117
- Smith BK, Jensen EA:
Critical review of the United Kingdom's "gold standard" survey of public attitudes to science.
Public Underst Sci.
2016; 25(2): 154–170. PubMed Abstract
| Publisher Full Text
