Internal Consistency of Terminology in Referencing Pilot Studies

In this section, we explore how pilot studies are reported in academic articles. To do this, we manually extracted all statements mentioning the keyword "pilot" from each article and analyzed them inductively based on grounded theory. We found that only a few mentions of pilot studies were made in each paragraph or short sentence, so we focused on the pilot studies conducted with crowdsourced participants.
We discovered that the term "pilot study" is the most commonly used term to refer to crowd pilot studies in the literature (see Figure 7). To ensure consistency in our analysis, we recommend using this term throughout the articles we reviewed.
To analyze how pilot studies are reported, we coded sections of articles where they were discussed. Initially, our coding scheme reflected the standard structure of academic articles, but we modified it slightly to accommodate the diverse methodological approaches used in the articles. The result of our coding is depicted in Figure 9, which shows the sections where authors report on their pilot studies.
In summary, this section demonstrates how we systematically analyzed and coded the reports of pilot studies in academic articles to gain a deeper understanding of how these studies are typically reported. By using a consistent term and coding scheme, we were able to identify patterns and trends in how authors discuss pilot studies in their articles.