Misplaced Pages

Seriousness check

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.
This article is an orphan, as no other articles link to it. Please introduce links to this page from related articles; try the Find link tool for suggestions. (February 2022)

The seriousness check is a technique that can be used in online research (also called Internet-based research, Web-based research, Web-based experiments) to improve data quality. Nowadays, many scientific studies with human participants are conducted online and are accessible to a large diversity of participants. Nonetheless, many people just want to look at the different pages of the questionnaire, instead of giving carefully chosen answers to the questions. The seriousness check addresses this problem: In this approach the respondents are asked about the seriousness of their participation or for a probability estimate that they will complete the entire study or experiment. Thus, by using the seriousness check irrelevant data entries can be easily identified and be excluded from the data analysis.

Method

Illustration of the Seriousness Check

Seriousness checks can be implemented both before and after participation in the study. However, it has been shown that the seriousness check is a good predictor of dropout rates when implemented in the first page of the experiment.

There are Web-based tools, e.g. WEXTOR, that implement the seriousness check by default.

Impact

Several studies have shown that performing a seriousness check at the start of a study best predicts motivation and dropout probability. It was observed that of those answering “I would like to look at the pages only" around 75% will drop, while of those answering “I would like to seriously participate now" only ca. 10-15% will drop during the study. Overall, about 30-50% of visitors will fail the seriousness check, i.e. answer “I would like to look at the pages only". Moreover, it was found that emphasizing seriousness increased information seeking in participants and the time they spent on the study. Following up on this, it was shown that motivation and self-reported seriousness significantly predict several data quality indicators

References

  1. ^ Reips, U.-D. (2009). Internet experiments: Methods, guidelines, metadata. Human Vision and Electronic Imaging XIV, Proc. SPIE, 7240, 724008. https://doi.org/10.1117/12.823416
  2. Musch, J., & Klauer, K. C. (2002). Psychological experimenting on the World Wide Web: Investigating content effects in syllogistic reasoning. In B. Batinic, U.-D. Reips, & M. Bosnjak (Eds.), Online social sciences (pp. 181–212). Hogrefe & Huber Publishers.
  3. Reips, U.-D. (2007). The methodology of Internet-based experiments. In A. Joinson, K. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford Handbook of Internet Psychology (pp. 373-390). Oxford University Press.
  4. ^ Reips, U.-D. (2002). Standards for internet-based experimenting. Experimental Psychology, 49(4), 243–256. https://doi.org/10.1026/1618-3169.49.4.243
  5. Reips, U.-D., & Krantz, J. H. (2010). Conducting true experiments on the Web. In S. D. Gosling & J. A. Johnson (Eds.), Advanced methods for conducting online behavioral research (p. 193–216). American Psychological Association. https://doi.org/10.1037/12076-01
  6. Aust, F., Diedenhofen, B., Ullrich, S., & Musch, J. (2013). Seriousness checks are useful to improve data validity in online research. Behavior Research Methods, 45(2), 527–535. https://doi.org/10.3758/s13428-012-0265-2
  7. Reips, U.-D.; Neuhaus, C. (2002). "WEXTOR: A web-based tool for generating and visualizing experimental designs and procedures" (PDF). Behavior Research Methods, Instruments, and Computers. 34 (2): 234–240. doi:10.3758/BF03195449. PMID 12109018. S2CID 39032912.
  8. Reips, U.-D. (2008). How Internet-mediated research changes science. In A. Barak (Ed.), Psychological aspects of cyberspace: Theory, research, applications (p. 268–294). Cambridge University Press. https://doi.org/10.1017/CBO9780511813740.013
  9. Bayram, A. Burcu (2018). "Serious subjects: A test of the seriousness technique to increase participant motivation in political science experiments". Research & Politics. 5 (2): 205316801876745. doi:10.1177/2053168018767453. ISSN 2053-1680. S2CID 149584146.
  10. Verbree, Anne-Roos; Toepoel, Vera; Perada, Dominique (2020). "The Effect of Seriousness and Device Use on Data Quality" (PDF). Social Science Computer Review. 38 (6): 720–738. doi:10.1177/0894439319841027. ISSN 0894-4393. S2CID 133438145.
Categories: