Cited By
View all- Angarita MKolnes MNolte A(2024)Startup Creation Beyond Hackathons – A Survey on Startup Development and SupportSoftware Business10.1007/978-3-031-53227-6_15(205-221)Online publication date: 9-Feb-2024
The CAT Effect: Exploring the Impact of Casual Affective Triggers on Online Surveys’ Response Rates
Article No.: 583, Pages 1 - 13
Abstract
We explore the impact of Casual Affective Triggers (CAT) on response rates of online surveys. As CAT, we refer to objects that can be included in survey participation invitations and trigger participants’ affect. The hypothesis is that participants who receive CAT-enriched invitations are more likely to respond to a survey. We conducted a study where the control condition received invitations without affective triggers, and the experimental condition received CAT-enriched invitations. We differentiated the triggers within the experimental condition: one-third of the population received a personalized invitation, one-third received a picture of the surveyor’s cat, and one-third received both. We followed up with a survey to validate our findings. Our results suggest that CATs have a positive impact on response rates. We did not find CATs to induce response bias.
Supplementary Material
Talk Video
- Download
- 28.23 MB
References
[1]
Yehuda Baruch and Brooks C Holtom. 2008. Survey response rate levels and trends in organizational research. Human relations 61, 8 (2008), 1139–1160.
[2]
Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological) 57, 1(1995), 289–300.
[3]
Adam J Berinsky, Michele F Margolis, and Michael W Sances. 2014. Separating the shirkers from the workers? Making sure respondents pay attention on self-administered surveys. American Journal of Political Science 58, 3 (2014), 739–753.
[4]
Anol Bhattacherjee. 2001. Understanding information systems continuance: an expectation-confirmation model. MIS quarterly (2001), 351–370.
[5]
Paul P Biemer and Lars E Lyberg. 2003. Introduction to survey quality. Vol. 335. John Wiley & Sons.
[6]
Irene-Angelica Chounta, Emanuele Bardone, Aet Raudsep, and Margus Pedaste. 2021. Exploring teachers’ perceptions of Artificial Intelligence as a tool to support their practice in Estonian K-12 education. International Journal of Artificial Intelligence in Education (2021), 1–31.
[7]
E Gil Clary, Mark Snyder, and Robert Ridge. 1992. Volunteers’ motivations: A functional strategy for the recruitment, placement, and retention of volunteers. Nonprofit Management and leadership 2, 4 (1992), 333–350.
[8]
Mick P Couper, Frederick G Conrad, and Roger Tourangeau. 2007. Visual context effects in web surveys. Public Opinion Quarterly 71, 4 (2007), 623–634.
[9]
Scott D Crawford, Mick P Couper, and Mark J Lamias. 2001. Web surveys: Perceptions of burden. Social science computer review 19, 2 (2001), 146–162.
[10]
Tim Curran and Jeanne Doyle. 2011. Picture superiority doubly dissociates the ERP correlates of recollection and familiarity. Journal of Cognitive Neuroscience 23, 5 (2011), 1247–1262.
[11]
Richard Curtin, Stanley Presser, and Eleanor Singer. 2005. Changes in telephone survey nonresponse over the past quarter century. Public opinion quarterly 69, 1 (2005), 87–98.
[12]
W De Heer and E De Leeuw. 2002. Trends in household survey nonresponse: A longitudinal and international comparison. Survey nonresponse 41(2002), 41–54.
[13]
Kristine De Valck, Fred Langerak, Peter C Verhoef, and Peeter WJ Verlegh. 2007. Satisfaction with virtual communities of interest: Effect on members’ visit frequency. British Journal of Management 18, 3 (2007), 241–256.
[14]
W Edwards Deming. 1944. On errors in surveys. American Sociological Review 9, 4 (1944), 359–369.
[15]
Nicole B Ellison, Charles Steinfield, and Cliff Lampe. 2007. The benefits of Facebook “friends:” Social capital and college students’ use of online social network sites. Journal of computer-mediated communication 12, 4 (2007), 1143–1168.
[16]
Joel R Evans and Anil Mathur. 2018. The value of online surveys: A look back and a look ahead. Internet Research (2018).
[17]
Weimiao Fan and Zheng Yan. 2010. Factors affecting response rates of the web survey: A systematic review. Computers in human behavior 26, 2 (2010), 132–139.
[18]
Anna Filippova, Erik Trainer, and James D Herbsleb. 2017. From diversity by numbers to diversity as process: supporting inclusiveness in software development teams with brainstorming. In Proceedings of the 39th International Conference on Software Engineering. IEEE Press, 152–163.
[19]
Richard J Fox, Melvin R Crask, and Jonghoon Kim. 1988. Mail survey response rate: A meta-analysis of selected techniques for inducing response. Public opinion quarterly 52, 4 (1988), 467–491.
[20]
Barbara L Fredrickson and Christine Branigan. 2005. Positive emotions broaden the scope of attention and thought-action repertoires. Cognition & emotion 19, 3 (2005), 313–332.
[21]
Adrian Furnham. 1986. Response bias, social desirability and dissimulation. Personality and individual differences 7, 3 (1986), 385–400.
[22]
James H Gray, Emily Reardon, and Jennifer A Kotler. 2017. Designing for parasocial relationships and learning: Linear video, interactive media, and artificial intelligence. In Proceedings of the 2017 Conference on Interaction Design and Children. 227–237.
[23]
Jerald Greenberg. 1986. Determinants of perceived fairness of performance evaluations.Journal of applied psychology 71, 2 (1986), 340.
[24]
Robert M Groves. 2006. Nonresponse rates and nonresponse bias in household surveys. Public opinion quarterly 70, 5 (2006), 646–675.
[25]
Theo Downes-Le Guin, Reg Baker, Joanne Mechling, and Erica Ruyle. 2012. Myths and realities of respondent engagement in online surveys. International Journal of Market Research 54, 5 (2012), 613–633.
[26]
Yimeng Guo, Jacek A Kopec, Jolanda Cibere, Linda C Li, and Charles H Goldsmith. 2016. Population survey features and response rates: a randomized experiment. American Journal of Public Health 106, 8 (2016), 1422–1426.
[27]
Heesup Han, Jinsoo Hwang, and Sanghyeop Lee. 2017. Cognitive, affective, normative, and moral triggers of sustainable intentions among convention-goers. Journal of Environmental Psychology 51 (2017), 1–13.
[28]
Dirk Heerwegh. 2005. Effects of personal salutations in e-mail invitations to participate in a web survey. Public Opinion Quarterly 69, 4 (2005), 588–598.
[29]
Donald Horton and R Richard Wohl. 1956. Mass communication and para-social interaction: Observations on intimacy at a distance. psychiatry 19, 3 (1956), 215–229.
[30]
James Jaccard and Jim Jaccard. 2001. Interaction effects in logistic regression. Number 135. Sage.
[31]
Lars Bo Jeppesen and Lars Frederiksen. 2006. Why do users contribute to firm-hosted user communities? The case of computer-controlled music instruments. Organization science 17, 1 (2006), 45–63.
[32]
Adam N Joinson and Ulf-Dietrich Reips. 2007. Personalized salutation, power of sender and response rates to Web-based surveys. Computers in Human Behavior 23, 3 (2007), 1372–1383.
[33]
Lars Kaczmirek. 2005. Web Surveys. A Brief Guide on Usability and Implementation Issues. In Usability professionals. 102–106.
[34]
Dennis O Kaldenberg, Harold F Koenig, and Boris W Becker. 1994. Mail survey response rate patterns in a population of the elderly: does response deteriorate with age?The Public Opinion Quarterly 58, 1 (1994), 68–76.
[35]
Michael D Kaplowitz, Timothy D Hadlock, and Ralph Levine. 2004. A comparison of web and mail survey response rates. Public opinion quarterly 68, 1 (2004), 94–101.
[36]
Malcolm Koo and Harvey Skinner. 2005. Challenges of internet recruitment: a case study with disappointing results. Journal of Medical Internet Research 7, 1 (2005), e6.
[37]
Frank J Landy, Janet L Barnes, and Kevin R Murphy. 1978. Correlates of perceived fairness and accuracy of performance evaluation.Journal of Applied psychology 63, 6 (1978), 751.
[38]
Howard Levene 1960. Contributions to probability and statistics. Essays in honor of Harold Hotelling(1960), 278–292.
[39]
Gerald S Leventhal, Jurgis Karuza, and William R Fry. 1980. Beyond fairness: A theory of allocation preferences. (1980).
[40]
Mingnan Liu, Noble Kuriakose, Jon Cohen, and Sarah Cho. 2016. Impact of web survey invitation design on survey participation, respondents, and survey responses. Social science computer review 34, 5 (2016), 631–644.
[41]
Peter McCullagh. 1980. Regression models for ordinal data. Journal of the Royal Statistical Society: Series B (Methodological) 42, 2(1980), 109–127.
[42]
J Scott Mizes, E Louis Fleece, and Cindy Roos. 1984. Incentives for increasing return rates: Magnitude levels, response bias, amd format. Public Opinion Quarterly 48, 4 (1984), 794–800.
[43]
Hendrik Müller and Aaron Sedley. 2015. Designing Surveys for HCI Research. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI EA ’15). Association for Computing Machinery, New York, NY, USA, 2485–2486. https://doi.org/10.1145/2702613.2706683
[44]
Hendrik Müller, Aaron Sedley, and Elizabeth Ferrall-Nunge. 2014. Designing Unbiased Surveys for HCI Research. In CHI ’14 Extended Abstracts on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI EA ’14). Association for Computing Machinery, New York, NY, USA, 1027–1028. https://doi.org/10.1145/2559206.2567822
[45]
Jessica Gall Myrick. 2015. Emotion regulation, procrastination, and watching cat videos online: Who watches Internet cats, why, and to what effect?Computers in human behavior 52 (2015), 168–176.
[46]
Alexander Nolte, Irene-Angelica Chounta, and James D Herbsleb. 2020. What Happens to All These Hackathon Projects? Identifying Factors to Promote Hackathon Project Continuation. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2(2020), 1–26.
[47]
Alexander Nolte, Ei Pa Pa Pe-Than, Abasi-amefon Obot Affia, Chalalai Chaihirunkarn, Anna Filippova, Arun Kalyanasundaram, Maria Angelica Medina Angarita, Erik Trainer, and James D Herbsleb. 2020. How to organize a hackathon–A planning kit. arXiv preprint arXiv:2008.08025(2020).
[48]
Mary Beth Oliver and Arthur A Raney. 2011. Entertainment as pleasurable and meaningful: Identifying hedonic and eudaimonic motivations for entertainment consumption. Journal of Communication 61, 5 (2011), 984–1004.
[49]
Reinhard Pekrun, Thomas Goetz, Wolfram Titz, and Raymond P Perry. 2002. Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational psychologist 37, 2 (2002), 91–105.
[50]
Marco Perugini and Richard P Bagozzi. 2001. The role of desires and anticipated emotions in goal-directed behaviours: Broadening and deepening the theory of planned behaviour. British Journal of Social Psychology 40, 1 (2001), 79–98.
[51]
David R Peryam and Francis J Pilgrim. 1957. Hedonic scale method of measuring food preferences.Food technology (1957).
[52]
Andraž Petrovčič, Gregor Petrič, and Katja Lozar Manfreda. 2016. The effect of email invitation elements on response rate in a web survey within an online community. Computers in Human Behavior 56 (2016), 320–329.
[53]
Stephen R Porter and Michael E Whitcomb. 2003. The impact of contact type on web survey response rates. The Public Opinion Quarterly 67, 4 (2003), 579–588.
[54]
Stephen R Porter and Michael E Whitcomb. 2005. E-mail subject lines and their effect on web survey viewing and response. Social Science Computer Review 23, 3 (2005), 380–387.
[55]
Huilian Sophie Qiu, Alexander Nolte, Anita Brown, Alexander Serebrenik, and Bogdan Vasilescu. 2019. Going farther together: The impact of social capital on sustained participation in open source. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 688–699.
[56]
Huilian Sophie Qiu, Yang Wen, and Alexander Nolte. 2021. Approaches to Diversifying the Programmer Community–The Case of the Girls Coding Day. In 2021 IEEE/ACM 13th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE). IEEE, 91–100.
[57]
Barbara A Schuldt and Jeff W Totten. 1994. Electronic mail vs. mail survey response rates. Marketing Research 6, 1 (1994), 3–7.
[58]
Kazim Sheikh and Stephen Mattingly. 1981. Investigating non-response bias in mail surveys.Journal of Epidemiology & Community Health 35, 4 (1981), 293–296.
[59]
Jesús H Trespalacios and Ross A Perkins. 2016. Effects of personalization and invitation email length on web-based survey response rates. TechTrends 60, 4 (2016), 330–335.
[60]
James C Witte, Roy P Pargas, Catherine Mobley, and James Hawdon. 2004. Instrument effects of images in web surveys: A research note. Social Science Computer Review 22, 3 (2004), 363–369.
[61]
Kevin B Wright. 2005. Researching Internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of computer-mediated communication 10, 3 (2005), JCMC1034.
[62]
Francis J Yammarino, Steven J Skinner, and Terry L Childers. 1991. Understanding mail survey response behavior a meta-analysis. Public Opinion Quarterly 55, 4 (1991), 613–639.
[63]
Doreen Zillmann, Andreas Schmitz, Jan Skopek, and Hans-Peter Blossfeld. 2014. Survey topic and unit nonresponse. Quality & quantity 48, 4 (2014), 2069–2088.
Index Terms
- The CAT Effect: Exploring the Impact of Casual Affective Triggers on Online Surveys’ Response Rates
Recommendations
Response Effects of Prenotification, Prepaid Cash, Prepaid Vouchers, and Postpaid Vouchers
In a web-based experiment with 1,750 randomly sampled university students, we investigated the effect of mailed prenotification plus prepaid cash, mailed prenotification plus a prepaid voucher, mailed prenotification plus a postpaid voucher, and mailed ...
Response Rates for Surveys Completed With Paper-and-Pencil and Computers: Using Meta-Analysis to Assess Equivalence
The increasing number of self-report surveys being collected using computers has led to a body of literature examining the response rates for computerized surveys compared with the more traditional paper-and-pencil method. However, results from individual ...
Response Rates and Data Quality in Web and Mail Surveys Administered to PhD Holders
Despite the extensive use of web surveys today, there are certain methodological factors related to participant cooperation and data quality, which remain unclear and require further study. Here, the authors compare responses to a survey administered in ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2022
10459 pages
ISBN:9781450391573
DOI:10.1145/3491102
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 April 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
CHI '22
Sponsor:
CHI '22: CHI Conference on Human Factors in Computing Systems
April 29 - May 5, 2022
LA, New Orleans, USA
Acceptance Rates
Overall Acceptance Rate 6,199 of 26,314 submissions, 24%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 246Total Downloads
- Downloads (Last 12 months)54
- Downloads (Last 6 weeks)5
Reflects downloads up to 01 Nov 2024
Other Metrics
Citations
Cited By
View all- Angarita MKolnes MNolte A(2024)Startup Creation Beyond Hackathons – A Survey on Startup Development and SupportSoftware Business10.1007/978-3-031-53227-6_15(205-221)Online publication date: 9-Feb-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format