research-article

Unique Exams: Designing Assessments for Integrity and Fairness

Authors:

Lisa YanAuthors Info & Claims

SIGCSE '21: Proceedings of the 52nd ACM Technical Symposium on Computer Science Education

Pages 1170 - 1176

https://doi.org/10.1145/3408877.3432556

Published: 05 March 2021 Publication History

Abstract

During the COVID-19 pandemic, many educators have had to rethink their methodology for summative assessment. Are timed and proctored exams appropriate---or even feasible---in this new open-internet, online learning environment? In this experience paper, we discuss our unique exams framework: our tool for generating exams that are uniquely identifiable but conceptually identical. In our university-level Probability for Computer Scientists Course, students completed unique exams generated from a common exam skeleton, with unique numeric variations per problem. With few deviations from the creation, administration, and grading processes of a traditional exam, our framework can provide a layer of security for both students and instructors about exam reliability for any classroom environment---in-person or online. In addition to sharing our experience designing unique exams, in this paper we also present a simple end-to-end tool and example question templates for different CS subjects that other instructors can adapt to their own courses.

References

[1]

[n.d.]. Chegg. https://www.chegg.com. Accessed: 2020-07-30.

[2]

[n.d.]. Create new possibilities with Pearson. Start learning today. https: //www.pearson.com/en-us.html

[3]

[n.d.]. EXAMSOFT PRODUCT SPECIFICATIONS. https://examsoft.com/productspecifications/exam-monitor. Accessed: 2020-07-30.

[4]

[n.d.]. StackOverflow. https://www.stackoverflow.com. Accessed: 2020-07-30.

[5]

[n.d.]. Student Guide: Exam Integrity with ExamID and ExamMonitor. https: //www.bcm.edu/sites/default/files/StudentGuide_ExamIntegrity.pdf. Accessed: 2020-07-30.

[6]

Kwame Anthony Appiah. [n.d.]. If My Classmates Are Going to Cheat on an Online Exam, Why Can't I? https://www.nytimes.com/2020/04/07/magazine/if-myclassmates-are-going-to-cheat-on-an-online-exam-why-cant-i.html. Accessed: 2020-08-28.

[7]

Tshepo Batane. 2010. Turning to Turnitin to fight plagiarism among university students. Journal of Educational Technology & Society 13, 2 (2010), 1--12.

[8]

Peter Brusilovsky and Sergey Sosnovsky. 2005. Individualized exercises for selfassessment of programming knowledge: An evaluation of QuizPACK. Journal on Educational Resources in Computing (JERIC) 5, 3 (2005), 6--es.

Digital Library

[9]

Binglin Chen, Sushmita Azad, Max Fowler, Matthew West, and Craig Zilles. 2020. Learning to Cheat: Quantifying Changes in Score Advantage of Unproctored Assessments Over Time. In Proceedings of the Seventh ACM Conference on Learning@ Scale. 197--206.

Digital Library

[10]

Binglin Chen, Matthew West, and Craig Zilles. 2018. How Much Randomization Is Needed to Deter Collaborative Cheating on Asynchronous Exams?. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale (L@S '18). Association for Computing Machinery, New York, NY, USA, 1--10. https://doi.org/10.1145/ 3231644.3231664

Digital Library

[11]

Jonathan Corley, Ana Stanescu, Lewis Baumstark, and Michael C. Orsega. 2020. Paper Or IDE? The Impact of Exam Format on Student Performance in a CS1 Course. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (SIGCSE '20). Association for Computing Machinery, New York, NY, USA, 706--712. https://doi.org/10.1145/3328778.3366857

[12]

Henry Corrigan-Gibbs, Nakull Gupta, Curtis Northcutt, Edward Cutrell, and William Thies. 2015. Deterring cheating in online environments. ACM Transactions on Computer-Human Interaction (TOCHI) 22, 6 (2015), 1--23.

Digital Library

[13]

Paul Denny, Sathiamoorthy Manoharan, Ulrich Speidel, Giovanni Russello, and Angela Chang. 2019. On the Fairness of Multiple-Variant Multiple-Choice Examinations. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. 462--468.

Digital Library

[14]

Chinedu Emeka and Craig Zilles. 2020. Student Perceptions of Fairness and Security in a Versioned Programming Exam. In Proceedings of the 2020 ACM Conference on International Computing Education Research (ICER '20). Association for Computing Machinery, New York, NY, USA, 25--35. https://doi.org/10.1145/ 3372782.3406275

Digital Library

[15]

Deirdre Fern. 2020. BU Investigating Whether Students Cheated on Online Exams. The Boston Globe (April 2020).

[16]

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. arXiv:1412.6572 [cs, stat] (March 2015). arXiv:1412.6572 [cs, stat]

[17]

Thomas R. Guskey. 2003. How Classroom Assessments Improve Learning. Educational, School, and Counseling Psychology Faculty Publications. (2003), 8.

[18]

Stylianos Hatzipanagos and Steven Warburton. 2009. Feedback as dialogue: exploring the links between formative assessment and social software in distance learning. Learning, Media and Technology 34, 1 (2009), 45--59. https://doi.org/10. 1080/17439880902759919 arXiv:https://doi.org/10.1080/17439880902759919

[19]

Jim Hollan and Scott Stornetta. 1992. Beyond Being There. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '92). Association for Computing Machinery, New York, NY, USA, 119--125. https://doi.org/10. 1145/142750.142769

Digital Library

[20]

Im Y Jung and Heon Y Yeom. 2009. Enhanced security for online exams using group cryptography. IEEE transactions on Education 52, 3 (2009), 340--349.

Digital Library

[21]

Jeffrey D. Karpicke and Janell R. Blunt. 2011. Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping. Science 331, 6018 (2011), 772--775. https://doi.org/10.1126/science.1199327 arXiv:https://science.sciencemag.org/content/331/6018/772.full.pdf

[22]

Peter T. Knight. 2002. Summative Assessment in Higher Education: Practices in disarray. Studies in Higher Education 27, 3 (Aug. 2002), 275--286. https: //doi.org/10.1080/03075070220000662 Publisher: Routledge.

[23]

Anita Krsak. 2007. Curbing academic dishonesty in online courses. In TCC. TCCHawaii, 159--170.

[24]

Ghader Kurdi, Jared Leo, Bijan Parsia, Uli Sattler, and Salam Al-Emari. 2020. A Systematic Review of Automatic Question Generation for Educational Purposes. International Journal of Artificial Intelligence in Education 30, 1 (2020), 121--204.

[25]

Tyler M. Miller and Lisa Geraci. 2011. Training metacognition in the classroom: the influence of incentives and feedback on exam predictions. Metacognition and Learning 6, 3 (Dec. 2011), 303--314. https://doi.org/10.1007/s11409-011-9083-7

[26]

Derek Newton. 2020. Another Problem with Shifting Education Online: A Rise in Cheating. Washington Post (July 2020).

[27]

Gary M. Olson and Judith S. Olson. 2000. Distance Matters. Human--Computer Interaction 15, 2--3 (Sept. 2000), 139--178. https://doi.org/10. 1207/S15327051HCI1523_4

Digital Library

[28]

Nick Parlante, Julie Zelenski, Daniel Zingaro, Kevin Wayne, Dave O'Hallaron, Joshua T. Guerin, Stephen Davies, Zachary Kurmas, and Keen Debby. 2012. Nifty Assignments. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (SIGCSE '12). Association for Computing Machinery, New York, NY, USA, 475--476. https://doi.org/10.1145/2157136.2157274

[29]

Chris Piech and Chris Gregg. 2018. BlueBook: A Computerized Replacement for Paper Tests in Computer Science. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE '18). Association for Computing Machinery, New York, NY, USA, 562--567. https://doi.org/10.1145/3159450.3159587

Digital Library

[30]

Justin Reich, Christopher J Buttimer, Dan Coleman, Richard D Colwell, Farah Faruqi, and Laura R Larke. 2020. What's Lost, What's Left, What's Next: Lessons Learned from the Lived Experiences of Teachers during the 2020 Novel Coronavirus Pandemic. https://doi.org/10.35542/osf.io/8exp9

[31]

Arjun Singh, Sergey Karayev, Kevin Gutowski, and Pieter Abbeel. 2017. Gradescope: A Fast, Flexible, and Fair System for Scalable Assessment of Handwritten Work. In Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale (L@S '17). Association for Computing Machinery, New York, NY, USA, 81--88. https://doi.org/10.1145/3051457.3051466

Digital Library

[32]

Adam Steinbaugh. 2020. ProctorU Threatens UC Santa Barbara Faculty over Criticism during Coronavirus Crisis.

[33]

Mark A. Stellmack, Nora K. Keenan, Rita R. Sandidge, Amy L. Sippl, and Yasmine L. Konheim-Kalkstein. 2012. Review, Revise, and Resubmit: The Effects of Self Critique, Peer Review, and Instructor Feedback on Student Writing. Teaching of Psychology 39, 4 (Oct. 2012), 235--244. https://doi.org/10.1177/0098628312456589

[34]

Hoi K. Suen. 2014. Peer Assessment for Massive Open Online Courses (MOOCs). International Review of Research in Open and Distributed Learning 15, 3 (2014), 312--327. https://doi.org/10.19173/irrodl.v15i3.1680

[35]

surang. [n.d.]. Icons made by surang from www.flaticon.com. https://www. flaticon.com/free-icon/test_2916270. Accessed: 2020-08-27.

[36]

G Whale. 1988. Plague: plagiarism detection using program structure. Dept. Technical Report. of Computer Science Technical Report 8805, University of NSW, Kensington. ..

[37]

Steven R. Wininger. 2005. Using Your Tests to Teach: Formative Summative Assessment. Teaching of Psychology 32, 3 (July 2005), 164--166. https://doi.org/ 10.1207/s15328023top3203_7 Publisher: SAGE Publications Inc.

[38]

Lisa Yan, Nick McKeown, Mehran Sahami, and Chris Piech. 2018. TMOSS: Using intermediate assignment work to understand excessive collaboration in large classes. In Proceedings of the 49th ACM technical symposium on computer science education. 110--115.

Digital Library

[39]

Craig Zilles, Robert Timothy Deloatch, Jacob Bailey, Bhuwan B. Khattar, Wade Fagen, and Cinda Heeren. 2015. Computerized testing: A vision and initial experiences. ASEE Annual Conference and Exposition, Conference Proceedings 122nd ASEE Annual Conference and Exposition: Making Value for Society, 122nd ASEE Annual Conference and Exposition: Making Value for... (1 Jan. 2015). 2015 122nd ASEE Annual Conference and Exposition; Conference date: 14-06-2015 Through 17-06-2015.

Cited By

Richards MWaugh KSlaymaker MPetre MWoodthorpe JGooch D(2024)Bob or Bot: Exploring ChatGPT's Answers to University Computer Science AssessmentACM Transactions on Computing Education10.1145/363328724:1(1-32)Online publication date: 14-Jan-2024
https://dl.acm.org/doi/10.1145/3633287
Reckinger SHummel JHeckman SStephenson BStone JBattestilli LRebelsky SShoop L(2024)Traditional vs. Flexible Modalities in a Data Structures ClassProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630952(1112-1118)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630952
Minagar SSakzad ATack GRudolph CSheard JStephenson BStone JBattestilli LRebelsky SShoop L(2024)ALAN: Assessment-as-Learning Authentic Tasks for NetworkingProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630774(853-859)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630774
Show More Cited By

Index Terms

Unique Exams: Designing Assessments for Integrity and Fairness
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
        CS1
      2. Student assessment

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCSE '21: Proceedings of the 52nd ACM Technical Symposium on Computer Science Education

March 2021

1454 pages

ISBN:9781450380621

DOI:10.1145/3408877

General Chairs:
Mark Sherriff
University of Virginia, USA
,
Laurence D. Merkle
Air Force Institute of Technology, USA
,
Program Chairs:
Pamela Cutter
Kalamazoo College, USA
,
Alvaro Monge
California State University, Long Beach, USA
,
Judithe Sheard
Monash University, Australia

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 March 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCSE '21

Sponsor:

SIGCSE

SIGCSE '21: The 52nd ACM Technical Symposium on Computer Science Education

March 13 - 20, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE Virtual 2024

Sponsor:
sigcse

1st ACM Virtual Global Computing Education Conference

December 5 - 8, 2024

Virtual Event , NC , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
311
Total Downloads

Downloads (Last 12 months)60
Downloads (Last 6 weeks)7

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Richards MWaugh KSlaymaker MPetre MWoodthorpe JGooch D(2024)Bob or Bot: Exploring ChatGPT's Answers to University Computer Science AssessmentACM Transactions on Computing Education10.1145/363328724:1(1-32)Online publication date: 14-Jan-2024
https://dl.acm.org/doi/10.1145/3633287
Reckinger SHummel JHeckman SStephenson BStone JBattestilli LRebelsky SShoop L(2024)Traditional vs. Flexible Modalities in a Data Structures ClassProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630952(1112-1118)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630952
Minagar SSakzad ATack GRudolph CSheard JStephenson BStone JBattestilli LRebelsky SShoop L(2024)ALAN: Assessment-as-Learning Authentic Tasks for NetworkingProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630774(853-859)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630774
Mdaka DMuchenje TTshilongamulenzhe TMathonsi T(2024)A Multimodal Authentication Method for Electronic Exams2024 International Conference on Artificial Intelligence, Big Data, Computing and Data Communication Systems (icABCD)10.1109/icABCD62167.2024.10645265(1-5)Online publication date: 1-Aug-2024
https://doi.org/10.1109/icABCD62167.2024.10645265
Bugajewski DBugajewska M(2024)System for Preparing and Executing Math Exams Online – A Case StudyArtificial intelligence and Machine Learning10.1007/978-3-031-62843-6_30(318-329)Online publication date: 29-Jun-2024
https://doi.org/10.1007/978-3-031-62843-6_30
Vicente MFernández Peris CMartínez M(2024)EVERSYS: Efficient exam versioning tool for linear circuits and other problem‐based subjectsComputer Applications in Engineering Education10.1002/cae.2271532:3Online publication date: 18-Jan-2024
https://doi.org/10.1002/cae.22715
Crick TPrickett TVasiliou CChitare NWatson ILaakso MMonga MSimon Sheard J(2023)Exploring Computing Students' Post-Pandemic Learning Preferences with Workshops: A UK Institutional Case StudyProceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 110.1145/3587102.3588807(173-179)Online publication date: 29-Jun-2023
https://dl.acm.org/doi/10.1145/3587102.3588807
Hoseana JLeonardo Louismono AStepanus O(2023)Using graph-colouring to organise multi-version assessmentsInternational Journal of Mathematical Education in Science and Technology10.1080/0020739X.2023.2255592(1-13)Online publication date: 13-Sep-2023
https://doi.org/10.1080/0020739X.2023.2255592
Braun JSchläfli RSchmocker DWilding B(2023)Experiences and Critical Reflection on Online-Assessment with Excel Case Studies – Review on a Successful Online-Assessment Practice as Well as the Adaptation to a Remote Setting Due to the COVID-19 PandemicStudent Assessment in Digital and Hybrid Learning Environments10.1007/978-3-658-42253-0_2(9-36)Online publication date: 1-Dec-2023
https://doi.org/10.1007/978-3-658-42253-0_2
Griffiths B(2022)Mitigating cheating during online proctored examsResearch on Education and Media10.2478/rem-2022-001614:2(9-14)Online publication date: 15-Dec-2022
https://doi.org/10.2478/rem-2022-0016
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents