Understanding Hackers’ Work: An Empirical Study of Offensive Security Practitioners
Pages 1669 - 1680
Abstract
Offensive security-tests are commonly employed to pro-actively discover potential vulnerabilities. They are performed by specialists, also known as penetration-testers or white-hat hackers. The chronic lack of available white-hat hackers prevents sufficient security test coverage of software. Research into automation tries to alleviate this problem by improving the efficiency of security testing. To achieve this, researchers and tool builders need a solid understanding of how hackers work, their assumptions, and pain points.
In this paper, we present a first data-driven exploratory qualitative study of twelve security professionals, their work and problems occurring therein. We perform a thematic analysis to gain insights into the execution of security assignments, hackers' thought processes and encountered challenges. This analysis allows us to conclude with recommendations for researchers and tool builders, to increase the efficiency of their automation and identify novel areas for research.
References
[1]
[n. d.]. Acunetix: Web Vulnerability Scanner. https://www.acunetix.com/ Accessed: 2022-09-30
[2]
[n. d.]. BloodHoundAD: Six Degrees of Domain Admin. https://github.com/BloodHoundAD/BloodHound Accessed: 2022-09-30
[3]
[n. d.]. Conti cyber attack on the HSE, Independent Post Incident Review. https://www.hse.ie/eng/services/publications/conti-cyber-attack-on-the-hse-full-report.pdf Accessed: 2022-09-30
[4]
[n. d.]. Conti’s Hacker Manuals — Read, Reviewed & Analyzed. https://www.akamai.com/blog/security/conti-hacker-manual-reviewed Accessed: 2022-09-30
[5]
[n. d.]. Delve: Software Tool to Analyze Qualitative Data. https://delvetool.com/ Accessed: 2022-10-01
[6]
[n. d.]. DirBuster. https://www.kali.org/tools/dirbuster/ Accessed: 2022-09-30
[7]
[n. d.]. GhostPack/Certify: Active Directory certificate abuse. https://github.com/GhostPack/Certify Accessed: 2022-09-30
[8]
[n. d.]. gobuster: Directory/File, DNS and VHost busting tool written in Go. https://github.com/OJ/gobuster Accessed: 2022-09-30
[9]
[n. d.]. https://nakedsecurity.sophos.com/2021/07/16/more-printnightmare-we-told-you-not-to-turn-the-print-spooler-back-on/. https://nakedsecurity.sophos.com/2021/07/16/more-printnightmare-we-told-you-not-to-turn-the-print-spooler-back-on/ Accessed: 2022-10-03
[10]
[n. d.]. Invicti: Web Application Security for Enterprise. https://www.invicti.com/ Accessed: 2022-09-30
[11]
[n. d.]. Metasploit: Penetration Testing Software. https://github.com/rapid7/metasploit-framework Accessed: 2022-09-30
[12]
[n. d.]. Methodology for Top 10. https://groups.google.com/a/owasp.org/g/leaders/c/pFLxDLE28ZA Accessed: 2022-09-30
[13]
[n. d.]. Nessus Vulnerability Assessment Solution. https://www.tenable.com/products/nessus/nessus-professional Accessed: 2022-09-30
[14]
[n. d.]. Nmap: the Network Mapper — Free Security Scanner. https://nmap.org Accessed: 2022-09-30
[15]
[n. d.]. Nuclei: Fast and customizable vulnerability scanner based on simple YAML based DSL. https://github.com/projectdiscovery/nuclei Accessed: 2022-09-30
[16]
[n. d.]. OWASP Zed Attack Proxy (ZAP). https://www.zapproxy.org/ Accessed: 2022-09-30
[17]
[n. d.]. PTES Technical Guidelines. http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines Accessed: 2022-09-30
[18]
[n. d.]. sqlmap: automatic SQL injection and database takeover tool. https://sqlmap.org/ Accessed: 2022-09-30
[19]
[n. d.]. Windows Print Spooler Remote Code Execution Vulnerability (CVE-2021-34527). https://msrc.microsoft.com/update-guide/vulnerability/CVE-2021-34527 Accessed: 2022-09-30
[20]
[n. d.]. Zero Day Initiative. https://www.zerodayinitiative.com/blog Accessed: 2022-09-30
[21]
2016-07-06. DIRECTIVE (EU) 2016/1148 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 6 July 2016 concerning measures for a high common level of security of network and information systems across the Union. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016L1148 Official Journal of the European Union, L 194 (2016-07-06), 1–30.
[22]
Maurício Aniche, Christoph Treude, Igor Steinmacher, Igor Wiese, Gustavo Pinto, Margaret-Anne Storey, and Marco Aurélio Gerosa. 2018. How modern news aggregators help development communities shape and share knowledge. In Proceedings of the 40th International conference on software engineering. 499–510.
[23]
Farzana Ahamed Bhuiyan, Akond Rahman, and Patrick Morrison. 2020. Vulnerability discovery strategies used in software projects. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Workshops. 13–18.
[24]
Loyd Blankenship. 1986. The Conscience of a Hacker. Phrack, 7 (1986), Jan., http://www.phrack.org/archives/issues/7/3.txt
[25]
Petar Boyanov. 2018. Educational exploiting the information resources and invading the security mechanisms of the operating system Windows 7 with the exploit Eternalblue and Backdoor Doublepulsar. Association Scientific and Applied Research, 14 (2018), 34.
[26]
Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health, 11, 4 (2019), 589–597.
[27]
Vit Bukac, Vaclav Lorenc, and Vashek Matyáš. 2014. Red queen’s race: APT win-win game. In Cambridge International Workshop on Security Protocols. 55–61.
[28]
Mariano Ceccato, Paolo Tonella, Cataldo Basile, Paolo Falcarin, Marco Torchiano, Bart Coppens, and Bjorn De Sutter. 2019. Understanding the behaviour of hackers while performing attack tasks in a professional setting and in a public challenge. Empirical Software Engineering, 24 (2019), 240–286.
[29]
Zakir Durumeric, Frank Li, James Kasten, Johanna Amann, Jethro Beekman, Mathias Payer, Nicolas Weaver, David Adrian, Vern Paxson, and Michael Bailey. 2014. The matter of heartbleed. In Proceedings of the 2014 conference on internet measurement conference. 475–488.
[30]
Stefano Ferretti, Silvia Mirri, Catia Prandi, and Paola Salomoni. 2016. Automatic web content personalization through reinforcement learning. Journal of Systems and Software, 121 (2016), 157–169.
[31]
Jill J Francis, Marie Johnston, Clare Robertson, Liz Glidewell, Vikki Entwistle, Martin P Eccles, and Jeremy M Grimshaw. 2010. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychology and health, 25, 10 (2010), 1229–1245.
[32]
Hugo Gascon, Christian Wressnegger, Fabian Yamaguchi, Daniel Arp, and Konrad Rieck. 2015. Pulsar: Stateful black-box fuzzing of proprietary network protocols. In Security and Privacy in Communication Networks: 11th EAI International Conference, SecureComm 2015, Dallas, TX, USA, October 26-29, 2015, Proceedings 11. 330–347.
[33]
Egon G Guba and Yvonna S Lincoln. 1994. Competing paradigms in qualitative research. Handbook of qualitative research, 2, 163-194 (1994), 105.
[34]
Greg Guest, Arwen Bunce, and Laura Johnson. 2006. How many interviews are enough? An experiment with data saturation and variability. Field methods, 18, 1 (2006), 59–82.
[35]
Aaron Guzman. [n. d.]. OWASP Firmware Security Testing Methodology. https://scriptingxss.gitbook.io/firmware-security-testing-methodology/ Accessed: 2022-09-30
[36]
Aaron Guzman and Cedric Bassem. 2020. OWASP IoT Security Verification Standard. https://github.com/OWASP/IoT-Security-Verification-Standard-ISVS/releases/download/1.0RC/OWASP_ISVS-1.0RC-en_WIP_.pdf
[37]
Andreas Happe and Cito Jürgen. 2023. Getting pwn’d by AI: Penetration Testing with Large Language Models. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2023). Association for Computing Machinery, New York, NY, USA. 5 pages. https://doi.org/10.1145/3611643.3613083
[38]
Richard Harang and Felipe N Ducau. 2018. Measuring the speed of the Red Queen’s Race. BlackHat: Las Vegas, NV, USA.
[39]
Carlos Holguera, Bernhard Müller, Sven Schleier, and Jeroen Willemsen. 2022. OWASP Mobile Application Security Verification Standard. https://github.com/OWASP/owasp-masvs/releases/latest/download/OWASP_MASVS-v1.4.2-en.pdf
[40]
Nicolas Huaman, Bennet von Skarczinski, Dominik Wermke, Christian Stransky, Yasemin Acar, Arne Dreiß igacker, and Sascha Fahl. 2021. A large-scale interview study on information security in and attacks against small and medium-sized enterprises. In In 30th USENIX Security Symposium.
[41]
(ISC)2. 2022. (ISC)2 CYBERSECURITY WORKFORCE STUDY 2022. https://www.isc2.org//-/media/ISC2/Research/2022-WorkForce-Study/ISC2-Cybersecurity-Workforce-Study.ashx Accessed: 2023-04-28
[42]
Ioannis Katakis, Grigorios Tsoumakas, Evangelos Banos, Nick Bassiliades, and Ioannis Vlahavas. 2009. An adaptive personalized news dissemination system. Journal of intelligent information systems, 32 (2009), 191–212.
[43]
James Kettle. 2019. HTTP Desync Attacks: Request Smuggling Reborn. https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn Accessed: 2023-08-18
[44]
James Kettle. 2022. Browser-Powered Desync Attacks: A New Frontier in HTTP Request Smuggling. https://portswigger.net/research/browser-powered-desync-attacks Accessed: 2023-08-18
[45]
Andrew G Kotulic and Jan Guynes Clark. 2004. Why there aren’t more information security research studies. Information & Management, 41, 5 (2004), 597–607.
[46]
Sydney Lake. 2022. The cybersecurity industry is short 3.4 million workers—that’s good news for cyber wages. https://fortune.com/education/articles/the-cybersecurity-industry-is-short-3-4-million-workers-thats-good-news-for-cyber-wages/ Accessed: 2023-04-28
[47]
Noella Mackenzie and Sally Knipe. 2006. Research dilemmas: Paradigms, methods and methodology. Issues in educational research, 16, 2 (2006), 193–205.
[48]
Nuthan Munaiah, Akond Rahman, Justin Pelletier, Laurie Williams, and Andrew Meneely. 2019. Characterizing attacker behavior in a cybersecurity penetration testing competition. In 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1–6.
[49]
Bruce Potter and Gary McGraw. 2004. Software security testing. IEEE Security & Privacy, 2, 5 (2004), 81–85.
[50]
Elie Saad and Rick Mitchell. 2020. OWASP Web Security Testing Guide. https://github.com/OWASP/wstg/releases/download/v4.2/wstg-v4.2.pdf
[51]
MNK Saunders and PC Tosey. 2013. The layers of research design. University of Surrey.
[52]
Sven Schleier, Bernhard Mueller, Carlos Holguera, and Jeroen Willemsen. 2022. OWASP Mobile Application Security Testing Guide. https://github.com/OWASP/owasp-mastg/releases/latest/download/OWASP_MASTG-v1.5.0.pdf
[53]
Leif Singer, Fernando Figueira Filho, and Margaret-Anne Storey. 2014. Software engineering at the speed of light: how developers stay current using twitter. In Proceedings of the 36th International Conference on Software Engineering. 211–221.
[54]
Justin Smith, Christopher Theisen, and Titus Barik. 2020. A Case Study of Software Security Red Teams at Microsoft. In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 1–10.
[55]
Blake E Strom, Andy Applebaum, Doug P Miller, Kathryn C Nickels, Adam G Pennington, and Cody B Thomas. 2018. Mitre att&ck: Design and philosophy. In Technical report. The MITRE Corporation.
[56]
Timothy C Summers. 2015. How hackers think: A mixed method study of mental models and cognitive patterns of high-tech wizards. Case Western Reserve University.
[57]
Ari Takanen, Jared D Demott, Charles Miller, and Atte Kettunen. 2018. Fuzzing for software security testing and quality assurance. Artech House.
[58]
Romina Torres and Boris Tapia. 2011. Improving web api discovery by leveraging social information. In 2011 IEEE International Conference on Web Services. 744–745.
[59]
Catia Trubiani, Pooyan Jamshidi, Jurgen Cito, Weiyi Shang, Zhen Ming Jiang, and Markus Borg. 2019. Performance Issues? Hey DevOps, Mind the Uncertainty. IEEE Software, 36, 02 (2019), 110–117.
[60]
Niek Jan van den Hout. 2019. Standardised Penetration Testing? Examining the Usefulness of Current Penetration Testing Methodologies. Ph. D. Dissertation.
[61]
Andrew van der Stork, Brian Glas, Neil Smithline, and Torsten Gigler. 2021. OWASP Top 10:2021. https://owasp.org/Top10/0x00-notice/
[62]
Andrew van der Stork, Josh Grossman, Daniel Cuthbert, Elar Lang, and Jim Manico. 2021. OWASP Application Security Verification Standard. https://github.com/OWASP/ASVS/raw/v4.0.3/4.0/OWASP+Application+Security+Verification+Standard+4.0.3-en.pdf
[63]
Chris Wysopal, Lucas Nelson, Elfriede Dustin, and Dino Dai Zovi. 2006. The art of software security testing: identifying software security flaws. Pearson Education.
[64]
Mengwei Xu, Feng Qian, Qiaozhu Mei, Kang Huang, and Xuanzhe Liu. 2018. Deeptype: On-device deep learning for input personalization service with minimal privacy concern. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2, 4 (2018), 1–26.
[65]
Kuat Yessenov, Ivan Kuraj, and Armando Solar-Lezama. 2017. DemoMatch: API discovery from demonstrations. ACM SIGPLAN Notices, 52, 6 (2017), 64–78.
Index Terms
- Understanding Hackers’ Work: An Empirical Study of Offensive Security Practitioners
Recommendations
Cyber Security & Ethical Hacking For SMEs
KMO '16: Proceedings of the The 11th International Knowledge Management in Organizations Conference on The changing face of Knowledge Management Impacting SocietyLiterature posits that 30,000 SME websites are hacked daily. Thus it is acknowledged that web-based tools are essential to prevent cyber criminals hacking into online networks to comprise their services and gain access to confidential data for improper ...
Guidelines for Discovering and Improving Application Security
CSCS '13: Proceedings of the 2013 19th International Conference on Control Systems and Computer ScienceThis paper analyzes current threats in computer security for web-based applications with a SQL database. We conduct a penetration test in a real-case scenario of multiple attacks against the network, the web application and the SQL database. The test ...
Enhancing the comprehension of network sniffing attack in information security education using a hands-on lab approach
SIGITE '14: Proceedings of the 15th Annual Conference on Information technology educationSniffing attack is a common network attack and a fundamental topic to information security education. The sniffing attack is usually used by malicious users to spy network traffic, and to collect confidential and sensitive information. With the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2023
2215 pages
ISBN:9798400703270
DOI:10.1145/3611643
- General Chair:
- Satish Chandra,
- Program Chairs:
- Kelly Blincoe,
- Paolo Tonella
Copyright © 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 November 2023
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
Conference
ESEC/FSE '23
Sponsor:
ESEC/FSE '23: 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
December 3 - 9, 2023
CA, San Francisco, USA
Acceptance Rates
Overall Acceptance Rate 112 of 543 submissions, 21%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 666Total Downloads
- Downloads (Last 12 months)666
- Downloads (Last 6 weeks)116
Reflects downloads up to 05 Nov 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in