Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models

P Vaithilingam, T Zhang, EL Glassman - Chi conference on human …, 2022 - dl.acm.org
Recent advances in Large Language Models (LLM) have made automatic code generation
possible for real-world programming tasks in general-purpose programming languages …

The programmer's assistant: Conversational interaction with a large language model for software development

SI Ross, F Martinez, S Houde, M Muller… - Proceedings of the 28th …, 2023 - dl.acm.org
Large language models (LLMs) have recently been applied in software engineering to
perform tasks such as translating code between programming languages, generating code …

Is ChatGPT the ultimate programming assistant--how far is it?

H Tian, W Lu, TO Li, X Tang, SC Cheung… - arXiv preprint arXiv …, 2023 - arxiv.org
Recently, the ChatGPT LLM has received great attention: it can be used as a bot for
discussing source code, prompting it to suggest changes, provide descriptions or even …

Chatgpt and software testing education: Promises & perils

S Jalil, S Rafi, TD LaToza, K Moran… - 2023 IEEE international …, 2023 - ieeexplore.ieee.org
Over the past decade, predictive language modeling for code has proven to be a valuable
tool for enabling new forms of automation for developers. More recently, we have seen the …

Do users write more insecure code with AI assistants?

N Perry, M Srivastava, D Kumar, D Boneh - Proceedings of the 2023 …, 2023 - dl.acm.org
AI code assistants have emerged as powerful tools that can aid in the software development
life-cycle and can improve developer productivity. Unfortunately, such assistants have also …

“What it wants me to say”: Bridging the abstraction gap between end-user programmers and code-generating large language models

MX Liu, A Sarkar, C Negreanu, B Zorn… - Proceedings of the …, 2023 - dl.acm.org
Code-generating large language models map natural language to code. However, only a
small portion of the infinite space of naturalistic utterances is effective at guiding code …

Synchromesh: Reliable code generation from pre-trained language models

G Poesia, O Polozov, V Le, A Tiwari, G Soares… - arXiv preprint arXiv …, 2022 - arxiv.org
Large pre-trained language models have been used to generate code, providing a flexible
interface for synthesizing programs from natural language specifications. However, they …

On the design of ai-powered code assistants for notebooks

AM McNutt, C Wang, RA Deline… - Proceedings of the 2023 …, 2023 - dl.acm.org
AI-powered code assistants, such as Copilot, are quickly becoming a ubiquitous component
of contemporary coding contexts. Among these environments, computational notebooks …

LLM is Like a Box of Chocolates: the Non-determinism of ChatGPT in Code Generation

S Ouyang, JM Zhang, M Harman, M Wang - arXiv preprint arXiv …, 2023 - arxiv.org
There has been a recent explosion of research on Large Language Models (LLMs) for
software engineering tasks, in particular code generation. However, results from LLMs can …

Investigating code generation performance of ChatGPT with crowdsourcing social data

Y Feng, S Vanam, M Cherukupally… - 2023 IEEE 47th …, 2023 - ieeexplore.ieee.org
The recent advancements in Artificial Intelligence, particularly in large language models and
generative models, are reshaping the field of software engineering by enabling innovative …