Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models

P Vaithilingam, T Zhang, EL Glassman - Chi conference on human …, 2022 - dl.acm.org
Recent advances in Large Language Models (LLM) have made automatic code generation
possible for real-world programming tasks in general-purpose programming languages …

Self-edit: Fault-aware code editor for code generation

K Zhang, Z Li, J Li, G Li, Z Jin - arXiv preprint arXiv:2305.04087, 2023 - arxiv.org
Large language models (LLMs) have demonstrated an impressive ability to generate codes
on competitive programming tasks. However, with limited sample numbers, LLMs still suffer …

What is it like to program with artificial intelligence?

A Sarkar, AD Gordon, C Negreanu, C Poelitz… - arXiv preprint arXiv …, 2022 - arxiv.org
Large language models, such as OpenAI's codex and Deepmind's AlphaCode, can
generate code to solve a variety of problems expressed in natural language. This …

No need to lift a finger anymore? assessing the quality of code generation by chatgpt

Z Liu, Y Tang, X Luo, Y Zhou… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Large language models (LLMs) have demonstrated impressive capabilities across various
natural language processing (NLP) tasks, such as machine translation, question answering …

In-ide code generation from natural language: Promise and challenges

FF Xu, B Vasilescu, G Neubig - ACM Transactions on Software …, 2022 - dl.acm.org
A great part of software development involves conceptualizing or communicating the
underlying procedures and logic that needs to be expressed in programs. One major …

Discovering the syntax and strategies of natural language programming with generative language models

E Jiang, E Toh, A Molina, K Olson, C Kayacik… - Proceedings of the …, 2022 - dl.acm.org
In this paper, we present a natural language code synthesis tool, GenLine, backed by 1) a
large generative language model and 2) a set of task-specific prompts that create or change …

Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation

J Liu, CS Xia, Y Wang, L Zhang - Advances in Neural …, 2024 - proceedings.neurips.cc
Program synthesis has been long studied with recent approaches focused on directly using
the power of Large Language Models (LLMs) to generate code. Programming benchmarks …

A study on robustness and reliability of large language model code generation

L Zhong, Z Wang - arXiv preprint arXiv:2308.10335, 2023 - arxiv.org
Recently, the large language models (LLMs) have shown extraordinary ability in
understanding natural language and generating programming code. It has been a common …

Craft: Customizing llms by creating and retrieving from specialized toolsets

L Yuan, Y Chen, X Wang, YR Fung, H Peng… - arXiv preprint arXiv …, 2023 - arxiv.org
Large language models (LLMs) are often augmented with tools to solve complex tasks. By
generating code snippets and executing them through task-specific Application …

Exploring parameter-efficient fine-tuning techniques for code generation with large language models

M Weyssow, X Zhou, K Kim, D Lo… - arXiv preprint arXiv …, 2023 - arxiv.org
Large Language Models (LLMs) possess impressive capabilities to generate meaningful
code snippets given natural language intents in zero-shot, ie, without the need for specific …