Recently, machine learning (ML) methods have been used to create powerful language
models for a broad range of natural language processing tasks. An important subset of this …

Large language models have been shown to achieve remarkable performance across a
variety of natural language tasks using few-shot learning, which drastically reduces the …

Program synthesis strives to generate a computer program as a solution to a given problem
specification, expressed with input-output examples or natural language descriptions. The …

Programming is a powerful and ubiquitous problem-solving tool. Systems that can assist
programmers or even generate programs themselves could make programming more …

Large pre-trained code generation models, such as OpenAI Codex, can generate syntax-
and function-correct code, making the coding of programmers more productive and our …

This paper explores the limits of the current generation of large language models for
program synthesis in general purpose programming languages. We evaluate a collection of …

The BigCode project is an open-scientific collaboration working on the responsible
development of large language models for code. This tech report describes the progress of …

Benchmark datasets have a significant impact on accelerating research in programming
language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster …

Neural code synthesis has reached a point where snippet generation is accurate enough to
be considered for integration into human software development workflows. Commercial …

While programming is one of the most broadly applicable skills in modern society, modern
machine learning models still cannot code solutions to basic problems. Despite its …