In 2006, Geoffrey Hinton proposed the concept of training “Deep Neural Networks (DNNs)”
and an improved model training method to break the bottleneck of neural network …

The immense amounts of source code provide ample challenges and opportunities during
software development. To handle the size of code bases, developers commonly search for …

GitHub and OpenAI recently launched Copilot, an" AI pair programmer" that utilizes the
power of Natural Language Processing, Static Analysis, Code Synthesis, and Artificial …

Semantic code search is the task of retrieving relevant code given a natural language query.
While related to other information retrieval tasks, it requires bridging the gap between the …

Pre-trained models have been shown effective in many code intelligence tasks. These
models are pre-trained on large-scale unlabeled corpus and then fine-tuned in downstream …

Recent research has achieved impressive results on understanding and improving source
code by building up on machine-learning techniques developed for natural languages. A …

A transcompiler, also known as source-to-source translator, is a system that converts source
code from a high-level programming language (such as C++ or Python) to another …

Code completion, which aims to predict the following code token (s) according to the code
context, can improve the productivity of software development. Recent work has proved that …

Pre-trained models like BERT have achieved strong improvements on many natural
language processing (NLP) tasks, showing their great generalizability. The success of pre …

Recent advances in self-supervised learning have dramatically improved the state of the art
on a wide variety of tasks. However, research in language model pre-training has mostly …