Export Citations
Publications
Save this search
Please login to be able to save your searches and receive alerts for new content matching your search criteria.
- research-article
Learning semantic representations to verify hardware designs
NIPS '21: Proceedings of the 35th International Conference on Neural Information Processing Systems•December 2021, Article No.: 1799, pp 23491-23504Verification is a serious bottleneck in the industrial hardware design cycle, routinely requiring person-years of effort. Practical verification relies on a "best effort" process that simulates the design on test inputs. This suggests a new research ...
- research-article
A Bayesian-symbolic approach to reasoning and learning in intuitive physics
NIPS '21: Proceedings of the 35th International Conference on Neural Information Processing Systems•December 2021, Article No.: 190, pp 2478-2490Humans can reason about intuitive physics in fully or partially observed environments even after being exposed to a very limited set of observations. This sample-efficient intuitive physical reasoning is considered a core domain of human common sense ...
- research-article
Training chain-of-thought via latent-variable inference
- Du Phan,
- Matthew D. Hoffman,
- David Dohan,
- Sholto Douglas,
- Tuan Anh Le,
- Aaron Parisi,
- Pavel Sountsov,
- Charles Sutton,
- Sharad Vikram,
- Rif A. Saurous
NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems•December 2023, Article No.: 3184, pp 72819-72841Large language models (LLMs) solve problems more accurately and interpretably when instructed to work out the answer step by step using a "chain-of-thought" (CoT) prompt. One can also improve LLMs' performance on a specific task by supervised fine-tuning,...
- research-article
LAMBDABEAM: neural program search with higher-order functions and lambdas
NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems•December 2023, Article No.: 2234, pp 51327-51346Search is an important technique in program synthesis that allows for adaptive strategies such as focusing on particular search directions based on execution results. Several prior works have demonstrated that neural models are effective at guiding ...
- research-article
PaLM: scaling language modeling with pathways
- Aakanksha Chowdhery,
- Sharan Narang,
- Jacob Devlin,
- Maarten Bosma,
- Gaurav Mishra,
- Adam Roberts,
- Paul Barham,
- Hyung Won Chung,
- Charles Sutton,
- Sebastian Gehrmann,
- Parker Schuh,
- Kensen Shi,
- Sashank Tsvyashchenko,
- Joshua Maynez,
- Abhishek Rao,
- Parker Barnes,
- Yi Tay,
- Noam Shazeer,
- Vinodkumar Prabhakaran,
- Emily Reif,
- Nan Du,
- Ben Hutchinson,
- Reiner Pope,
- James Bradbury,
- Jacob Austin,
- Michael Isard,
- Guy Gur-Ari,
- Pengcheng Yin,
- Toju Duke,
- Anselm Levskaya,
- Sanjay Ghemawat,
- Sunipa Dev,
- Henryk Michalewski,
- Xavier Garcia,
- Vedant Misra,
- Kevin Robinson,
- Liam Fedus,
- Denny Zhou,
- Daphne Ippolito,
- David Luan,
- Hyeontaek Lim,
- Barret Zoph,
- Alexander Spiridonov,
- Ryan Sepassi,
- David Dohan,
- Shivani Agrawal,
- Mark Omernick,
- Andrew M. Dai,
- Thanumalayan Sankaranarayana Pillai,
- Marie Pellat,
- Aitor Lewkowycz,
- Erica Moreira,
- Rewon Child,
- Oleksandr Polozov,
- Katherine Lee,
- Zongwei Zhou,
- Xuezhi Wang,
- Brennan Saeta,
- Mark Diaz,
- Orhan Firat,
- Michele Catasta,
- Jason Wei,
- Kathy Meier-Hellstern,
- Douglas Eck,
- Jeff Dean,
- Slav Petrov,
- Noah Fiedel
Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular ...
- research-article
Can large language models reason about program invariants?
ICML'23: Proceedings of the 40th International Conference on Machine Learning•July 2023, Article No.: 1144, pp 27496-27520Identifying invariants is an important program analysis task with applications towards program understanding, bug finding, vulnerability analysis, and formal verification. Existing tools for identifying program invariants rely on dynamic analysis, ...
- research-article
Learning discrete energy-based models via auxiliary-variable local exploration
NIPS '20: Proceedings of the 34th International Conference on Neural Information Processing Systems•December 2020, Article No.: 876, pp 10443-10455Discrete structures play an important role in applications like program language modeling and software engineering. Current approaches to predicting complex structures typically consider autoregressive models for their tractability, with some sacrifice ...
- research-article
Learning to execute programs with instruction pointer attention graph neural networks
NIPS '20: Proceedings of the 34th International Conference on Neural Information Processing Systems•December 2020, Article No.: 723, pp 8626-8637Graph neural networks (GNNs) have emerged as a powerful tool for learning software engineering tasks including code completion, bug finding, and program repair. They benefit from leveraging program structure like control flow graphs, but they are not ...
- poster
Open-vocabulary models for source code
ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings•June 2020, pp 294-295• https://doi.org/10.1145/3377812.3390806Statistical language modeling techniques have successfully been applied to large source code corpora, yielding a variety of new software development tools, such as tools for code suggestion, improving readability, and API migration. A major issue with ...
- research-article
Big code != big vocabulary: open-vocabulary models for source code
ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering•June 2020, pp 1073-1085• https://doi.org/10.1145/3377811.3380342Statistical language modeling techniques have successfully been applied to large source code corpora, yielding a variety of new software development tools, such as tools for code suggestion, improving readability, and API migration. A major issue with ...
- short-paper
How Often Do Single-Statement Bugs Occur?: The ManySStuBs4J Dataset
MSR '20: Proceedings of the 17th International Conference on Mining Software Repositories•June 2020, pp 573-577• https://doi.org/10.1145/3379597.3387491Program repair is an important but difficult software engineering problem. One way to achieve acceptable performance is to focus on classes of simple bugs, such as bugs with single statement fixes, or that match a small set of bug templates. However, it ...
- short-paper
Where should I comment my code?: a dataset and model for predicting locations that need comments
ICSE-NIER '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results•June 2020, pp 21-24• https://doi.org/10.1145/3377816.3381736Programmers should write code comments, but not on every line of code. We have created a machine learning model that suggests locations where a programmer should write a code comment. We trained it on existing commented code to learn locations that are ...
- research-article
Incremental sampling without replacement for sequence models
ICML'20: Proceedings of the 37th International Conference on Machine Learning•July 2020, Article No.: 815, pp 8785-8795Sampling is a fundamental technique, and sampling without replacement is often desirable when duplicate samples are not beneficial. Within machine learning, sampling is useful for generating diverse outputs from a trained model. We present an elegant ...
- research-article
GEMSEC: graph embedding with self clustering
ASONAM '19: Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining•August 2019, pp 65-72• https://doi.org/10.1145/3341161.3342890Modern graph embedding procedures can efficiently process graphs with millions of nodes. In this paper, we propose GEMSEC - a graph embedding algorithm which learns a clustering of the nodes simultaneously with computing their embedding. GEMSEC is a ...
- Article
Learning semantic annotations for tabular data
IJCAI'19: Proceedings of the 28th International Joint Conference on Artificial Intelligence•August 2019, pp 2088-2094The usefulness of tabular data such as web tables critically depends on understanding their semantics. This study focuses on column type prediction for tables without any meta data. Unlike traditional lexical matching-based methods, we propose a deep ...
- research-article
ColNet: embedding the semantics of web tables for column type prediction
AAAI'19/IAAI'19/EAAI'19: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence•January 2019, Article No.: 4, pp 29-36• https://doi.org/10.1609/aaai.v33i01.330129Automatically annotating column types with knowledge base (KB) concepts is a critical task to gain a basic understanding of web tables. Current methods rely on either table metadata like column name or entity correspondences of cells in the KB, and may ...
- research-article
Probabilistic programming with densities in SlicStan: efficient, flexible, and deterministic
Proceedings of the ACM on Programming Languages, Volume 3, Issue POPL•January 2019, Article No.: 35, pp 1-30 • https://doi.org/10.1145/3290348Stan is a probabilistic programming language that has been increasingly used for real-world scalable projects. However, to make practical inference possible, the language sacrifices some of its usability by adopting a block syntax, which lacks ...
- Article
HOUDINI: lifelong learning as program synthesis
NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing Systems•December 2018, pp 8701-8712We present a neurosymbolic framework for the lifelong learning of algorithmic tasks that mix perception and procedural reasoning. Reusing high-level concepts across domains and learning complex procedures are key challenges in lifelong learning. We show ...