Anchor Attention for Hybrid Crowd Forecasts Aggregation
Authors: 
Yuzhong Huang, Andrés Abeliuk, 
Fred Morstatter, Pavel Atanasov, Aram GalstyanAuthors Info & Claims
Pages 1869 - 1871
Abstract
Forecasting the future is a notoriously difficult task. One way to address this challenge is to "hybridize" the forecasting process, combining forecasts from a crowd of humans, as well as one or more machine models. However, an open challenge remains in how to optimally aggregate inputs from these pools into a single forecast. We proposed anchor attention for this type of sequence summary problem. Each forecast is represented by a trainable embedding vector. An anchor attention score is used to determine input weights. We evaluate our approach using data from a real-world forecasting tournament, and show that our method outperforms the current state-of-the-art aggregation approaches.
References
[1]
Pavel Atanasov, Phillip Rescober, Eric Stone, Samuel A Swift, Emile Servan-Schreiber, Philip Tetlock, Lyle Ungar, and Barbara Mellers. 2017. Distilling the wisdom of crowds: Prediction markets vs. prediction polls. Management science, Vol. 63, 3 (2017), 691--706.
[2]
Rob Hyndman and Yeasmin Khandakar. 2008. Automatic Time Series Forecasting: The forecast Package for R. Journal of Statistical Software, Articles, Vol. 27, 3 (2008), 1--22. https://doi.org/10.18637/jss.v027.i03
[3]
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. 2019. The M4 Competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting (2019). https://doi.org/10.1016/j.ijforecast.2019.04.014
[4]
Daniel Marbach, James C Costello, Robert Küffner, Nicole M Vega, Robert J Prill, Diogo M Camacho, Kyle R Allison, Andrej Aderhold, Richard Bonneau, Yukun Chen, et almbox. 2012. Wisdom of crowds for robust gene network inference. Nature methods, Vol. 9, 8 (2012), 796.
[5]
Fred Morstatter, Aram Galstyan, Gleb Satyukov, Daniel Benjamin, Andres Abeliuk, Mehrnoosh Mirtaheri, KSM Tozammel Hossain, Pedro Szekely, Emilio Ferrara, Akira Matsui, Mark Steyvers, Stephen Bennet, David Budescu, Mark Himmelstein, Michael Ward, Andreas Beger, Michele Catasta, Rok Sosic, Jure Leskovec, Pavel Atanasov, Regina Joseph, Rajiv Sethi, and Ali Abbas. 2019. SAGE: A Hybrid Geopolitical Event Forecasting System. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 6557--6559. https://doi.org/10.24963/ijcai.2019/955
[6]
Ville A Satopää, Jonathan Baron, Dean P Foster, Barbara A Mellers, Philip E Tetlock, and Lyle H Ungar. 2014. Combining multiple probability predictions using a simple logit model. International Journal of Forecasting, Vol. 30, 2 (2014), 344--356.
[7]
James Surowiecki. 2004. The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business. Economies, Societies and Nations, Vol. 296 (2004).
[8]
Philip E Tetlock. 2017. Expert political judgment: How good is it? How can we know? Princeton University Press.
[9]
Lyle Ungar, Barbara Mellers, Ville Satopää, Philip Tetlock, and Jon Baron. 2012. The good judgment project: A large scale test of different methods of combining expert predictions. In 2012 AAAI Fall Symposium Series .
[10]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Lawrence Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. In NIPS.
Index Terms
- Anchor Attention for Hybrid Crowd Forecasts Aggregation
Recommendations
Beyond Sum and Weighted Aggregation: An Efficient Mixed Aggregation Method with Multiple Weights for Image Search
Thematic Workshops '17: Proceedings of the on Thematic Workshops of ACM Multimedia 2017Image search with local descriptors represents an image usually by embedding and aggregating a set of patch descriptors into a single vector. Standard aggregation operations include sum and weighted aggregations. While showing high efficiency, sum ...
The Bayesian method of moments (BMOM) in some aggregation problems in econometrics: Research Articles
Bayesian Models in Business and IndustryThe BMOM is particularly useful for obtaining post-data moments and densities for parameters and future observations when the form of the likelihood function is unknown and thus a traditional Bayesian approach cannot be used. Also, even when the form of ...
The Accuracy of Combining Judgemental and Statistical Forecasts
Judgement based forecasts are widely used in practice either alone or in conjunction with computer prepared forecasts. This study empirically examines the improvement in accuracy which can be gained from combining judgemental forecasts, either with ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2020
2289 pages
ISBN:9781450375184
- General Chairs:
- Amal El Fallah Seghrouchni,
- Gita Sukthankar,
- Program Chairs:
- Bo An,
- Neil Yorke-Smith Yorke-Smith
Sponsors
Publisher
International Foundation for Autonomous Agents and Multiagent Systems
Richland, SC
Publication History
Published: 13 May 2020
Check for updates
Author Tags
Qualifiers
- Extended-abstract
Funding Sources
- Intelligence Advanced Research Projects Activity (IARPA)
Conference
AAMAS '19
Sponsor:
AAMAS '19: International Conference on Autonomous Agents and Multiagent Systems
May 9 - 13, 2020
Auckland, New Zealand
Acceptance Rates
Overall Acceptance Rate 1,155 of 5,036 submissions, 23%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 30Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in