short-paper

Public Access

Towards Confidence-aware Calibrated Recommendation

Authors:

Mohammadmehdi Naghiaei,

Hossein A. Rahmani,

Mohammad Aliannejadi,

Nasim SonboliAuthors Info & Claims

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Pages 4344 - 4348

https://doi.org/10.1145/3511808.3557713

Published: 17 October 2022 Publication History

Abstract

Recommender systems utilize users' historical data to learn and predict their future interests, providing them with suggestions tailored to their tastes. Calibration ensures that the distribution of recommended item categories is consistent with the user's historical data. Mitigating miscalibration brings various benefits to a recommender system. For example, it becomes less likely that a system overlooks categories with less interaction on a user's profile by only recommending popular categories. Despite the notable success, calibration methods have several drawbacks, such as limiting the diversity of the recommended items and not considering the calibration confidence. This work, presents a set of properties that address various aspects of a desired calibrated recommender system. Considering these properties, we propose a confidence-aware optimization-based re-ranking algorithm to find the balance between calibration, relevance, and item diversity, while simultaneously accounting for calibration confidence based on user profile size. Our model outperforms state-of-the-art methods in terms of various accuracy and beyond-accuracy metrics for different user groups.

Supplementary Material

MP4 File (CIKM22-sp1313.mp4)

Presentation video - This work, presents a set of properties that address various aspects of a desired calibrated recommender system. Considering these properties, we propose a confidence-aware optimization-based re-ranking algorithm to find the balance between calibration, relevance, and item diversity, while simultaneously accounting for calibration confidence based on user profile size.

Download
27.76 MB

References

[1]

Himan Abdollahpouri, Masoud Mansoury, Robin Burke, and Bamshad Mobasher. 2020. The connection between popularity bias, calibration, and fairness in recommendation. In Fourteenth ACM conference on recommender systems. 726--731.

Digital Library

[2]

Antonio Bella, Cèsar Ferri, José Hernández-Orallo, and Mar'ia José Ram'irez-Quintana. 2010. Calibration of machine learning models. In Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques. IGI Global, 128--146.

[3]

Aditya Bhaskara, Mehrdad Ghadiri, Vahab Mirrokni, and Ola Svensson. 2016. Linear relaxations for finding diverse elements in metric spaces. Advances in neural information processing systems, Vol. 29 (2016).

[4]

Antonin Chambolle. 2004. An algorithm for total variation minimization and applications. Journal of Mathematical imaging and vision, Vol. 20, 1 (2004), 89--97.

Digital Library

[5]

Van Dang and W Bruce Croft. 2012. Diversity by proportionality: an election-based approach to search result diversification. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. 65--74.

Digital Library

[6]

Farzad Eskandanian, Bamshad Mobasher, and Robin Burke. 2017. A Clustering Approach for Personalizing Diversity in Collaborative Recommender Systems. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (Bratislava, Slovakia) (UMAP '17). Association for Computing Machinery, New York, NY, USA, 280--284. https://doi.org/10.1145/3079628.3079699

Digital Library

[7]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web. 173--182.

Digital Library

[8]

Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE international conference on data mining. Ieee, 263--272.

Digital Library

[9]

Neil Hurley and Mi Zhang. 2011. Novelty and diversity in top-n recommendation--analysis and evaluation. ACM Transactions on Internet Technology (TOIT), Vol. 10, 4 (2011), 1--30.

Digital Library

[10]

David A Levin and Yuval Peres. 2017. Markov chains and mixing times. Vol. 107. American Mathematical Soc.

[11]

Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 world wide web conference. 689--698.

Digital Library

[12]

Kun Lin, Nasim Sonboli, Bamshad Mobasher, and Robin Burke. 2020. Calibration in Collaborative Filtering Recommender Systems: A User-Centered Analysis. In Proceedings of the 31st ACM Conference on Hypertext and Social Media (Virtual Event, USA) (HT '20). Association for Computing Machinery, New York, NY, USA, 197--206. https://doi.org/10.1145/3372923.3404793

Digital Library

[13]

Jinoh Oh, Sun Park, Hwanjo Yu, Min Song, and Seung-Taek Park. 2011. Novel recommendation based on personal popularity tendency. In 2011 IEEE 11th International Conference on Data Mining. IEEE, 507--516.

Digital Library

[14]

Eli Pariser. 2011. The filter bubble: What the Internet is hiding from you. Penguin UK.

Digital Library

[15]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012).

[16]

Aghiles Salah, Quoc-Tuan Truong, and Hady W Lauw. 2020. Cornac: A Comparative Framework for Multimodal Recommender Systems. Journal of Machine Learning Research, Vol. 21, 95 (2020), 1--5.

[17]

Sinan Seymen, Himan Abdollahpouri, and Edward C Malthouse. 2021. A Constrained Optimization Approach for Calibrated Recommendations. In Fifteenth ACM Conference on Recommender Systems. 607--612.

Digital Library

[18]

Guy Shani and Asela Gunawardana. 2011. Evaluating recommendation systems. In Recommender systems handbook. Springer, 257--297.

[19]

Nasim Sonboli, Farzad Eskandanian, Robin Burke, Weiwen Liu, and Bamshad Mobasher. 2020. Opportunistic Multi-Aspect Fairness through Personalized Re-Ranking. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization (Genoa, Italy) (UMAP '20). Association for Computing Machinery, New York, NY, USA, 239--247. https://doi.org/10.1145/3340631.3394846

Digital Library

[20]

Harald Steck. 2018. Calibrated recommendations. In Proceedings of the 12th ACM conference on recommender systems. 154--162.

Digital Library

[21]

Quoc-Tuan Truong, Aghiles Salah, Thanh-Binh Tran, Jingyao Guo, and Hady W Lauw. 2021. Exploring Cross-Modality Utilization in Recommender Systems. IEEE Internet Computing (2021).

[22]

Saúl Vargas, Linas Baltrunas, Alexandros Karatzoglou, and Pablo Castells. 2014. Coverage, redundancy and size-awareness in genre diversity for recommender systems. In Proceedings of the 8th ACM Conference on Recommender systems. 209--216.

Digital Library

[23]

Yining Wang, Liwei Wang, Yuanzhi Li, Di He, and Tie-Yan Liu. 2013. A theoretical analysis of NDCG type ranking measures. In Conference on learning theory. PMLR, 25--54.

[24]

Mi Zhang and Neil Hurley. 2008. Avoiding monotony: improving the diversity of recommendation lists. In Proceedings of the 2008 ACM conference on Recommender systems. 123--130. endthebibl

Digital Library

Cited By

Naghiaei MDehghan MRahmani HAzizi JAliannejadi MOosterhuis HBast HXiong C(2024)Personalized Beyond-accuracy Calibration in RecommendationProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672507(107-116)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672507
Rahmani HNaghiaei MDeldjoo Y(2024)A Personalized Framework for Consumer and Producer Group Fairness Optimization in Recommender SystemsACM Transactions on Recommender Systems10.1145/36511672:3(1-24)Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1145/3651167
Lin KMansoury MEskandanian FSabouri MMobasher BBoratto LGena CMarras MGermanakos PPopescus E(2024)Beyond Static Calibration: The Impact of User Preference Dynamics on Calibrated RecommendationAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3664869(86-91)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3631700.3664869
Show More Cited By

Index Terms

Towards Confidence-aware Calibrated Recommendation
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Calibrated Recommendations as a Minimum-Cost Flow Problem
WSDM '23: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining

Calibration in recommender systems has recently gained significant attention. In the recommended list of items, calibration ensures that the various (past) areas of interest of a user are reflected with their corresponding proportions. For instance, if a ...
Calibrated recommendations
RecSys '18: Proceedings of the 12th ACM Conference on Recommender Systems

When a user has watched, say, 70 romance movies and 30 action movies, then it is reasonable to expect the personalized list of recommended movies to be comprised of about 70% romance and 30% action movies as well. This important property is known as ...
A trust-aware recommendation method based on Pareto dominance and confidence concepts

We proposed a trust-based collaborative filtering method called CPD.Implicit trust statements are identified and used in the recommendation process.A reliability measure is used to evaluate quality of implicit trust statements.Most prominent users are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

October 2022

5274 pages

ISBN:9781450392365

DOI:10.1145/3511808

General Chairs:
Mohammad Al Hasan
Indiana University Purdue University, Indianapolis, USA
,
Li Xiong
Emory University, Atlanta, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

National Science Foundation

Conference

CIKM '22

Sponsor:

CIKM '22: The 31st ACM International Conference on Information and Knowledge Management

October 17 - 21, 2022

GA, Atlanta, USA

Acceptance Rates

CIKM '22 Paper Acceptance Rate 621 of 2,257 submissions, 28%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
209
Total Downloads

Downloads (Last 12 months)124
Downloads (Last 6 weeks)24

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Naghiaei MDehghan MRahmani HAzizi JAliannejadi MOosterhuis HBast HXiong C(2024)Personalized Beyond-accuracy Calibration in RecommendationProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672507(107-116)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672507
Rahmani HNaghiaei MDeldjoo Y(2024)A Personalized Framework for Consumer and Producer Group Fairness Optimization in Recommender SystemsACM Transactions on Recommender Systems10.1145/36511672:3(1-24)Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1145/3651167
Lin KMansoury MEskandanian FSabouri MMobasher BBoratto LGena CMarras MGermanakos PPopescus E(2024)Beyond Static Calibration: The Impact of User Preference Dynamics on Calibrated RecommendationAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3664869(86-91)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3631700.3664869
Kleinberg JRyu ETardos É(2024)Calibrated Recommendations for Users with Decaying AttentionAlgorithmic Game Theory10.1007/978-3-031-71033-9_25(443-460)Online publication date: 31-Aug-2024
https://doi.org/10.1007/978-3-031-71033-9_25
Wang CLiu YYu YMa WZhang MLiu YZeng HFeng JDeng C(2023)Two-sided Calibration for Quality-aware Responsible RecommendationProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608799(223-233)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608799
da Silva DDurão F(2023)Introducing a framework and a decision protocol to calibrated recommender systemsApplied Intelligence10.1007/s10489-023-04681-753:19(22044-22072)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1007/s10489-023-04681-7

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents