research-article

Integration of Multisensorial Stimuli and Multimodal Interaction in a Hybrid 3DTV System

Authors:

Francisco Pedro Luque,

Claudio Feijoo,

Carlos Alberto Martín,

Guillermo CisnerosAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 11, Issue 1s

Article No.: 16, Pages 1 - 22

https://doi.org/10.1145/2617992

Published: 01 October 2014 Publication History

Abstract

This article proposes the integration of multisensorial stimuli and multimodal interaction components into a sports multimedia asset under two dimensions: immersion and interaction. The first dimension comprises a binaural audio system and a set of sensory effects synchronized with the audiovisual content, whereas the second explores interaction through the insertion of interactive 3D objects into the main screen and on-demand presentation of additional information in a second touchscreen. We present an end-to-end solution integrating these components into a hybrid (internet-broadcast) television system using current 3DTV standards. Results from an experimental study analyzing the perceived quality of these stimuli and their influence on the Quality of Experience are presented.

Supplementary Material

a16-luque-apndx.pdf (luque.zip)

Supplemental movie, appendix, image and software files for, Integration of Multisensorial Stimuli and Multimodal Interaction in a Hybrid 3DTV System

Download
47.40 KB

References

[1]

A. T. Beck. 1961. An inventory for measuring depression. Archives Gen. Psychiatry 4, 6, 561.

[2]

C. Bracken, G. Pettey, and M. Wu. 2011. Telepresence and attention: Secondary task reaction time and media form. In Proceedings of the International Society for Presence Research Annual Conference.

[3]

J. Cha, M. Eid, and A. E. Saddik. 2009. Touchable 3D video system. ACM Trans. Multimedia Comput. Commun. Appl. 5, 4, 1--25.

Digital Library

[4]

M. Cobos, J. J. Lopez, J. M. Navarro, and G. Ramos. 2013. Subjective quality assessment of multichannel audio accompanied with video in representative broadcasting genres. Multimedia Syst.

Digital Library

[5]

B. De Ruyter and E. Aarts. 2004. Ambient intelligence. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI'04). ACM Press, New York, New York, 203.

Digital Library

[6]

ETSI. 2006. Digital video broadcasting (DVB); Multimedia home platform (MHP). European Telecomunications Standards Institute (ETSI) Technical Specification TS 101 812 v1.0.2.

[7]

ETSI. 2010a. Digital video broadcasting (DVB); DVB-HN (home network) reference model phase 1. Technical Specification TS 102 905 Ver. 1.1.1. European Telecomunications Standards Institute (ETSI).

[8]

ETSI. 2010b. Digital video broadcasting (DVB); Signalling and carriage of interactive applications and services in hybrid broadcast/broadband environments. Technical Specification TS 102 809 v1.1.1. European Telecomunications Standards Institute (ETSI).

[9]

ETSI. 2012a. Digital video broadcasting (DVB); Plano-stereoscopic 3DTV; Part 2: Frame compatible plano-stereoscopic 3DTV. Technical Specification TS 101 547-2 V1.2.1. European Telecomunications Standards Institute (ETSI).

[10]

ETSI. 2012b. HbbTV (hybrid broadcast broadband TV) technical specification. Technical Specification TS 102 796 v1.2.1. European Telecomunications Standards Institute (ETSI).

[11]

I. Galloso, F. P. Luque, L. Piovano, D. Garrido, E. Sánchez, and C. A. Feijoo. 2012. Foundations of a new interaction paradigm for immersive 3D multimedia. In Proceedings of the NEM Summit: Implementing Future Media Internet towards New Horizons.

[12]

G. Ghinea and O. Ademoye. 2012. The sweet smell of success: Enhancing multimedia applications with olfaction. ACM Trans. Multimedia Comput. Commun. Appl. 8, 1, 1--17.

Digital Library

[13]

E. B. Goldstein. 2010. Sensation and Perception. Cengage Learning.

[14]

D. W. Heightman. 1975. Digital video broadcasting (DVB); DVB-HN (home network) reference model phase 1. European Telecomunications Standards Institute (ETSI) Technical Specification TS 102 905 Ver. 1.1.1. Radio Electron. Eng. 45, 10, 559--569.

[15]

H. Hirschmüller. 2008. Stereo processing by semiglobal matching and mutual information. IEEE Trans. Patt. Anal. Mach. Intell. 30, 2, 328--41.

Digital Library

[16]

INSPIRA. 2006. Informe sobre el uso de la televisión digital interactiva por parte del público español. Coordinated by G&commat;TV-UPM. Sponsored by Spanish Ministry of Industry, Tourism and Trade.

[17]

ISO/IEC. 1998. Information technology—Coding of multimedia and hypermedia information—Part 6: Support for enhanced interactive applications. International Standard (ISO/IEC 13522-6). International Organization for Standardization.

[18]

ISO/IEC. 2000. Generic coding of moving pictures and associated audio information: Systems. International Standard (ISO/IEC 13818-1). International Organization for Standardization.

[19]

ISO/IEC. 2007. Home electronic system (HES) architecture Part 3--5. International Standard (ISO/IEC 14543-3). International Organization for Standardization.

[20]

ISO/IEC. 2013. Information technology—Media context and control—Part 3: Sensory Information. International Standard (ISO/IEC 23005-3). Organization for Standardization.

[21]

ITU-R. 2000. Subjective assessment of stereoscopic television pictures. International Telecommunication Union.

[22]

ITU-R. 2002. 500-11. Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union.

[23]

ITU-R. 2012. Methodology for the subjective assessment of the quality of television pictures BT Series Broadcasting service. International Telecommunication Union.

[24]

ITU-T. 1998. P.911: Subjective audiovisual quality assessment methods for multimedia applications. International Telecommunication Union.

[25]

B. R. Jones, H. Benko, E. Ofek, and A. D. Wilson. 2013. IllumiRoom. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'13). ACM Press, New York, New York, 869.

Digital Library

[26]

S. Jumisko-Pyykkö. 2011. User-centered quality of experience and its evaluation methods for mobile television. Master's Thesis, Tampere University of Technology.

[27]

J. Kowalczuk, E. Psota, and L. Perez. 2012. Real-time stereo matching on CUDA using an iterative refinement method for adaptive support-weight correspondences. IEEE Trans. Circuits Syst. Video Technol. 23, 1, 94--104.

Digital Library

[28]

P. Le Callet, S. Möller, and A. Perkis, eds. 2012. Qualinet white paper on definitions of quality of experience Qualinet.

[29]

P. Mills, A. Sheikh, G. Thomas, and P. Debenham. 2011. BBC R&D white paper WHP208. BBC.

[30]

N. Murray, Y. Qiao, B. Lee, A. K. Karunakar, and G.-M. Muntean. 2013. Subjective evaluation of olfactory and visual media synchronization. In Proceedings of the 4th ACM Multimedia Systems Conference (MMSys'13). 162--171.

Digital Library

[31]

R. L. Nabi and M. Krcmar. 2004. Conceptualizing media enjoyment as attitude: Implications for mass media effects research. Commun. Theory 14, 4, 288--310.

[32]

D. R. Olsen, D. Bunn, T. Boulter, and R. Walz. 2012. Interactive television news. ACM Trans. Multimedia Comput. Commun. Appl. 8, 2, 1--20.

Digital Library

[33]

G. Ramos and M. Cobos. 2013. Parametric head-related transfer function modeling and interpolation for cost-efficient binaural sound applications. J. Acoustical Soci. Am. 134, 3, 1735--8.

[34]

A. A. Raney and J. Bryant. 2002. Moral judgment and crime drama: An integrated theory of enjoyment. J. Commun. 52, 2, 402--415.

[35]

M. J. Schuemie, P. van der Straaten, M. Krijn, and C. van der Mast. 2001. Research on presence in virtual reality: A survey. Cyberpsychol. Behav. Impact Internet Multimedia Virtual Reality Behav. Soci. 4, 2, 183--201.

[36]

H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson. 2003. RTP: A transport protocol for real-time applications. Internet Engineering Task Force, RFC 3550.

Digital Library

[37]

M. Slater and S. Wilbur. 1997. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence-Teleoperators Virtual Environ. 6, 6, 603--616.

Digital Library

[38]

M. D. Slater. 2003. Alienation, aggression, and sensation seeking as predictors of adolescent use of violent film, computer, and website content. J. Commun. 53, 1, 105--121.

[39]

C. Timmerer, M. Waltl, B. Rainer, and H. Hellwagner. 2012. Assessing the quality of sensory experience for multimedia presentations. Signal Process. Image Commun. 27, 8, 909--916.

Digital Library

[40]

M. Waltl, B. Rainer, C. Timmerer, and H. Hellwagner. 2013. An end-to-end tool chain for sensory experience based on MPEG-V. Signal Process. Image Commun. 28, 2, 136--150.

Digital Library

[41]

M. Waltl, C. Timmerer, and H. Hellwagner. 2009. A test-bed for quality of multimedia experience evaluation of sensory effects. In Proceedings of the International Workshop on Quality of Multimedia Experience. IEEE, 145--150.

[42]

M. Waltl, C. Timmerer, and H. Hellwagner. 2010. Increasing the user experience of multimedia presentations with sensory effects. In Proceedings of the 11th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS). IEEE, 1--4.

[43]

I. Wechsung, M. Schulz, K.-P. Engelbrecht, J. Niemann, and S. Möller. 2011. All users are (not) equal - The influence of user characteristics on perceived quality, modality choice and performance. In Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. Springer, New York, NY, 175--186.

[44]

A. Weffers-Albu, S. de Waele, W. Hoogenstraaten, and C. Kwisthout. 2011. Immersive TV viewing with advanced Ambilight. In Proceedings of the IEEE International Conference on Consumer Electronics (ICCE). 753--754.

[45]

D. Winer. 1999. XML remote procedure call (XML-RPC) specification. XML-RPC.

[46]

Z. Yang, W. Wu, K. Nahrstedt, G. Kurillo, and R. Bajcsy. 2010. Enabling multi-party 3D tele-immersive environments with ViewCast. ACM Trans. Multimedia Comput. Commun. Appl. 6, 2, 1--30.

Digital Library

[47]

K. Yoon. 2013. End-to-end framework for 4-D broadcasting based on MPEG-V standard. Signal Process. Image Commun. 28, 2, 127--135.

Digital Library

[48]

D. Zillmann and P. Vorderer. 2000. Media Entertainment: The Psychology of Its Appeal. Lawrence Erlbaum Associates Publishers.

Cited By

Barreto Fde Abreu RJosué MMontevecchi EValentim PMuchaluat-Saade D(2022)Providing multimodal and multi-user interactions for digital tv applicationsMultimedia Tools and Applications10.1007/s11042-021-11847-382:4(4821-4846)Online publication date: 18-Jul-2022
https://dl.acm.org/doi/10.1007/s11042-021-11847-3
Souza GSilva DDelgado MRodrigues RMendes PAmorim GGuedes ASantos Jde Salles Soares Neto C(2020)Interactive 360-degree Videos in Ginga-NCL Using Head-Mounted-Displays as Second Screen DevicesProceedings of the Brazilian Symposium on Multimedia and the Web10.1145/3428658.3430972(289-296)Online publication date: 30-Nov-2020
https://dl.acm.org/doi/10.1145/3428658.3430972
Elsharif AMousa MIssa A(2020)Mulsemedia: Review and Perspective on Potentials in Rehabilitation and Disability Technological Developments2020 International Conference on Assistive and Rehabilitation Technologies (iCareTech)10.1109/iCareTech49914.2020.00026(97-102)Online publication date: Aug-2020
https://doi.org/10.1109/iCareTech49914.2020.00026
Show More Cited By

Index Terms

Integration of Multisensorial Stimuli and Multimodal Interaction in a Hybrid 3DTV System

Recommendations

Reality Tales: Facilitating User-Character Interaction with Immersive Storytelling
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

Reality Tales is a platform to facilitate interaction with fictional story characters for readers (13-20 years). The platform leverages binaural audio and multi-voice narration for an immersive story experience. Several paradigms for interactive ...
Mobile phones as second screen for TV, enabling inter-audience interaction
ACE '11: Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology

Despite the ever expanding forms of digital entertainment, there are still TV shows and events that create an audience desire to be part of a mass shared experience. In the past direct inter-audience interaction of such events has been restricted to ...
Using Interaction Data to Predict Engagement with Interactive Media
MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Media is evolving from traditional linear narratives to personalised experiences, where control over information (or how it is presented) is given to individual audience members. Measuring and understanding audience engagement with this media is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 11, Issue 1s

Special Issue on Multiple Sensorial (MulSeMedia) Multimodal Media : Advances and Applications

September 2014

260 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/2675060

Editors:
Ralf Steinmetz
Technische Universität Darmstadt, Germany
,
Gheorghita Ghinea,
Christian Timmerer,
Weisi Lin,
Stephen Gulliver,
Zheng-Jun Zha
Institute of Intelligent Machines, Chinese Academy of Sciences
,
Lei Zhang
Bing Multimedia Search, Microsoft Corporation
,
Max Mühlhäuser
Department of Computer Science, Technische Universität Darmstadt
,
Alan F. Smeaton
Insight Centre for Data Analytics, Dublin City University

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2014

Accepted: 01 May 2014

Revised: 01 March 2014

Received: 01 November 2013

Published in TOMM Volume 11, Issue 1s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Spanish Ministry of Industry Tourism and Commerce

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
274
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)2

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Barreto Fde Abreu RJosué MMontevecchi EValentim PMuchaluat-Saade D(2022)Providing multimodal and multi-user interactions for digital tv applicationsMultimedia Tools and Applications10.1007/s11042-021-11847-382:4(4821-4846)Online publication date: 18-Jul-2022
https://dl.acm.org/doi/10.1007/s11042-021-11847-3
Souza GSilva DDelgado MRodrigues RMendes PAmorim GGuedes ASantos Jde Salles Soares Neto C(2020)Interactive 360-degree Videos in Ginga-NCL Using Head-Mounted-Displays as Second Screen DevicesProceedings of the Brazilian Symposium on Multimedia and the Web10.1145/3428658.3430972(289-296)Online publication date: 30-Nov-2020
https://dl.acm.org/doi/10.1145/3428658.3430972
Elsharif AMousa MIssa A(2020)Mulsemedia: Review and Perspective on Potentials in Rehabilitation and Disability Technological Developments2020 International Conference on Assistive and Rehabilitation Technologies (iCareTech)10.1109/iCareTech49914.2020.00026(97-102)Online publication date: Aug-2020
https://doi.org/10.1109/iCareTech49914.2020.00026
Saleme ECovaci AMesfin GSantos CGhinea G(2019)Mulsemedia DIY: A Survey of Devices and a Tutorial for Building Your Own Mulsemedia EnvironmentACM Computing Surveys10.1145/331985352:3(1-29)Online publication date: 18-Jun-2019
https://dl.acm.org/doi/10.1145/3319853
Saleme ESantos CGhinea G(2019)A mulsemedia framework for delivering sensory effects to heterogeneous systemsMultimedia Systems10.1007/s00530-019-00618-825:4(421-447)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1007/s00530-019-00618-8
Saleme ESantos CFalbo RGhinea GAndres FChbeir RIshikawa HSumiya KHatano KKoeppen M(2018)Towards a reference ontology on mulsemedia systemsProceedings of the 10th International Conference on Management of Digital EcoSystems10.1145/3281375.3281378(23-30)Online publication date: 25-Sep-2018
https://dl.acm.org/doi/10.1145/3281375.3281378
Galloso IPalacios JFeijóo CSantamaría A(2016)On the influence of individual characteristics and personality traits on the user experience with multi-sensorial mediaMultimedia Tools and Applications10.1007/s11042-016-3360-z75:20(12365-12408)Online publication date: 1-Oct-2016
https://dl.acm.org/doi/10.1007/s11042-016-3360-z
Galloso IFeijóo CSantamaría A(2014)Novel Approaches to Immersive Media: From Enlarged Field-of-View to Multi-sensorial ExperiencesNovel 3D Media Technologies10.1007/978-1-4939-2026-6_2(9-24)Online publication date: 28-Oct-2014
https://doi.org/10.1007/978-1-4939-2026-6_2

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents