Article

Gaussian process classification for segmenting and annotating sequences

Authors:

Thomas Hofmann,

Alexander J. SmolaAuthors Info & Claims

ICML '04: Proceedings of the twenty-first international conference on Machine learning

Page 4

https://doi.org/10.1145/1015330.1015433

Published: 04 July 2004 Publication History

Abstract

Many real-world classification tasks involve the prediction of multiple, inter-dependent class labels. A prototypical case of this sort deals with prediction of a sequence of labels for a sequence of observations. Such problems arise naturally in the context of annotating and segmenting observation sequences. This paper generalizes Gaussian Process classification to predict multiple labels by taking dependencies between neighboring labels into account. Our approach is motivated by the desire to retain rigorous probabilistic semantics, while overcoming limitations of parametric methods like Conditional Random Fields, which exhibit conceptual and computational difficulties in high-dimensional input spaces. Experiments on named entity recognition and pitch accent prediction tasks demonstrate the competitiveness of our approach.

References

[1]

Altun, Y., Hofmann, T., & Johnson, M. (2003a). Discriminative learning for label sequences via boosting. Advances in Neural Information Processing Systems.

[2]

Altun, Y., Tsochantaridis, I., & Hofmann, T. (2003b). Hidden markov support vector machines. 20th International Conference on Machine Learning.

[3]

Bennett, K., Momma, M., & Embrechts, J. (2002). Mark: A boosting algorithm for heterogeneous kernel models. Proceedings of SIGKDD International Conference on Knowledge Discovery and Data Mining.

Digital Library

[4]

Collins, M. (2002). Discriminative training methods for Hidden Markov Models: Theory and experiments with perceptron algorithms. Empirical Methods of Natural Language Processing (EMNLP).

Digital Library

[5]

Crammer, K., & Singer, Y. (2001). On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2.

Digital Library

[6]

Csat'o, L., & Opper, M. (2002). Sparse on-line Gaussian Processes. Neural Computation, 14, 641--668.

Digital Library

[7]

Gibbs, M. N., & MacKay, D. J. C. (2000). Variational Gaussian Process Classifiers. IEEE-NN, 11, 1458.

Digital Library

[8]

Girosi, F. (1997). An equivalence between sparse approximation and support vector machines (Technical Report AIM-1606).

Digital Library

[9]

Greenberg, S., Ellis, D., & Hollenback, J. (1996). Insights into spoken language gleaned from phonetic transcripti on of the Switchboard corpus. ICSLP96.

[10]

Jaakkola, T. S., & Jordan, M. I. (1996). Computing upper and lower bounds on likelihoods in intractable networks. In Proc. of the Twelfth Conf. on UAI.

Digital Library

[11]

Kimeldorf, G., & Wahba, G. (1971). A correspondence between Bayesian estimation and on stochastic processes and smoothing by splines. Annals of Math. Stat., 41(2), 495--502.

[12]

Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional Random Fields: Probabilistic models for segmenting and labeling sequence data. Proc. 18th International Conf. on Machine Learning.

Digital Library

[13]

McCallum, A., Freitag, D., & Pereira, F. (2000). Maximum Entropy Markov Models for Information Extraction and Segmentation. Machine Learning: Proceedings of the Seventeenth International Conference (ICML 2000).

Digital Library

[14]

Minka, T. (2001). A family of algorithms for approximate Bayesian inference. PhD thesis, MIT Media Lab.

Digital Library

[15]

Opper, M., & Winther, O. (2000). Gaussian Processes for classification: Mean-field algorithms. Neural Computation, 12, 2655--2684.

Digital Library

[16]

Punyakanok, V., & Roth, D. (2000). The use of classifiers in sequential inference. Advances in Neural Information Processing Systems.

[17]

Schölkopf, B., & Smola, A. J. (2002). Learning with kernels. MIT Press.

[18]

Seeger, M., Lawrence, N. D., & Herbrich, R. (2003). Fast sparse Gaussian Process methods: The informative vector machine. Advances in Neural Information Processing Systems.

[19]

Smola, A. J., & Bartlett, P. L. (2000). Sparse greedy Gaussian Process regression. Advances in Neural Information Processing Systems.

[20]

Smola, A. J., & Schöölkopf, B. (2000). Sparse greedy matrix approximation for machine learning. Proc. 17th International Conf. on Machine Learning.

Digital Library

[21]

Taskar, B., Guestrin, C., & Koller, D. (2004). Max-margin markov networks. Advances in Neural Information Processing Systems.

[22]

Weston, J., & Watkins, C. (1999). Support vector machines for multi-class pattern recognition. Proceedings European Symposium on Artificial Neural Networks.

[23]

Williams, C. K. I., & Barber, D. (1998). Bayesian classification with Gaussian Processes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 1342--1351.

Digital Library

[24]

Williams, C. K. I., & Seeger, M. (2000). Using the nystrom method to speed up kernel machines. Advances in Neural Information Processing Systems.

[25]

Zhu, & Hastie, T. (2001). Kernel logistic regression and the import vector machine. Advances in Neural Information Processing Systems.

Cited By

Lu XChow T(2024)Partial Sequence Labeling With Structured Gaussian ProcessesIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.319172635:2(2783-2792)Online publication date: Feb-2024
https://doi.org/10.1109/TNNLS.2022.3191726
Raghavan AJohansson K(2023)Distributed Regression by Two Agents from Noisy Data2023 European Control Conference (ECC)10.23919/ECC57647.2023.10178232(1-6)Online publication date: 13-Jun-2023
https://doi.org/10.23919/ECC57647.2023.10178232
Nguyen TPhung SBouzerdoum A(2020)Hybrid Deep Learning-Gaussian Process Network for Pedestrian Lane Detection in Unstructured ScenesIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2020.296624631:12(5324-5338)Online publication date: Dec-2020
https://doi.org/10.1109/TNNLS.2020.2966246
Show More Cited By

Gaussian process classification for segmenting and annotating sequences
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches

Recommendations

Discriminative Gaussian process latent variable model for classification
ICML '07: Proceedings of the 24th international conference on Machine learning

Supervised learning is difficult with high dimensional input spaces and very small training sets, but accurate classification may be possible if the data lie on a low-dimensional manifold. Gaussian Process Latent Variable Models can discover low ...
Bayesian multi-instance multi-label learning using Gaussian process prior

Multi-instance multi-label learning (MIML) is a newly proposed framework, in which the multi-label problems are investigated by representing each sample with multiple feature vectors named instances. In this framework, the multi-label learning task ...
Annotating abstract anaphora

In this paper, we present first results from annotating abstract (discourse-deictic) anaphora in German. Our annotation guidelines provide linguistic tests for identifying the antecedent, and for determining the semantic types of both the antecedent and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '04: Proceedings of the twenty-first international conference on Machine learning

July 2004

934 pages

ISBN:1581138385

DOI:10.1145/1015330

Conference Chair:
Carla Brodley
Purdue University/Tufts University

Copyright © 2004 Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 July 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
593
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lu XChow T(2024)Partial Sequence Labeling With Structured Gaussian ProcessesIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.319172635:2(2783-2792)Online publication date: Feb-2024
https://doi.org/10.1109/TNNLS.2022.3191726
Raghavan AJohansson K(2023)Distributed Regression by Two Agents from Noisy Data2023 European Control Conference (ECC)10.23919/ECC57647.2023.10178232(1-6)Online publication date: 13-Jun-2023
https://doi.org/10.23919/ECC57647.2023.10178232
Nguyen TPhung SBouzerdoum A(2020)Hybrid Deep Learning-Gaussian Process Network for Pedestrian Lane Detection in Unstructured ScenesIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2020.296624631:12(5324-5338)Online publication date: Dec-2020
https://doi.org/10.1109/TNNLS.2020.2966246
Udendhran RBalamurugan M(2020)RETRACTED ARTICLE: Towards secure deep learning architecture for smart farming-based applicationsComplex & Intelligent Systems10.1007/s40747-020-00225-57:2(659-666)Online publication date: 10-Nov-2020
https://doi.org/10.1007/s40747-020-00225-5
Cohen S(2019)Bayesian Analysis in Natural Language Processing, Second EditionSynthesis Lectures on Human Language Technologies10.2200/S00905ED2V01Y201903HLT04112:1(1-343)Online publication date: 8-Apr-2019
https://doi.org/10.2200/S00905ED2V01Y201903HLT041
Jing YEastwood MTan BKonios AHamid ACollinson MHamdan HBoubiche DKlett F(2017)An intelligent well-being monitoring system for residents in extra care homesProceedings of the 1st International Conference on Internet of Things and Machine Learning10.1145/3109761.3109769(1-6)Online publication date: 17-Oct-2017
https://dl.acm.org/doi/10.1145/3109761.3109769
Cohen S(2016)Bayesian Analysis in Natural Language ProcessingSynthesis Lectures on Human Language Technologies10.2200/S00719ED1V01Y201605HLT0359:2(1-274)Online publication date: 9-Jun-2016
https://doi.org/10.2200/S00719ED1V01Y201605HLT035
Nguyen TBouzerdoum APhung S(2016)Variational inference for infinite mixtures of sparse Gaussian processes through KL-correction2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2016.7472143(2579-2583)Online publication date: Mar-2016
https://doi.org/10.1109/ICASSP.2016.7472143
Wang PLiu LShen CHuang ZHengel AShen H(2016)What’s Wrong with That Object? Identifying Images of Unusual Objects by Modelling the Detection Score Distribution2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR.2016.174(1573-1581)Online publication date: Jun-2016
https://doi.org/10.1109/CVPR.2016.174
Srijith PBalamurugan PShevade S(2016)Gaussian Process Pseudo-Likelihood Models forźSequence LabelingEuropean Conference on Machine Learning and Knowledge Discovery in Databases - Volume 985110.1007/978-3-319-46128-1_14(215-231)Online publication date: 19-Sep-2016
https://dl.acm.org/doi/10.1007/978-3-319-46128-1_14
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents