Self-attention-based conditional random fields latent variables model for sequence labeling

Yinan Shao, Jerry Chun Wei Lin, Gautam Srivastava, Alireza Jolfaei, Dongdong Guo, Yi Hu

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)
2 Downloads (Pure)


To process data like text and speech, Natural Language Processing (NLP) is a valuable tool. As on of NLP's upstream tasks, sequence labeling is a vital part of NLP through techniques like text classification, machine translation, and sentiment analysis. In this paper, our focus is on sequence labeling where we assign semantic labels within input sequences. We present two novel frameworks, namely SA-CRFLV-I and SA-CRFLV-II, that use latent variables within random fields. These frameworks make use of an encoding schema in the form of a latent variable to be able to capture the latent structure in the observed data. SA-CRFLV-I shows the best performance at the sentence level whereas SA-CRFLV-II works best at the word level. In our in-depth experimental results, we compare our frameworks with 4 well-known sequence prediction methodologies which include NER, reference parsing, chunking as well as POS tagging. The proposed frameworks are shown to have better performance in terms of many well-known metrics.

Original languageEnglish
Pages (from-to)157-164
Number of pages8
JournalPattern Recognition Letters
Publication statusPublished - May 2021
Externally publishedYes


  • Big data
  • Encoding schema
  • Latent CRF
  • Natural language processing
  • Sequence labeling
  • VQA


Dive into the research topics of 'Self-attention-based conditional random fields latent variables model for sequence labeling'. Together they form a unique fingerprint.

Cite this