A Transformer Based Approach for Activity Detection

Gulshan Sharma; Abhinav Dhall; Ramanathan Subramanian

doi:10.1145/3503161.3551598

A Transformer Based Approach for Activity Detection

Gulshan Sharma
, Abhinav Dhall
, Ramanathan Subramanian

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

13 Citations (Scopus)

Abstract

Non-invasive physiological sensors allow for the collection of user-specific data in realistic environments. In this paper, using physiological data, we investigate the effectiveness of Convolutional Neural Network (CNN) based feature embeddings and Transformer architecture for the human activity recognition task. 1D-CNN representation is used for the heart rate, and 2D-CNN is used for short-term Fourier transformation of the accelerometer data. Post fusion, the feature is input into a transformer. The experiments are performed on the harAGE dataset. The findings indicate the discriminative ability of the feature-fusion on transformer-based architecture, and the method outperforms the harAGE baseline by an absolute 3.7%.

Original language	English
Title of host publication	MM '22
Subtitle of host publication	Proceedings of the 30th ACM International Conference on Multimedia
Place of Publication	New York, NY
Publisher	Association for Computing Machinery, Inc
Pages	7155-7159
Number of pages	5
ISBN (Electronic)	9781450392037
DOIs	https://doi.org/10.1145/3503161.3551598
Publication status	Published - 10 Oct 2022
Externally published	Yes
Event	30th ACM International Conference on Multimedia, - Lisboa, Portugal Duration: 10 Oct 2022 → 14 Oct 2022 Conference number: 30th

Publication series

Name	Proceedings of the ACM International Conference on Multimedia
Volume	2022

Conference

Conference	30th ACM International Conference on Multimedia,
Abbreviated title	MM 2022
Country/Territory	Portugal
City	Lisboa
Period	10/10/22 → 14/10/22

Keywords

CNN
human activity recognition
transformers
Human-centered computing
empirical studies in human-computer interaction

Access to Document

10.1145/3503161.3551598Licence: In Copyright

Cite this

@inproceedings{ad833fee275942d4908be9d38a623a50,

title = "A Transformer Based Approach for Activity Detection",

abstract = "Non-invasive physiological sensors allow for the collection of user-specific data in realistic environments. In this paper, using physiological data, we investigate the effectiveness of Convolutional Neural Network (CNN) based feature embeddings and Transformer architecture for the human activity recognition task. 1D-CNN representation is used for the heart rate, and 2D-CNN is used for short-term Fourier transformation of the accelerometer data. Post fusion, the feature is input into a transformer. The experiments are performed on the harAGE dataset. The findings indicate the discriminative ability of the feature-fusion on transformer-based architecture, and the method outperforms the harAGE baseline by an absolute 3.7\%.",

keywords = "CNN, human activity recognition, transformers, Human-centered computing, empirical studies in human-computer interaction",

author = "Gulshan Sharma and Abhinav Dhall and Ramanathan Subramanian",

year = "2022",

month = oct,

day = "10",

doi = "10.1145/3503161.3551598",

language = "English",

series = "Proceedings of the ACM International Conference on Multimedia",

publisher = "Association for Computing Machinery, Inc",

pages = "7155--7159",

booktitle = "MM '22",

note = "30th ACM International Conference on Multimedia, , MM 2022 ; Conference date: 10-10-2022 Through 14-10-2022",

}

Sharma, G, Dhall, A & Subramanian, R 2022, A Transformer Based Approach for Activity Detection. in MM '22: Proceedings of the 30th ACM International Conference on Multimedia. Proceedings of the ACM International Conference on Multimedia, vol. 2022, Association for Computing Machinery, Inc, New York, NY, pp. 7155-7159, 30th ACM International Conference on Multimedia, , Lisboa, Portugal, 10/10/22. https://doi.org/10.1145/3503161.3551598

A Transformer Based Approach for Activity Detection. / Sharma, Gulshan; Dhall, Abhinav; Subramanian, Ramanathan.
MM '22: Proceedings of the 30th ACM International Conference on Multimedia. New York, NY: Association for Computing Machinery, Inc, 2022. p. 7155-7159 (Proceedings of the ACM International Conference on Multimedia; Vol. 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Transformer Based Approach for Activity Detection

AU - Sharma, Gulshan

AU - Dhall, Abhinav

AU - Subramanian, Ramanathan

N1 - Conference code: 30th

PY - 2022/10/10

Y1 - 2022/10/10

N2 - Non-invasive physiological sensors allow for the collection of user-specific data in realistic environments. In this paper, using physiological data, we investigate the effectiveness of Convolutional Neural Network (CNN) based feature embeddings and Transformer architecture for the human activity recognition task. 1D-CNN representation is used for the heart rate, and 2D-CNN is used for short-term Fourier transformation of the accelerometer data. Post fusion, the feature is input into a transformer. The experiments are performed on the harAGE dataset. The findings indicate the discriminative ability of the feature-fusion on transformer-based architecture, and the method outperforms the harAGE baseline by an absolute 3.7%.

AB - Non-invasive physiological sensors allow for the collection of user-specific data in realistic environments. In this paper, using physiological data, we investigate the effectiveness of Convolutional Neural Network (CNN) based feature embeddings and Transformer architecture for the human activity recognition task. 1D-CNN representation is used for the heart rate, and 2D-CNN is used for short-term Fourier transformation of the accelerometer data. Post fusion, the feature is input into a transformer. The experiments are performed on the harAGE dataset. The findings indicate the discriminative ability of the feature-fusion on transformer-based architecture, and the method outperforms the harAGE baseline by an absolute 3.7%.

KW - CNN

KW - human activity recognition

KW - transformers

KW - Human-centered computing

KW - empirical studies in human-computer interaction

UR - https://www.scopus.com/pages/publications/85151158328

U2 - 10.1145/3503161.3551598

DO - 10.1145/3503161.3551598

M3 - Conference contribution

AN - SCOPUS:85151158328

T3 - Proceedings of the ACM International Conference on Multimedia

SP - 7155

EP - 7159

BT - MM '22

PB - Association for Computing Machinery, Inc

CY - New York, NY

T2 - 30th ACM International Conference on Multimedia,

Y2 - 10 October 2022 through 14 October 2022

ER -

A Transformer Based Approach for Activity Detection

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this