GraphITTI: Attributed Graph-based Dominance Ranking in Social Interaction Videos

Garima Sharma; Shreya Ghosh; Abhinav Dhall; Munawar Hayat; Jianfei Cai; Tom Gedeon

doi:10.1145/3610661.3616184

GraphITTI: Attributed Graph-based Dominance Ranking in Social Interaction Videos

Garima Sharma, Shreya Ghosh, Abhinav Dhall, Munawar Hayat, Jianfei Cai, Tom Gedeon

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Estimating the most dominant person in a social interaction setting is a challenging feat even with the advancement of deep learning techniques due to problem complexity, non-availability of labelled data and subjective biases in annotations. This paper aims to reformulate the problem of detecting the Most Dominant Person (MDP) as a person ranking problem by utilizing person-specific attributes such as facial gestures, eye gaze, visual attention and speaking patterns. Our proposed framework, attributed Graph-based dominant person ranking in social InTeracTIon videos, GraphITTI, learns generic and robust person rankings on top of context level features. To inject domain knowledge into the GraphITTI framework, we consider inter-personal and intra-personal aspects along with spatiotemporal context patterns. Our extensive quantitative analysis suggests that GraphITTI framework performs favourably over the current state-of-the-art for dominant person detection and ranking. The code is available at https://github.com/shgnag/GraphITTI.

Original language	English
Title of host publication	ICMI '23 Companion
Subtitle of host publication	Companion Publication of the 25th International Conference on Multimodal Interaction
Place of Publication	New York, NY
Publisher	Association for Computing Machinery
Pages	323-329
Number of pages	7
ISBN (Electronic)	9798400703218
DOIs	https://doi.org/10.1145/3610661.3616184
Publication status	Published - 9 Oct 2023
Externally published	Yes
Event	International Conference on Multimodal Interaction - Paris, France Duration: 9 Oct 2023 → 13 Oct 2023 Conference number: 25th

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	International Conference on Multimodal Interaction
Abbreviated title	ICMI 2023
Country/Territory	France
City	Paris
Period	9/10/23 → 13/10/23

Bibliographical note

Publisher Copyright:
© 2023 ACM.

Keywords

Attributed Graph
Dominant Person Detection
Group Environment
Leadership
Social Interaction

Access to Document

10.1145/3610661.3616184Licence: In Copyright

Cite this

Sharma, G., Ghosh, S., Dhall, A., Hayat, M., Cai, J., & Gedeon, T. (2023). GraphITTI: Attributed Graph-based Dominance Ranking in Social Interaction Videos. In ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal Interaction (pp. 323-329). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3610661.3616184

@inproceedings{d942def3aeae447695051952dbf4503b,

title = "GraphITTI: Attributed Graph-based Dominance Ranking in Social Interaction Videos",

abstract = "Estimating the most dominant person in a social interaction setting is a challenging feat even with the advancement of deep learning techniques due to problem complexity, non-availability of labelled data and subjective biases in annotations. This paper aims to reformulate the problem of detecting the Most Dominant Person (MDP) as a person ranking problem by utilizing person-specific attributes such as facial gestures, eye gaze, visual attention and speaking patterns. Our proposed framework, attributed Graph-based dominant person ranking in social InTeracTIon videos, GraphITTI, learns generic and robust person rankings on top of context level features. To inject domain knowledge into the GraphITTI framework, we consider inter-personal and intra-personal aspects along with spatiotemporal context patterns. Our extensive quantitative analysis suggests that GraphITTI framework performs favourably over the current state-of-the-art for dominant person detection and ranking. The code is available at https://github.com/shgnag/GraphITTI.",

keywords = "Attributed Graph, Dominant Person Detection, Group Environment, Leadership, Social Interaction",

author = "Garima Sharma and Shreya Ghosh and Abhinav Dhall and Munawar Hayat and Jianfei Cai and Tom Gedeon",

note = "Publisher Copyright: {\textcopyright} 2023 ACM.; International Conference on Multimodal Interaction, ICMI 2023 ; Conference date: 09-10-2023 Through 13-10-2023",

year = "2023",

month = oct,

day = "9",

doi = "10.1145/3610661.3616184",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

pages = "323--329",

booktitle = "ICMI '23 Companion",

}

Sharma, G, Ghosh, S, Dhall, A, Hayat, M, Cai, J & Gedeon, T 2023, GraphITTI: Attributed Graph-based Dominance Ranking in Social Interaction Videos. in ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal Interaction. ACM International Conference Proceeding Series, Association for Computing Machinery, New York, NY, pp. 323-329, International Conference on Multimodal Interaction, Paris, France, 9/10/23. https://doi.org/10.1145/3610661.3616184

GraphITTI: Attributed Graph-based Dominance Ranking in Social Interaction Videos. / Sharma, Garima; Ghosh, Shreya; Dhall, Abhinav et al.
ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal Interaction. New York, NY: Association for Computing Machinery, 2023. p. 323-329 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - GraphITTI

T2 - International Conference on Multimodal Interaction

AU - Sharma, Garima

AU - Ghosh, Shreya

AU - Dhall, Abhinav

AU - Hayat, Munawar

AU - Cai, Jianfei

AU - Gedeon, Tom

N1 - Conference code: 25th

PY - 2023/10/9

Y1 - 2023/10/9

N2 - Estimating the most dominant person in a social interaction setting is a challenging feat even with the advancement of deep learning techniques due to problem complexity, non-availability of labelled data and subjective biases in annotations. This paper aims to reformulate the problem of detecting the Most Dominant Person (MDP) as a person ranking problem by utilizing person-specific attributes such as facial gestures, eye gaze, visual attention and speaking patterns. Our proposed framework, attributed Graph-based dominant person ranking in social InTeracTIon videos, GraphITTI, learns generic and robust person rankings on top of context level features. To inject domain knowledge into the GraphITTI framework, we consider inter-personal and intra-personal aspects along with spatiotemporal context patterns. Our extensive quantitative analysis suggests that GraphITTI framework performs favourably over the current state-of-the-art for dominant person detection and ranking. The code is available at https://github.com/shgnag/GraphITTI.

AB - Estimating the most dominant person in a social interaction setting is a challenging feat even with the advancement of deep learning techniques due to problem complexity, non-availability of labelled data and subjective biases in annotations. This paper aims to reformulate the problem of detecting the Most Dominant Person (MDP) as a person ranking problem by utilizing person-specific attributes such as facial gestures, eye gaze, visual attention and speaking patterns. Our proposed framework, attributed Graph-based dominant person ranking in social InTeracTIon videos, GraphITTI, learns generic and robust person rankings on top of context level features. To inject domain knowledge into the GraphITTI framework, we consider inter-personal and intra-personal aspects along with spatiotemporal context patterns. Our extensive quantitative analysis suggests that GraphITTI framework performs favourably over the current state-of-the-art for dominant person detection and ranking. The code is available at https://github.com/shgnag/GraphITTI.

KW - Attributed Graph

KW - Dominant Person Detection

KW - Group Environment

KW - Leadership

KW - Social Interaction

UR - http://www.scopus.com/inward/record.url?scp=85175860380&partnerID=8YFLogxK

U2 - 10.1145/3610661.3616184

DO - 10.1145/3610661.3616184

M3 - Conference contribution

AN - SCOPUS:85175860380

T3 - ACM International Conference Proceeding Series

SP - 323

EP - 329

BT - ICMI '23 Companion

PB - Association for Computing Machinery

CY - New York, NY

Y2 - 9 October 2023 through 13 October 2023

ER -

Sharma G, Ghosh S, Dhall A, Hayat M, Cai J, Gedeon T. GraphITTI: Attributed Graph-based Dominance Ranking in Social Interaction Videos. In ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal Interaction. New York, NY: Association for Computing Machinery. 2023. p. 323-329. (ACM International Conference Proceeding Series). doi: 10.1145/3610661.3616184

GraphITTI: Attributed Graph-based Dominance Ranking in Social Interaction Videos

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this