The eyes know it: FakeET- An Eye-tracking Database to Understand Deepfake Perception

Parul Gupta; Komal Chugh; Abhinav Dhall; Ramanathan Subramanian

doi:10.1145/3382507.3418857

The eyes know it: FakeET- An Eye-tracking Database to Understand Deepfake Perception

Parul Gupta, Komal Chugh, Abhinav Dhall, Ramanathan Subramanian

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

22 Citations (Scopus)

Abstract

We present FakeET - an eye-tracking database to understand human visual perception of deepfake videos. Given that the principal purpose of deepfakes is to deceive human observers, FakeET is designed to understand and evaluate the ability of viewers to detect synthetic video artifacts. FakeET contains viewing patterns compiled from 40 users via the Tobii desktop eye-tracker for 811 videos from the Google Deepfake dataset, with a minimum of two viewings per video. Additionally, EEG responses acquired via the Emotiv sensor are also available. The compiled data confirms (a) distinct eye movement characteristics for real vs fake videos; (b) utility of the eye-track saliency maps for spatial forgery localization and detection, and (c) Error Related Negativity (ERN) triggers in the EEG responses, and the ability of the raw EEG signal to distinguish between real and fake videos.

Original language	English
Title of host publication	ICMI '20
Subtitle of host publication	Proceedings of the 2020 International Conference on Multimodal Interaction
Place of Publication	New York, NY
Publisher	Association for Computing Machinery, Inc
Pages	519-527
Number of pages	9
ISBN (Electronic)	9781450375818
DOIs	https://doi.org/10.1145/3382507.3418857
Publication status	Published - 22 Oct 2020
Externally published	Yes
Event	22nd ACM International Conference on Multimodal Interaction - Virtual, Online, Netherlands Duration: 25 Oct 2020 → 29 Oct 2020 Conference number: 22nd

Publication series

Name	Proceedings of the International Conference on Multimodal Interaction
Volume	2020

Conference

Conference	22nd ACM International Conference on Multimodal Interaction
Abbreviated title	ICMI 2020
Country/Territory	Netherlands
City	Virtual, Online
Period	25/10/20 → 29/10/20

Keywords

deepfake
eeg
eye-tracking
visual perception

Access to Document

10.1145/3382507.3418857Licence: In Copyright

Cite this

Gupta, P., Chugh, K., Dhall, A., & Subramanian, R. (2020). The eyes know it: FakeET- An Eye-tracking Database to Understand Deepfake Perception. In ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction (pp. 519-527). (Proceedings of the International Conference on Multimodal Interaction; Vol. 2020). Association for Computing Machinery, Inc. https://doi.org/10.1145/3382507.3418857

Gupta, Parul ; Chugh, Komal ; Dhall, Abhinav et al. / The eyes know it : FakeET- An Eye-tracking Database to Understand Deepfake Perception. ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction. New York, NY : Association for Computing Machinery, Inc, 2020. pp. 519-527 (Proceedings of the International Conference on Multimodal Interaction).

@inproceedings{c13280d51efb49f195bf73ea69ac060c,

title = "The eyes know it: FakeET- An Eye-tracking Database to Understand Deepfake Perception",

abstract = "We present FakeET - an eye-tracking database to understand human visual perception of deepfake videos. Given that the principal purpose of deepfakes is to deceive human observers, FakeET is designed to understand and evaluate the ability of viewers to detect synthetic video artifacts. FakeET contains viewing patterns compiled from 40 users via the Tobii desktop eye-tracker for 811 videos from the Google Deepfake dataset, with a minimum of two viewings per video. Additionally, EEG responses acquired via the Emotiv sensor are also available. The compiled data confirms (a) distinct eye movement characteristics for real vs fake videos; (b) utility of the eye-track saliency maps for spatial forgery localization and detection, and (c) Error Related Negativity (ERN) triggers in the EEG responses, and the ability of the raw EEG signal to distinguish between real and fake videos.",

keywords = "deepfake, eeg, eye-tracking, visual perception",

author = "Parul Gupta and Komal Chugh and Abhinav Dhall and Ramanathan Subramanian",

year = "2020",

month = oct,

day = "22",

doi = "10.1145/3382507.3418857",

language = "English",

series = "Proceedings of the International Conference on Multimodal Interaction",

publisher = "Association for Computing Machinery, Inc",

pages = "519--527",

booktitle = "ICMI '20",

note = "22nd ACM International Conference on Multimodal Interaction, ICMI 2020 ; Conference date: 25-10-2020 Through 29-10-2020",

}

Gupta, P, Chugh, K, Dhall, A & Subramanian, R 2020, The eyes know it: FakeET- An Eye-tracking Database to Understand Deepfake Perception. in ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction. Proceedings of the International Conference on Multimodal Interaction, vol. 2020, Association for Computing Machinery, Inc, New York, NY, pp. 519-527, 22nd ACM International Conference on Multimodal Interaction, Virtual, Online, Netherlands, 25/10/20. https://doi.org/10.1145/3382507.3418857

The eyes know it: FakeET- An Eye-tracking Database to Understand Deepfake Perception. / Gupta, Parul; Chugh, Komal; Dhall, Abhinav et al.
ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction. New York, NY: Association for Computing Machinery, Inc, 2020. p. 519-527 (Proceedings of the International Conference on Multimodal Interaction; Vol. 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - The eyes know it

T2 - 22nd ACM International Conference on Multimodal Interaction

AU - Gupta, Parul

AU - Chugh, Komal

AU - Dhall, Abhinav

AU - Subramanian, Ramanathan

N1 - Conference code: 22nd

PY - 2020/10/22

Y1 - 2020/10/22

N2 - We present FakeET - an eye-tracking database to understand human visual perception of deepfake videos. Given that the principal purpose of deepfakes is to deceive human observers, FakeET is designed to understand and evaluate the ability of viewers to detect synthetic video artifacts. FakeET contains viewing patterns compiled from 40 users via the Tobii desktop eye-tracker for 811 videos from the Google Deepfake dataset, with a minimum of two viewings per video. Additionally, EEG responses acquired via the Emotiv sensor are also available. The compiled data confirms (a) distinct eye movement characteristics for real vs fake videos; (b) utility of the eye-track saliency maps for spatial forgery localization and detection, and (c) Error Related Negativity (ERN) triggers in the EEG responses, and the ability of the raw EEG signal to distinguish between real and fake videos.

AB - We present FakeET - an eye-tracking database to understand human visual perception of deepfake videos. Given that the principal purpose of deepfakes is to deceive human observers, FakeET is designed to understand and evaluate the ability of viewers to detect synthetic video artifacts. FakeET contains viewing patterns compiled from 40 users via the Tobii desktop eye-tracker for 811 videos from the Google Deepfake dataset, with a minimum of two viewings per video. Additionally, EEG responses acquired via the Emotiv sensor are also available. The compiled data confirms (a) distinct eye movement characteristics for real vs fake videos; (b) utility of the eye-track saliency maps for spatial forgery localization and detection, and (c) Error Related Negativity (ERN) triggers in the EEG responses, and the ability of the raw EEG signal to distinguish between real and fake videos.

KW - deepfake

KW - eeg

KW - eye-tracking

KW - visual perception

UR - http://www.scopus.com/inward/record.url?scp=85096672983&partnerID=8YFLogxK

UR - http://purl.org/au-research/grants/arc/DP190102919

U2 - 10.1145/3382507.3418857

DO - 10.1145/3382507.3418857

M3 - Conference contribution

AN - SCOPUS:85096672983

T3 - Proceedings of the International Conference on Multimodal Interaction

SP - 519

EP - 527

BT - ICMI '20

PB - Association for Computing Machinery, Inc

CY - New York, NY

Y2 - 25 October 2020 through 29 October 2020

ER -

Gupta P, Chugh K, Dhall A, Subramanian R. The eyes know it: FakeET- An Eye-tracking Database to Understand Deepfake Perception. In ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction. New York, NY: Association for Computing Machinery, Inc. 2020. p. 519-527. (Proceedings of the International Conference on Multimodal Interaction). doi: 10.1145/3382507.3418857

The eyes know it: FakeET- An Eye-tracking Database to Understand Deepfake Perception

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this