EMIP: The eye movements in programming dataset

Roman Bednarik; Teresa Busjahn; Agostino Gibaldi; Alireza Ahadi; Maria Bielikova; Martha Crosby; Kai Essig; Fabian Fagerholm; Ahmad Jbara; Raymond Lister; Pavel Orlov; James Paterson; Bonita Sharif; Teemu SirkiäFIn; Jan Stelovsky; Jozef Tvarozek; Hana Vrzakova; Ian van der Linde

doi:10.1016/j.scico.2020.102520

EMIP: The eye movements in programming dataset

Roman Bednarik^*, Teresa Busjahn, Agostino Gibaldi, Alireza Ahadi, Maria Bielikova, Martha Crosby, Kai Essig, Fabian Fagerholm, Ahmad Jbara, Raymond Lister, Pavel Orlov, James Paterson, Bonita Sharif, Teemu SirkiäFIn, Jan Stelovsky, Jozef Tvarozek, Hana Vrzakova, Ian van der Linde

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

21 Citations (Scopus)

Abstract

A large dataset that contains the eye movements of N=216 programmers of different experience levels captured during two code comprehension tasks is presented. Data are grouped in terms of programming expertise (from none to high) and other demographic descriptors. Data were collected through an international collaborative effort that involved eleven research teams across eight countries on four continents. The same eye tracking apparatus and software was used for the data collection. The Eye Movements in Programming (EMIP) dataset is freely available for download. The varied metadata in the EMIP dataset provides fertile ground for the analysis of gaze behavior and may be used to make novel insights about code comprehension.

Original language	English
Article number	102520
Number of pages	11
Journal	Science of Computer Programming
Volume	198
Early online date	4 Aug 2020
DOIs	https://doi.org/10.1016/j.scico.2020.102520
Publication status	Published - 15 Oct 2020

Keywords

eye-tracking, program comprehension, dataset
Program comprehension
Dataset
Eye-tracking

ASJC Scopus subject areas

Software
Information Systems
Computational Theory and Mathematics
Modelling and Simulation

Documents and Links

10.1016/j.scico.2020.102520

21 Citations
1 Conference contribution
1 Chapter (peer-reviewed)

Readability metrics for program code: how is reading ease reflected in gaze?
Paterson, J. H. & Hartmann, K., 2016, Reports and Studies in Forestry and Natural Sciences: Eye Movements in Programming: Models to Data, Proceedings of the Third International Workshop. Bednarik, R., Busjahn, T., Schulte, C. & Tamm, S. (eds.). Joenssu, Finland: University of Eastern Finland, Vol. 23. p. 12-13 2 p.
Research output: Chapter in Book/Report/Conference proceeding › Chapter (peer-reviewed) › peer-review
Eye movements in code reading: relaxing the linear order
Busjahn, T., Bednarik, R., Begel, A., Crosby, M., Paterson, J., Schulte, C., Sharif, B. & Tamm, S., 7 Sept 2015, 2015 IEEE 23rd International Conference on Program Comprehension. IEEE, p. 255-265 11 p.
Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Open Access
File
176 Citations (Scopus)

1185 Downloads (Pure)

Cite this

Bednarik, R., Busjahn, T., Gibaldi, A., Ahadi, A., Bielikova, M., Crosby, M., Essig, K., Fagerholm, F., Jbara, A., Lister, R., Orlov, P., Paterson, J., Sharif, B., SirkiäFIn, T., Stelovsky, J., Tvarozek, J., Vrzakova, H., & van der Linde, I. (2020). EMIP: The eye movements in programming dataset. Science of Computer Programming, 198, Article 102520. https://doi.org/10.1016/j.scico.2020.102520

@article{5dd036423e9c4f1aa28e415e7407f096,

title = "EMIP: The eye movements in programming dataset",

abstract = "A large dataset that contains the eye movements of N=216 programmers of different experience levels captured during two code comprehension tasks is presented. Data are grouped in terms of programming expertise (from none to high) and other demographic descriptors. Data were collected through an international collaborative effort that involved eleven research teams across eight countries on four continents. The same eye tracking apparatus and software was used for the data collection. The Eye Movements in Programming (EMIP) dataset is freely available for download. The varied metadata in the EMIP dataset provides fertile ground for the analysis of gaze behavior and may be used to make novel insights about code comprehension.",

keywords = "eye-tracking, program comprehension, dataset, Program comprehension, Dataset, Eye-tracking",

author = "Roman Bednarik and Teresa Busjahn and Agostino Gibaldi and Alireza Ahadi and Maria Bielikova and Martha Crosby and Kai Essig and Fabian Fagerholm and Ahmad Jbara and Raymond Lister and Pavel Orlov and James Paterson and Bonita Sharif and Teemu Sirki{\"a}FIn and Jan Stelovsky and Jozef Tvarozek and Hana Vrzakova and {van der Linde}, Ian",

note = "Acceptance from webpage AAM requested 27/08/20 ST and 15/9/20 ET (with note of REF policy deadline) ",

year = "2020",

month = oct,

day = "15",

doi = "10.1016/j.scico.2020.102520",

language = "English",

volume = "198",

journal = "Science of Computer Programming",

publisher = "Elsevier B.V.",

}

Bednarik, R, Busjahn, T, Gibaldi, A, Ahadi, A, Bielikova, M, Crosby, M, Essig, K, Fagerholm, F, Jbara, A, Lister, R, Orlov, P, Paterson, J, Sharif, B, SirkiäFIn, T, Stelovsky, J, Tvarozek, J, Vrzakova, H & van der Linde, I 2020, 'EMIP: The eye movements in programming dataset', Science of Computer Programming, vol. 198, 102520. https://doi.org/10.1016/j.scico.2020.102520

TY - JOUR

T1 - EMIP: The eye movements in programming dataset

AU - Bednarik, Roman

AU - Busjahn, Teresa

AU - Gibaldi, Agostino

AU - Ahadi, Alireza

AU - Bielikova, Maria

AU - Crosby, Martha

AU - Essig, Kai

AU - Fagerholm, Fabian

AU - Jbara, Ahmad

AU - Lister, Raymond

AU - Orlov, Pavel

AU - Paterson, James

AU - Sharif, Bonita

AU - SirkiäFIn, Teemu

AU - Stelovsky, Jan

AU - Tvarozek, Jozef

AU - Vrzakova, Hana

AU - van der Linde, Ian

N1 - Acceptance from webpage AAM requested 27/08/20 ST and 15/9/20 ET (with note of REF policy deadline)

PY - 2020/10/15

Y1 - 2020/10/15

N2 - A large dataset that contains the eye movements of N=216 programmers of different experience levels captured during two code comprehension tasks is presented. Data are grouped in terms of programming expertise (from none to high) and other demographic descriptors. Data were collected through an international collaborative effort that involved eleven research teams across eight countries on four continents. The same eye tracking apparatus and software was used for the data collection. The Eye Movements in Programming (EMIP) dataset is freely available for download. The varied metadata in the EMIP dataset provides fertile ground for the analysis of gaze behavior and may be used to make novel insights about code comprehension.

AB - A large dataset that contains the eye movements of N=216 programmers of different experience levels captured during two code comprehension tasks is presented. Data are grouped in terms of programming expertise (from none to high) and other demographic descriptors. Data were collected through an international collaborative effort that involved eleven research teams across eight countries on four continents. The same eye tracking apparatus and software was used for the data collection. The Eye Movements in Programming (EMIP) dataset is freely available for download. The varied metadata in the EMIP dataset provides fertile ground for the analysis of gaze behavior and may be used to make novel insights about code comprehension.

KW - eye-tracking, program comprehension, dataset

KW - Program comprehension

KW - Dataset

KW - Eye-tracking

U2 - 10.1016/j.scico.2020.102520

DO - 10.1016/j.scico.2020.102520

M3 - Article

VL - 198

JO - Science of Computer Programming

JF - Science of Computer Programming

M1 - 102520

ER -

EMIP: The eye movements in programming dataset

Abstract

Keywords

ASJC Scopus subject areas

Documents and Links

Fingerprint

Research output

Readability metrics for program code: how is reading ease reflected in gaze?

Eye movements in code reading: relaxing the linear order

Cite this