Investigating eye movements in natural language and C++ source code

Abstract

Is there an inherent difference in the way programmers read natural language text compared to source code? Does expertise play a role in the reading behavior of programmers? In order to start answering these questions, we conduct a controlled experiment with novice and non-novice programmers while they read small short snippets of natural language text and C++ source code. The study was conducted with 33 students recruited from an Introduction to Programming class in the Computer Science and Information Systems department at Youngstown State University. The students were each given ten tasks: a set of seven programs, and three natural language texts. The order of the tasks was randomized within each type. They were asked one of three random comprehension questions after each task. We use several linearity metrics that were presented recently in a similar eye tracking study and report on the findings. The results indicate that novices and non-novices both read source code less linearly than natural language text. We did not find any differences between novices and non-novices between natural language text and source code. We discuss the implications of this work along with directions for future work.