Automatic Difficulty Detection Public Deposited

Downloadable Content

Download PDF
Last Modified
  • March 19, 2019
  • Carter, Jason
    • Affiliation: College of Arts and Sciences, Department of Computer Science
  • Previous work has suggested that the productivity of developers increases when they help each other and as distance increases, help is offered less. One way to make the amount of help independent of distance is to develop a system that automatically determines and communicates developers' difficulty. It is our thesis that automatic difficulty detection is possible and useful. To provide evidence to support this thesis, we developed six novel components: * programming-activity difficulty-detection * multimodal difficulty-detection * integrated workspace-difficulty awareness * difficulty-level detection * barrier detection * reusable difficulty-detection framework Programming-activity difficulty-detection mines developers' interactions. It is based on the insight that when developers are having difficulty their edit ratio decreases while other ratios such as the debug and navigation ratios increase. This component has a low false positive rate but a high false negative rate. The high false negative rate limitation is addressed by multimodal difficulty-detection. This component mines both programmers' interactions and Kinect camera data. It is based on the insight that when developers are having difficulty, both edit ratios and postures often change. Integrated workspace-difficulty awareness combines continuous knowledge of remote users' workspace with continuous knowledge of when developers are having difficulty. Two variations of this component are possible based on whether potential helpers can replay developers' screen recordings. One limitation of this component is that sometimes, potential helpers spend a large amount of time trying to determine if they can offer help. Difficulty-level and barrier detection address this limitation. The former is based on the insight that when developers are having surmountable difficulties they tend to perform a cycle of editing and debugging their code; and when they are having insurmountable difficulties they tend to spend a large amount of time a) between actions and b) outside of the programming environment. Barrier detection infers two kinds of difficulties: incorrect output and design. This component is based the insight that when developers have incorrect output, their debug ratios increase; and when they have difficulty designing algorithms, they spend a large amount of time outside of the programming environment. The reusable difficulty-detection framework uses standard design patterns to enable programming-activity difficulty-detection to be used in two programming environments, Eclipse and Visual Studio. These components have been validated using lab and/or field studies.
Date of publication
Resource type
Rights statement
  • In Copyright
  • Stotts, P. David
  • Brooks, Edward F.
  • Wang, Wei
  • Kelly, Diane
  • Dewan, Prasun
  • Doctor of Philosophy
Degree granting institution
  • University of North Carolina at Chapel Hill Graduate School
Graduation year
  • 2014
Place of publication
  • Chapel Hill, NC
  • There are no restrictions to this item.

This work has no parents.