In this document, the authors examine some common methodological challenges in educational technology research and highlight new data collection approaches using examples from the literature and our own work. Given that surveys and questionnaires remain widespread and dominant tools across nearly all studies of educational technology, the authors first discuss the background and limitations of how researchers have traditionally used surveys to define and measure technology use (as well as other variables and outcomes). Through this discussion, they introduce our own work with a visual analog “sliding” scale as an example of a new approach to survey design and data collection that capitalizes on the technology resources increasingly available in schools. Next, they highlight other challenges and opportunities inherent in the study of educational technology, including the potential for computer adaptive surveying, and discuss the critical importance of aligning outcome measures with the technological innovation, concerns with computer-based versus paper-based measures of achievement, and the need to consider the hierarchical structure of educational data in the analysis of data for evaluating the impact of technology interventions.
High-quality technical assistance for results