Data-gathering (or measurement) can be done in many different ways, using different tools, depending on the type of data you wish to collect. One thing that is common to all forms of data collection is that they can be done well or done badly, and this can make a large difference to the success of a study. Even something as simple as a basic survey of student attitudes can provide high-quality data, but only if the survey is designed thoughtfully, so some effort to insure the quality of your measurement should be part of any educational investigation.
There are many ways of collecting data, each of which has advantages and disadvantages.
Ways of collecting data
Interviews
Simply speaking with people in a focused and intentional way can yield very useful information. Interviews excel at painting a richly contextualized portrait of a phenomenon from one person’s point of view, and they can range from tightly structured (using a predetermined set of questions in a specified order—an interview protocol--with each interviewee) to unstructured (a free-flowing conversation the direction of which is determined organically as the interview proceeds). Interviews generate large amounts of detailed data, which can be challenging to analyze.
Focus groups
Sometimes thought of as a subclass of interviews, focus groups involve speaking with several people in a group setting to generate detailed information from a variety of points of view. The interplay of perspectives can be quite revealing, particularly if the participants feel free to express both agreement and disagreement. Focus groups require skill to conduct successfully, because the group conversation must be guided and managed in a deft, tactful manner (Krueger & Casey 2014).
Surveys and questionnaires
Surveys are collections of questions that are usually delivered on paper or online to participants in a study. Due to the ease of constructing and implementing surveys, they are probably the most common measure used in educational research. However, it is easy to construct a survey that does not yield useful information because the questions are ambiguous, badly worded, or badly selected, or because of bias and error built into the response scales.
You can improve your survey design using the think-aloud technique, in which a few potential survey-takers are recruited to respond to a draft survey while speaking their thoughts aloud in the presence of a researcher who can use the feedback to improve the validity of the survey.
Tests, assignments, and other performance measures
Information about student learning is often provided by assignments associated with a class, or by standardized tests and other measures of student performance. When well-constructed, tests can yield important information about outcomes of great educational interest, but like surveys, tests that are developed in an informal way are often unreliable or lacking in other properties (such as consistency over time) that are necessary for them to produce data useful for educational studies (Thorndike 2005).
Class grades are usually compiled from a variety of assignments and are often used as indicators of student learning, but attention should be paid to their reliability and to the fact that they often have a distribution that is concentrated in a small area of the available scale.
Observations
Sometimes the phenomena you are interested in studying can be observed directly, such as the type and frequency of learning activities that occur in a class, the way an instructor moves through a classroom, or some aspects of student group interaction. When this is possible, observation has the advantage of providing large quantities of data that are not subject to the same biases that afflict self-reported data, although observers must be well-trained and aware of the biases they bring to the task.
Research, evaluation, and the IRB
After you have settled on research questions, a research design, and data collection methods, an important consideration is whether your investigation requires review by the University of Minnesota’s Institutional Review Board, or IRB.
The IRB reviews all studies at UMN that meet the federal definition of research to insure that participants in those studies are treated ethically. Broadly speaking, a study constitutes research if it is a systematic inquiry that involves human participants and is designed to produce generalizable knowledge.