Research Methods and Methodologies

Data alone, regardless of its type, does not mean anything until you interpret it. The processes that you use to collect, analyze, and organize your data are your research methods. Research methods are often categorized as quantitative, qualitative or mixed method. Some projects,  such as lab experiments, require the use of the scientific method of inquiry, observation, quantitative data collection, analysis, and conclusions to test a hypothesis. Other kinds of projects take a more deductive approach and gather both quantitative and qualitative evidence to support a position or recommendation. The research methods you choose will be determined by the goals and scope of your project, and by your intended audience’s expectations.

Data Collection

In terms of data collection, there are a variety of qualitative and quantitative methods available. A list of several common primary data collection methods is provided below. Note that each method follows a specific protocol both to ensure the validity of the data and to protect any human or animal subjects involved. For more on research that uses human participants, see the “Human Research Ethics” section later in this chapter.

Interviews. Interviews are one-on-one or small group question and answer sessions. Interviews will provide detailed information from a small number of people and are useful when you want to get an expert opinion on your topic.

Surveys/Questionnaires. Surveys are a form of questioning that is less flexible than interviews, as the questions are set ahead of time and cannot be changed. Surveys can be in print format or delivered electronically. This method can reach much larger groups of people than interviews, but it results in less detailed responses.

On-site research. These observations involve taking organized notes about occurrences at a determined research site. Research sites may be physical locations, such as a local gym or building site, or they may be virtual, such as an online forum or event. Observations allow you to gain objective information without the potentially biased viewpoint of an interview or survey.

Experiments. Whether in the lab or in the field, experiments are designed to test hypotheses and verify previous results. Experiments are prepared by using standard protocols and careful testing in order to protect the researchers and their subjects, as well as to isolate specific variables.

Simulations. Typically designed and run using computer programs, simulations are a type of experiment that tests hypotheses and solutions in a virtual setting that approximates the real world. Simulations are usually an option for when in-person experiments are not feasible.

Primary source documents. More common in text-based fields, original written, visual, and/or audio sources can be used to locate specific data for further analysis and interpretation. In this method, the data collected could be words, images, sounds, or movements.

Effective Primary Research Design

In a technical and professional writing class you will likely use a few common primary research methods involving human subjects: surveys, interviews, and on-site research (field, lab, or simulation). While you are not expected to be an expert in any of these methods, you should approach them ethically and thoughtfully so as to protect any participants and to generate reliable, generalizable data.

Survey Questions

When designing surveys, remember the rhetorical situation. What are the goals of your survey? Who are you hoping will complete the survey? What will they know? What will they not know? How long can you expect them to engage with your survey? What is the best method of surveying them (online, say through Google Forms, or in person)? How many responses do you hope to obtain? Use this information to inform the design of your survey and any preliminary materials you include. All surveys should feature clear statements of purpose, as well as specific directions for answering the questions and how to contact the researcher if participants have any questions.

After determining your audience and purpose, you will need to design your questions. Remember, in all online surveys you will not be there to provide immediate clarification, so your questions need to be carefully worded to avoid confusion and researcher bias. As a rule, your survey questions should

Be as specific as possible. Avoid ambiguity by providing specific dates, events, or descriptors as necessary.

Ask only one question at a time. Specifically, avoid survey questions that require the participant to answer multiple items at once. This will confuse the reader as to what you are looking for and will likely skew your data.

Be neutral. Present your survey questions without leading, inflammatory, or judgmental language. Common leading survey questions that you want to avoid include phrasing like “Do you agree that our enemies are a threat to our way of life?” You will also want to avoid using language that is sexist, racist, or ableist. See Chapter 4: Persuasion for more information about loaded language.

Be organized logically. Questions should be presented in a way that makes sense to the participant. For example, if you introduce a concept in Question 1, you do not want to return to it again in Question 12. Follow-up questions and linked questions should be asked in succession rather than separated.

Allow participants to decline answering. In general, you will want to be wary of questions that require participants to divulge sensitive information, even if they are answering anonymously. This information could include details such as a trauma, eating disorders, or drug use. For research projects that require these questions, consult your university’s IRB (Internal Review Board). They may need you to fill out special documentation that accounts for how you will protect your participants.

After designing the questions, you will also need to consider how your participants can answer them. Depending, you may opt for quantitative data, which includes yes/no questions, multiple choice, Likert scales, or ranking. Note that what makes this data “quantitative” is that it can be easily converted into numerical data for analysis.  Alternatively, you may opt for qualitative data, which includes questions that require a written response from the participant. A description and some of the advantages of these answer styles follow below:

Yes/No (Quantitative). These simple questions allow for comparison but not much else. They can be useful as a preliminary question to warm up participants or open up a string of follow-up questions.

Multiple choice (Quantitative). These questions allow for pre-set answers and are particularly useful for collecting demographic data. For example, a multiple-choice question might look something like this: “How many years have you attended your university?” Depending on the question, you may wish to allow for a write-in response.

Likert scale (Quantitative). One of the most common answer types, the Likert scale is a rating, usually on a 1-5 scale. At one end of the scale, you will have an option such as “Definitely Agree” and on the other you will have “Definitely Disagree.” In the middle, if you choose to provide it, is a neutral option. Some answers in this format may use a wider range (1-10, for example), offer a “Not Applicable” option, or remove the neutral option. Be mindful of what these choices might mean. A wider scale could, in theory, mean more nuance, but only if the distinctions between each option are clear.

Ranking (Quantitative). In a ranking-based answer, you provide a list of options and prompt your participant to place them in a certain order. For example, you may be offering five potential solutions to a specific problem. After explaining the solutions, you ask your reader to identify which of the five is the best, which is second best, and so on. Participants may assign these items a number or rearrange their order on a screen.

Written responses (Qualitative). Especially when you want detailed, individualized data, you may choose for participants to provide written answers to your questions. This approach is beneficial in that you may receive particularly detailed responses or ideas that the survey did not address. You might also be able to privilege voices that are often drowned out in large surveys. However, keep in mind that many participants do not like responding to essay-style questions. These responses work best as follow-up questions midway or later in the survey.

Finally, before officially publishing your survey online or asking participants in person, make sure that you conduct preliminary testing. This preliminary testing is crucial. When seeking feedback, have your reviewers note any confusion or ambiguity in question wording, lack of clarity in question order, typographical errors, technical difficulties, and how long the survey took for them to complete. Remember, surveys with unclear questions and sloppy formatting annoy participants and damage your credibility. Conversely, the more professional a survey looks and the easier it is for your reader to complete it, the more likely you will receive useful responses.

Writing Engaging Research Interview Questions

Preparing good interview questions takes time, practice, and testing. Many novice interviewers go into interviews with the assumption that they do not need to prepare and are merely having a conversation. While this approach can generate information, these interviewers often find that several important questions were not addressed. When designing interview questions, you will want not only to consider the content of the question but also where the question appears in your list.

When preparing for an interview, first contact your potential interviewee as soon as possible. Individuals, especially those who work outside academia, may operate on timelines that may feel odd to college and university students. You will also want to prepare any equipment (such as a recorder or smart phone, but request permission first before recording!), questions, and IRB approval, if applicable.

Carter McNamara offers the following suggestions for wording interview questions. This passage is quoted in its entirety:

Wording should be open-ended. Respondents should be able to choose their own terms when answering questions.

Questions should be as neutral as possible. Avoid wording that might influence answers, e.g., evocative, judgmental wording.

Questions should be asked one at a time. Avoid asking multiple questions at once. If you have related questions, ask them separately as a follow-up question rather than part of the initial query.

Questions should be worded clearly. This includes knowing any terms particular to the program or the respondents’ culture.

Be careful asking “why” questions. This type of question infers a cause-effect relationship that may not truly exist. These questions may also cause respondents to feel defensive, e.g., that they have to justify their response, which may inhibit their responses to this and future questions.^[1]

If you choose to have a face-to-face interview or interview over Zoom or Skype, show up on time and dress according to the level of the interviewee. Honoring the interviewee’s time by being punctual, having prepared questions, and not extending past an established time limit is crucial to both collecting good information and maintaining a positive relationship with the interviewee. For more information on designing effective interviews, see Appendix: Qualitative Interview Design.

Field Research

When conducting field research, or research that takes you outside of a lab or simulation, you will need to consider the following:

Gain appropriate permissions for researching the site. Your “site” is the location where you are conducting research. Sites could include potential locations for a community garden, a classroom where you’re observing student behaviors or a professor’s teaching strategies, or a local business. Certain sites will require specific permission from an owner or other individual. Depending on your study, you may also need to acquire IRB permission.

Know what you’re looking for. While people-watching is interesting, your most effective field research will be accomplished if you know roughly what you want to observe. For instance, say you are observing a large lecture from a 100-level class, and you are interested in how students use their laptops, tablets, or phones. In your observation, you would be specifically focusing on the students, with some attention to how they’re responding to the professor. You would not be as focused on the content of the professor’s lecture or if the students are doing non-electronic things such as doodling or talking to their classmates.

Take notes. Select your note-taking option and prepare backups. While in the field, you will be relying primarily on observation. Record as much data as possible and back up that data in multiple formats.

Be unobtrusive. In field research, you function as an observer rather than a participant. Therefore, do your best to avoid influencing what is happening at the research site.

Data Interpretation

Methods also include ways of interpreting and organizing data, either once it has been collected or simultaneously with data collection. More specific methodologies, such as ways to structure the analysis of your data, include the following:

Coding. Reviews transcripts of interview data and assigns specific labels and categories to the data. A common social science method.

Cost/benefit analysis. Determines how much something will cost versus what measurable benefits it will create.

Life-cycle analysis. Determines overall sustainability of a product or process, from manufacturing, through lifetime use, to disposal. You can also perform comparative life-cycle analyses or specific life cycle stage analysis.

Comparative analysis. Compares two or more options to determine which is the “best” solution given specific problem criteria such as goals, objectives, and constraints.

Process analysis. Studies each aspect of a process to determine if all parts and steps work efficiently together to create the desired outcome.

Sustainability analysis. Uses concepts such as the “triple bottom line” or “three pillars of sustainability” to analyze whether a product or process is environmentally, economically, and socially sustainable.

In all cases, the way you collect, analyze, and use data must be ethical and consistent with professional standards of honesty and integrity. Lapses in integrity may lead to poor quality reports not only in an academic context (poor grades and academic dishonesty penalties) but also in the workplace. These lapses can lead to lawsuits, job loss, and even criminal charges. Some examples of these lapses include

• Fabricating your own data (making it up to suit your purpose)
• Ignoring data that disproves or contradicts your ideas
• Misrepresenting someone else’s data or ideas
• Using data or ideas from another source without acknowledgment or citation of the source.