Lesson 2.4: Designing Surveys and Questionnaires

Introduction

Surveys are a powerful tool for gathering information and understanding people's opinions, behaviors, and preferences. In this lesson, we will explore how to design effective surveys and questionnaires. By the end of this lesson, you will be able to:

Differentiate between open and closed questions and understand rating scales, including Likert scales.
Identify the effects of wording in questions, such as leading, loaded, double-barreled, and ambiguous questions.
Learn about question order, social-desirability bias, and how respondent fatigue can impact your data.
Understand the importance of piloting a questionnaire and how to improve it based on pilot feedback.

Let's get started with a question to hook your interest: Have you ever filled out a survey that made you feel confused or frustrated? 🤔 Understanding how to frame questions can make all the difference!

Open vs Closed Questions

Open Questions

Open questions allow respondents to answer in their own words. They provide qualitative data, which can be rich and detailed. However, analyzing open responses can be time-consuming. For example:

What do you think about the new school cafeteria menu?

This gives the respondent freedom to express their opinion in any way they choose.

Closed Questions

Closed questions restrict responses to predefined options, making it easier to analyze results quantitatively. Common closed questions might include:

Do you like the new cafeteria menu? (Yes/No)
How satisfied are you with the new cafeteria menu? (Very Satisfied, Satisfied, Neutral, Unsatisfied, Very Unsatisfied)

Rating Scales

One popular type of closed question uses rating scales, like the Likert scale, which measures attitudes or feelings across a continuum. A typical Likert scale looks like this:

On a scale of 1 to 5, how much do you agree with the following statement: “The new cafeteria menu is healthy.”

$ - 1 = Strongly Disagree$

$ - 2 = Disagree$

$ - 3 = Neutral$

$ - 4 = Agree$

$ - 5 = Strongly Agree$

This method provides a structured way to gauge opinions, making it easy to compute averages and trends. For instance, if 60% of students rate the menu 4 or 5, this indicates a generally positive reception.

Wording Effects

Leading Questions

Leading questions prompt or suggest a particular answer. For example:

How great do you think the new cafeteria menu is?

This question might lead respondents to focus on positive aspects rather than offering an unbiased opinion.

Loaded Questions

Loaded questions carry emotional weight that can influence the response. For example:

What do you think about the terrible choices in the new cafeteria?

This question is biased because it assumes the choices are terrible before the respondent has an opportunity to express their thoughts.

Double-Barreled Questions

Double-barreled questions ask about two different issues but allow for only one response. For instance:

How satisfied are you with the cleanliness and variety of the cafeteria menu?

This can confuse respondents since they may feel differently about the cleanliness compared to the variety.

Ambiguous Questions

Ambiguous questions lack clarity, leading to varied interpretations. For example:

What do you think about improvements?

This does not specify what improvements are being referred to, leaving respondents guessing.

Question Order and Bias

Question Order

The order of questions can affect responses. For example, if you ask about satisfaction first and follow up with a question about specific issues, responses to the second question may be influenced by the first one. Avoid bias by randomizing question order when possible.

Social-Desirability Bias

Respondents may alter their answers to appear favorable in the eyes of others, known as social-desirability bias. To minimize this effect, consider using indirect questioning or ensuring anonymity in surveys.

Respondent Fatigue

Respondent fatigue occurs when participants become tired or bored; this can lead to careless responses or survey abandonment. Keep surveys concise, clear, and engaging to combat fatigue!

Piloting a Questionnaire

The Importance of Piloting

Before launching your survey, piloting is crucial. A pilot survey tests your questions and structure on a small sample to identify issues, such as confusing wording or poor flow.

After piloting, ask for feedback: Did they understand the questions? Did they feel fatigued?

Improving the Questionnaire

Use feedback from the pilot test to make necessary changes. For instance, if participants found certain questions confusing, consider rephrasing them or splitting them into simpler parts. Always aim for clarity and ease of understanding.

Conclusion

Designing effective surveys requires careful consideration of question types, wording, order, and testing. By employing open and closed questions wisely, avoiding leading or ambiguous wording, and piloting your questionnaire, you can gather accurate and meaningful data. Remember, the goal is to make things easy for respondents so that you can get the best insights from your surveys! 📝

Study Notes

Open questions → qualitative data; closed questions → quantitative data.
Likert scales measure attitudes on a defined scale, e.g., 1 to 5.
Avoid leading, loaded, double-barreled, and ambiguous questions for unbiased data collection.
Randomize question order to minimize bias.
Be aware of social-desirability bias and respondent fatigue; keep surveys short and engaging.
Piloting helps identify issues in your questionnaire for improvement.