无忧视频

Test all the components of your survey. Your survey is more than just the questions.

Take the time to design your survey

Constructing a survey instrument is an art in itself. There are numerous small decisions that must be made — about content, wording, format, placement — that can have important consequences for your results. While there’s no one perfect way to accomplish this, OIRE is available ([email]data@hmc.edu[/email]) to offer advice and guidance to help develop your survey. Additionally, the site is a good resource.

Check the HMC Survey Calendar

OIRE maintains a , which shows what surveys are active on campus at what times for students, faculty and staff. Please contact us ([email]data@hmc.edu[/email]) and let us know about your survey and we will add it to the calendar. This helps OIRE coordinate survey administration, make sure that HMC is not overburdening students, faculty and/or staff with survey requests and helps ensure that your survey has a better chance of getting a good response.

Think about the length of your survey instrument

Think about it again. Every additional question reduces the likelihood respondents will complete the survey. Don鈥檛 ask items you don鈥檛 know how you will use. It is good practice to provide survey respondents with an estimate of how many items are on the survey and/or how long it takes on average to complete the survey in the survey invitation/instructions.

Don鈥檛 ask if you know the answer

Basic demographic information (gender, race/ethnicity, class year, major, etc.) is available through OIRE. When appropriate, suppress these questions and OIRE can provide that data for you. OIRE uses HMC ID as the link between demographic data and survey responses in creating a datafile. HMC also participates in many standard surveys. If the results of these surveys can answer the questions you have, consider using these results rather than running your own independent survey. Not only will you get results faster and avoid asking HMC students questions they have already answered, you will know that the items have been independently tested and validated and you will be assured of a good response rate. A list of surveys that HMC participates in and the types of information available can be found at Institutional Results by Instrument.

Do not 鈥渇orce鈥� a response to a question

By doing this, you may annoy the respondent, causing them to abandon the entire survey, or encourage them to speak out in ways that negatively impact your survey. More importantly, voluntary participation refers to the exercise of free will in deciding whether to participate in any research activity, including answering a question on a survey. International law, national law, and the codes of conduct of scientific communities protect this right. Sample language from CGU鈥檚 Institutional Review Board to include in your instructions: 鈥淭his survey is entirely voluntary, and individuals may answer as few or as many questions as they wish.鈥� OIRE does not force or request a response to any survey or question. Allow the respondents to decide how much information they want to share.

Get feedback on your survey

It鈥檚 a good idea to pilot test your survey with a few volunteers in advance of sending it out. Sample language to invite pilot-testers: 鈥淚n advance of this survey going out to a wider audience, we are asking a select group of students/faculty/staff to respond to the survey early in order to get feedback on the questions, and to understand how long the survey takes to answer.鈥� If you make no changes from pilot test to launching the full version, this allows you to retain and use pilot test data. Many of us are concerned with getting as many respondents as possible. While there is no single method that will increase response rates for your survey, research (Dillman, 2007) does indicate several factors that contribute to higher response rates:

• Perceived importance of the survey (value to the student, the college, perceived legitimacy)
• Level of interest students/faculty/staff have in the topic
• Trust that the data will be used and maintained properly
• Perception of reward for participation
• Minimizing respondent burden

With these in mind, here are some practical considerations that may serve to help you get the best response to your survey:

Who the survey is 鈥渇rom鈥� matters

Who will your intended audience respond to? It may be a department chair, a Dean, or OIRE. 鈥淥ffice of the Dean of Students鈥� is less effective than 鈥淒ean Leslie鈥�. Framing who the survey is from can help you acquire more participation and better data.

The 鈥淪ubject鈥� line of your invitation email matters

Think of your subject line as a headline. It should be:

1. Short (because people don鈥檛 read)
2. Rich in information (clearly summarize the email)
3. Frontloaded with keywords (because people scan only the beginning of a list of items, and an email inbox is a list)
4. Understandable out of context (your email is competing for attention with other emails)

• Provide the reader with a reason to explore your message further.
• The words 鈥淗elp鈥� and 鈥淩eminder鈥� negatively affect survey participation rates.
• General Rule of Thumb: keep the subject line to 50 characters.
• Provide localization and context. 鈥淗MC Core Curriculum Alumni Survey鈥� or 鈥淎lumni Attitudes towards Core Curriculum鈥� is more effective than 鈥淎lumni Survey.鈥�

The proper 鈥淕reeting鈥� in the invitation will increase the likelihood of response

鈥淒ear Alex鈥� is better than 鈥淒ear member of the Class of 2021鈥� is better than 鈥淒ear HMC student鈥�. 鈥淒ear Professor Xu鈥� is better than 鈥淒ear Faculty鈥�.

If you are offering incentives, prizes or gifts for participation, check the rules

Any incentive program may require IRB approval. We recommend that you check with the as soon as you have made a decision regarding incentives. Other helpful hints around the use of incentives are:

• Incentives should be designed in a manner that maintains the voluntary nature of the survey. Please note, if you offer an incentive, your survey is no longer anonymous, as researchers will need to know the identity of those who participated in order to get them their incentive.
• Survey promotion (emails, flyers, etc.) should not emphasize the incentives to a degree that minimizes the requirement of survey participation.
• The amount or value of the incentive should not be so large as to appear coercive, interfere with a student鈥檚 financial aid or violate NCAA eligibility rules.
• If your incentive involves a drawing (i.e., raffle or lottery), you should consult CA state law to determine whether your lottery is legal.
• If you plan to utilize an incentive for your survey, you must add that information about incentives to the instructions all participants receive before deciding to participate in the survey. This section would describe, in detail, the incentive program including the amount that could be won, an estimate of the odds (if you are utilizing a drawing), and how any drawing or other incentive program would be conducted. If you are conducting a web administration, this information should be included on the 鈥淲elcome鈥� page.
• Alternative considerations include allowing the respondents to select their 鈥渋ncentive鈥� (e.g. please select your 鈥渢hank you鈥� gift: a) $10 Claremont Cash, b) $10 Amazon Gift Card; c) $10 donation to HMC鈥檚 Mary S Binder Prize or d) I do not wish to receive a gift) or making a donation for every valid survey response (e.g. donating $1.00 per survey response to a local charity or cause).

Set up your reporting before you launch the survey

You want the best survey possible. In order to obtain quality results, that means doing some work upfront.

• Enter variable names and values into your datafile.
• Generate test responses (pilot test the survey).
• Check and see that reporting is capturing data as you intend.
• Create an analysis strategy (e.g. list of analyses you will do).

For more helpful information, please consult:

• Survey Guidelines
• Survey Checklist