In an earlier post, we talked about the Gallup organization’s process for translating employee engagement surveys. Today, we want to discuss some of the cultural issues and biases that may taint survey results. Even if a survey has been painstakingly translated, some variation can be chalked up to cultural differences. Individuals from certain cultures may tend to respond to surveys in certain ways, regardless of what is actually being asked.
Or call 215-729-6765 to talk with a team member.
For example, a popular format for response categories is the Likert scale, in which a respondent is asked to indicate on a scale of one through five how strongly they agree or disagree with a statement. All the answers on the Gallup survey take this format.
Moderation or extremism?
One issue that consistently arises in debates over international comparisons of survey results is that members of some cultural groups are more likely to choose extreme answers (strongly agree, strongly disagree), and members of others are more likely to choose answers toward the middle (neither agree nor disagree), regardless of the content of the question. This may reflect different cultural values about how moderate one’s opinions should be, and whether it is acceptable to express extreme positive or negative opinions. Therefore, to better understand individual variation, the survey designer might use a scale of 1-10 rather than 1-5 because it allows for finer gradations of opinion.
Which end is up?
Another problem with Likert scales is that some cultures may be accustomed to seeing 1 as the most positive end of a scale and 5 as the most negative, while others may be used to the reverse. In the United States, some researchers have noticed that respondents tend to choose answers nearer to the far left of the scale regardless of content, possibly because of patterns of perception; one wonders whether cultures whose language reads right-to-left (Arabic, Hebrew) might favor the far right end of the scale? To control for this tendency, the survey designer might survey half the respondents using a scale that runs from left to right, and the other half with a scale that runs the opposite direction.
Can they ask me that?
Another source of confusion can come from demographics measures. Asking about income levels and ethnic heritage might feel intrusive and offensive in some cultures. Also, levels of education can present a problem insofar as they are not universally equivalent. It is not clear whether the O-levels in Pakistan are the educational equivalent of the GCSE in England and the GED in the United States, and the assessment of the value of a University degree versus a technical school certificate might be different for residents of one country than for another.
What do they want me to say?
Finally, a survey developer might consider whether respondents might be biased toward answering questions in ways that are socially acceptable. For example, if deference to authority is a widely held cultural value, invitations to rate or evaluate managers may trend toward favorable ratings. People may feel similarly regarding their peers. Even if you assure the respondents that individual answers are going be kept anonymous, faith in the preservation of “privacy” and “anonymity” vary widely across cultures. You might not know whether your survey measures what people actually think or what they know they are supposed to be thinking. Here is where a cultural assessment is crucial to design questions that are specific enough to gain useful data, but that are non-threatening to core social values.
When you ask people to participate in a survey, you are asking for a favor. An employee engagement survey does more than measure attitudes and beliefs—it also communicates to employees what management thinks is important. Although there are lots of things that might be “nice to know,” you must be careful not to include questions about issues or problems that you have no intention of addressing. Survey participants may resent giving answers to questions that were asked “in bad faith,” and this may reinforce feelings of mistrust. Finally, in global corporations employees may speak a different language than management. Be sure to translate the report of the survey findings to a language that employees understand (if you plan to share this information). And always rely on experienced partners to provide pre-translation cultural consulting as well as localization in order to get the most out of your project.