Market Researcher’s Secret: You Cannot “Use a Steam-Hammer to Crack Nuts”

Lyudmila Goryunova is the head of a Russian company which is engaged in market and consumer research. In an interview with Youth Time magazine she discussed what should be considered a correctly conducted opinion poll, how to identify a respondent-deceiver, and how to put together an appropriate questionnaire.        

Let’s begin with the most important question: how can an inexperienced person figure out whether he is looking at the results of a poll that has been conducted properly and determine that it is reliable?

This is an issue that sparks heated discussion, and it is a central topic of numerous books. It is due to the fact that there is no single indicator that gives us an understanding of whether a poll was carried out correctly or not. Research is a complex product, and it is important to keep the specifications in mind at every stage of its “production”. First of all, it is necessary to turn attention to the methodology of a poll – what tasks have been set and how they were implemented. There are no good or bad methods, only those that correspond or do not correspond with the tasks. For instance, if it is necessary to make an important strategic decision, subsequent to the results of the research, as well as to justify your calculations in reliance on this data, you have to conduct a quantitative research, since figures are required. If the main objective is to have the consumer reveal some interesting insights, then it is necessary to carry out a deep qualitative research project, because in this case quantitative surveys would be useless.

The choice of the target audience is very important. You can’t “use a steam-hammer to crack nuts” when conducting research (especially market research). It is essential to define the selection parameters for the participants in a research project accurately, focusing on particulars such as age, gender, income, and consumption characteristics. Judging from this information, you have to clarify the method – think about where you can find such people. Are they at home? Then you need a telephone survey, based on making calls on landline telephones. Are they spending a good deal of time surfing the Internet? Perhaps, it would be appropriate to launch an online survey. Will you be able to persuade them to participate in focus groups? Will they come to you? Or would it be better to pay them a personal visit and conduct an interview? All these factors have to be taken into account during the planning stage.

The next step would be to determine whether the actual business objective will be served by the proposed questionnaire or guidelines. “Direct” questions, when a person is asked rather clearly whether he knows or uses something, or what his stance on a particular issue is, may not provide the desired results. During qualitative research, oftentimes it is necessary to rely on projective techniques. After all, it is not of great importance that your brand is associated, for example, with Pinocchio. More significant is the question whether in the consumer’s understanding the Pinocchio character is perky, cheerful, curious and easy-going.                        

In your opinion, which global research agencies are the most trustworthy?

Judging from my knowledge of the research community, I can tell you that most agencies, regardless of whether they are privately owned or public, Russian or foreign, are doing their best to generate the most objective and accurate data. The research organizations have the option of affiliating with a good selection of professional associations, such as the World Association for Social, Opinion and Market Research (ESOMAR) (www.esomar.org). If the agency is a member of this organization, then it can serve, to some extent, to confirm that it adheres to the essential standards of research realization, which are formalized in the Code. Most big agencies have a highly functional system of data control, and they seek to improve it. For example, there are databases that allow you to detect dishonest respondents who try to make money by participating in focus groups by deceiving the organizers and pretending to be some person other than who they are. Nevertheless, there is a constant problem involving the human factor. Any agency, regardless of whether it is large or small, international or regional, can encounter such difficulties. Some questionnaires are always sifted out as a result of control procedures. If you’ve been told that customary procedures did not reveal a single violation, it could be a good reason to perform a recheck.            

What kind of research do you consider to be questionable?

In my opinion, there are multiple problems associated with research of this type. The first one is related to the challenge of identifying the people who should be invited to take part in the survey. Besides, the reason behind most falsifications is that the task of finding the target audience is overcomplicated. For example, in the process of searching for suitable respondents too many conditions may be set. All factors should coincide, such as age, income, usage of certain product, and apart from that the respondent has to watch TV no less than 5 hours a day, have a dog of a certain breed, as well as a blond second cousin who works in the accounting field. Certainly I’m exaggerating now, but from time to time the requirements to qualify respondents may be brought to the point of absurdity, so that it actually becomes way too difficult to find them in real life. That’s when the human factor plays its part yet again, as it is necessary to respond to the customer’s order, though not all researchers are willing to take a strong stand, even when they should. Sometimes they are tied to the client’s chariot, and only after some time do they begin to figure out how to accomplish the task sensibly.

The second problem is excessively voluminous questionnaires (primarily relevant to quantitative researches). In this case, there are two complications. If an interview lasts for more than 15 minutes, the respondent gets tired and is tempted to interrupt the conversation. Therefore, the most impatient ones, or those in a hurry, are time and again overlooked in the course of analysis (whilst sometimes they form an important part of our audience). The interviewer in these instances also has a temptation to cut down a questionnaire, miss some questions, or simplify something. The wording of questions poses a similar problem. Some questions are too complex, and they put respondent in a quandary, and some are repetitious and boring. If such questions form a critical part of the questionnaire, then the risk that an interview won’t be completed increases.

Regarding the methods, I can say that at the present time household surveys are the most questionable ones. In today’s world of intercoms, concierges, and the increased vigilance of people due to the fear of becoming a victim of theft or fraud, it is almost impossible to perform a random representative survey using this method. Ironically, street surveys, which are often accused of massive falsifications, since monitoring them is rather difficult, are fabricated by interviewers less often than the household ones, because, once again, it is easier to poll than to trump up. Besides, there is still a substantial risk of being caught.

Are there any situations when public opinion regarding an issue can shift dramatically over a short period of time?

This typically happens with the ratings of politicians at the regional as well as at the national level. They are most vulnerable to rapid shifts in the day-to-day situation of their country, city, or village, and the way such changes are portrayed in mass media. Sometimes one word, uttered by a politician, can skyrocket his rating or send it to the pit of hell. Introducing new brands and products can also cause radical changes in public opinion. Information travels at the speed of the Internet, for those who are connected to it, and the modern audience is very sensitive to advertising and other forms of communication. Therefore, novelties are spreading much faster than ever before in history. However, opinions about some relatively “old” and familiar categories are not changing so rapidly. If you are already familiar with something, you have to try it in order to believe that it has changed. Besides, it is not always possible to do it quickly (not everyone would want to do it, for example, because of a negative attitude). Indeed, the balance of long-standing competitors such as Coca-Cola and Pepsi does change, although this process is gradual.

Do you personally have any trust in public opinion, and do you rely on it when making your own choices? 

Yes, to some extent. Since I am an opinion surveyor by vocation, I’ve always paid attention to what the respondents were saying. I have had my own “discoveries”, which emerged “subsequent to the results of surveys”. For instance, if I saw that people who were similar to me, were choosing a certain product, I would try to comprehend why this was happening and consider that, perhaps, the product would suit me too. And occasionally I use public opinion on the principle of “ex contrario”. So, if I see that some product is overly popular, I may do the opposite and ignore it.

Since April 2014, Lyudmila Goryunova has run the Russian company “Knowledge Management Center”, which examines market conjunctures and reveals public opinion. She has 15 years of work experience in the field of research, as well as a background in political consulting and assisting during election campaigns. Lyudmila has also dealt with internal corporate investigations and reputation audits. She has three degrees (including those in the field of sociology and economics). Goryunova has an M.A degree in Sociology (University of Manchester) and she holds a PhD in Social Sciences (Russian Federation).                                  

Support us!

All your donations will be used to pay the magazine’s journalists and to support the ongoing costs of maintaining the site.