Online Research 2000-2004 at FESSEL-GfK
Peter Diem
January 23, 2004
Quantitative Research via the Web
Quantitative market research using the Internet as a carrier medium is going into its fifth year at FESSEL-GfK. Not counting a number of pilot questionnaires and projects offered but not ordered, around 200 research projects were conducted since June 2000. The software used for “Computer Assisted Web Interviews (CAWI)” has been the “Internet Research Manager (IRM)”, developed as proprietary software by Domestic Data/Vienna together with FESSEL-GfK online research. The IRM is a menu-based multi-language Internet research suite which can be operated without knowledge of HTML by any junior research assistant -that is without help by IT personnel. Under the licence negotiated with Domestic Data, the programme can be and is being used by any number of staff/computers at FESSEL-GfK and its subsidiaries in Central and Eastern Europe. At present, IRM works in 17 languages not only to translate the wording of questions but also to work entirely in one’s native language with the desk interface. The IRM software can handle all forms of questions possible on the Web including multi-media stimuli (image, sound, video etc.). Amongst other features, the IRM includes a recoding module for processing open questions. It enables customized online reporting and has a number of possibilities for recording paradata such as average question and interview duration, break-off statistics etc. Field reports give a precise picture of the functioning of a study on an hourly basis. Data can be processed by Quantum, SPSS or directly online and exported into MS Office programmes.
Quantitative online research is being conducted by FESSEL-GfK along four distinct methodological lines:
Pool surveys:
Here the sample is drawn from an address database recruited offline and, therefore, representative of the Austrian Internet user population (sometimes, but incorrectly, called “access panel”). In the case of FESSEL-GfK the address pool comprises about 25 000 persons of whom ca 15.000 are active at the moment. As an incentive, 8-20 points are being allotted per completed questionnaire. For 100 points respondents receive a shopping voucher worth 15 Euros.
Pop-up Surveys:
Every n visitor of a Website is confronted with a pop-up window inviting him/her to participate in a professional market research project. Thereby Website operators can find out about the demographic structure of their audience and their attitudes towards the site. Sometimes, pop-ups are also used for other subject matter. By a special method, respondents are informed that the pop-up window is not a commercial advertising. Double entry is prevented by a cookie.
Surveys based on customer-provided addresses:
Employee studies, customer surveys and membership projects are being conducted on the basis of address material held or provided by the client. A practical problem that may arise here is poor quality of addresses. If the client does not trust the institute and organizes the mailings himself, difficulties in timely reminders etc. may occur. In borderline cases anonymous market research tends to become CRM.
Link Surveys:
In special cases, research may be based on persons who are asked to follow a link on the Web or displayed on a PC-e.g., in a hotel lobby. Such surveys can record demographics and attitudes of visitors and guests of museums, fairs etc.
FESSEL-GfK has not created a special task force for online research but follows the policy to let each department decide for themselves if they want to use CAWI or not. The author is acting as a consultant.
Using the built-in performance statistics of IRM, it is possible to describe some of the major characteristics of the research work carried out by the various departments at FESSEL-GfK. One observation was quite surprising: the studies conducted over the last few years varied in their respective parameters much more than we opurselves would have expected of “typical” Internet research projects:
Few and many interviews
The median (“typical”) online project was based on n = 211 complete interviews-the result of a median gross sample of N = 505 people. The median response rate was 49 percent (min 12%, max 100%). Sample sizes varied considerably: from n = 10 complete interviews (in a Pop-up Test on a very lowtraffic -Website) to n = 6 091 complete interviews (in a transnational employee study for an Austrian bank). Short and long interviews
The median duration of online interviews was 9.4 minutes with a minimum of 1.3 minutes (“how much do you spend on your mobile phone?”) and a maximum of 46.3 minutes (“use of various cosmetics”).
Few and many questions
Considering that a respondent may not have to answer all questions due to filters, the mean number of questions per project was 34, with a minimum of 2 (“mobile phone” again) and a maximum of 194 questions (“cosmetics” again). The most frequent number of questions per project (mode) was 20.
Short and long field periods
Although there were some studies carried out within 1-2 days, the most frequent field period was 4 days. This is explained by the fact that relatively many poster pre-tests were conducted. In such studies with short field time, it is our policy to include the weekend in order to reach also “light surfers” and/or people who do most of their surfing at home. Median duration of field work was 14 days with a maximum of 102 days (in a business-to-business study).
The year 2003
Like everybody else we too suffered from the effects of economic slowdown and the backlash following the Internet hype of the years preceding 2002. Nevertheless, FESSEL-GfK carried out 60 quantitative online research projects last year. Besides the employee study and the cosmetics survey quoted above, major projects included the Austrian Internet Radar (AIR – 2 waves at n = 5000) and the study Austrian Youth Online (3 waves at n = 1 000). The AIR is the most important source of data on Internet usage in Austria. By means of the “logo recall method” 120 leading Internet sites are being polled online and their audience size and composition determined.
Besides the logfile statistics, published at http://www.oewa.at, the AIR is the only reliable basis for online advertizing in Austria, as long as there is no Internet audience size measurement panel.
Another interesting and successful online project is the so-called Poster Online Pretest (“POP”). This series of studies on billboards is carried out for a major Austrian outdoor advertizing firm. At regular intervals up to a dozen posters are judged by ca. 200 respondents.
Under the title “Online Express” colleagues in the consumer goods section and in other departments have started to sell a “genuine” Internet research product: a general advertising copy test online for pretesting concepts, ads,
logos, wrappings, TV or radio spots, etc. on the basis of n = 200 and at an attractive price of Euro 1000.- per tested item. It seems however, that the Austrian market is still rather reluctant to make use of such opportunities.
Finally, it should be mentioned that several Pop up-Tests were conducted in order to show the efficiency of Websites operated by commercial firms or state agencies. Return of investment is best proved by such surveys.
Qualitative Research about and via the Web
The main instrument to check usability and efficiency of a website is the Qualitative Webtest. On the institute premises a special room has been reserved for in-depth interviews with respondents who sit in front of a desktop PC and occupy themselves with a specific Website. In this way, e.g., one of the main offers of Austrian e-government was investigated. On the basis of n = 30 , a thorough analysis of practical use of this guide through government agencies could be made. A Pop up-Test to quantify results is to follow.
While studying Websites in group discussions or by personal “in front of the screen” interviews has become commonplace with FESSEL-GfK, the Online Focus Group has not yet entered practical use. In spite of the fact that a customized programme for easy use has been developed together with the Regensburg based software firm SPIN AG
(http://www.spingroups.de), the Austrian market is said to be still reluctant to accept Web-based discussion groups. Maybe the colleagues in Prague and Ljubljana, who have also tested our software, will be leading the way. At any rate, further virtual focus group experiments will be conducted in 2004 together with the psychology department at FESSEL-GfK.
Into the future
From the statistical evaluation of around 60 recent and 200 overall online studies carried out by FESSEL-GfK and its subsidiaries, a number of conclusions can be made:
1. Online research (O.R.) in order to be introduced on a broad level needs strong back-up by management. Workshops to train staff and inform clients are a must. At least one person per company must work fulltime in this field.
2. O.R. software like the IRM that can be understood and handled by practically everybody in the institute is a good basis for take-up of the
method. In many cases the media research department will take the lead and spread the knowledge.
3. O.R. needs imaginative leadership. The younger generation of researchers who are completely familiar with the possibilities of the digital age are not yet in management positions. But they can contribute new and fresh ideas provided such ideas are welcome.
4. Commercial calculation of cost has proved that O.R. if conducted professionally today is not a lot cheaper than conventional research. At least in the initial years the cost of software, of some new hardware and of necessary expertise and training must be carried as an additional load on prices. In the long run O.R. will however prove its cost efficiency vis á vis face-to-face and phone interviews – not counting its many methodological advantages.
5. O.R. has the capacity both to supplement and replace conventional surveys. Take a project to test 52 packages of hair cosmetics on a nationwide quantitative basis what can be cheaper and faster than O.R.? In many other applicationse.g., in social research with an emphasis on the total population – O.R. is not yet practical. In some instances – e.g., with respondent e-mail addresses provided by the client, O.R. cannot be beaten.
6. The basic philosophy of O.R. according to theory is “quick and straight”: short and easy-to-answer questionnaires administered within a few days with results delivered online for an attractive price. In practice, such products have been the exception. As long as there is no forced marketing for “typical” online services, conventional studies transferred to the Web seem to prevail. There is no objection against methodological “change-over” from conventional to online, but it would be desirable to design and sell more “true” online products too.
7. The decision whether to install a specific individual or working group responsible for O.R. or leave it entirely up to the respective departments to decide about using and marketing O.R. – is not easy. In the beginning, it seems to me from my own practice, it would be advisable to have both. Later on all branches in the company will be willing and able to go for O.R. without any central support.
8. One of the big advantages of O.R. is its trans-national potential. The built-in translation module of the IRM and its multi-language interface makes it possible for all CEE subsidiaries to work with it in their respective national language. Although “all business is local”, the opening of the
Central and Eastern European markets in an enlarged European Union will bring about many chances for cross-national, multi-language research projects. This should be born in mind regardless of the somewhat slower development of Internet use in the new CEE democracies.
9. While in the beginning, that is a few years ago, exchange of opinion on O.R. methodology and commercial practice in personal or digital form -was rather frequent, lately there has been little interchange of information between GfK online researchers. With economic growth and market chances recovering it would be desirable to re-establish such contacts.
10. There is hardly any field in which change is so noticeable and so fast as in Information Technology. O.R. is a typical product of the digital age.
In order to stay abreast of the time, it is advisable to invest a lot of energy (and some money) in this fascinating new method. FESSEL-GfK -as could be shown – has decided to do so.