Heretical Ideas – Abandon Focus Groups

Oct
7
2010

A snap shot in time of a Fountain is this what Focus Groups are about

To often we see focus groups being used as the core evaluation and research tool within a web design project. Why!?

Focus groups are in the main used to evaluate and get recommendations from customers on the proof of concepts or prototypes. They have a strength in gathering people’s judgements, emotions and possible interactive scenarios from the group as a whole.

If you don’t know I have no love for focus groups, I have yet to see them produce any results that have not been tainted or saturated in biases. To the extent that they were just unusable. With the research or evaluation having to be conducted by some other technique later on in the project. Often at an extra expense.

Why use Focus Groups

If focus groups are so great why are we still using them. Well mainly we are still using them due to several business driven factors:

  • Cost

    The biggest cost overall factor for all research methods is the cost of the consultant. Hence consider the cost of 1 focus group session verses 15 interviews at 1 hour each, plus transcription and analysis. You can se why they are attractive.

  • Ease of implementation

    Compared to ethnographic studies, the easy of gathering a group of customers or stakeholders is a very attractive factor. You have them there all in one room, and even a few hours later you have an outcome. Quick and easy.

  • Previously Known Technique

    I have found that often marketing firms will use this method and rebrand it as user research. Mainly because they see it as a trusted and tried technique and maybe they are under the misguided priniciple that market research is vary similar to customer (user) research.

The Issues

The issues all stem from the social interactions of the focus group. People after all for the norm are sociable, this can be a big issue. But it’s not the only issue:

  • Qualitative Results

    The information gathered from the focus group will be at best qualitative in nature, while this is good to give a strategic direction for an emotion response. It’s not what we are after overall. We need cold hard evidence more often than not.

  • What Customers Say

    To often, as we know, customers will tell you they would do one thing, but when observed they in fact do something completely different. This alteration of the discussed “reality” and the observed reality is extenuated by the group think process.

  • Group Think

    People tend to react differently when you get them in a group situation then what they do when you have then alone. Its the usual social dynamics in play, domination, pecking orders, social personality masking and so on. No matter how good a moderator is, they can’t stop the group think process from taking hold, they can only lessen it. Hence people will further suppress ideas, concept or critique in the fear of it’s not exceptable by the group. Focus groups in this way are lot like a jury without evidence.

  • Domination

    All it takes is a very dominate personality in a focus group and this will taint the group outcome dramatically. To the point that you are just in fact reporting on the view point of the dominate individuals.

  • In Depth Examination

    Yes focus groups can provide some in-depth group examination of issues. However there just isn’t the process to dig deeper into an issue and follow the trail to some of the core issues, as say would be examined in a one on one interview or ethnographic study.

    Yes you can have a good moderator who can reduce these aspects. However overall they are still going to occur, just to a lesser degree. How much of a lesser degree will depend on the experience and skills of the moderator.

The Alternatives

It’s all very good dismissing focus groups out of hand, but what are the alternatives:

  • User Testing

    Nothing beats user testing, working with the participants one on one, observing them. Reducing the bias via carefully worded questions on a pre determined scenario that you are evaluating. All the social issues, bias and the like are removed. Remember user testing doesn’t have to be expensive, it can be done in a very cost effectively. It’s just a matter of audience and budgetary scope.

  • Interviews

    When you are using a focus group to gather scenario or even user type data, you really would be better off with a series of interviews, even short ones. If you you suspect that your interview will be too short to cover all your bases then consider priming the participants before hand.

  • Participatory Design

    This really isn’t that new if you have been following a real customer focused design approach. If you are of the system design school then it maybe a little radical.

    Basically you let the customers design the product for you. Under the guiding hands of the design, development and business teams of course. Why should you be considering that all the good ideas are going to come from your team of designers, and then have them evaluated by the customers. Why not bring a key number of customers along for the journey of the design process. (I’m going to pester you more on this topical later in another post.)

  • Collaborative Analysis

    This allows a group of customers, usually one for each persona, the deisgn team, and client to evaluate a design or the like. The process allows for placement of the personas (customer) one at a time in centre stage and have them evaluate the design from their view point. Ultimately the customers will also assist in the design process, contributing to alternatives and the solving of any problems.

    The design team and client are only present in terms of an advisory role on any recommendations that is forth coming.

So maybe you like or even love focus groups; fine that is good.

However next time you go to use them, just stop and think about the altenatives. Is a focus group really going to serve you well.

What do you think are focus groups a waste of time or are they the best thing since sliced bread. You tell me.

Tags: , , , , , ,

3 comments

  1. I agree with most of your concerns with focus groups. I admit the method is flawed – although the problem of self-reported data is hardly unique to focus groups and affects most of the alternative approaches you recommend.

    Focus group is a research methodology that yields results and is appropriate for certain research needs, generally at the very beginning or very end of a project, viz. the exploratory and validation stages.

    If I were conducting research personally I probably wouldn’t go with focus groups but in the case of my current project where we outsourced the research (time, resource constraints, consultant factor etc) focus groups were the recommended approach and provided useful results. Results that need to be interpreted carefully but results nonetheless.

  2. Issue becomes that focus groups are used too often as the only research tool. And they are also often used for a lot more than just exploratory research. The process is tainted from its marketing roots, which sadly see it as a core tool.

    Again, as you say the results from a focus group need to be interpreted carefully.

    Too often a half hearted approach is given to the analysis and the results are taken as often the gospel truth. This is an area of extreme danger, the time needs to be taken to do a proper analysis.

  3. From a transcriptionist’s point of view, they are a total waste of time. It is next to impossible to “get” what everyone says as they are too far away from the microphone, speak in low voices, speak at the same time, mumble, etc. So what has been said does NOT get transcribed. They are a transcriptionist’s absolute nightmare, straining to try to decipher what’s being said. Extremely, EXTREMELY aggravating. I always wonder why the person moderating a focus group doesn’t: 1) Give everyone a good-quality mic, 2) INSIST that each person speak up,speak clearly, identify themselves each time they speak, 3) Ensure it’s conducted in a QUIET room, 4) Immediately intervene each time someone mumbles, speaks too low, interrupts. And, amazingly, the people who run focus groups are all astonished when they don’t get a clear accurate transcript. DUH!!! Focus groups are a nightmare to transcribe.

Comments are now closed, move along, nothing to see here.