by Craig Cochran

This article is the first in a five-part series discussing the requirements of ISO 9001:2000. The series offers tips for implementing the standard and adding value to your organization. Among other changes, the updated standard now requires explicit evidence of continual improvement. This month, Craig Cochran explores some less familiar tools you can use to assess customer satisfaction.

--Denise Robitaille
Series Editor

In December 2000, the ISO 9001:2000 standard introduced customer satisfaction to the quality management system lexicon. Although more than three years have since passed, when people hear the words “customer satisfaction,” they still think of surveys. This association is understandable yet limiting. Many other methods beyond surveys exist for measuring customer perceptions. Two lesser-known methods are focus groups and undercover customers.

Focus groups

Focus groups usually comprise fewer than 10 people who examine perceptions about a particular product, service, policy or idea. The point is to “focus” on a fairly short list of issues and explore it thoroughly. A facilitator is present to keep the group’s discussion moving in the right direction and ensure that pertinent details are recorded. The garnered information is typically qualitative and concerns subjective observations about the issues under discussion.

A focus group’s power lies in its ability to leverage multiple channels of communication and thought. In this respect it’s similar to a brainstorming session, which encourages interactive thinking to drill down and uncover aspects of an issue or problem. Because a focus group’s success relies on the participants’ interaction, two planning variables are critical:

Using a skilled facilitator who can manage group dynamics. Anyone who’s ever presided over a free-flowing discussion knows how difficult this can be.

Selecting participants who are willing to interact constructively. This often means choosing people with somewhat similar backgrounds and/or demographics. It can be difficult to encourage lively interaction among participants who are uncomfortable in one another’s company.

Generally speaking, certain people shouldn’t be asked to participate in the same focus group. These include bosses and subordinates, direct competitors, family members, experts and novices, and those who hold drastically different opinions from the other participants. Members won’t fully engage with the rest of the group if they’re intimidated or unduly influenced by the some of its members.

Keep in mind, too, that when participants are carefully selected, the group’s conclusions aren’t statistically valid representations of the population as a whole. A focus group represents what a fairly narrow slice of the population thinks about the topic at hand. It’s the opposite of a quantitative survey, which strives for a random selection of participants and statistical validity. There’s nothing wrong with a specifically selected group’s impressions, as long as decision makers recognize the built-in bias and don’t base their conclusions on the results of a single focus group.

Before the session, participants are informed about the general topic but not the specific angle the group will address. Thus, they might be told they’ll be discussing snack foods, or even just food, rather than the specific topic of potato chips. Why the secrecy about specific issues? Because many participants will research the subject if they know what it will be ahead of time. It’s human nature; everyone wants to appear as knowledgeable as possible. Unfortunately, advance research can also influence participants’ thoughts and opinions during the group discussion, resulting in flawed information for the researchers.

A focus group’s line of questions is exploratory and offers plenty of room for participants to stretch out and explore the topic from various angles. Typical lines of questioning include:

What do you like about this product/policy/idea?

What do you dislike?

How does it compare to other products/policies/ideas?

What would you change about it?

Keep the group focused

Participants’ interaction will branch out from these initial thoughts, and each of these questions could easily spawn more detailed discussions. In fact, that’s desirable. You want to start in a comfortable, generalized manner so participants will become comfortable with the topic and each other, and then focus on the details. Participants might be surprised by their thoughts and opinions on a subject, particularly when a skilled moderator encourages the group to expand on the qualities under discussion. A facilitator’s skill is critical at this juncture because a “10,000 foot rap session” that produces generalities won’t benefit anyone. Failing to keep the group focused results in unsatisfactory information. Picture someone trying to herd a flock of noisy, excited geese. That’s similar to what the facilitator is doing.

A focus group’s discussion is usually recorded. Audio recorders and/or video cameras are almost essential, particularly for recording the emotions, opinions, interactions and hot buttons the group uncovers. Don’t worry about spooking the participants. They’ll quickly forget about these devices and ease into free-flowing discussions. Experienced and skilled facilitators often capture relevant points with a flipchart and marker, but these are rarely used as recording techniques. Instead, they help the facilitator guide the group’s discussion in a visual manner that everyone can follow.

Participating in a lively focus group can consume a considerable amount of energy. For this reason, it shouldn’t go on all day. People can’t keep up the intensity level, even if they want to. Focus groups only rarely last more than two hours. Also, it’s common for participants to be compensated in some way for their time and trouble. Meals, cash payments, product samples, souvenirs and discounts can motivate participation.

When the focus group has completed its work, everyone should be heartily thanked. The facilitator should then quickly review the information that’s been recorded, especially if recorded manually. Memories are short, and the facilitator won’t remember what cryptic notes mean a week after the session took place. Clarify all the details while the issues are still fresh in your mind. If electronic recording devices were used, turn them off, secure them and remove the recording medium. If observers were used during the focus group, it’s helpful for them to compare notes and impressions immediately after the group adjourns.

“Focus groups are especially good at generating ideas early in the product development cycle, when it’s too expensive or too complex to develop multiple prototypes,” says Deborah Holden, a trained facilitator and manager with The Coca-Cola Co. in Atlanta. “The ideas that are gathered can then be explored further through quantitative techniques before any big decisions are made.”

Holden also points out a couple of potential problems with focus groups. “They’re not particularly good at getting to the truth of politically correct questions,” she notes. “If participants know the so-called ‘right’ answer, they’ll probably give it, even though it’s not an accurate representation of their thoughts and practices. This often comes up with focus groups that address health, fitness and nutrition. Few people will reveal the truth about these issues in front of strangers.”

Finally, Holden cautions facilitators to be on the lookout for the lone “expert” who might emerge in a focus group. You most likely won’t know who this is until the group is underway. “It’s been my experience that an expert will try to take over the group,” she says. “The group interaction is harmed because participants become afraid of saying something that the expert might disagree with. In extreme situations, the expert can even be removed from the group.”

Undercover customers

One of the most revealing ways to learn about customer satisfaction is to experience what your customers do when they interact with your company. You can speculate about what this experience is like, but until you’ve been on the receiving end of your own goods or services, you really don’t know. That’s the underlying theory behind the undercover customer concept: Find out firsthand what it’s like to be one.

This technique--also known as “mystery shopping”--is widely used in consumer industries such as retail, restaurants, hotels, mail order businesses and car dealerships. Someone impersonates a customer on behalf of the organization, paying close attention to product or service quality. The undercover customer can be an employee of the organization or someone hired for this sole purpose. In either case, employees within the organization must not know how or when the undercover customer will visit. It’s important to experience the transaction in the same way a normal customer would.

What attributes might be examined? Some typical variables include:

Timeliness of service

Accuracy of information received

Courtesy of personnel

Efficiency of the fulfillment process

Aggressiveness of sales personnel

Conformity of the final product

Adherence to organizational policies

There’s no doubt that this method can deliver valuable information about the customer experience. In fact, this is the only method that delivers customer perceptions firsthand. Other methods filter the customer encounter on a second- or third-hand basis. With undercover customers, you actually see, hear, smell, touch and live the experience, which provides pure, unadulterated information.

Organizations walk a fine line when using this method, however. It’s often perceived as a covert surveillance tool that puts people at a disadvantage. This can destroy morale and motivation and send valuable employees scurrying for other jobs. Clearly, the undercover customer must focus on the organization’s system, processes and procedures. Employees shouldn’t feel that they’re being persecuted. Everyone must understand that the undercover customer process will help the company better serve its customers; it’s not about trying to get people in trouble.

Some states--Nevada, for example--require that undercover customers act under the supervision of a licensed private investigator. Unfortunately, these statutes only reinforce the idea that employees are under surveillance. It’s worth examining your state’s laws before implementing an undercover customer process.

A number of companies specialize in providing undercover customer services. One of the more prominent is Mystique Shopper (www.mystiqueshopper.com) in Uniondale, New York. The company’s founder and president, John Saccheri, says mystery shopping:

Reveals patterns about the effectiveness of the organization’s processes

Enforces a certain level of discipline among personnel. When the boss is around, everyone is on his or her best behavior. When the mystery shopper is enlisted, the boss is always around.

Saccheri emphasizes that the organization must analyze the trends and not take action on isolated incidents. “When problems occur, they’re often system problems as opposed to people problems,” he explains. “Address the system, and people issues usually go away. Of course, it’s occasionally the case when the problem is a person, and mystery shopping will reveal this.”

The undercover customer’s findings are often positive, Saccheri notes. It’s a learning experience, and there are many opportunities to recognize people for good performances. “When the process is presented in a positive light, employees seem to enjoy and embrace the undercover customer concept,” he says. “The trick is in the way management uses the results. As with any tool, this can backfire if it’s used incorrectly.”

It’s possible for organizations to experiment with this method without outside assistance by simply using an employee as the undercover customer. This brings with it some special challenges, though. Potential problems include:

Employees can be recognized. In this case, they aren’t truly undercover. The employees serving them will recognize the undercover customers, and this will affect their behavior.

n Employees know how everything works. They’re more likely to maneuver within the known constraints of the company, taking into consideration weaknesses and obstacles. They’re less likely to make the same mistakes, and thus cover the same dissatisfactions, as real customers.

Employees have a hard time being objective. They already know who they like, who they don’t like, the organization’s problems, etc. Politics are rampant. Third parties are more likely to approach the task in a balanced and neutral manner.

Finally, the organization must design a tool for capturing undercover customer perceptions. The tool can include open-ended questions (such as: “What did you like?” and “What did you dislike?”), specific questions (“Were you served within 15 minutes?”), scaled questions (“Please rate courtesy of your salesperson on a scale from one to five.”) or a mix of all of these. Clearly, designing the tool presents its own challenges. As with so many other customer perception methods, it makes sense to start simply (i.e., with open-ended questions and specific questions), then move on to more sophisticated rating methodologies after the process matures.

Be creative

Don’t assume that a survey is the only valid tool for assessing customer satisfaction. Other techniques are also available; the approaches are limited only by your organization’s creativity. Researching what other companies have done can help you brainstorm alternatives. Call reports, field reports, comment cards, complaint systems, warranty analysis, after-order follow-ups, customer hospitality days, focus groups and undercover customers can all deliver valuable information. Try a method, and be prepared to switch gears when necessary. And no matter what methods you use, don’t forget to take action on the results.

About the author

Craig Cochran is a project manager with the Center for International Standards & Quality, part of Georgia Tech’s Economic Development Institute. He’s an RAB-certified QMS lead auditor and the author of Customer Satisfaction: Tools, Techniques and Formulas for Success and The Continual Improvement Process: From Strategy to the Bottom Line,

both available from Paton Press (www.patonpress.com). CISQ can be reached at (800) 859-0968 or on the Web at www.cisq.gatech.edu. Special thanks to John Saccheri, Deborah Holden, Troy Clarida, Ann Barrelle and Lynn Ann Pall for their assistance with this article.

About the series editor

Denise Robitaille is a consultant, writer and trainer. She’s also a lead assessor and certified quality auditor. Much of her work involves assisting companies with implementing and maintaining ISO 9001-compliant quality management systems. She is the author of The Corrective Action Handbook, The Preventive Action Handbook and The Management Review Handbook, all available from Paton Press (www.patonpress.com).