One Hand Clapping
by Dirk Dusharme
The Internet is incredible, particularly for the news and trade media. We can easily track down sources for stories, fact-check by identifying the primary information source, look up the full text of legislation on a state or federal level, interview people in other parts of the world, and on and on. It allows small publications with limited resources to do the same research as larger organizations. Including surveys.
Over the years Quality Digest has conducted dozens of surveys, including customer satisfaction, salary, Six Sigma usage, ISO 9001:2000 transition and many more. All these were possible because of the low cost to fax and, later, e-mail surveys.
Unfortunately, because of the proliferation of e-mail spam and unsolicited faxes, it has become increasingly difficult for Quality Digest and other publications to conduct large surveys. Data-gathering surveys conducted by news organizations are legal when there is no additional intent to solicit, but that doesn't stop technology or threatened lawsuits from hindering that data-gathering function. As avenues for sending e-mails or faxes have been closed off, either by technology or, shall we say, expediency, the response rate to our surveys has dropped drastically.
This month's ISO standards survey is no exception. The number of responses has dropped from a high of about 6,000 responses in 2001 to about 550 responses this year. About two-thirds of the drop is because we no longer fax surveys. The balance is due to those e-mails being blocked by spam engines or simply not being opened (we can track both of those figures).
So what's the problem? 550 responses offer plenty of data to analyze. That's true for an overall picture of a subject, where all 550 responses will be used for the analysis. For instance, when it comes to opinions on customer satisfaction with registrars, we have seen that there is little difference between looking at 550 responses or 4,000. The issue is granularity. With a small response pool, it isn't possible to dig down and dissect the data. In a salary survey, for instance, we could tell you the average salary for a certified quality engineer (CQE) in the United States, and we could give you the average CQE salary in the western United States, but it's getting increasingly difficult to give you the salary for a CQE in the western United States who works in the electronics industry. The data set is simply too small to divvy up.
The irony is that Quality Digest readers respond very favorably to the survey data that are collected via these annoying means. As a recipient of hundreds of spam e-mails a day, I sympathize with users who are inundated with spam. But I also want to know what's happening in industry--not from a press release or a survey conducted by a biased source, but by hearing from the people who actually work in the industry.
What's the solution? Honestly, we're not sure. For surveys directed at our readership, we can put a big, splashy, obnoxious, hard-to-miss page in the magazine that contains either a survey to be faxed back, or an invitation to take the survey online. In fact, look for just such a page in the March issue.
As usual, we turn to our readers for their suggestions. Quality Digest is your magazine, and we value your input. So here's a mini-survey: Do you get value from the surveys that appear in Quality Digest? Would you like to see them continue? Do you have any ideas on how to conduct such surveys that would have a decent response rate and not cost a fortune? Feel free to send your suggestions by clicking the feedback link below.
|
