by Richard J. Schonberger

In 1988, when total quality management reached its zenith and W. Edwards Deming and Joseph M. Juran strode like gods across the business landscape, the Malcolm Baldrige National Quality Award was launched. At the time, big industry was sold on the idea that quality can make a substantial difference, and the academic community was warming to it too. The American Society for Quality Control (which later dropped the word "control" from its name) helped to administer the award, which suggested, at least symbolically, that quality would be the award's foremost concern.

 The Baldrige Award's original scoring breakdown mostly fits with that suggestion. In the 1988 point-summary listing, the word "quality" was included 13 times. But what a difference 13 years make: In the 2001 scoring summary, "quality" doesn't appear at all, though quality-related terms appear sporadically among the 17 pages of detailed criteria.

 What happened during those intervening years to diminish quality's presence in the criteria? This article takes a look at where the Baldrige Award is heading and why. No one wants to imply that Baldrige winners are unworthy. The award process--for all its faults--is rigorous and fair, and the winners--for the most part--are highly deserving. But the award itself has strayed a good deal from its initial emphasis on quality. Listed in the table on this page are more specific conclusions.

Assessing the Baldrige's Shifts in Focus The award's focus has been shifting from quality management to general management.   Recent scoring criteria give increasing weight to success factors that are popular but untested over time and less weight to the quality sciences.   The award has turned sharply away from prescriptive criteria (i.e., what the candidate company should do) and toward probing, open-ended questions that might invite practices that weaken rather than strengthen competitiveness.   The award's increasing emphasis on business results introduces means/ends confusion.   Progress made toward instilling a belief in the primacy of managing for continuous process improvement is set back when process management receives increasingly fewer points in the award's scoring.   The award has become a compromise aimed at pleasing various interest groups, hampering its original quality-centered and customer-focused purpose.

Changes in criteria

 Each year since its 1988 inception, the Baldrige Award has seen incremental changes in its criteria. These have steadily moved the award away from the ideals of the quality management and quality engineering communities--away from "pure play" quality. Take leadership, the first of seven categories for both the 1988 and 2001 Baldrige criteria. The 1988 wording includes "addressing quality issues," "improved product or service quality" and "continuous improvement and total quality." The 2001 leadership criteria contain about four times as many words, but none of them are markers of quality. Instead, the 2001 language addresses leadership from a general-management point of view, for example, "How do senior leaders set and deploy organizational values, short- and long-term directions, and performance expectations…"

 The question is reasonable, but it differs widely from 1988's emphasis on quality. The same goes for most of the language in the 2001 document. It offers a strong general-management flavor but doesn't retain many quality ideals.

 The first Baldrige Award document places the "quality heartland" in category 5.0, Quality Assurance of Products and Services (150 points), as well as in category 6.0, Results from Quality Assurance of Products and Services (100 points). These two categories accounted for 40 percent of the total pages of the 1988 scoring criteria and 25 percent of the 1,000 possible points. Wording included such common quality concepts as "use of quality improvement process during the last three years," "evidence of periodic tightening of quality objectives," "level of scrap, rework or rejects per unit of output or per time period," and "percent change in warranty or guarantee claims during the last three years."

 In 2001, the closest counterpart of these categories would be category 6.0, Process Management. That category, ratcheted down to just 85 points out of 1,000, retains a few phrases reminiscent of quality: "inspection, tests, and process/performance audits to minimize warranty and/or rework costs." But, notably, the word "costs" was added at the end. In 1988, the wording was "number of" and "level of" but never "cost of." Injecting cost into the dynamic draws the category away from pure-play, and away from those true quality believers who subscribe to the Philip Crosby phrase, "quality is free" (i.e., it's beyond the need for monetary proof).

 The Baldrige Award scoring criteria's latest revisions also follow the trend away from quality. Changes from 2000 to 2001, for example, occur in two main areas, both in quality's heartland:

Subcategory 6.3, Supplier and Partnering Processes (15 points), was eliminated, and subcategory 7.4, Supplier and Partner Results (25 points), was dropped. Also, the point value for subcategory 6.1, Product and Service Processes, was reduced from 55 to 45 points.

 In addition to the shift away from quality language and toward general management, the latest Baldrige criteria include a striking addition: question marks. Nearly every sentence is a question. Under subcategory 6.1, Product and Service Processes (45 points), is: "How do you design your production/delivery systems and processes to meet all key operational performance requirements?" This construction, though not closing the door to quality, certainly opens it up to just about anything else. A company could, for example, make maximum output as its single-minded operational-performance requirement. Before the modern quality era, it wasn't at all unusual for companies to do just that, letting quality fall where it may.

 In 1988, the award was prescriptive. Examiners knew what companies should do and marked them high for doing it. Now, like news reporters, examiners ask questions and must judge the veracity and cogency of the answers. Previously, examiners knew their subject better than the company's managers, who would learn about improvement through the examiners' assessments. Now, examiners are in the weaker position of listening and learning from a company in order to make their assessments.

Missing language and misleading scores

 That brings us to another critical concern. The direction in which the Baldrige criteria are drifting raises the question, "Do we really know what we know?" That is, it seems possible for a company to score high points and possibly win an award without deserving it. Say that Acme Corp., along with its recent record of very strong business results, has a senior executive team whose top priority is winning a Baldrige Award. Examiners visit the company, hear impressive presentations from highly placed people and see fine charts all over the walls. Acme scores very well on business results (450 points possible), leadership (120 points) and strategic planning (85 points). Though it receives only average points for applying quality management tools and techniques, its total score is high. Acme wins a Baldrige.

 What's wrong with that? Points are points. No, they aren't, and here's why. Theories about managing an organization are divided into those that are sound and proven and those that are shaky. Quality management concepts and tools fall in the former camp. Leadership and strategic planning, categories 1.0 and 2.0 in the 2001 criteria, remain in the latter.

 Leadership comes in 57 varieties, from the quietly competent technocratic manager to the empathetic, humanistic leader to the hard-driving, results-at-all-costs capo di capo. Leadership styles range from hands-on and seat-of-the-pants to tightly controlling everything. For each, there are numerous effective examples and plenty of admiration. This doesn't diminish leadership as a valued Baldrige category. Although much of 1988's language under leadership referred to leaders' involvement in quality, not much of that survives in 2001. The same can be said for strategic planning, which in 1988 was called "strategic quality planning."

Confusing ends and means

 Scoring high points for business results is a more problematic issue. To be sure, any organization's ultimate goal is to achieve sustained, exceptional business results. The quality sciences are instrumental in that regard; leadership is, too, no matter how fuzzy our notions are about it. But for many companies, a more vital factor is innovation, especially as applied to generating new products. Despite this, we know little about managing innovation. It's all too easy for a company with an excellent record of new-product introductions to bet on the wrong horse and suffer for it in the marketplace.

 Formerly high-flying Baldrige Award winners Motorola and Xerox are cases in point. Motorola owes much of its recent misfortune to its failure to choose the right mobile-phone technology; Nokia cleaned up. Xerox chose to stick with large, supposedly high-margin printers, while Hewlett-Packard walked away with a huge, surprisingly profitable market for consumer PC printers and supplies. If Motorola and Xerox were to seek a Baldrige Award today, their recent spates of poor business results would yield point totals nowhere close to award-winning numbers.

 Low points for business results clearly indicate just one thing: poor business results. The causes, however, are murky. A panel of experts could argue all day as to whether the two companies had lost their edge in leadership, marketing, organization, engineering, financial management (in Xerox's case), lobbying (in Motorola's) or plain-old luck.

 One more problem: As the award's balance tilts ever more toward business results, redundancy and confusion factors slide in. The old Baldrige Award concerned itself with what companies did to manage and improve processes for customers' benefits. The new bent is toward outcomes: If the results are there, we think, surely the processes are well-managed and continually improving. We should know better. Not distinguishing means from ends introduces confusion and uncertainty. From a practical standpoint, that muddling can add to companies' time and costs of pursuing the award, and complicate examiners' roles as assessors and scorekeepers.

All hat and no cattle

 Then there are the Baldrige stock indexes, which consistently certify that Baldrige winners' shares outperform other organizations' on the S&P 500.1 And why not, when 450 of the award's 1,000 maximum assessment points are chalked up for business results? Fine performance in the stock market will be guaranteed when the Baldrige is awarded based on best business results; it's tautological.

 Thomas Johnson and Anders Bröms mount a strong attack on management by results in their book, Profit Beyond Measure (Free Press, 2000). Its central idea is managing by means; the quality counterpart is process management. The old Baldrige criteria emphasized creating capable processes, ensuring their alignment with customers' current and future needs and continually improving those processes. The new focus reverts, in part, to the results-oriented mindset that should've been laid to rest. Unfortunately, it hangs on, stoked by the award's changes in emphasis and at the expense of continuous process improvement.

 I visited one Baldrige winner while leading a group of foreign executives on a study mission in the United States. This group came away only half impressed with the company. They thought its presentations and trend charts were excellent but left wondering about its supposedly sound foundation of quality basics: We didn't receive altogether satisfactory answers to some of our questions concerning the latter. The expression, "All hat and no cattle," came to mind.

What happened?

 There are reasons for the Baldrige Award's about-face--from quality to general management, from hard science to whatever seems to work, from prescriptions to questions and from means clarity to means/ends confusion. As a prestigious national award, the Baldrige is beholden to certain special interests. It must compromise on its focus in order to offer something to each major interest group, four in particular:

  The quality community. This is still the award's main constituency, supplying most of its support, examiners, lore and impetus for use. The community believes that continually improving processes toward customer satisfaction is the key to every genuine, long-lasting measure of success.

  Executives. The Baldrige must court these individuals because they have the power to veto their organizations' award applications. Thus, it's tactical to include as Baldrige criteria various characteristics of the executive art, some of which are controversial, such as maximizing shareholder value or executive compensation vs. maximizing service. Executives' considerable influence is reflected by the ease with which their favored programs show up in the Baldrige criteria before they've stood the test of time. Two such examples made their appearances during the mid-1990s: "stretch targets," last seen under subcategory 4.1 in 2000 but withdrawn in 2001; and "economic value," from subcategory 7.2 in the 2001 criteria.

  The investment community. This group's interest is in short-term financial results, which conflicts with the longer-term, build-the-infrastructure affinity of the quality community.

  The news media, general public and political community. The general public and, by extension, the political establishment, will react to anything that's expounded in the news media, however sensational or unfair. The Baldrige criteria contain a rather complex mix of measures. So when any one of them (e.g., statistical process control or total quality management) receives bad press, other positive measures, such as beating the stock market, might be trotted out as a rebuttal.

 The last three categories support a "let's see the business results" disposition, based on the idea that quality is necessary but not sufficient. This is true, but beside the point. A quality award should be a quality award. Those who want an award for business success should endow another one. Business magazines regularly publish best-companies lists--biggest, highest-valued, best to work for, most environmentally responsible and so on. These publications feature best leaders as well; why clutter the Baldrige Award with them?

Role reversals

 Here's an irony: While the Baldrige Award, the European Quality Award and other related honors are moving away from quality, the ISO 9000 series is coming home to quality. Purists in the quality community have sneered at the ISO 9000 series, contending that it's too much a documentation exercise. However, the sneering may soon taper off: ISO 9001:2000 has made a sharp course correction, veering toward process improvement just as the Baldrige and European awards are going the other way. Though of the opinion that the ISO 9000 series is too broad in scope, author Tito Conti notes that it "finally incorporates many of the features of the so-called total quality management or excellence models."2

 ISO 9001:2000 is based on eight principles. Notable among them are customer focus, involvement of people, process approach, factual approach to decision making and mutually beneficial supplier relationships.3 These all could have come straight out of the W. Edwards Deming playbook.

Take action

 For all its flaws, the Baldrige Award still puts quality in the public eye. It still encourages companies to better themselves. It continues to promote sharing of best practices. And because the award isn't set in concrete, perhaps critiques of it, such as this one, will contribute to a course correction.

 As for the quality community, perhaps it will become agitated enough about the Baldrige's current drift to take some kind of action. We could, for example, work within professional organizations to draw the award back toward its quality roots. Or we could let the National Institute of Standards and Technology, the award's founder, take care of the Baldrige and instead create a new pure-play quality award.

 This article has raised some of the issues. I hope others will take the baton and run with it.

References

1. In 1995 the U.S. National Institute of Standards and Technology set up a hypothetical stock market fund made up of Baldrige Award recipients. Each year, $1,000 is "invested" so that those companies' stock performances may be compared with the S&P 500. One or more other Baldrige stock indexes have similar aims.

2. Tito A. Conti. "How to Find the Correct Balance Between Standardization and Differentiation." Quality Progress, April 2001, pp. 119–121.

3. David Drickhamer. "Standards Shake-Up." Quality Progress, March 5, 2001, pp. 37–40. The other two principles under ISO 9001:2000 are leadership and system approach to management.

About the author

 Richard J. Schonberger is an affiliate professor at the University of Washington and president of Schonberger & Associates Inc., which provides seminars and advisory services to industrial and service organizations. Best known as the originator of the term and concepts of "world-class manufacturing," Schonberger is the author of more than 100 articles and papers and eight books, including Let's Fix It! (Free Press, 2001), from which this article is adapted. E-mail him at rschonberger@qualitydigest.com .