Improvement can be achieved in one or more of the many characteristics of any given product or process. In most situations, however, improvement primarily implies that performance is enhanced. Experimental design is one technique that can be learned and applied to determine product or process design for improved performance.

Figure 1: Performance Before Experimental Study Figure 1: Performance Before Experimental Study

 For a high-volume manufactured part, the two statistical performance characteristics that manufacturers typically aim to achieve are improving the mean (average) and reducing the variability around the mean. For improvement, our goal is to move the performance of a population of parts to the target and minimize the variability around it (see figures 1 and 2). No matter the application, performance consistency is a desirable characteristic to achieve. Performance consistency is achieved when the performance is on-target most of the time.

 An effective way to improve performance is to optimize the engineering designs of products or processes by experimental means. A structured and economical way to study projects whose performance depends on many factors is to apply the experimental method known as "design of experiments," a statistical technique introduced in the 1920s by Ronald A. Fisher in England. In the 1950s, Genichi Taguchi of Japan proposed a much-standardized version of the technique for engineering applications. His prescription for experiment designs, a new strategy to incorporate the effects of uncontrollable factors and ability to quantify the performance improvement in terms of dollars by use of a loss function, made the DOE technique much more attractive to the practicing engineers and scientists in all kinds of industries.

 In January, John Wiley & Sons Inc. published my book on DOE/Taguchi technique. Intended primarily for the self-learner, the book takes the reader through the entire application and analysis process in 16 different steps. One who learns the topics covered in these 16 steps well will be able to handle more than 99 percent of the situations common to manufacturing and production activities. Following are the 16 steps you will need to master DOE using the Taguchi approach for your own product and process design improvement.

Step 1: Design of experiments and the Taguchi approach

 A quick review and understanding of the Taguchi version of DOE is essential before diving into the subject. The purpose here is to gather a clear understanding of what DOE is and understand how Taguchi standardized the experiment design process to make the technique easier to apply.

Step 2: Definition and measurement of improvement

 No experiment that lacks the means to measure its results is complete or useful. A clear definition of objectives and measurement methods allows us to compare two individual performances, but a separate yardstick is needed to compare performances of one population (multiple products or processes) with another. In general, individual performance measures are different for different projects, but consistency is the means by which we measure population performance. Consistent performance produces reduced variations around the target (when present) and results in reduction of scrap, rejection and warranty. In this step, you learn how population performances are measured and compared.

Step 3: Common experiments and analyses methods

 A common practice for studying single or multiple factors is to experiment with one factor at a time while holding all others fixed. This practice is attractive, as it's simple and supported by common sense. However, the results are often misleading and fail to reproduce conclusions drawn from such an exercise. A more effective method for these situations is to study their effect simultaneously by setting up experiments following the DOE technique. This step should lead to some understanding of basic DOE principles.

Step 4: Designing experiments using orthogonal arrays

 The word "design" in "design of experiments" implies a formal layout of the experiments that contains information about how many tests are to be carried out and the combination of factors included in the study. Once the project is identified, the objectives and factors and their levels are determined by following a recommended sequence of discussion in a planning meeting. There are many possible ways to lay out the experiment; the best method depends on the project. A number of standard orthogonal arrays (number tables) have been constructed to facilitate designs of experiments. Each of these arrays can be used to design experiments to suit several experimental situations. This step should be devoted to learning about the different orthogonal arrays and understanding how easy it is to design experiments by using them.

Step 5: Designing experiments with two-level factors

 Experiments that involve studies of factors with two levels are both simple and common. There are a set of orthogonal arrays (designated as L-4, L-8, L-12, L-16, L-32, L-64, etc.) created specifically for two-level factors. Experiments of all sizes can be easily designed using these arrays, as long as all factors involved are tested at two levels. By completing this step, you will learn how quickly experiments involving two-level factors can be designed and analyzed using the standard orthogonal arrays.

Step 6: Designing experiments with three-level and four-level factors

 When only two levels of factors are studied, the factors' behavior is necessarily assumed to be linear. When nonlinear effects are suspected, more than two levels of the factors are desirable. Although many larger two-level orthogonal arrays can be modified to accommodate three-level and four-level factors, a set of standard arrays such as L-9, L-18, L-27, modified L-16 and modified L-32 are also available for this purpose. This step should help you learn the design and analysis of these more complex experiments.

Step 7: Analysis of variance (ANOVA)

 Calculations of result averages and averages for factor-level effects, which only involve simple arithmetic operations, produce answers to major questions that were unconfirmed in the earlier steps about the project. However, questions concerning the influence of factors on the variation of results --in terms of discrete proportion --can only be obtained by performing analysis of variance. In this step, you'll learn how all analysis of variance terms are calculated. Utilize this step to review a number of example analyses to build your confidence in interpreting the experimental results.

Step 8: Designing experiments to study interactions between factors

 Interaction among factors, which is one factor's effect on another, is quite common in industrial experiments. When experiments with factors don't produce satisfactory results, or when interactions among factors are suspected, the experiment must accommodate interaction studies. In this step, your objective will be learning how to design experiments to include interaction and how to analyze the results to determine if interaction is present. You will also learn how to determine the most desirable condition in cases in which interaction is found to be significant. Although interactions among several factors, and between factors at three or four levels, are also present, studies and corrections for interaction between two two-level factors will suffice for most situations.

 The materials in steps 1-8 prepare you for many applications in the production floor. As long as the factors you want to study are all at the same level, you're able to design experiments using one of the available orthogonal arrays. You're also able to analyze the results of such experiments following the standard method of analysis, which uses the averages (means) of the multiple sample test results of individual experiments, and determine the optimum design conditions. With the knowledge you should gather in these steps, you can indeed apply the DOE to solve most production problems whose solutions lie in finding the proper combination of the controllable factors, instead of some special causes.

 The reality, however, is that you will often have factors at mixed levels; some will be at three-level, some at four-level, and many at two-level. You also need to learn how to analyze the results for variability. Recall that it's the reduction of variability, which instills performance consistency, that we're after. The following additional steps address these items and prepare you to handle most every type of experimental situation.

 If your applications always involve production problem solving, you may find that your knowledge up to this point is quite adequate for the job. Nevertheless, you may want to sharpen your application skills before proceeding to learn about the advanced concepts in the technique described in the eight steps that follow.

Step 9: Experiments with mixed-level factors

 Experiment designs with all of the factors at one level are easily handled using one of the available standard arrays. But these standard arrays can't always accommodate many mixed-factor situations that you might find in industrial settings. Most mixed-level designs, however, can be accomplished by altering the standard orthogonal arrays. Your goal will be to learn the procedure by which columns of an array are modified to upgrade and downgrade the number of levels in creating a new column. This way, a two-level array can be modified to have three-level and four-level columns. Conversely, to accommodate a factor with a lesser number of levels, a four-level column can be reduced to a three-level, and a three-level column to two-level, by a method known as "dummy treatment."

Step 10: Combination designs

 For some applications, the factors and levels are such that standard use of the orthogonal array doesn't produce an economical experimental strategy. In such situations, a special experiment design technique such as a combination design might offer a significant savings in number of samples. This step will familiarize you with the necessary assumptions that must be made in order to lay out experiments using combination design. With this technique, generally, two two-level factors are studied by assigning them to a three-level column.

Step 11: Robust design strategy

 Variations among parts manufactured to the same specifications are common even when attempts are made to keep all factors at their desired levels. Remember, variation reduction is our ultimate goal. When performance is consistently on-target (the desired value), the customer perceived quality of the product is favorably affected. Variation is most often due to factors that are not controllable or are too expensive to control. These are called the "noise factors." In robust design methodology, the approach is not to control the noise factors, but to minimize their influence by adjusting the controllable factors that are included in the study. This new strategy, promoted by Taguchi, reduces variability without actually removing the cause of variation.

Step 12: Analysis using signal-to-noise (S/N) ratios

 The traditional method of calculating average factor effects and thereby determining the desirable factor levels (optimum condition) is to look at the simple averages of the results. Although average calculation is relatively simple, it doesn't capture the variability of results within a trial condition. A better way to compare the population behavior is to use the mean-squared deviation, which combines effects of both average and standard deviation of the results. For convenience of linearity and to accommodate wide-ranging data, a logarithmic transformation of MSD (called the signal-to-noise ratio) is recommended for analysis of results. This step will teach you how MSD is calculated for different quality characteristics and how analysis using S/N ratios differs from the standard practice. When the S/N ratio is used for results analysis, the optimum condition identified from such analysis is more likely to produce consistent performance.

Step 13: Results analysis using multiple evaluation criteria

 Often, a product (or process) is expected to satisfy multiple objectives. The result in this case comprises multiple evaluation criteria, which represents performance in each of the objectives. It's common practice, however, to analyze only one criteria at a time because different objectives are likely to be evaluated by different criteria, each of which has different units of measurement and relative weighting. When the results are analyzed separately for different criteria and the desirable design conditions are determined, there is no guarantee that the factor combination will all be alike. An objective way to analyze the results is to combine the multiple evaluations into a single criterion, which incorporates the units of measurements and the relative weights of the individual criterion of evaluation. You should devote your time during this step to learning the principles involved in formulation of an overall evaluation criterion for analysis of multiple objectives, when present.

Step 14: Quantification of variation reduction and performance improvement

 Most of your DOE applications allow you to determine optimum design that is expected to produce an overall better performance. The improvement of performance often means that either the average or the variations (or both) have improved. When the new design is put into practice (i.e., the recommended design is incorporated), it's expected to reduce scrap and warranty costs. In turn, this reduction more than offsets the cost of the new design. The expected monetary savings from the improved design can be calculated by using Taguchi's loss function. In this step, you'll learn how to estimate the expected savings from the improvement predicted by the experimental results. Further, you'll also learn how the expected improvement in performance from the new design is expressed in terms of capability improvement indexes such as Cp and Cpk.

Step 15: Effective experiment planning

 As far as the benefits from the technique are concerned, experiment planning is the most important among the different application activities. Therefore, it's a required first and necessary step in the application process. Planning for DOE/Taguchi requires structured brainstorming with project team members. The nature of discussions in the planning session is likely to vary from project to project and is best facilitated by one who is well-versed in the technique. Your effort in this step will be to learn the structure of proven planning sessions documented by experienced application specialists.

Step 16: Review of example case studies

 The application knowledge gained in steps 1-15 could be overwhelming if you didn't have immediate projects on which to practice. One way to build more confidence and extend your application expertise is by familiarizing yourself with numerous types of case studies with complete experiment design and results analysis. In this final step, you should seek out and thoroughly review complete project application reports. Complete case studies should contain discussions under most of the following topics:

  Project title or problem definition

  Project objective(s)

  Evaluation criteria and quality characteristic

  Identified factors and levels and those that are included in the study

  Suspected interactions and those that are selected for the initial study

  Uncontrollable factors (noise factors) and how they were treated

  Sequence of running of the experimental conditions

  Measured results, which represent evaluation of different objectives

  Main effects indicating the trend of factors' influence

  Analysis of variance for relative influence of the factor to the variation of results

  Optimum condition and the expected performance

  Improvement and expected monetary savings

  Graphical representation of variation reduction expected from the improved design

 Now that you have an idea about the topics and the study sequence, one question remains: How do you actually go about learning them?

 To get yourself comfortable with DOE application knowledge, you will need to understand four phases in the application process: (1) experiment planning, (2) experiment design, (3) results analysis and (4) interpretation of results. Of these, you need not --and may not be able to afford the time --to be too good with experiment design and number crunching. These are mundane tasks, so feel comfortable letting a computer program do the work for you; your focus should be to learn the practiced and proven discipline of how to plan an experiment following a structured sequence of discussion. The experiment planning process requires more the art of teamwork than experimental science. Only the experienced can describe and share methods that have worked. Look for references that describe and teach the technique through application examples.

 Both experiment planning and interpretation analysis are areas you'll want to gain control over. The nature of discussions and findings in these areas are always project-specific. As the experimenter, you'll know far more about these two areas than anyone else. Good knowledge of the project objectives, how objectives are evaluated, how the factors included in the study were selected, and so on will help you confidently interpret results from the routine analysis. You will benefit most when your reference book stresses application rather than theory.

References

1. Taguchi, Genichi. System of Experimental Design. New York: UNIPUB, Kraus International Publications, 1987.

2. Roy, Ranjit K. A Primer on the Taguchi Method. Dearborn, Michigan: Society of Manufacturing Engineers, 1990.

3. Roy, Ranjit K. Design of Experiments Using the Taguchi Approach: 16 Steps to Product and Process Improvement. New York: John Wiley & Sons, 2001.

About the author

 Ranjit K. Roy, Ph.D., P.E. (M.E.) is an engineering consultant specializing in Taguchi approach of quality improvement. Roy has achieved international recognition as a consultant and trainer for his down-to-earth teaching style of the Taguchi experimental design technique. He is the author of the texts Design of Experiments Using Taguchi Approach: 16 Steps to Product, Process Improvement and A Primer on the Taguchi Method , and of Qualitek-4 software for design and analysis of Taguchi experiments.