by Deborah A. Sadowski and Mark R. Grabau

If you’re thinking of applying process simulation to help you make an informed decision about a project or system, take comfort in the fact that you’re not the first person to do so. Process simulation is a proven tool that, when applied correctly, boosts the quality and efficiency of systems and operations.

However, whether you’re new to the process or routinely use simulation to develop ideas and/or solve problems, it’s impossible to avoid the numerous pitfalls that any simulation project presents. The following tips will help you recognize and sidestep the worst of these and allow you to concentrate on obtaining the best results for stakeholders. As any successful simulation analyst will tell you, there are three critical keys to achieving your simulation goals: viewing a project as a whole rather than a model-building exercise, establishing attainable and well-defined milestones, and keeping an eye on the outcomes of the projects and decisions to be made.

Much has been written about modeling/analysis techniques and the software commonly used to perform simulation studies. The methods and supporting tools you select to do the work will contribute to the ultimate success of your project. However, the many supplemental aspects of conducting a study can influence your likelihood of success just as much as those thoroughly discussed methods and tools. Let’s take a look at some of those peripheral imperatives.

Three steps to simulation success

In the best scenarios, a successful simulation project is one that delivers useful information at the appropriate time to support a meaningful decision. Much of the attention paid to simulation studies focuses on getting the information. However, the other two legs of this triad—timing and the effort’s impact on decisions—are equally important.

n Find the right information. A simulation study’s ultimate purpose is to make more-informed decisions. The simulation project collects data that help to answer what-if questions through experimental exercises on system models. A focus on collecting the right information is often lost in the complex and time-consuming activities involved in performing the project.

The most important aspect of finding the right information is to put yourself in the position of the target audience—the decision makers. Think about what they need to know and why they need to know it, and do so in the context of what they’re going to do with this information to deliver value to your business. Their view of the system may be different from yours, so be careful to translate the data you collect into the information they desire.

n Choose the right timing. Deciding when to deliver meaningful information is critical to a project’s success. A high-fidelity answer that arrives too late to influence a decision is inferior to a rough-cut estimate that’s presented in time to help.

Also note that timing applies throughout a study, not just to its delivery. If you can provide preliminary insights into a system’s behavior early in a project, the owners of the design might change the options they’re considering or adjust the focus of the simulation efforts.

n Provide the right context. For a project to succeed, its results need to influence some decision. An accurate and detailed simulation model, robust statistical analysis, and eye-grabbing animation all completed on time are of little value if they aren’t delivered to the right person in the right context.

This is where your role as an analyst—not just a model builder—is crucial. The instincts you’ve developed about the system are likely as, or more, valuable than those of the experts with whom you’ve consulted. Although the people who perform or implement a process can master a great amount of detail concerning their particular system subset, a simulation analyst can offer both an informative overview and an understanding of the process details that will help identify relationships or risks that might not be apparent otherwise. Use this perspective to ask questions—not just answer them—and to emphasize your study’s important results so that attention is focused on conclusions that result in the greatest improvements.

Pitfalls and pratfalls

To succeed with simulation, you must first avoid certain common errors. You might not control all the factors affecting your project, but it’s invaluable to understand their importance as you plan your project and design reports and presentations.

n Tackling the wrong problem. Sometimes the biggest mistake is made at the outset of a simulation study. If you pick the wrong problem to explore, you may be setting yourself up for failure before you’ve made your first mouse click.

The most common misguided simulation studies are those with an overly ambitious or ill-defined scope. It’s difficult to figure out where the boundaries should be when studying a complex system because it often seems as if everything affects the performance parameters driving the decisions. You must decide in the early stages of a project what to exclude; although it’s hard to say no, it’s critical that you’re willing to do so, particularly when meeting decision-making deadlines.

It’s also easy to fall into the old “When you have a hammer (i.e., simulation), everything looks like a nail (i.e., career-or business-enhancing opportunity)” trap. Many problems can certainly best be resolved with the aid of simulation analysis, but other problems can be readily solved using such tools as queuing analysis, optimization or simple spreadsheet calculations. Before embarking on a simulation study, evaluate the alternative decision-support tools vis-à-vis the three success factors discussed earlier.

n Bad Timing. To increase your chance of providing the right answers at the right time, think carefully about when to start a simulation project and when to put the brakes on—even after you’ve established momentum. If the system/process designers are still considering widely differing ideas or brainstorming how to solve fundamental design issues, it may be premature to perform more than a rudimentary analysis.

It’s more difficult to identify timing problems once a project is underway. If there are regular and significant changes to a project’s nature, you might provide the best value by using simulation for very rough-cut analysis and holding off on a more detailed study.

There are also hazards to starting a simulation study too late. This often begins with a panicked call from a project manager who says that he or she “absolutely must have a simulation done, now!” If you’re presented with such a request, carefully lead the project manager through what can feasibly be completed and then negotiate changes in the project’s scope or analysis detail vs. risks to meeting deadlines.

n Missing the “data woes” warning signs. Ask any experienced simulation analyst what the most aggravating, challenging and dangerous aspect of a project is and you’re likely to hear “data” in reply. Data woes are somewhat analogous to the story of Goldilocks and the Three Bears: You can have too little, too much or just the right amount—and still find yourself in trouble.

If there are problems, they’re often because of a lack of reliable data. Yield percentages, defect rates, transfer times and other important aspects of a system’s dynamics might not already be collected for other business purposes. For systems that aren’t yet operational, these quantities might be questionable at best. Because getting this data can be time-consuming, it’s critical to establish your data requirements and identify their sources as early as possible.

In your search for data, you might find that there’s far too much information available. The particular data you need might exist, perhaps even electronically in a database or spreadsheet, but you might spend days trying to locate it amidst all the other data in the same place. In this circumstance, it’s imperative to secure help from someone who’s knowledgeable about the data and whom you can educate about your exact needs.

Be sure to understand what each piece of information really means. What you think of as cycle time, for instance, may be the standard process time in an ideal environment. The actual data stored in a cycle-time table, however, might include waiting time, breakdown times and other properties that are typically modeled separately in simulation.

n Adding unnecessary details. One of the easiest time-consuming traps to fall into is getting hooked on modeling—adding detail because you can, rather than because it’s needed. Whenever possible, keep the model simple unless you have the luxury of significant slack time in your schedule. It’s usually more important that you perform some level of analysis in a timely fashion than run the risk of having no results to deliver when they’re needed.

Be careful when enhancing your project’s animation. Playing with graphics can be addictive and can take valuable time away from other project activities such as testing and analysis.

Because it’s tedious, it’s easy to put off testing until late in the schedule, when there’s little time to address faults in the logic. Test early and often; this way, when deadlines approach, you can at least provide some rudimentary results based on a valid model.

Focus, planning and reassessment

With all of these challenges, it’s a wonder that anyone can possibly perform a successful simulation. But there are three simple habits you can develop that will substantially boost your likelihood of success.

n Establish a clear focus. A successful simulation study begins by identifying focused objectives to which all constituents agree and then establishing a reasonable scope and timeline to achieve those goals. The questions to be answered and decisions to be affected by the study should be documented and prioritized, and this list should drive the entire study’s efforts. During the project, objectives should be reviewed often and modified as needed. The project specification should also explicitly identify assumptions regarding the type of data being incorporated in the analysis runs, the level of detail in the process logic, and the meaning of inputs and outputs.

n Plan carefully and thoroughly. There’s a common misconception that a simulation study involves a sequence of steps (e.g., project definition, model formulation, verification, validation and analysis). Actually, all elements of a simulation project should be performed repeatedly throughout the effort, growing in scope as the model progresses. Schedule the project in complete phases with intermediate milestones, spaced no more than, perhaps, two weeks apart for a mid-size to large project. Milestone goals should include descriptions of model logic to be developed and to what level of detail; data requirements, including both the type and source of the information; performance measures to be collected by the model, including mockup reports; testing and validation scenarios to be passed; animation to be developed; and analysis scenarios to be performed.

Too often, the process of creating the model logic and drawing the animation consume so much of the allotted schedule that there’s little time left to test or think. Allocate time in each project segment to test the model (including “stress test” scenarios such as high demand), to collect and validate data and to run different analysis scenarios. It’s easy to work through most of a project looking only at a baseline configuration for the system; instead, be sure to exercise the model under different scenarios throughout the project. Also perform sensitivity analysis to gain insight into what really makes a difference in the system’s performance.

n Constantly review and reassess. The model and other deliverables should be reviewed often, more intensely as the project nears completion. Structured walk-throughs with colleagues and/or clients are ideal for discovering logic problems or errors in the model. In addition, the simulation team should review the model specifications, data analyses, animation, output reports and information to be presented to decision makers.

Throughout a project’s duration, flexibility is important. As situations arise such as scope changes, problems with the datacollection or lack of subject-matter expertise, the simulation team must look for new ways to solve problems or work around them. As these situations arise, their importance to a project’s goals should be assessed. This ability to look both forward and backward marks the difference between a capable model builder and a value-adding systems analyst.

About the authors

Deborah A. Sadowski is manager of customer success at Rockwell Software (formerly Systems Modeling). She has held many roles in design and development of simulation software, including vice president of development during the creation of Arena, and has conducted numerous training courses and consulting projects in simulation. She is co-author with W. David Kelton and R. P. Sadowski of the textbook, Simulation with Arena and currently serves as the chairman of the Winter Simulation Conference board of directors.

Mark R. Grabau is an Accenture executive as well as the Accenture Government Operating Unit’s lead for simulation modeling. He has more than 10 years of experience applying simulation modeling on consulting interventions in the transportation, pharmaceutical, telecommunications, manufacturing and government industries.

This article is based on materials published in the Proceedings of the 2000 Winter Simulation Conference.

Letters to the editor regarding this article can be e-mailed to letters@qualitydigest.com.