Johnson Space Center:
A Day in the Life of an Assessor If you've never been through an assessment by a
third-party assessor, you might not know what to expect.
The experience of Lee. C. Bravener, NQA USA lead assessor,
at NASA's Johnson Space Center may help to shed
light on what an assessment is really like. It was challenging to be named lead assessor of the five-man team for the ISO 9000 preassessment of NASA's Johnson Space Center in Houston. An assignment of this magnitude meant that my work would have to begin long before the actual start of the on-site assessment. And while I must confess that my assignment was a little intimidating because of the worldwide prestige of JSC, I had utmost confidence in my NQA USA team members. JSC is anything but a standard manufacturing operation -- it is a 24-hour-a-day operation with many unique and self-contained organizational elements. I would have to assemble a diverse group of assessors with not only aerospace experience but manufacturing, research and flight operations experience. Assembling the assessor team My first task was to contact JSC's ISO representative and introduce myself, opening a line of communication in order to further our teaming alliance. Those first discussions with the client are vital for the assessment team to provide the client with value-added service throughout the registration process. And it is important for the assessor to understand the client's point of view in order to eliminate any ambiguous expectations. Second, the client needs to develop a comfort factor with the assessment team. This makes it easier for the assessment team to properly evaluate the implementation of their quality system. After contacting the client, I needed to review the assessment team's background -- their skills, special expertise and also the proposed standard elements they were to assess. For the NASA assessment, we needed a team with a broad background in computer science and aerospace research and development. One of our assessors has a military aviation background, another possesses design engineering (hardware and software) experience, while a third comes from the testing and R&D sector. A fourth assessor has high-tech manufacturing credentials, and I have both project management and extensive quality system experience. To top it all off, the entire team is quite familiar with the governmental space exploration arena. Once the team was selected, I turned my attention to preparing the assessment package. This material included the scope statement and site information, a copy of the report from the documented quality system review, a preliminary planning schedule, blank hard-copy report forms and a 3.5" disk of electronic forms. It also included the ISO 9001 standard and supporting documents (AS9000, ISO 9003-3, TickIT guide, applicable NQA USA guidance notes, etc.) and a map of the site. The package contained an e-mail of other information that the NASA JSC management representative thought would be helpful. Day one An assessor on the job has little time for anything else, even breakfast. My mind is continually reviewing the upcoming assessment, deciding what I need to do and how I'm going to do it. Our day would start with the traditional sunrise breakfast meeting. At the meeting, we discussed our proposed plan of action. Each assessor was encouraged to comment and offer suggestions that might improve our way of handling the project. With the end of our discussion and a consensus on any adjustments reached, we left for JSC. Approaching the front gate of NASA JSC, we saw displays of the Apollo Capsule and the Saturn Booster. Seeing these impressive exhibits brought home the realization that we were here to assess what is probably the No. 1 technology center of our time. We were quickly guided through the lobby by the JSC management representative and up to our workroom on the 6th floor. We were told that the opening meeting scheduled for 8:30 a.m. would be in the executive conference room on the 9th floor. During the time we were waiting to go upstairs, our team was introduced to the NASA ISO implementation team. The opening briefing I was impressed by the size of the audience waiting for us and estimated it at more than 75 NASA management and staff. The impressive turnout ranged from the NASA center director and division directors to the ISO implementation team. I sensed a mixture of curiosity and anticipation along with a touch of caution in the room. As registrars, we often forget that our mission is something new to each organization and our audience really doesn't know what to expect from us in the days that follow. With that in mind, I began my briefing by trying to set everyone at ease about the upcoming events. I explained the purpose of the activity, then detailed NQA USA's policy on confidentiality. I told the audience that our team would be drawing upon a sample of their activities to determine the implementation status and that we would provide them with a written report at the activity's conclusion. With such a large audience, I thought it advisable to modify the normal entrance briefing and attempt to hit only the high points. Later on, I conducted a second entrance briefing for the NASA management representative and his team. That second briefing included a discussion of administrative requirements, duties of the guides, administrative support, organizing an activity schedule and various routine matters. After the briefing session, we moved from the conference room to our workroom and began the first day's activity. Although a tremendous amount of preplanning had been performed before the activity, the magnitude and diversity of JSC could foil even the best of plans. Beginning the assessment I thought it best that we immediately start the active assessment because we had a lot of ground to cover. I had scheduled as my first appointment a meeting with the NASA center director and his deputy. In the company of the NASA management representative, I went down the hall to an executive suite for my first in-depth briefing. Waiting for my appointment, I gazed around at an area that was awash in space flight history. Plaques, displays and photographs surrounded me. Mercury, Gemini, Apollo, John Glenn, Neil Armstrong and Sally Ride were names and images that I knew well. My meeting with the director was a "jumping off" point for our assessment. I normally perform an interview with the senior executives and the ISO management representative at their earliest convenience. This is in response to ISO 9001, Element 4.1, Management Responsibility. Senior executives often feel uncomfortable with the fact that they will also be called upon during an audit. I explained the purpose of my visit and what I hoped to accomplish. I asked questions like: "What is the quality policy? What does it mean? How are you implementing the quality system?" As I continued, I found that the director had become quite enthusiastic about developing and implementing an ISO 9001 quality system. I found this very satisfying and wrote this in my notes along with the actual words of his responses. I make sure to capture all information and impressions; I might need them later on during the assessment. Eventually, I thought that I had enough information to verify compliance to this element and moved on. Getting down to business My next stop was an interview with the director of operations. I queried with my usual questions and, as often happens during assessments, we moved into a line of questions that led me to suspect a possible nonconformance. The director of operations, however, assured me that he had evidence ensuring compliance, which he later supplied. When this meeting ended, I went back to our workroom for an interview with the NASA management representative. Walking into our workroom, I noticed with some satisfaction that the white board in the room was covered with a very detailed schedule that incorporated a few last-minute adjustments. I still had some time before my next appointment with Mark, the assessor assigned to design control. He had some issues he had run into and wanted my opinion. Just about every aerospace company in the United States has some type of representation at JSC. These companies may contract for a particular program, provide professional staffing or use the facility for research. Because of the magnitude and diversity at this huge facility, determining who falls within the scope of the assessment can at times be very confusing. Mark and I reviewed our ground rules governing this issue and presented them to the ISO management representative. He thought that our approach toward contractors was appropriate. It became immediately obvious, however, that he was eager for a more detailed discussion. This became quite evident from his answers to my set of questions. It was my intent to determine if the ISO 9000 program was a one-person show or if it was a focal point for executive management. As it turned out, the representative had been delegated a tremendous amount of responsibility, but he had also been provided with all the necessary support he needed. Making adjustments Rarely throughout the day is there a time when the assessment is not occupying everyone's attention. During lunch, I directed our team to the schedule for the remainder of the assessment and asked for comments and suggestions. There were a few minor changes due to schedule conflicts and considerations due to workloads at JSC. Because the space shuttle Discovery was in orbit, a U.S. astronaut was on the Russian Mir space station and pictures were coming in from the Mars Pathfinder probe, I certainly did not have any difficulty in adjusting our schedule to fit theirs. I then selected the appropriate team members to cover the second and third shifts later on in the week. After lunch, I again sat down with the management representative, this time to evaluate their corrective actions relating to the quality documentation review that had been conducted the previous month. My objective was to analyze not only how they addressed the nonconformances they had received, but also whether their responses were effective. In order to complete Element 4.1, it was now time to review the documentation covering management review. I was provided with appropriate objective evidence: meeting notes, agendas, minutes and action item logs. Satisfied that the standards requirements had been met, I moved on to Element 4.17, Internal Quality Audits. The internal audit lead auditor launched into an explanation of NASA's internal audit program and presented me with his work instructions. With only a few questions, we rapidly progressed through internal audits, audit training records, training programs, a sample of records of audits performed, reports prepared and the audit schedule. Satisfied, I returned to our workroom to prepare for the day's 4:30 p.m. team meeting. First day's wrap-up To begin the meeting, I asked each team member to report on what he had experienced throughout the day. Greg, our assessor who had formerly been a military aviator, had been to the astronaut office and had reviewed their selection program. Mark told us about the space station docking simulator used to test new software. Keith had been busy auditing in the shuttle mockup area, where fixtures were being tested for the robotics arm. Kevin, my deputy, had been to mission control. After our team discussion, we had a wrap-up meeting with the NASA team. In this meeting, we briefed the "issues" we had for the day, but did not assign any level of severity (major, minor, observation), primarily because at this stage, we hadn't finalized the grading system. At times, objective evidence indicating an area of concern or eliminating an item that had previously caused concern, can appear later, sometimes at the 11th hour. The meeting concluded with some questions and answers. It was nearly 6:00 p.m. before we were able to secure the day's on-site activities. An assessor's day never seems to end After dinner, our team continued discussions of the day's activities. We compared notes to ascertain if any major problems might surface. Various theories and hypotheses were presented and examined, and some were either discarded or set aside for further investigation. Before we broke up, I gave out a few homework assignments as a result of the day's activity that needed to be completed before morning. Then I suggested a few changes in the next day's activity to research certain questions that had arisen. Back at the hotel, I checked my messages, only to find that NASA's Marshall Space Flight Center and McDonnell Douglas had called, wanting to begin discussions regarding their upcoming assessment activity. Even after I turned out the light, I was turning over in my mind some of the questions I would be asking our team members at the next morning's 7:00 a.m. breakfast meeting. Sometime after 10:00 p.m., the first day of this assessment finally came to an end. About the author Lee C. Bravener, RAB lead assessor No. Q02130, has more than 20 years of direct quality assurance experience. Prior to joining NQA USA in 1993, he helped develop and implement quality systems to comply with ISO 9000, MIL-Q-9858A, MIL-STD-1520, NHB 5300, JSC 31000 and MIL-STD-2167/2168. At NQA USA, he has been lead auditor on numerous ISO 9000, AS9000 and QS-9000 assessments covering a variety of industries, including aerospace, electronics, printing, software, heavy and light manufacturing, biomedical, testing and services. He has recently been promoted to vice president of NQA USA. |
