Introduction to Reflecting (Defect)

and the PEP Tool

Lynn Robert Carter


Macro Lifecycle Model

At the heart of the Personal Engineering Process (PEP) is a belief that lessons from the past can be leveraged to produce better results on future projects. When people are open to the patterns of the work they do, it is easy for them to see recurring activities in their projects. These patterns of work are often called lifecycles. The most fundamental lifecycle in PEP consists of four major recurring activities:
  1. Plan
  2. Perform
  3. Reflect on actual versus plan
  4. Reflect on quality

The Customer's Perspective

Regardless of what we think about your work, it is the customer's opinion that ultimately matters. While the PEP System itself can not ensure customers will be satisfied with your product, the System can capture critical information and make it available to you at opportune times.

As with planning, gaining control over the quality of your work is a skill that is mastered by doing and can not be mastered by reading books or attending lectures. As with most skill-based activities, mastery comes from a sequence of steps:

  1. Contemplate what is required and how it can be done
  2. Perform the activity in alignment with the contemplation
  3. Compare actual performance with contemplated performance looking for deviations
  4. Evaluate the deviations to determine where or not they are significant
  5. For significant deviations, decide whether or not the impact of the deviation and/or the probability of recurrence warrants further attention
  6. For significant deviations with a high enough impact and a great enough probability of recurrence, assess the root cause of the deviation and how the activity might be done differently to correct the situation in the future
  7. Establish mechanisms likely to support changing behavior to align with the new insight

The Plan and Perform Phases of the PEP System address the first two items in this list. Reflection activities (such as Reflect (AvP) and this section, Reflect (Defect)) address items three through six. (The seventh step is also addressed by the planning and performing PEP activities, but it is discussed in the Leveraging Lessons Learned section.)

As with the other PEP System Main Pages, the Reflect (Defect) Main Page is both a progress indicator as well as a guide through the steps. As you progress through the steps, the system indicates your progress by display "Done" for those steps you have completed and "Active" for the task the System believes you should perform next. As with the other phases of the System, when a step is listed as "Done", you are still able to click on the link for that step and update information on that or previous steps. Nothing is truly finished until you click on the Finalize button or the finalize link.

Figure 1 shows a representation of the Reflect (Defect) Main Page. When you have completed all of the previous phases of the PEP process, you will arrive at this page when you click on the name of this assignment from the Course Main Page.

Figure 1

Updating the Defect Log

The first task you must perform is an update of the defect log. You must determine whether or not your professor has already logged defects against your work or whether you must log these defects yourself. To determine whether or not the professor has logged defects, click on the Defect Log - All link on the right side of the page shown in Figure 1. This will take you to the defect log. Look at the entries and see if your professor has logged defects. (It may well be the case that you have not logged defects for this assignment and neither has your professor. Should this be the case, the Defect Log - All link will not be active.)

If there are no defects in the log or a review shows that the only defects in the log are ones you have inserted, you may need to log quality defects against this assignment. You perform this activity by doing an analysis of your graded assignment. Any corrections or problems identified by your professor should be consider to be defects. Anything less than a perfect paper deserves effort on your part to understand what went wrong and effort to determine what you will do differently so you do not experience the same problem next time. The discipline of engineering is not about randomly changing things until they work. Your development as a professional depends on your willingness to recognize the need for improvement, some careful reflection on how you might improve, and tangible effort to change something.

Figure 1 shows the user clicking on the Update Defect Log link. Students do this when they have defects to log against their assignment. Figure 2 shows a sample page after the link has been pressed and the returned page has been filled out with defect information. In this case, the student it logging a defect against the assignment due to low marks earned during the Reflect (AvP) phase. If your professor has returned your assignment and there are problems marked, you need to verify whether or not these defects have been logged. If not, you will need to fill out pages similar to these, one for each defect. When you have finished entering a defect, be sure to click on the Submit button to log the defect as shown in Figure 2.

Figure 2

After each defect is submitted, a new blank defect form is presented to you. You may fill out this blank form with the next defect and continue to submit them until you have finished. After all of the defects have been captured, click on the Update capture and proceed button as shown in Figure 3.

The fields that need to be defined are clearly labeled. At this point in the PEP System process, most all of the defects will be Quality defects. (That is, defects reported to you by your customer.) If you have discovered other defects from your reflection on your actual versus plan performance that were not captured during that process, you may need to log a different type of defect.

Since you are unlikely to have corrected the defects your professor has found, those defects should be logged with a status of Open. Keeping track of what is open and what is closed can be most helpful in the future when you might consider building upon this piece of software. Closed defects imply that the problems have been resolved. Open defects imply work remains to be done.

Trying to determine when your defect generated waste can be a bit difficult to assess for quality defects. You need to consider the work you have done and try to imagine what of that work would have to be discarded should you take the time to correct the defect. With that work in mind, in which lifecycle task would the bulk of that work have been performed? This is what you should select for the Waste occurred during field.

Similarly, you need to assess when the root cause of the waste took place. Was this an understanding problem take goes all the way back into the planning phase? Is this a design or implementation problem? Being careful with this assessment. It is critical. The PEP System will use the value you select for this field to determine where this lessons should be made available in future iterations. If this is a problem that you believe goes all the way back to planning, it makes sense for the system to make this lesson available to you during planning. If this is a packaging issue, it would be wasteful to warn you about it during planning.

The amount of effort that was or would need to be discarded during the repair operation needs to be captured in the Effort wasted field. If you don't take time to think about it now, the probability is high that you will remember even less several days, weeks, or months from now. Don't kid yourself by saying the pain was so vivid you will never need to remind yourself of it. Human nature gives us both a favor and a curse. We are allowed to forget thinks, but in such a state of forgetfulness, we often do the same painful things over again. The size of the effort wasted is a critical component of information to help you determine whether or not some change might be cost effective.

It is easy to capture the Symptom. The professor may have written the words in red ink on your paper. Again, it is important to capture this information. The Root Cause can be more illusive. There are plenty of books that describe problem solving and how to move from a symptom to a potential root cause. (The Memory Jogger described at http://www.goalqpc.com/PRODUCT/MJIIprev.pdf is one of the standard references.) Regardless of how you get to the Root Cause, it is critical that you truly believe that correcting this root cause is a reasonable thing to do, is feasible, and would really result in avoiding the symptom in the future. It is a waste of your time to write down things that you know will never be implemented. Your goal is to provided help to yourself next time you go through the process. Do yourself a favor as opposed to just doing something to capture a few points from the grader.

When you have finished capturing defects and you've clicked the Update button in Figure 3, the PEP System will display a page similar to the one shown in Figure 4.

Figure 3

Assessing Defects

As with the previous reflection activity, you need to assess the significance, impact, and probability of recurrence of each defect. For the purposes of this class, any defect logged by your professor should be assessed as: Should you assess a defect at lower levels, the PEP System will not make the defect available for capturing lessons learned. Should you assess professor reported defects at a lower level, you will lose points during the grading process. To begin the assessment, click on the link shown in Figure 4.

Figure 4

Upon entering the assessment step, you are basically shown a modified version of the Defect Log. For each defect that has been captured for this assignment, a log entry will be displayed. Work your way through all of the defects listed in the page similar to Figure 5, clicking on the sequence number for each defect.

Figure 5

When you click on the sequence number of a defect, an assessment form is generated that asks you to answer the standard questions about this defect. Perform your assessment (being mindful of what you've read above.) When you've finished your assessment, click on the Save button as shown in Figure 6.

Figure 6

As you finish each defect, move on to the next. Should there me more than a page worth of defects, use the Next or Prev links to access the other sets of defects. After you have assessed all of the defects, click on the Save link as shown in Figure 7.

Figure 7

If you have rated any of the defects as:

or higher, a list of those defects will be presented to you when you click on the link as shown in Figure 8. Should you elect not to rate any of the defects at this level, the PEP System will recommend you skip this step.

Figure 8

Reflecting on Defects

In this example, the student had only one defect that matched the standard, and so the list of defects on which to reflect has grown smaller. Clicking on the sequence number link for a defect this time takes you to a page similar to that shown to you in Figure 10.

Figure 9

This is the point where you need to spend some time thinking. What information do you have wished someone had told you at just the right time during the early phases of the assignment? What would they have had to say to catch your attention? How could they have described the situation and the way to avoid it to you such that you would have actually done things differently? The information on the page shown in Figure 10 will be made available to you during future PEP activities based on when you asserted the defect occurred. Take time to capture words to yourself in a way that will be meaningful to you then.

Each assignment is different, so it doesn't do you much good to provide suggestions on how to write Cool programs better. You won't be writing any more of them for this course (other than as test data for the compiler you will be producing). You need to project into the future and think more broadly about the general lessons you should take away from this experience as opposed to just specific ideas aimed at just this previous assignment.

Figure 10

After reflecting on each and every defect in this list (and possibly other pages that you may need to use the Next or Prev links to get to), be sure to click on the Save button as shown in Figure 11.

Figure 11

Once you have finished the last reflection, the only remaining task is to capture your effort for this phase of the process. If you wish to produce more than one effort log, be sure to use the Effort Log links on the right side of the page. When you click on the link as shown in Figure 12, you begin a process of finalization.

Figure 12

If you get to the point of the page shown in Figure 13 and you are not ready to finalize the reflection, please click on the Return link on the right side of the page. If you fill out the form shown in Figure 13 and click on the Update button, the system will update the log and finalize this phase of the process. Clicking as shown below will result in you moving forward to Figure 14.

Figure 13

Figure 14 shows the Course Main Page after all of the PEP Phases have been performed. If you click on the assignment title at this point, as shown in Figure 14, the system will take you to a page where you can examine the data you captured as well as the grader's marks and comments. You will not be able to make changes.

Figure 14

Reviewing an Assignment

Figure 15 shows the Review Main page. From here, you can examine information captured about this assignment as well as information about your grade for the assignment. Should you click on this link as shown in Figure 15, the system will take you to a grade summary page that summarizes you PEP grade for this assignment as well as how you relate to the class minimum, average, and maximum for the assignment.

Figure 15

Now that you have been through the entire process, consider going back and rereading all of this documentation with a new set of eyes. The PEP System can be a powerful tool to assist you learn how to plan and how to improve both your performance and your quality. As with most things, nothing comes for free. If you invest in yourself and take time to produce well-considered reflections and lessons learned, you will see improvement by the end of the semester.


lrc@sei.cmu.edu
Revision 1.1 (1999/09/14 21:30:00)