11
The Pilot Class and Beyond
What’s Inside This Chapter
This chapter describes the next phase in your course development, the pilot class, including:
• helping out the SME-turned-trainer
• development opportunities from the pilot class
• post-course maintenance.
The chapter concludes with two worksheets to guide your evaluation and course updates.
11
The Pilot Class and Beyond
At this point, your course has been accepted, and you’ve made it to the pilot class stage. As happy as you may be, your job is not quite done. You need to first be able to effectively transfer the project to the instructor of the course, then you need to observe and analyze the course in action to determine whether your objectives and design are sound. In addition, you should put some steps in place so your material can be more easily updated and kept relevant. These items will be discussed in this chapter.
Helping Out the SME-Turned-Trainer
Sometimes, SMEs also double as instructors of technical training classes. While there are rare times when a complete nonexpert teaches a technical topic or when a SME is permanently paired with someone who specializes in classroom instruction, you will find these to be the exceptions rather than the rule. Most of the time, a SME will instruct the class themselves. As such, there are certain things you can do to help the SME-turned-trainer both before and during the pilot class.
Do a “Dry Run” Before the Pilot
You can help the SME-turned-trainer by having a run-through of the course materials before the pilot class. Explain the rationale of the organizational structure, the exercises, and any other particulars of the class. If possible, have the instructor practice the presentation or complete portions of the exercises.
You should go over course content verbally with the SME-turned-trainer, but you might also consider having an instructor’s guide printed out. Creating an instructor’s guide can really help ensure that your course will continue to be delivered as originally designed and developed. This instructor’s guide can be as simple as a one-page typed schedule, or it can be as detailed as a full-blown facilitator’s guide with descriptions, rationales and answers to exercises, notes included for the PowerPoint slides, and so on.
The course dry run is also a good time to review adult learning principles. If your instructor seems hesitant about interactivity or any of the other instructional features of the course, you might provide resources that prove that adult learning principles are tested and sound. It can ease your instructor’s mind if you explain that you will be there to assist in the class (for example, by helping to explain exercise instructions or to provide guidance in keeping the course on track). This not only can help calm your instructor’s nerves about trying something new but will also prepare the instructor for times when you may stand up and help out during the class.
Resist any urges to change material—unless it is incorrect—at this stage if the pilot class will be happening sooner than would allow for completion of the requested changes. Ask the SME to give the course a try as it is and then explain that you’ll be open to removing anything that doesn’t work after the pilot.
Think About This
“The pilot class can give you an opportunity to demonstrate a great new exercise or technique to your instructors.
“One time, I worked on developing a new introductory course for the oil services industry. The instructor was very experienced and knowledgeable but stuck in a slide-show lecture rut. I designed a fun, hands-on exercise that required the learners to work in groups using basic craft supplies to create a simple oil field tool called a “Poor Boy” basket, or finger basket. As this type of exercise was way outside my instructor’s comfort zone, he fought against its inclusion in the class, but grudgingly allowed me to demonstrate it during the pilot class.
“Well, the learners loved the exercise! They thought that it enhanced their understanding of the tool and that it gave them a fun way to break up the monotony of the afternoon lecture. Once my instructor saw the success of this exercise during the pilot class, he became much more receptive to my ideas for other classes.”
—Stefanie Matta, Contract Instructional Designer
Assistance During the Class
It is important for you to attend this pilot course. You must be able to observe the exercises and the lessons and see how the audience reacts to them. Don’t be afraid to stand up and help the instructor during the pilot course. If you notice the instructor getting completely off track, ask if the class can have a break and privately discuss this with the instructor.
If you have to jump in to help, it will usually be when the exercises occur. Flag the exercises the instructor has trouble with during the pilot. Make a note to add additional detailed written instructions and possibly even a sample completed problem and answer for the learners to view before completing the exercise themselves. These types of things can augment some of the instructor’s verbal directions.
In addition, you may need to rethink whether to include certain kinds of exercises in the course. You may have created the best exercise ever, but if the instructor doesn’t use it, the exercise will not be effective. The intricacies and detail included in administrating the exercises you design should match the facilitation skills of the instructor. Some instructors just may not have the skills or motivation to try an exercise with a lot of steps. In that case, the exercise may need to be replaced with something else.
Noted
Sometimes instructors will be nervous to have someone in their class observing them. Be conscientious about not appearing as a judge or critic.
Approach instructor tangents just as you did when developing and gathering material with your SME: warily, but with some leeway available. Sometimes tangents occur because the instructor has noticed a hole in the content and that a new objective is necessary. Sometimes instructor tangents are OK—they may answer learners’ questions and aid in their understanding. It will be up to you to determine whether the tangents are ambling mindlessly or are fit for your purpose.
Effectively briefing and supporting the SME-turned-trainer is just one aspect of the pilot class. The next section discusses how to analyze the parts of the pilot class so you can further develop your course.
Development Opportunities From the Pilot Class
During the pilot course, you and the instructor should be analyzing whether:
• Course objectives are met through the training content provided.
• There is a logical flow of material.
• Technical explanations are complete.
• Unnecessary information is removed.
• Course exercises are challenging, useful, and relevant.
Attendees in the course are necessary sources to help you determine whether the conditions listed describe your course. Attendees can be polled or pre- and post-tested to determine whether your course objectives really are on track and whether your design of material actually helps to meet course objectives.
When considering input, you have a couple of choices for the pilot class: You can tell the learners the course is a pilot run, or you can avoid letting on to anyone that this is the first time that this course has been taught.
Option 1: Tell the Learners It Is a Pilot Run
Sometimes, if a course is highly anticipated, it will be next to impossible to keep it a secret that this is the first time it has been run. In addition, many instructors choose to explain to learners that the course is a pilot run because those instructors are not confident with the material, so saying it is a pilot serves as a disclaimer that makes the instructors feel better and above potential criticism.
A benefit of sharing that it is a pilot class is that you can request feedback and hold periodic open and honest discussions about whether course content is effective. You are attempting to enlist the learners as co-creators and trying to make them feel invested in the outcome of the course. If you can achieve this, learners’ feedback will be especially valuable.
A possible drawback of this is that the learners may not be as objective with their comments and evaluations; they may be more forgiving because they may not think the content is “final.”
Option 2: Don’t Let on That It Is a Pilot Course
This allows you to get more objective reactions of course content from learners. You get to see what their reactions would be if the content presented was the finalized version, including what portions learners like and what portions may need more work. Of course, you will probably already have ideas about these things, but the feedback is still quite valuable and more “pure.” Often, you will be surprised with at least some aspects of what learners thought was effective or not effective.
Regardless of which route you choose, it is important that you tap into learners’ reactions during the pilot course. This will give you a chance to see how learners respond to the material, both positively and negatively. Keep an eye out during the class to see where they struggle and where they are not engaged. You can document questions learners have and different content areas that appear difficult to understand. Don’t be afraid to ask the learners directly for feedback. You should be tracking everything possible. Speaking to the target audience firsthand can help you to figure out what works and what does not, and which objectives are needed and which are not. You will probably find that the learners have good suggestions for activities and content, and many will even possess hidden gems of relevant technical information.
As the pilot class proceeds, you should analyze objectives, content, areas for interactivity, learners’ reactions, their questions, and extra things the instructor says (things not mentioned in the presentation or the materials). Here are specific items to consider when sitting through a pilot class:
• Are learners engaged in all sections of the course? Is there something we could add to further engage learners?
• Does the target audience feel all objectives were met? That all objectives were necessary? That any important objectives were missing?
• Do learners express positive ratings or comments about the course structure and content?
• In which areas of the course did learners have a lot of questions? Were these questions answered by the current course material?
• What new explanations did the instructor include as the class occurred? Do those explanations need to be formally included in the content?
• Did learners find any inconsistencies or mistakes in the material?
• Did learners reference other resources that might be useful to include in the course?
• Are there any areas in which we can include a more complete or more updated example in the material?
• Are there any ways we can make the learners more responsible for their own learning?
• Do we make any general statements or assumptions in the material that are confusing to learners? How should we adjust the text, or what specific examples can we include?
From the information you gather from the pilot class, you should be able to make one more major update and then be done with your course (or at least be done with the first version of it!).
Other Opportunities From the Pilot
There is nothing like a pilot class to find the final pieces of missing information you need. For example, in pilot classes you could:
• Give learners exercises to complete that—despite your best efforts—you were not able to get answers for yet. In cases in which you have extreme trouble getting answers, it is possible you will obtain them from learners during the pilot. After they complete the exercise and share their answers, you can document the correct input and make these answers part of the permanent instructor’s guide.
• Ask learners in the pilot course to create questions for a final exam. At the end of the class, divide the learners into groups and assign each group a course objective. Ask learners to create five test questions based on what they have learned about their assigned course objective. For the pilot class, you can use these questions as a trivia game, which temporarily functions as the final exam. After the pilot, you should be able to use these questions as a starting point for creating a permanent written exam for the next class.
These examples are a little scrappy, but sometimes measures like this are necessary. You can use the pilot as a way to find the rest of the technical information you are seeking for a course. It should be noted—importantly—that if you are going the route of an unfinished or unpolished pilot course, you must be aware of how you advertise the course, who attends, and what kind of “course credit” you give.
After you have updated your course based on the pilot class, your next step will be saying goodbye to your course for good … or at least for a little while.
Post-Course Maintenance
It is important to keep your course updated and relevant; yet you must also be ready to say good-bye to a project so that you can move on to your next priority. Post-course maintenance can be a tricky thing, but there are measures you can take to keep a project “closed” for an appropriate amount of time while still providing updated content to the learners.
Dealing With “Tweaks” in Material
It can be next to impossible to get your course perfect. Your SME (or you) will probably want to make a tweak here or there. Constantly opening and closing a project for small details, however, can be a major disruption in your work process and efficiencies.
You can meet this “tweak” challenge by logging all the small changes needed for a course and then waiting to deal with them until enough have built up to validate a major overhaul of the course. For example, as the course continues to be taught, provide a worksheet for your instructor to log any improvements they think could be made. Set up an electronic file in which the instructor can enter new information and requested changes. Keep this information as long-term data to be incorporated into the next major redevelopment of the course.
Basically, you should set up an electronic location for the instructor to keep track of eventual changes they want made, and then periodically return to the class once enough major changes to the material are needed to justify reopening the project.
Keeping Content Updated
It is important that technical content stay up-to-date. One suggestion is to provide your instructor with a sample addendum worksheet template so they can list additional important updates, changes, and so on that come up after launch. You can then give this addendum worksheet to learners along with the regular course materials. This allows the instructor some control over keeping the content up to date and prevents you from having to jump at every minor edit.
Noted
“It’s not you, it’s me.”
If you have been effective in establishing a partnership with your SME, it can be especially difficult for your SME to understand that you have closed the project. In some cases, the SME may feel disappointed or let down that you are no longer actively working on the course. If this is the case for you, first of all, cheer up! If the SME misses you on the project, this indicates that you must have done a good job and that you were successful in establishing a team atmosphere. Second, you can ease the pains for your SME by explaining that it is nothing personal toward them or the project; rather, it is your own workload that necessitates the closure. Assuring your SME that steps are in place to keep track of eventual updates can help as well.
Getting It Done
As happy as you may be to have made it to the pilot course stage, your job is not done. You need to first be able to transfer the project to the instructor of the course, then observe the course in action to determine that your objectives and design are sound and relevant. The pilot course is an opportunity to test your development project. As the pilot class proceeds, you should analyze objectives, content, areas for interactivity, learners’ reactions, their questions, and extra things the instructor says (Worksheet 11-1). In addition, you should put some steps in place so your material can be more easily updated and kept relevant (Worksheet 11-2).
Saying goodbye to technical training projects should be a happy event; from this day forward, you will have successfully provided the organization with a useful and relevant training course that can assist employee performance.
Worksheet 11-1. Pilot Course Evaluation—Developer and Instructor
Use this worksheet to evaluate the pilot course. You can ask your instructor or SME to fill out the worksheet as well.
Based on the Pilot Course
Objectives are:
• appropriate to the target audience
• relevant to the organization and business strategy
• specific, measurable, achievable, realistic, and time-bound.
Information is:
• complete
• accurate
• concise.
Exercises:
• meet course objectives
• engage learners
• include clear instructions
• are easy to administer.
In regard to course structure:
• overall organization of the course works
• flow of each section makes sense
• length of the course is appropriate
• level of content is appropriate.
Additional comments:
Worksheet 11-2. Sample “Updates to Content” Worksheet
This is an example of a worksheet that you can give to your instructor SME to allow them to provide learners with updated information related to course content that has not yet been formally incorporated in the course materials.
As you know, technology is always changing. Please note the following updates to the course material. Thank you.
Section 1
Section 2
Section 3
Additional comments: