Developing Computer-Based Training for the Transit Industry

If your organization is thinking about adding CBT to its repertoire of training delivery methods, there are some important nuts and bolts issues that need to be considered during the initial stages of the development process. This article is intended to help identify potential pitfalls and avoid some common mistakes so that new CBT modules are as effective and successful as they need to be.

Knowing When to Use CBT
Computer-based training is a wonderful tool, but it is not the solution for all of your training needs. There will always be a place for traditional classroom-based and practical, hands-on instruction in the training landscape. You can increase the number of possible training applications for CBT by using it in blended learning environments, essentially combining CBT with traditional instruction, but there are certain limits to its effectiveness as a stand-alone delivery mechanism. As a general rule, CBT is not the preferred method for delivering the following types of training:

• Long or multiple-day training programs.
It is generally accepted that a trainee will lose interest, and their overall retention rate will decrease after about 30 minutes of CBT time. You can get around this general limitation by breaking the content up into smaller, more manageable CBT modules. That being said, you can’t simply replace a 40-hour instructor-led training program with a similar-sized CBT. There’s just too much content to put into the CBT box. Remember, depending on your network capacity and the way you’re planning to launch the program, the bigger the CBT, the longer it will potentially take to load and the slower it will run. And from a budgeting perspective, the more time, effort and money it will cost.

• Programs that normally require a lot of interaction.
Without an instructor present, the only thing the trainee can interact with is the computer screen. Yes, you can add exercises, reference material, external links, avatars (animated characters), etc. to your CBT to make the content more comprehensive and interactive. If you have the appropriate IT infrastructure, you can even incorporate real-time communication with an instructor and/or other trainees. However, the more bells and whistles you add to the CBT, the more you run the risk of allowing the program to lose its focus or becoming too long. In addition, you may also inadvertently provide the trainee with opportunities to become lost by requiring them to drill down or interact with too many layers of information. And there is also the very real possibility the trainee will have questions that you just did not factor into your content.

• Programs that require practical or operational certification.
You can certainly use a CBT program to introduce a practical subject. CBT exercises and simulations can be powerful learning tools. But certification usually involves an instructor observing the trainee physically demonstrating the required skills in the real work environment.

• Programs with a short shelf life.
Unless you’re planning to buy an off-the-shelf product or you have experience producing CBTs in-house, developing a full blown CBT program for a topic that has a short lifespan can be an expensive exercise in both time and money.

• Programs with a small audience.
Similar to the above, if your CBT program is used infrequently it’s going to take a long time for you to receive a return on your development investment.

Selecting Appropriate Source Material
As with different types of training programs, not all content is suitable for development into CBT. Source material that hasn’t been validated, contains unstable information or depends on multiple resources is not the best subject for a CBT module. Use the following guidelines to determine whether your content is a good fit for a CBT program:

• Have the users validated the content?
Many agencies are spending a lot of time and effort to document their operational and maintenance procedures. Sometimes there is a disconnect, however, between what is written in the procedure and what actually happens on the shop floor. After developing a beautifully interactive CBT program, the last thing that you want to hear from an end user is, “That’s not the way we do things down here.” Make sure that your source material matches the reality of the business flow. Performing a dry run of the procedure is a good way to determine whether it’s ready to be developed as a CBT. Using actual practitioners as your subject matter experts will also help to mitigate this problem.

• Is the content stable?
Developing a CBT takes time and money. You don’t want to waste your valuable development resources on a subject matter that changes all the time. If a maintenance procedure is in a constant state of flux or contains products and tools that change frequently, then the resulting CBT will require constant revision. You can try to avoid this to some extent by stepping back and making your instructions less specific — more process than procedure. Unfortunately this can also make your content seem vague or down right confusing. I offer the following as a slightly exaggerated example, “Turn the approved nut in the appropriate direction using the approved tool after applying the approved lubricant.” Your best bet is to look for material that contains information that is fairly static and secure.

• Does the content have a discrete beginning, middle and ending?
While not impossible, source material that contains a lot of branching or decision blocks (“if this, then this…”), or material that depends on a lot of assumed knowledge and/or supporting information can be difficult to turn into an effective CBT. The more the trainee is channelled into smaller subsets of specialized information, the more difficult it can be to return to the main flow of the lesson.

Inspection-based procedures can be particularly tricky because the trainee is often required to follow a series of “if/then” decision blocks as they zero-in on a specific problem. Likewise, a trainee who is unfamiliar with a new work environment may be required to interrupt their progress through a set of work instructions in order to gain background information from supporting documentation and/or internal links.

Both of these situations have the potential for drawing the trainee further and further away from the main thrust of the CBT. Granted, most maintenance procedures are more complicated than simply moving from A-to-B-to-C, but subject matter that jumps from A-to-Q-or-R-or-S and then back to B can be very difficult to map out from an instructional design perspective.

Incorporating the Principles of Adult Learning Theory
Without going into a separate thesis, let’s acknowledge that adults learn differently from children. Adult learning theory is not something new; it’s been accepted for many, many years.

Let’s also acknowledge that the things we typically learn in the workplace are different from what we were required to learn in school. Taking the above as a given let’s now look at some simple concepts that you should consider incorporating in your CBTs to ensure that learning actually takes place. These concepts are based on the work of Malcolm Knowles and other experts in the field of adult education.

• Adults want to know why they are required to learn something.
Ensure that your CBT explains why a particular task, skill or behaviour is important to the trainee. Adults want to know, “What’s in this for me, why do I need to learn this?” Providing a reason and a context for the instruction will help motivate the trainee to learn the required skills.

• Adults learn best through practical application.
Since adults learn best by doing, try to include meaningful activities in your CBT. The use of exercises, simulations, quizzes and even games are common ways for the trainee to apply and re-enforce the skills and knowledge they are learning. You can also consider adding a demonstration piece or practical exercise on the subject when the trainee returns to the work site.

• Adults bring their own experience to the learning situation.
Adults draw on their experience to help them solve problems. They each come with a repertoire of skills and knowledge that they can use when called upon to learn something new. Unfortunately they also have different levels and types of knowledge and experience. And to further complicate matters, individual learning styles are often different from trainee to trainee. As a result, there are two things you should bear in mind when you’re designing a job-related CBT: 1) Use the knowledge that a trainee should already possess as the basis for teaching new information, and 2) Attempt to cover as many different learning styles as possible by presenting the information in several different formats. This second point involves the use of elements such as: text, audio, graphics and practical exercises to deliver the training content.

• Adults like to apply what they learn as soon as possible.
Like the old adage says, “If you don’t use it, you lose it.” Adults learn best when the subject matter has an immediate application for them. As a result CBT can work extremely well as just-in-time training in the work environment. You can use CBT:

  1. When an employee transfers into a new section.
  2. As a refresher before a worker starts a job that he or she hasn’t completed in a long time.
  3. When a new piece of equipment or a new business process arrives in the department.

Your CBT module should contain information that the trainees can use right away. It should allow them to learn at their own pace and gain insight from their mistakes in a safe and controlled environment.

Beware of Dancing Baloney
There is another old adage that says, “Just because you can, doesn’t mean you should.” It’s very easy to get carried away with all the bells and whistles that are available to the CBT developer.

If you’ve ever sat through a meeting where the speaker has fallen in love with all the slide transitions, animations, sound effects and special features their presentation software has, then you understand. It all becomes too distracting after a while. The same thing happens in a CBT. The more “stuff” put on screen, the harder it is for the trainee to find the really important information. Things to consider when designing the user interface:

• Create a style guide or template for the on-screen elements.
A little upfront planning and consistency will have a huge impact on the look and feel of your module. If your user interface is predictable and easy to understand, then the trainee will feel more comfortable. The less confusion they feel, the easier it will be for them to absorb your content:

  1. Choose a base font and font size for on-screen text.
  2. Ensure sufficient contrast between text and the background screen so that it can be comfortably read.
  3. Ensure your on-screen text is grammatically correct.
  4. Apply a consistent formatting scheme and to all text (for example, do not use underlining for emphasis — it makes the word or phrase look like a hypertext link).
  5. Establish a consistent hierarchy of headings, labels and titles.

• Nothing should appear on screen that doesn’t support or advance learning.
It’s easy to be seduced by glitzy effects and special features. But if the special effect doesn’t advance the learning — or worse still, gets in the way — what good is it? An interface that includes a cluttered screen, alarming colors, flashing lights, dramatic changes in font size, unnecessary sounds, animation for the sake of animation, and complicated screen transitions will cause the trainee to lose focus and concentrate on the medium instead of the message. Basically, it all boils down to two things: don’t distract the trainee and don’t interrupt the learning process. In the case of on-screen elements, more is definitely less.

• Ensure the trainee can always get back to the lesson.
If you’ve ever become lost while surfing the Internet, then you know how difficult it can be to find your way out. Jumping from link to link, or drilling down through multiple layers of information is like exploring a cave. You never know where the next twist or turn is going to take you. So, if you’re planning to include pop-ups, PDF files, hypertext links and other navigational elements in your CBT, ensure that:

  1. There are no dead end links.
  2. On-screen elements such as pop-ups can be easily closed.
  3. Viewing supporting information doesn’t require opening another application.
  4. Module travel is bi-directional.

Working With an External Vendor
If you want to incorporate CBTs into your training catalog, but you don’t have the time, resources or skills to develop them in-house, then you’re probably thinking about working with an outside vendor. There are lots of companies and individuals who specialize in this form of program development to choose from. Here are some things to consider when you’re going out into the CBT developer market:

• Off-the-shelf vs. custom content.
If you’re looking for a quick fix to a relatively generic problem, then purchasing an off-the-shelf product is a great way to get training where it is needed fast. There are lots of good communication, customer service, sales, safety, etc. CBT programs available from a wide range of reputable suppliers. There are a number of vendors who specialize in more industry-specific training programs, such as rail infrastructure or diesel equipment to name a couple of examples. Some of the larger equipment manufacturers also offer CBT modules as part of their training packages. But, if you’re looking for information that reflects your specific operational environment and equipment, then custom content development is probably the way to go.

• Look for a custom content developer, not a subject matter expert.
There are three elements involved in producing a successful CBT module:

  1. Content — the required information from several sources, including a subject matter expert.
  2. Instructional Design — organizing and presenting the content in a way enabling absorbtion, retention and ultimately demonstrative mastery of the required information.
  3. Computer Interface — the box that holds the content, as well as the plumbing that enables the user to move through the program.

Generally speaking, a good custom content developer has two of these three elements. They have the technical skills and equipment required to build a good box for your content. They also have the instructional design background necessary to transform your base content into a successful learning tool. As the course originator, you have to supply the third element, which is the actual subject material. You know what your organization wants and how it operates. Obviously, as the person paying the bills, you also have a say about how the other two development elements are going to be executed.

If you find a supplier who claims to be an expert in all three of these areas, be sure to review their credentials and ask for references, and insist upon seeing samples of their work. Solid instructional design skills are often the missing element in an external subject matter expert’s development equation. In other words, they may know what needs to be said, but they don’t know how to say it. Unless the principles of sound instructional design are incorporated into the CBT development process, the resulting module will probably contain factual information, it may even look good, but it will likely be useless as a learning tool.

Managing the Development Process
Regardless of whether you’re working with a vendor or producing the CBT yourself, you have to manage the development process. Now, some of the following roles may be collapsed in smaller organizations, but typically CBT production involves: trainers, subject matter experts, instructional designers and/or outside vendors, client departments and IT departments.

In other words, someone to build the module; someone who knows what should go into the module; someone who wants, or who will benefit from the module; and, someone who enables the module to be accessed within the organization. As a result it takes a lot of communication and collaboration to be successful. Here are some things I’ve learned that help eliminate surprises during the development process:

• Consider some upfront questions affecting module construction.
Before you begin to actually develop your CBT there are some important upfront questions you need to consider that will have a significant impact on how you design the module. Questions like:

What are the course objectives?

Who are the end users?

Will the program be used for initial training, refresher training or both?

Is the CBT module a stand-alone program, or is it part of a larger course?

Is there a course pass/fail requirement?

Is there a need for formalized testing?

Is there a practical or hands-on component?

Do you need to track and record trainee progress and performance?

The answers to all these questions and possibly many more should go into some kind of a design document for the module. Trust me; you’ll avoid a lot of headaches, revisions and expenses later on if you go into the development process with a plan.

• Ensure your infrastructure will support access to the CBT.
In a perfect world you already have a learning management system (LMS) to house, launch and track your CBT modules. There is sufficient access to computers in your classrooms and/or the client department’s facilities. And, all your end users have the appropriate access to your computer network. If your reality is something different, then you need to sort out how you’re going to deliver your modules to your audience. Here are some possible suggestions if you don’t have a lot of IT infrastructure:

  1. Let the vendor host the CBT.
  2. Put the CBT on a CD.
  3. Install the CBT on one computer.
  4. Incorporate the CBT into an existing instructor-led training program.

If you are planning to use a learning management system to launch your CBT module, then it has to be able to talk to your LMS. Without going into a lot of detail here, this means your module must adhere to certain industry standards and specifications that define communication between the module and the host system. This is typically referred to as sharable content object reference model (SCORM) compliance. If you are seriously thinking about CBT development, then I suggest you do some additional research on this topic.

• Hold a stakeholder kick-off meeting.
Bring all of the players together at the start of the project and discuss their roles and responsibilities. Explain how your plan is going to work. If you’re using an outside vendor, they’ve probably done this before, so let them discuss their CBT development process. Make sure everybody knows what’s expected of them and gain their commitment for the time and resources involved. Some of the typical collaborative activities include:

  1. Gathering content from client departments, subject matter experts and other sources.
  2. Capturing equipment, work processes and locations on video and digital photography.
  3. Reviewing, reviewing, reviewing.
  4. QA testing the beta modules in an actual IT environment.
  5. Roll-out and piloting modules with end users.

• Storyboard CBT content in its native environment.
The traditional CBT storyboard is usually a table that lists how the text, digital assets, special effects, links, etc. are going to appear and interact on a particular screen. The storyboard can be a little confusing or even intimidating for reviewers who are unfamiliar with the development process.

I’ve been dealing with them for years and I still have trouble visualizing what the finished screen will look like. In response to this we now ask our vendors to provide us with a mock-up of the screen that identifies the assets that are required and where everything will go in the final version. It helps us to plan what is needed, while giving us a better feel for the look of the finished product.

•Hold regular update/milestone meetings with stakeholders.
If you want to keep production moving, stakeholders have to know what’s going on. This is especially important for key individuals in the client departments who are on the periphery of the development process and have other jobs and responsibilities.

In my experience, very few CBT projects go as originally planned. Requirements change, resources become unavailable and production schedules stretch, to name just a few of the problems you’re likely to face. And of course, if you’re working with a vendor, you have to hold regular meetings to ensure that they are achieving all of the required project milestones.

Nigel Lindsey-King is an instructional designer with more than 25 years of experience developing and delivering training programs for a transit environment.