Leveraging Collaborative Learning Theory for Online Technical Training in Transportation: Part II
The Requirement
“Return on investment (ROI) has become one of the most challenging and intriguing issues facing the human resources development (HRD) and the performance improvement community(ies)… Pressure from clients and senior managers to show the return on their training investment is probably the most influential driver… In practice, ROI must be explored, considered, and ultimately implemented in most organizations.”1 And the transportation industry – most especially the transit component – needs ROI analyses badly.
Findings from many studies across different subject matters suggest a ‘rule of thirds,’ indicating that interactive instructional technology, [i.e., computer-based training] can:
- “reduce the cost of instruction by about one-third.
- reduce the time of instruction by about one-third.
- increase learning by about one-third.
“ITSs (Intelligent Tutoring Systems) are distinguished from other applications of interactive instructional technology by their capabilities to permit mixed initiative dialogue in which either the technology or the learner can initiate questions and to generate instructional material, strategies, and interactions on demand in real time.”2
In a recent telephone conversation with an APTA (American Public Transportation Association) staff researcher, this writer was informed that there are approximately 400,000 members in the transit industry workforce. This was a quote by the research staff member from the current APTA 2015 Factbook still in preparation for public release. According to the researcher, this above-mentioned figure can be found on page 20. Individuals in this workforce need to be capable of independent thought and action. They also need to continue their training and career growth even when they are far removed from schoolhouses, expert mentors, and/or others in their own career specialties.
The majority of employees (60 percent) work in bus service, followed by 16 percent in demand response, 13 percent in heavy rail, and 7 percent in commuter rail. The public transportation fleet comprises 135,000 vehicles in active service. Of this number, buses represent 56 percent; demand response vehicles, 26 percent; heavy rail cars, 8 percent; commuter rail cars, 4 percent; light rail cars, 1 percent; and all other modes, 5 percent.
However, the amount and complexity of information that individuals at all levels must integrate and prioritize is increasing. It has produced an increasing number of what Wulfeck and Wetzel-Smith (2008) and Wetzel-Smith and Wulfeck (2010) have described as “Incredibly Complex Tasks.”
Today about 15 percent of industrial tasks (to include all of those in the transportation industry) are abstract, multidimensional, non-linear, dynamic, and interdependent. The dynamic nature of these tasks and the evolving of the transportation operational and maintenance environments means that individuals must continually receive up-to-date training and performance assessment – training that is cost-effective and cost-beneficial. This is the focus of this article
The cost of training in most major industries has always been a topic of discussion among top- and middle-management – and certainly among training directors. When the economy is tight - albeit, frequently - training is one of the first components in the budget to get axed. Having managed projects on five continents, I find this decision to be universal. i.e., to say, cross-cultural.
When researching costs for training of whatever kind, I simply looked for how we, i.e., America, trained our military so effectively and efficiently – on such a short timeline - to prepare for WWII. We trained millions of men and women to meet the enemy. “The Army Specialized Training Program (ASTP) was a military training program instituted by the United States Army and the other military services during World War II to meet wartime demands both for junior officers and soldiers with technical skills conducted at a number of American universities. [They] offered training in such fields as engineering, foreign languages, and medicine.”3
The operational conditions of America’s infrastructure – most particularly the transportation component is best described in the following statement: “The physical infrastructure of the United States is in a state of dismal disrepair. All of the transportation modes - Air, Water, Highway, and Rail (to include transit Systems) - at many locations, worldwide, operate at, or above design capacity. Transit systems in particular are facing the challenges of delivering service in the 21st Century.” 4 We need a WWII approach to restore America’s infrastructure.
Most (perhaps all) transit agency General Managers are operating on tight budget constraints in an increasingly complex maintenance and operational environment imperative, wherein at the same time, are managing with the admonition to “do more with less” requirement. They must ensure that ‘learning’ - education, training, job tasks, and decision- aiding - is rapidly available on demand, anytime and anywhere, to individuals and teams, at all levels of responsibility for performing.
The Technical Opportunity: A Third Revolution in Learning
In response to this requirement, ADL (Advanced Distributive Learning – a term coined by IDA) is riding and contributing to a third revolutionary wave in learning. The first of these revolutions occurred about 5,000 years ago with the invention of writing – the use of graphic tokens to represent syllables of sound. The second revolution in learning occurred with the invention of books printed from moveable type – first in China around 1000 AD and then in Europe in the mid-1400s (Kilgour, 1998). With them, the dissemination of knowledge and skills through writing became scalable. Once content was produced, it could be made widely available and increasingly inexpensive as printing technology developed.5
“The IDA is a non-profit corporation that operates three federally funded research and development centers. This corporation is part of a larger umbrella organization of the Department of Defense –DARPA: the Defense Advanced Research Projects Agency - to provide objective analyses of national security issues, particularly those requiring scientific and technical expertise, and conduct related research on other national challenges.”6
Enter computer technology with its ability to adapt rapidly, in real time, to the changing demands, needs, and circumstances of learners and learning. Computer technology allows not just content but also instructional strategies, techniques, and interactions to become inexpensively ubiquitous and available on demand, anytime, anywhere. It may be fomenting a third revolution in learning. ADL is both a response and a contributor to this third revolutionary possibility.
Figure 1.7
The second revolution in learning occurred with the invention of books printed from moveable type – first in China around 1000 AD and then in Europe in the mid-1400s (Kilgour, 1998). With them, the dissemination of knowledge and skills through writing became scalable. Once content was produced, it could be made widely available and increasingly inexpensive as printing technology developed.6
But with writing and printing the dissemination of content was still passive. It lacked the tutorial interactivity that had been the foundation of learning for the first 100,000 years or so of human existence.
You will recall a quote from my first article wherein I say: “Going further back in history, it was Alexander the Great (B.C. 356-323), son of King Philip of Macedonia and his fourth wife, Olympias, at the age of ten – some historians say fifteen - was given Aristotle as his personal tutor….”8
‘Learning’ in ADL, then, is used as a catch-all designator for education, training, and problem solving. ‘Distributed’ in ADL is not just another word for distance. It signifies learning that can be provided in classrooms with a teacher [and instructor] present, in the field linking together widely dispersed instructors and students, and standing alone with no instructor other than the computer itself present. Finally, ‘Advanced’ in ADL implies affordable, interactive, adaptive, on-demand, instruction delivered using computer technology so that it is available anytime, anywhere. ADL relies on computer technology to accomplish its goals.
The ADL purpose has, from its inception, been to ensure access to the highest quality education, training, and performance/decision aiding tailored to individual needs, and delivered cost effectively, anytime and anywhere.
Evidence: Research and Development Foundations
What evidence is there that computer technology might be effecting this third revolution? What have we learned from research on computer uses in instruction? Some key findings may be summarized as follows:
- The instructional technologies targeted by ADL have been found to be more effective than typical classroom instruction across many instructional objectives and subject matters.
- ADL is generally less costly, offering greater return on investment than current instructional approaches, especially at scale when many widely dispersed students must be served.
- ADL allows education, training, and performance/decision aiding and problem solving to be delivered from the same knowledge bases on platforms ranging from hand-held devices to large desk-top computers to capabilities embedded in operational equipment.
- ADL allows education, training, and performance/decision aiding and problem solving to be delivered from the same knowledge bases on platforms ranging from hand-held devices to large desk-top computers to capabilities embedded in operational equipment.
Individualization: Tutorial Instruction
These arguments have been made for the computer-assisted approaches used by ADL for the last 40-50 years (e.g., Alpert & Bitzer (1970), Atkinson (1968), Coulson (1962), Galanter (1959), Suppes (1966). They have been repeatedly validated by empirical research and practical experience. The following sections discuss the research and development that supports these arguments and serves as background and foundation for ADL.
The argument for ADL technology begins with an issue that arises independently from applications of technology. It concerns the effectiveness of classroom instruction, involving one instructor for 20-30 (or more) students, compared to individual tutoring, involving one instructor for each student. Empirical results from comparisons of this sort are shown in Figure 2. below adapted from Bloom (1984).
Figure 2. Individual Tutoring Compared to Classroom Instruction
“Bloom combined findings from three empirical studies comparing tutoring with one-on-many classroom instruction. That such comparisons would show the tutored students to have learned more is not surprising. What is surprising is the size of the difference. Overall, as Figure 2 suggests, the difference was found to be two standard deviations. It suggests that, with instructional time held constant, one-on-one tutoring, - [vis-à-vis the computer] - can raise the performance of mid-level 50th percentile students roughly to that of 98th percentile students. These and similar empirical findings suggest that differences between one-on-one tutoring and typical classroom instruction are not only likely, but very large.
“The shapes of the distributions shown in Figure 2 further suggest the equity of learning that can be achieved through tutorial instruction. Research by Bloom and others suggests that the individualization permitted by tutorial instruction provides more equity in the amount of learning gains -- an issue examined more directly for instructional technology by Jamison, Fletcher, Suppes, and Atkinson (1976).” That is to say: that the same levels of success with humans as tutors, occurs as well with electronic tutors, e.g., the computer.
Return on Investment
“Knowing that we can use ADL technologies to reliably reduce learning time – remembering the aforementioned ‘rule of thirds’-, particularly time to learn journeyman skills such as remembering, understanding, and applying facts, simple concepts, and straight-forward procedures, what might the return be from investing in it?
“One way to answer this question is by returning to the findings presented earlier in Figure 2, which only considered savings. Using the analysis underlying that figure we can wrap in both the savings and the costs to achieve them using a return on investment (ROI) model. This model simply reduces to the ratio of the net return (savings in this case) to the costs as shown in the following generic formula:
(Savings – Costs)
Costs
“A final definition is offered in this article and is the basic definition of return on investment. The two common formulas offered are: Benefits/Costs Ratio (BCR) and ROI:
BCR = Program Benefits
Program Costs
ROI (%) = Net Program Benefits x 100
Program Costs
“The BCR uses the total benefits and costs. In the ROI formula, the costs are subtracted from the total benefits to produce net benefits which are then divided by the costs.”
“For example, a telemarketing sales training program at Hewlett-Packard Company produced benefits of $3,296,977 with a cost of $1,116,291 (Seagraves, 2001). Therefore the benefits/cost ratio is:
BCR = $3,296,977 = 2.95 (or 2.95:1)
$1,116,291
“As this calculation shows, for every $1 invested, $2.95 in benefits are returned. In this example, net benefits are $3,296,977 - $1,116,291 = $2,180,616, Thus ROI is:
ROI (%) = $2.,180,616 x 100 = 195%
$1,116,291
“This means that for each $1 invested in the program, there is a return of $1.95 in net benefits, after costs are covered. The benefits are usually expressed as annual benefits, representing the amount saved or gained for a complete year after program completion.8”
