My name is Geoff Newman, I am a Simulator Instructor currently working at the Leonardo Training Academy in Italy. I am a contractor and the views and ideas I contribute today are my own and in no way reflect the views, policies and opinions of the LTA.
The essence of Evidence Based Training is – SHOW ME! Show me you can do it, whatever it may be. No more box ticking, just show me you can perform.
The truth about learning to operate complex automation systems is that the necessary skills come easy to some but very, very slowly to many.
The breadth of this capability spectrum may be the cause of many of the competence issues prevailing today because we are focusing on the delivery of a syllabus rather than on competence. The global variation in piloting skills is quite extraordinary and even within the EASA family this wide variation can be found. This is a real headache for the course designers and is one reason why a competence based system is essential.
From my privileged position at the Instructor Operating Station in my AW139 Full Flight Simulator I can testify that much of the confusion that arises in the cockpit of this complex modern helicopter has its origins in mismanagement of the automation. Achieving competency with automation systems will make a great contribution to overall piloting standards. If the current ‘syllabus’ based system is in need of updating and improving to meet the demands of modern complex helicopters then maybe EBT can deliver the much- needed improvement in competency.
EBT focuses attention on the demonstration of a wider range of knowledge and skills so it has the potential, maybe not to replace the current system, but certainly to compliment it.
I’m here to talk to you about EBT in the context of effectively teaching the use of the automation systems but before I get down to details let me set the scene for you. We’ll get there, I promise, but first I want to create a better understanding of what we are up against. The learning process for a complex Flight Management System begins in the technical ground school.
As TRI’s and SFI’s we come to understand the importance of the ‘Learning Curve’ and how managing the passage of each student along this curve is a skill each flight instructor needs to acquire.
But enough of the ‘Learning Curve’ – I want to talk about the ‘Forgetting Curve’.
The Psychologist, Hermann Ebbinghaus, just over one hundred years ago conducted pioneering research into the ability of the human mind to retain knowledge. I’ll condense his sizeable papers into a simple conclusion.
SLIDE 1 – Nearly 80% of knowledge acquired via a classroom lecture will be lost within four weeks.
Think about this for a moment. A typical EASA approved two-week type technical course – which contains important elements relating to the automation – requires a pass mark of 75% in the end of course exam. This level of knowledge retention is deemed to be the minimum acceptable for the eventual issue of a type rating. The candidate will normally meet up with his new type – in the flesh at it were – four weeks after the exam and could begin operations with a theoretical knowledge level of 15% if Ebbinghaus’s 80% loss is realised during that period. This, you will observe is now 60% below the level apparently required for the issue of a TR.
Of course the reality is that things are not that simple, there will be some reinforcement of elements of the technical course during the simulator and flying training. Nonetheless this methodology does not reflect well on our understanding of the science of pedagogy. If the pass mark of 75% represents the minimum level of knowledge then surely steps should be taken to ensure that this minimum level is maintained. Many of the pilots I see at recurrent training sessions are surprisingly ignorant about basic technical issues. This particularly applies to those that were given a type rating with zero time in any kind of FSTD and missed that vital reinforcement. There may be shortcuts out there but I really do recommend a full factory course and all it has to offer. From where I sit cheaper is definitely not better.
SLIDE 2 – Knowledge can be consolidated by a series of reviews.
Ebbinghaus was able to prove that repetition consolidates knowledge and the more you repeat it the more you consolidate. ‘Over-training’, he called it and ‘over training’ is all about repeating the lesson until consolidation of the vital elements is achieved. If we could deliver an on-line post grad support system as a cost effective supplement to the standard type rating we would provide vital assistance to the very large number of helicopter pilots who work in relatively small, unsophisticated operations. The UK’s Open University has proven ‘distance learning’ techniques and these can be harnessed to achieve the periodic reviews required.
There is one area of our current training and testing system that has embraced the concept of repetition but I would argue that this has not necessarily been to our advantage. I spent the last forty years of my active flying career doing essentially the same LPC/OPC every six months. It meant we knew what to expect but it also meant that the same skills were demonstrated every time and no new skills entered this forum and no strategy existed within the licensing system to encourage a wider skill portfolio. Of course the OPC may expand the scope for learning new skills and polishing up some old ones but we should not underestimate the LST’s ability to shape the whole teaching and learning process. The exam content becomes the syllabus.
I find it hard to see those checks as anything other than jumping through hoops for they had little relevance to the job I did day in day out. For many years, during the 1980’s and 90’s there was even the extraordinary requirement to exclude the automation from every check flight because the old guard didn’t understand the need to train and check as you operate. It is therefore no surprise that there is legacy lack of emphasis on ensuring that all the aircrafts operating systems are properly understood and operating competence has been demonstrated throughout that system’s capabilities, not just on the elements required to perform the standard test items.
I am banging on about ‘competency’ so maybe before I go any further I should define what I mean by that word. I judge a pilot to be competent when he can perform his daily duties safely, effectively and in accordance with all the rules and regulations that apply to that task. In addition I would include the ability to deal effectively with any problems that occur during the flight. He should be able to deal with the problems encountered and deliver the aircraft, passengers and crew safely to terra firma.
I have had the pleasure of training more than 400 pilots from 52 different countries since I started teaching on the AW139. Whilst 99% passed the various courses they were on, more than 50% could not, I my opinion, meet my definition of being competent. This is mainly because they are not able to cope when things go wrong.
It is essential that the pilot understands and learns how to cope with things that not only go wrong but a solution to the problem cannot be found in the Flight Manual. There are many examples of system malfunctions and failures as well as unforeseen changes in circumstances that cannot simply be solved by looking in the Emergency Checklist. To describe this quality I use the term ‘resilience’. I believe the industry already recognises this quality and the word ‘resilience’ to describe it.
There are no unscheduled problems in the current LST – that’s the point. Pilots are intimately acquainted with its contents – they do the same thing every year and virtually the same thing every six months. EBT allows for carefully choreographed scenarios that enable the instructor to identify the candidates strengths and weaknesses in this respect.
Now to the question of EBT and how it can help us deliver greater overall competence and in particular competence operating the automation. Whilst I am an enthusiast for EBT I would hesitate to refer to myself as in any way an ‘expert’ on the subject. Whilst I use some EBT techniques, when and where appropriate, during simulator training sessions I have not had the opportunity to work within a regime totally dedicated to EBT. I did, however, come close to doing that at the beginning of last year and maybe there is a useful story to tell about that experience.
For the last twelve years a small team of rotary pilots and engineers has been working with Lufthansa Consulting on behalf of Petrobras, the Brazilian National Oil Company. The task was to improve the safety of their flight operations both on and offshore. I had the pleasure of being a founder member of that project and have contributed on and off during that period. Back in 2004 Petrobras were moving 45,000 people every month to over one hundred offshore installations using fifty plus helicopters of various types. Since those early days of the project the Lufthansa team has contributed to a 50% reduction in the helicopter accident rate. Since 2009 the operation has doubled in size and has recently generated many incidents that required some remedial analysis. I was asked to take on that task two years ago and finally got to grips with it in January last year.
When I took a long hard look at the pilot training methods used by the four major operators involved in the project there were two issues that dominated. The first was the weaknesses surrounding pilot recruitment. Brazil does not permit the employment of foreign pilots so the home market had to supply the close to three hundred pilots needed to meet the doubling of the helicopter fleet that took place over three or four years. I won’t go into detail but suffice it to say that where an operator referenced a ‘minimum’ level of training in his Operating Manual this appeared to become the norm so many new recruits were not being properly prepared for the demanding world of offshore operations.
I realised that turning such a huge problem around was never going to be quick and it was never going to be easy so I developed a medium term strategy that would create a better and more progressive training philosophy. To do this we needed a top class cabal of flight instructors. Unfortunately, and this was the second major issue, those in post varied in their capabilities and to ignore that would have hamstrung any attempt to simply devise a better syllabus. Believe it or not that was what my written instructions were from the Petrobras management – devise a better syllabus.
Instead, I recognised that simply devising a better syllabus would never work if the delivery system was essentially dysfunctional. I did my homework on EBT and found the EBT Manual, published by ICAO and through colleagues tracked down Mike Varney of the EBT Foundation. Mike allowed me to transcribe the EBT Manual designed for airliners to a format that may be more suitable for the rotary world. I then made a presentation of the EBT project to the Petrobras management and they could immediately see that it offered the prospect of an orderly step forward. They much prefer evolution rather than revolution.
The intention was to roll out the programme on one type and then review its effectiveness. The programme could then be fine-tuned over a number of years and then applied across the board to eventually include all types and all their operators. The project was made possible by the fact that Petrobras financed the existing simulator training and those resources would be reconfigured to deliver a substantial EBT programme. The existing LPC/OPC system would continue in the normal way so as not to upset the relationship between the Civil Aviation Department and the operators. The intention was to involve the CAD as observers and take them along with us during the experimental and developmental phases. The EBT Manual does make provision for this style of training programme and is a very good way of transitioning from one licensing regime to another.
I have to explain at this point that the single most attractive part of the EBT project for Petrobras was, from my point of view, the fact that the scheme required all flight instructors to take part in training programme designed to deliver the skills necessary to optimise the EBT philosophy. This would provide the opportunity to identify the weak instructors and schedule remedial training for them. We would simultaneously give the training pilots a skill uplift and provide an exciting project for them to buy into. Without their commitment any hope of improving standards would have been wishful thinking. We really needed them to be ‘on side’.
At that time I had not heard about ATQP. Maybe if I had then maybe I would have put that forward as a solution rather than EBT. They are both progressive training philosophies that seek to move us away from the old style of same old, same old metronomic checking system.
By way of postscript the Petrobras project came to a shuddering halt when the corruption scandal broke in May last year. It will be sometime before there is an opportunity to pick up the threads although I have heard from contacts in Brazil that at least one operator is trying to keep the programme moving forward.
One of the major tasks undertaken during the Petrobras EBT project was translate the EBT Manual into helicopter-speak. This required a review of the matrices in the appendix and the need to find a way to make them relevant to helicopter operations. These matrices provided a detailed framework for the course designers to use in their deliberations. The original was broken down into jets and turbo props and further into the different ‘generations’ of these fixed wing types. There was no equivalent categorisation available for helicopters so I decided to use the helicopter type as the differentiator.
SLIDE 3 SAMPLE RECURRENT TRAINING AND ASSESSMENT MATRIX (page 1 of 12)
My expertise was on the AW 139 so the template I created was for that type. I invited the senior trainers in the four main operators to submit their interpretation of the matrix for the type they had selected to be the ‘launch’ candidate for the EBT programme. Three chose the AW139 so went along with my submission and one chose the S92. The differences between the training matrix for the S92 and AW139 turned out to be minimal so it is shown in my document as a combined matrix. The question mark in the title signifies that no other S92 operator has had the opportunity to either agree or disagree with this assertion.
The title of my presentation implies that I might have some answers to the question how can we improve competence in the way we teach the use of automation. It’s tempting to say that the answer is simple. We just need to devote more time to training and more FSTD resources need to be available close to the worksite. All it takes is money…… (pause for hollow laughter) ….. but seriously, modern FMS systems are hugely capable devices that not only include the capabilities found in a fixed wing airliner but also offer the possibility of fully automated Search and Rescue functions.
In my opinion it is unhelpful to define the automation syllabus using the License Skill Test as the template. The Honeywell Epic System Phase 7, has a huge bath-full of capabilities but to deliver the type rating course we take just a couple of buckets full of them. When do we get to deliver an understanding about what is left behind? We need to either expand the course or provide more support training using lower level training devices. These should, as far as possible, offer a truly realistic operating environment without the complications associated with flight simulation. We are working hard to provide these facilities at the Leonardo Training Academy but operators need to be convinced that they are worth paying that little bit extra in order to include them in the course they sign up to.
EBT is mainly about the training and on-going assessment of type-qualified pilots. It relies on the use of well-designed scenarios to deliver opportunities for analysis and assessment over a series of simulator flights rather than relying on one single session. It also provides for a library of these scenarios so that every time there is a six-monthly check flight there is something different to challenge the candidate. Some scenarios can be designed to focus on automation issues so where weaknesses in this area are observed lessons can be targeted to help improve performance.
The authorities will be looking for certain elements to be included in every formal check flight and these can be accommodated within the planned simulator sessions. The EBT matrix provides for this and identifies them in the column entitled ‘Frequency’.
EBT may primarily be aimed at recurrent training but the same techniques can be included within the type- rating course. Simple scenarios can be included allowing the instructor to assess both the rate at which the student is absorbing the material and also the depth of his understanding of the subject. Simply ploughing through a syllabus and ticking boxes is nowhere near as revealing. An example of this kind of simple scenario would be to announce, at an appropriate point in the lesson, that the briefed runway is no longer available for the planned ILS approach and sit back and watch how the candidate handles the situation. The way he behaves will reveal not only a lot about how much he has absorbed about operating the automation but also how his non-technical skills are developing – CRM, airmanship, communication skills and much more besides.
In my view EBT wins hands down over the way we are doing things now. Maybe ATQP is equal to or even better than EBT I don’t yet know enough about ATQP to say but my heart warms at the prospect that we may at last be contemplating sophisticated training systems to deal with equally sophisticated training needs.