OEM Development testing v In-service Experiance
OEM Development testing v In-service Experiance
This is a common theme across all industries. Automotive, aeronautical, industrial, marine, consumer goods.
An OEM launches a new product, after possibly several years and millions apon millions of testing hrs/miles/cycles etc.
But still, problems don't show up until after the first year of in-sevice use by the customer. This is particularly true in the automotic world.
Is there a flaw with the way development is done, or the test and development regime. Or, is it it simply not possible to test in a real world way that simulates the experience of thousands of customers. It's not for want of trying though. car manufacturers spend millions of dollars flying prototypes all over the world looking for the worst conditions possible, GM for example did several million miles of testing, yet problems didn;t show up until the trucks were in the hands of customers for a year.
Here is an example
An OEM launches a new product, after possibly several years and millions apon millions of testing hrs/miles/cycles etc.
But still, problems don't show up until after the first year of in-sevice use by the customer. This is particularly true in the automotic world.
Is there a flaw with the way development is done, or the test and development regime. Or, is it it simply not possible to test in a real world way that simulates the experience of thousands of customers. It's not for want of trying though. car manufacturers spend millions of dollars flying prototypes all over the world looking for the worst conditions possible, GM for example did several million miles of testing, yet problems didn;t show up until the trucks were in the hands of customers for a year.
Here is an example
Join Date: Mar 2019
Location: Zurich
Posts: 1
Likes: 0
Received 0 Likes
on
0 Posts
You are using the wrong analogy. Modern cars are NOT tested like they used to. When Peugeot wanted to test their cars, they'd simply drive them from France to Senegal and see what broke down. That's how Paris-Dakkar Rally was born. And that's why French cars would lose all plastic panels in the first two years but would then go on forever. Mercedes would run their engines till they broke down and saw where and how it broke.
These days, testing is done "in a more scientific way". Car manufacturers are NOT "looking for the worst conditions possible". They send several prototypes to "testing grounds", in Rovaniemi for winter testing, in South Africa or Mohave Desert for high-temperature runs, and to their "proving grounds" in Indonesia and Malaysia for "humid climate tests". Once there, they have meticulously laid out "courses" with strictly measured bumps and cracks, strictly defined "cycles" each car should drive on each stretch of track, etc. Kontuirlicher Verbesserungprocess. Of course, as any "scientific" testing, it has to be verifiable, recordable and repeatable, which by definition cannot model every life situation the car will meet on the road. Cars are simply no longer tested to destruction.
Now as for the airplanes, I have NO idea how those two are connected because by definition every flight or service of an airplane is, in theory at least, verifiable, recordable and repeatable, i.e. absolutely identical.
These days, testing is done "in a more scientific way". Car manufacturers are NOT "looking for the worst conditions possible". They send several prototypes to "testing grounds", in Rovaniemi for winter testing, in South Africa or Mohave Desert for high-temperature runs, and to their "proving grounds" in Indonesia and Malaysia for "humid climate tests". Once there, they have meticulously laid out "courses" with strictly measured bumps and cracks, strictly defined "cycles" each car should drive on each stretch of track, etc. Kontuirlicher Verbesserungprocess. Of course, as any "scientific" testing, it has to be verifiable, recordable and repeatable, which by definition cannot model every life situation the car will meet on the road. Cars are simply no longer tested to destruction.
Now as for the airplanes, I have NO idea how those two are connected because by definition every flight or service of an airplane is, in theory at least, verifiable, recordable and repeatable, i.e. absolutely identical.
ENTREPPRUNEUR
Join Date: Jun 2001
Location: The 60s
Posts: 566
Likes: 0
Received 0 Likes
on
0 Posts
Firstly you have to understand you cannot verify a system by testing it. You can never cover all the modes/events that could occur in practice. So testing only reduces risk rather than eliminate it. It is thus very important to design a system as thoroughly as possible to avoid faults rather than discover them in testing.
To test a system you need to know precisely what it is supposed to do, and then you need to know all things it will encounter. Humans are not good at these sorts of things. They like to take simplifications and elaborate them as necessary. Obviously there is a great temptation to stop elaborating when you think you've got far enough on your testing journey
Normally testing starts at a granular/component level since the overall system is the sum of its parts. You then test ever larger aggregations of functionality. However the best testing is that which tests the whole system at once. With aircraft that must include such factors as the people who fly and maintain the aircraft. the supply chain, the missions presented, varied weather conditions and so on. There may be just too many possible combinations to ever do this systematically so live line experience is the most reasonable way of picking representative test cases.
This is all of interest because the Boeing Max saga could be characterised as situations that nobody clearly foresaw. They didn't think through completely how an AOA sensor failure would create a challenging flightdeck workload. The individual effects would have been considered but perhaps not how they ganged up together against the pilots. Or Boeing may have actually realised this could happen and suppressed detailed investigation into how various levels of pilot skill might affect the safety outcome. Certainly the Max situation asks a lot of questions about testing
To test a system you need to know precisely what it is supposed to do, and then you need to know all things it will encounter. Humans are not good at these sorts of things. They like to take simplifications and elaborate them as necessary. Obviously there is a great temptation to stop elaborating when you think you've got far enough on your testing journey
Normally testing starts at a granular/component level since the overall system is the sum of its parts. You then test ever larger aggregations of functionality. However the best testing is that which tests the whole system at once. With aircraft that must include such factors as the people who fly and maintain the aircraft. the supply chain, the missions presented, varied weather conditions and so on. There may be just too many possible combinations to ever do this systematically so live line experience is the most reasonable way of picking representative test cases.
This is all of interest because the Boeing Max saga could be characterised as situations that nobody clearly foresaw. They didn't think through completely how an AOA sensor failure would create a challenging flightdeck workload. The individual effects would have been considered but perhaps not how they ganged up together against the pilots. Or Boeing may have actually realised this could happen and suppressed detailed investigation into how various levels of pilot skill might affect the safety outcome. Certainly the Max situation asks a lot of questions about testing
