Why do we still expect “successful failures” on rocket launches and not just scale up or scale down the same design on successful rocket ships and launch pads to make bigger or smaller ships with more stable structural material?
In: 1598
At some point you just gotta test a full sized version. You can simulate until you’re blue in the face, but your simulations are only as good as the data points they measure, i.e. if there is a real world factor that would make your rocket go boom but your simulations don’t cover it because maybe you haven’t discovered that factor yet, then you’ll still do a few real world tests to cover your ass for the unknown.
Or to use a coding analogy. Devs write unit tests alongside their regular code (usually) that will test the part of the code that’s written. These tests might test the positive success scenario and/or a negative failure scenario and everything works and tests great. So the dev promotes it into the codebase. But only after it has been combined and run with everything else, do new avenues/possibilities for unexpected behavior reveal themselves due to the increased complexity of the system your component is now just one small part of.
So that’s why we will do both larger QA testing (wider, more system-wide testing, but still typically automated, scripted or scenario based due to time/money factors) and then hopefully if you’re not in the game industry, before you’ll ship it, you’ll do several rounds of UAT which is user acceptance testing followed by bug fixing, then rinsing and repeating the cycle until the general belief is that all major showstopping issues are either known about and accepted or patched out.
What is UAT in a nutshell? Real world exposure without necessarily real world consequences. I.e. a test area where the tool/app can be tested without any major consequences but it behaves identically as if it had been deployed for real and then you just let your users hammer on it. You might do that through a selected subset of users (closed UAT) or you might open it up a bit more for a wider range of testing exposure, like an open alpha or beta (the difference there is simply down to the definition of overall completeness with alphas being in a much more unfinished state than a beta).
The point being, the UAT sessions typically find the really weird esoteric bugs that could typically never have been found through a combination of unit testing or QA alone.
So, consider this rocket launch a round of UAT, where bugs will be identified through the data telemetry they would have received right up to failure, they’ll ‘patch’ the issue, adjust all of their testing scenarios to test for that in the future and launch another rocket.
Latest Answers