In Test Like You Fly - Part 2 - Basics, we discussed the fundamentals of how you practice a TLYF strategy. No systems operate in such isolation that they are not reliant on some other code foundations. And those foundations rely on other foundations.
Our applications don't live in a vacuum. They take advantage of the operating systems, databases, caches, networks, and other external services available today in the cloud deployment environment. Let's see what we should think about in regard to Testing Like We Fly.
Specificity is an invaluable ally in operating environments that have been confirmed through testing. Do not make the fatal inference: “It should work”.
Let's say we have a database running MySQL 8.0.18. We are tasked with upgrading this database to MySQL 8.0.19. Based on semantic versioning, this looks like an innocent patch update. Unfortunately, this is far from the truth. In this update, the code was changed to assume that
TINYINT(1) should be treated as a boolean. However, this was NOT the case any longer if the column was
UNSIGNED when the schema was extracted. When schemas from MySQL 8.0.18 were dumped and loaded into another database running MySQL 8.0.19, these boolean columns became
TINYINT(4) columns. Existing databases were unaffected, but any new databases loading the old schemas signalled to applications that these columns were no longer boolean.
Moral: Always test the full cycle with any change including downstream practices like data transfer/replication
One of the most difficult contexts to emulate in testing is interfacing with networks. APIs with credentials that serve us in production should be secured from testing access. So how do we sufficiently TLYF while still abiding by network security best practices?
If network access is crucial, then taking the network down before and randomly during test execution provides critical insight into how your application handles these inevitable scenarios.
If possible, use or at least test using the same cloud credentials as will be used in production. This will ensure the accessible “user” context is tested. Testing Like You Fly in this respect gives us the confidence in knowing that if we can hit production cloud services in the test environment, the odds are good that we can hit them in production as well. Make absolutely sure that any test system you establish cannot access the production system directly - protect the production environment from mutation during tests.
In the age of cloud computing, physical distance between your servers is often overlooked. Consider the availability needs and data processed by the servers physically closest to the user locations they service.
Short but important considerations for all parties involved in launching a successful application. Next up is Timing.