So far in the TLYF series we have covered the Principles, the Basics, and the Impact of the external components that support your mission. Lets expand on that with the one dimension we have little ability to stop - Time.
It's about time, it's about space, About two men in the strangest place. It's about time, it's about flight. Traveling faster than the speed of light.*
In the real world, not everything happens at once. Probably a good thing. And some activities take much longer than others. Like going to the office could mean a drive to work in heavy morning traffic, or putting on your slippers and walking into the other room.
Applications are no different. Requests take different amounts of time to respond. Operations occur in some order; not all at once nor always in parallel. And sometimes we only need to perform a task once - like swapping one database column for another during a migration.
Let's explore considerations from the Test Like You Fly perspective.
Performance means speed. Not only do tasks need to complete within some finite amount of time, but they need to meet reasonable expectations supporting user processes. Cascading and serial operations must complete in an overall duration that supports the business objectives. These may be difficult to reproduce in a test environment - especially if this is a recently inaugurated application without a reservoir of built up experience in real production data. However, it is absolutely necessary that the first time real production volumes of data are experienced NOT be in production.
Most applications today are used in more than a single time zone. In truth most applications of any size are globally accessible. From the perspective of the app's user they are in their own zone and most likely want to see all their information in that context. Databases typically are served in a consistent single time zone, with internal data stored with that time zone basis. This is usually UTC, but may be a different zone depending on the evolution of the app.
Ensure that different timezones are considered when determining dates from timestamps and date+time values. Time zone awareness, conversion between zones, and handling of notions like Summertime and Daylight Saving Time are critical as time is portrayed in the app. Calculations which cross DST boundaries (April and November currently) need to acknowledge the loss or gain of an hour. Transitioning across Year boundaries is another concern. The Space Shuttle Team was concerned about launching in one year and being in space when New Year's Eve transpired. It's critical to simulate all of this before putting the application in production. Be aware that not all timezones change their clocks on the same date. Is that handled? Is it even tested?
Applications that serve other applications, such as APIs, often have throttling built in to ensure they are not overwhelmed or artificially susceptible to Denial of Service attacks. When we rely on those APIs, it is imperative that we test our application's behavior when throttled by these external "forces".
Getting ahead of ourselves, especially in the world of multi-core apps and serverless/microservices is a critical concern. Everything has an order. Order may be implicit or explicit. The more explicit the ordering - such as via creating pipelines of activities - the easier to ensure the sequences are maintained.
Many test frameworks randomize tests, which is important. But purposely running tests in a planned sequence that proves graceful processing in an out-of-sequence manner is also important. Do not trust that randomness will generate all combinations.
System complexity is at the heart of why TLYF is critical yet difficult. Isolating components from each other is the antithesis of this type of testing. This could lead to exponential combinatorial test scenarios. That could be a smell, or it could be a requirement.
These are some of the most difficult areas to simulate, however perhaps also the ones that can unexpectedly be the most problematic. Methods that work successfully once, may fail on subsequent invocations merely because of prior state. Methods that alter their input parameters invisibly can behave fine, then fail the next time.
Single occurrence events are not just single first-time activities, but also single last-time activities. You may only get one chance in the production environment to get these correct. Much like single-execution missions (i.e. expendable rockets), migrations and other corrective actions often dissuade time spent on testing. After all, we only need this to be correct once.
I hope you have found Part 4 on Time and Timing valuable. Next up in our TLYF series - "What Was Your Intent?"