19 July, 2017
We all like to solve problems, it’s something that gives us all a great deal of satisfaction. When we attempt to solve problems, we can’t consider them solved unless we prove it through a test of some sort. Some tests we perform seem so simple and are repeated so many times, that we stop consciously thinking about them as tests - they become part of our habits and daily lives.
When you run the shower, you might place your hand under the shower head before stepping into the flow. You don’t do this to offer the water a handshake, you’re doing this to check the temperature of the water. That check is a test – you are ensuring that the temperature of the water leaving the showerhead meets your requirements: not too hot, but not too cold. You’re checking that the input you have given to the shower (setting the temperature) has the expected output (turning the faucet to hot doesn’t produce cold water - after enough time for the boiler to slump into action).
When we solve problems, we are using the principles of software testing - whether we realise it or not. We create specifications, we develop, and then – most importantly – test to prove that what we have done has worked.
When we make changes, just like in professional testing, we test to make sure that they work the way that we expect them to. In testing terms, you are changing the specifications when the environment changes.
For example, if you changed from showering to taking a bath, your requirements for a bath might share those of a shower (they provide hot water for you to clean yourself in), but they might add new requirements (when you fill the bath, you don’t want to over fill it so it flows over the sides when you step in). It’s practical application is the same, but the change of environment makes it necessary to change the requirements.
We can examine situations to prove this concept. Consider a Gardener, who walks into his greenhouse to find that an infestation of Greenfly has taken over his chili plants. He sets in motion a plan to get rid of the “bugs”.
The Gardener is, in this scenario, a representation of a one-man Project Team. He is not only the end user, but also needs to be the Project manager, the Product Specialist, the Developer and the Tester. He undertakes each of these roles while solving the problem.
He begins to create his specifications for this project. The first requirement is “Get rid of the Greenfly”. But he considers further, and realises he must find a way to get rid of the Greenfly in a way that doesn’t damage his plants.
He could just get rid of the plants – thus solving the problem of the Greenfly being a pest to his plants – but by doing so he doesn’t so much solve the problem as create a new environment. It’s a valid consideration, and I’m sure you can probably think of several examples of times where this might be practical or necessary. One might be where a network is infected with malicious software, and it would be cheaper to simply replace the network than to treat each individual system on the network to remove the infection.
So, let’s say that the gardener is not willing to lose his crop, he wants to keep these chilis and get a bumper harvest.
What if he uses a pesticide? The pesticide might make his plants inedible, or reduce growth. However, the pesticide is a valid solution – it could kill the Greenfly; however, it might kill the plants. So, he decides to test it on a single plant. This is a test plan.
He sets up a precondition:
The pesticide must not kill the plant, or contain ingredients that would make the plant inedible.
He then scripts his steps, and his expected results:
Step: The plant is sprayed with the pesticide
Expected Result: The Greenflies no longer appear on the plant, and the plant is healthy
He may perform the test and find a Greenfly on the plant that has survived, and so the test has failed due to a defect in the pesticide – it is allowing the bug to survive.
Or perhaps he may have a “showstopper” – the plant is killed by the pesticide. He loses the plant, so no further testing can be done on that plant until a new environment (a new plant) is created. Traditionally, a “showstopper” would be a bug which prevents the Tester from performing any further tests – meaning the system stops responding or goes down.
However, for our poor Gardener, the outcomes mean he must return to the drawing board and find another way of dealing with the Greenfly.
The Gardener sends the problem back to the product specialist (himself) to come up with another solution, so that he can test to prove its functionality. The product specialist does a little hunting around, looking at documentation about chili plants and the greenfly they have discovered.
He discovers an existing solution in another product – the Ladybird – which if implemented in their greenhouse system, would enable the risk of greenfly bugs to be drastically – if not totally – reduced. The Ladybirds, and their larvae, would eat the Greenfly, whilst not damaging the plants, and would continue to stay in the greenhouse to prevent further outbreaks, if they are kept in the greenhouse.
The product specialist requests from the Project Manager for an allocation of live Ladybirds to be deployed on the environment. With the Ladybirds delivered, it is up to the Tester to ensure that the desired functionality is delivered (the greenfly get all eaten up) and that the system remains stable (the Ladybirds don’t get a hankering for chili).
The test plan is created and a script is written – albeit in the Gardener’s head:
Step: Place Ladybird larvae on the leaves of the chilli plant
Expected Result: The Ladybird larvae eat the greenfly as expected, and do not damage the plant.
The script could be expanded:
Step: After a week, examine the chili plants
Expected Results: The Greenflies are gone, and the plants are undamaged by the Ladybirds. Ladybirds may be present
Let’s say the test passes, and the Greenfly are now gone. The Gardener can “Sign off” the project – it has succeeded. If he is a smart Tester he will make a little note in his gardening book “When Greenfly start attacking my plants – put some Ladybird larvae on the plants to kill them”.
In the real world, this would be like documenting a release. The development company (in this case, the humble Gardener) has found a solution to a problem, and so they should make a note of the solution so that if the problem reappears, they can exploit their knowledge.
If our Gardener doesn’t document, then what happens when two years later, the Greenfly appears on his habaneros? The Gardener, like all of us, is not perfect. It would not be unreasonable for him not to remember how to get rid of the dastardly Greenfly. He would have to go out and spend time to figure out how to get rid of them, rather than look at a concise document for a solution.
So, has the greenhouse scenario satisfied our concepts of ‘what makes a good test’?
• The test is repeatable. The Gardener could go to his chili growing club and tell its members that he got rid of his Greenfly by using Ladybirds, and each member would be able to repeat the test in their own environments – provided they had access to the existing functionality (some live Ladybirds).
• The test is “complete” – what you could consider to be a win or loss condition. If the Greenflies survive, the Gardener’s test has failed – and so the test ends. If the Greenfly die, the Gardener’s test has succeeded – and so the test ends.
• The test is measurable – the Gardener can see at a glance if the requirements have been met as he can count the Greenfly on a plant (should he desire to) and the make sure that number is reduced.
• The test is testable – it can demonstrate that the requirement has been met.
• It’s feasible – it can be implemented with the resources available. The Gardener can order Ladybirds, it is not out with his means to do so, but were he required to put his chili plants in the Rings of Saturn to rid them of Greenfly this would be unfeasible, and thus not a very good test at all. If you can’t feasibly do something, you can’t feasibly test.
• And finally, it is beneficial – the requirement of the Gardener being rid of Greenfly allows him to have the fruit from his chili plants, and all the benefits that may arise from that.
We often don’t realise that what we are doing has a name, or has a practical value in the business world. But looking at the above example, it can see that really, testing is quite a simple practise. Naturally, for a more complex system than removing Greenfly from a chili plant, the testing too becomes more complex, however, the underlying concepts remain the same. You set out to prove that something works, and that it works without adversely affecting the environment.
So, when you go about your day to day business, consider how often you perform a test. It could be as simple as looking left and right before crossing the road to verify that the road is clear before crossing or checking that the temperature of the water in your morning shower is just right. You may surprise yourself, you are really a Tester.
By Colin Campbell, Test Analyst at Edge Testing
Back to Blog