Gimme Gimme Gimme more Assertions

I can’t forget the words my colleague gave me when I was just taking my first baby steps as a QA, and it was a great QA-related piece of advice: “Never assume anything.” I can assure you that the more time progresses, the more this makes sense.

Today’s IT systems we work with are more complicated than ever before, and we need to be completely sure of the outcome of any part of the system that we are either testing, exploring, or developing. If we are not sure about the expected behavior, there is always a specification or some other type of documentation that is supposed to clarify the enigma. If there is no specification and the documentation is lacking, we can always check with the project manager, product owner, developer, or anyone who is familiar with the system.

Sure, for many features we can assume that the outcome is within the scope of expectations. For example, if we click on a log out button, then we will be logged out. But, as there is always a but - if we click the Logout button, are we supposed to be presented with some dialog confirmation? Are we supposed to be forwarded to the login page? Is all unsaved work being saved as a draft or completely forgotten? You see, such a simple action has an outcome we can assume, but there are many questions that arise. Just imagine the amount and complexity of questions we might think of in the case of more challenging actions and features.

I always write a test specification with automation in mind, trying to prepare all the data, steps, and descriptions to be ready if (regression) automation will be developed. That if is purposely accentuated, because even if automation is never developed for the system under test, you still have perfectly defined assertions, which are very useful while performing manual (regression) testing. This is also a nice way to validate all the states that follow the executed steps.

Examples are my best friend, so let’s clarify my words with some test cases. A simple CRUD process is always a good and most anticipated example, so I will try to bring this whole thesis closer with a simple test case titled User can be created.

First of all, let’s start with the title, which we already mentioned:

User can be created

Now we can already assume (ah yes, that magical word once again) what the process might be, but let’s define it better with some steps to reproduce. We are going to use the Gherkin syntax, as I am using it in the current project I am working on and I find it quite useful:

Given I am logged in as an Admin

And I have the user dashboard opened

When I click on the “Add user” button

And I type “cosmo.kramer@kramerica.me” into the “Email” textbox

And I type “Pass123!” into the “Password” textbox

And I click on the “Submit” button

Then…

If you are familiar with the Gherkin syntax, you know that the expected result comes with the Then keyword, and here starts the fun with assertions. What are we expecting as the outcome of this test case? For the user to be created. And how do we confirm that?

Then a banner notification with “User successfully created” pops up

Perfect. We have an assertion that confirms that the user is created and the test case User can be created has passed. Although, this might not be enough and this is exactly the point of this article. What additional assertions might we think of? What if the banner notification tells us that the user is created, but there was some error and the user is not actually created? Then we might add another assertion:

Then a banner notification with “User successfully created” pops up

And a user with the email “cosmo.kramer@kramerica.me” is visible in the users list

This assertion makes us a lot more confident. Also, to make sure that the user is not created only in the current browser session and accidentally not stored in the database, let’s add just one more assertion:

Then a banner notification with “User successfully created” pops up

And a user with the email “cosmo.kramer@kramerica.me” is visible in the users list

And a user with the email “cosmo.kramer@kramerica.me” is present in the Users table in the database

The variants are endless - be creative with assertions and let them be robust. Also, keep in mind not to overcomplicate tests with assertions. For this particular example, I would check the database during the development process of the system under test, but for regression tests I would omit that step because it might be time-consuming for a tester to manually access the database, and letting automation frameworks access the database might be a bit tricky from a security perspective (API endpoint might be the solution, but more on this in some future topic).

Additional assertions that are not defined in Gherkin Then steps might be added for automation purposes. These assertions don’t need to be defined in the test specification, but rather in the middle of the automation code. They are useful to check if something is visible, if something has changed state, or any other behavior that might cause the automation to fail. Having said that, situations that might cause flakiness are a great use case for additional assertions. Some examples are:

• Check if the Edit button is enabled before clicking on it
• Check if the app has an internet connection before logging in

With these kinds of assertions, we want to get quick confirmation when tests fail, as they can quickly reveal why the test failed. I tend to use these mid-code assertions when I don’t have enough confidence in some element or feature, so I want to double-check it before proceeding with the test.

And now to close this article - when I was starting to write this post, I asked AI to give me some comic ideas for the title. The suggestions were okay, but I decided to use my own idea. I don’t know - ABBA was probably playing in my head at that moment, so it was really hard to eliminate it.

If you want to know more about Gherkin, visit this article:
https://automationpanda.com/2017/01/30/bdd-101-writing-good-gherkin/