How Would You Approach Exploratory API Testing?

Day 2 of MoT’s 30 Days of Testing asks the question “How would you approach exploratory API Testing?”.  From the perspective of someone who has never done testing that I feel falls into the realm of ‘real’ exploratory testing, this is a tough question to answer.  I’m going to approach this as “You’ve been told to test an API you have no documentation for.  You work at the company creating the API.”

There’s a couple of things I should tell you before getting into the exploratory testing:

1.  The company I work for is a software developer with many different departments. We’re all friendly, and knowing more about something is as easy as going to someone’s desk and asking if they’ve got time to help you
2. We’re big into writing specs for things, and having test scripts based on those specs.

The second point typically means that exploratory testing is moved to the sideline.  We will do some loose exploratory testing as we follow the script – based on any quirks we notice in the implementation – but 95% of the time we’re executing test cases that we knew existed far in advance of any code being delivered.

With that out of the way, how would I go about exploratory testing an API with no information about it?

Find Out More

APIs are tricky to exploratory test due to their nature.  There’s no GUI to lead you on your exploratory path.  How can we make it easier on ourselves, as someone in direct contact with the people creating the API?  Talk to them.  Even without a solid spec to work from, business analysts should have an idea of the problem the client wants to solve with the API.  The first thing I do is talk to the BAs and make some notes while having that conversation.  I translate them into a mind map later on.  This gets me one part of puzzle: what we intended to make.

After speaking with the BAs, the next port of call are the developers who worked on implementing the API.  Armed with the knowledge of what needed to be made, I ask the developers how they implemented solutions to each of the problems they were presented with.  This will typically result in a list of methods/paths that allow access to those solutions via the API.  At this point, I ask them to provide sample requests, as well as all parameters that it’s possible to set in each request.  This is important because often there are optional parameters that developers may not think to provide in an example request.  Optional parameters can potentially change how that request is processed, and lead to failures or exceptions that would not be able to find without knowing that the parameter existed in the first place.

Document Stuff

Having had those discussions, I go back to my desk and think about what I’ve learned.  My basic pre-testing discovery period is now complete.  I spend some time attempting to match up what was implemented with what was intended.  If I see any areas where I think the two have significantly diverged, I mark those in some way for testing as soon as I’m confident the very basics of the API are up and running.  This allows for any potential implementation problems to be found and talked about quickly.

I also make sure that what I’ve learned is available to my colleagues.  We keep test scripts in Excel, and maintain an internal Wiki where we write articles about what projects are, where their documentation is, and any information likely to be helpful for someone picking up testing the project for the first time.

Tools

My tool of choice for any kind of API testing is RestAssured.  I’m primarily an automated tester so I’m more comfortable in an IDE than I am in Postman or SoapUI.  Any tests that I made note of earlier that seem like they would be good long-term tests are written using RestAssured, where they can be re-run any time we need do a release of the API.

Conclusion

The easiest way to deal with a situation where you need to test an API where there’s no specification is to ask people about it.  This may all be cheating when it comes to exploratory testing, I’m not sure.  I can only come at this from a perspective I’m familiar with based on the culture where I work.  If we don’t know, we ask.

I’m looking forward to the other posts about this subject and hopefully learning about what people do when there’s nobody to ask.