Autonomous API Testing with Semantic Reasoning

To succeed, software development teams need to release features at high velocity and of high quality. High quality means the application logic is free of functional defects and security vulnerabilities that put the company at risk. Perimeter security is insufficient to protect against flaws lurking in the application's business logic exposed by an API. To ensure the security of modern applications, the applications' API should be made inherently secure by validating the application logic before release.

Aptori Semantic Reasoning Platform

At Aptori, we built our Semantic Reasoning Platform to unburden developers by making API testing autonomous. We designed a targeted approach to API testing that autonomously interrogates an API efficiently and effectively so teams can rely on Aptori to validate an API quickly in the build stage of their SDLC.

We've heard from multiple teams that all lament the time spent on testing and the uncertainty of knowing when enough testing has been done. And for a good reason. The test space for a typical API is astronomical when one considers the following:

• The number of permutations of operations to execute in sequence grows exponentially with sequence length.
• The number of combinations of possible input values for each operation.
• The number of checks to verify the behavior of the API.

No wonder software engineering teams are overwhelmed by the effort to test their APIs.

A key insight about APIs is that operations rarely exist in isolation. There will be cases in which data in the response of one operation is used as input to another operation. The design of our Semantic Reasoning Platform leverages the relationships between operations in an API to reduce the number of permutations and to choose input values that result in successful requests to the API. For example, a resource-oriented API often has "CRUD'' operations to Create, Read, Update, and Delete objects of a particular resource type. Given an API definition that describes the operations of an API, our Semantic Reasoning Platform identifies each operation's resource and action type and constructs an API call graph that enumerates sequences of operations that make sense as workflows. Using this graph, Aptori can emulate how a human user uses the API, whether a customer or an attacker.

Let's consider a simple example. To test an operation that updates an object, one needs to have a valid identifier of an object in the application. The likelihood of guessing a valid identifier at random is near impossible. Instead, the approach a tester would take - and the technique Aptori performs autonomously - is to identify an operation, such as the create operation, that provides a valid object identifier in its response. The update operation can be tested with a valid identifier by invoking the create operation first. This dramatically improves the effectiveness of the test by exercising the application logic of the update operation beyond the input validation code. The relationship between the create and update operations also guides how the API call graph is constructed to reduce the exponential space of possible sequences to a linear amount.

Any engineer that has written an API test is familiar with the approach described above. It follows a common test pattern known as Arrange-Act-Assert (also similar to Given-When-Then), which means the test first sets up the initial state, then performs an action to be tested, and finally validates the expected result. For API testing, tools like Postman allow an engineer to manually build a stateful sequence of operations necessary to arrange an application into the initial state. Unfortunately, it is a laborious effort to write and maintain such tests manually, not to mention the number of tests that must be written.

Aptori unburdens developers from writing tests manually. Using an API definition as input, our Semantic Reasoning Platform constructs a stateful API call graph and autonomously walks the graph interrogating the API, uncovering functional defects and business logic vulnerabilities. Our Semantic Reasoning Platform can traverse the API call graph in multiple ways, from a minimal set of sequences that execute all operations at least once to sequences that repeat operations multiple times for performance and load testing.