Axway API-Manager allows you to easily apply quotas to your APIs on System- and Application-Level which makes sure, that
only a configured number of requests will be processed by an API or API-Method in a configured timeframe.
However, in some situations backend-applications providing the services, may not be fully functional
(e.g. during maintenance, etc.) and perform API-Requests slower than expected. In that situations, more API-Requests
are coming in, than backend-service can process. As a result a queue of so called Inflight-API-Requests
is growing up, until all API-Worker threads in the API-Gateway are used. The API-Gateway becomes unresponsive also for other APIs.
In that situations not the number of API-Requests over time matters, instead it is important to limit the total number of concurrent/parallel API-Requests.
The Request- and Routing-Policies delivered with this projects allow you to configure by API, by API-Method or if nothing specific is configured as a default a concurrency limit. The maximum number of Inflight-Requests for an API. The configuration is saved inside the Key-Property-Store (KPS) and validated at runtime by the API-Gateway.
This artefact can be used with Axway API Management version 7.5.3 and higher
Nothing special
If you would like to protect your backend-systems from getting overloaded by too many API-Requests at the same time or make sure a slow-perfoming APIs isn't slowing down your platform, you can install the so called Call-Limiter policies delivered with this project by the following steps:
- Clone this project on your local harddisk
git clone https://github.com/Axway-API-Management-Plus/concurrent-call-limiter.git - Import the Policy-Fragment from file:
src/API-Call-Limiter-Polices.xml
This gives you two new policies in container:Plattform Protection Tools:
MaxConcurrentApiRequestsRequestPolicy&MaxConcurrentApiRequestsResponsePolicy - Additionally a Cache-Config is imported used to store the actual number of Inflight-API-Requests
- Import the KPS-Collection from file:
src/API-Call-Limiter-KPS.xml - The new Policies are meant to be API-Manager Request- and Response-Policies and executed on every API-Request.
As it is very likely, that you are already using Request- and Response-Policies in your API-Manager, you need to wire up the new Call-Limiter-Policies into your existing policies for instance using a Policy-Shortcut.
If you are using API-Manager version 7.6.2 or higher it's recommended to register the Call-Limiter-Policies as Global Request- and Response-Policies to be executed by every API. - Once you have imported everything and deployed the configuration set, you can configure the behavior of the Call-Limiter.
Important note: You have to use both, the Request- AND Response-Policy, as the requests-policy is increment the Inflight-Number and the Response-Policy is decrementing it.
This is an overview about what you have after you have imported both XML-Fragments:
Assuming you have correctly configured the call limiter policies and they are now running during API request processing, a globally configured threshold of 100 concurrent requests per API! is already active:
In order to configure the total number of concurrent API-Requests you can use KPS-Web-UI or REST-API provided by
the API-Gateway Manager:
To override the hardcoded default limit of 100 concurrent requests create the following default entry into the table: Concurrency Limit:
To define a limit for an API, no matter which API-Method is called, create an entry like this:
The API-Path you define is the exposure path of the API-Manager API. Not the path of the Backend-API.
Lastly you can define a limit on a per method basis like this:
The columns comment and lastUpdate are just text-field and fully optional.
- 0.0.1 - 21.08.2019
- Initial version
- Define a limit per Methods using different HTTP-Verbs
Please read Contributing.md for details on our code of conduct, and the process for submitting pull requests to us.
Axway Team