API Rate Limiting

When we provide an API service, we need to ensure fair usage for every user so that the system resources are effectively and fairly serving all. We want to make sure that the majority of users are getting good server performance; therefore, we need to apply restrictions. By limiting a small number of high-traffic users, we can make sure that the majority of users are satisfied.

The way to do that is to set a limit per user. For example, we can limit the number of requests per user to be no more than 100 per second. This number will be enough for the normal usage of our API. If there is any particular case where a user is firing 100+ requests per second, the excess requests will not be handled. This is to reserve system resources (such as CPU processing and bandwidth resources) for other users.

To achieve this, we introduce the concept of rate-limiting. By limiting the "rate" of our API service per user, we guarantee that the majority of our users are...