diff --git a/_images/rate_limiter/fixed_window.svg b/_images/rate_limiter/fixed_window.svg new file mode 100644 index 00000000000..83d5f6e79ac --- /dev/null +++ b/_images/rate_limiter/fixed_window.svg @@ -0,0 +1,84 @@ + + + + + + + + + + + + + + + + + + + + + + 10:00 + + + 10:30 + + + 11:00 + + + 11:30 + + + 12:00 + + + + + + + + 12:30 + + + 13:00 + + + + + + + + + + + + + + + + + + + + + + 1 hour window + + + 1 hour window + + + + + + 1 hour window + + + + + 13:15 + + + diff --git a/_images/rate_limiter/sliding_window.svg b/_images/rate_limiter/sliding_window.svg new file mode 100644 index 00000000000..2c565615441 --- /dev/null +++ b/_images/rate_limiter/sliding_window.svg @@ -0,0 +1,65 @@ + + + + + + + + + + 10:00 + + + 10:30 + + + 11:00 + + + 11:30 + + + 12:00 + + + + + + 12:30 + + + 13:00 + + + + + + + + + + + + + + + + + + + + + + 1 hour window + + + + + + 13:15 + + + + + + diff --git a/_images/rate_limiter/token_bucket.svg b/_images/rate_limiter/token_bucket.svg new file mode 100644 index 00000000000..29d6fc8f103 --- /dev/null +++ b/_images/rate_limiter/token_bucket.svg @@ -0,0 +1,83 @@ + + + + 10:00 + + + 10:30 + + + 11:00 + + + 11:30 + + + 12:00 + + + + + + + + 12:30 + + + 13:00 + + + + + + + + + + + + + + + + + + + + + + + + + + + 13:15 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/_images/sources/rate_limiter/fixed_window.dia b/_images/sources/rate_limiter/fixed_window.dia new file mode 100644 index 00000000000..16282a2dcce Binary files /dev/null and b/_images/sources/rate_limiter/fixed_window.dia differ diff --git a/_images/sources/rate_limiter/sliding_window.dia b/_images/sources/rate_limiter/sliding_window.dia new file mode 100644 index 00000000000..e16275d8995 Binary files /dev/null and b/_images/sources/rate_limiter/sliding_window.dia differ diff --git a/_images/sources/rate_limiter/token_bucket.dia b/_images/sources/rate_limiter/token_bucket.dia new file mode 100644 index 00000000000..16761971337 Binary files /dev/null and b/_images/sources/rate_limiter/token_bucket.dia differ diff --git a/rate_limiter.rst b/rate_limiter.rst index 63e073a1e92..1d54faba331 100644 --- a/rate_limiter.rst +++ b/rate_limiter.rst @@ -28,22 +28,44 @@ Fixed Window Rate Limiter ~~~~~~~~~~~~~~~~~~~~~~~~~ This is the simplest technique and it's based on setting a limit for a given -interval of time. For example: 5,000 requests per hour or 3 login attempts -every 15 minutes. +interval of time (e.g. 5,000 requests per hour or 3 login attempts every 15 +minutes). + +In the diagram below, the limit is set to "5 tokens per hour". Each window +starts at the first hit (i.e. 10:15, 11:30 and 12:30). As soon as there are +5 hits (the blue squares) in a window, all others will be rejected (red +squares). + +.. raw:: html + + Its main drawback is that resource usage is not evenly distributed in time and -it can overload the server at the window edges. In the previous example, a user -could make the 4,999 requests in the last minute of some hour and another 5,000 -requests during the first minute of the next hour, making 9,999 requests in -total in two minutes and possibly overloading the server. These periods of -excessive usage are called "bursts". +it can overload the server at the window edges. In the previous example, +there are 6 accepted requests between 11:00 and 12:00. + +This is more significant with bigger limits. For instance, with 5,000 requests +per hour, a user could make the 4,999 requests in the last minute of some +hour and another 5,000 requests during the first minute of the next hour, +making 9,999 requests in total in two minutes and possibly overloading the +server. These periods of excessive usage are called "bursts". Sliding Window Rate Limiter ~~~~~~~~~~~~~~~~~~~~~~~~~~~ The sliding window algorithm is an alternative to the fixed window algorithm -designed to reduce bursts. To do that, the rate limit is calculated based on -the current window and the previous window. +designed to reduce bursts. This is the same example as above, but then +using a 1 hour window that slides over the timeline: + +.. raw:: html + + + +As you can see, this removes the edges of the window and would prevent the +6th request at 11:45. + +To achieve this, the rate limit is approximated based on the current window and +the previous window. For example: the limit is 5,000 requests per hour; a user made 4,000 requests the previous hour and 500 requests this hour. 15 minutes in to the current hour @@ -66,6 +88,18 @@ continuously updating budget of resource usage. It roughly works like this: * If the bucket still contains tokens, the event is allowed; otherwise, it's denied; * If the bucket is at full capacity, new tokens are discarded. +The below diagram shows a token bucket of size 4 that is filled with a rate +of 1 token per 15 minutes: + +.. raw:: html + + + +This algorithm handles more complex back-off algorithm to manage bursts. +For instance, it can allow a user to try a password 5 times and then only +allow 1 every 15 minutes (unless the user waits 75 minutes and they will be +allowed 5 tries again). + Installation ------------