By FDE Interview Prep, a SourceStrongAI company

In this post11 sections
  1. What the question asks and what it is testing
  2. The token bucket in one paragraph
  3. A short Python solution with tests
  4. Follow-up: make it thread safe
  5. Follow-up: one bucket per client without leaking memory
  6. Follow-up: share the limit across processes
  7. Token bucket or sliding window: how to choose out loud
  8. Is this really an FDE interview question?
  9. Questions people ask
  10. Keep reading
  11. More from the blog

“Implement a rate limiter” sounds like the easy question on the list. You write a class that says no when there are too many requests, then the interviewer asks about threads, then every customer’s API key, then several servers, and the warm-up has become the whole round. This post gives a short, tested Python , then each follow-up with the words to say. For where the coding round sits in the loop, read the FDE interview guide.

The short answer first. A token bucket stores two numbers per client: how many tokens it holds and when it last updated. On each request, add the tokens that dripped in since then (elapsed * rate), cap the total at capacity, and spend one if there is one. No timers and no background thread. The rest is follow-ups.

What the question asks and what it is testing

The prompt usually arrives in one line: “Write an allow() method that says whether a request may go through. Limit each user to some number of requests per second, with bursts.” The code fits on one screen. What gets judged is everything around it:

  • Do you pin the contract before typing? Limit per what: user, API key, IP? Is a refused request rejected or delayed? Do some requests cost more than one token?
  • Do you handle time correctly? Which clock, and how do you test refill without calling sleep?
  • Do you find the edge cases yourself? Palantir’s coding guide tells candidates to think about edge cases and the ways their code could break. Source 1Writing Good CodePublisherPalantirSource typecompany hiring page
  • Can you extend the design when the interviewer changes the problem? This is where the round is decided.

Here are the exact words we suggest for the opening, before any code:

“So allow(client) returns true or false. Each client can burst up to capacity requests, then gets rate per second after that. A refused request spends nothing. I’ll inject the clock so I can test refill without sleeping. Should a refused caller be told when to retry?”

If the answer to that last question is yes, you will need the wait time for the HTTP response, and the solution below computes it.

Is this really an question? The public evidence is thin; we weigh it near the end. Our post on what no one publishes about FDE interviews shows how to check claims like it.

The token bucket in one paragraph

Picture a bucket that holds at most capacity tokens. Tokens drip in at rate per second. Each request takes one token, or cost tokens if requests differ in weight. When the bucket is empty, the request is refused or waits. Capacity sets the burst you allow; rate sets the long-run limit. The trick that keeps it cheap is lazy refill: nothing adds tokens on a schedule. On each call you work out how many would have dripped in since the last one. That gives constant time per call and two numbers per client. The token bucket glossary entry has the same idea in formula form.

Walk one example out loud before you code. With capacity=3 and rate=1:

  • at t=0, three requests pass and the fourth is refused;
  • at t=0.5, the bucket holds half a token, so the next request is refused;
  • at t=1.0, the halves add up to one token, so it passes;
  • after an hour idle, the bucket holds 3 tokens, not 3600.

The last line is the bug most first drafts have. Say it before the interviewer asks.

A short Python solution with tests

Standard library only. The clock is a parameter so tests can move time by hand.

import math
import time


class TokenBucket:
    def __init__(self, capacity, rate, clock=time.monotonic):
        if capacity <= 0 or rate <= 0:
            raise ValueError("capacity and rate must be > 0")
        # the largest burst
        self.capacity = capacity
        # tokens added per second
        self.rate = rate
        self.clock = clock
        # start full
        self.tokens = capacity
        self.last = clock()

    def allow(self, cost=1):
        if not 0 < cost <= self.capacity:
            raise ValueError("cost must be in (0, capacity]")
        now = self.clock()
        elapsed = max(0.0, now - self.last)
        self.tokens = min(self.capacity,
                          self.tokens + elapsed * self.rate)
        self.last = max(self.last, now)
        # float refill can land just short
        if self.tokens + 1e-9 >= cost:
            self.tokens = max(0.0, self.tokens - cost)
            return True
        return False

    def retry_after(self, cost=1):
        # call right after a refused allow()
        if self.tokens + 1e-9 >= cost:
            return 0
        return math.ceil((cost - self.tokens) / self.rate)

Say why each line is there as you write it:

  • time.monotonic, not time.time. The wall clock can jump backwards when the host syncs its time, and a jump forward would mint free tokens. A monotonic clock only moves forward.
  • max(...) on both the elapsed time and last means a clock that steps backwards, such as a badly written fake, can neither remove tokens nor refill the same second twice.
  • A cost outside (0, capacity] raises. A cost above capacity can never pass, so refusing it forever and telling the caller to retry would be a lie. A zero or negative cost would mint tokens: without the check, allow(-5) on a full bucket with capacity=3 returns true and leaves tokens=8.
  • 1e-9 absorbs float error. With rate=10 and the clock stepping by 0.1, one step can refill 0.9999999999999998 tokens instead of 1. Without the tolerance, the float-steps test below fails; we checked.
  • retry_after is the wait a refused caller needs: the missing tokens divided by the rate. It rounds up, because the HTTP Retry-After header takes whole seconds (more on that below), so a fast limit still says 1, and a client that obeys it is not refused again.

Now the tests. A small callable object stands in for the clock:

class FakeClock:
    def __init__(self):
        self.now = 0.0

    def __call__(self):
        return self.now


def test_burst_then_refill():
    clock = FakeClock()
    b = TokenBucket(capacity=3, rate=1, clock=clock)
    got = [b.allow() for _ in range(4)]
    assert got == [True, True, True, False]
    clock.now = 0.5
    # half a token is not enough
    assert not b.allow()
    clock.now = 1.0
    # the two halves add up
    assert b.allow()


def test_idle_never_overfills():
    clock = FakeClock()
    b = TokenBucket(capacity=3, rate=1, clock=clock)
    clock.now = 1000.0
    assert sum(b.allow() for _ in range(10)) == 3


def test_float_steps():
    clock = FakeClock()
    b = TokenBucket(capacity=1, rate=10, clock=clock)
    for step in range(1, 51):
        clock.now = round(step * 0.1, 10)
        assert b.allow() and not b.allow()


def test_retry_after_rounds_up():
    clock = FakeClock()
    b = TokenBucket(capacity=3, rate=2, clock=clock)
    assert all(b.allow() for _ in range(3))
    clock.now = 0.2
    # 0.4 tokens: 0.3 s to wait, sent as 1
    assert not b.allow() and b.retry_after() == 1

Nothing sleeps, so say so: “I can test refill without a single sleep, so these tests will never be flaky in CI.”

If the limiter guards an HTTP API, a refused request gets status 429 Too Many Requests, defined in RFC 6585, section 4, which says the response may include a Retry-After header. RFC 9110, section 10.2.3 defines that header as an HTTP date or a whole number of seconds. So a refused request gets status 429 with Retry-After set to b.retry_after(), which the test above pins: 0.4 tokens left, 0.3 seconds to wait, sent as 1.

Follow-up: make it thread safe

The interviewer asks: “Your server handles requests on a thread pool. Is this safe?”

It is not. allow() reads the token count, decides, then writes it back. Two threads can both read the last token, both decide yes, and both spend it. Python’s global interpreter lock does not save you, because it can switch threads between the check and the subtraction. And on the free-threaded builds of Python 3.13 and later there is no global lock at all. The fix is to make refill, check and spend one step:

import threading


class LockedBucket(TokenBucket):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self._lock = threading.Lock()

    def allow(self, cost=1):
        # refill, check and spend as one step
        with self._lock:
            return super().allow(cost)

The test is where you can stand out. A naive thread test proves very little: in our runs, the bucket without a lock passed it every time at Python’s default thread switch interval. Shorten the interval and the race shows:

import sys


def test_threads_never_overspend():
    old = sys.getswitchinterval()
    # switch threads often so races show
    sys.setswitchinterval(1e-6)
    try:
        b = LockedBucket(capacity=100, rate=1e-9,
                         clock=lambda: 0.0)
        wins = []

        def work():
            wins.append(sum(b.allow() for _ in range(1000)))

        workers = [threading.Thread(target=work)
                   for _ in range(8)]
        for w in workers:
            w.start()
        for w in workers:
            w.join()
        assert sum(wins) == 100
    finally:
        sys.setswitchinterval(old)

With the short interval, the bucket without a lock failed this test on some runs and passed on others, and the locked one passed every run. Say that out loud: “A passing concurrency test is weak evidence, so I made the race likely and then showed the lock closes it.”

One Blind commenter, who did not state their role, wrote in September 2024 that in their opinion Anthropic’s coding questions end with verbal follow-ups on scaling and edge cases. Source 2Anthropic interview advice (Blind)PublisherBlindSource typecandidate report on Blind That is one opinion, not about rate limiters or FDE roles specifically.

What about asyncio? If the server is a single asyncio event loop and allow() has no await inside it, no other task can run in the middle, so it needs no lock. The moment the bucket moves to a network store, that changes.

Follow-up: one bucket per client without leaking memory

Next: “Now every API key gets its own limit.” A dictionary of buckets keyed by client does it, and it leaks: every key ever seen stays in memory.

The insight that fixes it cleanly: a bucket left idle for capacity / rate seconds is full, and a full bucket behaves exactly like a brand-new one. So you can delete it without changing any answer the limiter gives. Keep the buckets in least-recently-used order and you only ever need to look at the oldest one:

from collections import OrderedDict


class PerClientLimiter:
    def __init__(self, capacity, rate, clock=time.monotonic):
        self.capacity = capacity
        self.rate = rate
        self.clock = clock
        # idle this long means full
        self.full_after = capacity / rate
        # least recently used first
        self.buckets = OrderedDict()
        self.lock = threading.Lock()

    def allow(self, client, cost=1):
        # reject bad input before touching the dict
        if not 0 < cost <= self.capacity:
            raise ValueError("cost must be in (0, capacity]")
        with self.lock:
            now = self.clock()
            # drop buckets that are full anyway
            while self.buckets:
                oldest = next(iter(self.buckets.values()))
                if now - oldest.last < self.full_after:
                    break
                self.buckets.popitem(last=False)
            b = self.buckets.pop(client, None)
            if b is None:
                b = TokenBucket(self.capacity, self.rate,
                                self.clock)
            # move to the newest end
            self.buckets[client] = b
            return b.allow(cost)

And its test:

def test_clients_are_independent_and_idle_ones_go():
    clock = FakeClock()
    lim = PerClientLimiter(capacity=2, rate=1, clock=clock)
    assert lim.allow("a") and lim.allow("a")
    assert not lim.allow("a")
    # a's empty bucket does not affect b
    assert lim.allow("b")
    # both idle past capacity / rate
    clock.now = 5.0
    assert lim.allow("c")
    assert list(lim.buckets) == ["c"]

The oldest bucket always has the oldest last, because every call moves its client to the newest end and the clock only moves forward. That is also why the cost check comes first: a call that raised after moving a bucket would break the order. Each bucket is evicted at most once, so the cleanup costs constant time per call on average.

Then name the case this does not cover. Memory is now bounded by the clients active in the last capacity / rate seconds. If clients are keyed by IP address and someone sprays requests from many addresses, that is still unbounded, so add a hard cap on the number of buckets and decide what happens past it: refuse new clients, or put them in one shared bucket.

Follow-up: share the limit across processes

Next: “We run several app servers behind a load balancer.” Now each process has its own dictionary, and a client gets the full limit once per process.

You have three options. Say them in order of cost:

  1. Split the limit. Give each process capacity / N and rate / N. No shared state and no new failure mode, but it is only right when the load balancer spreads each client’s requests evenly. With sticky sessions, a client gets one process’s share.
  2. Share the state, naively. Store each bucket in Redis, then read it, compute and write it back. This brings back the thread race, across machines: two servers read the last token and both spend it.
  3. Share the state, atomically. Move the arithmetic into the store so refill, check and spend run as one step. In Redis that is a Lua script, and Redis runs a script atomically. Two details make it correct. The script reads Redis’s own TIME, so servers with skewed clocks agree. And it sets a key expiry of capacity / rate seconds, the same “idle means full” argument as the eviction above. The full script is in the model answer to the token bucket question.

Then the question that separates candidates: “What happens when Redis is slow or down?” There is no free answer, so give the options and ask:

  • Fail open. Allow everything. Right when the limiter protects your own servers and a short overload is survivable.
  • Fail closed. Refuse everything. Right when the limit protects a partner’s metered quota or something that costs money.
  • Fall back. Each process switches to a local bucket with its share of the limit. It is approximate, but it degrades gently.

Whichever you pick, give the Redis client a short timeout, or every request waits out a long one while the store is down. Here is what to say:

“If Redis is down, I’d fall back to a local bucket with each server’s share of the limit, and alert on it. If this limit protects the partner’s quota, I’d fail closed instead. Which failure hurts the customer less?”

That last question is the FDE part: the trade-off belongs to the customer.

Token bucket or sliding window: how to choose out loud

Expect: “Why not a sliding window?” Know both well enough to compare them without notes.

Token bucketSliding-window log
BurstsUp to capacity at onceNever over the limit in any window
State per clientTwo numbersA timestamp per recent request
Work per callConstantDrop expired timestamps first
Easy to state“rate per second, bursts of capacity”“At most limit in any window”

A third option sits between them. A sliding-window counter keeps two fixed-window counts per client and weights the previous one by how much of it still overlaps the window, so it approximates the log with constant state. And name the one to avoid: a plain fixed-window counter lets a burst at the end of one window and another at the start of the next both pass. The sliding-window question has the log in code and the memory arithmetic.

How to choose out loud:

“If the contract says ‘at most this many per minute’ and a partner will audit it, I’d use a sliding log, because it matches the wording exactly. If the goal is to protect our servers and short bursts are fine, a token bucket is cheaper and friendlier to clients. Which one describes your limit?”

Finally, flip sides. In FDE work you may be the client being limited, not the server doing the limiting. A client that gets HTTP 429 should wait for Retry-After when it is present and back off with when it is not, or every client retries at the same moment. The retry jitter question covers that code, and our post on the API integration coding round puts it together with pagination and idempotency.

Is this really an FDE interview question?

You will see the claim repeated. One prep site lists “Implement a rate limiter (Anthropic favorite)” among common FDE coding patterns and cites no source. Source 3Forward Deployed Engineer Interview: The Definitive 2026 GuidePublisherExponent (Aced)Source typeinterview prep siteSource 4Anthropic Technical Interview Questions: Complete Guide 2026PublisherJobrightSource typeinterview prep site Another prep site lists “Design a rate limiter” as an Anthropic concurrency question and links one candidate’s Medium post. Source 3Forward Deployed Engineer Interview: The Definitive 2026 GuidePublisherExponent (Aced)Source typeinterview prep siteSource 4Anthropic Technical Interview Questions: Complete Guide 2026PublisherJobrightSource typeinterview prep site Neither shows what Anthropic asks.

What candidates and public repos show:

These are single reports and repos, not a pattern. The reason to practice anyway is the job: FDE code talks to customer and partner APIs you do not control, and those can refuse you with HTTP 429, as the Forerunner mock API does. Our lesson on what FDE coding rounds test sorts the evidence by company, and do FDE interviews have LeetCode? covers the split between practical builds and algorithm problems.

Before your coding round

  • Write the bucket from memory with an injected clock, and time yourself.
  • Test burst, partial refill, the cap after a long idle, float steps and the rounded-up Retry-After.
  • Explain the race in one sentence, then close it with a lock.
  • Explain why an idle bucket can be deleted without changing any answer.
  • Describe the atomic shared version and pick fail-open, fail-closed or fall back, with a reason.
  • Compare the token bucket with a sliding window, and end with a question for the customer.

Set a 25-minute timer, write the bucket from memory with an injected clock, then open the token bucket question and answer its follow-ups out loud before you read the model answer, which has the full Redis script and the fallback when Redis is down. Our question bank holds 180 interview questions, and the token bucket, sliding-window and retry jitter questions are free to read.

GlossaryToken bucketA rate-limiting algorithm that refills tokens at a fixed rate and allows bursts up to a capacity.More on Token bucketGlossaryForward deployed engineerA software engineer who builds and ships production systems inside a customer’s problem and environment, accountable to that customer’s outcome.More on Forward deployed engineerGlossaryExponential backoff with jitterRetrying with growing, randomized delays so that clients recovering from the same failure do not retry in lockstep.More on Exponential backoff with jitter

Questions people ask

How does a token bucket rate limiter work?

A bucket holds up to a fixed number of tokens and refills at a steady rate. Each request takes a token, and when none is left the request is rejected or delayed. The capacity sets the burst you allow, and the refill rate sets the long-run limit.

Should I implement a token bucket or a sliding window?

A token bucket allows short bursts and keeps constant state per client. A sliding-window log counts requests in the last window exactly but stores a timestamp for every request. Ask which behavior the customer needs, then say why you chose.

Is the rate limiter an Anthropic interview question, as prep sites say?

One prep site says it is an Anthropic favorite and cites no source, and another prep site lists it as an Anthropic concurrency question, linking one candidate’s blog post. Treat both as unverified, and practice it anyway, because the APIs an FDE integrates with can rate-limit you.Source 3Forward Deployed Engineer Interview: The Definitive 2026 GuidePublisherExponent (Aced)Source typeinterview prep siteSource 4Anthropic Technical Interview Questions: Complete Guide 2026PublisherJobrightSource typeinterview prep site

How do I make a rate limiter work across several servers?

Keep the bucket state in a shared store and update it atomically, for example with a Redis script that refills and takes a token in one step. Then say what happens when the store is slow or down, because failing open or failing closed is a decision for the customer.

Keep reading

Guides

Lessons

Questions

More in Interview rounds