How we improved our Rails app’s performance with Conditional Get Requests

Photo by Veri Ivanova on Unsplash

HTTP provides a method of client-side caching known as Conditional Get Requests. This style of caching allows a client to cache the content of a response locally (in your browser cache or mobile device). When the client makes a subsequent request to your application for the same resource, it includes a token or timestamp from their previous request. Based on this token or timestamp, if your application determines that the response body would be the same as the last request, the server may respond with a short, head-only, 304 Not Modified response. This instructs the requesting client to fetch the response body from its local cache store instead.

This method of caching can reduce latency and server burden, since it saves our application from having to fetch many records from our database and from having to render view templates when a resource hasn’t changed. For resources that tend not to change too often, or are requested multiple times by the same client without changing, Conditional Get Requests can be a simple and powerful tool for improving performance.

In May 2022, we conducted an experiment to determine if we could use this method of caching at Cookpad to improve response times on selected API endpoints. This post documents how we used this feature to shave up-to 150ms off our response times.

Beyond default behaviour

Modern web frameworks and networking libraries are quite full-featured. Our mobile clients (Android and iOS) were already sending the required headers for Conditional Get Requests with every GET request to our backend, but our API—a Ruby on Rails app—was effectively ignoring these and rendering responses afresh with each request.

Rails does, of course, support Conditional Get Requests straight out of the box, and our application would respond with a 304 Not Modified if it recognised a resource hadn't changed since the previous request. But our default implementation was still doing the work of fetching the same records from the database and rendering the same view template, regardless of the Conditional Get headers, so the performance benefit was minimal. To reap the benefits of client-side caching required some thoughtful configuration and refactoring.

Choosing our test subject

To find good candidates for client-side caching, we used Charles Proxy to monitor our own personal use of the mobile application. Certain endpoints stood out as being likely to be visited by a browsing user more than once in a short period of time, without the response content having changed. This made them a good candidate for client-side caching.

We nominated the /user/:user_id/recipes endpoint as a test subject. This is an endpoint in our API that presents all of the recipes belonging to a given user. Next, we discussed which method we would use to determine if a resource was changed or not.

HTTP supports more than one way of validating whether a resource has changed or not (called validators), and more than one validation method can be used in the same request. In this particular case, this recipes endpoint can present different data based on query parameters (such as page and per_page), so we decided that ETags were a more accurate way to validate if a resource had changed than a last modified timestamp.

Rails sends an ETag header with each response by default, which represents a hash digest (like a fingerprint) of the response body that is essentially unique. As mentioned before, our mobile clients were already sending this ETag back to the server in subsequent requests (as a header named HTTP-IF-NONE-MATCH). All that was required then, was that we update our controllers to only load resources and render views if the hash digest was different from the one expected.

Updating our controllers

To add this behaviour, we used two Rails methods: #stale? and #etag.

#stale?

The #stale? method provided by Rails will compare the hash digest of a given value or object with the value provided in the HTTP-IF-NONE-MATCH request header. If the two values are the same, #stale? will return false and the action can respond with a 304 Not Modified without having to load and render additional data. If the two values differ, #stale? returns true and the page should be loaded and rendered in full.

Here’s an example to help you visualise how this might look:

def index
  @user = User.find(params[:user_id])
  if stale?(etag: @user.updated_at)
    @recipes = @user.recipes
  else
    # head 304
  end
end

In this example, the ETag validator includes the current user’s updated_at timestamp. Behind the scenes, Rails will add this (and potentially other) values to an array and then hash all of those values to generate a unique hexadecimal string-the ETag. If that string matches the value in the HTTP-IF-NONE-MATCH header, #stale? will return false and the controller will respond 304 Not Modified.

#etag

Calculating whether a response matches an ETag or not is not always as straightforward as checking just a simple timestamp. Some endpoints include query parameters (such as pagination), and some endpoints render a resource differently depending on which user is making this request.

To factor these variable values into the entire digest for a response ETag, Rails provides the etag method. Simply call this method in the controller, with a block, to add any additional validators to the array of values for the final ETag digest.

For example:

etag { params[:page] }

etag { Current.user.id }

def index
  # ...
end

To make our lives easier, we added an additional class method to our controllers .include_params_in_etags to simplify adding parameters and their values to the validators array.

module ETagParamKeys
  extend ActiveSupport::Concern

  module ClassMethods
    private

    def include_params_in_etags(*param_names)
      Set.new(param_names).each { |p| include_param_in_etags(p) }
    end

    def include_param_in_etags(param_name)
      etag { params.key?(param_name) && "#{param_name}-#{params[param_name]}" }
    end
  end
end

A complete example

Here’s a complete example that connects the three concepts described above:

# The value of these parameters will be included as part of
# the array used to create the ETag digest
include_params_in_etags :page, :per_page

# The current User's ID will be added to the ETag, making it unique
# per authenticated User
etag { Current.user&.id }

def show
  if stale?(etag: etag_value)
    # Slow query we'd rather not make if we can avoid it
    @recipes = user.recipes.published.page(params[:page]).per(params[:per_page])
  else
    # head 304
  end
end

private

  def etag_value
    # This value is store in an in-memory cache and is super quick!
    @_etag_value ||= user.latest_recipe_updated_at
  end

  def user
    @_user ||= User.find(params[:user_id])
  end

Our implementation in this experiment was essentially the same as described above, but with some additional lines of code for collecting performance metrics and other such things.

To invalidate or not to invalidate?

There are only two hard things in Computer Science: cache invalidation and naming things.

— Phil Karlton

Implementing these changes in the code was fairly straightforward. The real difficulty came in determining the most effective yet reliable way of invalidating our cache.

Our initial PR was blocked, because other engineers started to spot potential cases where the response data might have changed and our ETag value wouldn’t detect it. This led to quite a long and detailed discussion in which we tried to identify as many potential cases as possible. In the end, we opted for a cached timestamp that would be invalidated every time a user or any of their recipes changed. This additional method call added an extra few milliseconds to the response time, but not enough to outweigh the overall benefit.

Results

We used Prometheus to record the response times for each request to this endpoint, and also to track whether the ETag was stale or not (a cache “miss” or a cache “hit”).

The results of this experiment are shown in the charts below:

Chart showing the “hit”, “miss”, and average response times (ms) for this endpoint

In this first chart, the Y-axis shows the response time (ms), and the X-axis is the time. The three lines indicate cache hits, cache misses, and the average response time for the endpoint (both hits and misses).

Chart showing the cache hit rate (%) for this endpoint

In this second chart, the Y-axis shows the % cache hits, and the X-axis is the time.

As you can see from these results, cache miss responses tended to range between 220ms and 310ms. Whereas cache hits were consistently around 95ms. The cache hit rate was around 25%, but we saw this growing steadily over time. This had the effect of bringing the average response time down between 30ms and 70ms, roughly 10–20%, and that improvement has increased with the cache hit rate.

Adding Conditional Get Requests improved response time dramatically for about 25% of requests, and improved the average response time for all requests to this endpoint. Not bad for a few hours of work!

Caveats

Implementing Conditional Get Requests in Rails is surprisingly simple.

However, it is important to ensure that the logic for determining a stale vs fresh cache is sound, and covers all scenarios. It is very easy to miss certain edge cases or certain combinations of variables, and doing so could mean serving stale, out of date content to a user.

We ended up writing several tests to thoroughly cover the cases we thought of, and to provide a layer of insurance that future changes wouldn’t break the cache behaviour.

Cache lifespan

Cached responses may live on a device for a long time. This means that improperly configured ETag values could result in users experiencing bugs or viewing out-of-date information for several days or weeks. Since these errors take place on the client-side, and not server-side, it is very difficult to know if problems arise unless our users start to complain!

This issue can be mitigated by customising the :cache_control option in the #stale? method to set a shorter cache expiry time, but that is outside of the scope of this article.

Proxies and privacy

It’s possible that, depending on the cache-control headers you set, caching a response body may also happen in one of the proxies along a network. You should ensure you understand the cache-control headers, and how these are configured for your application. Make sure this also aligns with your company’s privacy policy and other legal requirements too; if in doubt, err on the side of caution.

YMMV

The above described experiment demonstrates an average improvement of 10%-20% in response times for this endpoint. However, the impact of conditional get requests is likely to vary significantly for each endpoint. Your Mileage May Vary. I would encourage teams to A/B test the effectiveness of their implementation as part of their workflow, and compare the results with the default implementation, to prove that the overall benefit justifies developing and managing this feature.

Conclusion

Conditional Get Requests offer an easy and potentially powerful way to improve the response times for a given endpoint. In our experiment, we were impressed by how high the cache hit rate was, and how much faster cached responses were. However, it is essential that you ensure the conditions you have identified for cache validity are realistic and comprehensive.

Teams are advised to measure and benchmark their implementation to gain a deeper understanding of the impact (if any) their changes have made.

Remember to share your findings with others 🤓