CF_CTile

What is it?

CF_CTile is a caching proxy for the /ct/v1/get-entries and /ct/v1/get-sth endpoints of an RFC6962 Certificate Transparency (CT) log, implemented in JavaScript as a Cloudflare Snippet.

This project was developed for use with Sectigo's RFC6962 CT logs. It has a very similar goal to Let's Encrypt's CTile project, but it uses Cloudflare's CDN caching instead of S3 object storage.

Why is a caching proxy needed?

RFC6962 section 4.6 gives clients complete freedom when choosing values for the start and end parameters of get-entries requests. Consequently, get-entries responses cannot be cached effectively. Many real-world CT log deployments have struggled to handle the volume of get-entries requests that they receive.

How does it work?

CF_CTile borrows the concept of "tiles" from the Static CT API. Each tile has a fixed size of 256 entries. Whenever a client sends a get-entries request, CF_CTile rounds down the start parameter to the nearest multiple of 256 and requests exactly 256 entries from the backend. The responses to these backend requests are cached by the Cloudflare CDN. When a backend request is successful, CF_CTile generates the get-entries response by copying from the tile zero or more of the entries requested by the client.

RFC6962 allows the get-entries API to return fewer entries than requested by the client. For simplicity and due to Cloudflare's restrictions on subrequests to the same origin, CF_CTile never attempts to returns entries from multiple tiles in the same response. Consequently, no more than 256 entries will be returned for any get-entries request. If the start parameter is one less than the end of a tile, the response will include only a single entry. If the start parameter exceeds the most recent entry in a cached "partial" tile, the response will contain zero entries.

For get-sth requests, CF_CTile instructs Cloudflare to cache the response for 60 seconds. This dramatically reduces the volume of get-sth requests that the log server needs to handle, whilst ensuring that get-sth responses are still fairly fresh when delivered to clients.

Deployment instructions

Caching configuration

To ensure that "partial" tiles (i.e., get-entries responses containing less than the 256 entries requested by CF_CTile) are not cached for too long:

• Change Caching -> Configuration -> Browser Cache TTL in your Cloudflare account from the default setting (4 hours) to Respect Existing Headers.
• Create an API token that has the Cache Purge permission.

Creating the Snippet

• Click Rules -> Snippets -> Create Snippet.
• In the Snippet editor:
  • Paste the contents of cf-ctile.js.
  • Replace THE_ZONE_ID with your Cloudflare Zone ID.
  • Replace THE_API_TOKEN with your Cache Purge API token.
• Click Deploy, selecting All incoming requests as the Snippet Rule.

Log server requirements

Maximum get-entries size

It is essential that the log server is capable of emitting "full" tiles (i.e., get-entries responses containing 256 entries). For Trillian/CTFE, this means setting max_get_entries to at least 256; or if align_getentries is enabled, then max_get_entries must be set to exactly 256.

Caching headers

CF_CTile relies on the origin log server to set appropriate cache headers when returning a "full" or "partial" tile.

To ensure a high cache hit rate, "full" tiles should be cached "forever", or at least for a relatively long time. Trillian/CTFE sets Cache-Control: public, max-age=86400 to instruct Cloudflare to cache "full" tiles for 24 hours.

Caching of "partial" tiles can help when there are lots of clients tailing the log, but it is necessary to ensure that "partial" tiles are only cached for a short time. Trillian/CTFE sets Cache-Control: public, max-age=60 to instruct Cloudflare to cache "partial" tiles for only 1 minute.