Add support for tiled logs to ctclient

[lukevalenta]

ctclient does not appear to support crawling tiled logs from https://www.gstatic.com/ct/log_list/v3/all_logs_list.json.

For example:

./ctclient get-entries --first 0 --last 0 --log_name "Let's Encrypt Willow2025h1b"
F0310 14:01:50.693967    8341 root.go:131] No log with name like "Let's Encrypt Willow2025h1b" found in loglist "https://www.gstatic.com/ct/log_list/v3/all_logs_list.json"

[oz-mo]

Hello! I'm just checking in to see if there are any updates on adding support for tiled logs. With log providers beginning to transition to static/tiled APIs, this feature would be very helpful for the community.

[phbnf]

Hello!

What specific subcommands would you like to use with static-ct-api logs? I've been experimenting with a small client for static-ct-api logs in the TesseraCT repo over there transparency-dev/tesseract#556, which you might find handy! Once submitted I'm planning to add features to it as I need them.

At the moment, we're not planning to add support for static-ct-api logs in this google/certificate-transaprency-go. I'm hopping that two different tools (one for rfc6962 logs here, one for static-ct-api logs over there) could do the job to start with, and then we could merge them later if need be and if that makes sense from an API point of view.

[BrandesEric]

I would like an equivalent of the https://github.com/google/certificate-transparency-go/blob/master/scanner/scanner.go capability for tiled logs if possible?

Is there another client library that already does this in a different repo?

[lukevalenta]

Check out https://github.com/FiloSottile/sunlight/blob/main/client.go. Depending on whether you care if the tiles are authenticated, you might be interested in UnauthenticatedTrimmedEntries.

[BrandesEric]

@lukevalenta thank you! I think I could adapt this for my purposes. I didn't realize there was an UnauthenticatedTrimmedEntries option. That could be useful. Thanks!

[lukevalenta]

Sure thing! See https://groups.google.com/a/chromium.org/g/ct-policy/c/5T0VUHVghuU/m/hRDHM-39AgAJ for more discussion on why this was added. Clients should be authenticating tiles from CT (as logs aren't themselves meant to be trusted), but since many clients don't do this, the UnauthenticatedTrimmedEntries option provides a way to limit bandwidth costs both for those clients and the log operator.

[phbnf]

Thanks for the input! This library has some capabilities to tell precert and certs apart, do you use these features? Do I understand that you're looking for a library, rather than a binary for your usecase?

In the meantime:

• We've got the fsck tool in the TesseraCT repo, it iterates over entries in a tree. At the moment, it uses internal libraries, but we could change this based on input signals and as TesseraCT gets more mature. In the meantime, you can also copy the code.
• Sunlight's got a client as Luke mentioned! +1 to what Luke mentions for unauthenticated trimmed entries. But also, if you're a log client, it's a good idea to verify that entries roll up to a checkpoint - that's the whole point of a log.