Using filters (ex. Filter: blob:none)

[sbrptdev2]

How can filters be used to avoid downloading blobs for example? This is useful in a context where only specific meta-data is wanted, for example if you are writing a commit log inspection tool.

TL;DR:

Trying to implement git clone --bare --filter=blob:none --single-branch with gix.

[Byron]

Even though on protocol level, the filter can be specified, it's not currently wired so from gix that's not available.
You would probably be able to apply such a patch yourself, as it's really just about passing additional arguments to the fetch command at the protocol level. However, it's complex enough for me not to have a quick tip handy.

[sbrptdev2]

Bummer, I was hoping it would be more simple already. git2 allows you to do this (apparently) with fetch_opts.custom_headers(&["Filter: blob:none"]);

I'm also interested in streaming, rather than a full download prior to obtaining results. This is probably even less trivial to accomplish. An additional bonus will be adding in-memory object storage (ex. no disk writes needed). Of course you could hack this together with an out of band tmpfs (or similar ramdisk approach) mountpoint, but that's not as neat.

[Byron]

Bummer, I was hoping it would be more simple already. git2 allows you to do this (apparently) with fetch_opts.custom_headers(&["Filter: blob:none"]);

That's how I imagine it to work here as well one day. That would be the aforementioned 'wiring'.

I'm also interested in streaming, rather than a full download prior to obtaining results. This is probably even less trivial to accomplish.

One would have to drop onto protocol level to get control over the packed stream. In practice, it's already streamed to a file on disk while it's being indexed. Without having resolved the pack it's probably not very useful, as there is no easy object-access possible otherwise.

[sbrptdev2]

Low level is OK if the pack format allows to index in a way that can be abused via HTTP range headers. Using this, and proper offset arithmetic(s), you can likely skip across the stream to get only the data you want (meta-data).

[Byron]

Without an index for the pack, that's not possible unfortunately.