"Indexes" or "Indices"

[westonpace]

Both are valid words and both are used in the code today (in comments, variables, and APIs) "Indexes" is probably more common in the DB world. "Indices" is more common in our code today.

We should be consistent.

I don't think there is an objectively correct answer so just chime in if you have any opinion and in a week or two we can pick one and start to clean things up.

[westonpace]

I have a slight favor for "indexes" as it is the term used in other databases.

[Xuanwo]

indexes looks more natural to me

[jackye1995]

I remember we discussed this once (forgot where, maybe internally only) and we arrived at the conclusion that "indexes" is more commonly used, but we already have most places using "indices" and it is a bit hard to change it at this moment.

[westonpace]

Yeah, looking through things I think I am going to change my vote. We use indices in a lot of places that will be difficult to change (e.g. in the manifest)

[Xuanwo]

Sad, indexes is more easy to type for me 😂

[wjones127]

Related question: we really need a standard term for the individual index pieces (fragments? deltas? shards?). What should we call those?

[jackye1995]

To me there is a difference between terms like zone, fragment vs shard, partition, that the later ones carry a meaning that the divide has some sort of logical importance, e.g. based on a logical partition key or criteria like the vector space centroids. Zone and fragment are more like just you can draw any arbitrary boundary to divide the rows and each part is still a valid zone/fragment.

[westonpace]

Yeah, although I don't think we necessarily treat the "base" any differently, so in that sense delta is a misnomer.

Maybe not at read time, so maybe we don't need the distinction. However, when we train a delta index we reuse the centroids from the primary index, which I think was the original inspiration for the name. It does appear we still write the centroids in the delta index so a delta can "stand alone".

I think "Index Fragment" is somewhat viable. In some ways we can re-use more terms here. For example, merging two indexes could be called "index compaction".

I just worry about confusion with our existing fragment concept. For example, users might expect there to be one index fragment for each data fragment.

[westonpace]

Another potential confusion is that each fragment will have a fragment bitmap

[wjones127]

Fair points on "Fragment" being confusing.

How about "Index segment"?

This term is used by Lucene, for example: https://stackoverflow.com/questions/2703432/what-are-segments-in-lucene

[westonpace]

Ok, I can live with segment (I just switched to shard 😆 😢 )