Support for Indexes

[mdreizin]

Thanks @destel for creating rill — it’s truly amazing and saves a lot of time.

I’m wondering why it doesn’t support indexes. Currently, every callback like func(A) only accepts A, but in some cases it would be very useful to also have the index. For example, this would allow different processing for the first, second, even/odd, last, etc.

Is there a specific reason index support is missing?

There’s a workaround using a key/value pair or a tuple of two, but that requires iterating twice (first to create a temporary slice, then to process it), which isn’t ideal.

[destel]

Thank you very much for the feedback. I really appreciate it!

I originally built rill around the use cases I had in production: basic concurrency, stream processing, batching, and ordering. I just didn’t have a scenario where indices were needed, since I was mostly processing streams of unknown size (for example, streaming rows from a DB, transforming them, and then batch-inserting them into another DB).

That said, I’m open to adding index support directly to the library if there’s enough demand.

The good news is that rill doesn’t hide its internals, so it’s easy to create a reusable WithIndices function that attaches indices to items as they arrive. This approach is stream-friendly, doesn’t require multiple passes, but does add a few extra channel operations per item.

Full example here: https://go.dev/play/p/apYY3le10Tw

And here’s a copy-paste friendly version of the implementation:

arr := []string{"a", "b", "c", "d", "e", "f", "g", "h"}

in := rill.FromSlice(arr, nil)

inIndexed := WithIndices(in)

err := rill.ForEach(inIndexed, 2, func(item ValueWithIndex[string]) error {
	fmt.Println(item)
	return nil
})

fmt.Println("Done:", err)

Here's the playground link and a copy-paste friendly version of the implementation:

https://go.dev/play/p/apYY3le10Tw

type ValueWithIndex[A any] struct {
	Value A
	Index int
}

func makeValueWithIndex[A any](value A, index int) ValueWithIndex[A] {
	return ValueWithIndex[A]{Value: value, Index: index}
}

func WithIndices[A any](in rill.Stream[A]) rill.Stream[ValueWithIndex[A]] {
	res := make(chan rill.Try[ValueWithIndex[A]])
	go func() {
		defer close(res)
		i := 0
		for item := range in {
			res <- rill.Wrap(makeValueWithIndex(item.Value, i), item.Error)
			i++
		}
	}()
	return res
}

func WithoutIndices[A any](in rill.Stream[ValueWithIndex[A]]) rill.Stream[A] {
	res := make(chan rill.Try[A])
	go func() {
		defer close(res)
		for item := range in {
			res <- rill.Wrap(item.Value.Value, item.Error)
		}
	}()
	return res
}

[mdreizin]

Thanks @destel for taking the time to create a snippet showing how to use indices in rill. I came up with a different, not-so-rill-idiomatic approach, but I really like your snippet :)