Tracking Issue for slice::array_chunks

[lcnr]

The feature gate for the issue is #![feature(array_chunks)].

Also tracks as_(r)chunks(_mut) in #![feature(slice_as_chunks)] -- these were FCPed starting at #74985 (comment) below.

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also uses as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

• Restrict N to values greater than 0 at compile time instead of using a runtime panic. Make <[T]>::array_* methods fail to compile on 0 len arrays #99471
• Support calls where N == 0 is unreachable due to conditionals Tracking Issue for slice::array_chunks #74985 (comment)
  • It was decided that a runtime panic is appropriate: Tracking Issue for slice::array_chunks #74985 (comment)
• Adjust documentation (see instructions on rustc-dev-guide)
• Stabilization PR (see instructions on rustc-dev-guide)
  • slice_as_chunks: Stabilize slice_as_chunks library feature #139656
  • array_chunks

Unresolved Questions

• Iterator vs slices, Does [T]::array_chunks really need to be an iterator? #76354

Implementation history

• implemented in add slice::array_chunks to std #74373
• mut version in Add slice::array_chunks_mut #75021
• as_chunks{_mut} in Add [T]::as_chunks(_mut) #76635
• as_rchunks{_mut} and unchecked versions in Add as_rchunks (and friends) to slices #78818
• const as_(r)chunk in constify slice_as_chunks (unstable) #111453
• Stabilize slice_as_chunks library feature #139656

[thomcc]

Is there any intention of providing the reverse versions of these functions?

What would that look like?

[joshlf]

Same API, but iterating backwards. I have a use case in which I need to convert either a prefix or a suffix of a slice to an array reference. This API is perfect for the prefix use case, and I'd love to be able to use it for the suffix use case as well.

[newpavlov]

@joshlf
Both slice of arrays and custom iterator type would implement the DoubleEndedIterator, so your use case does not need a separate reverse method.

[joshlf]

That's only true if N evenly divides the length of the slice (see array_chunks docs). Otherwise, the remaining elements are only accessible via the remainder method, and won't be iterated over in the DoubleEndedIterator implementation.

[newpavlov]

Ah, I misunderstood your use case, you want [0, 1, 2, 3, 4, 5, 6] to be split into [4, 5, 6], [1, 2, 3], [0], not into [6], [3, 4, 5], [0, 1, 2]. It indeed requires a separate method.

[joshlf]

Yep, exactly 😃

[thomcc]

Yeah, array_rchunks is definitely worth having.

[sffc]

Also useful would be the inverse operation: [[0, 1, 2], [3, 4, 5]] to [0, 1, 2, 3, 4, 5]. (from &[[T; 3]] to &[T])

[ChayimFriedman2]

Do we need to wait until const_evaluatable_unchecked is stabilized so we can deny N = 0 at compile time to stabilize this feature? Perhaps we can just mention in the documentation that currently it panics but it may be uplifted to a compile time error in the future.

[fzyzcjy]

Hi, is there any alternatives before this is stable? Thanks!

[CryZe]

Hi, is there any alternatives before this is stable? Thanks!

Yeah bytemuck::cast_slice and bytemuck::cast_slice_mut can do mostly the same (with a Pod requirement though)

[fzyzcjy]

Thanks!

[sffc]

Another workaround is to map the slice chunks to array chunks.

my_slice.chunks(3).map(|s| {
    let mut a = [0; 3];
    a.copy_from_slice(s);
    a
})  // now you have an array_chunks iterator

[clarfonthey]

One potential use case that might be worth having a dedicated method:

impl<T> [T] {
    fn get_chunk<const N: usize>(&self, idx: usize) -> &[T; N] {
        &self[idx..idx + N].as_chunks::<N>().0[0]
    }
    fn get_chunk_mut<const N: usize>(&mut self, idx: usize) -> &mut [T; N] {
        &mut self[idx..idx + N].as_chunks_mut::<N>().0[0]
    }
}

If no one takes it up, at some point, I might implement this myself. Trouble is figuring out a name.

[jDomantas]

Do we need to wait until const_evaluatable_unchecked is stabilized so we can deny N = 0 at compile time to stabilize this feature? Perhaps we can just mention in the documentation that currently it panics but it may be uplifted to a compile time error in the future.

That is a stricter requirement - valid code then must prove that N != 0 in addition to avoiding calling array_chunks::<0>(). With your suggestion otherwise valid programs could start failing to compile:

fn foo<const N: usize>() {
    for chunk in get_slice().array_chunks() {
        //                   ^^^^^^^^^^^^ error: can't prove that N != 0
        do_whatever(chunk);
    }
}

fn main() {
    foo::<3>();
}

[Liamolucko]

Has a version of array_chunks for iterators rather than slices been considered?

[clarfonthey]

Most of the iterator adapters that have been slowly added to libstd made their start in itertools, so, you should probably start by looking what that offers, and then after that make a case for why it should be in libstd.

From what I understand, though, a lot of those methods are slowly being added to the Iterator trait directly, though.

This isn't really the place for that kind of discussion, though, and it might be best to open a separate issue in the RFCs repo, on Zulip, or elsewhere.

[mina86]

Should we also have a TryFrom implementation which converts slice into
chunks and fails if the slice cannot be evenly divided? I found that
I often need such an operation and it could be more concise if there
was a try_from that I could map_err or unwrap when checking for empty
reminder. With as_chunks I need to interrupt the flow of the code by
adding an if !reminder.is_empty() { ... } check.

#105316 is how this would look.

[joboet]

With auto traits and negative impls, the check for empty arrays can be made via the trait system, so it applies even in cargo check:

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=67c14ab8d25e50d6b2047c2c2ce85091

[lcnr]

it can not, because partially applicable negative impls aren't really supported for auto traits rn, and once/if they are, they won't do what you expect them to do:

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=c0ed93fccc6cfd16e067e22bc834956e

[mina86]

Is https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=c7bf56d1a1d409c85e4fc514512bbd355 not an option?

[yizhepku]

I think there should be a safe version of as_chunks_unchecked.

Usually, an "unchecked" version of a function does the same thing as their checked counterpart. Currently, this is not the case.

I propose we rename the current as_chunks_unchecked to as_chucks_exact_unchecked and then add as_chucks_exact.

[c410-f3r]

What is the status of this feature?

[c410-f3r]

Looks like that checking of N == 0 is the only blocker, right? If so, then we probably already have the answer from the responsible party.

• Make <[T]>::array_* methods fail to compile on 0 len arrays #99471
• split_array: remove runtime panics #109049

In other words, it seems that officially runtime checks are preferable over compile-time checks so can we proceed with stabilization?

[jhpratt]

Permitting something to compile knowing with 100% certainty that it will panic seems like an odd decision to make, given that we have the capability to prevent it. Post-monomorphization errors aren't ideal, but surely it's better than an unconditional runtime panic?

[jDomantas]

The problem is that it's not "100% certainty that it will panic". There was an example in another thread (#99471 (comment)) of code that is perfectly legal, but adding post-monomorphization error makes it fail to compile:

fn f<const N: usize>() {
    if N == 0 {
        ...
    } else {
        foo.as_chunks::<N>()...
    }
}

fn main() {
    f::<0>();
}

[jhpratt]

Ah, that makes sense. So unless we can solve the halting problem, a runtime error it is…

[mina86]

If the choice is between stabilising it as is with a runtime error, and keeping it experimental until static if and where N > 0 or N: NonZeroUsize becomes a thing, I vote for keeping this experimental.

[joboet]

I haven't found the time to work on it for the last few weeks, but I think my pre-RFC for pattern types has a good solution for this when combined with rust-lang/project-const-generics#26.

[suprohub]

very long isue

[jhpratt]

Now that inline const is stabilized, we should be able to make compilation fail when N == 0. It's not perfect (as mentioned in #99471, but it would be closer to the desired behavior. Likewise for array_windows (#75027)

[Xiretza]

I think the lang team has made it clear repeatedly that they prefer runtime panics over post-mono errors. Personally I don't agree, but I don't think it's worth arguing over anymore.

[jhpratt]

I don't agree either, but it's also not up to me. This issue has it crossed out, but that doesn't appear to be a formal decision in either direction. The docs currently say

This check will most probably get changed to a compile time error before this method gets stabilized.

So it seems to remain an open question?

[c410-f3r]

A reduced example that should compile according to the responsible team.

pub fn foo<const N: usize>() -> i32 {
    const {
        assert!(N != 0);
    }
    1
}

pub fn bar<const N: usize>() -> i32 {
    if N == 0 {
        2
    } else {
        foo::<N>()
    }
}

fn main() {
    assert_eq!(bar::<0>(), 2);
}

Maybe something about the PR that unblocked inline constants (#122568) could help here?

There are then 4 scenarios for stabilization:

1. Compile-time checks with the current CTFE behavior.
2. Compile-time checks with a machinery that allows the above snippet.
3. Tracking Issue for slice::array_chunks #74985 (comment)
4. Runtime checks, which I also personally disagree.

cc rust-lang/rfcs#3582
cc #122301

[jhpratt]

Thanks for that example. I've added it to the top comment.

[its-the-shrimp]

I haven't seen it discussed here yet, but another way to tackle the case of N == 0 is to not tackle it and just make the iterator return [T; 0] until the end of time, what do you all think about this?

[scottmcm]

and just make the iterator return [T; 0] until the end of time

That would mean the iterator can't be ExactSizeIterator, so I think it's unacceptable.

If people want iter::repeat([]) they should use that; I don' think getting that from array_chunks is ever what anyone calling array_chunks actually wants.

[ais523]

On the subject of the "what should happen if the const argument is 0" debate, I'd like to present what I think is an analogous situation:

fn divide<const D: u32>(n: u32) -> u32 {
    n / D
}

fn divide_checked<const D: u32>(n: u32) -> Option<u32> {
    if D == 0 {
        None
    } else {
        Some(n / D)
    }
}

fn main() {
    dbg!(divide_checked::<5>(99));
    dbg!(divide_checked::<0>(99));
    dbg!(divide::<5>(99));
    dbg!(divide::<0>(99));
}

Playground.

The current behaviour of Rust in this situation is to compile with no errors or warnings (not even on the call to divide::<0>), and to panic at runtime. In other words, if the divisor is coming from a const generic, there's no check for validity, even though division by 0 is always meaningless, and even though divide::<0> always divides by 0 and can easily be observed at compile time to do so.

Meanwhile, if you just write 99/0 directly, you get a deny-by-default lint, unconditional_panic, which makes sense given that the code can never run successfully.

I don't think the situation of, e.g., [T]::as_chunks::<0> is significantly different from that of division by constant zero – nor do I think there would be any particular benefit from making them act differently. I also suspect that there is unlikely to be a better solution than "[T]::as_chunks::<0> panics at runtime if it is ever called, and produces a deny-by-default lint if you write the 0 explicitly, but not if the 0 is calculated based on a constant generic" – this gives the compile-time error we want in the cases where the statement is obviously meaningless, but allows the code to compile in cases where it might not be called due to expanded generic code.

It's also worth noting an edge case:

let fn_pointer = <[u32]>::as_chunks::<0>;

Do we want to allow this, or not? With the explicit ::<0>, I can see a reasonable argument for throwing the unconditional_panic lint, with a message along the lines of "this function pointer will panic if it is ever called". I can't see a use for writing it directly (although I can see how it could potentially happen in expanded generic code, which could end up never calling the function in cases where the parameter is 0).

[Noratrieb]

The remaining issue here is what to do on as_chunks::<0>(). The current implementation panics, but with inline const it would also be possible to turn it into a post-mono error instead.

#74985 (comment) it is mentioned above that apparently the lang team prefers runtime panics over post-mono errors, but I don't see any links to discussion about this or justification for why. Runtime errors do have the advantage that they won't be triggered in dead code.

The arguments for making it a post-mono error are clear: getting errors earlier (and at all if the code is not tested). Post mono errors won't error in cargo check, but will error in cargo build. Runtime panics won't error in cargo check, won't error in cargo build either, but will only (potentially) error in cargo run/cargo test/cargo oops-its-prod-already.

Other comments above make additional arguments in favor of post-mono errors, and the documentation of the methods also intended to make it a post-mono error eventually.

[traviscross]

@rustbot labels +I-lang-nominated

I'm not sure whether what was mentioned about our lang preferences is actually true, so let's nominate to discuss.

cc @rust-lang/lang

[tgross35]

I don't remember what exactly the argument against post-mono errors was but I do remember thinking "that does makes sense", even though I prefer an earlier error. I want to say @BurntSushi may have originally expressed the concern?

If we do go with the runtime panic, we should document that a future Rust version is allowed to reject N = 0 earlier. (It would be nice if this could be a where N > 0 bound at some point in the future, but that would be more breakage than runtime->post mono).

[cuviper]

I think #74985 (comment) is a good example why a runtime check is desirable, so users can be flexible in their own APIs around this. If I write as_chunks::<N>() where N might be zero in unreachable code, then I don't want an error.

[mina86]

That’s current limitation in expressiveness of the language, no? I can easily imagine:

impl [T] {
    fn as_chunks<const N: NonZeroUsize>() -> (&[[T; N]], &[T]);
}

fn foo<const N: usize>(slice: &[T]) -> Option<i32> {
    let chunks = const if let Some(N) = NonZeroUsize::new(N) {
        slice.as_chunks::<N>().0;
    } else {
        return None;
    };
    todo!()
}

That would require a few things from the language:

• Ability to take non-primitive types as const generic arguments.
• A ‘const if’ construct similar to C++’s ‘if constexpr’.
• An implicit conversion from integer literals to NonZeroX types. (Not strictly necessary but ergonomics would suffer greatly without it).

To me it seems that the question is whether we want to wait for all that or stabilise.

[traviscross]

Maybe neither here nor there, but note that the following works:

const fn divide<const D: u64>(n: u64) -> u64 {
    const { assert!(D > 0) }; // <--- Post-mono assertion.
    n / D
}

const fn divide_checked<const D: u64>(n: u64) -> Option<u64> {
    match D {
        0 => None,
        1.. => Some(divide::<D>(n)),
    }
}

fn main() {
    dbg!({const { divide_checked::<1>(1) }});
    dbg!({const { divide_checked::<0>(1) }}); // <--- Look here.
    dbg!(divide_checked::<1>(1));
}

Playground link

That is, if the function within which we want to compile-time assert is parametric over a type or const generic, is const, and we're always in a const context when we'd see the "bad" generic argument for it, then something like what we want here does work currently.

(The question of whether we'll guarantee to not later disallow this is #122301.)

Interestingly, this is the odd case of a call that we allow in a const context but disallow outside of one.

[the8472]

From the options in #74985 (comment) we would like a combination of point 2 and 3.

Runtime panics are not great because they fail it compiles, it works or make illegal states non-representatble principles, for precisely the thing that we're making legible to the type system.

The current CTFE approach, i.e. post-mono errors, is suboptimal because it would not be forwards-compatible with pattern types or where bounds. It would also bet difficult to work with for a more-generic caller. The const-trick from #122301 ¹² would help with the latter issue but not with the former. It's also a somewhat obscure trick.

So we'd like where N > 0 or where N: usize is 1.. bounds on array_chunks. That's the first language feature we'd need. But on its own that would be insufficient for generic callers because they wouldn't have any way to generalize to all N via some fallback.

So the second desired feature is a way for generic callers to locally discharge the restriction. E.g. const if, inline where expressions or unifying const generic impls

// std function
fn chunks<const N: usize>()
where
    const { N > 0 },
{
    // ...
}

// user code
fn use_chunks<const N: usize>()  {
    where N > 0 {
       chunks<N>();
    } else {
       fallback();
    }
}

¹ That issue was actually motivated by this one, but it turned out insufficient.
² Additional prior discussion: rust-lang/rfcs#3582 (comment)

[Noratrieb]

I agree with you that this would be a nice way to implement this. But I think this feature is so incredibly far off that we really shouldn't be blocking this very useful helper (that replaced chunks_exact, which has a lot more room for bugs) for literal decades because of one of its weird error edge cases.
Passing 0 is a mistake, and I don't think a lot of people will run into it. It's an obscure edge case, and I think both suggested ways of detecting it (post mono error or runtime error) are perfectly fine in practice, so I really think we should stabilize it with one of the two (whatever you prefer) instead of blocking it for decades.

[Noratrieb]

I would suggest going with a post mono error, as that has significant advantages. The only disadvantage is that it may impede flexibility. After stabilizing, in the post mono error, we could link to a tracking issue where people can complain if their use case was blocked by this inflexibility, and we should be able to make it a runtime error in case it's a problem (since that's backwards compatible as long as we document that people shouldn't rely on it being a post mono error).

[workingjubilee]

Would any of the people involved in the current state of the relevant const generic bounds involving pattern (types?)... I'm not entirely sure if I should be asking @oli-obk or @BoxyUwU... be able to weigh in on the projected development cycle for such a feature as libs-api is asking? @Noratrieb has projected "literal decades" and I have no reason to doubt such a guess, currently, but are we talking 1 or 10?

[oli-obk]

Uh, please do not put pattern types into the consideration for any API decisions. They are an internal language feature and will stay so for the forseeable future. Getting them anywhere production ready is definitely off far enough that waiting for them is at present like waiting for specialization to be stable. If you'd like to know more, join https://rust-lang.zulipchat.com/#narrow/channel/481660-t-lang.2Fpattern-types

[BoxyUwU]

I think there is basically no universe in which we get N > 0 bounds in the next 2-3 years, it wouldn't surprise me if it took much longer than that even. Supporting fn foo<const N: NonZeroU32> is likely to be available much sooner (maybe the next year or so?), though the ability to take a generic const N: usize and pass that to a const M: NonZeroU32 is once again very far off. rust-lang/project-const-generics#26 I also do not see happening for a long time, it would definitely be blocked on -Znext-solver being stable, and I don't know that anyone has really sat down and properly thought out how it should even be implemented (so lets also say definitely not happening in the next 2-3 years for this too).

One thing that could be interesting is adding an unstable attribute for const generics that requires the argument to be fully concrete, i.e. forbid foo::<N> and only allow foo::<1> or foo::<{ MY_CONST_ITEM }>. This combined with PMEs would be fully forwards compatible (ish?) with whatever design we wind up with down the road for bounding const parameters (though it would rule out using const N: NonZeroUsize in the future, but possible we could add some hacks to the compiler if that winds up being the desired route). This wouldn't be as flexible on stabilization as desired but it seems more flexible than an error saying the function is unstable.

[the8472]

Ok then our options are

• runtime panic, no forward compatibility, more-general callers can use regular control flow
• postmono error as-is, no forward compatibility, more-general callers have to use What are the guarantees around which constants (and callees) in a function get monomorphized? #122301 forever
• wait for const N: NonZeroUsize to become usable. more-general callers would have to use What are the guarantees around which constants (and callees) in a function get monomorphized? #122301 but could (much) later upgrade if new language features that make it possible to construct one from a where N > 0 or something like that
• request a lang feature to require concrete calls so we can maybe upgrade later, but it [the upgrade] will be much later too

Uh, please do not put pattern types into the consideration for any API decisions.

I was gesturing at a general category of features to solve our problems by limiting the range of valid const values. If NonZeroUsize is easier than pattern types or where bounds that's useful information. If there are any other options then we'd like to hear them too.

[oli-obk]

I think just a mono time error seems fine. There are enough APIs that we would like to improve with better const checks, this one isn't particularly special. We can just ship it and handle it however we'll handle these in the future. Even a panic doesn't seem bad, we do that for lots of APIs in libstd

[the8472]

There are enough APIs that we would like to improve with better const checks

Which stable ones are you thinking of? There are a bunch of array ones like this one that have been kept unstable due to this issue.

We can just ship it and handle it however we'll handle these in the future.

I don't think anyone has mentioned a plan how to handle these other than living with the limitation.

[crumblingstatue]

One thing that could be interesting is adding an unstable attribute for const generics that requires the argument to be fully concrete,

I think this might be worth looking into, if it's not too difficult to implement.
It would allow maximum forwards compatibility, and may also be helpful in stabilizing other const generics dependent APIs, like array_windows.

[BoxyUwU]

request a lang feature to require concrete calls so we can maybe upgrade later, but it'll be much later too

for the record I don't expect the lang feature for concrete calls to take very long. it's the kind of thing that could probably be implemented in a day or so. but the transitioning off it to a proper type system feature for requiring N > 0 would be much later

[traviscross]

@rustbot labels -I-lang-nominated

We discussed this in lang triage today. In particular, it had been asserted and then repeated in this thread that the lang team prefers runtime panics over post-mono errors. That's not the case. We like errors to happen as soon as possible. So we prefer pre-mono errors over post-mono errors when possible, but similarly we prefer post-mono errors over runtime errors when that is possible and practical.

It's of course for libs-api to decide what's practical in this case.

Regarding what counsel we can provide on the language direction and timeline here, probably all of us want something in the spirit of const if, but as has been discussed on this thread, that or other similar solutions along this axis could take awhile.

[workingjubilee]

Sounds good.

It seems like our preferred paths are the slightly hacky #[rustc_your_argument_must_be_this_concrete_to_pass] or the monomorphizing error.

As the compiler magic that prevents jamming in random expressions is

• forward-compat with moving to a monomorphizing error
• forward-compat with moving to a panic
• NOT forward-compat with moving to a NonZeroUint type unless...
• ...with coercion from uint types to NonZeroUint, where it is forward-compat

It seems like the logical first move would be the enforced concrete arguments, then?
It feels to me that we would not want NonZeroUint for this argument without something that could coerce values found to be in the, er, nonzero range? Because I expect we'll see something like this a lot in code: data.array_chunks::<16>()

[crumblingstatue]

One of the truly magical things about array_chunks (and similar APIS, like array_windows) is that the length can be inferred, so you can write very ergonomic code like

for [a, b, c] in slice.array_chunks() {
    // do stuff with a, b, and c
}

If we want a solution that enforces a non zero uint, we should take care that it doesn't break inference.

EDIT: Also, the inferred length should count as "concrete", if we implement a concrete enforcing attribute.

[scottmcm]

I would say that lang wants compile-time checks where feasible, but also wants to avoid nuisance compilation failures that you can't suppress even though the thing it's complaining about is only in dead code.

TBH I'm tempted to just give this a runtime assert and a custom lint that's deny-by-default if it notices N is ever always zero. That'll catch the more likely mistake of somehow unintentionally inferring it as zero (maybe you typed if let ([[]], tail) instead of if let ([], tail), or something) without being annoying for the "look, I have a runtime check for N != 0 before calling it" case. And it's a really useful helper (though I'm of course biased) so I'd rather it be stable than perfect.

I'm reminded of the runtime assert in https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.as_flattened, where I just think it's fine, and better than blocking arrays-of-ZSTs entirely in post-mono. (Though admittedly that one requires the combination of two unlikely things, not just one.)

[sffc]

Maybe just give chunk size 0 a well-defined behavior such as never evaluating the loop expression?

And there can be a runtime panic in debug mode like for integer overflow.

I'd like to avoid a runtime panic being unconditionally compiled into code for N > 0. If we can assume that the compiler will eliminate it, maybe that's fine.

[GoldsteinE]

It’s an if on a const expression, so surely it will be optimized out.

[Qix-]

Just a thought, could array chunks simply return an Option and document that it returns None if the window size is 0? It could even be extended such that if the windows size is 0 and the slice length is 0, that's well defined, returning Some.

Since the window size is constant, then in most cases it'd return Some unconditionally, which I assume would make any downstream users subject to constant folding optimizations. Also makes the behavior explicit, without needing any unstable syntax stuff.

[its-the-shrimp]

Just a thought, could array chunks simply return an Option and document that it returns None if the window size is 0? It could even be extended such that if the windows size is 0 and the slice length is 0, that's well defined, returning Some.

Since the window size is constant, then in most cases it'd return Some unconditionally, which I assume would make any downstream users subject to constant folding optimizations. Also makes the behavior explicit, without needing any unstable syntax stuff.

This'd make the runtime value depend on the compile-time argument, which is not too good in terms of developer experience

[sffc]

Maybe just give chunk size 0 a well-defined behavior such as never evaluating the loop expression?

Thought: with this approach, the loop expression becomes dead code, so it will generate a compiler warning.

[Kixunil]

Because I expect we'll see something like this a lot in code: data.array_chunks::<16>()

This is true. In a large-ish codebase I co-maintain we have 10 instances of chunks_exact. Every single one of them has a constant argument. Getting array_chunks there would be great.

Honestly I can't even think of a case when it'd make sense to need if N == { fallback() }. I don't think it's realistic that anyone will ever need it. This method is generally used for very specific purposes and there's no way to fulfill that purpose if your argument is 0. So postponing stabilization just to allow this fringe use cases doesn't make sense.

Regarding NonZeroUsize, I'd very much like to see that with coercion. It's indeed true that 9 out of those 10 cases I found use a literal, the other one uses a constant. Even having a general mechanism for these coercions could be very useful for other things as well:

pub struct EvenLen(usize); // useful for hex stuff?

impl EvenLen {
    #[const_coerce_literals] // this function gets implicitly called in const context when a literal is coerced
    pub const fn from_usize_unwrap(len: usize) -> Self {
        assert_eq!(len % 2, 0);
        EvenLen(len)
    }
}

But still, this can be added later. My point is that the functions really are used with literals so having NonZeroUsize without coercion might be even worse than runtime panic.

So in conclusion, I think the proposal requiring the generic argument to be concrete makes a lot of sense, is already very useful and and I don't see anything wrong with it. I'm even going to give a shot writing a helper extension trait to have array_chunks in stable.

Also what @crumblingstatue said about inferred values is interesting but ultimately if it's going to delay stabilization I think it's not worth it since spelling the constant is not that big deal.

[scottmcm]

Come to think of it, the fact that we now have split_first_chunk and friends would help avoid a bunch of the places where .as_chunks was particularly helpful for conditional things. .as_chunks()[0] is no longer worth doing, since .first_chunk().unwrap() is better -- and supports N == 0 just fine.

---

To address something from the notes, I do think that inferring the length is useful, even if there was a restriction that it needs to be concrete.

For example, it's handy to

let (chunks, tail) = foo.as_chunks();
for x in chunks {
    bar(usize::from_le_bytes(*x));
}

which feel really nicely magical with the inferred length.

So it'd be nice if that can still work, even if a generic parameter as the length were blocked for a bit.

---

@Kixunil Out of curiosity, of your chunks_exact calls, how many of them would prefer array_chunks and how many would be as_chunks? It would be interesting to know which things are better when (cc #76354).

[Kixunil]

@scottmcm good points!

All of the cases use iterators but that's survivorship bias - chunks_exact doesn't support random access so if we need random access anywhere we are not using the method there but something else. However I can't think of a good regexp that would find those places and I also can't think of where we would need it. Everything is simple enough to use iteration.

Except splitting/subslicing of arrays.

We have a bunch of places where we need to split arrays, not slices into sub-arrays of known length. So the operation is statically known to not fail in theory but not directly provable to the compiler. I wrote an extension trait for arrays that allows such splitting (also compile-time asserting that the length of the sub-arrays sums up to the original array) and even found a bug in our code. Pretty cool. So code like let foo: &[u8; 8] = bar[0..8].try_into().expect("statically known len"); becomes let foo = bar.split_array::<8, 8>(); if bar: [u8; 16] with the failed check causing post-mono error. I think this would be likely useful addition to core as well but more hairy than as_chunks/array_chunks because, as opposed to the .. operator, this one takes two lengths of the arrays and also one of the lengths is in principle useless. So I think long-term core will want to have just one parameter and stabilizing a method with two parameters is not great.

Side note: I've implemented as_chunks rather than array_chunks so that once its iterated it has the unstable TrustedLen implemented to get access to optimizations. However now that I think about it I could've just added chunk_arrays (too) returning core::slice::Iter<'_, [T; N]>. But it does inform about #76354 a bit: in one case there was a check for remainder being empty which got much simpler (though could've been that simple with chunks_exact too) and in all other cases we were iterating within array where typing the extra .0 was not too obnoxious (we were also passing it into zip, so no .iter() was needed. But it also reminds me we probably want to have something like array_split_divisible which compile-time-asserts that remainder is empty.

[fintelia]

In my code, the two versions that would be useful are as_chunks for cases where I want to process the remainder, and a different method that returned only the chucks but panicked if the slice length wasn't divisible by the chunk size. For instance, when working with RGBA data represented as a byte slice, you probably don't want to continuously add assertions about the slice length but it feels a bit ugly to use array_chunks and just silently ignore the the remainder.

[Kixunil]

@fintelia sounds like you may want to pass around &[[u8; 4]], proving divisibility, instead of &[u8] so that you don't need to check (and panic) and only perform the check in the top-most function. I wouldn't be surprised if this also has additional performance benefits.

[fintelia]

@Kixunil yeah that could help for certain cases. Though something like this part of the png crate couldn't really benefit. The problem is that there's multiple different pixel formats supported, with some code generically handling them all as byte slices and other parts matching on each of the possible sizes and doing array operations on them.

[Kixunil]

@fintelia I'd use enum Pixels<'a> { One(&'a [u8]), Two(&[[u8; 2]]), Three(&[[u8; 3]], ...) instead of having BytesPerPixel and the pixels separated. I also see a lot of opportunity to reduce duplication in that code.

[Noratrieb]

This is getting really off topic, please stay on topic on tracking issues to keep them organized and avoid sending out too many notifications to people that are subscribed.

[Amanieu]

We discussed this in the libs-api meeting today. Overall we are happy with the current behavior using a runtime panic. Although a compile-time warning would be helpful, the scenario where it would be most useful likely involves generic wrappers forwarding N and we can only really warn at the immediate call site of array_chunks where N is not yet known.

@rfcbot merge

[rfcbot]

Team member @Amanieu has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[Kixunil]

@Amanieu can someone actually show a production code that needs it to be panic rather than post mono error? Not hypothetical. So far I've only seen snippets saying if N == 0 { what_the_hell_do_you_even_want_to_do_here()??? } else { do_something_senible() } but none of them in real code.

I personally highly doubt this will ever be needed but that's not even that important: going from compile-time error to runtime error is not a breaking change but the reverse is. The more conservative approach (stabilizing with post-mono error) could've been taken for now and revisited in the future if there are people explaining how it limits them.

Frankly, I don't think Rust should add panicking APIs lightly. They lead people to get used to panics. I have a fairly recent cautionary tale. Because Rust had historically poor support for arrays (this got significantly better recently) a security-critical library I co-maintain had to operate on slices. We started changing that but we had a bunch of code in the form of u32::from_le_bytes(foo[x..(x+4)].try_into().expect("len is 4")) because there wasn't a better way in the standard library and until recently nobody realized it's possible to cause post-mono error. And because we had a bunch of code people got used to it to the point that two reviewers approved similar conversion in completely invalid context where the input comes from user. This could've lead to DoS vulnerability.

Thankfully, soon after that merged and before release I happened to realize that we can statically eliminate these panics by stuffing them into an extension trait that causes post-mono errors if used incorrectly. And that made me find that panic, along with another unrelated bug. (BTW we had a similar issue with unsafe creeping in that was resolved similarly. Thankfully we had no bugs there.)

To be clear, I'm not saying that runtime panic is certainly wrong here. If my assumption, that all code uses statically-known values, is true then all code will just hit those proposed warnings. But it does look very weird that a certain error is turned to runtime panic just to allow edge cases that could've had a separate _rt_checked method. And it also isn't great to enforce the "panicking is usual API design" sentiment.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[suprohub]

Yo finally

[scottmcm]

@Amanieu Can you clarify exactly which APIs you're stabilizing as part of this? Is it all of

• https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.array_chunks
• https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.array_chunks_mut
• https://doc.rust-lang.org/nightly/std/slice/struct.ArrayChunks.html
• https://doc.rust-lang.org/nightly/std/slice/struct.ArrayChunksMut.html
• https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.as_chunks
• https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.as_chunks_mut
• https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.as_chunks_unchecked
• https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.as_chunks_unchecked_mut
• https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.as_rchunks
• https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.as_rchunks_mut

or just a subset?

(This issue grew a bunch of stuff over the last almost-half-a-decade...)

---

And it also isn't great to enforce the "panicking is usual API design" sentiment.

Note that I think there's a huge difference between "panics based on runtime value" and "panics based on const-generic value". Testing all the possible runtime values is hard, and they can come from user input. But panicking on a const generic, in a way that's "well if you ever run the monomorphization you'll see it", is way less of an issue.

It's like how, yes, https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.offset_from_unsigned can panic, but it only panics if the pointee is a ZST. And thus it's not that big of a deal, as the specific monomorphization either always panics or never panics.

[Kixunil]

But panicking on a const generic, in a way that's "well if you ever run the monomorphization you'll see it", is way less of an issue.

That is certainly true in the vacuum. My story gives some perspective on what happens if it becomes a common pattern. The thing is, in that example all relevant values were hard-coded. If they were wrong they'd also always panic. But the issue is we got used to those conversions being the standard so much that a wrong conversion looked like the right one.

To map it to this method, one way this could go wrong is if people indeed started using if N == 0 often they might get so used to it that one day they don't notice that the check is if n == 0 and n is untrusted, so setting it to 0 is a DoS exploit. If used in a nested generic function it'd not emit any warning. And maybe for most, if not all, the people there's a more appropriate solution than doing that check but because the check is easy they weren't force to a more creative/principled solution. Just like we've completely missed that get_array::<const LEN: usize>(offset: usize) -> Option<&[T; LEN]> in an extension trait is possible and a good idea.

[RalfJung]

I thought the current policy was to always do FCP on the stabilization PR, not the tracking issue?

In this case, I think const-stabilizing these functions will require const-stabilizing an intrinsic (exact_div), which requires t-lang fcp. Preparing the stabilization PR would uncover such points so we don't need to wait for a second round of FCP.

[traviscross]

I thought the current policy was to always do FCP on the stabilization PR, not the tracking issue?

It was reconsidered in the last libs-api meeting and decided that if that were to be changed, other related changes would need to be made to facilitate it, such as moving the discussions off of the tracking issues (or some other solution) so that the conversations would not be split between the tracking issue and the stabilization PR. It was mentioned that libs-api had previously done the FCP on the stabilization PR and that the experience was considered to have been a failure, leading to the process of doing it on the tracking issue. So the reasons behind the previous experience of failure with the process need to be explored and the causes resolved.

The divergence in experience is quite interesting, as on the lang side we've found doing the stabilization FCP on the stabilization PR to be a significant process improvement.

In this case, I think const-stabilizing these functions will require const-stabilizing an intrinsic (exact_div), which requires t-lang fcp. Preparing the stabilization PR would uncover such points so we don't need to wait for a second round of FCP.

Agreed. Similarly, apropos of @scottmcm's question, doing it on the stabilization PR helps to make the exact scope of the stabilization more clear.

[traviscross]

In this case, I think const-stabilizing these functions will require const-stabilizing an intrinsic (exact_div), which requires t-lang fcp.

On this matter, of course, whoever is handling the stabilization, please put up a PR to stabilize the intrinsics and lang-nominate that. We could make our FCP contingent on libs-api deciding to stabilize the functions here so the FCPs could run in parallel.

[RalfJung]

@rust-lang/libs-api it would be really good to get clarity on what exact set of functions and types is being proposed for stabilization here, as currently it is unclear.

[scottmcm]

I put up and FCP for exact_div in #139163 -- I think we might as well just const-stabilize it, because it's not doing anything scary in the first place, and it's not like we can't still delete the intrinsic later if it comes to that anyway.

[RalfJung]

It was reconsidered in the last libs-api meeting and decided that if that were to be changed, other related changes would need to be made to facilitate it, such as moving the discussions off of the tracking issues (or some other solution) so that the conversations would not be split between the tracking issue and the stabilization PR. It was mentioned that libs-api had previously done the FCP on the stabilization PR and that the experience was considered to have been a failure, leading to the process of doing it on the tracking issue. So the reasons behind the previous experience of failure with the process need to be explored and the causes resolved.

The divergence in experience is quite interesting, as on the lang side we've found doing the stabilization FCP on the stabilization PR to be a significant process improvement.

Yeah that is somewhat surprising. In particular with const stabilization, it can be hard to predict what exactly needs to be stabilized for this to work, such as extra intrinsics or some use of #[rustc_allow_const_fn_unstable]. So even if the FCP happens on the tracking issue, I would recommend a requirement that the stabilization PR must already exist so there is a clear idea of what is being proposed for stabilization.

[Amanieu]

@rust-lang/libs-api it would be really good to get clarity on what exact set of functions and types is being proposed for stabilization here, as currently it is unclear.

It wasn't discussed in the detail in the meeting, we mostly focused on the N == 0 issue. I'll bring it up again in tomorrow's meeting.

[Amanieu]

@rfcbot cancel

[rfcbot]

@Amanieu proposal cancelled.

[Amanieu]

In the @rust-lang/libs-api meeting today we discussed the specific methods that we are considering stabilizing. We decided to only stabilize the as_chunks methods which return slices and leave the methods returning an iterator as unstable.

Methods being stabilized are:

impl [T] {
    const fn as_chunks<const N: usize>(&self) -> (&[[T; N]], &[T]);
    const fn as_rchunks<const N: usize>(&self) -> (&[T], &[[T; N]]);
    const unsafe fn as_chunks_unchecked<const N: usize>(&self) -> &[[T; N]];
    const fn as_chunks_mut<const N: usize>(&mut self) -> (&mut [[T; N]], &mut [T]);
    const fn as_rchunks_mut<const N: usize>(&mut self) -> (&mut [T], &mut [[T; N]]);
    const unsafe fn as_chunks_unchecked_mut<const N: usize>(&mut self) -> &mut [[T; N]];
}

@rfcbot merge

[rfcbot]

Team member @Amanieu has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[BurntSushi]

The docs for these functions currently state:

This check will most probably get changed to a compile time error before this method gets stabilized.

That should probably get fixed before stabilization or as part of stabilization.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[joboet]

The docs for these functions currently state:

This check will most probably get changed to a compile time error before this method gets stabilized.

That should probably get fixed before stabilization or as part of stabilization.

Couldn't we also remove the panic for these methods and return the entire slice as remainder if N is zero?

[cuviper]

Couldn't we also remove the panic for these methods and return the entire slice as remainder if N is zero?

That's possible, but this statement in the docs would have to make an exception on the "strictly less" part:

Splits the slice into a slice of N-element arrays, starting at the beginning of the slice, and a remainder slice with length strictly less than N.

[scottmcm]

Couldn't we also remove the panic for these methods and return the entire slice as remainder if N is zero?

We could do lots of things, but the big question is whether the alternatives are more useful than they are surprising. Like #74985 (comment) discussed another option above.

In general, I think its good when we panic because something cannot meet the natural post-condition. It's always possible to expand a post-condition to make a panic unnecessary, but that has its own problems when the programmer ends up trying to figure out why something so weird is happening in an edge case rather than getting a really obvious failure from the «sorry, I cannot do that» panic.

Does someone calling .as_chunks() ever want the whole thing to be in the remainder? None of my uses for as_chunks that I can think of ever wanted that. And it breaks the pattern that held for everything else, where as you use a smaller and smaller chunk size the remainder gets shorter and shorter, or where as the input gets longer the remainder gets longer until you fill a chunk and the remainder goes back to zero. Said otherwise, I can't imagine anyone ever calling .as_chunks::<0>() to get that behaviour, and if it happens in generic code it feels more likely that the generic code will do something wrong with the result than be happy with that result -- for example, maybe they notice the input length is longer than the chunk size so of course they can look at the first chunk, but that's not true if it just all got put in the remainder. But we could fix that by returning not (&[], input) but (&[[]; usize::MAX], input)! So would that be better? In some ways I think it is, since now you always have foo.as_chunks::<N>().0.len() >= foo.as_chunks::<N+1>().0.len() again, which is a good property. But is it what people actually want? No, I still don't think it's better than a panic.

It's like how we could say that 0 / 0 is 0 in rust, and that would even be handy for some people sometimes, but we don't and it panics. Or how we could say that indexing has wrapping behaviour, which would be handy for some people sometimes, but we don't and it panics for too-large indices. Or we could have said that 0.ilog2() returns -1 as Self when overflow checking is off, which is in some sense a very natural result, but we didn't and it always panics. Or how originally .step_by(0) repeated the first item forever, but before it was stabilized that was changed, so now that just always panics.

[Kixunil]

I'd like to note that my request for explaining why take the less conservative approach and which real-world code needs to check N wasn't addressed yet. I don't find the responses so far satisfying because they do not include realistic code which needs the panic rather than compile error.

To put it differently, I don't have a problem with panics if some code actually needs them but I highly doubt there is any such production code. A single example would convince me that panic is better.

[joboet]

But we could fix that by returning not (&[], input) but (&[[]; usize::MAX], input)!

I didn't think about the length of the slice being the (mathematically undefined) dividend – I guess my mind got these methods mixed up with split_first_chunk... please ignore my previous comment.

[BurntSushi]

I'd like to note that #74985 (comment) for explaining why take the less conservative approach and which real-world code needs to check N wasn't addressed yet.

I personally didn't respond because to me, examples aren't what convinces me to prefer panics. Post-mono errors are what convinces me to prefer panics. (Because I think post-mono errors should be avoided at almost all costs.)

[scottmcm]

I guess my mind got these methods mixed up with split_first_chunk...

I'll just second this point, actually, because I think it's a good one. I used to write .as_chunks().0[0] in examples because there wasn't a .first_chunk().unwrap() so it was by far the easiest way to get an array. But thankfully we have better ways than that horrible snippit now 🙂

the length of the slice being the (mathematically undefined) dividend

A really interesting (but slightly off-topic) thing this makes me think about: Okasaki chapter 9 is about numerical representations, and the equivalence between a way of writing a number and a container of that length. The simplest example is that a linked list is a unary representation of its own length. The book does a neat job of extending that to other things too, like what the container equivalent of a binary number would be -- and how the push/concat implementation on the data structure ends up with the same complexity as the increment/addition operator on a binary number.

So from that perspective, &[T] is a number, and first_chunk is checked_sub on that number, as_chunks is /+% on that number, and as_flattened is *-by-a-constant on it.

[joboet]

A really interesting (but slightly off-topic) thing this makes me think about: Okasaki chapter 9 is about numerical representations, and the equivalence between a way of writing a number and a container of that length. The simplest example is that a linked list is a unary representation of its own length. The book does a neat job of extending that to other things too, like what the container equivalent of a binary number would be -- and how the push/concat implementation on the data structure ends up with the same complexity as the increment/addition operator on a binary number.

So from that perspective, &[T] is a number, and first_chunk is checked_sub on that number, as_chunks is /+% on that number, and as_flattened is *-by-a-constant on it.

This is getting very off-topic, but this equivalence is what I found so incredibly fascinating about lambda calculus. It's taught me that data is defined by how you interact with it – and these interactions are basically nothing but folds.