[Variant] Integration tests for reading parquet w/ Variants

[alamb]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

• part of [EPIC] [Parquet] Implement Variant type support in Parquet #6736

We are implementing Variant, and part of doing so is ensuring that the Rust implementation is compatible with other implementations

Describe the solution you'd like
I would like a harness that

1. For each test listed in cases.json
2. Reads the expected variant.bin file and creates a Variant from it
3. Reads the data from the Parquet file (should be a StructArray)
4. Construct a VariantArray
5. Read the value from the VariantArray (using both VariantArray::value and the variant_get kernel)
6. Verify that the value from the VariantArray matches the Variant read from the .bin file

Many of the cases will not to pass initially -- that is fine, we should just mark them as #[ignore] or something and we will enable them as we work through the Variant implementation

Describe alternatives you've considered

I recommend using this PR from @rdblue

• Add shredded Variant reader test cases parquet-testing#90

• Here is an example of such an integration test for the go implementation ci(parquet/pqarrow): integration tests for reading shredded variants DO NOT MERGE YET arrow-go#455 written by @zeroshade (parquet/pqarrow/variant_test.go specifically)

Additional context

[alamb]

Update here @carpecodeum has added the initial basic tests in

• [VARIANT] Initial integration tests for variant reads #8104

There are a few more follow on items:

1. Test reading the values as VariantArray (this requires some finagling with feature flags / dependencies that I am working on)
2. Add #test style tests for all the cases (@carpecodeum / @mprammer is this something you able to help with?)
3. Potential cleanups from [VARIANT] Initial integration tests for variant reads #8104

[carpecodeum]

Update here @carpecodeum has added the initial basic tests in

• [VARIANT] Initial integration tests for variant reads #8104

There are a few more follow on items:

1. Test reading the values as VariantArray (this requires some finagling with feature flags / dependencies that I am working on)
2. Add #test style tests for all the cases (@carpecodeum / @mprammer is this something you able to help with?)
3. Potential cleanups from [VARIANT] Initial integration tests for variant reads #8104

Yep we can help with that, sorry missed this comment

[alamb]

Thanks @carpecodeum -- I filed this ticket to track adding a feature flag / making it easier to use Variant with

• [Variant] Add variant feature to parquet crate #8132

I think it will be tricky (b/c I don't really know what to suggest / don't have a clear goal yet)

Perhaps you can work on 2 and 3 while I try and sort out #8132 ?

[alamb]

Here is a PR to make the variant code available in the parquet crate

• Add variant_experimental feature to parquet crate #8133