Rust: RecordBatches to Dataframe after Collect

[Cambyst]

Hi guys,

Apologies for the very basic question, but I have had a look at the documentation and also online for quite some time but couldn't find anything: Once you have called Collect and got your result back as Vec, how do you cast this back into a Dataframe for (potentially) more processing later in your program ? I can see from_batch but that's only for a single batch. Many thanks in advance !

Best Regards

[jonahgao]

Is it possible to handle it by cloning the DataFrame:

async fn test() {
    let df = test_table()
            .await?
            .select_columns(&["c2", "c3"])?
            .cache() // optionally cache the results as a memory table
            .await?;

        // clone and collect the results
        let results = df.clone().collect().await?;
        pretty::print_batches(&results)?;

        // processing later
        let df = df.filter(col("c2").eq(lit(3)))?;
        df.show().await?;
}

[Cambyst]

Hi @jonahgao , that did the trick, thank you very much !