Open
Description
Describe the bug
apache/arrow-rs#7159 only avoids overflow of remainder(%), the divide operation still has similar issues as #1412.
In spark, the / operation will be converted to double/decimal division, so only integral divide operation of Long.MinValue div -1 has inconsistent behavior.
Steps to reproduce
test case:
test("COMET-1412: test smallest signed integer value div -1") {
Seq[(String, Any)](
// ("short", Short.MinValue),
// ("int", Int.MinValue),
("long", Long.MinValue)
// ("double", Double.MinValue)
).foreach { case (t, v) =>
withTable("t1") {
sql(s"create table t1(c1 $t, c2 short) using parquet")
sql(s"insert into t1 values($v, -1)")
withSQLConf(CometConf.COMET_ENABLED.key -> "false "){
sql("select c1 div c2, c1 div -1 from t1 order by c1").show()
}
withSQLConf(CometConf.COMET_ENABLED.key -> "true "){
sql("select c1 div c2, c1 div -1 from t1 order by c1").show()
}
}
}
}
result of vanilla spark:
+--------------------+--------------------+
| (c1 div c2)| (c1 div -1)|
+--------------------+--------------------+
|-9223372036854775808|-9223372036854775808|
+--------------------+--------------------+
error on comet:
org.apache.comet.CometNativeException: Arithmetic overflow: Overflow happened on: -9223372036854775808 / -1
at org.apache.comet.Native.executePlan(Native Method)
Expected behavior
No response
Additional context
No response