-
Notifications
You must be signed in to change notification settings - Fork 232
Allowing omitting the "T" separator from a datetime string #5337
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
9a25307 to
bc595ea
Compare
roborivers
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Coding style check: Error. ⚠.
Smoke testing: Success ✓.
Cbuild submission: Error ⚠.
Regression testing: 3/625 tests failed ⚠.
The first 10 failing tests are:
udf
consumer_non_atomic_default_consumer_generated
consumer
|
If we allow strings without T to be converted to datetimes, we have the following conundrum: is "20250101" a datetime or an epoch value? |
de0a305 to
c8c985d
Compare
If the string contains only digits, it's treated as a unix timestamp (code). So "20250101" will still be an epoch value! I've also added a test for it! |
roborivers
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Coding style check: Error. ⚠.
Smoke testing: Success ✓.
Cbuild submission: Success ✓.
Regression testing: 5/625 tests failed ⚠.
The first 10 failing tests are:
sc_transactional_rowlocks_generated
udf
consumer_non_atomic_default_consumer_generated
consumer
sc_downgrade
|
This breaks UDF and consumer tests? |
Our customers have issue using spark to create a dataframe on a datetime column, because spark interpolates the provided datetime string, but drops the "T" separator when filling the range. It seems that omitting the "T" character is fairly acceptable, and ISO 8601 does allow it to be omitted in a few cases. This is a very simple patch to allow it. Signed-off-by: Rivers Zhang <[email protected]>
c8c985d to
cdb7bd0
Compare
There's one case that I overlooked, eg |
roborivers
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Coding style check: Error. ⚠.
Smoke testing: Success ✓.
Cbuild submission: Success ✓.
Regression testing: 2/625 tests failed ⚠.
The first 10 failing tests are:
cdb2_close
sc_downgrade
Our customers have issue using spark to create a dataframe on a datetime column, because spark interpolates the provided datetime string, but drops the "T" separator when filling the range.
It seems that omitting the "T" character is fairly acceptable, and ISO 8601 does allow it to be omitted in a few cases. This is a very simple patch to allow it.