Skip to content

Conversation

@AbinayaJayaprakasam
Copy link
Contributor

@AbinayaJayaprakasam AbinayaJayaprakasam commented Nov 27, 2025

What changes were proposed in this pull request?

I renamed _LEGACY_ERROR_TEMP_1007 to TEMP_VIEW_WRITE_NOT_ALLOWED and added sqlState 42809.

The changes include:

  • New error class TEMP_VIEW_WRITE_NOT_ALLOWED in error-conditions.json with sqlState 42809
  • Updated QueryCompilationErrors.scala to reference the new error class
  • Refactored 3 test cases in DataFrameWriterV2Suite to use checkError() instead of string matching

Why are the changes needed?

Users frequently encounter this error when attempting to write to temporary views using the DataFrameWriter V2 API. The legacy error name is non-descriptive and lacks an sqlState, which makes it difficult to handle programatically and reduces compatibility with JDBC/ODBC clients.

Does this PR introduce any user-facing change?

No. The error message text stays the same. User will only see the same error message, but the underlying error class is now properly named and includes a sqlState for better tooling support.

How was this patch tested?

I tested this change in multiple ways:

  • Manually triggered the error in spark-shell to verify the new error class is used
  • Updated 3 existing tests in DataFrameWriterV2Suite to use checkError() for structured validation of error class and parameters
  • Ran the full test suite: all tests pass successfully

Manual Testing in Spark-Shell

Before (legacy error):

scala> val sourceData = Seq((1, "a"), (2, "b"), (3, "c")).toDF("id", "name")
scala> sourceData.createOrReplaceTempView("source")
scala> spark.range(10).createOrReplaceTempView("temp_view")
scala> spark.table("source").writeTo("temp_view").append()

org.apache.spark.sql.AnalysisException: Cannot write into temp view temp_view as it's not a data source v2 relation.
at org.apache.spark.sql.errors.QueryCompilationErrors$.writeIntoTempViewNotAllowedError(QueryCompilationErrors.scala:551)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$12.$anonfun$applyOrElse$73(Analyzer.scala:1237)

After (new error with class and sqlState):

scala> val sourceData = Seq((1, "a"), (2, "b"), (3, "c")).toDF("id", "name")
scala> sourceData.createOrReplaceTempView("source")
scala> spark.range(10).createOrReplaceTempView("temp_view")
scala> spark.table("source").writeTo("temp_view").append()

org.apache.spark.sql.AnalysisException: [TEMP_VIEW_WRITE_NOT_ALLOWED] Cannot write into temp view temp_view as it's not a data source v2 relation. SQLSTATE: 42809
at org.apache.spark.sql.errors.QueryCompilationErrors$.writeIntoTempViewNotAllowedError(QueryCompilationErrors.scala:551)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$12.$anonfun$applyOrElse$73(Analyzer.scala:1237)

Note: The error now displays the error class [TEMP_VIEW_WRITE_NOT_ALLOWED] and includes sqlState 42623 (accessible via getSqlState()).

Ran the full DataFrameWriterV2Suite test suite: all tests pass successfully

[info] Test run finished: 0 failed, 0 ignored, 6 total, 0.786s
[info] ScalaTest
[info] Run completed in 9 seconds, 738 milliseconds.
[info] Total number of tests run: 48
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 48, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 54, Failed 0, Errors 0, Passed 54
[success] Total time: 43 s, completed 27-Nov-2025, 11:21:46 pm

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the SQL label Nov 27, 2025
"message" : [
"Cannot write into temp view <quoted> as it's not a data source v2 relation."
],
"sqlState" : "42623"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain the choice of sqlState 42623? The definition is "DEFAULT clause cannot be specified", which doesn't seem to match this error's purpose. Have you considered 42809 instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you .. You're absolutely right.

I initially chose 42623 because I saw it used for other write constraint errors in Spark, but after reviewing the SQL state definitions, 42809 ("The identified object is not the type of object to which the statement applies") is indeed more semantically accurate for this error.

The error occurs because the temp view is not a DataSource V2 relation, which is exactly an "object type mismatch" scenario that 42809 is designed for.

I've updated the PR to use 42809. Thank you for the careful review!

References:

### What changes were proposed in this pull request?

I renamed _LEGACY_ERROR_TEMP_1007 to TEMP_VIEW_WRITE_NOT_ALLOWED and added sqlState 42809.

The changes include:
- New error class TEMP_VIEW_WRITE_NOT_ALLOWED in error-conditions.json with sqlState 42809
- Updated QueryCompilationErrors.scala to reference the new error class
- Refactored 3 test cases in DataFrameWriterV2Suite to use checkError() instead of string matching
- Removed the old _LEGACY_ERROR_TEMP_1007 entry to complete the migration

### Why are the changes needed?

Users frequently encounter this error when attempting to write to temporary views using the DataFrameWriter V2 API. The legacy error name is non-descriptive and lacks a sqlState, which makes it difficult to handle programmatically and reduces compatibility with JDBC/ODBC clients.

### Does this PR introduce any user-facing change?

No. The error message text stays the same. Users will see the same error message, but the underlying error class is now properly named and includes a sqlState for better tooling support.

### How was this patch tested?

I tested this change in multiple ways:
- Manually triggered the error in spark-shell to verify the new error class is used
- Updated 3 existing tests in DataFrameWriterV2Suite to use checkError() for structured validation of error class and parameters
- Ran the full test suite: all 48 tests pass successfully
- Verified sqlState 42809 is correctly set based on reviewer feedback

### Was this patch authored or co-authored using generative AI tooling?

no
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants