-
Notifications
You must be signed in to change notification settings - Fork 38
Open
Description
tdengine版本:3.3.6.9
spark版本:3.3.2
使用taos-jdbcdriver版本:3.6.2
通过spark读取数据即df.show(5), 没有数据,时间戳类型的数据是返回null, string类型的直接返回的字段名,double类型的会报转换错误
不管是用RestfulDriver或者WebSocketDriver驱动都一样
代码如下:
`
SparkSession spark = SparkSession.builder()
.appName("appSparkTest")
.master("local[*]")
.getOrCreate();
// connect
String url = "jdbc:TAOS-RS://192.168.1.1:6041/test?user=root&password=taosdata";
String driver = "com.taosdata.jdbc.rs.RestfulDriver";
// String url = "jdbc:TAOS-WS://192.168.1.1:6041/test?user=root&password=taosdata";
// String driver = "com.taosdata.jdbc.ws.WebSocketDriver";
int timeout = 60; // seconds
try {
// query sql
DataFrameReader reader = spark.read()
.format("jdbc")
.option("url", url)
.option("driver", driver)
.option("queryTimeout", timeout);
// map table
String dbtable = "(select ts, mete_code from xxx) as t;";
Dataset<Row> df = reader.option("dbtable", dbtable).load();
String log = String.format("------------ show dbtable read:%s -----------", dbtable);
System.out.println(log);
// show schema
df.printSchema();
// show data
df.show(5);
`
结果如下:
`
root
|-- ts: timestamp (nullable = true)
|-- mete_code: string (nullable = true)
+----+---------+
| ts|mete_code|
+----+---------+
|null|mete_code|
|null|mete_code|
|null|mete_code|
|null|mete_code|
|null|mete_code|
+----+---------+
only showing top 5 rows
`
但是把taos-jdbcdriver版本换成3.0.0 使用RestfulDriver驱动是能正常获取数据的
`
root
|-- ts: timestamp (nullable = true)
|-- mete_code: string (nullable = true)
+-------------------+--------------------+
| ts| mete_code|
+-------------------+--------------------+
|2024-06-27 10:15:00|ACTIVE_THRUST_BRG...|
|2024-06-27 10:16:00|ACTIVE_THRUST_BRG...|
|2024-06-27 10:18:00|ACTIVE_THRUST_BRG...|
|2024-06-27 10:20:00|ACTIVE_THRUST_BRG...|
|2024-06-27 10:22:00|ACTIVE_THRUST_BRG...|
+-------------------+--------------------+
only showing top 5 rows
`
Metadata
Metadata
Assignees
Labels
No labels