Как исправить ошибку компиляции spark?

Question

[Korifa]

Подскажите, пож-так, как это исправить?

Comments

[Dmitry Roo]

Во-первых, почему скриншот, а не текст?
Во-вторых, почему не полныйл лог?

[Dmitry Roo]

3.5.4 у меня собралась без тестов:

[INFO] Reactor Summary for Spark Project Parent POM 3.5.4:
[INFO] 
[INFO] Spark Project Parent POM ........................... SUCCESS [  2.715 s]
[INFO] Spark Project Tags ................................. SUCCESS [  5.559 s]
[INFO] Spark Project Sketch ............................... SUCCESS [  8.635 s]
[INFO] Spark Project Local DB ............................. SUCCESS [  9.708 s]
[INFO] Spark Project Common Utils ......................... SUCCESS [ 10.996 s]
[INFO] Spark Project Networking ........................... SUCCESS [  9.804 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [  9.095 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [  8.645 s]
[INFO] Spark Project Launcher ............................. SUCCESS [  9.636 s]
[INFO] Spark Project Core ................................. SUCCESS [01:40 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [ 34.917 s]
[INFO] Spark Project GraphX ............................... SUCCESS [ 24.251 s]
[INFO] Spark Project Streaming ............................ SUCCESS [ 48.439 s]
[INFO] Spark Project SQL API .............................. SUCCESS [ 29.325 s]
[INFO] Spark Project Catalyst ............................. SUCCESS [02:23 min]
[INFO] Spark Project SQL .................................. SUCCESS [02:39 min]
[INFO] Spark Project ML Library ........................... SUCCESS [01:51 min]
[INFO] Spark Project Tools ................................ SUCCESS [  5.691 s]
[INFO] Spark Project Hive ................................. SUCCESS [01:13 min]
[INFO] Spark Project REPL ................................. SUCCESS [ 13.569 s]
[INFO] Spark Project Assembly ............................. SUCCESS [  7.522 s]
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [ 25.131 s]
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [ 26.261 s]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS [ 36.872 s]
[INFO] Spark Project Examples ............................. SUCCESS [ 41.898 s]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [  3.779 s]
[INFO] Spark Avro ......................................... SUCCESS [ 26.169 s]
[INFO] Spark Project Connect Common ....................... SUCCESS [ 45.059 s]
[INFO] Spark Protobuf ..................................... SUCCESS [ 27.211 s]
[INFO] Spark Project Connect Server ....................... SUCCESS [ 31.598 s]
[INFO] Spark Project Connect Client ....................... SUCCESS [01:06 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  09:58 min (Wall Clock)
[INFO] Finished at: 2025-01-06T00:43:27+03:00
[INFO] ------------------------------------------------------------------------
➜  spark-3.5.4 java --version                       
java 21.0.5 2024-10-15 LTS
Java(TM) SE Runtime Environment (build 21.0.5+9-LTS-239)
Java HotSpot(TM) 64-Bit Server VM (build 21.0.5+9-LTS-239, mixed mode, sharing)

[Korifa]

Dmitry Roo, Подскажи, пож-та, какую можно указать версию hive при сборке?
Я правильно понимаю, что для hive 4.0.1 еще нет актуальной версии spark?

[Dmitry Roo]

Korifa, я на счет хайва не в курсе

[Korifa]

Dmitry Roo, я вот такой командой собирал
./dev/make-distribution.sh --pip --tgz -Phive -Dhive.version=4.0.1 -Dhadoop.version=3.3.6 -Phive-thriftserver -Pyarn

[Korifa]

Dmitry Roo, Насколько я понимаю падает из-за того что пока не поддерживается версия hive 4.0.1

[Dmitry Roo]

Korifa, попробуйте версию апнуть