Блог

Получение ошибки при запуске «spark-shell» или «pyspark» для оболочки питания anaconda. Локальная установка.

Установлен SPARK_HOME Установлен
HADOOP_HOME установлен
JAVA_HOME установлен

 (base) PS C:Users....> spark-shell
Exception in thread "main" java.lang.NoSuchMethodError: scala.Some.value()Ljava/lang/Object;
        at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:134)
        at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111)
        at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105)
        at org.apache.spark.deploy.SparkSubmit.initializeLogIfNecessary(SparkSubmit.scala:75)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:83)
        at org.apache.spark.deploy.SparkSubmit$anon$2.doSubmit(SparkSubmit.scala:1039)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
(base) PS C:Users....>
 

При выполнении pyspark

 (base) PS C:Usersthyago.carvalho> pyspark
Python 3.8.8 (default, Apr 13 2021, 15:08:03) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
Exception in thread "main" java.lang.NoSuchMethodError: scala.Some.value()Ljava/lang/Object;
        at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:134)
        at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111)
        at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105)
        at org.apache.spark.deploy.SparkSubmit.initializeLogIfNecessary(SparkSubmit.scala:75)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:83)
        at org.apache.spark.deploy.SparkSubmit$anon$2.doSubmit(SparkSubmit.scala:1039)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Traceback (most recent call last):
  File "C:sparkspark-3.1.2-bin-hadoop3.2pythonpysparkshell.py", line 35, in <module>
    SparkContext._ensure_initialized()  # type: ignore
  File "C:sparkspark-3.1.2-bin-hadoop3.2pythonpysparkcontext.py", line 331, in _ensure_initialized
    SparkContext._gateway = gateway or launch_gateway(conf)
  File "C:sparkspark-3.1.2-bin-hadoop3.2pythonpysparkjava_gateway.py", line 108, in launch_gateway
    raise Exception("Java gateway process exited before sending its port number")
Exception: Java gateway process exited before sending its port number
>>>