Блог

Для проекта мне нужно добавить зависимость ignite-spark, но добавление приведенной ниже строки и синхронизации приводит к тому, что модули с сообщением об ошибке были разрешены с конфликтующими суффиксами разных версий в ProjectRef.

libraryDependencies = "org.apache.ignite" % "ignite-spark_2.10" % "2.3.0"

Также пробовал

libraryDependencies = "org.apache.ignite" %% "ignite-spark" % "2.3.0"

Версия Scala: 2.11.12 Искра:2.3.0 Воспламенение: 2.10

сборка.sbt

 name := "ScalaProjectDemo2.11.12"

version := "0.1"

scalaVersion := "2.11.12"
libraryDependencies  = "org.apache.spark" %% "spark-sql" % "2.3.0" % "provided"
libraryDependencies  = "org.apache.ignite" % "ignite-spring" % "2.10.0"
libraryDependencies  = "org.apache.ignite" % "ignite-indexing" % "2.10.0"
libraryDependencies  = "org.apache.ignite" % "ignite-core" % "2.10.0"
libraryDependencies  = "org.apache.spark" %% "spark-core" % "2.3.0"
libraryDependencies  = "org.apache.ignite" % "ignite-spark_2.10" % "2.3.0"
 

 D:/Java/jdk1.8.0_301/bin/java.exe -Djline.terminal=jline.UnsupportedTerminal -Dsbt.log.noformat=true -Dfile.encoding=UTF-8 -Didea.managed=true -Dfile.encoding=UTF-8 -jar C:/Users/admin/AppData/Roaming/JetBrains/IdeaIC2020.1/plugins/Scala/launcher/sbt-launch.jar
[info] Loading global plugins from C:Usersadmin.sbt1.0plugins
[info] Loading project definition from C:UsersadminIdeaProjectsScalaProjectDemo2.11.12project
[info] Loading settings for project scalaprojectdemo2-11-12 from build.sbt ...
[info] Set current project to ScalaProjectDemo2.11.12 (in build file:/C:/Users/admin/IdeaProjects/ScalaProjectDemo2.11.12/)
[info] sbt server started at local:sbt-server-800963285a52579c1956
sbt:ScalaProjectDemo2.11.12>
[info] Defining Global / sbtStructureOptions, Global / sbtStructureOutputFile and 1 others.
[info] The new values will be used by cleanKeepGlobs
[info]  Run `last` for details.
[info] Reapplying settings...
[info] Set current project to ScalaProjectDemo2.11.12 (in build file:/C:/Users/admin/IdeaProjects/ScalaProjectDemo2.11.12/)
[info] Applying State transformations org.jetbrains.sbt.CreateTasks from C:/Users/admin/AppData/Roaming/JetBrains/IdeaIC2020.1/plugins/Scala/repo/org.jetbrains/sbt-structure-extractor/scala_2.12/sbt_1.0/2018.2.1 4-88400d3f/jars/sbt-structure-extractor.jar
[info] Reapplying settings...
[info] Set current project to ScalaProjectDemo2.11.12 (in build file:/C:/Users/admin/IdeaProjects/ScalaProjectDemo2.11.12/)
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
[error] Modules were resolved with conflicting cross-version suffixes in ProjectRef(uri("file:/C:/Users/admin/IdeaProjects/ScalaProjectDemo2.11.12/"), "scalaprojectdemo2-11-12"):
[error]    org.apache.spark:spark-launcher _2.10, _2.11
[error]    org.apache.spark:spark-sketch _2.10, _2.11
[error]    org.json4s:json4s-ast _2.10, _2.11
[error]    org.apache.spark:spark-catalyst _2.10, _2.11
[error]    org.apache.spark:spark-network-shuffle _2.10, _2.11
[error]    com.twitter:chill _2.10, _2.11
[error]    org.apache.spark:spark-sql _2.11, _2.10
[error]    org.json4s:json4s-jackson _2.11, _2.10
[error]    com.fasterxml.jackson.module:jackson-module-scala _2.10, _2.11
[error]    org.json4s:json4s-core _2.10, _2.11
[error]    org.apache.spark:spark-unsafe _2.11, _2.10
[error]    org.apache.spark:spark-tags _2.11, _2.10
[error]    org.apache.spark:spark-core _2.10, _2.11
[error]    org.apache.spark:spark-network-common _2.10, _2.11
[error] stack trace is suppressed; run 'last update' for the full output
[error] stack trace is suppressed; run 'last ssExtractDependencies' for the full output
[error] (update) Conflicting cross-version suffixes in: org.apache.spark:spark-launcher, org.apache.spark:spark-sketch, org.json4s:json4s-ast, org.apache.spark:spark-catalyst, org.apache.spark:spark-network-shuffle, com.twitter:chill, org.apache.spark:spark-sql, org.json4s:json4s-jackson, com.fasterxml.jackson.module:jackson-module-scala, org.json4s:json4s-core, org.apache.spark:spark-unsafe, org.apache.spark:spark-tags, org.apache.spark:spark-core, org.apache.spark:spark-network-common
[error] (ssExtractDependencies) Conflicting cross-version suffixes in: org.apache.spark:spark-launcher, org.apache.spark:spark-sketch, org.json4s:json4s-ast, org.apache.spark:spark-catalyst, org.apache.spark:spark-network-shuffle, com.twitter:chill, org.apache.spark:spark-sql, org.json4s:json4s-jackson, com.fasterxml.jackson.module:jackson-module-scala, org.json4s:json4s-core, org.apache.spark:spark-unsafe, org.apache.spark:spark-tags, org.apache.spark:spark-core, org.apache.spark:spark-network-common
[error] Total time: 9 s, completed Sep 18, 2021 6:11:55 AM
[info] shutting down sbt server
 

Пожалуйста, предоставьте свои предложения.

Заранее спасибо.

Смотрю на хранилище Maven.

Мы видим, что функция 2.3.0 ignite-spark поддерживает только Scala 2.10 (и, следовательно, также зависит от более старых версий Spark).

Возможно, вы захотите обновить по крайней мере 2.7.6 до той (только), которая поддерживает Scala 2.11 и основана на Spark 2.3 ; это та же версия, которую вы использовали.