- Installing Scala and Spark on Windows
- Download
- The most popular way to get Scala is either using Scala through sbt, the Scala build tool, or to use Scala through an IDE.
- First, make sure you have the Java 8 JDK (or Java 11 JDK) installed.
- Then, install Scala:
- Other ways to install Scala
- Release Notes
- Other Releases
- Other resources
- License
- Scala — Настройка среды
- Шаг 1: Проверьте вашу установку Java
- Шаг 2: Настройте свою среду Java
- Шаг 3: Установите Scala
Installing Scala and Spark on Windows
Jose Marcial Portilla
Dec 16, 2016 · 3 min read
Hi! I’m Jose Portilla and I teach over 200,000 students about programming, data science, and machine learning on Udemy! You can check out all my courses here.
If you’re interested in learning Python for Data Science and Machine learning, check out my course here. (I also teach Full Stack Web Development with Django!)
Quick guide to installing a basic Scala and Spa r k set-up on Windows. Spark is written in Scala, which is written in Java, which means we need all three of these things to make sure everything works out! Here are the general steps (if you’re enrolled in my course: Scala and Spark for Big Data and Machine Learning you can always follow along with the video lecture).
Step 1: Download the latest Java Development Kit that matches your system (32-bit vs 64-bit). You can find the download website from Oracle here or just Googling “Java Development Kit”.
Step 2: Go to apache.spark.org and download a pre-built version of Spark (pre-built for Hadoop 2.7 and Later) and preferably Spark 2.0 or later.
Step 3: Download winutils.exe in order to make sure that Hadoop works correctly on your computer for Windows. You can find this file as a resource in the video lecture, but you may need to Google for another version if you are running an older version of Windows. (Just google “Spark+winutils” and you should see plenty of links with different sources for the download.
Step 4: Go to your downloaded jdk file and run the installation program, just use all the defaults.
Step 5: Extract the downloaded spark-2.0.2-bin-hadoop2.7-tar.gz file. You may need to extract this twice in order to get the full folder to show.
Step 6: Once you have this folder, go to your C drive and create a new folder called Spark and copy and paste the contents of the unzipped spark-2.0.2-bin-hadoop2.7-tar.gz file to this new Spark folder you just created.
Step 7: Create a new folder under your C drive called winutils. Then inside of this folder create a new folder called bin. Inside of this bin folder place your downloaded winutils.exe file.
Now its time to tell your Windows machine where to find everything, which means we need to edit our environment variables.
Step 8: Go to Control Panel > System and Security > System > Advanced System Settings
Step 9: In the window that pops up, click on the button Environment Variables
Step 10: You should see two panels, User Variables and System Variables, click New… on the User Variables to create a new variable name and value combination. You’ll create the following variables (your full paths may differ slightly):
Variable name: SPARK_HOME
Variable value: C:\Spark
Variable name: JAVA_HOME
Variable value: C:\Program Files\Java\jdk1.8.0_101
Variable name: HADOOP_HOME
Variable value: C:\winutils
Step 11: Then under your User Variables you should see a variable called PATH that is already there. Select it and click Edit
Step 12: You should see a bunch of environment variables already there, but now we’re going to add our own paths. Click on new and enter:
Then repeat it again and add:
Everything should now be installed. Let’s test it. Open a command prompt and use cd to change directory to C:\Spark and then type:
then hit Enter and you should eventually the Spark Shell display. You can type :q to exit out of this.
Hope this was helpful!
A quick note: It is common once you edit and do changes in the PATH that some changes are not recognized. If you follow the steps as shown here and get the ‘not recognized’ error when trying to run spark-shell, just restart Windows, that should fix the problem.
Download
The most popular way to get Scala is either using Scala through sbt, the Scala build tool, or to use Scala through an IDE.
First, make sure you have the Java 8 JDK (or Java 11 JDK) installed.
To check, open the terminal and type:
java -version (Make sure you have version 1.8 or 11.)
(If you don’t have it installed, download Java from Oracle Java 8, Oracle Java 11, or AdoptOpenJDK 8/11. Refer JDK Compatibility for Scala/Java compatiblity detail.
Then, install Scala:
. either by installing an IDE such as IntelliJ, or sbt, Scala’s build tool.
Best if you prefer a full-featured IDE (recommended for beginners)
Best if you are familiar with the command line
Compared to other programming languages, installing Scala is a bit unusual. Scala is unusual because it is usually installed for each of your Scala projects rather than being installed system-wide. Both of the above options manage (via sbt) a specific Scala version per Scala project you create.
But it’s also possible to «install» Scala in numerous other ways; e.g., grab Scala binaries and use Scala from the command line or use Scala in your browser!
Other ways to install Scala
- Download the Scala binaries for
Need help running the binaries? - Using SDKMAN!, you can easily install Scala with sdk install scala
- On macOS you can also use Homebrew and existing Scala Formulae
brew update
brew install scala - With MacPorts, you can get Scala using sudo port install scala2.x command.
For example to install Scala 2.12 simply use sudo port install scala2.12 - Use Scastie to run single-file Scala programs in your browser using multiple Scala compilers; the production Scala 2.x compilers, Scala.js, Dotty, and Typelevel Scala. Save and share executable Scala code snippets.
- Try Scala in the browser via ScalaFiddle. This lets you run single-file Scala programs in your browser using Scala.js, including graphical/interactive examples such as Oscilloscope or Ray Tracer
- Get Ammonite, a popular Scala REPL
Or are you looking for previous releases of Scala?
Release Notes
Other Releases
You can find the links to prior versions or the latest development version below. To see a detailed list of changes for each version of Scala please refer to the changelog.
Note that different major releases of Scala (e.g. Scala 2.11.x and Scala 2.12.x) are not binary compatible with each other.
- Latest 2.12.x maintenance release:
- Scala 2.12.13 — Released on January 12, 2021
- Last 2.11.x maintenance release:
- Scala 2.11.12 — Released on November 9, 2017
- Last 2.10.x maintenance release:
- Scala 2.10.7 — Released on November 9, 2017
- Nightly builds
- Changelog
- All previous releases
Other resources
You can find the installer download links for other operating systems, as well as documentation and source code archives for Scala 2.13.5 below.
| Archive | System | Size |
|---|---|---|
| scala-2.13.5.tgz | Mac OS X, Unix, Cygwin | 21.98M |
| scala-2.13.5.msi | Windows (msi installer) | 130.48M |
| scala-2.13.5.zip | Windows | 22.02M |
| scala-2.13.5.deb | Debian | 641.63M |
| scala-2.13.5.rpm | RPM package | 130.73M |
| scala-docs-2.13.5.txz | API docs | 57.73M |
| scala-docs-2.13.5.zip | API docs | 111.89M |
| scala-sources-2.13.5.tar.gz | Sources | 8.0M |
License
The Scala distribution is released under the Apache License, Version 2.0.
java -version (Make sure you have version 1.8.)
java -version (Make sure you have version 1.8.)
java -version (Make sure you have version 1.8.)
- Launch the Windows Start menu
- Click on Programs
- Find the Java program listing
- Click About Java to see the Java version
Scala — Настройка среды
Scala может быть установлен на любой UNIX или Windows-системе. Перед тем, как начать установку Scala на свой компьютер, на вашем компьютере должна быть установлена Java 1.8 или выше.
Следуйте инструкциям ниже, чтобы установить Scala.
Шаг 1: Проверьте вашу установку Java
Прежде всего, вам необходимо установить Java Software Development Kit (SDK) в вашей системе. Чтобы убедиться в этом, выполните любую из следующих двух команд в зависимости от платформы, на которой вы работаете.
Если установка Java была выполнена правильно, то на ней отобразится текущая версия и спецификация вашей установки Java. Пример вывода приведен в следующей таблице.
| Платформа | команда | Пример вывода | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Windows |
| Sr.No | Платформа и описание | ||||||
|---|---|---|---|---|---|---|---|
| 1 |
| Sr.No | Платформа и описание | |||
|---|---|---|---|---|
| 1 |
| Платформа | Команда и выход | Описание |
|---|---|---|
| Windows |
