Developing a dockerized web app on Windows Subsystem for Linux (WSL)

It’s been a long time since I last used Windows for development. I mostly work with open source languages and tools, so when I started my professional career, around 10 years ago, I switched to Linux, after briefly using Windows for web development. Five years later, I got myself a Macbook Air and used that, along with my desktop PC which was still running Linux. Last summer, Microsoft released the Anniversary Update for Windows 10 and I was surprised that it also included the Ubuntu userspace (also known as Windows Subsystem for Linux). It’s also quite easy to install.

Last month, I decided to upgrade to a new laptop, and I switched back to PC. I was going to install Linux on it, anyway, but since it came with Windows 10 as well, I thought I would give WSL a try. I didn’t expect much, but I was pleasantly surprised. I could run Git, Vim and Ruby on Windows just as well as on Linux. That said, the Windows command line is terrible compared to any Linux terminal. I quickly switched to wsltty. It still doesn’t provide all the shortcuts and features of, say gnome-terminal, but it does come close. But I digress, this post isn’t about the Windows console, but WSL and Docker.

For the past few months, I have used Docker for all the projects I’m working on. It’s very easy to set up a development environment, regardless of the host operating system. Since my new laptop came with Windows 10 Pro, I could install Docker for Windows. If you’re fine with cmd.exe or Powershell, that’s all you need. However, I really wanted to use WSL, since I find it much easier to use zsh or bash. Git and Vim are available as Windows binaries, but I think it’s simpler to just install them via apt-get and use them from WSL. Not to mention, you don’t have to mess with (CRLF/LF) line endings this way.

While the WSL is pretty close to production-ready, it’s still in beta. Running simple command line programs just works, but setting up your whole development workflow is full of little incompatibilities and annoyances. You can probably search for each one of them and find a solution, but setting up everything took me a few days, so I thought it was a good idea to document the whole process.

The guide

First, you need to install WSL which is pretty easy, and adds a bash shortcut into your start menu. When you open it, you are greeted with the default bash prompt. Next, install Git and Vim, with apt-get:

Installing Docker, takes a bit more work. There’s a guide for Ubuntu 14.04 on Docker’s website, which is the Ubuntu version that comes with WSL. I’ve summarized the relevant steps below:

Now, Docker engine runs natively on Windows, so you only need the docker client. You don’t have to install any kernel packages and you can simply skip the service docker start command.

By this point, one would think that docker is ready to run, right? Well, it’s not that simple. Try running docker and this is what you get:

Problem #1

With the Docker daemon running on the host OS (Windows), the Linux client can’t connect to the default Unix socket (it does not exist). The solution is pretty easy though, as documented in this useful reddit post. Just add a variable to your .bashrc :

Docker running fine! Now, just a git clone and we’re up and running! Not so fast…

Problem #2

The project I’m using for this example, mounts the code directory in the container. It also uses Docker compose to set up other services. Follow this guide to install compose before continuing.

This is the service definition in the docker-compose.yml file:

Pretty standard for a Rails app. Running docker-compose build builds the images as defined in the docker-compose.yml file. What happens if you want to launch the rails console? Let’s see:

That’s unexpected! Digging further, we can see that the /opt/rails_app is completely empty, which means the volume is not mounted at all. Here’s the proof:

What’s wrong? Well, 2 things:

• In order to mount a host directory into a container, it must be shared via Windows file sharing. You can do that from Docker’s settings (see below).
• Docker for Windows expects Windows paths. If you simply git clone your project inside WSL, your working directory will be something like /home/my-user/my-project . It should be something like C:\Users\my-user\my-project .

The first issue is easy to resolve. Right-click the Docker icon and select “Settings”. Then, simply enable sharing for drive C and follow the prompts. Here’s a screenshot:

The other issue is harder. WSL files are stored somewhere in your home directory (the path is C:\Users\win-user\AppData\Local\lxss ). You could technically find a way to translate the WSL path into a Windows path. Then write a wrapper script that calls docker with the Windows path instead. If you use docker-compose , however, you need to edit your docker-compose.yml file. Well, that kind of defeats the purpose, doesn’t it? We want to use Docker to set up our development workstation regardless of its operating system.

After googling around, I found a satisfactory solution. If you look carefully at the screenshot above, you can see that you can use a forward slash for Windows paths with Docker. However, this doesn’t solve our problem, since Linux has no concept of drive letters, and paths start with a forward slash. In fact, WSL auto-mounts Windows drives under /mnt . For example, your C:\Users\win-user\Code\my-project path is accessible on WSL as /mnt/c/Users/win-user/Code/my-project . We’re getting closer! Well, it appears Docker can translate a path such as /c/Code/my-project to C:\Code\my-project . That’s pretty good! The only thing remaining is to have our C: drive appear on WSL as /c . How do we do this? Simply, by symlinking /mnt/c as /c :

Note: If you are running the latest April 2018 update there is a better way to do this. Scroll to the bottom of the article to read more.

And we’re done! The rest is the usual stuff:

Well, there’s one more thing. Using the dot to denote the current working directory won’t work… You have to use $PWD , so edit the relevant docker-compose.yml line, and you’re truly done:

Update: Compatibility with Windows (PowerShell and cmd.exe)

While the solution above works on Linux, MacOS and WSL, it is not compatible with the native Windows terminals (PowerShell and cmd.exe). The reason is simple. There is no PWD environment variable. Both shells support environment variables, but neither sets PWD to the current working directory. If you need to keep your docker-compose.yml compatible with these shells, there’s an easy fix. You just have to provide a fallback value when the PWD variable is not set:

This was added to Docker Compose v1.9 and needs version 2.1 of the configuration file format. That’s why I have also included the first line in the code snippet above. Also, keep in mind this is supported on Docker engine 1.12 and above.

Is it worth it?

Well, for me, yes. If you have to use Windows, you can keep your Linux, terminal-heavy workflow by using WSL. If you don’t use Docker, it’s much easier. But using Docker on WSL is feasible. There is only one minor issue with the solution above. Git is slower when the repo is located in a Windows-accessible path ( /mnt/x/.. ). My prompt calls git to retrieve the current branch and other info and this results in a half-second delay before the prompt appears. That’s not a deal-breaker, however, and I can be just as productive on Windows, as I am on Linux.

That’s it! Hope you liked my guide. Do you use WSL and Docker? I’d love to hear your experience.

November 2017 — Fall Creators Update

It’s been many months since I’ve written this tutorial. During that time, Windows 10 has been updated twice, and with it, WSL has been updated and is now out of beta! What has changed? Not much. Here’s a list of the most relevant changes:

• File I/O is noticeably faster now so git operations are quick enough for large repos.
• The bundled Ubuntu version is now 16.04 and it’s distributed through the Windows Store. This means that the instructions to install Docker are a bit different.

You don’t have to do anything with your existing installation, but you’re encouraged to re-install WSL through the Store. The old WSL installation is deprecated and updates are only available through the Store.

April 2018 update —Configuration with /etc/wsl.conf

The latest update came out a bit later than expected, but has an exciting new feature. You can now configure WSL, using the new /etc/wsl.conf file. Among the things you can configure, is the directory where all fixed drives are mounted. This renders the previous symlink solution obsolete, so remove any symlinks, if you still have them. You have to create the file and add this content:

You need to restart Windows (or log out/log in again — haven’t tried) for this to take effect. All your fixed drives will appear directly in the root directory now. The options line enables Linux metadata under WSL, which means you can change the owner of files and directories (used to default to root before) and their permissions (the read-write-execute bits, which used to default to 0777). The WSL team has a more detailed post about this.

Another exciting addition is support for Unix sockets. This enables another approach to connecting the Linux Docker client to the Docker for Windows server, which is explained in detail in this post.

An In Depth Tutorial on Linux Development on Windows with WSL and Visual Studio Code

In an earlier blog post, Take your Linux development experience in Windows to the next level with the Windows Subsystem for Linux (WSL) and Visual Studio Code Remote, we introduced an overview of the VS Code Remote – WSL extension, which simplifies Linux development on Windows Subsystem on Linux (WSL). Put on your SCUBA gear, because in this follow up we’ll give you a deep dive tutorial on how to setup WSL and VS Code for Python development by creating a Python “Hello World” application.

Windows: A great platform for building Linux Apps

Windows is the most popular operating system in the world, with almost 50% of developers using it every day. At the same time, many of these developers are building applications that are deployed to Linux-based servers running in the cloud or on-premises.

With WSL and VS Code, you can now seamlessly develop Linux-based applications on Windows. WSL lets you run a full Linux distro on Windows, where you can install platform-specific toolchains, utilities, and runtimes.

VS Code and the WSL extension let you develop in the context of the Linux environment, using those tools and runtimes, from the comfort of Windows. All of your VS Code settings are maintained across Windows and Linux, making it easy to switch back and forth. Commands and workspace extensions are run directly in Linux, so you don’t have to worry about pathing issues, binary compatibility, or other cross-OS challenges. You’re able to use VS Code in WSL just as you would from Windows. One tool, two operating systems.

If it sounds magical, that’s because it is! But, don’t take our word for it. Let’s get our hands dirty and build a simple Python3 application so you can experience the magic for yourself.

Install and set up WSL

You install WSL from the Microsoft Store. You can search for “Linux” in the Microsoft store to see a sub section of distributions in the store. Choose the Linux distribution you want to install and follow the prompts. You can also search for distributions in the search bar.

Select Install.

And when done, select Launch to get started. This will open a Linux terminal and complete the installation. You’ll need to create a user ID and password since we’re setting up a full Linux instance, but once that’s done, boom! You are running Linux on Windows.

Python development

If you don’t have Python already installed, run the following commands to install Python3 and pip, the package manager for Python, into your Linux installation.

And to verify, run:

This isn’t intended to be a Python tutorial, so we’ll do the canonical “Hello World” app. Create a new folder called “helloWorld” and then add a Python file that will print a message when run:

Clearly, echo isn’t a great way to do development. In a remote Linux environment (this WSL distro is technically another machine without UI, that just happens to be running locally on your computer), your development tools and experiences are pretty limited. You can run Vim in the terminal to edit your file or you can edit the sources on the Windows side through the \\wsl$ mount:

The problem with this model is that the Python runtime, pip, or any conda packages for that matter, are not installed on Windows.

Remember, Python is installed in the Linux distro, which means if we’re editing Python files on the Windows side, we can’t run or debug them unless we install the same Python development stack on Windows. And that defeats the purpose of having an isolated Linux instance set up with all our Python tools and runtimes!

What can we do? That’s where Visual Studio Code and the Remote – WSL extension comes to the rescue.

Visual Studio Code

VS Code is a lightweight, cross platform source code editor, built on open source. It comes with built-in support for modern web development with JavaScript, TypeScript, Node.js, CSS, etc. It also has a rich ecosystem of extensions (10K+) providing support for 100s of languages and frameworks, such as Python, Go, PHP, Java, C++, and C#.

If you don’t already have VS Code, download it now. It’s about 50 MB to download on Windows and sets up in less than a minute.

Now we just need the magic, and that is the Remote – WSL extension. Open the Extensions view in VS Code (Ctrl+Shift+X) and search for “wsl”. Choose the Remote – WSL extension as seen below (it should be at the top of the list) and press Install.

Once installed, head back over the WSL terminal, make sure you are in the helloWorld folder, and type in “code .” to launch VS Code (the “.” tells VS Code to open the current folder).

The first thing you’ll see is a message about “Installing VS Code Server” (the c7d83e57… number is the version of the VS Code Server that matches the client-side tools you just installed). VS Code is installing a small server on the Linux side that the desktop VS Code will then talk to.

That server will then install and host extensions in WSL, so that they run in the context of the tools and frameworks installed in WSL. In other words, your Python extension will run against the Python installed in WSL, not against what is installed on the Windows side, as it should for the proper development experience.

The next thing that happens is VS Code will start and open the helloWorld folder. You may see a quick notification telling you that VS Code is connecting to WSL, and you may be prompted to allow access to the Node.js-based server.

Now, when we hover over hello.py, we get the proper Linux path.

Integrated Terminal

If that doesn’t convince you we’re connected to the Linux subsystem, run Terminal > New Terminal (Ctrl+`) to open a new terminal instance.

You’ll start a new instance of the bash shell in WSL, again from VS Code running on Windows.

Tip: In the lower left corner of the Status Bar, you can see that we’re connected to our WSL: Ubuntu instance.

Click on it to bring up a set of Remote – WSL extension commands.

Installing the Python extension (and additional tools)

Click on hello.py to open it for editing. We are prompted with what we call an “Important” extension recommendation, in this case to install the Python extension, which will give us rich editing and debugging experiences. Go ahead and select Install and reload if prompted.

To prove that the extension is installed in WSL, open the Extensions view again (Ctrl+Shift+X). You will see a section titled WSL – Installed and you can see any extensions that are installed on the WSL side.

Upon reload, you’ll also get prompted telling you that the pylint linter is not installed. Linters are used to give us errors and warnings in our source code. Go ahead and select Install.

Now, when you edit your code, you get rich colorization and completions.

And when you save your file (Ctrl+S), you’ll get linting errors and warnings on the file.

Debugging

With our tools set up, let’s take this one step further. Set a breakpoint on line 1 of hello.py by clicking in the gutter to the left of the line number or by putting the cursor on the line and pressing F9.

Now, press F5 to run your application. You will be asked how to run the application, and since this is a simple file, just choose Python File.

The app will start, and you’ll hit the breakpoint. You can inspect variables, create watches, and navigate the call stack.

Press F10 to step and you’ll see the output of the print statement in the debug console.

You get the full development experience of Visual Studio Code, using the Linux instance installed in WSL.

If you want to open another folder in WSL, open the File menu and choose Open Folder. You’ll get a minimal file and folder navigator for the Linux file system, not the Windows file system.

If you want to switch back to the Windows, select the Show Local option and you’ll get the standard Windows File Open dialog.

In conclusion

Source

More information

With WSL, VS Code and the Remote – WSL extension, your Windows machine becomes an awesome box for developing Linux applications. Your tools run on Windows while your application runs where it will be deployed, on Linux.

Linux development is not limited to WSL. Using the Remote – SSH extension, you can develop against remote SSH hosts with the same fidelity as shown here, all from your Windows desktop.

For more information, check out the following resources:

Finally, if you really want to supercharge your Windows dev box, try out the new Windows Terminal!

This is part 2 of our 3 part series. You can find the full series here:

Craig Loewen

Program Manager, Windows Developer Platform