Do not change Linux files using Windows apps and tools

November 17th, 2016

Update – May 2019 Windows 10 version 1903 has now been released and allows an important update to WSL that allows Windows applications and tools to access Linux files directly. To do this, WSL hosts a new 9P fileserver, which exposes distro filesystems to Windows apps and tools via \\wsl$\ \ ! You can read all about the feature in the blog post “What’s new for WSL in Windows 10 version 1903“.

The post below contains the original guidance, which also remains true: DO NOT access the Linux files inside of your %LOCALAPPDATA% folder, since you should only be accessing them . If you’d like to learn more about why you shouldn’t go digging through the Appdata folder for your Linux files, please read on!

There is one hard-and-fast rule when it comes to WSL on Windows:

DO NOT, under ANY circumstances , access, create, and/or modify Linux files inside of your `%LOCALAPPDATA%` folder using Windows apps, tools, scripts, consoles, etc.

Opening > files using some Windows tools may read-lock the opened files > and/or folders, preventing updates to file contents and/or metadata, > essentially resulting in corrupted files/folders.

Creating/changing Linux files in your Appdata folder from Windows will likely result in data corruption and/or damage your Linux environment requiring you to uninstall & reinstall your distro!

Note: In beta versions of WSL, your “Linux files” are any of the files and folders under %localappdata%\lxss – which is where the Linux filesystem – distro and your own files – are stored on your drive. In supported versions of WSL, the filesystems for the distros you install via the Microsoft Store are stored elsewhere on disk … for good reason!

What SHOULD I do?

To work on files using both Windows and Linux tools, store files in your Windows filesystem – this will enable you to access the same files from both Windows and from your Linux distros via /mnt/ /

When you access files on your Windows filesystem from within Bash, WSL honors the NT filesystem behaviors (e.g. case-insensitivity), permissions, etc. so you can easily access the same files using both Windows tools and Bash tools without having to copy files back and forth between filesystems.

Therefore, be sure to follow these three rules in order to avoid losing files, and/or corrupting your data:

1. DO store files in your Windows filesystem that you want to access/create/modify using Windows tools AND Linux tools
2. DO access / create / modify files in your Linux distros’ filesystems from Windows apps, tools, scripts or consoles using \\wsl$\ \

DO NOT access / create / modify files in your Linux distros’ filesystems from Windows apps, tools, scripts or consoles by finding the files in Appdata

Remember: There’s a reason we store your distros’ filesystems in non-obvious locations!

Why is this?

File metadata (e.g. permissions, ownership, timestamps, etc.) is stored for every file, folder, etc., on your storage devices. Alas, file metadata representation differs from one OS to another: Windows file metadata is different from Linux file metadata.

While it’s the OS’ job to store and update your file metadata, most of Windows doesn’t know anything about Linux, nor Linux file metadata, and doesn’t automatically add or update Linux file metadata for all Windows files because that would impose an unnecessary overhead on the vast majority of Windows users who will never run WSL.

It’s WSL’s job to write/update Linux file metadata for all the files under your Linux filesystem root (i.e. /), storing the Linux metadata in each file’s NTFS extended attributes. WSL also synthesizes pseudo metadata for most of the files in your Windows filesystem.

The problem arises when, for example, you use a Windows app/tool to open, create and/or modify a file under your distro root: Since the file was created with a Windows tool, the file won’t have any Linux file metadata (e.g. permissions, owner, access/update timestamps, etc.). Thus, to Linux, (which only receives Linux file metadata), the file may be reported as empty, may not even exist, or may have some metadata, but that metadata may not reflect the file’s details resulted in the file’s contents being corrupted.

Further, as in Linux, some Windows tools implement unusual filesystem access patterns to handle file updating and don’t actually edit files in-place: When apps/tools save changes to a file, the original files are often deleted and re-created, etc. During such operations, NT file “extended properties” (i.e. Linux file permissions) are often not copied and are “lost”.

For more background into how the WSL filesystem infrastructure works, be sure to read/watch this AWESOME blog & video which explains things in much more detail.

Please help us share this guidance far and wide – tweet, post and blog, linking back to this post!!

• [2017-03-14] – Updated “Why is this”; thanks to @mn in comments
• [2019-01-15] – Minor updates & improvements
• Last Updated: [2019-05-29] – Added an edit to the top of the post to describe the new feature allowing users to access Linux files from Windows

How to convert DOS/Windows newline (CRLF) to Unix newline (LF) in a Bash script

How can I programmatically (i.e., not using vi ) convert DOS/Windows newlines to Unix?

The dos2unix and unix2dos commands are not available on certain systems. How can I emulate these with commands like sed , awk , and tr ?

22 Answers 22

You can use tr to convert from DOS to Unix; however, you can only do this safely if CR appears in your file only as the first byte of a CRLF byte pair. This is usually the case. You then use:

Note that the name DOS-file is different from the name UNIX-file ; if you try to use the same name twice, you will end up with no data in the file.

You can’t do it the other way round (with standard ‘tr’).

If you know how to enter carriage return into a script ( control-V , control-M to enter control-M), then:

where the ‘^M’ is the control-M character. You can also use the bash ANSI-C Quoting mechanism to specify the carriage return:

However, if you’re going to have to do this very often (more than once, roughly speaking), it is far more sensible to install the conversion programs (e.g. dos2unix and unix2dos , or perhaps dtou and utod ) and use them.

If you need to process entire directories and subdirectories, you can use zip :

This will create a zip archive with line endings changed from CRLF to CR. unzip will then put the converted files back in place (and ask you file by file — you can answer: Yes-to-all). Credits to @vmsnomad for pointing this out.

What is the Windows Subsystem for Linux?

The Windows Subsystem for Linux lets developers run a GNU/Linux environment — including most command-line tools, utilities, and applications — directly on Windows, unmodified, without the overhead of a traditional virtual machine or dualboot setup.

• Choose your favorite GNU/Linux distributions from the Microsoft Store.
• Run common command-line tools such as grep , sed , awk , or other ELF-64 binaries.
• Run Bash shell scripts and GNU/Linux command-line applications including:
  • Tools: vim, emacs, tmux
  • Languages: NodeJS, Javascript, Python, Ruby, C/C++, C# & F#, Rust, Go, etc.
  • Services: SSHD, MySQL, Apache, lighttpd, MongoDB, PostgreSQL.
• Install additional software using your own GNU/Linux distribution package manager.
• Invoke Windows applications using a Unix-like command-line shell.
• Invoke GNU/Linux applications on Windows.

What is WSL 2?

WSL 2 is a new version of the Windows Subsystem for Linux architecture that powers the Windows Subsystem for Linux to run ELF64 Linux binaries on Windows. Its primary goals are to increase file system performance, as well as adding full system call compatibility.

This new architecture changes how these Linux binaries interact with Windows and your computer’s hardware, but still provides the same user experience as in WSL 1 (the current widely available version).

Individual Linux distributions can be run with either the WSL 1 or WSL 2 architecture. Each distribution can be upgraded or downgraded at any time and you can run WSL 1 and WSL 2 distributions side by side. WSL 2 uses an entirely new architecture that benefits from running a real Linux kernel.

WSL commands and launch configurations

Ways to run WSL

There are several ways to run a Linux distribution with WSL once it’s installed.

1. Open your Linux distribution by visiting the Windows Start menu and typing the name of your installed distributions. For example: «Ubuntu».
2. From Windows Command Prompt or PowerShell, enter the name of your installed distribution. For example: ubuntu
3. From Windows Command Prompt or PowerShell, to open your default Linux distribution inside your current command line, enter: wsl.exe .
4. From Windows Command Prompt or PowerShell, to open your default Linux distribution inside your current command line, enter: wsl [command] .

Which method you should use depends on what you’re doing. If you’ve opened a WSL command line within a Windows Prompt or PowerShell window and want to exit, enter the command: exit .

Launch WSL by distribution

Running a distribution using it’s distro-specific application launches that distribution in it’s own console window.

It is the same as clicking «Launch» in the Microsoft store.

You can also run the distribution from the command line by running [distribution].exe .

The disadvantage of running a distribution from the command line in this way is that it will automatically change your working directory from the current directory to the distribution’s home directory.

Example: (using PowerShell)

wsl and wsl [command]

The best way to run WSL from the command line is using wsl.exe .

Example: (using PowerShell)

Not only does wsl keep the current working directory in place, it lets you run a single command along side Windows commands.

Example: (using PowerShell)

Example: (using PowerShell)

Managing multiple Linux Distributions

In Windows 10 Version 1903 and later, you can use wsl.exe to manage your distributions in the Windows Subsystem for Linux (WSL), including listing available distributions, setting a default distribution, and uninstalling distributions.

Each Linux distribution independently manages its own configurations. To see distribution-specific commands, run [distro.exe] /? . For example ubuntu /? .

List distributions

wsl -l , wsl —list
Lists available Linux distributions available to WSL. If a distribution is listed, it’s installed and ready to use.

wsl —list —all Lists all distributions, including ones that aren’t currently usable. They may be in the process of installing, uninstalling, or are in a broken state.

wsl —list —running Lists all distributions that are currently running.

Set a default distribution

The default WSL distribution is the one that runs when you run wsl on a command line.

wsl -s , wsl —setdefault

Sets the default distribution to .

Example: (using PowerShell)
wsl -s Ubuntu would set my default distribution to Ubuntu. Now when I run wsl npm init it will run in Ubuntu. If I run wsl it will open an Ubuntu session.

Unregister and reinstall a distribution

While Linux distributions can be installed through the Microsoft store, they can’t be uninstalled through the store. WSL Config allows distributions to be unregistered/uninstalled.

Unregistering also allows distributions to be reinstalled.

Caution: Once unregistered, all data, settings, and software associated with that distribution will be permanently lost. Reinstalling from the store will install a clean copy of the distribution.

wsl —unregister
Unregisters the distribution from WSL so it can be reinstalled or cleaned up.

For example: wsl —unregister Ubuntu would remove Ubuntu from the distributions available in WSL. When I run wsl —list it will not be listed.

To reinstall, find the distribution in the Microsoft store and select «Launch».

Run as a specific user

wsl -u , wsl —user

Run WSL as the specified user. Please note that user must exist inside of the WSL distribution.

Change the default user for a distribution

Change the default user that for your distribution log-in. The user has to already exist inside the distribution in order to become the default user.

For example: ubuntu config —default-user johndoe would change the default user for the Ubuntu distribution to the «johndoe» user.

If you are having trouble figuring out the name of your distribution, see List distributions for the command to list the official name of the installed distributions.

Run a specific distribution

wsl -d , wsl —distribution

Run a specified distribution of WSL, can be used to send commands to a specific distribution without having to change your default.

Managing multiple Linux Distributions in earlier Windows versions

In Windows 10 prior to version 1903, the WSL Config ( wslconfig.exe ) command-line tool should be used to manage Linux distributions running on the Windows Subsystem for Linux (WSL). It lets you list available distributions, set a default distribution, and uninstall distributions.

While WSL Config is helpful for settings that span or coordinate distributions, each Linux distribution independently manages its own configurations. To see distribution-specific commands, run [distro.exe] /? . For example ubuntu /? .

To see all available options for wslconfig, run: wslconfig /?

To list distributions, use:

wslconfig /list
Lists available Linux distributions available to WSL. If a distribution is listed, it’s installed and ready to use.

wslconfig /list /all
Lists all distributions, including ones that aren’t currently usable. They may be in the process of installing, uninstalling, or are in a broken state.

To set a default distribution that runs when you run wsl on a command line:

wslconfig /setdefault Sets the default distribution to .

Example: (using PowerShell)
wslconfig /setdefault Ubuntu would set my default distribution to Ubuntu. Now when I run wsl npm init it will run in Ubuntu. If I run wsl it will open an Ubuntu session.

To unregister and reinstall a distribution:

wslconfig /unregister
Unregisters the distribution from WSL so it can be reinstalled or cleaned up.

For example: wslconfig /unregister Ubuntu would remove Ubuntu from the distributions available in WSL. When I run wslconfig /list it will not be listed.

To reinstall, find the distribution in the Microsoft store and select «Launch».

Configure per distro launch settings with wslconf

Available in Windows Build 17093 and later

Automatically configure functionality in WSL that will be applied every time you launch the subsystem using wsl.conf . This includes automount options and network configuration.

wsl.conf is located in each Linux distribution in /etc/wsl.conf . If the file is not there, you can create it yourself. WSL will detect the existence of the file and will read its contents. If the file is missing or malformed (that is, improper markup formatting), WSL will continue to launch as normal.

Here is a sample wsl.conf file you could add into your distributions:

When launching multiple Linux shells for the same distribution, you must wait until the Linux subsystem stops running, this can take approximately 8 seconds after closing the last instance of the distribution shell. If you launch a distribution (ie. Ubuntu), modify the wsl.conf file, close the distribution, and then re-launch it. You might assume that your changes to the wsl.conf file have immediately gone into effect. This is not currently the case as the subsystem could still be running. You must wait

8 seconds for the subsystem to stop before relaunching in order to give enough time for your changes to be picked up. You can check to see whether your Linux distribution (shell) is still running after closing it by using PowerShell with the command: wsl —list —running . If no distributions are running, you will receive the response: «There are no running distributions.» You can now restart the distribution to see your wsl.conf updates applied.

Configuration Options

In keeping with .ini conventions, keys are declared under a section.

WSL supports four sections: automount , network , interop , and user .

automount

key	value	default	notes
enabled	boolean	true	true causes fixed drives (i.e C:/ or D:/ ) to be automatically mounted with DrvFs under /mnt . false means drives won’t be mounted automatically, but you could still mount them manually or via fstab .
mountFsTab	boolean	true	true sets /etc/fstab to be processed on WSL start. /etc/fstab is a file where you can declare other filesystems, like an SMB share. Thus, you can mount these filesystems automatically in WSL on start up.
root	String	/mnt/	Sets the directory where fixed drives will be automatically mounted. For example, if you have a directory in WSL at /windir/ and you specify that as the root, you would expect to see your fixed drives mounted at /windir/c
options	comma-separated list of values	empty string	This value is appended to the default DrvFs mount options string. Only DrvFs-specific options can be specified. Options that the mount binary would normally parse into a flag are not supported. If you want to explicitly specify those options, you must include every drive for which you want to do so in /etc/fstab.

By default, WSL sets the uid and gid to the value of the default user (in Ubuntu distro, the default user is created with uid=1000,gid=1000). If the user specifies a gid or uid option explicitly via this key, the associated value will be overwritten. Otherwise, the default value will always be appended.

Note: These options are applied as the mount options for all automatically mounted drives. To change the options for a specific drive only, use /etc/fstab instead.

Mount options

Setting different mount options for Windows drives (DrvFs) can control how file permissions are calculated for Windows files. The following options are available:

Key	Description	Default
uid	The User ID used for the owner of all files	The default User ID of your WSL distro (On first installation this defaults to 1000)
gid	The Group ID used for the owner of all files	The default group ID of your WSL distro (On first installation this defaults to 1000)
umask	An octal mask of permissions to exclude for all files and directories	000
fmask	An octal mask of permissions to exclude for all files	000
dmask	An octal mask of permissions to exclude for all directories	000
metadata	Whether metadata is added to Windows files to support Linux system permissions	disabled
case	Determines directories treated as case sensitive and whether new directories created with WSL will have the flag set. See Per-directory case sensitivity and WSL for a detailed explanation of the options.	dir

Note: The permission masks are put through a logical OR operation before being applied to files or directories.

network

Section label: [network]

key	value	default	notes
generateHosts	boolean	true	true sets WSL to generate /etc/hosts . The hosts file contains a static map of hostnames corresponding IP address.
generateResolvConf	boolean	true	true set WSL to generate /etc/resolv.conf . The resolv.conf contains a DNS list that are capable of resolving a given hostname to its IP address.

interop

Section label: [interop]

These options are available in Insider Build 17713 and later.

key	value	default	notes
enabled	boolean	true	Setting this key will determine whether WSL will support launching Windows processes.
appendWindowsPath	boolean	true	Setting this key will determine whether WSL will add Windows path elements to the $PATH environment variable.

Section label: [user]

These options are available in Build 18980 and later.

key	value	default	notes
default	string	The initial username created on first run	Setting this key specifies which user to run as when first starting a WSL session.

Configure global options with .wslconfig

Available in Windows Build 19041 and later

You can configure global WSL options by placing a .wslconfig file into the root directory of your users folder: C:\Users\ \.wslconfig . Many of these files are related to WSL 2, please keep in mind you may need to run wsl —shutdown to shut down the WSL 2 VM and then restart your WSL instance for these changes to take affect.

Here is a sample .wslconfig file:

This file can contain the following options:

WSL 2 Settings

Section label: [wsl2]

These settings affect the VM that powers any WSL 2 distribution.

key	value	default	notes
kernel	string	The Microsoft built kernel provided inbox	An absolute Windows path to a custom Linux kernel.
memory	size	50% of total memory on Windows or 8GB, whichever is less; on builds before 20175: 80% of your total memory on Windows	How much memory to assign to the WSL 2 VM.
processors	number	The same number of processors on Windows	How many processors to assign to the WSL 2 VM.
localhostForwarding	boolean	true	Boolean specifying if ports bound to wildcard or localhost in the WSL 2 VM should be connectable from the host via localhost:port.
kernelCommandLine	string	Blank	Additional kernel command line arguments.
swap	size	25% of memory size on Windows rounded up to the nearest GB	How much swap space to add to the WSL 2 VM, 0 for no swap file.
swapFile	string	%USERPROFILE%\AppData\Local\Temp\swap.vhdx	An absolute Windows path to the swap virtual hard disk.

• Note: This value is true for Windows Build 19041 and may be different in Windows builds in the Insiders program

Entries with the path value must be Windows paths with escaped backslashes, e.g: C:\\Temp\\myCustomKernel

Entries with the size value must be a size followed by a unit, for example 8GB or 512MB .