Unix newlines to Windows newlines (on Windows)

Is there a way (say PowerShell, or a tool) in Windows that can recurse over a directory and convert any Unix files to Windows files.

I’d be perfectly happy with a way in PowerShell to at least detect a Unix file.

It’s easy do this for one single file, but I’m after something a bit more scalable (hence leaning towards a PowerShellish solution).

11 Answers 11

Here is the pure PowerShell way if you are interested.

Finding files with at least one Unix line ending (PowerShell v1):

Here is how you find and covert Unix line endings to Windows line endings. One important thing to note is that an extra line ending (\r\n) will be added to the end of the file if there isn’t already a line ending at the end. If you really don’t want that, I’ll post an example of how you can avoid it (it is a bit more complex).

The above works because PowerShell will automatically split the contents on \n (dropping \r if they exist) and then add \r\n when it writes each thing (in this case a line) to the file. That is why you always end up with a line ending at the end of the file.

Also, I wrote the above code so that it only modifies files that it needs to. If you don’t care about that you can remove the if statement. Oh, make sure that only files get to the ForEach-Object. Other than that, you can do whatever filtering you want at the start of that pipeline.

Windows command to convert Unix line endings?

Is there a Windows command to convert line endings of a file?

We have a test.bat which we need to run to start our server. We use Perforce and we need to have unix line endings in our workspace. For some reason, we are not allowed to change line endings to Windows in our workspaces. However, the server runs on Windows.

Everytime I have to run the bat file, I open it in Notepad++ and choose Edit→EOL conversion→Windows. Is there a way to automate this so that we won’t need to manually change the line endings everytime we sync with Perforce?

Thanks in advance.

17 Answers 17

This can actually be done very easily using the more command which is included in Windows NT and later. To convert input_filename which contains UNIX EOL (End Of Line) \n to output_filename which contains Windows EOL \r\n , just do this:

The more command has additional formatting options that you may not be aware of. Run more/? to learn what else more can do.

Use unix2dos utility. You can download binaries here.

I was dealing with CRLF issues so I decided to build really simple tool for conversion (in NodeJS):

So if you have NodeJS with npm installed you can try it:

Path might be configured dynamically by using Glob regex (same regex as in shell).

So if you can use NodeJS, it’s really simple and you can integrate this command to convert whole workspace to desired line endings.

You can do this without additional tools in VBScript:

Put the above lines in a file unix2dos.vbs and run it like this:

You can also do it in PowerShell:

which could be further simplified to this:

The above statement works without an explicit replacement, because Get-Content implicitly splits input files at any kind of linebreak (CR, LF, and CR-LF), and Set-Content joins the input array with Windows linebreaks (CR-LF) before writing it to a file.

Windows’ MORE is not reliable, it destroys TABs inevitably and adds lines.

unix2dos is part also of MinGW/MSYS, Cygutils, GnuWin32 and other unix binary port collections — and may already be installed.

When python is there, this one-liner converts any line endings to current platform — on any platform:

Or put the one-liner into a .bat / shell script and on the PATH according to your platform:

and use that tool like

Building on TampaHaze’s and MD XF’s helpful answers.

This will change all .txt files in place in the current directory from from LF to CRLF in Command Prompt

If you don’t want to verify every single change

To include subdirectories change

To do all this in a batch file including subdirectories without prompting use below

My contribution for this, converting several files in a folder: for %%z in (*.txt) do (for /f «delims=» %%i in (%%z) do @echo %%i)>%%z.tmp

You could create a simple batch script to do this for you:

Then run and will be instantly converted to DOS line endings.

I cloned my git project using the git bash on windows . All the files then had LF endings. Our repository has CRLF endings as default.

I deleted the project, and then cloned it again using the Windows Command Prompt . The CRLF endings were intact then. In my case, if I had changed the endings for the project, then it would’ve resulted in a huge commit and would’ve caused trouble for my teammates. So, did it this way. Hope this helps somebody.

Based on Endoro’s answer but to keep the blanks, try this:

If you have bash (e.g. git bash), you can use the following script to convert from unix2dos:

similarly, to convert from dos2unix:

Late to the party, but there is still no correct answer using a FOR /F loop.
(But you don’t need a FOR loop at all, the solution from @TampaHaze works too and is much simpler)

The answer from @IR relevant has some drawbacks.
It drops the exclamation marks and can also drop carets.

The trick is to use findstr /n to prefix each line with
: , this avoids skipping of empty lines or lines beginning with ; .
To remove the : the FOR «tokens=1,* delims=:» option can’t be used, because this would remove all leading colons in a line, too.

Therefore the line number is removed by set «line=!line:*:=!» , this requires EnableDelayedExpansion.
But with EnableDelayedExpansion the line set «line=%%L» would drop all exclamation marks and also carets (only when exclams are in the line).

That’s why I disable the delayed expansion before and only enable it for the two lines, where it is required.

The (echo(!line!) looks strange, but has the advantage, that echo( can display any content in !line! and the outer parenthesis avoids accidentials whitespaces at the line end.

Here’s a simple unix2dos.bat file that preserves blank lines and exclamation points:

The output goes to standard out, so redirect unix2dos.bat output to a file if so desired.

It avoids the pitfalls of other previously proposed for /f batch loop solutions by:
1) Working with delayed expansion off, to avoid eating up exclamation marks.
2) Using the for /f tokenizer itself to remove the line number from the findstr /n output lines.
(Using findstr /n is necessary to also get blank lines: They would be dropped if for /f read directly from the input file.)

But, as Jeb pointed out in a comment below, the above solution has one drawback the others don’t: It drops colons at the beginning of lines.

So 2020-04-06 update just for fun, here’s another 1-liner based on findstr.exe, that seems to work fine without the above drawbacks:

The additional tricks are:
3) Use digits 0-9 as delimiters, so that tokens=* skips the initial line number.
4) Use the colon, inserted by findstr /n after the line number, as the token separator after the echo command.

I’ll leave it to Jeb to explain if there are corner cases where echo:something might fail 🙂
All I can say is that this last version successfully restored line endings on my huge batch library, so exceptions, if any, must be quite rare!

Convert Unix line endings to Windows

I recently moved back to Windows from Linux. I have some files with CRLFs, some with LFs and some that are mixed. Is there a utility that will help me find all my Unix-touched files and convert them to proper CRLF terminated files?

The utility must run on Windows, not Linux. I have already moved. I’d rather not install Cygwin if I can avoid it.

14 Answers 14

You can convert them with the unix2dos utility on your Linux platform. There are unix2dos versions available for Windows as well.

If you have Perl installed you can also use this one liner:

Here is an easy and quick way.

Drag and drop the text file into Chrome (I don’t know about other browsers) and then cut and paste back into the original file 🙂

The one I found best for recursively going through folders, allowing file filters and allowing a simple search for «\r\n» and replacing it with just «\n» was Notepad++.

Notepad++ is one of the best, free, open source notepad programs for Windows. It is very simple and powerful. It handled the line ending search/replace just fine. A contractor check a bunch of .c and .h files in to our repository with Linux \r\n line endings, but since most people have standardized on Windows/Eclipse build tools, the files won’t build until the line endings are converted.

For example: sfk addcr -dir . -file .txt -norec
changes LF endings into CR/LF for Windows, on all .txt files of the current directory, but NOT within subdirectories (no recursion).

But this program does a lot more than just that.

On Cygwin, you can convert between Unix and «DOS» AKA Windows files using two built-in utilities:

Convert to DOS CR/LF format:

Convert back to Unix CR format:

The file is left in place with the same name.

I’m going to throw this solution out there. Git will do this. See this post about it

So theoretically you could do this to convert an entire tree

Change crlf to lf if you want to go the other way. NOTE: you’re not done yet, keep reading

Type git status to see which files will be affected. You might have to add lines like

etc to .gitattributes to avoid converting certain files. You can also explicit mark certain files as text

Then just repeat these 2 lines after you’ve edited .gitattributes

Then use git status again to see which files will be changed. When you’re sure all the files you want affected are listed by git status then commit

now check all the files out again

They should now have whatever your desired line endings are

** NOTE: If you were already using git skip the first 3 commands git commands. If you were not using git you can now delete the .gitattributes file and the .git folder.

** Back up your files: the git rm —cached -r deletes them all (although they are theoretically in your git repo (the .git folder) which is how they get restored by the last command git reset —hard . It’s just since files are getting deleted it’s probably best to back them up.

How to convert DOS/Windows newline (CRLF) to Unix newline (LF) in a Bash script

How can I programmatically (i.e., not using vi ) convert DOS/Windows newlines to Unix?

The dos2unix and unix2dos commands are not available on certain systems. How can I emulate these with commands like sed , awk , and tr ?

22 Answers 22

You can use tr to convert from DOS to Unix; however, you can only do this safely if CR appears in your file only as the first byte of a CRLF byte pair. This is usually the case. You then use:

Note that the name DOS-file is different from the name UNIX-file ; if you try to use the same name twice, you will end up with no data in the file.

You can’t do it the other way round (with standard ‘tr’).

If you know how to enter carriage return into a script ( control-V , control-M to enter control-M), then:

where the ‘^M’ is the control-M character. You can also use the bash ANSI-C Quoting mechanism to specify the carriage return:

However, if you’re going to have to do this very often (more than once, roughly speaking), it is far more sensible to install the conversion programs (e.g. dos2unix and unix2dos , or perhaps dtou and utod ) and use them.

If you need to process entire directories and subdirectories, you can use zip :

This will create a zip archive with line endings changed from CRLF to CR. unzip will then put the converted files back in place (and ask you file by file — you can answer: Yes-to-all). Credits to @vmsnomad for pointing this out.

Converting from Windows-style to UNIX-style line endings

The Problem

In a plain text file, to tell the computer that a line of text doesn’t continue forever, the end of each line is marked by a sequence of one or more invisible characters, called control characters. While there are many control characters for different purposes, the relevant ones for line endings are the carriage return (CR) and line feed (LF) characters.

Unfortunately, the programmers of different operating systems have represented line endings using different sequences:

• All versions of Microsoft Windows represent line endings as CR followed by LF.
• UNIX and UNIX-like operating systems (including Mac OS X) represent line endings as LF alone.

Therefore, a text file prepared in a Windows environment will, when copied to a UNIX-like environment such as a NeSI cluster, have an unnecessary carriage return character at the end of each line. To make matters worse, this character will normally be invisible, though in some text editors it will show up as ^M or similar.

Many programs, including the Slurm and LoadLeveler batch queue schedulers, will give errors when given a file containing carriage return characters as input.

Therefore, you will need to convert any such file so it has only UNIX-style line endings before using it on a NeSI cluster.

The Symptoms

In the Slurm job scheduler

If you submit (using sbatch ) a Slurm submission script with Windows-style line endings, you will likely receive the following error:

In other programs

Some UNIX or Linux programs are tolerant to Windows-style line endings, while others give errors. The text of the error is almost infinitely variable, but program behaviours might include the following responses:

• Explicitly stating the problem with line endings
• Complaining more vaguely that the input data is incomplete or corrupt or that there are problems reading it
• Failing in a more serious way such as a segmentation fault

Checking a file’s line ending format

If you have what you think is a text file on the cluster but you don’t know whether its line endings are in the correct format or not, you can run the following command:

Depending on the contents of foo.txt , the output of this command may vary, but if the output has «CR» or «CRLF» in it, you will need to convert foo.txt to UNIX format line endings if you want to use it on the cluster.

How to Convert

Converting using Notepad++

In the Windows text editing program Notepad++ (not to be confused with ordinary Notepad), there is a function to prepare text files with UNIX-style line endings.

To write your file in this way, while you have the file open, go to the Edit menu, select the «EOL Conversion» submenu, and from the options that come up select «UNIX/OSX Format». The next time you save the file, its line endings will, all going well, be saved with UNIX-style line endings.

You can check what format line endings you are currently editing in by looking in the status bar at the bottom of the window. Between the range box (a box containing Ln, Col and Sel entries) and the text encoding box (which will contain UTF-8, ANSI, or some other technical string) will be a box containing the current line ending format.

• In most cases, this box will contain the text «DOS\Windows».
• In a few cases, such as the file having been prepared on a UNIX or Linux machine or a Mac, it will contain the text «UNIX».
• It is possible, though highly unlikely by now, that the file may have old-style (pre-OSX) Mac line endings, in which case the box will contain the text «Macintosh».

Please note that if you change a file’s line ending style, you must save your changes before copying the file anywhere, including to a cluster.

Converting using dos2unix

Suppose, though, that you’ve copied a text file to the cluster already, and you realise you need to convert it to UNIX format. How do you do that?

Simple: Use the program dos2unix .

Just give the name of your file to dos2unix as an argument, and it will convert the file’s line endings to UNIX format:

There are other options in the rare case that you don’t want to just modify your existing file; run man dos2unix for details.