Git status ignore line endings / identical files / windows & linux environment / dropbox / mled

ignore line ending differences?

I use randomly Windows and Linux to work on the project. The project is in Dropbox.

I found a lot about how do make git diff ignore line endings. Since i use meld git diff opens meld for each file. And meld says «identical file».

So how do I avoid this. Git should only open meld for changed files. And git status should not report files as changed if only the file ending is different.

EDIT: Cause:

This happened because of this setting on Windows

So I checked out the working copy on Linux and set core.autocrlf false on Windows.

It would be still nice to know how to make git status ignore different new lines.

6 Answers 6

Try setting core.autocrlf value like this :

This answer seems relevant since the OP makes reference to a need for a multi-OS solution. This Github help article details available approaches for handling lines endings cross-OS. There are global and per-repo approaches to managing cross-os line endings.

Global approach

Configure Git line endings handling on Linux or OS X:

Configure Git line endings handling on Windows:

Per-repo approach:

In the root of your repo, create a .gitattributes file and define line ending settings for your project files, one line at a time in the following format: path_regex line-ending-settings where line-ending-settings is one of the following:

• text
• binary (files that Git should not modify line endings for — as this can cause some image types such as PNGs not to render in a browser)

The text value can be configured further to instruct Git on how to handle line endings for matching files:

• text — Changes line endings to OS native line endings.
• text eol=crlf — Converts line endings to CRLF on checkout.
• text eol=lf — Converts line endings to LF on checkout.
• text=auto — Sensible default that leaves line handle up to Git’s discretion.

Here is the content of a sample .gitattributes file:

More on how to refresh your repo after changing line endings settings here. Tldr:

backup your files with Git, delete every file in your repository (except the .git directory), and then restore the files all at once. Save your current files in Git, so that none of your work is lost.

git commit -m «Saving files before refreshing line endings»

Remove the index and force Git to rescan the working directory.

Rewrite the Git index to pick up all the new line endings.

Show the rewritten, normalized files.

In some cases, this is all that needs to be done. Others may need to complete the following additional steps:

Add all your changed files back, and prepare them for a commit. This is your chance to inspect which files, if any, were unchanged.

It is perfectly safe to see a lot of messages here that read[s] «warning: CRLF will be replaced by LF in file.»

Commit the changes to your repository.

git commit -m «Normalize all the line endings»

How to convert Windows end of line in Unix end of line (CR/LF to LF)

I’m a Java developer and I’m using Ubuntu to develop. The project was created in Windows with Eclipse and it’s using the Windows-1252 encoding.

To convert to UTF-8 I’ve used the recode program:

This command gives this error:

Convert line endings from CR/LF to a single LF: Edit the file with Vim, give the command :set ff=unix and save the file. Recode now should run without errors.

Nice, but I’ve many files to remove the CR/LF character from, and I can’t open each to do it. Vi doesn’t provide any option to command line for Bash operations.

Can sed be used to do this? How?

8 Answers 8

There should be a program called dos2unix that will fix line endings for you. If it’s not already on your Linux box, it should be available via the package manager.

sed cannot match \n because the trailing newline is removed before the line is put into the pattern space, but it can match \r , so you can convert \r\n (DOS) to \n (Unix) by removing \r:

Warning: this will change the original file

However, you cannot change from Unix EOL to DOS or old Mac ( \r ) by this. More readings here:

Actually, Vim does allow what you’re looking for. Enter Vim, and type the following commands:

The first of these commands sets the argument list to every file matching **/*.java , which is all Java files, recursively. The second of these commands does the following to each file in the argument list, in turn:

• Sets the line-endings to Unix style (you already know this)
• Writes the file out iff it’s been changed
• Proceeds to the next file

The tr command can also do this:

and should be available to you.

You’ll need to run tr from within a script, since it cannot work with file names. For example, create a file myscript.sh:

Running myscript.sh would process all the java files in the current directory and its subdirectories.

I’ll take a little exception to jichao’s answer. You can actually do everything he just talked about fairly easily. Instead of looking for a \n , just look for carriage return at the end of the line.

To change from Unix back to DOS, simply look for the last character on the line and add a form feed to it. (I’ll add -r to make this easier with grep regular expressions.)

Theoretically, the file could be changed to Mac style by adding code to the last example that also appends the next line of input to the first line until all lines have been processed. I won’t try to make that example here, though.

Warning: -i changes the actual file. If you want a backup to be made, add a string of characters after -i . This will move the existing file to a file with the same name with your characters added to the end.

Convert Unix line endings to Windows

I recently moved back to Windows from Linux. I have some files with CRLFs, some with LFs and some that are mixed. Is there a utility that will help me find all my Unix-touched files and convert them to proper CRLF terminated files?

The utility must run on Windows, not Linux. I have already moved. I’d rather not install Cygwin if I can avoid it.

14 Answers 14

You can convert them with the unix2dos utility on your Linux platform. There are unix2dos versions available for Windows as well.

If you have Perl installed you can also use this one liner:

Here is an easy and quick way.

Drag and drop the text file into Chrome (I don’t know about other browsers) and then cut and paste back into the original file 🙂

The one I found best for recursively going through folders, allowing file filters and allowing a simple search for «\r\n» and replacing it with just «\n» was Notepad++.

Notepad++ is one of the best, free, open source notepad programs for Windows. It is very simple and powerful. It handled the line ending search/replace just fine. A contractor check a bunch of .c and .h files in to our repository with Linux \r\n line endings, but since most people have standardized on Windows/Eclipse build tools, the files won’t build until the line endings are converted.

For example: sfk addcr -dir . -file .txt -norec
changes LF endings into CR/LF for Windows, on all .txt files of the current directory, but NOT within subdirectories (no recursion).

But this program does a lot more than just that.

On Cygwin, you can convert between Unix and «DOS» AKA Windows files using two built-in utilities:

Convert to DOS CR/LF format:

Convert back to Unix CR format:

The file is left in place with the same name.

I’m going to throw this solution out there. Git will do this. See this post about it

So theoretically you could do this to convert an entire tree

Change crlf to lf if you want to go the other way. NOTE: you’re not done yet, keep reading

Type git status to see which files will be affected. You might have to add lines like

etc to .gitattributes to avoid converting certain files. You can also explicit mark certain files as text

Then just repeat these 2 lines after you’ve edited .gitattributes

Then use git status again to see which files will be changed. When you’re sure all the files you want affected are listed by git status then commit

now check all the files out again

They should now have whatever your desired line endings are

** NOTE: If you were already using git skip the first 3 commands git commands. If you were not using git you can now delete the .gitattributes file and the .git folder.

** Back up your files: the git rm —cached -r deletes them all (although they are theoretically in your git repo (the .git folder) which is how they get restored by the last command git reset —hard . It’s just since files are getting deleted it’s probably best to back them up.

Windows command to convert Unix line endings?

Is there a Windows command to convert line endings of a file?

We have a test.bat which we need to run to start our server. We use Perforce and we need to have unix line endings in our workspace. For some reason, we are not allowed to change line endings to Windows in our workspaces. However, the server runs on Windows.

Everytime I have to run the bat file, I open it in Notepad++ and choose Edit→EOL conversion→Windows. Is there a way to automate this so that we won’t need to manually change the line endings everytime we sync with Perforce?

Thanks in advance.

17 Answers 17

This can actually be done very easily using the more command which is included in Windows NT and later. To convert input_filename which contains UNIX EOL (End Of Line) \n to output_filename which contains Windows EOL \r\n , just do this:

The more command has additional formatting options that you may not be aware of. Run more/? to learn what else more can do.

Use unix2dos utility. You can download binaries here.

I was dealing with CRLF issues so I decided to build really simple tool for conversion (in NodeJS):

So if you have NodeJS with npm installed you can try it:

Path might be configured dynamically by using Glob regex (same regex as in shell).

So if you can use NodeJS, it’s really simple and you can integrate this command to convert whole workspace to desired line endings.

You can do this without additional tools in VBScript:

Put the above lines in a file unix2dos.vbs and run it like this:

You can also do it in PowerShell:

which could be further simplified to this:

The above statement works without an explicit replacement, because Get-Content implicitly splits input files at any kind of linebreak (CR, LF, and CR-LF), and Set-Content joins the input array with Windows linebreaks (CR-LF) before writing it to a file.

Windows’ MORE is not reliable, it destroys TABs inevitably and adds lines.

unix2dos is part also of MinGW/MSYS, Cygutils, GnuWin32 and other unix binary port collections — and may already be installed.

When python is there, this one-liner converts any line endings to current platform — on any platform:

Or put the one-liner into a .bat / shell script and on the PATH according to your platform:

and use that tool like

Building on TampaHaze’s and MD XF’s helpful answers.

This will change all .txt files in place in the current directory from from LF to CRLF in Command Prompt

If you don’t want to verify every single change

To include subdirectories change

To do all this in a batch file including subdirectories without prompting use below

My contribution for this, converting several files in a folder: for %%z in (*.txt) do (for /f «delims=» %%i in (%%z) do @echo %%i)>%%z.tmp

You could create a simple batch script to do this for you:

Then run and will be instantly converted to DOS line endings.

I cloned my git project using the git bash on windows . All the files then had LF endings. Our repository has CRLF endings as default.

I deleted the project, and then cloned it again using the Windows Command Prompt . The CRLF endings were intact then. In my case, if I had changed the endings for the project, then it would’ve resulted in a huge commit and would’ve caused trouble for my teammates. So, did it this way. Hope this helps somebody.

Based on Endoro’s answer but to keep the blanks, try this:

If you have bash (e.g. git bash), you can use the following script to convert from unix2dos:

similarly, to convert from dos2unix:

Late to the party, but there is still no correct answer using a FOR /F loop.
(But you don’t need a FOR loop at all, the solution from @TampaHaze works too and is much simpler)

The answer from @IR relevant has some drawbacks.
It drops the exclamation marks and can also drop carets.

The trick is to use findstr /n to prefix each line with
: , this avoids skipping of empty lines or lines beginning with ; .
To remove the : the FOR «tokens=1,* delims=:» option can’t be used, because this would remove all leading colons in a line, too.

Therefore the line number is removed by set «line=!line:*:=!» , this requires EnableDelayedExpansion.
But with EnableDelayedExpansion the line set «line=%%L» would drop all exclamation marks and also carets (only when exclams are in the line).

That’s why I disable the delayed expansion before and only enable it for the two lines, where it is required.

The (echo(!line!) looks strange, but has the advantage, that echo( can display any content in !line! and the outer parenthesis avoids accidentials whitespaces at the line end.

Here’s a simple unix2dos.bat file that preserves blank lines and exclamation points:

The output goes to standard out, so redirect unix2dos.bat output to a file if so desired.

It avoids the pitfalls of other previously proposed for /f batch loop solutions by:
1) Working with delayed expansion off, to avoid eating up exclamation marks.
2) Using the for /f tokenizer itself to remove the line number from the findstr /n output lines.
(Using findstr /n is necessary to also get blank lines: They would be dropped if for /f read directly from the input file.)

But, as Jeb pointed out in a comment below, the above solution has one drawback the others don’t: It drops colons at the beginning of lines.

So 2020-04-06 update just for fun, here’s another 1-liner based on findstr.exe, that seems to work fine without the above drawbacks:

The additional tricks are:
3) Use digits 0-9 as delimiters, so that tokens=* skips the initial line number.
4) Use the colon, inserted by findstr /n after the line number, as the token separator after the echo command.

I’ll leave it to Jeb to explain if there are corner cases where echo:something might fail 🙂
All I can say is that this last version successfully restored line endings on my huge batch library, so exceptions, if any, must be quite rare!