Line ending mac os

Random Stuff

Various ramblings of sysadmin, programmer, dancer, coffee snob, food lover and Winnipegger.

Monday, April 16, 2012

Convert Unix, Windows, Mac line endings using OS X command

Today I had to copy some MySQL data from Debian server into test environment on my MacBook. While importing data from tab delimited text files, I noticed warnings that data in the last column of several tables was being truncated. I looked at the tables and noticed MySQL doing some very strange formatting when printing them. It looked almost as if last column was padded with a bunch of white space. I opened import file in TextWrangler and it appeared fine, but when I looked in document options, I saw this:

The good ol’ EOL (end-of-line) character.

Different operating systems use different characters to mark the end of line:

• Unix / Linux / OS X uses LF (line feed, ‘\n‘, 0x0A)
• Macs prior to OS X use CR (carriage return, ‘\r‘, 0x0D)
• Windows / DOS uses CR+LF (carriage return followed by line feed, ‘\r\n‘, 0x0D0A)

I’m guessing the person who sent me those files first transferred them to his Windows machine in ASCII mode, so newline characters got automatically converted during transfer.

Since some of the files were very big, instead of changing line endings in TextWrangler I decided to use command line (shocking, I know).

First I executed
to confirm existence of the dreaded ^M (carriage return) at the end of every line, and then ran
to generate new files without CR characters.

tr (translate character) is a nice little utility that does just that, substitutes one character with another or deletes it (like in my example). It’s available on pretty much any *nix distro so no need to install additional software.

Источник

Configure Visual Studio to use UNIX line endings

We would like to use Visual Studio 2005 to work on a local copy of an SVN repository. This local copy has been checked out by Mac OS X (and updates and commits will only be made under Mac OS X, so no problem there), and as a consequence the line endings are UNIX-style.

We fear that Visual Studio will introduce Windows-style line endings. Is it possible to force Visual Studio to use UNIX line endings?

9 Answers 9

Warning: This solution no longer works for Visual Studio 2017 and later. Instead, both of the answers by jcox and Munther Jaber are needed. I have combined them into one answer.

As OP states «File > Advanced Save Options», select Unix Line Endings.

This will only affect new files that are created. Fixing any that were previously created can be done file-by-file or you can search for tools that will fix on-bulk.

Here are some options available for Visual Studio Community 2017

1. «File > Advanced Save Options» has been removed by microsoft due to «uncommon use». Whatever that means. https://developercommunity.visualstudio.com/content/problem/8290/file-advanced-save-options-option-is-missed.html You can add it back by going to «Tools>Customize», then «Commands» tab, select the drop down next to «Menu Bar» select «File» then «Add Command»>File>Advanced Save Options..». You can then reorder it in the file menu by using «move down».

I don’t know if you will have to then set the advanced save options for each and every file, but it might prevent the issue I was having where my Visual Studio kept adding CL RF line endings into my files that were uniformly LF.

But I took it one step further and I added an extension called «Line Endings Unifier» by going to «Tools>Extensions and Updates>Online» and then searching for «line endings» in the search bar to the right. I will use this to automatically force all of my scripts to save with uniform line endings of my choice, but you can do more with it. https://marketplace.visualstudio.com/items?itemName=JakubBielawa.LineEndingsUnifier

strip’em is another solution that does something similar to Line Endings Unifier. http://www.grebulon.com/software/stripem.php

I am not sure how they differ or the advantages/disadvantages of either. I’m mainly using Line Endings Unifier just because it was in the Visual Studio Marketplace. I think I’ve used all of these methods in the past, but my memory is fuzzy.

Источник

How to make git understand Mac (CR) line endings

For some reasons one of my files contains old style Mac line endings (after editing on OSX). These are «CR» (carriage return) characters and show up as ^M in git diff .

Git does not understand that they are line ending codes (really how hard can it be?) and interprets the whole file as a single line.

I know that I can convert the files to LF or CRLF endings and then commit them back, however since git automatically converts my Windows (CRLF) line endings to LF, I was hoping that it would take care of CR line endings as well.

Is there a way to make git interpret CR as a line ending?

2 Answers 2

Create a filter driver plus .gitattributes : create a smudge filter that runs tr ‘\n’ ‘\r’ and a clean filter that runs tr ‘\r’ ‘\n’ , and mark the file(s) in question as using this filter. Store the file inside Git using LF-only line endings. (The filter driver is defined in a .git/config or $HOME/.gitconfig file and the names or name-patterns for the files go in .gitattributes .)

As you have seen, Git strongly prefers newline-terminated lines. (It can work with newline-separated lines, where the last line is missing the terminator, but this means that adding a line results in a change to the previous final line, since it now has a newline terminator while the new final line is missing the newline terminator.) This does not matter for the individual snapshots, but does matter for producing useful diffs.

Modern MacOS uses newlines, like everyone else. Only ancient backwards-compatible formats have CR-only line endings. See, e.g., this SuperUser Stack Exchange web site posting.

Git does not have a built in filter for converting to or from such line endings. Git does, however, have a general purpose mechanism for making alterations in work-tree files.

Remember that when Git stores any file in a snapshot, the file is represented by what Git calls a blob object, which is stored internally in a special, compressed (sometimes highly compressed), Git-only form. This form is not useful to anything but Git, so when you get the files in a useful form—via git checkout , for instance—Git expands them into their usual form for your computer. Meanwhile, any time you take a normal file like this and convert it to Git-only form, Git compresses the file down to its Git-only form. That happens whenever you copy a file back into Git’s index using git add .

The index copy of each file exists while you have the work-tree in place, just like the committed copy. The index copy is in the same Git-only format. The key difference here is that the committed copy can’t be changed, but the index copy can be changed. Running git commit takes a snapshot of whatever is in the index right at that point, and makes that the new snapshot for the new commit. Hence the index acts as what will go into the next commit. Using git checkout , you copy some existing commit into the index, and have Git expand it into the work-tree; then using git add , you selectively replace particular index copies with compressed versions of the work-tree files that you have changed.

This copying, to or from index and work-tree, is the ideal point at which to do Windows-style LF-to-CRLF conversions, or vice versa, so this is where Git does it. If you have some other conversion to perform, not directly built in to Git, this is where you tell Git to do it.

Smudge and clean filters

A smudge filter is one that Git applies when converting a file from compressed index copy to work-tree copy. Here, if you’ve chosen to have newline characters replaced with CRLF Windows-style line enders-or-separators, Git has an internal converter that will do that: eol=crlf . A clean filter is one that Git applies when converting a file from uncompressed work-tree copy to compressed index copy; here again, eol=crlf directs Git to do the backwards conversion.

If you want to replace newlines with CR-only, you must invent your own converters. Let’s say you call the overall process convert-cr :

(instead of *.csv eol=crlf ). This line goes into .gitattributes (which is a commit-able file, and you should commit it).

Now you must define the convert-cr filter. This goes in a Git configuration file, and here we find a minor flaw: the configuration file is not commit-able. This is a security issue: Git will run arbitrary commands here, and if I could commit this file and you clone it, you’ll run the commands I specify, without getting a chance to vet them first. So you must put this into your .git/config yourself, or into your global configuration ( git config —global —edit for instance):

Now whenever Git converts from Git-only format, it will translate the newlines to CRs, and whenever Git converts to Git-only format, it will translate the CRs to newlines.

This does not help with existing stored files

Any existing snapshots that you have today that have \r inside them, are stored that way forever. Git will never change any existing stored file! Stored data are precious and inviolate. There is nothing you can do about this. Well, there is almost nothing: you can throw out those commits entirely, making new and improved commits that you use instead. But that’s quite painful: every commit remembers its parent commits, so if you replace an early commit in your repository, you must replace every child, grandchild, and so on, so that they all remember this new sequence of commits. ( git filter-branch does this job.)

Источник

Remap «Home» and «End» to beginning and end of line

Most of my computing time is spent in Linux (with occasional time in Windows). On these platforms the Home and End keys universally jump to the beginning or end of the current line.

I am gradually getting used to using the one Apple computer in my life, but it is still frustrating than these keys seem to be bound to the beginning and end of the document and there doesn’t seem to be any key at all for the line based actions.

1. Are there existing keybindings I’m missing for beginning and end of line? (I mostly use a Terminal, Chrome and LibreOffice)
2. Is there a way to rebind the functions of these keys to the functions I am familiar with? I am willing to give up the document based versions which seem of little use to me.

Also two caveats:

1. I am not the only user on the system. It would be nice if my user settings didn’t break other peoples experience. If this isn’t possible, I think I probably have the strongest preference!
2. I do use multiple languages and switch between keyboard layouts, so any solutions would be expected to work even when toggling between multiple keyboard layouts.

12 Answers 12

The default shortcuts for moving to beginning or end of (wrapped) lines are ⌘ ← and ⌘ → . ⌥ ↑ and ⌥ ↓ or ⌃ A and ⌃ E move to the beginning or end of unwrapped lines (or paragraphs). ⌥ ← and ⌥ → move backwards/forward by words, and all of these are compatible with holding Shift to select during the corresponding moves.

You could remap home and end by creating

/Library/KeyBindings/ and saving a property list like this as DefaultKeyBinding.dict :

Most of the keybindings for editing text in OS X are defined in /System/Library/Frameworks/AppKit.framework/Resources/StandardKeyBinding.dict .

Applying changes requires reopening applications. DefaultKeyBinding.dict is ignored by some old versions of Xcode (works with latest version 6.3.1), Terminal, and many cross-platform applications.

See Cocoa Text System and my website for more information about the customizable keybindings.

Terminal’s keybindings can be customized in Preferences > Settings > Keyboard. \033OH moves to the beginning of a line and \033OF to the end of a line.

In Eclipse, key bindings should be modified in Preferences > General > Keys. You need to modify default bindings for commands Line Start and Line End (replace ⌘← by ↖ and ⌘→ by ↘). For selection to work, also modify Select Line Start and Select Line End.

PS: You may need to logout and login again for the

Источник

Converting newline formatting from Mac to Windows

I need a conversion utility/script that will convert a .sql dump file generated on Mac to one readable on Windows. This is a continuation of a problem I had here. The issue seems to be with newline formatting in text files, but I can’t find a tool to make the conversion.

12 Answers 12

Windows uses carriage return + line feed for newline:

Unix only uses Line feed for newline:

In conclusion, simply replace every occurence of \n by \r\n .
Both unix2dos and dos2unix are not by default available on Mac OSX.
Fortunately, you can simply use Perl or sed to do the job:

This is an improved version of Anne’s answer — if you use perl, you can do the edit on the file ‘in-place’ rather than generating a new file:

You can install unix2dos with Homebrew

Then you can do this:

You can also convert dos files to unix:

Just do tr delete:

You probably want unix2dos:

You can either run unix2dos on your DOS/Windows machine using cygwin or on your Mac using MacPorts.

1. Install dos2unix with homebrew
2. Run find ./ -type f -exec dos2unix <> \; to recursively convert all line-endings within current folder

Here’s a really simple approach, worked well for me, courtesy Davy Schmeits’s Weblog:

Where foo is the file that has the Control+M characters at the end of the line, and foo2 the new file you are creating.

vim also can convert files from UNIX to DOS format. For example:

The following is a complete script based on the above answers along with sanity checking and works on Mac OS X and should work on other Linux / Unix systems as well (although this has not been tested).

On Yosemite OSX, use this command:

where the ^M sequence is achieved by pressing Ctrl + V then Enter .

Expanding on the answers of Anne and JosephH, using perl in a short perl script, since i’m too lazy to type the perl-one-liner very time.
Create a file, named for example «unix2dos.pl» and put it in a directory in your path. Edit the file to contain the 2 lines:

Assuming that «which perl» returns «/usr/bin/perl» on your system. Make the file executable (chmod u+x unix2dos.pl).

Example:
$ echo «hello» > xxx
$ od -c xxx (checking that the file ends with a nl)
0000000 h e l l o \n

$ unix2dos.pl xxx
$ od -c xxx (checking that it ends now in cr lf)
0000000 h e l l o \r \n

In Xcode 9 in the left panel open/choose your file in project navigator. If file is not there, drug-and-drop it into the project navigator.

On right panel find Text Settings and change Line Endings to Windows (CRLF) .

XCode screendump

Источник