Configuring Git to handle line endings
To avoid problems in your diffs, you can configure Git to properly handle line endings.
About line endings
Every time you press return on your keyboard you insert an invisible character called a line ending. Different operating systems handle line endings differently.
When you’re collaborating on projects with Git and GitHub, Git might produce unexpected results if, for example, you’re working on a Windows machine, and your collaborator has made a change in macOS.
You can configure Git to handle line endings automatically so you can collaborate effectively with people who use different operating systems.
Global settings for line endings
The git config core.autocrlf command is used to change how Git handles line endings. It takes a single argument.
On macOS, you simply pass input to the configuration. For example:
On Windows, you simply pass true to the configuration. For example:
On Linux, you simply pass input to the configuration. For example:
Optionally, you can configure a .gitattributes file to manage how Git reads line endings in a specific repository. When you commit this file to a repository, it overrides the core.autocrlf setting for all repository contributors. This ensures consistent behavior for all users, regardless of their Git settings and environment.
The .gitattributes file must be created in the root of the repository and committed like any other file.
A .gitattributes file looks like a table with two columns:
- On the left is the file name for Git to match.
- On the right is the line ending configuration that Git should use for those files.
Here’s an example .gitattributes file. You can use it as a template for your repositories:
You’ll notice that files are matched— *.c , *.sln , *.png —, separated by a space, then given a setting— text , text eol=crlf , binary . We’ll go over some possible settings below.
text=auto Git will handle the files in whatever way it thinks is best. This is a good default option.
text eol=crlf Git will always convert line endings to CRLF on checkout. You should use this for files that must keep CRLF endings, even on OSX or Linux.
text eol=lf Git will always convert line endings to LF on checkout. You should use this for files that must keep LF endings, even on Windows.
binary Git will understand that the files specified are not text, and it should not try to change them. The binary setting is also an alias for -text -diff .
Refreshing a repository after changing line endings
When you set the core.autocrlf option or commit a .gitattributes file, you may find that Git reports changes to files that you have not modified. Git has changed line endings to match your new configuration.
To ensure that all the line endings in your repository match your new configuration, backup your files with Git, delete all files in your repository (except the .git directory), then restore the files all at once.
- Save your current files in Git, so that none of your work is lost.
- Add all your changed files back and normalize the line endings.
- Show the rewritten, normalized files.
- Commit the changes to your repository.
Источник
jonlabelle / crlf.py
| #!/usr/bin/env python |
| «»»Replace line breaks, from one format to another.»»» |
| from __future__ import print_function |
| import argparse |
| import glob |
| import os |
| import sys |
| import tempfile |
| from stat import ST_ATIME , ST_MTIME |
| LF = ‘ \n ‘ |
| CRLF = ‘ \r \n ‘ |
| CR = ‘ \r ‘ |
| def _normalize_line_endings ( lines , line_ending = ‘unix’ ): |
| r»»»Normalize line endings to unix (\n), windows (\r\n) or mac (\r). |
| :param lines: The lines to normalize. |
| :param line_ending: The line ending format. |
| Acceptable values are ‘unix’ (default), ‘windows’ and ‘mac’. |
| :return: Line endings normalized. |
| «»» |
| lines = lines . replace ( CRLF , LF ). replace ( CR , LF ) |
| if line_ending == ‘windows’ : |
| lines = lines . replace ( LF , CRLF ) |
| elif line_ending == ‘mac’ : |
| lines = lines . replace ( LF , CR ) |
| return lines |
| def _copy_file_time ( source , destination ): |
| «»»Copy one file’s atime and mtime to another. |
| :param source: Source file. |
| :param destination: Destination file. |
| «»» |
| file1 , file2 = source , destination |
| try : |
| stat1 = os . stat ( file1 ) |
| except os . error : |
| sys . stderr . write ( file1 + ‘ : cannot stat \n ‘ ) |
| sys . exit ( 1 ) |
| try : |
| os . utime ( file2 , ( stat1 [ ST_ATIME ], stat1 [ ST_MTIME ])) |
| except os . error : |
| sys . stderr . write ( file2 + ‘ : cannot change time \n ‘ ) |
| sys . exit ( 2 ) |
| def _create_temp_file ( contents ): |
| «»»Create a temp file. |
| :param contents: The temp file contents. |
| :return: The absolute path of the created temp file. |
| «»» |
| tf = tempfile . NamedTemporaryFile ( mode = ‘wb’ , suffix = ‘txt’ , delete = False ) |
| tf . write ( contents ) |
| tf . close () |
| return tf . name |
| def _delete_file_if_exists ( filepath ): |
| «»»Delete the file if it exists. |
| :param filepath: The file path. |
| «»» |
| if os . path . exists ( filepath ): |
| os . remove ( filepath ) |
| def _read_file_data ( filepath ): |
| «»»Read file data. |
| :param filepath: The file path. |
| :return: The file contents. |
| «»» |
| data = open ( filepath , ‘rb’ ). read () |
| return data |
| def _write_file_data ( filepath , data ): |
| «»»Write file data. |
| :param filepath: The file path. |
| :param data: The data to write. |
| «»» |
| f = open ( filepath , ‘wb’ ) |
| f . write ( data ) |
| f . close () |
| def main (): |
| «»»Main.»»» |
| parser = argparse . ArgumentParser ( |
| prog = ‘crlf’ , |
| description = ‘Replace CRLF (windows) line endings with LF (unix) ‘ |
| ‘line endings in files, and vice-versa’ ) |
| parser . add_argument ( |
| ‘-q’ , ‘—quiet’ , |
| help = ‘suppress descriptive messages from output’ , |
| action = ‘store_true’ , |
| default = False ) |
| parser . add_argument ( |
| ‘-n’ , ‘—dryrun’ , |
| help = ‘show changes, but do not modify files’ , |
| action = ‘store_true’ , |
| default = False ) |
| parser . add_argument ( |
| ‘-w’ , ‘—windows’ , |
| help = ‘replace LF (unix) line endings with CRLF (windows) line endings’ , |
| action = ‘store_true’ , |
| default = False ) |
| parser . add_argument ( |
| ‘-u’ , ‘—unix’ , |
| help = ‘replace CRLF (windows) line endings with LF (unix) ‘ |
| ‘line endings (default)’ , |
| action = ‘store_true’ , |
| default = False ) |
| parser . add_argument ( |
| ‘-t’ , ‘—timestamps’ , |
| help = «maintains the modified file’s time stamps (atime and mtime)» , |
| action = ‘store_true’ , |
| default = False ) |
| parser . add_argument ( |
| ‘files’ , |
| nargs = ‘+’ , |
| help = «a list of files or file glob patterns to process» , |
| default = ‘.’ ) |
| if len ( sys . argv ) 2 : |
| parser . print_help () |
| sys . exit ( 2 ) |
| args = parser . parse_args () |
| if args . windows is True and args . unix is True : |
| sys . stderr . write ( «Ambiguous options specified, ‘unix’ and ‘windows’. « |
| «Please choose one option, or the other. \n » ) |
| sys . exit ( 2 ) |
| files_to_process = [] |
| for arg_file in args . files : |
| files_to_process . extend ( glob . glob ( arg_file )) |
| if len ( files_to_process ) 0 : |
| if args . quiet is False : |
| sys . stderr . write ( ‘No files matched the specified pattern. \n ‘ ) |
| sys . exit ( 2 ) |
| if args . dryrun is True and args . quiet is False : |
| print ( ‘Dry-run only, files will NOT be modified.’ ) |
| for file_to_process in files_to_process : |
| if os . path . isdir ( file_to_process ): |
| if args . quiet is False : |
| print ( «- ‘<0>‘ : is a directory (skip)» . format ( file_to_process )) |
| continue |
| if os . path . isfile ( file_to_process ): |
| data = _read_file_data ( file_to_process ) |
| if ‘ \\ 0’ in data : |
| if args . quiet is False : |
| print ( «- ‘<0>‘ : is a binary file (skip)» . format ( file_to_process )) |
| continue |
| if args . windows is True : |
| new_data = _normalize_line_endings ( data , line_ending = ‘windows’ ) |
| else : |
| new_data = _normalize_line_endings ( data , line_ending = ‘unix’ ) |
| if new_data != data : |
| if args . quiet is False : |
| if args . windows is True : |
| if args . dryrun is True : |
| print ( «+ ‘<0>‘ : LF would be replaced with CRLF» . format ( file_to_process )) |
| else : |
| print ( «+ ‘<0>‘ : replacing LF with CRLF» . format ( file_to_process )) |
| else : |
| if args . dryrun is True : |
| print ( «+ ‘<0>‘ : CRLF would be replaced with LF» . format ( file_to_process )) |
| else : |
| print ( «+ ‘<0>‘ : replacing CRLF with LF» . format ( file_to_process )) |
| tmp_file_path = «» |
| if args . dryrun is False : |
| try : |
| if args . timestamps is True : |
| # create a temp file with the original file |
| # contents and copy the old file’s atime a mtime |
| tmp_file_path = _create_temp_file ( data ) |
| _copy_file_time ( file_to_process , tmp_file_path ) |
| # overwrite the current file with the modified contents |
| _write_file_data ( file_to_process , new_data ) |
| if args . timestamps is True : |
| # copy the original file’s atime and mtime back to |
| # the original file w/ the modified contents, |
| # and delete the temp file. |
| _copy_file_time ( tmp_file_path , file_to_process ) |
| _delete_file_if_exists ( tmp_file_path ) |
| except Exception as ex : |
| sys . stderr . write ( ‘error : <0>\n ‘ . format ( str ( ex ))) |
| sys . exit ( 1 ) |
| else : |
| if args . quiet is False : |
| if args . windows is True : |
| print ( «- ‘<0>‘ : line endings already CRLF (windows)» . format ( file_to_process )) |
| else : |
| print ( «- ‘<0>‘ : line endings already LF (unix)» . format ( file_to_process )) |
| else : |
| sys . stderr . write ( «- ‘<0>‘ : file not found \n » . format ( file_to_process )) |
| sys . exit ( 1 ) |
| if __name__ == ‘__main__’ : |
| main () |
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Источник
Will Anderson
Docker for Windows: Dealing With Windows Line Endings
One of the issues with Docker (or any Linux/macOS based system) on Windows is the difference in how line endings are handled. Windows ends lines in a carriage return and a linefeed \r\n while Linux and macOS only use a linefeed \n . This becomes a problem when you try to create a file in Windows and run it on a Linux/macOS system, because those systems treat the \r as a piece of text rather than a newline.
As a concrete example, if you try to clone the official WordPress docker image and build the image on Windows, you’ll run into problems when it tries to execute the docker-entrypoint.sh file.
The first line of that file is #!/bin/bash , which is the shebang syntax to say “run this file using /bin/bash”
However, if the file originated in windows, that first line will be interpreted as “run this file using /bin/bash\r”, and “bash\r” of course doesn’t exist, so you get this error:
There are a couple of ways to handle this issue.
Converting Line Endings During Build
Unix has a handy CLI tool for converting line endings called dos2unix . If you want to create a robust image, you can install do2unix as a dependency, then convert any files that you copy into the image. Then as a cleanup step, you can uninstall do2unix from the image (unless the image depends on it after the build).
Your Dockerfile might look something like this:
The main idea is to copy the script onto the machine, then use dos2unix to convert the line endings, and finally remove dos2unix from the machine (and clean up the files created by apt-get).
This is a good option if you’re managing the image, but what if you’re trying to build an image that someone else maintains?
Cloning Git Projects With Unix Line Endings
If you just want to clone and build an existing Docker image, you can use a Git flag to store the repository locally with Unix style line endings.
However, it’s worth noting that any new files you create will likely have Windows line endings, so you’ll still need to convert them before using them inside the Docker image.
Conclusion
That should cover the basics of line endings between Windows and Linux/macOS. These techniques apply beyond just Docker, but hopefully the Docker-specific details will help someone who’s struggling with Docker for Windows.
I haven’t configured comments for this blog, but if you want to get in touch, you can find me on Twitter
Источник
