Linux remove empty lines

How to delete empty lines using sed command under Linux / UNIX

Sed Delete Empty Line Syntax

The syntax is as follows to delete empty lines using sed:
sed ‘/^$/d’
echo «LINE» | sed ‘/^$/d’
echo «$VAR» | sed ‘/^$/d’
sed ‘/^[[:space:]]*$/d’ input_file

Examples

So to delete all empty lines from a file called /tmp/data.txt, enter:
$ sed ‘/^$/d’ /tmp/data.txt
To store output to another file use redirection operator:
$ sed ‘/^$/d’ /tmp/data.txt > /tmp/output.txt
OR update the file using -i option:
$ sed -i ‘/^$/d’ /tmp/data.txt

Removing all whitespace only using sed

Consider the following file:
more /tmp/data.txt
Sample outputs:

To delete all empty lines, run:
sed ‘/^[[:space:]]*$/d’ input.file
sed -i ‘/^[[:space:]]*$/d’ input.file
sed -r ‘/^\s*$/d’ input.file > output.file
sed -i ‘/^[[:space:]]*$/d’ /tmp/data.txt
Verify with the help of cat command:
cat output.file
cat /tmp/data.txt

• No ads and tracking
• In-depth guides for developers and sysadmins at Opensourceflare✨
• Join my Patreon to support independent content creators and start reading latest guides:
  • How to set up Redis sentinel cluster on Ubuntu or Debian Linux
  • How To Set Up SSH Keys With YubiKey as two-factor authentication (U2F/FIDO2)
  • How to set up Mariadb Galera cluster on Ubuntu or Debian Linux
  • A podman tutorial for beginners – part I (run Linux containers without Docker and in daemonless mode)
  • How to protect Linux against rogue USB devices using USBGuard

Join Patreon ➔

Deleting a line that matches a pattern

You can also match a word or a pattern to delete. For example
$ cat data.txt
Output:

To delete all lines that contain a ‘Windows’ word, enter:
$ sed ‘/Windows/d’ /tmp/data.txt > /tmp/output.data.txt
GNU Sed support -i option to edit files in place:
$ sed -i ‘/Windows/d’ /tmp/data.txt

A note about awk

The awk command offers a much simpler solution as compared to sed command. In this example, remove all empty lines using awk:
awk ‘NF’ my_input > my_output
cat my_output

Conclusion

In short, use any one of the following sed syntax to remove all empty lines from a text file under Linux, macOS, *BSD and Unix-like systems:
sed ‘/^[[:space:]]*$/d’ input > output
sed ‘/^\s*$/d’ in > out
sed ‘/^$/d’ input > output
sed -n ‘/^\s*$/!p’ in.txt > out.txt
See more sed examples from our page:

🐧 Get the latest tutorials on Linux, Open Source & DevOps via

Источник

This post and this website contains affiliate links. See my disclosure about affiliate links.

how to remove empty or blank lines from text files in linux

You may need to remove empty lines from a text file in order to compact them, for providing as input to another utility or just for readability. These could be any kind of files such as a badly formatted html file, a log file with lots of empty or blank lines or even source code files with double spacing.

Using text editors

If these are files of manageable length and size, and there are not a whole lot of them, then you could probably use a text editor such as vi or kate to perform the task.

Vi Editor

vi or vim editor is one of the most commonly used editor in Linux. It can be used from a command prompt without access to a X based system, which is often useful. Vi is a very powerful tool, which support many of the formats and command line syntaxes discussed later in the post.

This flexibility means that there are several different options available in the editor. All of it involves commands from the escape mode of the editor. To enter escape mode in vi, hit the escape key at any time from within the editor.

the ‘g‘ in the expression represents global mode and executes on all lines of the file. The second part is the regular expression that needs to match. The ‘^‘ matches the start of the line and ‘$‘ matches the end of the line. Together ‘^$‘ matches an empty line. The ‘d‘ at the end stands for the delete command. This will delete all empty lines, but will not remove lines that contain a blanks, spaces or white spaces.

Modifying the regular expression as above to match one or more occurrence of all white space will make sure that it will delete empty lines as well as the blank lines.

This is actually the negation of the previous syntax. The ‘v‘ stands for the reverse global mode, which means the command will execute on lines that does not match the expression. The ‘d‘ at the end is the same as before and stands for the delete command.

There are almost limitless options here to pretty much do anything, such as deleting all lines that match three blanks, condense multiple blank file into a single blank line, etc etc…..Also, the expressions used above or a variation thereof is used later in the post with the command line utilities as well.

Kate Editor

Kate is another graphical editor. It is more or less the default text editor for KDE. You can of course delete the blank lines manually one after another as in any text editor, but Kate does provide some easy to use commands to let you do this.

Kate supports a Editor Component Command Line feature that you can access from View->Switch to Command Line in the menu bar or by using the shortcut F7.

rmblank: In the command line, type in the command rmblank to remove all empty lines in the current document. This will not however remove any lines with blanks or white spaces in them.

rtrim: This will trim the lines and get rid of any trailing spaces from the end of the lines. Doing a rtrim before rmblank will allow you to remove both the empty lines as well as the blank lines.

Many other advanced text editors such as UltraEdit support search and replace functionality using regular expressions. It is very likely that you can do this replacement using regex expressions as well in these editors.

Using command line utilities

Although using text editor is a good option, using a command line utility is faster and easier. It is also very useful if you many different files to process. There are several utilities in Linux that can be used to remove empty lines in text files. The most popular and commonly used ones are awk, sed, and grep.

bash$ awk ‘/./’ filename.txt > newfile.txt

awk is a very versatile command. The above command removes all the empty lines in a file and prints the result out to the standard output. You can redirect the output to a file to save the result as a new file. The above command will remove all the empty lines in a file named filename.txt and saves it to newfile.txt.

The above command will not remove lines that has white spaces in them such as spaces or other non-printable characters. If you want to remove both the empty lines as well as the lines with spaces, then try the following command.

bash$ awk NF filename.txt > newfile.txt

sed is another popular utility for processing text. This follows a similar pattern to what was mentioned earlier in the vi editor section. The expressions used are very similar, mostly because as with most Linux commands it uses the regular expressions to match the input text.

bash$ sed ‘/^$/d’ filename.txt > newfile.txt

The above command will remove empty lines but not the lines that may contain spaces or other non-printable characters. In order to remove the empty lines as well as the blank lines use the following sed command.

bash$ sed ‘/^\s*$/d’ filename.txt > newfile.txt

The following command is actually just a negation of the first sed command. This will not remove (notice the !d) any lines that matches the regular expression, which matches all lines with any character in them.

bash$ sed ‘/./!d’ filename.txt > newfile.txt

grep is more of a matcher utility compared to the sed command mentioned earlier. Both will take similar regular expressions to match the empty and blank lines in a file. You will need to use the -v option with grep to negate the expression and print out the non-matching lines.

The example below will delete all the empty lines in text file filename.txt, but not the ones with whitespaces.

bash$ grep ‘.’ filename.txt > newfile.txt

In order to remove empty lines and as well all the lines with just white spaces in them, use the following grep command.

bash$ grep -v ‘^\s*$’ filename.txt > newfile.txt

All of the above utilities gets the job done pretty much the same way by using very similar expressions to match the empty or blank lines in the file. You can now extend these to remove any matching lines not just the empty lines. For example, if you wish to remove all comments in a script file, you can use the following variation.

Assuming that the comments in the script file always occur in a single line by itself and starts with a # (hash or pound), the following command will remove all of those lines.

bash$ sed ‘/^\s*#/d’ filename.txt > newfile.txt

You may further modify the expression to suit your needs.

Источник

How to Remove/Delete the empty lines from a file in Linux

Some times you may wants to remove or delete the empty lines in a file in Linux.It can be done in many ways but we have listed few simple methods in this article.

You may aware of that grep, awk and sed commands are specialized for textual data manipulation.

Navigate to the following URL, if you would like to read more similar kind of topics.

These fall in advanced commands category because these are used in most of the shell script to do required things.

It can be done using the following 5 methods.

• sed Command: Stream editor for filtering and transforming text.
• grep Command: Print lines that match patterns.
• cat Command: It concatenate files and print on the standard output.
• tr Command: Translate or delete characters.
• awk Command: The awk utility shall execute programs written in the awk programming language, which is specialized for textual data manipulation.
• perl Command: Perl is a programming language specially designed for text editing.

To test this, I had already created the file 2daygeek.txt with some texts and empty lines.

Now everything is ready and i’m going to test this in multiple ways.

How to Remove/Delete the empty lines from a file in Linux using sed Command?

Sed is a stream editor used to perform basic text transformations on an input stream (a file or input from a pipeline).

Details are follow:

• sed: It’s a command
• //: It holds the searching string.
• ^: Matches start of string.
• $: Matches end of string.
• d: Delete the matched string.
• 2daygeek.txt: Source file name.

How to Remove/Delete the empty lines from a file in Linux using grep Command?

grep searches for PATTERNS in each FILE. PATTERNS is one or patterns separated by newline characters, and grep prints each line that matches a pattern.

Details are follow:

• grep: It’s a command
• .: Replaces any character.
• ^: matches start of string.
• $: matches end of string.
• E: For extended regular expressions pattern matching.
• e: For regular expressions pattern matching.
• v: To select non-matching lines from the file.
• 2daygeek.txt: Source file name.

How to Remove/Delete the empty lines from a file in Linux using awk Command?

The awk utility shall execute programs written in the awk programming language, which is specialized for textual data manipulation. An awk program is a sequence of patterns and corresponding actions.

Details are follow:

• awk: It’s a command
• //: It holds the searching string.
• ^: matches start of string.
• $: matches end of string.
• .: Replaces any character.
• !: Delete the matched string.
• 2daygeek.txt: Source file name.

How to Remove/Delete the empty lines from a file in Linux using cat and tr Command?

cat stands for concatenate. It is very frequently used in Linux to reads data from a file.

cat is one of the most frequently used commands on Unix-like operating systems. It offers three functions which is related to text file such as display content of a file, combine multiple files into the single output and create a new file.

Translate, squeeze, and/or delete characters from standard input, writing to standard output.

Details are follow:

• cat: It’s a command
• tr: It’s a command
• |: Pipe symbol. It pass first command output as a input to another command.
• s: Replace each sequence of a repeated character that is listed in the last specified SET.
• \n: To add a new line.
• 2daygeek.txt: Source file name.

How to Remove/Delete the empty lines from a file in Linux using perl Command?

Perl stands in for “Practical Extraction and Reporting Language”. Perl is a programming language specially designed for text editing. It is now widely used for a variety of purposes including Linux system administration, network programming, web development, etc.

Источник

Remove empty lines in a text file via grep

How can I remove all the empty new lines in this FILE ?

Output of command:

11 Answers 11

grep . FILE

(And if you really want to do it in sed, then: sed -e /^$/d FILE )

(And if you really want to do it in awk, then: awk /./ FILE )

Try the following:

Here is a solution that removes all lines that are either blank or contain only space characters:

If removing empty lines means lines including any spaces, use:

Try this: sed -i ‘/^[ \t]*$/d’ file-name

It will delete all blank lines having any no. of white spaces (spaces or tabs) i.e. (0 or more) in the file.

Note: there is a ‘space’ followed by ‘\t’ inside the square bracket.

The modifier -i will force to write the updated contents back in the file. Without this flag you can see the empty lines got deleted on the screen but the actual file will not be affected.

it gives as output only lines with at least 2 characters.

See also the results with grep ‘^’ my_file outputs

and also with grep ‘^.’ my_file outputs

For multiple files (edit in-place):

Without modifying the file (just print on the standard output):

Perl might be overkill, but it works just as well.

Removes all lines which are completely blank:

Removes all lines which are completely blank, or only contain whitespace:

Variation which edits the original and makes a .bak file:

If you want to know what the total lines of code is in your Xcode project and you are not interested in listing the count for each swift file then this will give you the answer. It removes lines with no code at all and removes lines that are prefixed with the comment //

Run it at the root level of your Xcode project.

If you have comment blocks in your code beginning with /* and ending with */ such as:

then these will get included in the count. (Too hard).

Источник