Linux delete one line from file

Your Own Linux.

Linux How To’s | Bash Scripting | Python

Monday, 20 April 2015

Sed Command in Linux — Delete Lines from a File

This is the third article of the «Super sed ‘ Series», in which we will learn how to delete lines from a file using line numbers and regular expressions. In the first two articles, we have learned, how to print lines using sed and how to append and insert lines in a file using sed. If you are familiar with sed print syntaxes, then this article would be pretty easy to understand.

For those learners who are new to sed , let’s have a brief introduction of the Super sed :

• sed stand for Stream EDitor and it being based on the ed editor, it borrows most of the commands from the ed . It was developed by Lee E. McMahon of Bell Labs.
• sed offers large range of text transformations that include printing lines, deleting lines, editing line in-place, search and replace, appending and inserting lines, etc.
• sed is useful whenever you need to perform common editing operations on multiple lines without using ‘vi’ editor.
• Whenever sed is executed on an input file or on the contents from stdin, sed reads the file line-by-line and after removing the trailing newline, places it in the «Pattern space», where the commands are executed on them after conditions (as in case of regex matching) are verified, and then printed on the stdout.

Deleting Lines from a File using sed

Before we start, just remember two points:

1. sed «d» command lets us print specific lines based on the line number or regex provided.
2. When ^ means beginning of the line and $ denotes end of the line, ^$ makes a «Blank Line», very useful while removing empty lines from a file.

For our better understanding, let us have a file sedtest.txt with contents as follows:

A. sed — Delete Lines with Line Number

1. Delete ‘N’th line

This will remove ‘N’th line in the FILE.txt .

Example:
To delete 1st line,

While, to remove last line,

If you run above commands and inspect the file sedtest.txt , you would find that, the original contents of that file would not change. In case you want to remove lines in the file and save the changes (i.e. edit the file in place), you will have to use the option -i as shown in below example:

2. Delete all Lines starting from ‘M’th up to ‘N’th

This will remove the block of lines starting at line number M and ending at line number N .

Example:
To delete block of lines from 3rd line to 8th line.

Similarly, in order to delete lines starting from 5th up to last line, you would run-

3. Delete Every ‘M’th Line Starting from ‘N’th Line

This will start from Nth line and delete every Mth line coming after that. Note that, Nth line will also be deleted.

Example:
To delete every alternate line staring from 2nd one.

B. sed — Delete Lines using Regular Expression/Pattern

1. Delete lines containing a specific Pattern

This will delete all those lines that contain the pattern provided.

2. Delete all those lines not containing the Pattern

This will delete all those lines which do not include the pattern provided.

3. Delete block of lines starting from pattern matching line

This will remove the lines from the one where pattern matches, till ‘N’th line.

Similarly, so as to delete all the lines starting from pattern matching line till the end, you would use ‘/PATTERN/,$ d’ as follows —

4. Delete block of lines starting from Nth and ending at pattern matching line

This will delete the lines starting from the ‘N’th line, till the one where pattern matches.

5. Delete block of lines between two Pattern matches

This will start deleting lines from 1st pattern match till 2nd pattern match.

6. Deleting all blank lines

7. Deleting lines containing either of two patterns

That was all about the third article on sed command. More articles on sed are coming soon. So, stay tuned. Of course, do not forget to share your feedback in the comment section below.

Источник

Remove the last line from a file in Bash

I have a file, foo.txt , containing the following lines:

I want a simple command that results in the contents of foo.txt being:

15 Answers 15

The -i option does not exist in GNU sed versions older than 3.95, so you have to use it as a filter with a temporary file:

Of course, in that case you could also use head -n -1 instead of sed .

MacOS:

On Mac OS X (as of 10.7.4), the equivalent of the sed -i command above is

This is by far the fastest and simplest solution, especially on big files:

if You want to delete the top line use this:

which means output lines starting at line 2.

Do not use sed for deleting lines from the top or bottom of a file — it’s very very slow if the file is large.

For large files

I had trouble with all the answers here because I was working with a HUGE file (

300Gb) and none of the solutions scaled. Here’s my solution:

Or alternatively, as a one liner:

In words: Find out the length of the file you want to end up with (length of file minus length of length of its last line, using bc ), and set that position to be the end of the file (by dd ing one byte of /dev/null onto it).

This is fast because tail starts reading from the end, and dd will overwrite the file in place rather than copy (and parse) every line of the file, which is what the other solutions do.

NOTE: This removes the line from the file in place! Make a backup or test on a dummy file before trying it out on your own file!

Источник

How can I remove the first line of a text file using bash/sed script?

I need to repeatedly remove the first line from a huge text file using a bash script.

Right now I am using sed -i -e «1d» $FILE — but it takes around a minute to do the deletion.

Is there a more efficient way to accomplish this?

100MB. MacBook Air 2013 with SSD.

16 Answers 16

-n x : Just print the last x lines. tail -n 5 would give you the last 5 lines of the input. The + sign kind of inverts the argument and make tail print anything but the first x-1 lines. tail -n +1 would print the whole file, tail -n +2 everything but the first line, etc.

GNU tail is much faster than sed . tail is also available on BSD and the -n +2 flag is consistent across both tools. Check the FreeBSD or OS X man pages for more.

The BSD version can be much slower than sed , though. I wonder how they managed that; tail should just read a file line by line while sed does pretty complex operations involving interpreting a script, applying regular expressions and the like.

Note: You may be tempted to use

but this will give you an empty file. The reason is that the redirection ( > ) happens before tail is invoked by the shell:

1. Shell truncates file $FILE
2. Shell creates a new process for tail
3. Shell redirects stdout of the tail process to $FILE
4. tail reads from the now empty $FILE

If you want to remove the first line inside the file, you should use:

The && will make sure that the file doesn’t get overwritten when there is a problem.

You can use -i to update the file without using ‘>’ operator. The following command will delete the first line from the file and save it to the file.

For those who are on SunOS which is non-GNU, the following code will help:

You can easily do this with:

on the command line; or to remove the first line of a file permanently, use the in-place mode of sed with the -i flag:

No, that’s about as efficient as you’re going to get. You could write a C program which could do the job a little faster (less startup time and processing arguments) but it will probably tend towards the same speed as sed as files get large (and I assume they’re large if it’s taking a minute).

But your question suffers from the same problem as so many others in that it pre-supposes the solution. If you were to tell us in detail what you’re trying to do rather then how, we may be able to suggest a better option.

For example, if this is a file A that some other program B processes, one solution would be to not strip off the first line, but modify program B to process it differently.

Let’s say all your programs append to this file A and program B currently reads and processes the first line before deleting it.

You could re-engineer program B so that it didn’t try to delete the first line but maintains a persistent (probably file-based) offset into the file A so that, next time it runs, it could seek to that offset, process the line there, and update the offset.

Then, at a quiet time (midnight?), it could do special processing of file A to delete all lines currently processed and set the offset back to 0.

It will certainly be faster for a program to open and seek a file rather than open and rewrite. This discussion assumes you have control over program B, of course. I don’t know if that’s the case but there may be other possible solutions if you provide further information.

Источник

Delete specific line number(s) from a text file using sed?

I want to delete one or more specific line numbers from a file. How would I do this using sed?

7 Answers 7

If you want to delete lines from 5 through 10 and line 12th:

This will print the results to the screen. If you want to save the results to the same file:

This will back the file up to file.bak , and delete the given lines.

Note: Line numbers start at 1. The first line of the file is 1, not 0.

You can delete a particular single line with its line number by

This will delete the line on 33 line number and save the updated file.

and awk as well

This is very often a symptom of an antipattern. The tool which produced the line numbers may well be replaced with one which deletes the lines right away. For example;

(where deletelines is the utility you are imagining you need) is the same as

Having said that, if you are in a situation where you genuinely need to perform this task, you can generate a simple sed script from the file of line numbers. Humorously (but perhaps slightly confusingly) you can do this with sed .

This accepts a file of line numbers, one per line, and produces, on standard output, the same line numbers with d appended after each. This is a valid sed script, which we can save to a file, or (on some platforms) pipe to another sed instance:

On some platforms, sed -f does not understand the option argument — to mean standard input, so you have to redirect the script to a temporary file, and clean it up when you are done, or maybe replace the lone dash with /dev/stdin or /proc/$pid/fd/1 if your OS (or shell) has that.

As always, you can add -i before the -f option to have sed edit the target file in place, instead of producing the result on standard output. On *BSDish platforms (including OSX) you need to supply an explicit argument to -i as well; a common idiom is to supply an empty argument; -i » .

Источник

Unix Sed Command to Delete Lines in File — 15 Examples

Sed Command to Delete Lines: Sed command can be used to delete or remove specific lines which matches a given pattern or in a particular position in a file. Here we will see how to delete lines using sed command with various examples.

The following file contains a sample data which is used as input file in all the examples:

Sed Command to Delete Lines — Based on Position in File

In the following examples, the sed command removes the lines in file that are in a particular position in a file.

1. Delete first line or header line

The d option in sed command is used to delete a line. The syntax for deleting a line is:

Here N indicates Nth line in a file. In the following example, the sed command removes the first line in a file.

2. Delete last line or footer line or trailer line

The following sed command is used to remove the footer line in a file. The $ indicates the last line of a file.

3. Delete particular line

This is similar to the first example. The below sed command removes the second line in a file.

4. Delete range of lines

The sed command can be used to delete a range of lines. The syntax is shown below:

Here m and n are min and max line numbers. The sed command removes the lines from m to n in the file. The following sed command deletes the lines ranging from 2 to 4:

5. Delete lines other than the first line or header line

Use the negation (!) operator with d option in sed command. The following sed command removes all the lines except the header line.

6. Delete lines other than last line or footer line

7. Delete lines other than the specified range

Here the sed command removes lines other than 2nd, 3rd and 4th.

8. Delete first and last line

You can specify the list of lines you want to remove in sed command with semicolon as a delimiter.

9. Delete empty lines or blank lines

The ^$ indicates sed command to delete empty lines. However, this sed do not remove the lines that contain spaces.

Sed Command to Delete Lines — Based on Pattern Match

In the following examples, the sed command deletes the lines in file which match the given pattern.

10. Delete lines that begin with specified character

^ is to specify the starting of the line. Above sed command removes all the lines that start with character ‘u’.

11. Delete lines that end with specified character

$ is to indicate the end of the line. The above command deletes all the lines that end with character ‘x’.

12. Delete lines which are in upper case or capital letters

13. Delete lines that contain a pattern

14. Delete lines starting from a pattern till the last line

Here the sed command removes the line that matches the pattern fedora and also deletes all the lines to the end of the file which appear next to this matching line.

15. Delete last line only if it contains the pattern

Here $ indicates the last line. If you want to delete Nth line only if it contains a pattern, then in place of $ place the line number.

Note: In all the above examples, the sed command prints the contents of the file on the unix or linux terminal by removing the lines. However the sed command does not remove the lines from the source file. To Remove the lines from the source file itself, use the -i option with sed command.

If you dont wish to delete the lines from the original source file you can redirect the output of the sed command to another file.

If you like this article, then please share it or click on the google +1 button.

Источник