Linux merge files to one

Linux merge command

On Unix-like operating systems, the merge command performs a three-way file merge.

The merge process analyzes three files: a base version, and two conflicting modified versions. It attempts to automatically combine both sets of modifications, based on the shared base version, into a single merged file. If automatic merge is not possible, it facilitates manual merging.

This page describes the GNU/Linux version of merge.

Description

merge is part of the RCS version control package. It is used to perform a three-way file merge.

merge analyzes three files — an original file, and two modified versions of the original — and compares them, line-by-line, attempting to resolve the differences between the two sets of modifications to create a single, unified file which represents both sets of changes.

Depending on the differences between the two sets of changes, this may be an automatic process or may require user input.

If neither set of changes conflicts with the other, merge can usually figure out what to do on its own. But if the two sets of changes conflict — for example, if the same line of text is worded differently in both modified files — merge shows the conflict in the resulting merged file.

How merging works

merge incorporates all changes that lead from file2 to file3 into file1. The result ordinarily goes into file1.

Suppose file2 is the original, and both file1 and file3 are modifications of file2. Then merge combines both changes.

A conflict occurs if both file1 and file3 have changes in a common segment of lines. If a conflict is found, merge normally outputs a warning and brackets the conflict with » >>>>>>» lines. For instance, a typical conflict looks like this:

If there are conflicts, the user should edit the result and delete one of the alternatives.

Syntax

Options

-A	Output conflicts using the -A style of diff3(1) (if supported by diff3). This merges all changes leading from file2 to file3 into file1, and generates the most verbose output.
-E, -e	These options specify conflict styles that generate less information than -A. See diff3(1) for details. The default is -E. With -e, merge does not warn about conflicts.
-L label	This option may be given up to three times, and specifies labels to be used in place of the corresponding file names in conflict reports. That is, merge -L x -L y -L z a b c generates output that looks like it came from files x, y, and z instead of from files a, b, and c.
-p	Send results to standard output instead of overwriting file1.
-q	Quiet mode. Do not warn about conflicts.
-V	Print RCS’s version number.

Examples

Let’s say we have a file named orig.txt with the following contents.

. and a file named mod1.txt, which is a modified version of orig.txt:

. and a file named mod2.txt, which is also a modified version of orig.txt:

. and we run merge as follows:

It analyzes all three files, write to mod1.txt, and display the following warning:

This means the merge was successful, but we should be aware that there was a conflict. If we open mod1.txt — which by default is the file where the merge is written — we find that it now contains the following text:

It is up to us to decide which «Oranges are. » line to keep (or to combine them in our own way), and make the edit to the file manually.

Related commands

diff — Identify the differences between two files.

Источник

how to merge multiple files into one single file in linux

Many a times you may have multiple files that needs to merged into one single file. It could be that you previously split a single file into multiple files, and want to just merge them back or you have several log files that you want merged into one. Whatever the reason, it is very easy to merge multiple text files into a single file in Linux.

The command in Linux to concatenate or merge multiple files into one file is called cat. The cat command by default will concatenate and print out multiple files to the standard output. You can redirect the standard output to a file using the ‘>‘ operator to save the output to disk or file system.

Another useful utility to merge files is called join that can join lines of two files based on common fields. It can however work only on two files at a time, and I have found it to be quite cumbersome to use. We will cover mostly the cat command in this post.

Merge Multiple files into One in Order

The cat command takes a list of file names as its argument. The order in which the file names are specified in the command line dictates the order in which the files are merged or combined. So, if you have several files named file1.txt, file2.txt, file3.txt etc…

bash$ cat file1.txt file2.txt file3.txt file4.txt > ./mergedfile.txt

The above command will append the contents of file2.txt to the end of file1.txt. The content of file3.txt is appended to the end of merged contents of file1.txt and file2.txt and so on…and the entire merged file is saved with the name mergedfile.txt in the current working directory.

Many a time, you might have an inordinately large number of files which makes it harder to type in all the file names. The cat command accepts regular expressions as input file names, which means you can use them to reduce the number of arguments.

bash$ cat file*.txt my*.txt > mergedfile.txt

This will merge all the files in the current directory that start with the name file and has a txt extension followed by the files that start with my and has a txt extension. You have to be careful about using regular expressions, if you want to preserve the order of files. If you get the regular expression wrong, it will affect the exact order in which the files are merged.

A quick and easy way to make sure the files get merged in the exact order you want, is to use the output of another file listing program such as ls or find and pipe it to the cat command. First execute the find command with the regular expression and verify the file order…

bash$ find . -name «file*.txt» -o -name «my*.txt»

This will print the files in order such that you can verify it to be correct or modify it to match what you want. You can then pipe that output into the cat command.

bash$ find . -name «file*.txt» -o -name «my*.txt» | xargs cat > ./mergedfile.txt

When you merge multiple files into one file using regular expressions to match them, especially when it is piped and where the output file is not very obvious, make sure that the regular expression does not match the filename of the merged file. In the case that it does match, usually the cat command is pretty good at error-ing out with the message “input file is output file”. But it helps to be careful to start with.

Merge Two Files at Arbitrary Location

Sometimes you might want to merge two files, but at a particular location within the content of a file. This is more like the process of inserting contents of one file into an another at a particular position in the file.

If the file sizes are small and manageable, then vi is a great editor tool to do this. Otherwise the option is to split the file first and then merge the resulting files in order. The easiest way is to split the file is based on the line numbers, exactly at where you want to insert the other file.

bash$ split -l 1234 file1.txt

You can split the file into any number of output files depending on your requirement. The above example will split the file file1.txt to chunks of 1234 lines. It is quite possible that you might end up with more than two files, named xaa, xab, xac etc..You can merge all of it back using the same cat command as mentioned earlier.

bash$ cat xaa file2.txt xa

The above command will merge the files in order with the contents of file2.txt in between the contents of xaa and xab.

Another use case is when you need to merge only specific parts of certain files depending on some condition. This is especially useful for me when I have to analyze several large log files, but am only interested in certain messages or lines. So, I will need to extract the important log messages based on some criteria from several log files and save them in a different file while also maintaining or preserving the order of the messages.

Though you can do this using cat and grep commands, you can do it with just the grep command as well.

bash$ grep -h «[Error]» logfile*.log > onlyerrors.log

The above will extract all the lines that match the pattern [Error] and save it to another file. You will have to make sure that the log files are in order when using the regular expression to match them, as mentioned earlier in the post.

Источник

Split and merge files from the command line

Original Article by Juan Valencia

Although some file archivers offer us the option of split the files, this can be easily accomplished with two commands: split and cat .

Splitting a file with split

split just needs the size of the parts that we want to create, and the file that we want to split, e.g.:

split -b 1024 file_to_split.bin

If this file is 6 kibibytes long, it will create 6 files of 1 kibibyte each, named xaa , xab , xac , xad , xae and xaf .

The parameter -b is what defines the size of the resulting parts. You can use suffixes, either the SI suffixes KB (Kilobyte: 1000 bytes), MB (Megabyte: 1000×1000 bytes), GB (Gigabyte: 1000×1000×1000 bytes), TB, PB, EB, ZB, YB. Or you can use the EIC suffixes K (Kibibyte: 1024 bytes), M (Mibibyte: 1024×1024 bytes), G (Gibibyte: 1024×1024×1024 bytes), T, P, E, Z, Y. E.g.:

split -b 1K file_to_split.bin split -b 10M file_to_split.bin split -b 1KB file_to_split.bin split -b 10MB file_to_split.bin

The past examples would create parts of 1,024 bytes, 10,485,760 bytes, 1,000 bytes and 10,000,000 bytes respectively.

If we don’t want the default prefix x, we can change it by adding the new prefix after the name of the file that we want to split, e.g.:

split -b 1024 file_to_split.bin a_part_

If we are splitting a file of 6 kibibytes as in the first example, this would generate the files: a_part_aa , a_part_ab , a_part_ac , a_part_ad , a_part_ae and a_part_af .

We can change the length of the suffix in the resulting files, and we can choose between an alphabet based suffix (the default) or a numeric suffix. How many parts we can create depends of this two features of the suffix. If we keep the default length of 2, and don’t use a numeric suffix, we can split a file in up to 676 parts. If split runs out of suffixes, it will fail, leaving us with the files created until the moment it failed.

To change the length of the suffix use the parameter -a followed by a number. E.g.:

split -b 1024 -a 4 file_to_split.bin

Following the first given example again, we would end with the files: xaaaa , xaaab , xaaac , xaaad , xaaae and xaaaf .

To use a numeric suffix use the parameter -d , of course with a numeric suffix and a length of 2 we can split a file in up to 100 parts. E.g.:

split -b 1024 -d file_to_split.bin

And using the first example one last time, we would end with the files: x00 , x01 , x02 , x03 , x04 and x05 .

Merging the parts that were created with split

Since the files created with the split command are sequential, we can simply use cat to merge this files into a new file, e.g.:

cat x?? > file_ricostruito.bin

The question mark acts as a wild-card character. How many question marks do we use depends of course of the length of the suffixes used when creating the parts.

Split a file per lines

split also allow us to split a text file per lines rather than per the size of the resulting parts. I am sure this was very useful at some point in the past, but I can’t think of a reason to split a text file per lines other than for experimental or didactic purposes, text processors are quite capable of dealing with very large text files (great, just days after writing this I found a good application, splitting long lists of URLs and splitting those long MySQL files full of commands so we can upload them to those web-based systems that can’t handle big files, I left this comment for humorous purposes). Nevertheless, the parameter to split a text file per lines is -l followed by the number of lines. E.g.:

split -l 20 file_to_split.txt

This will create one file for every chunk of 20 lines in the original file, so if the file had 54 lines, it would create the files xaa (lines 1-20), xab (lines 21-40) and xac (lines 41-54).

There is another option in split , a mix between splitting the file per size in bytes and per number of lines. In this mode we specify a maximum size in bytes for the parts, and split will fit as many complete lines as possible in each part without exceeding the specified size in bytes. For this we use the parameter -C followed by a number representing the size in bytes, you can use any of the suffixes that are valid for the option -b . E.g.: Assume that we have a file that contains 20 lines of 100 characters each, totalling 2000 bytes, and we use the following command:

split -C 512 file_to_split.txt

This would give us the files: xaa (lines 1-5, since the sixth line, having 100 characters, would not fit in the 512 bytes that we set as limit), xab (lines 6-10), xac (lines 11-15) and xad (lines 16-20). Each file would have a size of 500 bytes.

Источник

How to merge or combine multiple files

If you would like to merge multiple Word, Excel or text files together, it can be done with relative ease and at no extra cost. Follow the steps below for the type of file you’re trying to merge.

For many users, it’s easier to copy and paste the contents of multiple files into a new file as a form of merging or combining files. See: How to copy and paste text in a document or another program.

Merging Microsoft Word documents

To merge Word documents, you can merge those documents within Microsoft Word itself. To do this, open the first file in Microsoft Word, and follow the steps for your version of Word. The steps are different because of the changes between the file menu and the Office Ribbon.

Microsoft Word 2007 or later (Ribbon)

In the Word Ribbon, click the Insert tab, click the down arrow next to Object, and select the Text from File option, as shown below.

Select the file you want to merge into the current document and click Insert. Once completed, the text and other information from the document will be merged into the current document. These steps can be completed as many times as you want if you want to merge multiple files.

If there are multiple files you want to merge at the same time, you can select multiple files by holding down the Ctrl and selecting each file you want to merge.

Microsoft Word 2003 or earlier (file menu)

In Word, click Tools in the top menu and select the Compare and Merge Documents option, as shown below.

Find the document you want to merge. You have the option of merging the selected document into the currently open document or merging the two documents into a new document. To choose the merge option, click the arrow next to the Merge button and select the desired merge option. Once complete, the files are merged.

If there are multiple files you want to merge at once, you can select multiple files by holding down the Ctrl and selecting each file you want to merge.

Merging Microsoft Excel files

To merge Microsoft Excel files together, it is best to save them as CSV files first. Open the Excel files and in the menu bar, click File, then Save As. In the Save as type drop-down list, select CSV (comma delimited) (*.csv) from the list.

Do this for each Excel file you want to merge, then place all the CSV files in the same folder. For ease, place them in a folder in the root of the C: drive (e.g., c:\csvfiles).

Open the Windows command prompt and navigate to the folder containing the CSV files. Type dir to view the files in the folder and ensure all the files are there.

Type in the following command to merge all CSV files in the folder into a new CSV file titled «newfile.csv» (any name could be used).

After the new file is created, open the new CSV file in Microsoft Excel and save it as an Excel file.

Merge a text (.txt) file in the Windows command line (MS-DOS)

Place each of the text files you want to merge in the same folder. For ease, place them in a folder in the root of the C: drive (e.g., c:\textfiles) and make sure the folder only contains text files you want to merge.

Before merging text files, you may want to make sure there is a blank line or at least one carriage return (pressing Enter ) to help separate each file.

Open the Windows command prompt and navigate to the folder containing the text files. Type dir to view the files in the folder and ensure all the files are there.

How to merge two files into one file

Type the following command to merge two text files into a new file or overwrite an existing file.

The above command would copy (merge) the contents of the file «first.txt» and «second.txt» into the new «third.txt» file.

How to merge all text files into one file

Type in the following command to merge all TXT files in the current directory into the file named newfile.txt (any name could be used).

Now you can open the text file and see everything merged together as one file.

Merge a file in the Linux command line

Linux users can merge two or more files into one file using the merge command or lines of files using the paste command.

Merge PDF files

PDF documents can also be merged. You can use a full version of Adobe Acrobat to do this, but this program is a bit pricy (several hundred dollars).

Another option is to find a free utility on the Internet to merge your PDF files. One of the better free utilities is PDF Split and Merge. It is an online tool that lets you merge two or more PDF files into one PDF file with a few clicks of your mouse button. You can also download and install a version of the Batch PDF Merger program, which costs about $30.

There are other free utilities online that offer this service. However, if there is any confidential information contained in the PDF files, use caution when merging them online. We recommend you use a utility on your computer for these types of PDF file mergers, to ensure the confidential data is kept confidential.

• MergePDF — Online utility to merge PDF files up to 30 MB.
• PDFMerge — Another great free utility to merge PDF documents.
• Sejda — An additional utility to merge PDF files up to 50 MB.

Источник