Merge two folders linux

how to merge two or more directories in linux

Before we merge two or more directories, we need to define what we mean by “merge” in the context of directories. There can be a variety of requirements that could be named as merging of directories. Let’s assume that merging of (two or more) directories means “to create a new directory with files from all of the merged directories“. There can be other requirements associated with this, but we will tackle those as we go along.

Creating a new directory with files from multiple directories itself is not very complicated. The issue itself occurs when there are files in the directories with same or conflicting names. In such cases, we need to figure out what the name and content of the resulting file needs to be.

The merge process is essentially a combination of moving, renaming and deleting files. That means there are several different ways to achieve the required result by using different Linux commands such as find, mv, cp, rm, rsync etc.

Let’s assume that we want to merge two different folders named first/ and second/ …into a new folder named third/. You should be able extrapolate the other cases such as merging first/ and second/ into the same folder second/ etc.

Merging two directories with unique file names

Let say we want to merge two different directories (first/ and second/) each of which contain a set of files and some sub-directories in each of them. They contain files but all of those files have unique file names that do not clash with each other. This is probably the easiest of the cases that you would encounter.

We can use the cp or copy command here, as there are no file name conflicts. Each of the directories and it sub-directories will copy its contents into the new folder.

You can remove the directories using the rm command after you have copied the files to the third/ folder.

Although the cp will work in this situation, I would recommend that you use rsync utility to perform large scale directory syncing. When you don’t know for sure, always assume that there are file collisions. Let’s see the options that rsync provide to merge directories ….

Merging two directories with files with file name collisions

We will now assume the previous use case but with file names that may be the same which will cause file name collisions when merged. So, the simplest thing to do when there is a file name collision is to delete the previous file and overwrite it with the new file.

-a or –archive: The archive option is a shortcut for many options (-rlptgoD), that supports recursion through folders, preserves file metadata etc.

-v or –verbose: verbose option that prints out what is being copied

-z or –compress: performs compression before transferring files. This may not be necessary if the directories on the same machine.

You can choose to overwrite the files only if the file is newer than the previous version. You can also choose to delete the source version, once the files have been successfully copied. The option to do this is -u (or –update) and –remove-source-files.

-u or –update: skips overwriting the file if the existing file has a newer modification time than the incoming file.

–delete-after: performs file deletions on the destination side after the transfer.

–remove-source-files: Remove the source file after the file has been transferred.

You can choose not to overwrite the files as well. This will result in the first file winning and all other files of the same name getting lost. You can use the –ignore-existing command line option to do that.

Источник

merge two folders and keeping the files have same name

I have multiple sources folder(these folders have a lot of files named such as ip address Ex: 192.168.2.1 ), I want to merge them in a target folder.

What is the ways of doing this operation on a Linux using terminal.

Source 1

Source 2

Source 3

Source 4

Source 5

Source 6

Target

original files have no file extension I just named them as what they are but I am opening them in gedit or any text editor. The duplicated file suffix might be (‘192.168.2.3.copy or 192.168.2.3_2 or anything just needs to be different)

What is the way of doing this operation with cp command, shell script or any other command in Linux?

2 Answers 2

Just note that this will not add .copy suffix to any files that are in source2 but not in source1 . That is, .copy will only be added for duplicate file names.

For multiple source folders, you can do something like:

Replace n with the your folder number. This will put a .

for the first copy, .

for the second copy and so on.

The answer by Munir is perfect. My reputation is too low to comment on his solution, but I would like to mention on it why this works:

Normally —backup will backup the file in the target folder, by appending a «

» at the end. The suffix option changes the «

» with copy. So why does this solution change the name of the source file and not the target? It’s the addition of the -f option in the line, which is a radically different behavior than what -f normally does. This is documented in the last paragraph of the man page:

As a special case, cp makes a backup of SOURCE when the force and backup options are given and SOURCE and DEST are the same name for an existing, regular file.

Источник

How to merge two directories in Ubuntu 16.04? [duplicate]

I have two folders /var/first/app and /var/second/app . I have different files within both folders and few are same. I want to merge /var/second/app to /var/first/app . How can I do that?

3 Answers 3

This should do the trick:

Use something like:

or change cp -r to cp -a to preserve ownership and timestamps.

You can also use -i to make sure what is going on. it’s going to prompt you before overwriting anything.

You may first backup your destination folder (just in case) :

If you don’t care overwriting files :

It will copy recursively the second folder into the first one, overwriting files with same names.

If you don’t want to overwrite existing files :

If all is ok, you can remove the backup :

Not the answer you’re looking for? Browse other questions tagged 16.04 command-line files or ask your own question.

Linked

Related

Hot Network Questions

site design / logo © 2021 Stack Exchange Inc; user contributions licensed under cc by-sa. rev 2021.10.8.40416

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.

Источник

How do I merge one directory into another using Bash?

I’m looking for shell script that merge files from one directory into another.

html/a/b.html was replaced by html_new/a/b.html
html/a/b2.html was copied from html_new/a/b2.html
html/index.html was kept untouched

8 Answers 8

All files and directories in source will end up in destination . For example, source/file1 will be copied to destination/file1 .

The -T flag stops source/file1 from being copied to destination/source/file1 instead. (Unfortunately, cp on macOS does not support the -T flag.)

You probably just want cp -R $1/* $2/ — that’s a recursive copy.

(If there might be hidden files (those whose names begin with a dot), you should prefix that command with shopt -s dotglob; to be sure they get matched.)

Take a look at rsync

Rsync got alot of flags to set so look at rsync manpage for details

Just use rsync — it’s a great tool for local file copy and merging in addition to remote copying.

Note that the trailing slash on the source folder is necessary to copy only the contents of source_folder to the destination. If you leave it off, it will copy the source_folder and it’s contents, which is probably not what you are looking for since you want to merge folders.

Even though this question and its accepted answer are ancient, I am adding my answer because the presently existing ones using cp either don’t handle some edge-cases or require working interactively. Often edge-cases/scriptability/portability/multiple-sources don’t matter though, in which case simplicity wins, and it is better to use cp directly with less flags (as in other answers) to reduce cognitive load — but for those other times (or for a robustly reusable function) this invocation/function is useful, and incidentally isn’t bash-specific (I realise this question was about bash though, so that’s just a bonus in this case). Some flags can be abbreviated (e.g. with -a ), but I have included all explicitly in long-form (except for -R , see below) for the sake of explanation. Obviously just remove any flags if there is some feature you specifically don’t want (or you are on a non-posix OS, or your version of cp doesn’t process that flag — I tested this on GNU coreutils 8.25’s cp ):

• -R : has subtly different semantics from -r / —recursive on some systems (particularly with respect to special files in source dirs) as explained in this answer
• —no-dereference : never follow symbolic links in SOURCE
• —preserve=all : preserve the specified attributes (default: mode,ownership,timestamps), if possible additional attributes: context, links, xattr, all
• —force : if an existing destination file cannot be opened, remove it and try again
• —one-file-system : stay on this file system
• —no-target-directory : treat DEST as a normal file (explained in in this answer, namely: If you do a recursive copy and the source is a directory, then cp -T copies the content of the source into the destination, rather than copying the source itself. )
• [piped input from yes ]: even with —force , in this particular recursive mode cp still asks before clobbering each file, so we achieve non-interactiveness by piping output from yes to it
• [piped output to /dev/null ]: this is to silence the messy string of questions along the lines of cp: overwrite ‘xx’?
• [return-val & early exit]: this ensures the loop exits as soon as there is a failed copy, and returns 1 if there was an error

• A funky new flag which I also use with this on my system is —reflink=auto for doing so-called «light copies» (copy-on-write, with the same speed benefits as hard-linking, and the same size benefits until and in inverse proportion to how much the files diverge in the future). This flag is accepted in recent GNU cp , and does more than a no-op with compatible filesystems on recent Linux kernels. YMWV-a-lot on other systems.

Источник

How to copy-merge two directories?

I have two directories images and images2 with this structure in Linux:

. and other 4000 folders

and the other is like:

. and other 4000 folders

Each of these folders contain images and the directories’ names under images and images2 are exactly the same, however their content is different. Then I want to know how I can copy-merge the images of /images2/ad into images/ad, the images of /images2/foo into images/foo and so on with all the 4000 folders..

8 Answers 8

This is a job for rsync. There’s no benefit to doing this manually with a shell loop unless you want to move the file rather than copy them.

(Note trailing slash on images2 , otherwise it would copy to /images/images2 .)

If images with the same name exist in both directories, the command above will overwrite /images/SOMEPATH/SOMEFILE with /images2/SOMEPATH/SOMEFILE . If you want to replace only older files, add the option -u . If you want to always keep the version in /images , add the option —ignore-existing .

If you want to move the files from /images2 , with rsync, you can pass the option —remove-source-files . Then rsync copies all the files in turn, and removes each file when it’s done. This is a lot slower than moving if the source and destination directories are on the same filesystem.

The best choice, as already posted, is of course rsync . Nevertheless also unison would be a great piece of software to do this job, though typically requires a package install. Both can be used in several operating systems.

Rsync

rsync synchronizes in one direction from source to destination. Therefore the following statement

syncs everything from Source to Destination. The merged folder resides in Destination.

-a means «archive» and copies everything recursively from source to destination preserving nearly everything.

-v gives more output («verbose»).

-h for human readable.

—progress to show how much work is done.

If you want only update the destination folder with newer files from source folder:

Unison

unison synchronizes in both directions. Therefore the following statement

syncs both directories in both directions and finally source equals destination. It’s like doing rsync twice from source to dest and vice versa.

For more advanced usages look at the man pages or the following websites:

Loop over all the contents of images2 using an expanded glob (to avoid the problems with parsing ls ) then mv the contents of those items to the matching entry in images . Uses basename to strip the leading images2 from the globbed path.

There are faster and much more space-efficient ways of merging two directories using the —link option to cp if the directories are on the same file system, described in the multiple varied answers in a related article here: (The title of the article doesn’t exactly match the user’s question, and the answers address the title topic, merging, more than they address the user’s actual question.)

The —link option to cp means no file data is copied. An example of this, where everything in /images2 replaces any older items in /images is:

cp —force —archive —update —link /images2/. /images

After the merge into /images , you can then rm -rf /images2

This solution will fail if anywhere in the file tree the merge tries to merge a directory onto an existing file or symlink with the same name, i.e. it won’t merge a directory named /images2/x onto an existing file or symlink with the same name /images/x and if you get such an error you can manually delete the file or symlink and just re-run the command.

The nice thing about —link is that no data is moved to merge the directories.

Источник