Linux copy many files to one

How do I copy multiple files by wildcard?

I have a folder with a number of files in it ABC.* (there are roughly 100 such files). I want to duplicate them all to new files with names starting with DEF.*

What is the simplest way to do this with a batch command (in BASH or similar)? I am thinking something involving sed or awk or xargs, but I’m having difficulty figuring out the syntax. I could write a Python script, but I’m thinking there is probably a command line solution that is not too complicated.

6 Answers 6

How about something like this in bash:

you can test it by putting echo in front of the cp command:

To do this efficiently with a large number of files, it is better to avoid a starting a different cp process for each one. One way would be to copy then rename them using prename ( rename is symlinked to this by default on Debian based distros). Using this and the Linux mktemp :

Update

Actually pax may be a better way to go here:

Unfortunately, despite pax being both POSIX and Linux Standard Base, few distros currently include it by default.

tar will do this for you really fast.

First I created 2 directories and 10 files:

Then I copied them:

TRANSFORM

So GNU tar will accept a sed —transform=EXPRESSION for file renaming. This can even rename only some of the files. For instance:

So that’s one advantage.

STREAM

Also consider that this is only two tar processes — and that will not alter regardless of your file count.

tar is as optimized as you could want it to be. This will never have problem argument counts or runaway child processes. This is just A > B done.

ARGUMENTS

I use 7 distinct arguments combined between my two tar processes here. The most important one is listed here first:

— stdout/stdin — this informs tar that it will be streaming either its input or output to or from stdin/stdout which it will interpret correctly depending on whether or not it is building or extracting an archive.

-c create — this tells tar to build the archive. The next argument tar expects is.

-f file — we specify that tar will be working with a file object rather than a tape-device or whatever. And the file it will be working with, as noted above, is stdin/stdout — in other words, our |pipe .

./* all $PWD/files — not too much to explain here except that the archive argument comes first, so — then ./* .

. and on the other side of the |pipe .

-C change directory — this informs tar that it needs to change to the directory I specify before performing any other action, so effectively it just cd ../test2 before extraction.

—transform=’s/ed/EXPR/’ — as has already been mentioned, this did the renaming. But the docs indicate that it can take any sed expression or //flag .

-x extract — after tar changes to our target directory and receives our renaming instructions we instruct it to begin extracting all of the files into its current directory from the -f — |pipe archive file. No mystery.

Источник

Copying multiple specific files from one folder to another

I have a large folder of pictures (thousands), and I have a long list of files, by exact file name, that I need to copy to another folder. I want to know if there is a way I can select several specific files from this folder, by name, and copy them to another folder, using the terminal, without copying them individually?

7 Answers 7

Simply copy multiple files at once from command line

There are several ways you could achieve this. The easiest I have seen is to use the following.

The syntax uses the cp command followed by the path to the directory the desired files are located in with all the files you wish to copy wrapped in brackets and separated by commas.

Make sure to note that there are no spaces between the files. The last part of the command, /home/usr/destination/ , is the directory you wish to copy the files into.

or if the all the files have the same prefix but different endings you could do something like this:

Where file1,file2,file3 and file4 would be copied.

From how you worded the question I believe this is what you’re looking for but it also sounds like you might be looking for a command to read from a list of files and copy all of them to a certain directory. If that is the case let me know and i’ll edit my answer.

Dealing with duplicates with python

So I wrote a little python script that I believe should get the job done. However, I am not sure how well versed you are in python (if versed at all) so I will try explaining how to use this script the best I can and please ask as many questions about it as you need.

This script should be relatively simple to use. First off, copy the above code into the program gedit (should be pre-installed in Ubuntu) or any other text editor.

After that is complete, save the file as move.py in your home directory (it can be any directory but for ease of instruction lets just use the home directory) or add the directory the file is contained in to your PATH. Then cd to your home directory (or whatever directory you saved move.py in) from the terminal and type the following command:

This should copy all of the files that are listed from the source directory to the destination directory with duplicates taking the format pic(1).jpg, pic(2).jpg and so on. file.txt should be a file that lists all the pictures you would like to copy with each entry on its own separate line.

In no way should this script effect the source directory, however just make sure to enter the correct paths to the source and destination directory and the worst that could happen is you copy the files to the wrong directory.

Источник

How to Copy Files and Directories in Linux

Home » SysAdmin » How to Copy Files and Directories in Linux

This guide will show you how to copy files and directories in Linux by executing commands from the command line. Furthermore, the commands listed below detail how to create system-wide backups or filter out and copy only specific files.

Note: These Linux commands can only be run from a terminal window. If your version of Linux boots to a desktop graphical interface, launch a terminal window by pressing CTRL-ALT-F2 or CTRL-ALT-T.

Using the cp Command to Copy Files and Directories in Linux

The cp command is the primary method for copying files and directories in Linux. Virtually all Linux distributions can use cp . The basic format of the command is:

This Linux command creates a copy of the my_file.txt file and renames the new file to my_file2.txt.

By default, the cp command runs in the same directory you are working in. However, the same file cannot exist twice in the same directory. You’ll need to change the name of the target file to copy in the same location. Some users will add _old, some will add a number, and some will even change the three-letter extension (e.g., .bak instead of .txt).

You may not get a warning before Linux overwrites your file – be careful, or see below for the –i option.

Additional Options

Additional options can be used in combination with the cp command:

• –v verbose: shows the progress of multiple copied files
• –ppreserve: keeps the same attributes, like creation date and file permissions
• –f force: force the copy by deleting an existing file first
• –i interactive: prompts for confirmation, highly advised
• –Rrecursive: copies all files and subfolders in a directory
• –u update: copy only if source is newer than destination

Note: The -p (preserve) option forces the system to preserve the following source file attributes: modification time, access time, user ID (UID), group ID (GID), file flags, file mode, access control lists (ACLs), and extended attributes (EAs).

How to Copy File to Another Directory in Linux

To copy a file from the directory you’re working in to a different location, use the command:

You don’t need to rename the file unless there’s already one with the same name in the target directory.

To specify a path for the source file:

This lets you copy without having to change directories. The cp command will create the /new_directory if it doesn’t exist.

To rename and copy a file to a different path:

This option is useful for creating backups of configuration files, or for copying data to a storage device.

Note: Learn how to move directories in Linux.

Copy Multiple Files from One Directory to Another in Linux

You may need to copy more than one file at a time.

List each file to be copied before the target directory:

This example created a copy of all three files in the /new_directory folder.

Use a wildcard to specify all files that share a string of characters:

This would find all the files with the .jpg extension in the /pictures directory, and copy them into the /new_directory folder.

To copy an entire folder and its subfolders and files, use the –R option:

–R stands for recursive, which means “everything in that location.” This would copy all the files, as well as all the directories, to the /new_directory folder.

Copy Using rsync Command

The rsync command in Linux is used to synchronize or transfer data between two locations. Usage is similar to cp , but there are a few key differences to note.

To copy a single file, enter the following into a terminal:

• The –a option means all, and is included with rsync commands – this preserves subdirectories, symbolic links, and other metadata.
• Replace the my_file.txt file in the working directory.
• Replace /new_directory/ with the destination.
• Using my_file_backup.txt as the target indicates the file will be renamed during the copy.

To copy a directory with rsync, enter the following:

This copies the contents of the /etc/docker/ directory to /home/backup/docker/. Make sure to keep the slashes. Omitting the slash on the source directory will copy the contents into a subdirectory.

To omit files from being copied, check out our guide on how to exclude files and directories in data transfer using rsync command.

Other Options

The ls command is a handy partner to the cp command in Linux.

To list the contents of a directory enter the command:

The example above displays all the files in /directory. Use this command after copying to verify the files were copied successfully.

To change directories, use cd and the name of the directory. For example:

The command prompt will change to display that you’ve changed directories.

Now you understand how to copy files in Linux. The cp command is a versatile and powerful tool for managing and backing up files.

Источник

How to append contents of multiple files into one file

I want to copy the contents of five files to one file as is. I tried doing it using cp for each file. But that overwrites the contents copied from the previous file. I also tried

and it did not work.

I want my script to add the newline at the end of each text file.

eg. Files 1.txt, 2.txt, 3.txt. Put contents of 1,2,3 in 0.txt

11 Answers 11

You need the cat (short for concatenate) command, with shell redirection ( > ) into your output file

Another option, for those of you who still stumble upon this post like I did, is to use find -exec :

In my case, I needed a more robust option that would look through multiple subdirectories so I chose to use find . Breaking it down:

Look within the current working directory.

Only interested in files, not directories, etc.

Whittle down the result set by name

Execute the cat command for each result. «+» means only 1 instance of cat is spawned (thx @gniourf_gniourf)

As explained in other answers, append the cat-ed contents to the end of an output file.

if you have a certain output type then do something like this

If all your files are in single directory you can simply do

Files 1.txt,2.txt, .. will go into 0.txt

If all your files are named similarly you could simply do:

I found this page because I needed to join 952 files together into one. I found this to work much better if you have many files. This will do a loop for however many numbers you need and cat each one using >> to append onto the end of 0.txt.

as brought up in the comments:

Another option is sed :

Or without redirection .

Note that last line write also merge.txt (not wmerge.txt !). You can use w»merge.txt» to avoid confusion with the file name, and -n for silent output.

Of course, you can also shorten the file list with wildcards. For instance, in case of numbered files as in the above examples, you can specify the range with braces in this way:

if your files contain headers and you want remove them in the output file, you can use:

If the original file contains non-printable characters, they will be lost when using the cat command. Using ‘cat -v’, the non-printables will be converted to visible character strings, but the output file would still not contain the actual non-printables characters in the original file. With a small number of files, an alternative might be to open the first file in an editor (e.g. vim) that handles non-printing characters. Then maneuver to the bottom of the file and enter «:r second_file_name». That will pull in the second file, including non-printing characters. The same could be done for additional files. When all files have been read in, enter «:w». The end result is that the first file will now contain what it did originally, plus the content of the files that were read in.

All of the (text-) files into one

xargs makes the output-lines of find . the arguments of cat.

find has many options, like -name ‘*.txt’ or -type.

you should check them out if you want to use it in your pipeline

If you want to append contents of 3 files into one file, then the following command will be a good choice:

It will combine the contents of all files into file4, throwing console output to /dev/null .

Not the answer you’re looking for? Browse other questions tagged linux bash unix or ask your own question.

Linked

Related

Hot Network Questions

Subscribe to RSS

To subscribe to this RSS feed, copy and paste this URL into your RSS reader.

site design / logo © 2021 Stack Exchange Inc; user contributions licensed under cc by-sa. rev 2021.10.8.40416

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.

Источник