Linux count all files recursively

Count Number of Files in a Directory in Linux

I presume you are aware of the wc command for counting number of lines. We can use the same wc command with ls command to count the number of files in a directory.

This task seems simple but could soon turn slightly complex based on your need and definition of counting files. Before I confuse you further, let’s see about various use cases of counting the number of files in Linux.

Count number of files in directory in Linux

Let me first show you the content of the test directory I am going to use in this tutorial:

You can see that it has 9 files (including one hidden file) and 2 sub-directories in that directory. But you don’t have to do it manually. Let’s count the number of files using Linux commands.

Count number of files and directories (without hidden files)

You can simply run the combination of the ls and wc command and it will display the number of files:

This is the output:

There is a problem with this command. It counts all the files and directories in the current directories. But it doesn’t see the hidden files (the files that have name starting with a dot).

This is the reason why the above command showed me a count of 10 files instead of 11 (9 files and 2 directories).

Count number of files and directories including hidden files

You probably already know that -a option of ls command shows the hidden files. But if you use the ls -a command, it also displays the . (present directory) and .. (parent directory). This is why you need to use -A option that displays the hidden files excluding . and .. directories.

This will give you the correct count of files and directories in the current directory. Have a look at the output that shows a count of 11 (9 files and 2 directories):

You can also use this command to achieve the same result:

Note that it the option used is 1 (one) not l (L). Using the l (L) option displays an additional line at the beginning of the output (see ‘total 64’ in the directory output at the beginning of the article). Using 1 (one) lists one content per line excluding the additional line. This gives a more accurate result.

Count number of files and directories including the subdirectories

What you have see so far is the count of files and directories in the current directory only. It doesn’t take into account the files in the subdirectories.

If you want to count the number of files and directories in all the subdirectories, you can use the tree command.

This command shows the directory structure and then displays the summary at the bottom of the output.

As you can see in the output, it shows that there are 7 directories and 20 files in total. The good thing about this result is that it doesn’t count directories in the count of files.

Count only the files, not directories

So far, all the solutions we have seen for counting the number of files, also take directories into account. Directories are essentially files but what if you want to count only the number of files, not directories? You can use the wonderful find command.

You can run this command:

The above command searched for all the files (type f) in current directory and its subdirectories.

Count only the files, not directories and only in current directory, not subdirectories

That’s cool! But what if you want to count the number of files in the current directory only excluding the files in the subdirectories? You can use the same command as above but with a slight difference.

All you have to do is to add the ‘depth’ of your find. If you set it at 1, it won’t enter the subdirectories.

Here’s the output now:

In the end…

In Linux, you can have multiple ways to achieve the same goal. I am pretty sure there can be several other methods to count the number of files in Linux. If you use some other command, why not share it with us?

I hope this Linux tutorial helped you learn a few things. Stay in touch for more Linux tips.

Источник

Recursively count all the files in a directory [duplicate]

I have a really deep directory tree on my Linux box. I would like to count all of the files in that path, including all of the subdirectories.

For instance, given this directory tree:

If I pass in /home , I would like for it to return four files. Or, bonus points if it returns four files and two directories. Basically, I want the equivalent of right-clicking a folder on Windows and selecting properties and seeing how many files/folders are contained in that folder.

How can I most easily do this? I have a solution involving a Python script I wrote, but why isn’t this as easy as running ls | wc or similar?

5 Answers 5

find . -type f | wc -l

Explanation:
find . -type f finds all files ( -type f ) in this ( . ) directory and in all sub directories, the filenames are then printed to standard out one per line.

This is then piped | into wc (word count) the -l option tells wc to only count lines of its input.

Together they count all your files.

The answers above already answer the question, but I’ll add that if you use find without arguments (except for the folder where you want the search to happen) as in:

the search goes much faster, almost instantaneous, or at least it does for me. This is because the type clause has to run a stat() system call on each name to check its type — omitting it avoids doing so.

This has the difference of returning the count of files plus folders instead of only files, but at least for me it’s enough since I mostly use this to find which folders have huge ammounts of files that take forever to copy and compress them. Counting folders still allows me to find the folders with most files, I need more speed than precision.

Источник

How do I count all the files recursively through directories

I want to see how many files are in subdirectories to find out where all the inode usage is on the system. Kind of like I would do this for space usage

which will give me the space used in the directories off of root, but in this case I want the number of files, not the size.

10 Answers 10

Thanks to Gilles and xenoterracide for safety/compatibility fixes.

The first part: find . -maxdepth 1 -type d will return a list of all directories in the current working directory. (Warning: -maxdepth is a GNU extension and might not be present in non-GNU versions of find .) This is piped to.

The second part: while read -r dir; do (shown above as while read -r dir (newline) do ) begins a while loop – as long as the pipe coming into the while is open (which is until the entire list of directories is sent), the read command will place the next line into the variable dir . Then it continues.

The third part: printf «%s:\t» «$dir» will print the string in $dir (which is holding one of the directory names) followed by a colon and a tab (but not a newline).

The fourth part: find «$dir» -type f makes a list of all the files inside the directory whose name is held in $dir . This list is sent to.

The fifth part: wc -l counts the number of lines that are sent into its standard input.

The final part: done simply ends the while loop.

So we get a list of all the directories in the current directory. For each of those directories, we generate a list of all the files in it so that we can count them all using wc -l . The result will look like:

Источник

How to count number of files in each directory?

I am able to list all the directories by

I attempted to list the contents of each directory and count the number of files in each directory by using the following command

But this summed the total number of lines returned by

Is there a way I can count the number of files in each directory?

18 Answers 18

This prints the file count per directory for the current directory level:

Assuming you have GNU find, let it find the directories and let bash do the rest:

• find . -type f to find all items of the type file , in current folder and subfolders
• cut -d/ -f2 to cut out their specific folder
• sort to sort the list of foldernames
• uniq -c to return the number of times each foldername has been counted

You could arrange to find all the files, remove the file names, leaving you a line containing just the directory name for each file, and then count the number of times each directory appears:

The only gotcha in this is if you have any file names or directory names containing a newline character, which is fairly unlikely. If you really have to worry about newlines in file names or directory names, I suggest you find them, and fix them so they don’t contain newlines (and quietly persuade the guilty party of the error of their ways).

If you’re interested in the count of the files in each sub-directory of the current directory, counting any files in any sub-directories along with the files in the immediate sub-directory, then I’d adapt the sed command to print only the top-level directory:

The first pattern captures the start of the name, the dot, the slash, the name up to the next slash and the slash, and replaces the line with just the first part, so:

The second replace captures the files directly in the current directory; they don’t have a slash at the end, and those are replace by ./ . The sort and count then works on just the number of names.

Источник

Fast Linux file count for a large number of files

I’m trying to figure out the best way to find the number of files in a particular directory when there are a very large number of files (more than 100,000).

When there are that many files, performing ls | wc -l takes quite a long time to execute. I believe this is because it’s returning the names of all the files. I’m trying to take up as little of the disk I/O as possible.

I have experimented with some shell and Perl scripts to no avail. How can I do it?

17 Answers 17

By default ls sorts the names, which can take a while if there are a lot of them. Also there will be no output until all of the names are read and sorted. Use the ls -f option to turn off sorting.

Note: This will also enable -a , so . , .. , and other files starting with . will be counted.

The fastest way is a purpose-built program, like this:

From my testing without regard to cache, I ran each of these about 50 times each against the same directory, over and over, to avoid cache-based data skew, and I got roughly the following performance numbers (in real clock time):

That last one, dircnt , is the program compiled from the above source.

EDIT 2016-09-26

Due to popular demand, I’ve re-written this program to be recursive, so it will drop into subdirectories and continue to count files and directories separately.

Since it’s clear some folks want to know how to do all this, I have a lot of comments in the code to try to make it obvious what’s going on. I wrote this and tested it on 64-bit Linux, but it should work on any POSIX-compliant system, including Microsoft Windows. Bug reports are welcome; I’m happy to update this if you can’t get it working on your AIX or OS/400 or whatever.

As you can see, it’s much more complicated than the original and necessarily so: at least one function must exist to be called recursively unless you want the code to become very complex (e.g. managing a subdirectory stack and processing that in a single loop). Since we have to check file types, differences between different OSs, standard libraries, etc. come into play, so I have written a program that tries to be usable on any system where it will compile.

There is very little error checking, and the count function itself doesn’t really report errors. The only calls that can really fail are opendir and stat (if you aren’t lucky and have a system where dirent contains the file type already). I’m not paranoid about checking the total length of the subdir pathnames, but theoretically, the system shouldn’t allow any path name that is longer than than PATH_MAX . If there are concerns, I can fix that, but it’s just more code that needs to be explained to someone learning to write C. This program is intended to be an example of how to dive into subdirectories recursively.

EDIT 2017-01-17

I’ve incorporated two changes suggested by @FlyingCodeMonkey:

1. Use lstat instead of stat . This will change the behavior of the program if you have symlinked directories in the directory you are scanning. The previous behavior was that the (linked) subdirectory would have its file count added to the overall count; the new behavior is that the linked directory will count as a single file, and its contents will not be counted.
2. If the path of a file is too long, an error message will be emitted and the program will halt.

EDIT 2017-06-29

With any luck, this will be the last edit of this answer 🙂

I’ve copied this code into a GitHub repository to make it a bit easier to get the code (instead of copy/paste, you can just download the source), plus it makes it easier for anyone to suggest a modification by submitting a pull-request from GitHub.

The source is available under Apache License 2.0. Patches * welcome!

• «patch» is what old people like me call a «pull request».

Источник