SEARCH

Enter your search query in the box above ^, or use the forum search tool.

You are not logged in.

#1 2011-10-07 13:14:57

xaos52
The Good Doctor
From: Planet of the @s
Registered: 2011-06-24
Posts: 4,602

Find the largest files on your system from the command line.

I have been using this perl script for years.
Found it on the internet.
Can not remember where though:

#!/usr/bin/perl
@queue = @ARGV;
while ($current = shift (@queue) ) {
       opendir DH,$current;
       @content = readdir DH;
       foreach $item (@content) {
               next if ($item =~ /^\.{1,2}$/);
               $fullname = "$current/$item";
               if (-l $fullname) { next; }
               if (-d $fullname) {
                       push @queue,$fullname;
                       next;
                       }
               if (-s $fullname > 100000) {
                       $big{$fullname} = -s $fullname;
                       }
               }
       }
foreach $item (sort {$big{$b}<=>$big{$a}} keys %big) {
       printf ("%8d %s\n",$big{$item}, $item);
       }

Save this script in a directory on your PATH (e.g. find-biggest-files.pl )and make it executable.
There is a hard limit in there of 100000. Smaller files are ignored.
To find the biggest files in your root dir:

find-biggest-files.pl /

PS:
1. it will search all mounted partions. If you dont want that, umount them first.
2. you can specify multiple directories to be searched as arguments on the command line.

hth

Offline

Be excellent to each other!

#2 2011-10-07 13:48:13

luc
#! Die Hard
From: Munich, Germany
Registered: 2010-03-21
Posts: 597

Re: Find the largest files on your system from the command line.

on my machine this runs much faster:

find -x some/dir/foo ! -type d -exec du -d 0 {} + | sort -n

or if you do not care to skip the directories:

du -xa some/dir/foo | sort -n

Note: the -x option will case both find and du to stay on the partition so you do not have to unmount something first..

Offline

#3 2011-10-07 14:01:40

ErSandro
#! CrunchBanger
Registered: 2009-09-19
Posts: 108

Re: Find the largest files on your system from the command line.

Very cool.

Try ncdu too, it's in the repositories.

Offline

#4 2011-10-07 14:18:58

Awebb
The Singularity
Registered: 2009-07-23
Posts: 2,812

Re: Find the largest files on your system from the command line.

@luc: What du implementation is this? I'm @work, sitting in front of a recent Ubuntu box, seing no -d switch in the manpage. Doh!


I'm so meta, even this acronym

Offline

#5 2011-10-07 15:41:40

luc
#! Die Hard
From: Munich, Germany
Registered: 2010-03-21
Posts: 597

Re: Find the largest files on your system from the command line.

@awebb: thats bsd du. you have to use "--max-depth=0" or "--summarize". Sorry didn't think about that wink

Offline

#6 2011-10-07 16:01:38

johnraff
nullglob
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 4,148
Website

Re: Find the largest files on your system from the command line.

^me neither on Squeeze. Could '-d 0' be an alias for  '--max-depth?=0' ?

edit oops, sorry Luc, you got in with the answer while I was posting that.

Last edited by johnraff (2011-10-08 04:53:22)


John
--------------------
( a boring Japan blog , Japan Links, idle twitterings  and GitStuff )
#! forum moderator    BunsenLabs

Offline

#7 2011-10-07 16:46:50

xaos52
The Good Doctor
From: Planet of the @s
Registered: 2011-06-24
Posts: 4,602

Re: Find the largest files on your system from the command line.

OK,
replaced perl script by

du -xa some/dir/foo | sort -n

thanks luc

Offline

#8 2011-10-07 18:00:16

luc
#! Die Hard
From: Munich, Germany
Registered: 2010-03-21
Posts: 597

Re: Find the largest files on your system from the command line.

I apologize for the confusion about the -d option. I just retested my code and I think it is not needed at all, as directories are not picked up by find due to "! -type d". But you where right john. It is the BSD version of the GNU option "--max-depth".
The new code is now:

 find -x some/dir/foo ! -type d -exec du {} + | sort -n

Offline

#9 2011-10-07 18:29:35

xaos52
The Good Doctor
From: Planet of the @s
Registered: 2011-06-24
Posts: 4,602

Re: Find the largest files on your system from the command line.

On debian:

find some/dir/foo -xdev ! -type d -exec du {} + | sort -n

so I propose:

find some/dir/foo -xdev ! -type d -exec du {} + | sort -nr | head

for top 10 biggest files.

Offline

#10 2011-10-08 05:03:48

johnraff
nullglob
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 4,148
Website

Re: Find the largest files on your system from the command line.

Maybe add du's -h option for human-readable sizes?

find some/dir/foo -xdev ! -type d -exec du -h {} + | sort -nr | head

John
--------------------
( a boring Japan blog , Japan Links, idle twitterings  and GitStuff )
#! forum moderator    BunsenLabs

Offline

#11 2011-10-08 06:51:23

luc
#! Die Hard
From: Munich, Germany
Registered: 2010-03-21
Posts: 597

Re: Find the largest files on your system from the command line.

but can "sort" sort that in the right way? I get

...
9K  file0
10M  file1
10K  file2
...

if I use "sort -n"

Offline

#12 2011-10-11 17:39:20

johnraff
nullglob
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 4,148
Website

Re: Find the largest files on your system from the command line.

^good point. I can't think of any way round that.


John
--------------------
( a boring Japan blog , Japan Links, idle twitterings  and GitStuff )
#! forum moderator    BunsenLabs

Offline

#13 2011-10-11 17:43:08

Awebb
The Singularity
Registered: 2009-07-23
Posts: 2,812

Re: Find the largest files on your system from the command line.

AWK! That's not only the sound the creature on your userpic makes ;-)


I'm so meta, even this acronym

Offline

#14 2011-10-12 05:01:24

johnraff
nullglob
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 4,148
Website

Re: Find the largest files on your system from the command line.

^...true... any suggestions as to how to get 2M to display above 10k with awk?

...or maybe better to sort first, then convert 20000 to 20K etc?


John
--------------------
( a boring Japan blog , Japan Links, idle twitterings  and GitStuff )
#! forum moderator    BunsenLabs

Offline

#15 2011-10-12 07:40:18

Awebb
The Singularity
Registered: 2009-07-23
Posts: 2,812

Re: Find the largest files on your system from the command line.

Yes. Sort first, then convert the numbers. I must admit I'd write an ugly chain of sed+regex, because my AWK-fu is still in the sign of a very low Kyū...


I'm so meta, even this acronym

Offline

#16 2011-10-12 09:03:00

aiBo
#! CrunchBanger
Registered: 2010-11-08
Posts: 243

Re: Find the largest files on your system from the command line.

luc wrote:

but can "sort" sort that in the right way? I get

...
9K  file0
10M  file1
10K  file2
...

if I use "sort -n"

This can be achieved with

$ sort -h
sort man page wrote:

-h, --human-numeric-sort
compare human readable numbers (e.g., 2K 1G)

Offline

#17 2011-10-12 10:07:47

Awebb
The Singularity
Registered: 2009-07-23
Posts: 2,812

Re: Find the largest files on your system from the command line.

Haha, sometimes a man page is a man page is a man page.


I'm so meta, even this acronym

Offline

#18 2011-10-12 11:09:31

johnraff
nullglob
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 4,148
Website

Re: Find the largest files on your system from the command line.

aiBo wrote:

This can be achieved with

$ sort -h

WOW  vYTUxZQ


John
--------------------
( a boring Japan blog , Japan Links, idle twitterings  and GitStuff )
#! forum moderator    BunsenLabs

Offline

#19 2011-10-12 18:49:29

luc
#! Die Hard
From: Munich, Germany
Registered: 2010-03-21
Posts: 597

Re: Find the largest files on your system from the command line.

blame on me sad sorry guys for making a fuss after not rtfm.

Offline

#20 2011-10-13 16:56:03

johnraff
nullglob
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 4,148
Website

Re: Find the largest files on your system from the command line.

^we're all equally guilty. neutral


John
--------------------
( a boring Japan blog , Japan Links, idle twitterings  and GitStuff )
#! forum moderator    BunsenLabs

Offline

Board footer

Powered by FluxBB

Copyright © 2012 CrunchBang Linux.
Proudly powered by Debian. Hosted by Linode.
Debian is a registered trademark of Software in the Public Interest, Inc.
Server: acrobat

Debian Logo