[LUAU] How do you delete over 500,000 files in a directory

[Jim Thompson]

Tim Newsham wrote:

> You'd think that by 2005, arbitrary length command lines wouldn't
> be an issue (assuming they could all fit in memory, or kmem, or some
> suitable place).  I wonder if anyone has made a more up-to-date
> unix like system that fixes these warbles..

The UNIX operating system traditionally has a fixed limit for the amount 
of memory that can be used for a program environment and argument list 
combined. You can use getconf to return that limit. On my Linux system 
(2.4.26) that amount is 128k. On my FreeBSD system (6.0) that amount is 
256k. It can vary per operating system. POSIX only requires 20k which 
was the traditional value used for probably 20 years. Newer operating 
system releases usually increase that somewhat.

jim at he-colo:~$ uname -a
Linux he-colo.netgate.com 2.4.26 #3 SMP Thu Jul 15 20:01:46 PDT 2004 
i686 GNU/Linux
jim at he-colo:~$ getconf ARG_MAX
131072
jim at he-colo:~$ grep ARG_MAX /usr/src/linux/include/linux/limits.h
#define ARG_MAX       131072    /* # bytes of args + environ for exec() */
jim at he-colo:~$
   

%uname -a
FreeBSD shuttle.netgate.com 6.0-RC1 FreeBSD 6.0-RC1 #1: Fri Oct 28 
23:46:37 HST 2005     
root at shuttle.netgate.com:/usr/obj/usr/src/sys/GENERIC+ath  i386
%getconf ARG_MAX
262144
%grep ARG_MAX /usr/include/sys/syslimits.h
#define ARG_MAX                 65536   /* max bytes for an exec function */
#define ARG_MAX                 262144  /* max bytes for an exec function */
%

(the first line on the freebsd definition is for alpha and powerpc, the 
second is "everything else".)

Some may find the reason for the first of two increases (40960, 65536) 
on FreeBSD intesting:

    "Increase ARG_MAX so that `make clean' in src/lib/libc works again.
    (Adding YP pushed it over the limit.)"

http://www.FreeBSD.org/cgi/cvsweb.cgi/src/sys/sys/syslimits.h

Other systems have their own reasons for their various limits.

Note that the expansion itself is no problem, rather it's almost always 
an exec(2) system call which fails returning E2BIG.
Remember that all that crap has to be copied into kernel space for an 
exec (in this instance, so '/bin/rm' can be run with the name(s) of the 
files to be deleted), and rather than have an "unlimited" array to be 
checked (to make sure it doesn't exceed the limits of the current 
process) and copied (always slower than you'd like), Unix (and linux) 
just tell you to find a slightly different way.

Thus the solutions using the shell built-in "echo" (and a pipeline or an 
iteration) work very well, though my preference is the find | xargs 
style solutions that others presented.

> Oh yah, plan9. ;-)

Woof.

If you want to use something with some architecture, go get a lisp machine.

jim