Xargs: Difference between revisions

Browse history interactively ← Previous edit Next edit →Content deleted Content addedVisual WikitextInline

Revision as of 14:31, 19 December 2011 edit130.225.125.174 (talk) →The separator problem ← Previous edit		Revision as of 22:55, 7 February 2012 edit undoTenPoundHammer (talk \| contribs)Autopatrolled, Extended confirmed users, Page movers, Pending changes reviewers, Rollbackers278,939 edits how-toNext edit →
Line 1:		Line 1:
	{{multiple issues\|cleanup=September 2011\|lead too long=September 2011}}
	{{NOT\|date=September 2011}}
	{{lowercase\|title=xargs}}		{{lowercase\|title=xargs}}
	'''xargs''' is a command on ] and most ] operating systems used to build and execute command lines from ]. Under the ] before version 2.6.23, arbitrarily long lists of parameters could not be passed to a command,<ref></ref> so xargs breaks the list of arguments into sublists small enough to be acceptable.		'''xargs''' is a command on ] and most ] operating systems used to build and execute command lines from ]. Under the ] before version 2.6.23, arbitrarily long lists of parameters could not be passed to a command,<ref></ref> so xargs breaks the list of arguments into sublists small enough to be acceptable.

	For example, commands like:

	<source lang="bash">
	rm /path/*
	</source>
	or
	<source lang="bash">
	rm `find /path -type f`
	</source>
	will fail with an error message of "Argument list too long" if there are too many files in <code>/path</code>.

	However the version below (functionally equivalent to <code>rm `find /path -type f`</code>) will not fail:
	<source lang="bash">
	find /path -type f -print0 \| xargs -0 rm
	</source>
	In the above example, the ] feeds the input of <code>xargs</code> with a long list of file names. <code>xargs</code> then splits this list into sublists and calls <code>rm</code> once for every sublist.

	The previous example is more efficient than this functionally equivalent version which calls <code>rm</code> once ''for every'' single file:
	<source lang="bash">
	find /path -type f -exec rm '{}' \;
	</source>

	Note however that with modern versions of <code>find</code>, the following variant does the same thing as the <code>xargs</code> version:
	<source lang="bash">
	find /path -type f -exec rm '{}' +
	</source>

	'''xargs''' often covers the same functionality as the ] (`) feature of many ], but is more flexible and often also safer, especially if there are blanks or special characters in the input. It is a good companion for commands that output long lists of files like ], ] and ], but only if you use -0, since xargs without -0 deals badly with file names containing ', " and space. ] is the perfect companion to ], ] and ] if file names may contain ', " and space (newline still requires -0).

	==Examples==

	<source lang="bash">
	find . -name "*.foo" \| xargs grep bar
	</source>

	The above is equivalent to:

	<source lang="bash">
	grep bar `find . -name "*.foo"`
	</source>

	Note that the above command uses backticks (<code>`</code>), not single quotes (<code>'</code>). It searches all files in the current ] and its subdirectories which end in <code>.foo</code> for occurrences of the ] <code>bar</code>. These commands will not work as expected if there are whitespace characters, including ]s, in the filenames. In order to avoid this limitation one may use:

	<source lang="bash">
	find . -name "*.foo" -print0 \| xargs -0 grep bar
	</source>

	The above command uses GNU specific extensions to <code>find</code> and <code>xargs</code> to separate filenames using the ];


	<source lang="bash">
	find . -name "*.foo" -print0 \| xargs -0 -t -r vi
	</source>

	The above command is similar to the former one, but launches the ] editor for each of the files. The <code>-t</code> prints the command to stderr before issuing it. The <code>-r</code> is a GNU extension that tells <code>xargs</code> not to run the command if no input was received.


	<source lang="bash">
	find . -name "*.foo" -print0 \| xargs -0 -I {} mv {} /tmp/trash
	</source>

	The above command uses <code>-I</code> to tell <code>xargs</code> to replace <code>{}</code> with the argument list. Note that not all versions of <code>xargs</code> supports the <code>{}</code> syntax. In those cases you may specify a string after <code>-I</code> that will be replaced, e.g.


	<source lang="bash">
	find . -name "*.foo" -print0 \| xargs -0 -I xxx mv xxx /tmp/trash
	</source>

	The above command uses string <code>xxx</code> instead of <code>{}</code> as the argument list marker.


	<source lang="bash">
	find . -maxdepth 1 -type f -name "*.ogg" -print0 \| xargs -0 -r cp -v -p --target-directory=/home/media
	</source>

	The command above does the same as:

	<source lang="bash">
	cp -v -p *.ogg /home/media
	</source>

	however, the former command which uses <code>find</code>/<code>xargs</code>/<code>cp</code> is more resource efficient and will not halt with an error if the number of files is too large for the <code>cp</code> command to handle. Another way to do it (choosing where to put your arguments) is:

	<source lang="bash">
	find . -maxdepth 1 -type f -name "*.ogg" -print0 \| xargs -0 -I MYFILES cp MYFILES /home/media
	</source>

	The <code>-I</code> in the above command tells <code>xargs</code> what replacement string you want to use (otherwise it adds the arguments to the end of the command). You can also use <code>-L</code> to limit the number of arguments. If you do that, the command will be run repeatedly until it is out of arguments. Thus, <code>-L1</code> runs the command once for each argument (needed for tools like tar and such).

	==The separator problem==

	Many UNIX utilities are line oriented. These may work with xargs as long as the lines do not contain <tt>'</tt>, <tt>"</tt> or space. Some of the UNIX utilities can use ] as record separator (e.g. ] (requires -0 and \0 instead of \n), ] (requires using -0), ] (requires using <tt>-print0</tt>), ] (requires <tt>-z</tt> or <tt>-Z</tt>), ] (requires using <tt>-z</tt>)). Using <tt>-0</tt> for <tt>xargs</tt> deals with the problem, but many UNIX utilities cannot use NULL as separator (e.g. ], ], ], ], ], ] -v, ], ]).

	But often people forget this and assume xargs is also line oriented.<ref></ref>

	The separator problem is illustrated here:

	<source lang="bash">
	touch important_file
	touch 'not important_file'
	find -name not\* \| tail \| xargs rm
	mkdir -p '12" records'
	find \! -name . -type d \| tail \| xargs rmdir
	</source>

	Running the above will cause <tt>important_file</tt> to be removed and will remove neither the directory called <tt>12" records</tt>, nor the file called <tt>not important_file</tt>.

	The proper fix is to use <tt>find -print0</tt>, but <tt>tail</tt> (and other tools) do not support NULL terminated strings:

	<source lang="bash">
	touch important_file
	touch 'not important_file'
	find -name not\* -print0 \| xargs -0 rm
	mkdir -p '12" records'
	find \! -name . -print0 \| xargs -0 rmdir
	</source>

	When using the syntax <tt>find -print0</tt>, entries are separated by a null character instead of a end-of-line. This is equivalent to the more verbose command:

	<source lang="bash">
	find -name not\* \| tr \\n \\0 \| xargs -0 rm
	</source>

	] is an alternative to xargs that is designed to have the same options, but be line oriented. Thus, using ] instead, the above would work as expected.<ref></ref>

	For Unix environments where xargs does not support the -0 option (e.g. Solaris), the following can not be used as it does not deal with ' and " (] would work on Solaris, though):

	<source lang="bash">
	find -name not\* \| sed 's/ /\\ /g' \| xargs rm
	</source>

	==References==		==References==

Revision as of 22:55, 7 February 2012

xargs is a command on Unix and most Unix-like operating systems used to build and execute command lines from standard input. Under the Linux kernel before version 2.6.23, arbitrarily long lists of parameters could not be passed to a command, so xargs breaks the list of arguments into sublists small enough to be acceptable.

References

GNU Core Utilities FAQ

External links

xargs: construct argument lists and invoke utility – Shell and Utilities Reference, The Single UNIX Specification, Version 4 from The Open Group

Manual pages

xargs(1) – GNU Findutils reference
xargs(1): construct argument list(s) and execute utility – FreeBSD General Commands Manual
xargs(1): construct argument list(s) and execute utility – NetBSD General Commands Manual
xargs(1): construct argument list(s) and execute utility – OpenBSD General Commands Manual
xargs(1): construct argument lists and invoke utility – Solaris 11.4 User Commands Reference Manual

v t e Unix command-line interface programs and shell builtins
File system	cat chattr chmod chown chgrp cksum cmp cp dd du df file fuser ln ls mkdir mv pax pwd rm rmdir split tee touch type umask
Processes	at bg crontab fg kill nice ps time
User environment	env exit logname mesg talk tput uname who write
Text processing	awk basename comm csplit cut diff dirname ed ex fold head iconv join m4 more nl paste patch printf read sed sort strings tail tr troff uniq vi wc xargs
Shell builtins	alias cd echo test unset wait
Searching	find grep
Documentation	man
Software development	ar ctags lex make nm strip yacc
Miscellaneous	bc cal expr lp od sleep true and false
Categories Standard Unix programs Unix SUS2008 utilities List

Categories: