25

I need to compress 80.000 files into multiple zip files. This is the command I use:

zip -s 200M photos_test/*

However I get the following error:

-bash: /usr/bin/zip: Argument list too long

What can I do to solve the issue, beside manually splitting the folder files ?

thanks

shgnInc
  • 435
  • 7
  • 17
aneuryzm
  • 2,105
  • 10
  • 37
  • 46
  • 1
    The error `-bash: /usr/bin/zip: Argument list too long` may cause in to case: 1- because of not using `-r` switch, 2- there too many files for archiving. So in first case @Mat's answer is true and in the second case the @IgnacioVazquez-Abrams's answer is true. – shgnInc Jul 06 '14 at 06:57

4 Answers4

19

If you want the whole directory, you could simply use the -r switch:

zip -r -s 200M myzip photos_test

That will include all subdirectories of photos_test though.

Mat
  • 8,043
  • 1
  • 33
  • 32
  • I've done what you suggest and I have a small myzip.zip (7mb) and the segments (200mb each). However, I cannot unzip the content, I'm running unzip unix myzip.zip but I get "bad zipfile offset (lseek)". Furthermore, I need to extract them in Windows environment as well, and there I only have Windows 7 extractor I guess. – aneuryzm Apr 19 '11 at 10:08
  • those are different questions, open another question for that (link to this one for reference). make sure you post the names of the generated zip files (not all of them, but first and last at least), and the **exact** command line you use. – Mat Apr 19 '11 at 10:11
15

The problem seems to be the expansion of the "*". Use folder name or ".":

If you want to include the root folder within the zip:

zip -r my.zip folder_with_80k_files

If you don´t want to include the root folder inside the zip:

cd folder_with_80k_files
zip -r my.zip .
stoffen
  • 251
  • 2
  • 4
6
find photos_test/ -mindepth 1 -maxdepth 1 | zip -@ -s 200M
Ignacio Vazquez-Abrams
  • 111,361
  • 10
  • 201
  • 247
  • 4
    Use `find . -mindepth 1 -maxdepth -name '*.json' | zip {YOURZIPFILENAME}.zip -@` if you don't need to split and want to select files by extension. – Daniel Sokolowski Jun 12 '17 at 15:03
  • 1
    `find . -mindepth 1 -maxdepth 1 -name '*.json' | zip {YOURZIPFILENAME}.zip -@` (there was missing `1` after -maxdepth). – Lucas Oct 28 '21 at 14:55
  • I am seeing errors : `zip error: Invalid command arguments (cannot write zip file to terminal)`. – jdhao Apr 19 '22 at 03:27
4

ls photos_test | zip -s 200M -@ photos

  • -@ will cause zip to read a list of files from stdin
  • | will pipe an output of ls into the input of zip command

man zip:

USE
⋮
   -@ file lists.  If a file list is specified as -@ [Not on MacOS], zip takes
   the  list  of  input  files from standard input instead of from the command
   line.  For example,

          zip -@ foo

   will store the files listed one per line on stdin in foo.zip.

   Under Unix, this option can be used to powerful effect in conjunction  with
   the  find (1)  command.   For example, to archive all the C source files in
   the current directory and its subdirectories:

          find . -name "*.[ch]" -print | zip source -@

   (note that the pattern must be quoted to keep the shell from expanding it).
⋮
Mr. Tao
  • 468
  • 1
  • 4
  • 13