1

I am looking for a way to save all attachments from an IMAP folder with 3000+ DMARC reports each of which is attached inside the individual messages as .zip or .xml.gz file. I am using Claws Mail which has a way to automate this (See Examples -> Save) through munpack. Unfortunately on openSUSE Leap 42.3 (which I use) there doesn't seem to be a package which provides munpack. After some web searching I found an alternative: metamail -wy suggested as an answer to this question.

Trying that directly in claws gave me some error (as not using properly the program). So I saved manually one message in a temp directory and tried from console:

[/tmp/download]: metamail -yw msg
Date: Mon, 03 Nov 2014 10:13:53 +0000
Subject: Report domain: ████████ Submitter: ████████
From: [email protected]
To: ████████


This message contains data in an unrecognized format, application/zip,
which can either be viewed as text or written to a file.

What do you want to do with the application/zip data?
1 -- See it as text
2 -- Write it to a file
3 -- Just skip it

2
Please enter the name of a file to which the data should be written
(Default: ████████.zip) >
Wrote file ████████.zip

In other words: It works per se but it requires interactive input for each individual message - surely not an option for 3000+ mails.

My question is:

How can I use metamail in a way which doesn't require interactive input, so that it can be iterated (e.g. in a script or through Claws Mail's functinality)?

george
  • 115
  • 1
  • 4
  • There is also [ripmime](https://pldaniels.com/ripmime/) as suggested in [this answer](https://superuser.com/a/187184/53547). (`apt install ripmime` on Debian and derivatives). – mivk Sep 26 '19 at 21:12

2 Answers2

1

I'm not sure how the data you have actually looks like, so the best I can do is some educated guesses.

I used metamail -q -w -x file_name successfully in the past in scripts to extract attachments. As you can see from man metamail, -w tells metamail not to consult the mailcap file, but just extract the attachments, -q tells it to be quiet, because scripts don't like the extra output, and -x tells it that it's definitely not running on a terminal, so it shouldn't ask stupid questions.

You already have -w, and you are also using -y, which shouldn't be necessary if the mail is properly formatted, but maybe yours isn't - as I said, I don't know what data you have. The man page also warns about strange effects that can happen with -y.

So first drop -y, see if it still works. Then add -x, see if it stops asking. Finally add -q to remove unwanted output.

Edit

After being accused to be too lazy to try, I did the following steps:

1) Mailed myself a zip file with one empty file using mutt. Here's the complete mail, with some headers removed/edited:

From mail  Sat Mar  3 12:49:13 2018
Envelope-to: <dirk@...>
...
Message-ID: <20180303114913.xr7xvpx2pso6wurn@feanor>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="gsrhzrdopn4ddt7t"
Content-Disposition: inline
User-Agent: NeoMutt/20170609 (1.8.3)
Content-Length: 480
Lines: 18


--gsrhzrdopn4ddt7t
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Test

--gsrhzrdopn4ddt7t
Content-Type: application/zip
Content-Disposition: attachment; filename="foo.zip"
Content-Transfer-Encoding: base64

UEsDBAoAAAAAALplY0wAAAAAAAAAAAAAAAAHABwAenp6L2Zvb1VUCQAD8IqaWvCKmlp1eAsA
AQToAwAABOgDAABQSwECHgMKAAAAAAC6ZWNMAAAAAAAAAAAAAAAABwAYAAAAAAAAAAAApIEA
AAAAenp6L2Zvb1VUBQAD8IqaWnV4CwABBOgDAAAE6AMAAFBLBQYAAAAAAQABAE0AAABBAAAA
AAA=

--gsrhzrdopn4ddt7t--

2) Saved it into a mailbox called yyy, then ran 

METAMAIL_TMPDIR=. metamail -q -x -w ~/Mail/yyy

3) As expected, metamail doesn't ask, and saves all parts as two files called 1-M64bZlz and 1-MKfamEn. The second is the zip file, the first is the body text.

And now? It works for me, it doesn't work for you.

dirkt
  • 16,421
  • 3
  • 31
  • 37
  • Thanks for the quick reply. I tried what you suggest but it gives me (I hope this will be formatted properly): `[/tmp/download]: metamail -w -x -q msg This message contains 'application/zip`-format data. which is being decoded and written to the file named "/tmp/████.zip". If you do not want this data, you probably should delete that file. Wrote file /tmp/████.zip`. IOW: it is not really quite and also always outputs the file in `/tmp`. Do you know how I could direct it to save the files to a particular directory and hopefully make it quiet so that I can mark your answer as complete? – george Mar 02 '18 at 16:47
  • The input file `msg` is simply the mail message source. – george Mar 02 '18 at 16:51
  • Can you pick one of the smaller files with that problem, upload it somewhere and edit your question with a link so I can have a look, assuming it doesn't contain sensitive data? – dirkt Mar 02 '18 at 17:00
  • You can create a sample file by sending an email message to yourself with an attachment (e.g. zip) and then picking "View source" in your email client for the received message. BTW I think I have found the right command: `export METAMAIL_TMPDIR=/tmp/download; metamail -w -x -q %F > /tmp/export.log` where `%F` is the name of the input file. However *for some* DMARC reports it creates also a separate text file with a name similar to `mm.EcPvNM` which is sometimes 0 bytes and sometimes contains a short text message. Is there a way to optimize the command to extract only the attachments? – george Mar 02 '18 at 18:02
  • ETA: If you would like please edit your answer to contain the complete solution so we can mark it for others as "the answer" :) – george Mar 02 '18 at 18:03
  • I can certainly create a sample file by sending an email to myself, but it will work on those files, so I can't find out the issue with *your file* this way. :-) And it will extract *all* mime parts, which sometimes means parts of zero size if the mailer that created it screwed up. Other mailers like to include short text messages, which is probably what you see. Another reason I'd like to see one of *your* messages. :-) – dirkt Mar 03 '18 at 11:13
  • I am sorry but I cannot share publicly my messages. This is internal data. But I have given steps to reproduce as it is exactly the same principle. Have you tried that? Or are you simply denying to try because you haven't? :) – george Mar 03 '18 at 11:44
  • Thank you for testing. The last thing is: how can we make this work without saving the body text in a separate file, i.e. - to extract only the attachment? Is that possible please. (ETA: nobody is accusing you of anything) – george Mar 03 '18 at 12:15
  • 1
    You can't. It will extract *all* mime parts, and depending on the creating mailer, there can be *many* mime parts. You can parse the output of `metamail` with `grep -B2 application/zip` or similar to get the name of *all* mime parts with this content type. Or you can use `file` to determine the type of the extracted files. Or anything else that works for you. – dirkt Mar 03 '18 at 12:45
0
  1. ripmime is a really good utility for extracting attachments from an email (I use a script which is invoke from procmail)

2a) if you really want to use munpack, then you can probably use the binary found inside the package management file for a previous release. For example, I needed munpack on CentOS-7 but could not find an rpm for mpack so I used and rpmfinder to download this "mpack-1.6-2.el6.rf.x86_64.rpm" then used this command to unpack the contents.

rpm2cpio mpack-1.6-2.el6.rf.x86_64.rpm | cpio -idmv

(you might want to do this in an empty subfolder; use tree to find the files; mine were under .usr/bin)

2b) munpack-1.6 has an undocumented switch "-t" to perform better text extractions (not shown when you use "-?")

neilrieck
  • 1
  • 1