From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id C6FC76DE102D for ; Sat, 30 Mar 2019 16:53:55 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.956 X-Spam-Level: X-Spam-Status: No, score=-0.956 tagged_above=-999 required=5 tests=[AWL=-0.307, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B-J4fbLdtJtv for ; Sat, 30 Mar 2019 16:53:54 -0700 (PDT) Received: from aibo.runbox.com (aibo.runbox.com [91.220.196.211]) by arlo.cworth.org (Postfix) with ESMTPS id 52BC66DE0F37 for ; Sat, 30 Mar 2019 16:53:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=runbox.com; s=rbselector1; h=Content-Transfer-Encoding:Content-Type:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=Hn4LBMaKYisfcSDlk3ApD4XCg2SzAwZytqg8aKlfrxg=; b=BOzFMBgsC7Q8f27z+3HNu9WtSq AwSDjRH/XKPYgMdFO3JWfxR7sOGg+RBiPS1uh0bJSysLcLTEKkWh+BhQKBKwL8UFP1FWuoznHx4dI bsVf2XDdBndwRAlD4wmMr12p61GGIJ2X2LMLjIBxUC4YkpD/z8JQIjidRkZsU6p9ULTgdsA7H5n3L SMo5pi9nIpt/QYspWrQNxZh8LD2MlXwuPMVTu34FnUP/D4I+XfkniY1trfIUbCeXh/AxAU8wE6JfL sgqm/OXzE+un0JBqJY0i9vw1A6p5dHsx4qcazvRuKITmX+6sxTlwChChDToFjdqM50L4DUifNvGku 98foxNDQ==; Received: from [10.9.9.211] (helo=mailfront11.runbox.com) by mailtransmit02.runbox with esmtp (Exim 4.86_2) (envelope-from ) id 1hANnI-0005u2-FV; Sun, 31 Mar 2019 00:53:48 +0100 Received: by mailfront11.runbox.com with esmtpsa (uid:850688 ) (TLS1.2:RSA_AES_256_CBC_SHA256:256) (Exim 4.82) id 1hANn2-0004nc-9V; Sun, 31 Mar 2019 00:53:33 +0100 From: "Alexei Gilchrist" To: "David Bremner" Cc: notmuch@notmuchmail.org Subject: Re: notmuch ignoring alot of emails Date: Sun, 31 Mar 2019 10:53:27 +1100 X-Mailer: MailMate (1.12.4r5594) Message-ID: In-Reply-To: <877ecgy505.fsf@tethera.net> References: <9359F7FC-94C5-48CA-B4C9-2FEB2C7EC398@runbox.com> <877ecgy505.fsf@tethera.net> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=_MailMate_3A318F7F-A56E-4A80-961B-0C122222493F_=" Content-Transfer-Encoding: 8bit X-Clacks-Overhead: GNU Terry Pratchett X-Mailman-Approved-At: Sun, 31 Mar 2019 00:17:24 -0700 X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Mar 2019 23:53:55 -0000 --=_MailMate_3A318F7F-A56E-4A80-961B-0C122222493F_= Content-Type: text/plain; charset=utf-8; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit >> When I run notmuch I get a bunch (hundreds) of emails that are >> ignored >> with: >> >> Note: Ignoring non-mail file: ... >> >> The files are valid maildir files but have a paragraph somewhere in >> the >> body where someone has written "From ". >> > > And do they also have have a line starting with "From " as the first > line? This makes them mbox files. The second "From " makes them mbox > files with multiple messages. Notmuch thinks your MDA (the thing that > made those files) is misconfigured, assuming my guess about the format > is correct. Every message file begins with “From “. This is true of all messages downloaded by both offlineimap (with type = Maildir) and mbsync. neomutt has no issues dealing with these files as maildir and mu has no issues indexing them either. I’m assuming that stating with “From “ is part of the maildir spec. The problem occurs specifically with notmuch. If someone sends a message with a line that begins with “From “ in the *body* then it confuses notmuch. mu can correctly index these messages but my mu is linked against libgmime-2.6, my notmuch (0.28.3) is linked against libgmime-3.0. >> Is there a fix to force the recognition of maildir files in this >> case? I >> thought this was a solved problem with gmime since 2.6.7. > > There is not currently a way to do that. It's not a GMime problem, > it's > a design choice of notmuch to avoid parsing multiple message > mbox's. That was originally added as a safety feature, and I think it > should probably stay the default. If someone wants work on adding a > configuration switch I can point them in the right direction. This is a poor design decision. It means anyone on the internet can break your mail setup simply by sending a message with a line starting with “From “. (and using usual quoted-printable Content-Transfer-Encoding). Try it. Send yourself a message with the line “From bad parsing comes chaos” and see if your notmuch can find it. My version can’t. --=_MailMate_3A318F7F-A56E-4A80-961B-0C122222493F_= Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

When I run notmuch I get a bunch (hundreds) of emails tha= t are ignored
with:

Note: Ignoring non-mail file: ...

The files are valid maildir files but have a paragraph so= mewhere in the
body where someone has written "From ".

And do they also have have a line starting with "From " a= s the first
line? This makes them mbox files. The second "From " makes them mbox
files with multiple messages. Notmuch thinks your MDA (the thing that
= made those files) is misconfigured, assuming my guess about the format is correct.

Every message file begins with =E2=80=9CFrom =E2=80=9C. T= his is true of all messages downloaded by both offlineimap (with type =3D= Maildir) and mbsync.
neomutt has no issues dealing with these files as maildir and mu has no i= ssues indexing them either. I=E2=80=99m assuming that stating with =E2=80= =9CFrom =E2=80=9C is part of the maildir spec.

The problem occurs specifically with notmuch. If someone = sends a message with a line that begins with =E2=80=9CFrom =E2=80=9C in t= he body then it confuses notmuch.

mu can correctly index these messages but my mu is linked= against libgmime-2.6, my notmuch (0.28.3) is linked against libgmime-3.0= =2E

Is there a fix to force the recognition of maildir files = in this case? I
thought this was a solved problem with gmime since 2.6.7.

There is not currently a way to do that. It's not a GMime= problem, it's
a design choice of notmuch to avoid parsing multiple message
mbox's. That was originally added as a safety feature, and I think it
= should probably stay the default. If someone wants work on adding a
configuration switch I can point them in the right direction.

This is a poor design decision. It means anyone on the in= ternet can break your mail setup simply by sending a message with a line = starting with =E2=80=9CFrom =E2=80=9C.
(and using usual quoted-printable Content-Transfer-Encoding).

Try it. Send yourself a message with the line =E2=80=9CFr= om bad parsing comes chaos=E2=80=9D and see if your notmuch can find it. = My version can=E2=80=99t.

--=_MailMate_3A318F7F-A56E-4A80-961B-0C122222493F_=--