unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* Possible bug: incorrect processing of multipart, base64-encoded subject lines.
@ 2022-09-21 14:15 Thibault Polge
  2022-09-23 23:48 ` [PATCH] test: add regression test for Subject with newline David Bremner
  2022-10-14 14:49 ` Possible bug: incorrect processing of multipart, base64-encoded subject lines Jakub Wilk
  0 siblings, 2 replies; 5+ messages in thread
From: Thibault Polge @ 2022-09-21 14:15 UTC (permalink / raw)
  To: notmuch

Hi all,

Notmuch doesn't parse correctly some Subject lines consisting of
multiple base64-encoded snippets.  As an example, this is the raw
subject line of an e-mail from Amazon:

Subject: =?UTF-8?B?TGl2cmFpc29uIHByw6l2dWUgcG91ciBhdWpvdXJk4oCZaHU=?=
 =?UTF-8?B?aTogUkVTVFJBUCBTYWRkbGUgQmFnIFNhY2NvY2hlLi4u?=

Notmuch displays the subject as only the decoded contents of the first
fragment (or first line), "Livraison prévue pour aujourd’hu", fully
ignoring the second base64 fragment, "i: RESTRAP Saddle Bag
Saccoche...".  The complete subject line should thus read:

> Livraison prévue pour aujourd’hui: RESTRAP Saddle Bag Saccoche...

Regards,
Thibault\r

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH] test: add regression test for Subject with newline.
  2022-09-21 14:15 Possible bug: incorrect processing of multipart, base64-encoded subject lines Thibault Polge
@ 2022-09-23 23:48 ` David Bremner
  2022-10-10 21:24   ` David Bremner
  2022-10-14 14:49 ` Possible bug: incorrect processing of multipart, base64-encoded subject lines Jakub Wilk
  1 sibling, 1 reply; 5+ messages in thread
From: David Bremner @ 2022-09-23 23:48 UTC (permalink / raw)
  To: Thibault Polge, notmuch

This tests the issue reported by Thibault in id:87wn9w4xus.fsf@thb.lt
---

I could not duplicate the problem here. Maybe it depends on the version of gmime?
I have 3.2.9 here.

 test/T050-new.sh                         |  4 ++++
 test/corpora/indexing/subject-newline:2, | 10 ++++++++++
 2 files changed, 14 insertions(+)
 create mode 100644 test/corpora/indexing/subject-newline:2,

diff --git a/test/T050-new.sh b/test/T050-new.sh
index cb67889c..0ac9fd7c 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -463,4 +463,8 @@ notmuch search id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain > EXPECT
 notmuch search id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain and ersatz > OUTPUT
 test_expect_equal_file_nonempty EXPECTED OUTPUT
 
+test_begin_subtest "base64 subject with newline"
+output=$(notmuch show id:subject-with-newline@shopping.com | grep ^Subject:)
+test_expect_equal "${output}" \
+		  "Subject: Livraison prévue pour aujourd’hui: RESTRAP Saddle Bag Saccoche..."
 test_done
diff --git a/test/corpora/indexing/subject-newline:2, b/test/corpora/indexing/subject-newline:2,
new file mode 100644
index 00000000..bb2dde55
--- /dev/null
+++ b/test/corpora/indexing/subject-newline:2,
@@ -0,0 +1,10 @@
+From: "Thibault" <thibault@example.com>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 02:08:10 -0800
+Subject: =?UTF-8?B?TGl2cmFpc29uIHByw6l2dWUgcG91ciBhdWpvdXJk4oCZaHU=?=
+ =?UTF-8?B?aTogUkVTVFJBUCBTYWRkbGUgQmFnIFNhY2NvY2hlLi4u?=
+Message-ID: <subject-with-newline@shopping.com>
+
+The subject should be
+
+> Livraison prévue pour aujourd’hui: RESTRAP Saddle Bag Saccoche...
-- 
2.35.2
\r

^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [PATCH] test: add regression test for Subject with newline.
  2022-09-23 23:48 ` [PATCH] test: add regression test for Subject with newline David Bremner
@ 2022-10-10 21:24   ` David Bremner
  2022-12-27 15:50     ` David Bremner
  0 siblings, 1 reply; 5+ messages in thread
From: David Bremner @ 2022-10-10 21:24 UTC (permalink / raw)
  To: Thibault Polge, notmuch; +Cc: Daniel Kahn Gillmor, Jakub Wilk

David Bremner <david@tethera.net> writes:

> This tests the issue reported by Thibault in id:87wn9w4xus.fsf@thb.lt
> ---
>
> I could not duplicate the problem here. Maybe it depends on the version of gmime?
> I have 3.2.9 here.

Now that I have gmime 3.2.13 I can confirm your bug (my previously
posted test fails).  It seems the purely a gmime issue (the following
reproducer is thanks to Jakub Wilk; on debian it needs the package
gir1.2-gmime-3.0).

#!/usr/bin/python3

import pathlib
import tempfile

import gi
gi.require_version('GMime', '3.0')
from gi.repository import GMime
GMime.init()

msg = b'''\
Subject: =?UTF-8?B?SGVsbG8=?= =?UTF-8?B?IHdvcmxk?=

.
'''

with tempfile.NamedTemporaryFile() as tmpfile:
    tmpfile.write(msg)
    tmpfile.flush()
    fp = GMime.StreamFile.open(tmpfile.name, 'r')
    parser = GMime.Parser.new_with_stream(fp)
    msg = parser.construct_message()
    subject = msg.subject
    expected = 'Hello world'
    assert subject == expected, f'{subject!r} != {expected!r}'


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Possible bug: incorrect processing of multipart, base64-encoded subject lines.
  2022-09-21 14:15 Possible bug: incorrect processing of multipart, base64-encoded subject lines Thibault Polge
  2022-09-23 23:48 ` [PATCH] test: add regression test for Subject with newline David Bremner
@ 2022-10-14 14:49 ` Jakub Wilk
  1 sibling, 0 replies; 5+ messages in thread
From: Jakub Wilk @ 2022-10-14 14:49 UTC (permalink / raw)
  To: notmuch; +Cc: Thibault Polge, Daniel Kahn Gillmor

* Thibault Polge <thibault@thb.lt>, 2022-09-21 16:15:
>Subject: =?UTF-8?B?TGl2cmFpc29uIHByw6l2dWUgcG91ciBhdWpvdXJk4oCZaHU=?=
> =?UTF-8?B?aTogUkVTVFJBUCBTYWRkbGUgQmFnIFNhY2NvY2hlLi4u?=
>
>Notmuch displays the subject as only the decoded contents of the first 
>fragment

I believe this is now fixed in GMime upstream:
https://github.com/jstedfast/gmime/commit/1a33a55baafc73b0

-- 
Jakub Wilk

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH] test: add regression test for Subject with newline.
  2022-10-10 21:24   ` David Bremner
@ 2022-12-27 15:50     ` David Bremner
  0 siblings, 0 replies; 5+ messages in thread
From: David Bremner @ 2022-12-27 15:50 UTC (permalink / raw)
  To: Thibault Polge, notmuch; +Cc: Daniel Kahn Gillmor, Jakub Wilk

David Bremner <david@tethera.net> writes:

> David Bremner <david@tethera.net> writes:
>
>> This tests the issue reported by Thibault in id:87wn9w4xus.fsf@thb.lt
>> ---
>>
>> I could not duplicate the problem here. Maybe it depends on the version of gmime?
>> I have 3.2.9 here.
>
> Now that I have gmime 3.2.13 I can confirm your bug (my previously
> posted test fails).  It seems the purely a gmime issue (the following
> reproducer is thanks to Jakub Wilk; on debian it needs the package
> gir1.2-gmime-3.0).

I think I won't go through the hassle of marking this broken for a
narrow range of gmime versions. Dropping the patch from the review queue
for now.

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2022-12-27 15:50 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-21 14:15 Possible bug: incorrect processing of multipart, base64-encoded subject lines Thibault Polge
2022-09-23 23:48 ` [PATCH] test: add regression test for Subject with newline David Bremner
2022-10-10 21:24   ` David Bremner
2022-12-27 15:50     ` David Bremner
2022-10-14 14:49 ` Possible bug: incorrect processing of multipart, base64-encoded subject lines Jakub Wilk

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).