bug#15984: 24.3; Problem with combining characters in attachment filename

[handa@gnu.org (K. Handa)]

In article <87eh574qmm.fsf@gnu.org>, handa@gnu.org (K. Handa) writes:

> I'll keep trying to find why the trunk doesn't crash with
> you recipe, and once I find the whole story, I'll install a
> proper patch (which may be the same as what I sent) to the
> trunk.

I couldn't reproduce that bug with the trunk code.  I
rewinded back to the day 2013-03-11 which is the day 24.3
was released and I can reproduce the bug with 24.3.  So, I
am now very puzzled.

Anyway, I installed that fix to the trunk because the
previous code was apparently wrong.

---
Kenichi Handa
handa@gnu.org

PS. I've just noticed that recent mails exchanged on this
matter were not CC:ed to 15984@debbugs.gnu.org.  So, to
provide the context, I attach some key mails here.

-1--------------------------------------------------------------------
From: nisse@lysator.liu.se (Niels Möller)
To: handa@gnu.org (K. Handa)
Subject: Re: bug#15984: 24.3;	Problem with combining characters in attachment filename

handa@gnu.org (K. Handa) writes:

> In article <83siue58mq.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:
>
>> I hope Handa-san will be able to find the problem.  The crash is 100%
>> reproducible with the steps described above and a mail message that
>> Niels can send you off-list.
>
> Could you please send me that mail message?  I'll delete it
> as soon as I can find a fix.

I believe the smaller bounce message I posted in the bugtracker exhibits
the the problem. That's the same file Eli was using when reproducing the
problem. Described at

  http://debbugs.gnu.org/cgi/bugreport.cgi?msg=14;bug=15984

actual messge (gzipped):

  http://debbugs.gnu.org/cgi/bugreport.cgi?msg=14;filename=bounce.gz;att=1;bug=15984

Steps to reproduce the problem (this info spread out in the bug thread):

1. Create a new directory, say mail-tmp. Copy the message (uncompressed)
   into that directory, with filename "1".

2. Start emacs in tty mode, with a latin-1 locale, like

   HOME=$HOME/tmp LC_CTYPE=sv_SE.ISO8859-1 src/emacs -Q -l bug.el

   with bug.el containing

     (setq gnus-init-file nil)
     (setq gnus-nntp-server nil)
     (gnus-no-server)

3. Then, in Gnus' *Group* buffer, create the group with G d, pointing
   out the mail-tmp directory, enter the group (RET), view the message
   (RET), try to write out the attachment ("o" on the attachment
   button). Still crashes for me.

Let me know if you need anything further info.

Regards,
/Niels

-- 
Niels Möller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

-2--------------------------------------------------------------------
From: Eli Zaretskii <eliz@gnu.org>
Subject: Re: bug#15984: 24.3;	Problem with combining characters in attachment filename
To: handa@gnu.org (K. Handa)
Cc: nisse@lysator.liu.se, handa@gnu.org

> From: handa@gnu.org (K. Handa)
> Cc: eliz@gnu.org, handa@gnu.org
> Date: Fri, 13 Dec 2013 23:15:00 +0900
> 
> In article <nn4n6dag53.fsf@bacon.lysator.liu.se>, nisse@lysator.liu.se (Niels Möller) writes:
> 
> > And tty mode, no X frame (I used an xterm, started in a latin-1 locale).
> 
> Yes.  I surely add "-nw" argument, and I tried the recipe
> with xterm and lxterminal.

I cannot reproduce this either, with today's trunk.  Perhaps you
could try with the trunk as it was on Nov 30, or with Emacs 24.3?

> By the way, I noticed that buffer-file-coding-system of
> Gnus's message buffer (the buffer showing that bounce mail)
> is raw-text-unix.  Is it the same with you?

Yes.  This might be part of the problem, or it could be the trigger
for the crash.


-3---------------------------------------------------------------------
From: handa@gnu.org (K. Handa)
To: nisse@lysator.liu.se (Niels Möller)
Cc: eliz@gnu.org, handa@gnu.org
Subject: Re: bug#15984: 24.3;	Problem with combining characters in attachment filename

In article <nn4n6dag53.fsf@bacon.lysator.liu.se>, nisse@lysator.liu.se (Niels Möller) writes:

> And tty mode, no X frame (I used an xterm, started in a latin-1 locale).

Yes.  I surely add "-nw" argument, and I tried the recipe
with xterm and lxterminal.

By the way, I noticed that buffer-file-coding-system of
Gnus's message buffer (the buffer showing that bounce mail)
is raw-text-unix.  Is it the same with you?

---
Kenichi Handa
handa@gnu.org

-4--------------------------------------------------------------------
From: nisse@lysator.liu.se (Niels Möller)
To: handa@gnu.org (K. Handa)
Cc: eliz@gnu.org
Subject: Re: bug#15984: 24.3;	Problem with combining characters in attachment filename

handa@gnu.org (K. Handa) writes:

> By the way, I noticed that buffer-file-coding-system of
> Gnus's message buffer (the buffer showing that bounce mail)
> is raw-text-unix.  Is it the same with you?

Yes. Probably wasn't in the original mail (if you like, I can look into
that further, but I don't want to crash the emacs I'm writing this in
right now).

Regards,
/Niels

-- 
Niels Möller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

-5--------------------------------------------------------------------
From: handa@gnu.org (K. Handa)
To: Eli Zaretskii <eliz@gnu.org>
Cc: nisse@lysator.liu.se, handa@gnu.org
Subject: Re: bug#15984: 24.3;	Problem with combining characters in attachment filename

In article <838uvo6cjx.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

> I cannot reproduce this either, with today's trunk.  Perhaps you
> could try with the trunk as it was on Nov 30, or with Emacs 24.3?

With Emacs 24.3, I could reproduce the bug and the patch
attached at the tail seems to fix it.  Could you please try
it?  It is applicable to the latest code too.

But, with the trunk, I have not yet succeeded in reproducing
the bug.  I tried from the revision of Nov 30 and went
back to April one month by one.

> > By the way, I noticed that buffer-file-coding-system of
> > Gnus's message buffer (the buffer showing that bounce mail)
> > is raw-text-unix.  Is it the same with you?

> Yes.  This might be part of the problem, or it could be the trigger
> for the crash.

With Emacs 24.3, the bug can be reproduced with a multibyte
buffer.

---
Kenichi Handa
handa@gnu.org

=== modified file 'src/composite.c'
--- src/composite.c	2013-01-01 09:11:05 +0000
+++ src/composite.c	2013-12-19 13:49:53 +0000
@@ -1426,7 +1426,7 @@
       cmp_it->width = 0;
       for (i = cmp_it->nchars - 1; i >= 0; i--)
 	{
-	  c = XINT (LGSTRING_CHAR (gstring, i));
+	  c = XINT (LGSTRING_CHAR (gstring, cmp_it->from + i));
 	  cmp_it->nbytes += CHAR_BYTES (c);
 	  cmp_it->width += CHAR_WIDTH (c);
 	}


-6--------------------------------------------------------------------
From: nisse@lysator.liu.se (Niels Möller)
To: handa@gnu.org (K. Handa)
Cc: Eli Zaretskii <eliz@gnu.org>
Subject: Re: bug#15984: 24.3;	Problem with combining characters in attachment filename

handa@gnu.org (K. Handa) writes:

> With Emacs 24.3, I could reproduce the bug and the patch
> attached at the tail seems to fix it.  Could you please try
> it?  It is applicable to the latest code too.

I compiled 24.3.1 with the patch applied. It no longer crashes. Great!

Behavior is that on saving the attachment, the default filename is
displayed as "Brev aktiea?gar 131127.pdf", where the question mark
really is a COMBINING DIAERESIS (according to C-u C-x =). When I press
enter, the file is saved under the file name "Brev aktiea gar
131127.pdf", with the combining diaeresis replaced by a SPC character
(checked with GNU ls -N | od -tx1c).

Regards,
/Niels

-- 
Niels Möller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

-7---------------------------------------------------------------------
From: handa@gnu.org (K. Handa)
To: nisse@lysator.liu.se (Niels Möller)
Cc: eliz@gnu.org, handa@gnu.org
Subject: Re: bug#15984: 24.3;	Problem with combining characters in attachment filename

In article <nn4n64m18i.fsf@bacon.lysator.liu.se>, nisse@lysator.liu.se (Niels Möller) writes:

> handa@gnu.org (K. Handa) writes:
> > With Emacs 24.3, I could reproduce the bug and the patch
> > attached at the tail seems to fix it.  Could you please try
> > it?  It is applicable to the latest code too.

> I compiled 24.3.1 with the patch applied. It no longer crashes. Great!

Thank you for testing that.

> Behavior is that on saving the attachment, the default filename is
> displayed as "Brev aktiea?gar 131127.pdf", where the question mark
> really is a COMBINING DIAERESIS (according to C-u C-x =). When I press
> enter, the file is saved under the file name "Brev aktiea gar
> 131127.pdf", with the combining diaeresis replaced by a SPC character
> (checked with GNU ls -N | od -tx1c).

This just my guess, but, as far as you are in ISO-8859-1
locale, there's no way to encode that combining diaeresis,
so gnus uses SPC as a replacement character.

Perhaps, gnus should warn you about that and ask you how to
encode the file name.

Anyway that is completely different matter than bug#15984.

I'll keep trying to find why the trunk doesn't crash with
you recipe, and once I find the whole story, I'll install a
proper patch (which may be the same as what I sent) to the
trunk.

---
Kenichi Handa
handa@gnu.org