23.0.60; RMAIL reading of mail file

23.0.60; RMAIL reading of mail file

[jpff]

I have been performing further experiments on this question (reading
my RMAIL file into latest CVS emacs).  On a 2GHz AMD64x2 machine it
took 118minutes of CPU, basically 100% of one processor for 2 hours to
obey M-x rmail-mode after I had loaded the file with M-x
find-file-literally.  This seems in line with my belief that the
algorithm being used is n^2, and for my 95Kb file is totally
unacceptable.  The good news is that it does run....... eventually.
On my laptop this translates to over 3hrs which is comparable with
the time between getting up and going to work, or being at work.  I am
willing to help but I do not understand what it is trying to do; but
it is reading data more than once.

==John ffitch

23.0.60; RMAIL reading of mail file

[Bob Rogers]

   From: <jpff@cs.bath.ac.uk>
   Date: Thu,  7 Feb 2008 16:46:05 +0000

   I have been performing further experiments on this question (reading
   my RMAIL file into latest CVS emacs).  On a 2GHz AMD64x2 machine it
   took 118minutes of CPU, basically 100% of one processor for 2 hours to
   obey M-x rmail-mode after I had loaded the file with M-x
   find-file-literally.  This seems in line with my belief that the
   algorithm being used is n^2, and for my 95Kb file is totally
   unacceptable . . .

   ==John ffitch

I cannot reproduce this on my 1.8GHz i686 Athlon with 256M of RAM, using
emacsen built from trunk today and on 27-Jan (i.e. both before and after
the Unicode merge).  I observe linear CPU-time growth between files of
127M and 254M (albeit artificially constructed ones), to a total of
about 18sec and 34sec respectively.

   I wonder, how much RAM do you have on this system?

					-- Bob Rogers
					   http://rgrjr.dyndns.org/

Re: 23.0.60; RMAIL reading of mail file

[jpff]

Sorry for  delay -- your IP address in the the abuse list and so was
filed as junk
X-Warning: 98.216.94.251 is listed at rbl-plus.mail-abuse.ja.net

>>   I wonder, how much RAM do you have on this system?

4Gb

When I was tracing the operation what I notices was that it has found
a \033 character, and that triggered a call to detect_coding_iso_2022
on line 7379 on src/coding.c
  The detect_coding_iso_2022 looks at all the remaining characters of
the file as far as i could see, via
    ONE_MORE_BYTE (c);
      switch (c)

That seemed to get almost to the end of the file (end?) and then
return.  The main scan started again until the next \033 when the
whole procedure was repeated, scanning to the end.

I can confirm that the 118mins is repeatable, and emacs 22 did the
same in 8secs

You can have my RMAIL file if necessary -- not sure there is much
sensitive in it.  Or I could send snapshot values from gdb
investigations.


BTW do you work on rmail in general? There is a long standing bug
with reading mail from two places in the the same g command

==John ffitch

Re: 23.0.60; RMAIL reading of mail file

[Eli Zaretskii]

> Date: Sun, 10 Feb 2008 12:17:23 +0000
> From: jpff <jpff@codemist.co.uk>
> Cc: emacs-pretest-bug@gnu.org
> 
> When I was tracing the operation what I notices was that it has found
> a \033 character, and that triggered a call to detect_coding_iso_2022
> on line 7379 on src/coding.c
>   The detect_coding_iso_2022 looks at all the remaining characters of
> the file as far as i could see, via
>     ONE_MORE_BYTE (c);
>       switch (c)
> 
> That seemed to get almost to the end of the file (end?) and then
> return.  The main scan started again until the next \033 when the
> whole procedure was repeated, scanning to the end.

Could you please post here an example of a file that causes this
slowness?  Please make it a binary attachment, to protect it from
munging by the various mail agents in between.

> BTW do you work on rmail in general?

I'm not him, but I do use Rmail all the time.

> There is a long standing bug with reading mail from two places in
> the the same g command

Where can I find the details?

Re: 23.0.60; RMAIL reading of mail file

[Kenichi Handa]

In article <4681-Sun10Feb2008121723+0000-jpff@codemist.co.uk>, jpff <jpff@codemist.co.uk> writes:
[...]
> When I was tracing the operation what I notices was that it has found
> a \033 character, and that triggered a call to detect_coding_iso_2022
> on line 7379 on src/coding.c
>   The detect_coding_iso_2022 looks at all the remaining characters of
> the file as far as i could see, via
>     ONE_MORE_BYTE (c);
>       switch (c)

> That seemed to get almost to the end of the file (end?) and then
> return.  The main scan started again until the next \033 when the
> whole procedure was repeated, scanning to the end.

> I can confirm that the 118mins is repeatable, and emacs 22 did the
> same in 8secs

> You can have my RMAIL file if necessary -- not sure there is much
> sensitive in it.  Or I could send snapshot values from gdb
> investigations.

I'd like to test with your RMAIL file (of 95Mb).  If
possible, please put it somewhere I can reach by anonymous
ftp or wget.

By the way, are those \033 characters a part of real
iso-2022-based encoding?  If not, please try with setting
inhibit-iso-escape-detection to t.

> BTW do you work on rmail in general? There is a long standing bug
> with reading mail from two places in the the same g command

I'm using rmail, but reading mails from only one point.  So,
I don't know about that problem.

---
Kenichi Handa
handa@ni.aist.go.jp