bug#1650: 23.0.60; raw-text-dos memory corruption

bug#1650: 23.0.60; raw-text-dos memory corruption

[Johan =?UTF-8?Q?Bockg=C3=A5rd]

GNU Emacs 23.0.60.12 (x86_64-unknown-linux-gnu, GTK+ Version 2.14.4) of
2008-12-20

emacs -Q

  ;; dictd is running on port 2628
  (let* ((coding-system-for-read 'raw-text-dos)
         (coding-system-for-write 'raw-text-dos)
         (proc (open-network-stream "foo" "foo" "localhost" 2628)))
    ;; The crash goes away if the next line is uncommented
    ;; (sit-for .1)
    (process-send-string proc "define * \"vice\"\r\n"))

=> Crash (backtrace below)

I can reproduce the crash on Emacs versions after 2008-03-25, whereas I
don't see the problem on versions before 2008-03-02. Intermediate
versions don't crash, but instead hang and consume all memory.


2008-03-25  Stefan Monnier  <monnier@iro.umontreal.ca>

        [...]

	* process.h (struct Lisp_Process): Remove filter_multibyte.
	* process.c (QCfilter_multibyte): Remove.
	(setup_process_coding_systems): Don't use filter_multibyte.
	(Fstart_process, Fmake_network_process): Don't set filter_multibyte.
	(read_process_output): Don't adjust multibyteness to filter_multibyte.
	(Fset_process_filter_multibyte): Change the coding-system to
	approximate the previous behavior.
	(Fprocess_filter_multibyte_p): Get the multibyteness straight from the
	coding-system.

	* coding.c (decode_coding_object): When not decoding into a buffer,
	obey the coding system's preference of (uni|multi)byte.


2008-03-02  Kenichi Handa  <handa@m17n.org>

	* coding.c (decode_coding_utf_8): When eol-type of CODING is
	`dos', don't decode '\r' if that is the last in the source.
	(decode_coding_utf_16, decode_coding_emacs_mule)
	(decode_coding_iso_2022, decode_coding_sjis, decode_coding_big5)
	(decode_coding_raw_text, decode_coding_charset): Likewise.
	(produce_chars): Don't decode EOL here.  Use EMACS_INT.


*** glibc detected *** /home/bojohan/vc/emacs/src/emacs: malloc(): memory corruption: 0x00000000022f83e0 ***
======= Backtrace: =========
/lib/libc.so.6[0x7f4f73ed2cff]
/lib/libc.so.6(__libc_malloc+0x98)[0x7f4f73ed4538]
/home/bojohan/vc/emacs/src/emacs[0x5497ce]
[...]

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7f4f78df7770 (LWP 7357)]
0x00007f4f73e88fd5 in raise () from /lib/libc.so.6
(gdb) bt
#0  0x00007f4f73e88fd5 in raise () from /lib/libc.so.6
#1  0x00007f4f73e8ab43 in abort () from /lib/libc.so.6
#2  0x00007f4f73ec9fa8 in ?? () from /lib/libc.so.6
#3  0x00007f4f73ed2cff in ?? () from /lib/libc.so.6
#4  0x00007f4f73ed4538 in malloc () from /lib/libc.so.6
#5  0x00000000005497ce in lisp_malloc (nbytes=7357, type=7357) at alloc.c:861
#6  0x000000000054a09a in allocate_string_data (s=0xdc7510, nchars=8136, 
    nbytes=8136) at alloc.c:1991
#7  0x000000000054ab90 in make_uninit_multibyte_string (nchars=8136, 
    nbytes=8136) at alloc.c:2508
#8  0x000000000054ac87 in make_uninit_string (length=7357) at alloc.c:2486
#9  0x00000000005587fd in make_buffer_string_both (start=1, start_byte=1, 
    end=8137, end_byte=<value optimized out>, props=1) at editfns.c:2420
#10 0x0000000000481c35 in decode_coding_object (coding=0xab4800, 
    src_object=11030241, from=0, from_byte=0, to=<value optimized out>, 
    to_byte=<value optimized out>, dst_object=11030337) at coding.c:7307
#11 0x000000000059bc8e in read_process_output (proc=16927316, channel=153)
    at process.c:5409
#12 0x000000000059f4f1 in wait_reading_process_output (time_limit=30, 
    microsecs=0, read_kbd=-1, do_display=1, wait_for_cell=11030241, 
    wait_proc=0x0, just_wait_proc=0) at process.c:4987
#13 0x0000000000415645 in sit_for (timeout=240, reading=1, do_display=1)
    at dispnew.c:6637
#14 0x00000000004f9f75 in read_char (commandflag=1, nmaps=2, 
    maps=0x7fff80e2ecc0, prev_event=11030241, used_mouse_menu=0x7fff80e2edd4, 
    end_time=0x0) at keyboard.c:2892
#15 0x00000000004fb8dd in read_key_sequence (keybuf=0x7fff80e2ee60, 
    bufsize=30, prompt=11030241, dont_downcase_last=0, 
    can_return_switch_frame=1, fix_current_buffer=1) at keyboard.c:9343
#16 0x00000000004fd73a in command_loop_1 () at keyboard.c:1621
#17 0x00000000005608cf in internal_condition_case (
    bfun=0x4fd520 <command_loop_1>, handlers=11117457, 
    hfun=0x4f5dd0 <cmd_error>) at eval.c:1511
#18 0x00000000004f524e in command_loop_2 () at keyboard.c:1338
#19 0x00000000005609e7 in internal_catch (tag=<value optimized out>, 
    func=0x4f5230 <command_loop_2>, arg=11030241) at eval.c:1247
#20 0x00000000004f5c10 in command_loop () at keyboard.c:1317
#21 0x00000000004f601c in recursive_edit_1 () at keyboard.c:942
#22 0x00000000004f6194 in Frecursive_edit () at keyboard.c:1004
#23 0x00000000004eb057 in main (argc=2, argv=0x7fff80e2f678) at emacs.c:1786

bug#1650: 23.0.60; raw-text-dos memory corruption

[Jason Rumney]

Johan Bockgård wrote:
> I can reproduce the crash on Emacs versions after 2008-03-25, whereas I
> don't see the problem on versions before 2008-03-02. Intermediate
> versions don't crash, but instead hang and consume all memory.
>   

This seems to be the same as bug #1035, which you reported in September, 
and which I fixed yesterday.

> 2008-03-02  Kenichi Handa  <handa@m17n.org>
>
> 	* coding.c (decode_coding_utf_8): When eol-type of CODING is
> 	`dos', don't decode '\r' if that is the last in the source.
> 	(decode_coding_utf_16, decode_coding_emacs_mule)
> 	(decode_coding_iso_2022, decode_coding_sjis, decode_coding_big5)
> 	(decode_coding_raw_text, decode_coding_charset): Likewise.
> 	(produce_chars): Don't decode EOL here.  Use EMACS_INT.
>   

bug#1650: 23.0.60; raw-text-dos memory corruption

[Johan =?UTF-8?Q?Bockg=C3=A5rd]

Jason Rumney <jasonr@f2s.com> writes:

> Johan Bockgård wrote:
>> I can reproduce the crash on Emacs versions after 2008-03-25, whereas I
>> don't see the problem on versions before 2008-03-02. Intermediate
>> versions don't crash, but instead hang and consume all memory.
>>   
>
> This seems to be the same as bug #1035, which you reported in
> September, and which I fixed yesterday.

I thought so too; that's why I haven't reported this one earlier. But it
isn't--it still crashes in the same way even after your change (which I
did notice).

-- 
Johan Bockgård

bug#1650: 23.0.60; raw-text-dos memory corruption

[Chong Yidong]

> emacs -Q
>
>   ;; dictd is running on port 2628
>   (let* ((coding-system-for-read 'raw-text-dos)
>          (coding-system-for-write 'raw-text-dos)
>          (proc (open-network-stream "foo" "foo" "localhost" 2628)))
>     ;; The crash goes away if the next line is uncommented
>     ;; (sit-for .1)
>     (process-send-string proc "define * \"vice\"\r\n"))
>
> => Crash (backtrace below)

I can't reproduce this.  Do you still see the bug with latest CVS?

bug#1650: 23.0.60; raw-text-dos memory corruption

[Johan =?UTF-8?Q?Bockg=C3=A5rd]

Chong Yidong <cyd@stupidchicken.com> writes:

>> emacs -Q
>>
>>   ;; dictd is running on port 2628
>>   (let* ((coding-system-for-read 'raw-text-dos)
>>          (coding-system-for-write 'raw-text-dos)
>>          (proc (open-network-stream "foo" "foo" "localhost" 2628)))
>>     ;; The crash goes away if the next line is uncommented
>>     ;; (sit-for .1)
>>     (process-send-string proc "define * \"vice\"\r\n"))
>>
>> => Crash (backtrace below)
>
> I can't reproduce this.  Do you still see the bug with latest CVS?

Yes.

Note that a similar crash was reported in the following message:

Subject: core dump in malloc
http://lists.gnu.org/archive/html/emacs-devel/2009-02/msg00449.html

bug#1650: 23.0.60; raw-text-dos memory corruption

[Chong Yidong]

> emacs -Q
>
>   ;; dictd is running on port 2628
>   (let* ((coding-system-for-read 'raw-text-dos)
>          (coding-system-for-write 'raw-text-dos)
>          (proc (open-network-stream "foo" "foo" "localhost" 2628)))
>     ;; The crash goes away if the next line is uncommented
>     ;; (sit-for .1)
>     (process-send-string proc "define * \"vice\"\r\n"))
>
> => Crash (backtrace below)

I haven't been able to reproduce this.  Can anyone on this list
reproduce this?  If so, please try to debug it.

bug#1650: 23.0.60; raw-text-dos memory corruption

[Johan =?UTF-8?Q?Bockg=C3=A5rd]

Chong Yidong <cyd@stupidchicken.com> writes:

> I haven't been able to reproduce this. Can anyone on this list
> reproduce this? If so, please try to debug it.

Ok, I've found a rather minimal recipe.


Start one Emacs and evaluate the following:

    (make-network-process
     :name "foo" :server t
     :host "localhost" :service 8888
     :sentinel (lambda (proc stat)
                 (when (string-match "^open from" stat)
                   (process-send-string proc (make-string 5000 ?\r)))))

Start a second Emacs and run this:

    (let* ((coding-system-for-read 'raw-text-dos))
      (open-network-stream "foo" "foo" "localhost" 8888))

bug#1650: 23.0.60; raw-text-dos memory corruption

[Andreas Schwab]

I've checked in a fix.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#1650: marked as done (23.0.60; raw-text-dos memory corruption)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 874 bytes --]


Your message dated Sat, 02 May 2009 09:49:55 -0400
with message-id <878wlfaa2k.fsf@cyd.mit.edu>
and subject line Re: bug#1650: 23.0.60; raw-text-dos memory corruption
has caused the Emacs bug report #1650,
regarding 23.0.60; raw-text-dos memory corruption
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@emacsbugs.donarmstrong.com
immediately.)


-- 
1650: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=1650
Emacs Bug Tracking System
Contact owner@emacsbugs.donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 6726 bytes --]

From: bojohan+mail@dd.chalmers.se (Johan Bockgård)
To: emacs-pretest-bug@gnu.org
Subject: 23.0.60; raw-text-dos memory corruption
Date: Sat, 20 Dec 2008 16:55:54 +0100
Message-ID: <yoijiqpeu9h1.fsf@remote1.student.chalmers.se>


GNU Emacs 23.0.60.12 (x86_64-unknown-linux-gnu, GTK+ Version 2.14.4) of
2008-12-20

emacs -Q

  ;; dictd is running on port 2628
  (let* ((coding-system-for-read 'raw-text-dos)
         (coding-system-for-write 'raw-text-dos)
         (proc (open-network-stream "foo" "foo" "localhost" 2628)))
    ;; The crash goes away if the next line is uncommented
    ;; (sit-for .1)
    (process-send-string proc "define * \"vice\"\r\n"))

=> Crash (backtrace below)

I can reproduce the crash on Emacs versions after 2008-03-25, whereas I
don't see the problem on versions before 2008-03-02. Intermediate
versions don't crash, but instead hang and consume all memory.


2008-03-25  Stefan Monnier  <monnier@iro.umontreal.ca>

        [...]

	* process.h (struct Lisp_Process): Remove filter_multibyte.
	* process.c (QCfilter_multibyte): Remove.
	(setup_process_coding_systems): Don't use filter_multibyte.
	(Fstart_process, Fmake_network_process): Don't set filter_multibyte.
	(read_process_output): Don't adjust multibyteness to filter_multibyte.
	(Fset_process_filter_multibyte): Change the coding-system to
	approximate the previous behavior.
	(Fprocess_filter_multibyte_p): Get the multibyteness straight from the
	coding-system.

	* coding.c (decode_coding_object): When not decoding into a buffer,
	obey the coding system's preference of (uni|multi)byte.


2008-03-02  Kenichi Handa  <handa@m17n.org>

	* coding.c (decode_coding_utf_8): When eol-type of CODING is
	`dos', don't decode '\r' if that is the last in the source.
	(decode_coding_utf_16, decode_coding_emacs_mule)
	(decode_coding_iso_2022, decode_coding_sjis, decode_coding_big5)
	(decode_coding_raw_text, decode_coding_charset): Likewise.
	(produce_chars): Don't decode EOL here.  Use EMACS_INT.


*** glibc detected *** /home/bojohan/vc/emacs/src/emacs: malloc(): memory corruption: 0x00000000022f83e0 ***
======= Backtrace: =========
/lib/libc.so.6[0x7f4f73ed2cff]
/lib/libc.so.6(__libc_malloc+0x98)[0x7f4f73ed4538]
/home/bojohan/vc/emacs/src/emacs[0x5497ce]
[...]

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7f4f78df7770 (LWP 7357)]
0x00007f4f73e88fd5 in raise () from /lib/libc.so.6
(gdb) bt
#0  0x00007f4f73e88fd5 in raise () from /lib/libc.so.6
#1  0x00007f4f73e8ab43 in abort () from /lib/libc.so.6
#2  0x00007f4f73ec9fa8 in ?? () from /lib/libc.so.6
#3  0x00007f4f73ed2cff in ?? () from /lib/libc.so.6
#4  0x00007f4f73ed4538 in malloc () from /lib/libc.so.6
#5  0x00000000005497ce in lisp_malloc (nbytes=7357, type=7357) at alloc.c:861
#6  0x000000000054a09a in allocate_string_data (s=0xdc7510, nchars=8136, 
    nbytes=8136) at alloc.c:1991
#7  0x000000000054ab90 in make_uninit_multibyte_string (nchars=8136, 
    nbytes=8136) at alloc.c:2508
#8  0x000000000054ac87 in make_uninit_string (length=7357) at alloc.c:2486
#9  0x00000000005587fd in make_buffer_string_both (start=1, start_byte=1, 
    end=8137, end_byte=<value optimized out>, props=1) at editfns.c:2420
#10 0x0000000000481c35 in decode_coding_object (coding=0xab4800, 
    src_object=11030241, from=0, from_byte=0, to=<value optimized out>, 
    to_byte=<value optimized out>, dst_object=11030337) at coding.c:7307
#11 0x000000000059bc8e in read_process_output (proc=16927316, channel=153)
    at process.c:5409
#12 0x000000000059f4f1 in wait_reading_process_output (time_limit=30, 
    microsecs=0, read_kbd=-1, do_display=1, wait_for_cell=11030241, 
    wait_proc=0x0, just_wait_proc=0) at process.c:4987
#13 0x0000000000415645 in sit_for (timeout=240, reading=1, do_display=1)
    at dispnew.c:6637
#14 0x00000000004f9f75 in read_char (commandflag=1, nmaps=2, 
    maps=0x7fff80e2ecc0, prev_event=11030241, used_mouse_menu=0x7fff80e2edd4, 
    end_time=0x0) at keyboard.c:2892
#15 0x00000000004fb8dd in read_key_sequence (keybuf=0x7fff80e2ee60, 
    bufsize=30, prompt=11030241, dont_downcase_last=0, 
    can_return_switch_frame=1, fix_current_buffer=1) at keyboard.c:9343
#16 0x00000000004fd73a in command_loop_1 () at keyboard.c:1621
#17 0x00000000005608cf in internal_condition_case (
    bfun=0x4fd520 <command_loop_1>, handlers=11117457, 
    hfun=0x4f5dd0 <cmd_error>) at eval.c:1511
#18 0x00000000004f524e in command_loop_2 () at keyboard.c:1338
#19 0x00000000005609e7 in internal_catch (tag=<value optimized out>, 
    func=0x4f5230 <command_loop_2>, arg=11030241) at eval.c:1247
#20 0x00000000004f5c10 in command_loop () at keyboard.c:1317
#21 0x00000000004f601c in recursive_edit_1 () at keyboard.c:942
#22 0x00000000004f6194 in Frecursive_edit () at keyboard.c:1004
#23 0x00000000004eb057 in main (argc=2, argv=0x7fff80e2f678) at emacs.c:1786



[-- Attachment #3: Type: message/rfc822, Size: 1566 bytes --]

From: Chong Yidong <cyd@stupidchicken.com>
To: Andreas Schwab <schwab@linux-m68k.org>
Cc: bojohan+mail@dd.chalmers.se (Johan Bockgård), 1650-done@emacsbugs.donarmstrong.com, emacs-devel@gnu.org
Subject: Re: bug#1650: 23.0.60; raw-text-dos memory corruption
Date: Sat, 02 May 2009 09:49:55 -0400
Message-ID: <878wlfaa2k.fsf@cyd.mit.edu>

Andreas Schwab <schwab@linux-m68k.org> writes:

> I've checked in a fix.

Thanks.