Emacs 23.1.93 pretest

Emacs 23.1.93 pretest

[Chong Yidong]

Emacs pretest 23.1.93 is now available for download via FTP, at the
following location:

  ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-23.1.93.tar.gz

The xdelta against the Emacs 23.1.92 pretest is here:

  ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-23.1.92-23.1.93.xdelta

This is the fourth pretest for what will be the Emacs 23.2 release.

Developers: as explained in a previous email, in a few days we will cut
the 23.2 branch and open the trunk for Emacs 24 development.  Only
regression bugfixes and documentation changes will be accepted into the
branch.  So consider this the very last call for non-regression fixes:
if there is one that you would like to make, please discuss on
emacs-devel now.

Pretesters: please send me an email reporting success or failure on your
build platform.  Report bugs via M-x report-emacs-bugs, or email
emacs-pretest-bug@gnu.org.  For questions, email emacs-devel@gnu.org.

Thanks.

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Fri, 26 Feb 2010 22:40:05 -0500
> 
> Emacs pretest 23.1.93 is now available for download via FTP, at the
> following location:
> 
>   ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-23.1.93.tar.gz
> 
> The xdelta against the Emacs 23.1.92 pretest is here:
> 
>   ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-23.1.92-23.1.93.xdelta
> 
> This is the fourth pretest for what will be the Emacs 23.2 release.

It crashes for me on MS-Windows, while loading my desktop file.  I'm
trying to debug this.

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> Date: Sat, 27 Feb 2010 11:05:01 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > From: Chong Yidong <cyd@stupidchicken.com>
> > Date: Fri, 26 Feb 2010 22:40:05 -0500
> > 
> > Emacs pretest 23.1.93 is now available for download via FTP, at the
> > following location:
> > 
> >   ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-23.1.93.tar.gz
> > 
> > The xdelta against the Emacs 23.1.92 pretest is here:
> > 
> >   ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-23.1.92-23.1.93.xdelta
> > 
> > This is the fourth pretest for what will be the Emacs 23.2 release.
> 
> It crashes for me on MS-Windows, while loading my desktop file.  I'm
> trying to debug this.

The problem is this line from my desktop file:

  (setq regexp-search-ring '("[^]$" "\\bmac" "\\`n" "\\`m" "\\\\`ma" "\\. [^ ]" "^`" "^\\\\" "/[A-Za-z] " "/. ." " $" "/.*ş" "/.*\naba/f" "^" "/.*H" "[[^ ]*[^]] "))

There's a Latin-2 character `ş' there.  If I delete the "/.*ş" part of
the regexp-search-ring, Emacs starts correctly.  Emacs 23.1.92 loads
the same desktop file without any trouble, so this is a regression.

From this and other symptoms (see below), it looks like the emacs-mule
decoding is broken in this pretest.  For example, visiting the
offending desktop file with the Latin-2 character in place causes many
characters following this Latin-2 character to disappear from display,
as if they didn't exist in the file.  Killing the buffer and visiting
the file again magically restores the characters that disappeared.

I cannot produce a GDB backtrace: I could not find a way of
reproducing the crash under GDB.  All I get is a weird
wrong-type-argument error:

   Debugger entered--Lisp error: (wrong-type-argument symbolp (quote ((tab-width . 8) (buffer-file-coding-system . iso-latin-1-unix) (case-fold-search . t))))

But I think this is because the failure to decode the desktop file
correctly causes Emacs to try evaluating a butchered expression.

FWIW, here's the backtrace of the crash produced by Dr MinGW, a GIT
debugger that pops up when Emacs crashes.  Note that it crashes in
decode_coding_emacs_mule, yet another sign that this is the culprit.

    emacs.exe caused an Access Violation at location 0112b000 in module emacs.exe Reading from location 03194000.

    Registers:
    eax=03194000 ebx=00000000 ecx=03194000 edx=00002c50 esi=00000000 edi=0081572c
    eip=0112b000 esp=0080a550 ebp=0080a5b8 iopl=0         nv up ei ng nz ac pe cy
    cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000293

    Call stack:
    0112B000  emacs.exe:0112B000  decode_coding_emacs_mule  coding.c:2497

	    ...
	    c = byte_after_cr, byte_after_cr = -1;
		  else
    >       ONE_MORE_BYTE (c);

		  if (c < 0 || c == 0x80)
	    ...

    01134F4F  emacs.exe:01134F4F  decode_coding  coding.c:7158

	    ...
		  coding->charbuf_used = carryover;
		  (*(coding->decoder)) (coding);
    >             coding_set_destination (coding);
		  carryover = produce_chars (coding, translation_table, 0);
		  if (coding->annotated)
	    ...

    01138823  emacs.exe:01138823  decode_coding_gap  coding.c:7666

	    ...
	      current_buffer->text->inhibit_shrinking = 1;
	      decode_coding (coding);
    >         current_buffer->text->inhibit_shrinking = 0;

	      attrs = CODING_ID_ATTRS (coding->id);
	    ...

    0104E7B8  emacs.exe:0104E7B8  Finsert_file_contents  fileio.c:4083

	    ...
		  decode_coding_gap (&coding, inserted, inserted);
		  inserted = coding.produced_char;
    >             coding_system = CODING_ID_NAME (coding.id);
		}
	      else if (inserted > 0)
	    ...

    0100C5AD  emacs.exe:0100C5AD  Ffuncall  eval.c:3038

	    ...
	      goto done;
	    case 5:
    >         val = (*XSUBR (fun)->function) (internal_args[0], internal_args[1],
	      internal_args[2], internal_args[3],
	      internal_args[4]);
	    ...

    0111E0F6  emacs.exe:0111E0F6  Fbyte_code  bytecode.c:679

	    ...
		  }
	    #endif
    >           TOP = Ffuncall (op + 1, &TOP);
		AFTER_POTENTIAL_GC ();
		break;
	    ...

    0100C012  emacs.exe:0100C012  funcall_lambda  eval.c:3216

	    ...
		}

    >         return unbind_to (count, val);
	    }

	    ...

    0100C3F6  emacs.exe:0100C3F6  Ffuncall  eval.c:3093

	    ...
	     done:
	      CHECK_CONS_LIST ();
    >         lisp_eval_depth--;
	      if (backtrace.debug_on_exit)
		val = call_debugger (Fcons (Qexit, Fcons (val, Qnil)));
	    ...

    0100C78E  emacs.exe:0100C78E  call4  eval.c:2880

	    ...
	      RETURN_UNGCPRO (Ffuncall (5, &fn));
	    #endif /* not NO_ARG_ARRAY */
    >       }

	    /* Call function fn with 5 arguments arg1, arg2, arg3, arg4, arg5 */
	    ...

    0107AC20  emacs.exe:0107AC20  Fload  lread.c:1225

	    ...
		   NILP (noerror) ? Qnil : Qt,
		   (NILP (nomessage) || force_load_messages) ? Qnil : Qt);
    >         return unbind_to (count, val);
	    }
		}
	    ...

    0100C5AD  emacs.exe:0100C5AD  Ffuncall  eval.c:3038

	    ...
	      goto done;
	    case 5:
    >         val = (*XSUBR (fun)->function) (internal_args[0], internal_args[1],
	      internal_args[2], internal_args[3],
	      internal_args[4]);
	    ...

    0111E0F6  emacs.exe:0111E0F6  Fbyte_code  bytecode.c:679

	    ...
		  }
	    #endif
    >           TOP = Ffuncall (op + 1, &TOP);
		AFTER_POTENTIAL_GC ();
		break;
	    ...

    0100C012  emacs.exe:0100C012  funcall_lambda  eval.c:3216

	    ...
		}

    >         return unbind_to (count, val);
	    }

	    ...

    0100C3F6  emacs.exe:0100C3F6  Ffuncall  eval.c:3093

	    ...
	     done:
	      CHECK_CONS_LIST ();
    >         lisp_eval_depth--;
	      if (backtrace.debug_on_exit)
		val = call_debugger (Fcons (Qexit, Fcons (val, Qnil)));
	    ...

    0111E0F6  emacs.exe:0111E0F6  Fbyte_code  bytecode.c:679

	    ...
		  }
	    #endif
    >           TOP = Ffuncall (op + 1, &TOP);
		AFTER_POTENTIAL_GC ();
		break;
	    ...

    0100C012  emacs.exe:0100C012  funcall_lambda  eval.c:3216

	    ...
		}

    >         return unbind_to (count, val);
	    }

	    ...

    0100C3F6  emacs.exe:0100C3F6  Ffuncall  eval.c:3093

	    ...
	     done:
	      CHECK_CONS_LIST ();
    >         lisp_eval_depth--;
	      if (backtrace.debug_on_exit)
		val = call_debugger (Fcons (Qexit, Fcons (val, Qnil)));
	    ...

    0100CB8B  emacs.exe:0100CB8B  run_hook_with_args  eval.c:2683

	    ...
		{
		  args[0] = XCAR (val);
    >             ret = Ffuncall (nargs, args);
		}
	    }
	    ...

    0100CD1C  emacs.exe:0100CD1C  Frun_hooks  eval.c:2534

	    ...
	      register int i;

    >         for (i = 0; i < nargs; i++)
		{
		  hook[0] = args[i];
	    ...

    0100C6B4  emacs.exe:0100C6B4  Ffuncall  eval.c:3005

	    ...
		  if (XSUBR (fun)->max_args == MANY)
	    {
    >         val = (*XSUBR (fun)->function) (numargs, args + 1);
	      goto done;
	    }
	    ...

    0111E0F6  emacs.exe:0111E0F6  Fbyte_code  bytecode.c:679

	    ...
		  }
	    #endif
    >           TOP = Ffuncall (op + 1, &TOP);
		AFTER_POTENTIAL_GC ();
		break;
	    ...

    0100C012  emacs.exe:0100C012  funcall_lambda  eval.c:3216

	    ...
		}

    >         return unbind_to (count, val);
	    }

	    ...

    0100C3F6  emacs.exe:0100C3F6  Ffuncall  eval.c:3093

	    ...
	     done:
	      CHECK_CONS_LIST ();
    >         lisp_eval_depth--;
	      if (backtrace.debug_on_exit)
		val = call_debugger (Fcons (Qexit, Fcons (val, Qnil)));
	    ...

    0111E0F6  emacs.exe:0111E0F6  Fbyte_code  bytecode.c:679

	    ...
		  }
	    #endif
    >           TOP = Ffuncall (op + 1, &TOP);
		AFTER_POTENTIAL_GC ();
		break;
	    ...

    0100C012  emacs.exe:0100C012  funcall_lambda  eval.c:3216

	    ...
		}

    >         return unbind_to (count, val);
	    }

	    ...

    0100D1AD  emacs.exe:0100D1AD  apply_lambda  eval.c:3135

	    ...
		}
	      backtrace_list->evalargs = 0;
    >         tem = funcall_lambda (fun, XINT (numargs), arg_vector);

	      /* Do the debug-on-exit now, while arg_vector still exists.  */
	    ...

    0100B906  emacs.exe:0100B906  Feval  eval.c:2413

	    ...
	      CHECK_CONS_LIST ();

    >         lisp_eval_depth--;
	      if (backtrace.debug_on_exit)
		val = call_debugger (Fcons (Qexit, Fcons (val, Qnil)));
	    ...

    01053927  emacs.exe:01053927  top_level_2  keyboard.c:1370

	    ...
	    {
	      return Feval (Vtop_level);
    >       }

	    Lisp_Object
	    ...

    0100A16A  emacs.exe:0100A16A  internal_condition_case  eval.c:1491

	    ...

	      val = (*bfun) ();
    >         catchlist = c.next;
	      handlerlist = h.next;
	      return val;
	    ...

    01053959  emacs.exe:01053959  top_level_1  keyboard.c:1382

	    ...
	      else
		message ("Bare Emacs (standard Lisp code not loaded)");
    >         return Qnil;
	    }

	    ...

    0100A09F  emacs.exe:0100A09F  internal_catch  eval.c:1226

	    ...
	      /* Call FUNC.  */
	      if (! _setjmp (c.jmp))
    >           c.val = (*func) (arg);

	      /* Throw works by a longjmp that comes right here.  */
	    ...

    010536F9  emacs.exe:010536F9  command_loop  keyboard.c:1339

	    ...
		    any_kboard_state ();
	    #endif
    >       internal_catch (Qtop_level, command_loop_2, Qnil);
	    executing_kbd_macro = Qnil;

	    ...

    010537B0  emacs.exe:010537B0  recursive_edit_1  keyboard.c:955

	    ...

	      val = command_loop ();
    >         if (EQ (val, Qt))
		Fsignal (Qquit, Qnil);
	      /* Handle throw from read_minibuf when using minibuffer
	    ...

    010538D1  emacs.exe:010538D1  Frecursive_edit  keyboard.c:1017

	    ...

	      recursive_edit_1 ();
    >         return unbind_to (count, Qnil);
	    }

	    ...

    01002E2F  emacs.exe:01002E2F  main  emacs.c:1836

	    ...
	      /* NOTREACHED */
	      return 0;
    >       }

	    /* Sort the args so we can find the most important ones
	    ...

    0100124B  emacs.exe:0100124B
    01001298  emacs.exe:01001298
    7C816D4F  kernel32.dll:7C816D4F  RegisterWaitForInputIdle

Re: Emacs 23.1.93 pretest

[Juanma Barranquero]

2010/2/27 Eli Zaretskii <eliz@gnu.org>:

> I cannot produce a GDB backtrace: I could not find a way of
> reproducing the crash under GDB.  All I get is a weird
> wrong-type-argument error:

This is an optimized build, but I can get a crash by doing C-h h. I
get this backtrace:

Breakpoint 1, w32_abort () at w32fns.c:7345
7345      button = MessageBox (NULL,
(gdb) bt
#0  w32_abort () at w32fns.c:7345
#1  0x0101ef48 in die (msg=0x14c453c "assertion failed:
CONSP((rest))", file=0x14c3bb8 "w32uniscribe.c", line=676) at
alloc.c:6259
#2  0x01228124 in uniscribe_check_otf (font=0x88ca60,
otf_spec=20161510) at w32uniscribe.c:676
#3  0x012241cb in font_matches_spec (logical_font=0x88ca60,
physical_font=0x88c878, font_type=4, lParam=8965156) at w32font.c:1313
#4  add_font_entity_to_list (logical_font=0x88ca60,
physical_font=0x88c878, font_type=4, lParam=8965156) at w32font.c:1428
#5  0x77561f91 in GDI32!D3DKMTGetDeviceState () from
C:\Windows\syswow64\gdi32.dll
#6  0x0088ca60 in ?? ()
#7  0x0088c878 in ?? ()
#8  0x00000004 in ?? ()
#9  0x0088cc24 in ?? ()
#10 0x00cb7668 in ?? ()
#11 0x00cb7668 in ?? ()
#12 0x00000001 in ?? ()
#13 0x00000024 in ?? ()
#14 0x0000001a in ?? ()
#15 0x0000000a in ?? ()
#16 0x00000004 in ?? ()
#17 0x00000000 in ?? ()

I'll do an unoptimized build and report back.

    Juanma

Re: Emacs 23.1.93 pretest

[Juanma Barranquero]

On Sat, Feb 27, 2010 at 12:28, Juanma Barranquero <lekktu@gmail.com> wrote:

> I'll do an unoptimized build and report back.

Same result. Transcript follows.

    Juanma


w32uniscribe.c:676: Emacs fatal error: assertion failed: CONSP((rest))

Breakpoint 1, w32_abort () at w32fns.c:7345
7345      button = MessageBox (NULL,
(gdb) bt
#0  w32_abort () at w32fns.c:7345
#1  0x0104d987 in die (msg=0x1663908 "assertion failed:
CONSP((rest))", file=0x1662008 "w32uniscribe.c", line=676) at
alloc.c:6259
#2  0x01328ac7 in uniscribe_check_otf (font=0x88d630,
otf_spec=21217310) at w32uniscribe.c:676
#3  0x01321769 in font_matches_spec (type=4, font=0x88d448,
spec=48831493, backend=48927794, logfont=0x88d630) at w32font.c:1313
#4  0x01321a43 in add_font_entity_to_list (logical_font=0x88d630,
physical_font=0x88d448, font_type=4, lParam=8968168) at w32font.c:1428
#5  0x77561f91 in GDI32!D3DKMTGetDeviceState () from
C:\Windows\syswow64\gdi32.dll
#6  0x0088d630 in ?? ()
#7  0x0088d448 in ?? ()
#8  0x00000004 in ?? ()
#9  0x0088d7e8 in ?? ()
#10 0x00978d18 in ?? ()
#11 0x00978d18 in ?? ()
#12 0x00000001 in ?? ()
#13 0x00000024 in ?? ()
#14 0x0000001a in ?? ()
#15 0x0000000a in ?? ()
#16 0x00000004 in ?? ()
#17 0x00000000 in ?? ()
(gdb) frame 2
#2  0x01328ac7 in uniscribe_check_otf (font=0x88d630,
otf_spec=21217310) at w32uniscribe.c:676
676       lang = XCAR (rest);
(gdb) p *font
$1 = {
  lfHeight = 36,
  lfWidth = 19,
  lfEscapement = 0,
  lfOrientation = 0,
  lfWeight = 400,
  lfItalic = 0 '\000',
  lfUnderline = 0 '\000',
  lfStrikeOut = 0 '\000',
  lfCharSet = 0 '\000',
  lfOutPrecision = 3 '\003',
  lfClipPrecision = 2 '\002',
  lfQuality = 1 '\001',
  lfPitchAndFamily = 49 '1',
  lfFaceName = "Courier
New\000xÖ\210\000)ö\020\001\000\000\000\000\001\000\000\000T\000\000"
}
(gdb) p otf_spec
$2 = 21217310
(gdb) pr
(mymr)
(gdb) frame 3
#3  0x01321769 in font_matches_spec (type=4, font=0x88d448,
spec=48831493, backend=48927794, logfont=0x88d630) at w32font.c:1313
1313                      if (!uniscribe_check_otf (logfont, val))
(gdb) p *font
$3 = {
  ntmTm = {
    tmHeight = 36,
    tmAscent = 26,
    tmDescent = 10,
    tmInternalLeading = 4,
    tmExternalLeading = 0,
    tmAveCharWidth = 19,
    tmMaxCharWidth = 24,
    tmWeight = 400,
    tmOverhang = 0,
    tmDigitizedAspectX = 96,
    tmDigitizedAspectY = 96,
    tmFirstChar = 30 '\036',
    tmLastChar = 255 '\377',
    tmDefaultChar = 31 '\037',
    tmBreakChar = 32 ' ',
    tmItalic = 0 '\000',
    tmUnderlined = 0 '\000',
    tmStruckOut = 0 '\000',
    tmPitchAndFamily = 54 '6',
    tmCharSet = 0 '\000',
    ntmFlags = 2359360,
    ntmSizeEM = 2048,
    ntmCellHeight = 2320,
    ntmAvgWidth = 1229
  },
  ntmFontSig = {
    fsUsb = {3758107391, 3221256259, 9, 0},
    fsCsb = {1073742335, 4294901760}
  }
}
(gdb) p spec
$4 = 48831493
(gdb) pr
#<font-spec uniscribe outline Courier\ New mono iso10646-1 nil nil nil
nil nil nil nil ((:otf mymr))>
(gdb) p backend
$5 = 48927794
(gdb) pr
uniscribe
(gdb) p logfont
$6 = (LOGFONT *) 0x88d630
(gdb) p *logfont
$7 = {
  lfHeight = 36,
  lfWidth = 19,
  lfEscapement = 0,
  lfOrientation = 0,
  lfWeight = 400,
  lfItalic = 0 '\000',
  lfUnderline = 0 '\000',
  lfStrikeOut = 0 '\000',
  lfCharSet = 0 '\000',
  lfOutPrecision = 3 '\003',
  lfClipPrecision = 2 '\002',
  lfQuality = 1 '\001',
  lfPitchAndFamily = 49 '1',
  lfFaceName = "Courier
New\000xÖ\210\000)ö\020\001\000\000\000\000\001\000\000\000T\000\000"
}
(gdb) frame 4
#4  0x01321a43 in add_font_entity_to_list (logical_font=0x88d630,
physical_font=0x88d448, font_type=4, lParam=8968168) at w32font.c:1428
1428          || !font_matches_spec (font_type, physical_font,
(gdb) p *logical_font
$8 = {
  elfLogFont = {
    lfHeight = 36,
    lfWidth = 19,
    lfEscapement = 0,
    lfOrientation = 0,
    lfWeight = 400,
    lfItalic = 0 '\000',
    lfUnderline = 0 '\000',
    lfStrikeOut = 0 '\000',
    lfCharSet = 0 '\000',
    lfOutPrecision = 3 '\003',
    lfClipPrecision = 2 '\002',
    lfQuality = 1 '\001',
    lfPitchAndFamily = 49 '1',
    lfFaceName = "Courier
New\000xÖ\210\000)ö\020\001\000\000\000\000\001\000\000\000T\000\000"
  },
  elfFullName = "Courier New\000\v\000\000\000b\000\000@\v", '\000'
<repeats 25 times>, "•\000\000\000\000\000èÕ\210\000Ñ<'\003Ä\377\210",
  elfStyle = "Normal\000\000þ\377\377\377#;ÑwN;ÑwH@\000\000\000\000\000\000\000\000\000",
  elfScript = "Occidental\000\000\001\000\000\000$Ö\210\000;\f\000\000Ä\377\210\000\035\004Õw"
}
(gdb) p *physical_font
$9 = {
  ntmTm = {
    tmHeight = 36,
    tmAscent = 26,
    tmDescent = 10,
    tmInternalLeading = 4,
    tmExternalLeading = 0,
    tmAveCharWidth = 19,
    tmMaxCharWidth = 24,
    tmWeight = 400,
    tmOverhang = 0,
    tmDigitizedAspectX = 96,
    tmDigitizedAspectY = 96,
    tmFirstChar = 30 '\036',
    tmLastChar = 255 '\377',
    tmDefaultChar = 31 '\037',
    tmBreakChar = 32 ' ',
    tmItalic = 0 '\000',
    tmUnderlined = 0 '\000',
    tmStruckOut = 0 '\000',
    tmPitchAndFamily = 54 '6',
    tmCharSet = 0 '\000',
    ntmFlags = 2359360,
    ntmSizeEM = 2048,
    ntmCellHeight = 2320,
    ntmAvgWidth = 1229
  },
  ntmFontSig = {
    fsUsb = {3758107391, 3221256259, 9, 0},
    fsCsb = {1073742335, 4294901760}
  }
}
(gdb) p lParam
$10 = 8968168

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Sat, 27 Feb 2010 13:11:45 +0100
> Cc: cyd@stupidchicken.com, emacs-devel@gnu.org
> 
> w32uniscribe.c:676: Emacs fatal error: assertion failed: CONSP((rest))
> 
> Breakpoint 1, w32_abort () at w32fns.c:7345
> 7345      button = MessageBox (NULL,
> (gdb) bt
> #0  w32_abort () at w32fns.c:7345
> #1  0x0104d987 in die (msg=0x1663908 "assertion failed:
> CONSP((rest))", file=0x1662008 "w32uniscribe.c", line=676) at
> alloc.c:6259
> #2  0x01328ac7 in uniscribe_check_otf (font=0x88d630,
> otf_spec=21217310) at w32uniscribe.c:676
> #3  0x01321769 in font_matches_spec (type=4, font=0x88d448,
> spec=48831493, backend=48927794, logfont=0x88d630) at w32font.c:1313
> #4  0x01321a43 in add_font_entity_to_list (logical_font=0x88d630,
> physical_font=0x88d448, font_type=4, lParam=8968168) at w32font.c:1428
> #5  0x77561f91 in GDI32!D3DKMTGetDeviceState () from
> C:\Windows\syswow64\gdi32.dll
> #6  0x0088d630 in ?? ()
> #7  0x0088d448 in ?? ()
> #8  0x00000004 in ?? ()
> #9  0x0088d7e8 in ?? ()
> #10 0x00978d18 in ?? ()
> #11 0x00978d18 in ?? ()
> #12 0x00000001 in ?? ()
> #13 0x00000024 in ?? ()
> #14 0x0000001a in ?? ()
> #15 0x0000000a in ?? ()
> #16 0x00000004 in ?? ()
> #17 0x00000000 in ?? ()

Thanks.  I get a similar crash, but it isn't an assert violation.

  Program received signal SIGSEGV, Segmentation fault.
  0x0118191d in uniscribe_check_otf (font=0x3, otf_spec=44922882)
      at w32uniscribe.c:684
  684         features[1] = XCAR (rest);

otf_spec is nil in my case:

  (gdb) p otf_spec
  $1 = 44922882
  (gdb) xtype
  Lisp_Symbol
  (gdb) xsymbol
  $2 = (struct Lisp_Symbol *) 0x2ad7800
  "nil"

But then how come this defense in uniscribe_check_otf did not catch
it?

  /* Check the spec is in the right format.  */
  if (!CONSP (otf_spec) || Flength (otf_spec) < 3)
    return 0;

Some GC perhaps?

See below for the full backtrace.

Btw, what version of GDB are you using?  It seems like it has some
trouble showing a useful backtrace.  My GDB is 7.0; if you have
anything older, I recommend installing 7.0 from the MinGW site.

Here's my backtrace:

  Program received signal SIGSEGV, Segmentation fault.
  0x0118191d in uniscribe_check_otf (font=0x3, otf_spec=44922882)
      at w32uniscribe.c:684
  684         features[1] = XCAR (rest);
  (gdb) bt
  #0  0x0118191d in uniscribe_check_otf (font=0x3, otf_spec=44922882)
      at w32uniscribe.c:684
  #1  0x0117d640 in add_font_entity_to_list (logical_font=0x82ca7c,
      physical_font=0x82c894, font_type=4, lParam=8571968) at w32font.c:1313
  #2  0x77f3afb1 in GDI32!CreateFontA () from C:\WINDOWS\system32\gdi32.dll
  #3  0x77f345be in GetOutlineTextMetricsA () from C:\WINDOWS\system32\gdi32.dll
  #4  0x77f29f0b in GDI32!EnumFontFamiliesA ()
     from C:\WINDOWS\system32\gdi32.dll
  #5  0x77f3ead3 in GDI32!EnumFontFamiliesExA ()
     from C:\WINDOWS\system32\gdi32.dll
  #6  0x0117e5e3 in w32font_list_internal (frame=-8, font_spec=8806376,
      opentype_only=1) at w32font.c:760
  #7  0x01180476 in uniscribe_list (frame=3, font_spec=45038597)
      at w32uniscribe.c:81
  #8  0x010abe27 in font_list_entities (frame=50487301, spec=54573445)
      at font.c:2926
  #9  0x010ac51c in font_find_for_lface (f=0x3003922, attrs=0x307f740,
      spec=45035776, c=-1) at font.c:3487
  #10 0x010d6bb4 in fontset_find_font (fontset=50521093, c=4121,
      face=0x307f700, id=143, fallback=0) at fontset.c:636
  #11 0x010d7387 in fontset_font (fontset=50523141, c=4121, face=0x307f700,
      id=143) at fontset.c:756
  #12 0x010d7971 in font_for_char (face=0x307f700, c=4121, pos=49053658,
      object=3) at fontset.c:1049
  #13 0x010a8fa1 in font_range (pos=417, limit=0x82cfbc, w=0x1019,
      face=0x307f700, string=44922882) at font.c:3997
  #14 0x010e1206 in autocmp_chars (cft_element=417, charpos=416, bytepos=664,
      limit=432, win=0x3026c00, face=0x3, string=44922882) at composite.c:973
  #15 0x010e2722 in composition_reseat_it (cmp_it=0x82deb8, charpos=416,
      bytepos=664, endpos=3131, w=0x3026c00, face=0x3, string=44922882)
      at composite.c:1147
  #16 0x0101a59e in next_element_from_buffer (it=0x82db00) at xdisp.c:6532
  #17 0x01018552 in get_next_display_element (it=0x82db00) at xdisp.c:5675
  #18 0x0102397d in display_line (it=0x82db00) at xdisp.c:16570
  #19 0x01024ccd in try_window (window=50490373, pos=..., check_margins=1)
      at xdisp.c:14028
  #20 0x0102d5fd in redisplay_window (window=50490373, just_this_one_p=0)
      at xdisp.c:13651
  #21 0x0102e937 in redisplay_window_0 (window=50490373) at xdisp.c:12283
  #22 0x01009ed5 in internal_condition_case_1 (
      bfun=0x102e90d <redisplay_window_0>, arg=50490373, handlers=44907174,
      hfun=0x101e27c <redisplay_window_error>) at eval.c:1538
  #23 0x0101e049 in redisplay_windows (window=49053658) at xdisp.c:12262
  #24 0x01030765 in redisplay_internal (preserve_echo_area=3) at xdisp.c:11834
  #25 0x0105bed2 in read_char (commandflag=1, nmaps=3, maps=0x82fae0,
      prev_event=44922882, used_mouse_menu=0x82fb78, end_time=0x0)
      at keyboard.c:2727
  #26 0x0105e332 in read_key_sequence (keybuf=0x82fcb0, bufsize=30,
      prompt=44922882, dont_downcase_last=0, can_return_switch_frame=1,
      fix_current_buffer=1) at keyboard.c:9512
  #27 0x01060377 in command_loop_1 () at keyboard.c:1643
  #28 0x0100a16a in internal_condition_case (bfun=0x10601cd <command_loop_1>,
      handlers=44979418, hfun=0x1059fa1 <cmd_error>) at eval.c:1490
  #29 0x0105390a in command_loop_2 () at keyboard.c:1360
  #30 0x0100a09f in internal_catch (tag=3, func=0x10538e7 <command_loop_2>,
      arg=44922882) at eval.c:1226
  #31 0x01053717 in command_loop () at keyboard.c:1339
  #32 0x010537b0 in recursive_edit_1 () at keyboard.c:954
  #33 0x010538d1 in Frecursive_edit () at keyboard.c:1016
  #34 0x01002e2f in main (argc=2, argv=0xa32770) at emacs.c:1833
  (gdb)

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> Date: Sat, 27 Feb 2010 15:15:35 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: cyd@stupidchicken.com, emacs-devel@gnu.org
> 
> Thanks.  I get a similar crash, but it isn't an assert violation.
> 
>   Program received signal SIGSEGV, Segmentation fault.
>   0x0118191d in uniscribe_check_otf (font=0x3, otf_spec=44922882)
>       at w32uniscribe.c:684
>   684         features[1] = XCAR (rest);
> 
> otf_spec is nil in my case:
> 
>   (gdb) p otf_spec
>   $1 = 44922882
>   (gdb) xtype
>   Lisp_Symbol
>   (gdb) xsymbol
>   $2 = (struct Lisp_Symbol *) 0x2ad7800
>   "nil"

This sounds like a separate problem, though: it did not exist in
revision 99563 (from yesterday's noon [UTC+2]), but exists in the
current revision 99569 and in the pretest.  By contrast, the problem
with emacs-mule decoding exists in both revisions.

> But then how come this defense in uniscribe_check_otf did not catch
> it?
> 
>   /* Check the spec is in the right format.  */
>   if (!CONSP (otf_spec) || Flength (otf_spec) < 3)
>     return 0;

Answering my own question here: because otf_spec is a cons cell of the
right length, but it looks not according to what uniscribe_check_otf
expects:

    (gdb) p otf_spec
    $7 = 19320654
    (gdb) xcons
    $8 = (struct Lisp_Cons *) 0x126cf48
    {
      car = 0x2ec7fda,
      u = {
	cdr = 0x2ad7802,
	chain = 0x2ad7802
      }
    }
    (gdb) p otf_spec
    $9 = 19320654
    (gdb) xcdr
    $10 = 0x2ad7802
    (gdb) xcdr
    $11 = 0x0

The change which causes it is this:

  === modified file 'lisp/international/fontset.el'
  --- lisp/international/fontset.el       2010-01-13 08:35:10 +0000
  +++ lisp/international/fontset.el       2010-02-27 13:55:36 +0000
  @@ -415,6 +415,9 @@
	(sinhala ,(font-spec :registry "iso10646-1" :otf '(sinh nil (akhn))))
	(malayalam ,(font-spec :registry "iso10646-1" :otf '(mlym nil (akhn))))

  +     (myanmar ,(font-spec :registry "iso10646-1" :otf '(mymr))
  +             ,(font-spec :registry "iso10646-1" :script 'myanmar))
  +
	(lao ,(font-spec :registry "iso10646-1" :otf '(lao\  nil nil (mark)))
	    ,(font-spec :registry "iso10646-1" :script 'lao)
	    (nil . "MuleLao-1"))
  @@ -548,7 +551,6 @@
		      armenian
		      syriac
		      thaana
  -                   myanmar
		      georgian
		      cherokee
		      canadian-aboriginal

Perhaps Handa-san could take a look at this.  The entry you added
seems to violate the doc string of `font-spec', which says:

    `:otf'

    VALUE must be a list (SCRIPT-TAG LANGSYS-TAG GSUB [ GPOS ]) to specify
    required OpenType features.

      SCRIPT-TAG: OpenType script tag symbol (e.g. `deva').
      LANGSYS-TAG: OpenType language system tag symbol,
	 or nil for the default language system.
      GSUB: List of OpenType GSUB feature tag symbols, or nil if none required.
      GPOS: List of OpenType GPOS feature tag symbols, or nil if none required.

Btw, the `:otf' spec does not seem to be documented in the ELisp manual.

Re: Emacs 23.1.93 pretest

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> But then how come this defense in uniscribe_check_otf did not catch
> it?
>
>   /* Check the spec is in the right format.  */
>   if (!CONSP (otf_spec) || Flength (otf_spec) < 3)
>     return 0;

Because `Flength (otf_spec) < 3' is bogus.  It should be `XINT (Flength
(otf_spec)) < 3'.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: Juanma Barranquero <lekktu@gmail.com>,  cyd@stupidchicken.com,  emacs-devel@gnu.org
> Date: Sat, 27 Feb 2010 15:31:30 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > But then how come this defense in uniscribe_check_otf did not catch
> > it?
> >
> >   /* Check the spec is in the right format.  */
> >   if (!CONSP (otf_spec) || Flength (otf_spec) < 3)
> >     return 0;
> 
> Because `Flength (otf_spec) < 3' is bogus.  It should be `XINT (Flength
> (otf_spec)) < 3'.

Right you are.

Re: Emacs 23.1.93 pretest

[Lennart Borgman]

On Sat, Feb 27, 2010 at 3:54 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Cc: Juanma Barranquero <lekktu@gmail.com>,  cyd@stupidchicken.com,  emacs-devel@gnu.org
>> Date: Sat, 27 Feb 2010 15:31:30 +0100
>>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> > But then how come this defense in uniscribe_check_otf did not catch
>> > it?
>> >
>> >   /* Check the spec is in the right format.  */
>> >   if (!CONSP (otf_spec) || Flength (otf_spec) < 3)
>> >     return 0;
>>
>> Because `Flength (otf_spec) < 3' is bogus.  It should be `XINT (Flength
>> (otf_spec)) < 3'.
>
> Right you are.


Are you checking in a change? Then could you perhaps tell me so I can
build from the Launchpad mirror when the change has arrived there?

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> From: Lennart Borgman <lennart.borgman@gmail.com>
> Date: Sat, 27 Feb 2010 15:59:07 +0100
> Cc: lekktu@gmail.com, cyd@stupidchicken.com,
> 	Andreas Schwab <schwab@linux-m68k.org>, emacs-devel@gnu.org
> 
> On Sat, Feb 27, 2010 at 3:54 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> >> From: Andreas Schwab <schwab@linux-m68k.org>
> >> Cc: Juanma Barranquero <lekktu@gmail.com>,  cyd@stupidchicken.com,  emacs-devel@gnu.org
> >> Date: Sat, 27 Feb 2010 15:31:30 +0100
> >>
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >>
> >> > But then how come this defense in uniscribe_check_otf did not catch
> >> > it?
> >> >
> >> >   /* Check the spec is in the right format.  */
> >> >   if (!CONSP (otf_spec) || Flength (otf_spec) < 3)
> >> >     return 0;
> >>
> >> Because `Flength (otf_spec) < 3' is bogus.  It should be `XINT (Flength
> >> (otf_spec)) < 3'.
> >
> > Right you are.
> 
> 
> Are you checking in a change?

Andreas already did.

Re: Emacs 23.1.93 pretest

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

>   +     (myanmar ,(font-spec :registry "iso10646-1" :otf '(mymr))
>   +             ,(font-spec :registry "iso10646-1" :script 'myanmar))
>   +
>
> Perhaps Handa-san could take a look at this.  The entry you added
> seems to violate the doc string of `font-spec'

I've checked in a fix for the :otf spec.

> Btw, the `:otf' spec does not seem to be documented in the ELisp manual.

I've just added it to display.texi.

It's unfortunate that the pretest has an easily triggerable crash on
Windows, but that's life.  We should probably have another pretest soon,
for the benefit of Windows testers---let's say Monday, March 8th, a bit
more than a week from now.

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Cc: Juanma Barranquero <lekktu@gmail.com>, emacs-devel@gnu.org
> Date: Sat, 27 Feb 2010 10:22:56 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >   +     (myanmar ,(font-spec :registry "iso10646-1" :otf '(mymr))
> >   +             ,(font-spec :registry "iso10646-1" :script 'myanmar))
> >   +
> >
> > Perhaps Handa-san could take a look at this.  The entry you added
> > seems to violate the doc string of `font-spec'
> 
> I've checked in a fix for the :otf spec.
> 
> > Btw, the `:otf' spec does not seem to be documented in the ELisp manual.
> 
> I've just added it to display.texi.

Thanks.

> It's unfortunate that the pretest has an easily triggerable crash on
> Windows, but that's life.  We should probably have another pretest soon,
> for the benefit of Windows testers---let's say Monday, March 8th, a bit
> more than a week from now.

I would suggest to wait until the problem with decoding emacs-mule is
fixed as well.

Re: Emacs 23.1.93 pretest

[Kenichi Handa]

In article <87tyt21z5b.fsf@stupidchicken.com>, Chong Yidong <cyd@stupidchicken.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
> >   +     (myanmar ,(font-spec :registry "iso10646-1" :otf '(mymr))
> >   +             ,(font-spec :registry "iso10646-1" :script 'myanmar))
> >   +
> >
> > Perhaps Handa-san could take a look at this.  The entry you added
> > seems to violate the doc string of `font-spec'

> I've checked in a fix for the :otf spec.

Thank you, but I changed the spec to ":otf '(mymr nil nil))"
because the only Burmese OTF font I know is from SIL and it
doesn't contain "LangSys" "brm" nor "GPOS" "liga".

Do you know any Burmese OTF fonts that matche with the spec
"(mymr brm (liga mark))"?

---
Kenichi Handa
handa@m17n.org

Re: Emacs 23.1.93 pretest

[Jason Rumney]

Kenichi Handa <handa@m17n.org> writes:

> Thank you, but I changed the spec to ":otf '(mymr nil nil))"
> because the only Burmese OTF font I know is from SIL and it
> doesn't contain "LangSys" "brm" nor "GPOS" "liga".

There is another here:

http://www.myanmarnlp.net.mm/opentype.htm

But I don't know which LangSys and GPOS info it contains (I had it
installed while developing the uniscribe backend on Windows, but not on
my current GNU/Linux system.

Re: Emacs 23.1.93 pretest

[Juanma Barranquero]

On Sat, Feb 27, 2010 at 14:15, Eli Zaretskii <eliz@gnu.org> wrote:

> Btw, what version of GDB are you using?  It seems like it has some
> trouble showing a useful backtrace.  My GDB is 7.0; if you have
> anything older, I recommend installing 7.0 from the MinGW site.

7.0.1. Apparently, it does not work very well on 64-bit Windows 7.

    Juanma

Re: Emacs 23.1.93 pretest

[Stefan Monnier]

> But then how come this defense in uniscribe_check_otf did not catch
> it?

>   /* Check the spec is in the right format.  */
>   if (!CONSP (otf_spec) || Flength (otf_spec) < 3)
>     return 0;

Once again I recommend that people compile with -DUSE_LISP_UNION_TYPE to
catch such bugs.  They're trivial to catch this way.


        Stefan

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> Date: Sat, 27 Feb 2010 12:21:03 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 
> 
> > Date: Sat, 27 Feb 2010 11:05:01 +0200
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: emacs-devel@gnu.org
> > 
> For example, visiting the offending desktop file with the Latin-2
> character in place causes many characters following this Latin-2
> character to disappear from display, as if they didn't exist in the
> file.

In the MS-DOS port, it actually displays 15450 null characters instead
of the real characters in the file.

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> Date: Sat, 27 Feb 2010 13:57:36 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 
> 
> > Date: Sat, 27 Feb 2010 12:21:03 +0200
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: 
> > 
> > > Date: Sat, 27 Feb 2010 11:05:01 +0200
> > > From: Eli Zaretskii <eliz@gnu.org>
> > > Cc: emacs-devel@gnu.org
> > > 
> > For example, visiting the offending desktop file with the Latin-2
> > character in place causes many characters following this Latin-2
> > character to disappear from display, as if they didn't exist in the
> > file.
> 
> In the MS-DOS port, it actually displays 15450 null characters instead
> of the real characters in the file.

"bzr bisect" points to this change as the reason for this bug:

    2010-02-05  Chong Yidong  <cyd@stupidchicken.com>

	    * charset.c (load_charset_map_from_file): Allocate large
	    charset_map_entries structure on the heap rather than the stack.
	    (Bug#5526).

The revisions before this change works correctly; all revisions after
it fail as described above.

Any ideas?

Re: Emacs 23.1.93 pretest

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

> "bzr bisect" points to this change as the reason for this bug:
>
>     2010-02-05  Chong Yidong  <cyd@stupidchicken.com>
>
> 	    * charset.c (load_charset_map_from_file): Allocate large
> 	    charset_map_entries structure on the heap rather than the stack.
> 	    (Bug#5526).
>
> The revisions before this change works correctly; all revisions after
> it fail as described above.

Hmm, this is strange.  This change (actually the succeeding 2010-02-06
change to the same place) switches from using alloca to SAFE_ALLOCA
(i.e. malloc, since the desired structure is large).  But the only way I
can see for this code to crash is if load_charset_map somehow makes a
pointer into the allocated structure.  But in that case, the old alloca
case should have crashed too.

If you remove the SAFE_FREE () calls, does that prevent the crash?

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Cc: emacs-devel@gnu.org
> Date: Sat, 27 Feb 2010 16:37:47 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > "bzr bisect" points to this change as the reason for this bug:
> >
> >     2010-02-05  Chong Yidong  <cyd@stupidchicken.com>
> >
> > 	    * charset.c (load_charset_map_from_file): Allocate large
> > 	    charset_map_entries structure on the heap rather than the stack.
> > 	    (Bug#5526).
> >
> > The revisions before this change works correctly; all revisions after
> > it fail as described above.
> 
> Hmm, this is strange.  This change (actually the succeeding 2010-02-06
> change to the same place) switches from using alloca to SAFE_ALLOCA
> (i.e. malloc, since the desired structure is large).  But the only way I
> can see for this code to crash is if load_charset_map somehow makes a
> pointer into the allocated structure.  But in that case, the old alloca
> case should have crashed too.

Yes, it _is_ weird.  But the effect (see below) does look like we are
freeing memory being used, or maybe overwriting some allocated buffer,
or in some other way thrashing the arena.

> If you remove the SAFE_FREE () calls, does that prevent the crash?

There's only one SAFE_FREE call that I see; if I remove it, temacs
crashes at loadup time, when it loads mule-conf.  So I cannot even get
as far as building Emacs.

Btw, the problem I was trying to reproduce with "bzr bisect" was not a
crash, but rather the fact that visiting an emacs-mule encoded desktop
file with that Latin-2 character in it caused some 15K characters
following the Latin-2 one be overwritten with nulls.  The original
crash somehow happens only when I click on an icon that invokes
runemacs.exe, and I cannot reproduce it with the -Q switch.  But since
both issues seem to be related to decoding emacs-mule, and they both
happen when visiting or loading the .emacs.desktop file, I'm assuming
that these are different manifestations of the same problem.

Re: Emacs 23.1.93 pretest

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

>> If you remove the SAFE_FREE () calls, does that prevent the crash?
>
> There's only one SAFE_FREE call that I see; if I remove it, temacs
> crashes at loadup time, when it loads mule-conf.  So I cannot even get
> as far as building Emacs.

Sorry yeah---we need the SAFE_FREE call because otherwise the specpdl
gets confused.

How about if you replace the SAFE_ALLOC calls with xmalloc and omit the
free() call at the end?

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Sat, 27 Feb 2010 20:25:10 -0500
> Cc: emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> If you remove the SAFE_FREE () calls, does that prevent the crash?
> >
> > There's only one SAFE_FREE call that I see; if I remove it, temacs
> > crashes at loadup time, when it loads mule-conf.  So I cannot even get
> > as far as building Emacs.
> 
> Sorry yeah---we need the SAFE_FREE call because otherwise the specpdl
> gets confused.
> 
> How about if you replace the SAFE_ALLOC calls with xmalloc and omit the
> free() call at the end?

If I do that, temacs dies during loadup saying it's out of memory:

    "./oo-spd/i386/temacs.exe" -batch -l loadup dump
    Loading loadup.el (source)...
    Using load-path (../lisp)
    Loading emacs-lisp/byte-run...
    Loading emacs-lisp/backquote...
    Loading subr...
    Loading version.el (source)...
    Loading widget...
    Loading custom...
    Loading emacs-lisp/map-ynp...
    Loading cus-start...
    Loading international/mule...
    Loading international/mule-conf...
    Memory exhausted--use M-x save-some-buffers then exit and restart Emacs
    make: *** [oo-spd/i386/emacs.exe] Error -1

Re: Emacs 23.1.93 pretest

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

>> Hmm, this is strange.  This change (actually the succeeding 2010-02-06
>> change to the same place) switches from using alloca to SAFE_ALLOCA
>> (i.e. malloc, since the desired structure is large).  But the only way I
>> can see for this code to crash is if load_charset_map somehow makes a
>> pointer into the allocated structure.  But in that case, the old alloca
>> case should have crashed too.
>
> Yes, it _is_ weird.  But the effect (see below) does look like we are
> freeing memory being used, or maybe overwriting some allocated buffer,
> or in some other way thrashing the arena.

Hmm, I think I may see the problem.  Does this patch help?

=== modified file 'src/charset.c'
*** src/charset.c	2010-02-06 13:23:33 +0000
--- src/charset.c	2010-02-28 01:45:17 +0000
***************
*** 530,535 ****
--- 530,536 ----
       large (larger than MAX_ALLOCA).  */
    SAFE_ALLOCA (head, struct charset_map_entries *,
  	       sizeof (struct charset_map_entries));
+   bzero (head, sizeof (struct charset_map_entries));
    entries = head;
  
    n_entries = 0;
***************
*** 556,561 ****
--- 557,563 ----
  	{
  	  SAFE_ALLOCA (entries->next, struct charset_map_entries *,
  		       sizeof (struct charset_map_entries));
+ 	  bzero (entries->next, sizeof (struct charset_map_entries));
  	  entries = entries->next;
  	}
        idx = n_entries % 0x10000;
***************
*** 595,600 ****
--- 597,603 ----
       large (larger than MAX_ALLOCA).  */
    SAFE_ALLOCA (head, struct charset_map_entries *,
  	       sizeof (struct charset_map_entries));
+   bzero (head, sizeof (struct charset_map_entries));
    entries = head;
  
    n_entries = 0;
***************
*** 631,636 ****
--- 634,640 ----
  	{
  	  SAFE_ALLOCA (entries->next, struct charset_map_entries *,
  		       sizeof (struct charset_map_entries));
+ 	  bzero (entries->next, sizeof (struct charset_map_entries));
  	  entries = entries->next;
  	}
        idx = n_entries % 0x10000;

Re: Emacs 23.1.93 pretest

[Andreas Schwab]

Chong Yidong <cyd@stupidchicken.com> writes:

> Hmm, I think I may see the problem.  Does this patch help?
>
> === modified file 'src/charset.c'
> *** src/charset.c	2010-02-06 13:23:33 +0000
> --- src/charset.c	2010-02-28 01:45:17 +0000
> ***************
> *** 530,535 ****
> --- 530,536 ----
>        large (larger than MAX_ALLOCA).  */
>     SAFE_ALLOCA (head, struct charset_map_entries *,
>   	       sizeof (struct charset_map_entries));
> +   bzero (head, sizeof (struct charset_map_entries));

If that fixes anything then it was already broken before.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs 23.1.93 pretest

[Chong Yidong]

Andreas Schwab <schwab@linux-m68k.org> writes:

> If that fixes anything then it was already broken before.

Yes, I think it was indeed broken before, and it's probably an accident
that it worked before.  The entries->next pointer needs to be
initialized.  (So we need the patch anyway, and I've checked it in.)

Re: Emacs 23.1.93 pretest

[Andreas Schwab]

Chong Yidong <cyd@stupidchicken.com> writes:

> The entries->next pointer needs to be initialized.

Does it?  AFAICS it is only used when the current table is full, in
which case it will point to the next table anyway.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Sun, 28 Feb 2010 09:25:04 -0500
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> 
> Andreas Schwab <schwab@linux-m68k.org> writes:
> 
> > If that fixes anything then it was already broken before.
> 
> Yes, I think it was indeed broken before, and it's probably an accident
> that it worked before.  The entries->next pointer needs to be
> initialized.  (So we need the patch anyway, and I've checked it in.)

Well, unfortunately, it doesn't seem to solve the problem.  And I
agree with Andreas: if this were the reason, Emacs would have needed
an unreasonable amount of luck to work before this change.  I don't
think the stack is zeroed out on Windows; I'm _positive_ it does not
get zeroed out in the MSDOS build (just checked in the DJGPP library
sources to be sure).

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> Date: Sun, 28 Feb 2010 19:32:42 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: schwab@linux-m68k.org, emacs-devel@gnu.org
> 
> Well, unfortunately, it doesn't seem to solve the problem.

I made the following experiment:

  . restore the calls to alloca -- the bug goes away

  . now add an xmalloc call to allocate a `struct charset_map_entries'
    right before alloca and an xfree call to free that memory after
    the call to load_charset_map, without doing anything with the
    allocated buffer in between the calls to xmalloc and to xfree --
    the bug is back!

So I think the change in charset.c itself did not cause the bug, it
just exposed a bug elsewhere, because it reshuffles the heap.

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> Date: Sun, 28 Feb 2010 21:31:21 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 
> 
> So I think the change in charset.c itself did not cause the bug, it
> just exposed a bug elsewhere, because it reshuffles the heap.

Actually, it turns out it's not ``reshuffling the heap'' that triggers
the bug, but rather the fact that when we call xmalloc, Emacs might
relocate buffer text.

IOW, I found the reason for the bug.  One of the callers of
load_charset_map_from_file maintains pointers into buffer text, so
when that gets relocated, ... you get the idea.

The first naive attempt to solve it was unsuccessful, so there's
probably something else at work here.  Hmm...

Re: Emacs 23.1.93 pretest

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

> Actually, it turns out it's not ``reshuffling the heap'' that triggers
> the bug, but rather the fact that when we call xmalloc, Emacs might
> relocate buffer text.
>
> IOW, I found the reason for the bug.  One of the callers of
> load_charset_map_from_file maintains pointers into buffer text, so
> when that gets relocated, ... you get the idea.

Could you point out where this happens?

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 02 Mar 2010 14:53:51 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Actually, it turns out it's not ``reshuffling the heap'' that triggers
> > the bug, but rather the fact that when we call xmalloc, Emacs might
> > relocate buffer text.
> >
> > IOW, I found the reason for the bug.  One of the callers of
> > load_charset_map_from_file maintains pointers into buffer text, so
> > when that gets relocated, ... you get the idea.
> 
> Could you point out where this happens?

Yes, I can, now that I know that myself ;-)

The problem was in two places: in emacs_mule_char and in
decode_coding_emacs_mule (which calls emacs_mule_char).
emacs_mule_char called DECODE_CHAR, which could result in a call to
decode_char, which could call load_charset_map_from_file (through
load_charset).  Both emacs_mule_char and decode_coding_emacs_mule walk
through buffer text with pointers, and those need to be fixed-up after
the call to load_charset_map_from_file.

I replaced the call to DECODE_CHAR with CODING_DECODE_CHAR, which
wraps DECODE_CHAR with code that fixes up the pointers to buffer text
if a charset map was loaded by DECODE_CHAR.  decode_coding_emacs_mule
needed a similar fixup for its own pointers to buffer text.

This is now fixed in the repository.  I think this fixes the original
problem; at least my .emacs.desktop file with a Latin-2 character now
loads correctly, both in the MS-Windows build and in the MS-DOS build.

Perhaps Handa-san could look at the two other callers of DECODE_CHAR
in coding.c, and see if they, too, need to be replaced with
CODING_DECODE_CHAR.

Re: Emacs 23.1.93 pretest

[Kenichi Handa]

In article <83zl2q30pa.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

> The problem was in two places: in emacs_mule_char and in
> decode_coding_emacs_mule (which calls emacs_mule_char).
> emacs_mule_char called DECODE_CHAR, which could result in a call to
> decode_char, which could call load_charset_map_from_file (through
> load_charset).  Both emacs_mule_char and decode_coding_emacs_mule walk
> through buffer text with pointers, and those need to be fixed-up after
> the call to load_charset_map_from_file.

> I replaced the call to DECODE_CHAR with CODING_DECODE_CHAR, which
> wraps DECODE_CHAR with code that fixes up the pointers to buffer text
> if a charset map was loaded by DECODE_CHAR.  decode_coding_emacs_mule
> needed a similar fixup for its own pointers to buffer text.

> This is now fixed in the repository.  I think this fixes the original
> problem; at least my .emacs.desktop file with a Latin-2 character now
> loads correctly, both in the MS-Windows build and in the MS-DOS build.

Thank you for fixing it.

> Perhaps Handa-san could look at the two other callers of DECODE_CHAR
> in coding.c, and see if they, too, need to be replaced with
> CODING_DECODE_CHAR.

Two other callers of DECODE_CHAR Fdecode_sjis_char and
Fdecode_big5_char, and they are ok.

---
Kenichi Handa
handa@m17n.org

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

Could this be due to some missing GCPRO?  Can one of the functions
called by load_charset_map GC?

Re: Emacs 23.1.93 pretest

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

> Could this be due to some missing GCPRO?  Can one of the functions
> called by load_charset_map GC?

AFAIK load_charset_map does not evaluate anything, but you can check by
binding inhibit_garbage_collection around the call to load_charset_map
and checking if the problem persists.

Re: Emacs 23.1.93 pretest

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Cc: emacs-devel@gnu.org
> Date: Sat, 27 Feb 2010 20:45:45 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Hmm, this is strange.  This change (actually the succeeding 2010-02-06
> >> change to the same place) switches from using alloca to SAFE_ALLOCA
> >> (i.e. malloc, since the desired structure is large).  But the only way I
> >> can see for this code to crash is if load_charset_map somehow makes a
> >> pointer into the allocated structure.  But in that case, the old alloca
> >> case should have crashed too.
> >
> > Yes, it _is_ weird.  But the effect (see below) does look like we are
> > freeing memory being used, or maybe overwriting some allocated buffer,
> > or in some other way thrashing the arena.
> 
> Hmm, I think I may see the problem.  Does this patch help?

No, it doesn't :-(

RE: Emacs 23.1.93 pretest

[Drew Adams]

> Originally, I planned to make the release branch and switch 
> the trunk to Emacs 24 development on Monday...
> let's postphone the branching, and see where we stand in
> about a week.

>> Emacs pretest 23.1.93 is now available for download via
>> FTP, at the following location:
>> ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-23.1.93.tar.gz

How's feedback from Windows users working out for ya (and I don't mean just
build reports)? Gettin' any?

There's no 23.1.93 at http://alpha.gnu.org/gnu/emacs/pretest/windows/, even
though that's the advertised purpose of this GNU directory. The last post there
is 23.1.91.

(And no 23.1.93 binary on Lennart's site.)

Re: Emacs 23.1.93 pretest

[Chong Yidong]

"Drew Adams" <drew.adams@oracle.com> writes:

> How's feedback from Windows users working out for ya (and I don't mean
> just build reports)? Gettin' any?

The latest pretest has a couple of serious bugs on Windows, so IMO there
is little point making the binary for it.

RE: Emacs 23.1.93 pretest

[Drew Adams]

> The latest pretest has a couple of serious bugs on Windows, 
> so IMO there is little point making the binary for it.

Fair enough. Is there a point in publishing a new release without testing on
Windows, in particular, testing the fixes for those serious bugs?

Or did you plan to publish another pretest with the bugs fixed? That would be
good (normal). That wasn't clear from your mails.

Re: Emacs 23.1.93 pretest

[Chong Yidong]

"Drew Adams" <drew.adams@oracle.com> writes:

> Fair enough. Is there a point in publishing a new release without testing on
> Windows, in particular, testing the fixes for those serious bugs?
>
> Or did you plan to publish another pretest with the bugs fixed? That would be
> good (normal). That wasn't clear from your mails.

There should be at least two more pretests before the 23.2 release.

bug#5679: Emacs 23.1.93 pretest

[Sergei Organov]

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

Chong Yidong <cyd@stupidchicken.com> writes:

> Emacs pretest 23.1.93 is now available for download via FTP,

It incorrectly renders terminus oblique fonts from Debian distribution
(xfonts-terminus-oblique Debian package):

-xos4-terminus-medium-o-*
-xos4-terminus-bold-o-*

Small example screenshots from emacs 22.2.1 (correct rendering) and
emacs 23.1.93 (wrong rendering) are attached. Please notice how ugly
"Global Mark Ring" looks when rendered by emacs23.

NOTE: emacs 23.1.1 also renders wrong.


[-- Attachment #2: emacs22 correct rendering --]
[-- Type: image/png, Size: 7950 bytes --]

[-- Attachment #3: emacs23 wrong rendering --]
[-- Type: image/png, Size: 9031 bytes --]