bug#52295: 28.0.90; Killing text results in coding system complaint

bug#52295: 28.0.90; Killing text results in coding system complaint

[luangruo--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Trying to kill text on MS-Windows 9x results in the following complaint:

w32--set-selection: Coding system is invalid or doesn't have an eol
variant for dos line ends: nil [2 times]

In GNU Emacs 28.0.90 (build 1, i686-pc-mingw32) of 2021-12-05 built on MACHINE

Windowing system distributor ‘Microsoft Corp.', version 4.10.2222
System Description: Microsoft Windows 98 (v4.10.2222)
Configured using:

‘configure --with-gnutls=ifavailable --prefix=/emacs-28.0.90/../prefix'
Configured features:
ACL MODULES NOTIFY W32NOTIFY PDUMPER SOUND THREADS TOOLKIT SCROLL_BARS
Important settings:

value of $LANG: enu

locale-coding-system: cp1252
Major mode: Fundamental

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

> Date: Sun, 05 Dec 2021 14:40:04 +0800
> From: luangruo--- via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Trying to kill text on MS-Windows 9x results in the following complaint:
> 
> w32--set-selection: Coding system is invalid or doesn't have an eol
> variant for dos line ends: nil [2 times]

What are the values of the following variables on that system?

  selection-coding-system
  next-selection-coding-system

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sun, 05 Dec 2021 14:40:04 +0800
>> From: luangruo--- via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> 
>> Trying to kill text on MS-Windows 9x results in the following complaint:
>> 
>> w32--set-selection: Coding system is invalid or doesn't have an eol
>> variant for dos line ends: nil [2 times]
>
> What are the values of the following variables on that system?
>
>   selection-coding-system
>   next-selection-coding-system

Both variables are nil.

Thanks.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: 52295@debbugs.gnu.org
> Date: Sun, 05 Dec 2021 16:58:27 +0800
> 
> >> w32--set-selection: Coding system is invalid or doesn't have an eol
> >> variant for dos line ends: nil [2 times]
> >
> > What are the values of the following variables on that system?
> >
> >   selection-coding-system
> >   next-selection-coding-system
> 
> Both variables are nil.

That's the problem.  But I don't understand why is that so.  The value
of selection-coding-system is set in globals_of_w32select, which is
called during startup, and where we have this code:

  ANSICP = GetACP ();
  OEMCP = GetOEMCP ();

  QANSICP = coding_from_cp (ANSICP);
  QOEMCP = coding_from_cp (OEMCP);

  if (os_subtype == OS_SUBTYPE_NT)
    Vselection_coding_system = Qutf_16le_dos;
  else if (inhibit_window_system)
    Vselection_coding_system = QOEMCP;
  else
    Vselection_coding_system = QANSICP;

On Windows 9X, this should assign the ANSI codepage to
selection-coding-system.  Since your codepage seems to be 1252, it
should assign the symbol 'cp1252-dos' to selection-coding-system, see
coding_from_cp.

Why doesn't this happen in your case?

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Po Lu <luangruo@yahoo.com>
>> Cc: 52295@debbugs.gnu.org
>> Date: Sun, 05 Dec 2021 16:58:27 +0800
>> 
>> >> w32--set-selection: Coding system is invalid or doesn't have an eol
>> >> variant for dos line ends: nil [2 times]
>> >
>> > What are the values of the following variables on that system?
>> >
>> >   selection-coding-system
>> >   next-selection-coding-system
>> 
>> Both variables are nil.
>
> That's the problem.  But I don't understand why is that so.  The value
> of selection-coding-system is set in globals_of_w32select, which is
> called during startup, and where we have this code:
>
>   ANSICP = GetACP ();
>   OEMCP = GetOEMCP ();
>
>   QANSICP = coding_from_cp (ANSICP);
>   QOEMCP = coding_from_cp (OEMCP);
>
>   if (os_subtype == OS_SUBTYPE_NT)
>     Vselection_coding_system = Qutf_16le_dos;
>   else if (inhibit_window_system)
>     Vselection_coding_system = QOEMCP;
>   else
>     Vselection_coding_system = QANSICP;
>
> On Windows 9X, this should assign the ANSI codepage to
> selection-coding-system.  Since your codepage seems to be 1252, it
> should assign the symbol 'cp1252-dos' to selection-coding-system, see
> coding_from_cp.
>
> Why doesn't this happen in your case?

Maybe that variable is dumped during the build process, with a value
appropriate for NT, and when Emacs starts on a 9X system some other code
sees that it is invalid and sets it to nil?

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: 52295@debbugs.gnu.org
> Date: Sun, 05 Dec 2021 17:26:11 +0800
> 
> >> >   selection-coding-system
> >> >   next-selection-coding-system
> >> 
> >> Both variables are nil.
> >
> > That's the problem.  But I don't understand why is that so.  The value
> > of selection-coding-system is set in globals_of_w32select, which is
> > called during startup, and where we have this code:
> >
> >   ANSICP = GetACP ();
> >   OEMCP = GetOEMCP ();
> >
> >   QANSICP = coding_from_cp (ANSICP);
> >   QOEMCP = coding_from_cp (OEMCP);
> >
> >   if (os_subtype == OS_SUBTYPE_NT)
> >     Vselection_coding_system = Qutf_16le_dos;
> >   else if (inhibit_window_system)
> >     Vselection_coding_system = QOEMCP;
> >   else
> >     Vselection_coding_system = QANSICP;
> >
> > On Windows 9X, this should assign the ANSI codepage to
> > selection-coding-system.  Since your codepage seems to be 1252, it
> > should assign the symbol 'cp1252-dos' to selection-coding-system, see
> > coding_from_cp.
> >
> > Why doesn't this happen in your case?
> 
> Maybe that variable is dumped during the build process, with a value
> appropriate for NT, and when Emacs starts on a 9X system some other code
> sees that it is invalid and sets it to nil?

No, the code above is run in the dumped Emacs, when it starts, not in
temacs.  So the values should reflect what happens when you start
Emacs on the Windows 9X system.

Could you perhaps add some printf's there to see what actually
happens?  It almost looks like that code doesn't get called (which
would be strange by itself), since coding_from_cp cannot possibly
return nil.  Or maybe the initial assignment does work, but then some
code resets the variable back to nil?

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Po Lu <luangruo@yahoo.com>
>> Cc: 52295@debbugs.gnu.org
>> Date: Sun, 05 Dec 2021 17:26:11 +0800
>> 
>> >> >   selection-coding-system
>> >> >   next-selection-coding-system
>> >> 
>> >> Both variables are nil.
>> >
>> > That's the problem.  But I don't understand why is that so.  The value
>> > of selection-coding-system is set in globals_of_w32select, which is
>> > called during startup, and where we have this code:
>> >
>> >   ANSICP = GetACP ();
>> >   OEMCP = GetOEMCP ();
>> >
>> >   QANSICP = coding_from_cp (ANSICP);
>> >   QOEMCP = coding_from_cp (OEMCP);
>> >
>> >   if (os_subtype == OS_SUBTYPE_NT)
>> >     Vselection_coding_system = Qutf_16le_dos;
>> >   else if (inhibit_window_system)
>> >     Vselection_coding_system = QOEMCP;
>> >   else
>> >     Vselection_coding_system = QANSICP;
>> >
>> > On Windows 9X, this should assign the ANSI codepage to
>> > selection-coding-system.  Since your codepage seems to be 1252, it
>> > should assign the symbol 'cp1252-dos' to selection-coding-system, see
>> > coding_from_cp.
>> >
>> > Why doesn't this happen in your case?
>> 
>> Maybe that variable is dumped during the build process, with a value
>> appropriate for NT, and when Emacs starts on a 9X system some other code
>> sees that it is invalid and sets it to nil?
>
> No, the code above is run in the dumped Emacs, when it starts, not in
> temacs.  So the values should reflect what happens when you start
> Emacs on the Windows 9X system.

> Could you perhaps add some printf's there to see what actually
> happens?  It almost looks like that code doesn't get called (which
> would be strange by itself), since coding_from_cp cannot possibly
> return nil.  Or maybe the initial assignment does work, but then some
> code resets the variable back to nil?

`globals_of_w32select' is called, but QANSICP is nil:

  Fprint (QANSICP, Qexternal_debugging_output);

Prints "nil" onto the console.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Po Lu <luangruo@yahoo.com> writes:

> `globals_of_w32select' is called, but QANSICP is nil:
>
>   Fprint (QANSICP, Qexternal_debugging_output);
>
> Prints "nil" onto the console.

??? It appears that coding_from_cp is actually returning nil in this
case.

This must be a bug in intern, because I don't see how else it could
possibly return nil.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

On December 7, 2021 7:51:32 AM GMT+02:00, Po Lu <luangruo@yahoo.com> wrote:
> Po Lu <luangruo@yahoo.com> writes:
> 
> > `globals_of_w32select' is called, but QANSICP is nil:
> >
> >   Fprint (QANSICP, Qexternal_debugging_output);
> >
> > Prints "nil" onto the console.
> 
> ??? It appears that coding_from_cp is actually returning nil in this
> case.
> 
> This must be a bug in intern, because I don't see how else it could
> possibly return nil.
> 

Maybe 'intern' signals an error?  There's a check_obarray test inside intern_1, which could signal an error.

If you print the value which 'intern' returns in coding_from_cp, do you see a nil value printed?

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Maybe 'intern' signals an error?  There's a check_obarray test inside
> intern_1, which could signal an error.

No error is signalled:

> If you print the value which 'intern' returns in coding_from_cp, do
> you see a nil value printed?

`nil' is printed in this case.

Thanks.

(I will not have access to that machine for a while, so I can't perform
any further testing on it for the next week or so.)

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

On December 7, 2021 11:20:10 AM GMT+02:00, Po Lu <luangruo@yahoo.com> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Maybe 'intern' signals an error?  There's a check_obarray test inside
> > intern_1, which could signal an error.
> 
> No error is signalled:
> 
> > If you print the value which 'intern' returns in coding_from_cp, do
> > you see a nil value printed?
> 
> `nil' is printed in this case.
> 
> Thanks.
> 
> (I will not have access to that machine for a while, so I can't perform
> any further testing on it for the next week or so.)
> 

So I think there's no way around stepping into intern_1 and the functions it calls, and figuring out what happens there.  I thought you said GDB doesn't work on the target?  Then perhaps printfs will help?

Thanks.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> On December 7, 2021 11:20:10 AM GMT+02:00, Po Lu <luangruo@yahoo.com> wrote:
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Maybe 'intern' signals an error?  There's a check_obarray test inside
>> > intern_1, which could signal an error.
>> 
>> No error is signalled:
>> 
>> > If you print the value which 'intern' returns in coding_from_cp, do
>> > you see a nil value printed?
>> 
>> `nil' is printed in this case.
>> 
>> Thanks.
>> 
>> (I will not have access to that machine for a while, so I can't perform
>> any further testing on it for the next week or so.)
>> 

> So I think there's no way around stepping into intern_1 and the
> functions it calls, and figuring out what happens there.  I thought
> you said GDB doesn't work on the target?  Then perhaps printfs will
> help?

Yes, GDB doesn't work there, but I modified coding_from_cp to look like
this:

static Lisp_Object
coding_from_cp (UINT codepage)
{
  char buffer[30];
  Lisp_Object sym;
  sprintf (buffer, "cp%d-dos", (int) codepage);
  sym = intern (buffer);
  Fprint (sym, Qexternal_debugging_output);
  return sym;
}

And it printed nil.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

On December 7, 2021 12:33:26 PM GMT+02:00, Po Lu <luangruo@yahoo.com> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > So I think there's no way around stepping into intern_1 and the
> > functions it calls, and figuring out what happens there.  I thought
> > you said GDB doesn't work on the target?  Then perhaps printfs will
> > help?
> 
> Yes, GDB doesn't work there, but I modified coding_from_cp to look like
> this:
> 
> static Lisp_Object
> coding_from_cp (UINT codepage)
> {
>   char buffer[30];
>   Lisp_Object sym;
>   sprintf (buffer, "cp%d-dos", (int) codepage);
>   sym = intern (buffer);
>   Fprint (sym, Qexternal_debugging_output);
>   return sym;
> }
> 
> And it printed nil.
> 

Yes.  But I meant you should go deeper, into 'intern' and its subroutines, starting from intern_1.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: 52295@debbugs.gnu.org
> Date: Tue, 07 Dec 2021 13:51:32 +0800
> 
> Po Lu <luangruo@yahoo.com> writes:
> 
> > `globals_of_w32select' is called, but QANSICP is nil:
> >
> >   Fprint (QANSICP, Qexternal_debugging_output);
> >
> > Prints "nil" onto the console.
> 
> ??? It appears that coding_from_cp is actually returning nil in this
> case.
> 
> This must be a bug in intern, because I don't see how else it could
> possibly return nil.

It could also be some memory-related snafu, or some stupid omission
during startup when we run on Windows 9X.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> ??? It appears that coding_from_cp is actually returning nil in this
>> case.
>> 
>> This must be a bug in intern, because I don't see how else it could
>> possibly return nil.

> It could also be some memory-related snafu, or some stupid omission
> during startup when we run on Windows 9X.

FWIW, the problem doesn't appear with an unexec build (though I did have
to fix the build process somewhat to get unexec to work on the build
system running MSYS).

I will build a pdumper build with all the usual debug options and see
what comes up.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: 52295@debbugs.gnu.org
> Date: Sun, 12 Dec 2021 18:58:22 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> ??? It appears that coding_from_cp is actually returning nil in this
> >> case.
> >> 
> >> This must be a bug in intern, because I don't see how else it could
> >> possibly return nil.
> 
> > It could also be some memory-related snafu, or some stupid omission
> > during startup when we run on Windows 9X.
> 
> FWIW, the problem doesn't appear with an unexec build

So I guess some code we need to run at startup doesn't get run in the
pdumper build.

> (though I did have to fix the build process somewhat to get unexec
> to work on the build system running MSYS).

Please tell the details, we should fix that.

> I will build a pdumper build with all the usual debug options and see
> what comes up.

Thanks.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Please tell the details, we should fix that.

I had to remove `-Wl,-disable-dynamicbase' from LD_SWITCH_SYSTEM_TEMACS,
because the linker complained that option did not exist.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: 52295@debbugs.gnu.org
> Date: Sun, 12 Dec 2021 20:47:37 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Please tell the details, we should fix that.
> 
> I had to remove `-Wl,-disable-dynamicbase' from LD_SWITCH_SYSTEM_TEMACS,
> because the linker complained that option did not exist.

What version of Binutils do you have there, and where did you download
it?

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Po Lu <luangruo@yahoo.com>
>> Cc: 52295@debbugs.gnu.org
>> Date: Sun, 12 Dec 2021 20:47:37 +0800
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Please tell the details, we should fix that.
>> 
>> I had to remove `-Wl,-disable-dynamicbase' from LD_SWITCH_SYSTEM_TEMACS,
>> because the linker complained that option did not exist.
>
> What version of Binutils do you have there, and where did you download
> it?

It's binutils 2.32, which I downloaded from the MingGW Installation
Manager.

Thanks.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: 52295@debbugs.gnu.org
> Date: Mon, 13 Dec 2021 08:50:02 +0800
> 
> >> I had to remove `-Wl,-disable-dynamicbase' from LD_SWITCH_SYSTEM_TEMACS,
> >> because the linker complained that option did not exist.
> >
> > What version of Binutils do you have there, and where did you download
> > it?
> 
> It's binutils 2.32, which I downloaded from the MingGW Installation
> Manager.

Ah, okay.  This is an old version, so it doesn't support those
switches.  (You may wish to get a newer version from ezwinports.)  So
I think it is okay for such an old version to require manual changes.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Po Lu <luangruo@yahoo.com> writes:

> I will build a pdumper build with all the usual debug options and see
> what comes up.

An unoptimized pdumper build with CFLAGS='-O0'
--enable-checking="yes,glyphs" crashes when a dump file is present, so
there is definitely more here than meets the eye, but I have no idea how
to get a working debugger onto that Windows 9x system.

Is there some way to save a core dump (or backtrace) there for debugging
on a more recent MS-Windows machine?

Thanks.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: 52295@debbugs.gnu.org
> Date: Sat, 18 Dec 2021 08:29:21 +0800
> 
> Po Lu <luangruo@yahoo.com> writes:
> 
> > I will build a pdumper build with all the usual debug options and see
> > what comes up.
> 
> An unoptimized pdumper build with CFLAGS='-O0'
> --enable-checking="yes,glyphs" crashes when a dump file is present, so
> there is definitely more here than meets the eye, but I have no idea how
> to get a working debugger onto that Windows 9x system.
> 
> Is there some way to save a core dump (or backtrace) there for debugging
> on a more recent MS-Windows machine?

Is this a real crash (segfault), or an abort where we pop up the Emacs
Abort dialog?  Since you compiled with --enable-checking, I think it's
the latter.

If it's a real crash, you could use the Windows Event log to find out
the address where it crashes, and then use GDB on another system to
see where that address is in the sources (but the address printed by
Windows needs to be "translated" into addresses that you submit to GDB
by adding some constant, AFAIR).

The other alternative is to insert many fprintf's to stderr into the
sources, and use that to determine where it crashes.

If it's an abort, then saying NO to the dialog will produce an
emacs_backtrace.txt file with addresses of the backtrace, which you
could take to another Windows system and use addr2line (from Binutils)
to recover the file names, function names, and line numbers that
correspond to the addresses; see the node "Crashing" in the Emacs
manual for how to do that.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Eli Zaretskii]

> Date: Sat, 18 Dec 2021 08:48:07 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 52295@debbugs.gnu.org
> 
> If it's a real crash, you could use the Windows Event log to find out
> the address where it crashes, and then use GDB on another system to
> see where that address is in the sources (but the address printed by
> Windows needs to be "translated" into addresses that you submit to GDB
> by adding some constant, AFAIR).
> 
> The other alternative is to insert many fprintf's to stderr into the
> sources, and use that to determine where it crashes.
> 
> If it's an abort, then saying NO to the dialog will produce an
> emacs_backtrace.txt file with addresses of the backtrace, which you
> could take to another Windows system and use addr2line (from Binutils)
> to recover the file names, function names, and line numbers that
> correspond to the addresses; see the node "Crashing" in the Emacs
> manual for how to do that.

Btw, there's one more alternative: to debug Emacs on Windows 9X
remotely, via gdbserver.  Did you try that?  The advantage is that
gdbserver is a much smaller and simpler program, so chances are it
will be able to run on Windows 9X.  GDB itself will run on a different
system (could even be GNU/Linux, if you have GDB that was built to
support debugging Windows PE executables), where all the problems of
running a modern GDB don't exist.  You just need the target system to
be connected to a network, so that GDB and gdbserver could
communicate.

bug#52295: 28.0.90; Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> You just need the target system to be connected to a network, so that
> GDB and gdbserver could communicate.

Thanks, I'll try that.  The difficult part here is to connect the
Windows 9x system to a network from which I can access it.

bug#52295: windows 98: Killing text results in coding system complaint

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Is this a real crash (segfault), or an abort where we pop up the Emacs
> Abort dialog?  Since you compiled with --enable-checking, I think it's
> the latter.

> If it's a real crash, you could use the Windows Event log to find out
> the address where it crashes, and then use GDB on another system to
> see where that address is in the sources (but the address printed by
> Windows needs to be "translated" into addresses that you submit to GDB
> by adding some constant, AFAIR).

It's a real crash: the system pops up a dialog that says:

  This program has performed an illegal operation and will be shut down.
  If the problem persists, contact the program vendor.

Underneath, it displays the contents of the program counter: 011c62d6.

But I have no idea how to convert that address with gdb on the machine
that built the binary.  Any ideas?

gdbserver complained about missing symbols in kernel32.dll, so that
didn't work either.

> The other alternative is to insert many fprintf's to stderr into the
> sources, and use that to determine where it crashes.

I will try that, thanks.