bug#25203: 25.1; crash during message, infinite recursion

bug#25203: 25.1; crash during message, infinite recursion

[Hin-Tak Leung]

emacs -Q -batch -l cjk-enc.el -f batch-force-cjk-write-file Big5.tex

where cjk-enc.el is a slight enhancement of
"cjk/utils/lisp/emacs/cjk-enc.el" from http://cjk.ffii.org , with

diff --git a/emacs/cjk-enc.el b/emacs/cjk-enc.el
index c8e6706..096ade7 100644
--- a/emacs/cjk-enc.el
+++ b/emacs/cjk-enc.el
@@ -881,7 +881,8 @@
               (setq last-pos (point))
               (message "Converting: %2d%%"
                        (/ (* 100 (point)) (point-max)))))
-
+        (message "temp-buf is %s..." (buffer-name temp-buf))
+        (message "work-buf is %s..." (buffer-name work-buf))
         ;; Advance to the next character and loop.
         (forward-char 1))
 

and Big5.tex is cjk/examples/Big5.tex



(gdb) bt
#0  0x00007f6ecfe1e48f in raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:58
#1  0x00000000004f1174 in terminate_due_to_signal ()
#2  0x000000000050945e in  ()
#3  0x0000000000509689 in  ()
#4  0x00000000005096ef in handle_sigsegv ()
#5  0x00007f6ecfe1e5c0 in <signal handler called> () at /lib64/libpthread.so.0
#6  0x000000000048c2d5 in consume_chars ()
#7  0x0000000000495fe9 in encode_coding ()
#8  0x000000000049c775 in encode_coding_object ()
#9  0x000000000049d7c8 in code_convert_string ()
#10 0x0000000000431ad5 in message_to_stderr ()
#11 0x000000000045657c in message3 ()
#12 0x000000000055ee5c in Fmessage ()
#13 0x00000000005651e0 in eval_sub ()
#14 0x00000000005681cd in Flet ()
#15 0x0000000000565014 in eval_sub ()
#16 0x00000000005682fd in Fwhile ()
#17 0x0000000000565014 in eval_sub ()
#18 0x00000000005681cd in Flet ()
#19 0x0000000000565014 in eval_sub ()
#20 0x00000000005681cd in Flet ()
#21 0x0000000000565014 in eval_sub ()
#22 0x00000000005657bd in funcall_lambda ()
#23 0x0000000000565a4b in Ffuncall ()
#24 0x0000000000564516 in internal_condition_case_n ()
#25 0x000000000042efc8 in safe.call ()
#26 0x000000000043b97c in safe_call ()
#27 0x000000000049c27e in encode_coding_object ()
---Type <return> to continue, or q <return> to quit---
#28 0x000000000049d7c8 in code_convert_string ()
#29 0x0000000000431ad5 in message_to_stderr ()
#30 0x000000000045657c in message3 ()
...
repeats at a cycle of 19
...
#923 0x000000000045657c in message3 ()
---Type <return> to continue, or q <return> to quit---
#924 0x000000000055ee5c in Fmessage ()
#925 0x00000000005651e0 in eval_sub ()
#926 0x00000000005681cd in Flet ()
#927 0x0000000000565014 in eval_sub ()
#928 0x00000000005682fd in Fwhile ()
#929 0x0000000000565014 in eval_sub ()
#930 0x00000000005681cd in Flet ()
#931 0x0000000000565014 in eval_sub ()
#932 0x00000000005681cd in Flet ()
#933 0x0000000000565014 in eval_sub ()
#934 0x00000000005657bd in funcall_lambda ()
#935 0x0000000000565a4b in Ffuncall ()
#936 0x0000000000564516 in internal_condition_case_n ()
#937 0x000000000042efc8 in safe.call ()
#938 0x000000000043b97c in safe_call ()
#939 0x000000000049c27e in encode_coding_object ()
#940 0x00000000005205fa in e_write ()
#941 0x000000000052766b in write_region ()
#942 0x00000000005283ef in Fwrite_region ()
#943 0x000000000056503d in eval_sub ()
#944 0x00000000005681cd in Flet ()
#945 0x0000000000565014 in eval_sub ()
#946 0x000000000056535d in Fprogn ()
#947 0x000000000055efc7 in Fsave_excursion ()
#948 0x0000000000565014 in eval_sub ()
#949 0x00000000005681cd in Flet ()
#950 0x0000000000565014 in eval_sub ()
#951 0x00000000005657bd in funcall_lambda ()
---Type <return> to continue, or q <return> to quit---
#952 0x0000000000564ac8 in apply_lambda ()
#953 0x0000000000564e2a in eval_sub ()
#954 0x0000000000565014 in eval_sub ()
#955 0x00000000005653a5 in Fif ()
#956 0x0000000000565014 in eval_sub ()
#957 0x00000000005653a5 in Fif ()
#958 0x0000000000565014 in eval_sub ()
#959 0x00000000005681cd in Flet ()
#960 0x0000000000565014 in eval_sub ()
#961 0x00000000005682fd in Fwhile ()
#962 0x0000000000565014 in eval_sub ()
#963 0x00000000005657bd in funcall_lambda ()
#964 0x0000000000564ac8 in apply_lambda ()
#965 0x0000000000564e2a in eval_sub ()
#966 0x00000000005657bd in funcall_lambda ()
#967 0x0000000000565a4b in Ffuncall ()
#968 0x000000000059ad23 in exec_byte_code ()
#969 0x0000000000565a4b in Ffuncall ()
#970 0x000000000059ad23 in exec_byte_code ()
#971 0x0000000000565a4b in Ffuncall ()
#972 0x000000000059ad23 in exec_byte_code ()
#973 0x0000000000564ac8 in apply_lambda ()
#974 0x0000000000564e2a in eval_sub ()
#975 0x00000000005686fc in Feval ()
#976 0x0000000000564362 in internal_condition_case ()
#977 0x00000000004f3d70 in top_level_1 ()
#978 0x0000000000564303 in internal_catch ()
#979 0x00000000004f15bd in command_loop ()
---Type <return> to continue, or q <return> to quit---
#980 0x00000000004f5dd7 in recursive_edit_1 ()
#981 0x00000000004f6128 in Frecursive_edit ()
#982 0x0000000000419d52 in main ()

In GNU Emacs 25.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 3.22.1)
 of 2016-10-12 built on buildvm-18.phx2.fedoraproject.org
Windowing system distributor 'Fedora Project', version 11.0.11900000
System Description:	Fedora release 25 (Twenty Five)

Configured using:
 'configure --build=x86_64-redhat-linux-gnu
 --host=x86_64-redhat-linux-gnu --program-prefix=
 --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr
 --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc
 --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64
 --libexecdir=/usr/libexec --localstatedir=/var
 --sharedstatedir=/var/lib --mandir=/usr/share/man
 --infodir=/usr/share/info --with-dbus --with-gif --with-jpeg --with-png
 --with-rsvg --with-tiff --with-xft --with-xpm --with-x-toolkit=gtk3
 --with-gpm=no --with-xwidgets build_alias=x86_64-redhat-linux-gnu
 host_alias=x86_64-redhat-linux-gnu 'CFLAGS=-DMAIL_USE_LOCKF -O2 -g
 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4
 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
 -m64 -mtune=generic' LDFLAGS=-Wl,-z,relro
 PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GCONF GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XWIDGETS

Important settings:
  value of $LANG: C
  locale-coding-system: nil

Major mode: Fundamental

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
Loading /usr/share/emacs/site-lisp/site-start.d/ess-init.el (source)...
Finding all versions of R on your system...
Sorry, no version of R could be found on your system.
Loading /usr/share/emacs/site-lisp/site-start.d/ess-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/git-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/lilypond-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/mercurial-site-start.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/rpmdev-init.el (source)...done
Loading /usr/share/emacs/site-lisp/site-start.d/systemtap-init.el (source)...done
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
/usr/share/emacs/site-lisp/lilypond-init hides /usr/share/emacs/site-lisp/site-start.d/lilypond-init

Features:
(shadow sort mail-extr emacsbug message idna dired rfc822 mml mml-sec
epg epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mail-utils ido seq ess-toolbar ess-mouse mouseme thingatpt browse-url
ess-menu ess-swv ess-noweb ess-noweb-font-lock-mode ess-bugs-l essd-els
ess-sas-d ess-sas-l ess-sas-a ess-sta-d ess-sta-l cc-vars cc-defs
make-regexp ess-sp6-d ess-dde ess-sp3-d ess-julia julia-mode ert pp
find-func ewoc debug ess-r-d ess-r-syntax ess-r-completion ess-roxy
essddr hideshow ess-help ess-r-package ess-s-l ess ess-inf ess-tracebug
compile tramp tramp-compat auth-source cl-seq eieio byte-opt bytecomp
byte-compile cl-extra cconv eieio-core gnus-util mm-util help-fns
help-mode mail-prsvr password-cache tramp-loaddefs cl-macs trampver
ucs-normalize shell pcomplete comint ansi-color ring format-spec advice
ess-mode ess-noweb-mode ess-utils ess-generics cl gv cl-loaddefs pcase
cl-lib ess-custom executable easymenu ess-compat ess-site time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset
image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote dbusbind inotify dynamic-setting
system-font-setting font-render-setting xwidget-internal move-toolbar
gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 260299 10067)
 (symbols 48 27442 0)
 (miscs 40 105 141)
 (strings 32 40868 7895)
 (string-bytes 1 1149641)
 (vectors 16 41418)
 (vector-slots 8 760116 3898)
 (floats 8 296 105)
 (intervals 56 326 108)
 (buffers 976 19))

bug#25203: 25.1; crash during message, infinite recursion

[Werner LEMBERG]

> where cjk-enc.el is a slight enhancement of
> "cjk/utils/lisp/emacs/cjk-enc.el" from http://cjk.ffii.org , with

http://git.savannah.gnu.org/gitweb/?p=cjk.git;a=blob_plain;f=utils/lisp/emacs/cjk-enc.el;hb=HEAD

> and Big5.tex is cjk/examples/Big5.tex

http://git.savannah.gnu.org/gitweb/?p=cjk.git;a=blob_plain;f=examples/Big5.tex;hb=HEAD

(Attention: this file is encoded in Big5.)


    Werner

bug#25203: 25.1; crash during message, infinite recursion

[Eli Zaretskii]

> From: Hin-Tak Leung <hintak.leung@gmail.com>
> Date: Thu, 15 Dec 2016 00:04:53 +0000
> Cc: Hin-Tak Leung <htl10@users.sourceforge.net>
> 
> emacs -Q -batch -l cjk-enc.el -f batch-force-cjk-write-file Big5.tex
> 
> where cjk-enc.el is a slight enhancement of
> "cjk/utils/lisp/emacs/cjk-enc.el" from http://cjk.ffii.org , with
> 
> diff --git a/emacs/cjk-enc.el b/emacs/cjk-enc.el
> index c8e6706..096ade7 100644
> --- a/emacs/cjk-enc.el
> +++ b/emacs/cjk-enc.el
> @@ -881,7 +881,8 @@
>                (setq last-pos (point))
>                (message "Converting: %2d%%"
>                         (/ (* 100 (point)) (point-max)))))
> -
> +        (message "temp-buf is %s..." (buffer-name temp-buf))
> +        (message "work-buf is %s..." (buffer-name work-buf))
>          ;; Advance to the next character and loop.
>          (forward-char 1))
> 
> and Big5.tex is cjk/examples/Big5.tex

AFAICT, cjk-enc.el shoots itself in the foot by invoking 'message'
from the pre-write-conversion function.  This is a very dangerous
thing to do, as the analysis below shows.  In fact, any I/O that might
require encoding should be completely avoided in pre-write-conversion.
It should be a function that does its work silently without any I/O,
because it is invoked in a context of I/O.  If it does need to do I/O,
it absolutely must make sure that I/O will not use the same encoding
for which the pre-write-conversion function was written.

Here's what happens here:

 . cjk-enc defines a coding-system with a pre-write-conversion
   function cjk-encode.

 . batch-force-cjk-write-file eventually calls cjk-write-file, which
   does this:

    (message "Saving %s and %s" bufname newbufname)
    (let ((coding-system-for-write 'cjk-coding))
      (write-region (point-min) (point-max) newbufname))

 . write-region invokes cjk-encode as part of encoding the text in the
   region.

 . cjk-encode calls 'message', due to the above changes.

 . since this is batch mode, 'message' eventually calls
   message_to_stderr, which attempts to encode the text before writing
   it to the terminal.  But since coding-system-for-write is let-bound
   to a non-nil value, message_to_stderr uses that value for encoding
   (as all the encoding functions do), and that results in recursive
   invocation of cjk-encode, which again tries to output a message to
   the terminal, etc. etc., ad nauseam.

The simplest fix for this that requires no changes in Emacs core is
this:

   (let ((coding-system-for-write locale-coding-system))
     (message "temp-buf is %s..." (buffer-name temp-buf))
     (message "work-buf is %s..." (buffer-name work-buf)))

That is, wrap the above 'message' calls in a let-binding of
coding-system-for-write that avoids recursion, and still does TRT wrt
encoding messages to the terminal in batch mode.  After all, this is
(presumably) debugging code, so it can go some extra length to avoid
interfering with the code it's supposed to debug.

If someone is going to argue that Lisp code should never crash Emacs,
then I can see the following alternatives:

  1. Inhibit messages while calling pre-write-conversion.
  2. Signal an error when pre-write-conversion is called recursively.

I think none of these is a good idea, as they disallow perfectly valid
use cases which don't hit this problem.  The 2nd one is also error
prone in its naïve implementation, and too complex IMO in non-naïve
ones.

So my suggestion is to fix your debugging code as indicated above.

Thanks.

bug#25203: 25.1; crash during message, infinite recursion

[Noam Postavsky]

On Thu, Dec 15, 2016 at 10:55 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>
> If someone is going to argue that Lisp code should never crash Emacs,
> then I can see the following alternatives:
>
>   1. Inhibit messages while calling pre-write-conversion.
>   2. Signal an error when pre-write-conversion is called recursively.
>
> I think none of these is a good idea, as they disallow perfectly valid
> use cases which don't hit this problem.  The 2nd one is also error
> prone in its naïve implementation, and too complex IMO in non-naïve
> ones.

Shouldn't max-specdl or max-lisp-eval-depth be catching too deep
recursive calls?

bug#25203: 25.1; crash during message, infinite recursion

[Hin-Tak Leung]

I'll have to think a bit about the lisp advice - that script is complicated as it is... I just like to comment that emacs 24.5 does not crash with the additional "message" code, so this is  a regression with emacs 25. And yes, I think no lisp code should crash emacs...

I also tried running the windows emacs 25 binary under wine - it dies there also. This eliminates possibility about miscompiling or platform issues.

My attempt at adding such temporary code is due to the bare unmodified script not working correctly with emacs 25 (still looking), and lisp debugging apparently does not work either 
(https://debbugs.gnu.org/cgi/bugreport.cgi?bug=8108).

bug#25203: 25.1; crash during message, infinite recursion

[Werner LEMBERG]

> In fact, any I/O that might require encoding should be completely
> avoided in pre-write-conversion.

Is this documented?  If yes, I've missed it.

> So my suggestion is to fix your debugging code as indicated above.

Thanks for your analysis.  If it was only possible to debug
pre-write-conversion functions on the lisp level, I probably would
have found this by myself...


    Werner

bug#25203: 25.1; crash during message, infinite recursion

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Thu, 15 Dec 2016 11:11:38 -0500
> Cc: Hin-Tak Leung <hintak.leung@gmail.com>, htl10@users.sourceforge.net, 
> 	25203@debbugs.gnu.org
> 
> Shouldn't max-specdl or max-lisp-eval-depth be catching too deep
> recursive calls?

This is not called in Lisp, it's called from C.  982 call frames
divided by 19 and multiplied by 5 (the number of calls to eval_sub)
gives 258, much smaller than max-lisp-eval-depth.

bug#25203: [cjk] Fwd: HELP with emacs 25.1 and cjk-enc.el

[Hin-Tak Leung]

Thanks for the advice from Eli Zaretskii
( https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25203#11 )

I think I fixed one breakage of cjk-enc.el with emacs 25. It is doing a "percentage progress report"
for any sizeable input. With the following change:

--- a/utils/lisp/emacs/cjk-enc.el
+++ b/utils/lisp/emacs/cjk-enc.el
@@ -879,8 +879,9 @@
         (if (> (- (point) last-pos) 1000)
             (progn
               (setq last-pos (point))
-              (message "Converting: %2d%%"
-                       (/ (* 100 (point)) (point-max)))))
+              (let ((coding-system-for-write 'cjk-coding))
+                (message "Converting: %2d%%"
+                         (/ (* 100 (point)) (point-max))))))
 
         ;; Advance to the next character and loop.
         (forward-char 1))

 I could get emacs 25 to run cjk-enc.el correctly to process Thai tis620 and mule encoded inputs now.

I still can't process Big5 input correctly - but thanks, the crash is gone -, Big5 input
seems to be sensitive to LANG/LC_*.
The original poster (not fully specified LANG/LC_*=zh_TW.Big5 ) can run it with 24.x but
I (LANG/LC_*=en_GB.UTF-8) can't currently beyond 22.x. I think I was able with 23.x ;
some some of the sensivity of  cjk-enc.el to LANG/LC_* needs looking /documenting at better.
  

bug#25203: [cjk] Fwd: HELP with emacs 25.1 and cjk-enc.el

[Hin-Tak Leung]

Sorry, should be this (note "locale-coding-system"):

--- a/utils/lisp/emacs/cjk-enc.el
+++ b/utils/lisp/emacs/cjk-enc.el
@@ -879,8 +879,9 @@
         (if (> (- (point) last-pos) 1000)
             (progn
               (setq last-pos (point))
-              (message "Converting: %2d%%"
-                       (/ (* 100 (point)) (point-max)))))
+              (let ((coding-system-for-write locale-coding-system))
+                (message "Converting: %2d%%"
+                         (/ (* 100 (point)) (point-max))))))
 
         ;; Advance to the next character and loop.
         (forward-char 1))

--------------------------------------------
On Thu, 15/12/16, Hin-Tak Leung <htl10@users.sourceforge.net> wrote:
 
 Thanks for the advice from Eli Zaretskii
 ( https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25203#11 )
 

bug#25203: [cjk] Fwd: HELP with emacs 25.1 and cjk-enc.el

[Eli Zaretskii]

> Date: Thu, 15 Dec 2016 22:57:50 +0000 (UTC)
> From: Hin-Tak Leung <htl10@users.sourceforge.net>
> Cc: cjk-list@nongnu.org, by@moscito.org
> 
> -              (message "Converting: %2d%%"
> -                       (/ (* 100 (point)) (point-max)))))
> +              (let ((coding-system-for-write 'cjk-coding))
> +                (message "Converting: %2d%%"
> +                         (/ (* 100 (point)) (point-max))))))

I don't think using cjk-coding is correct here, you should use
locale-coding-system instead.  This message is going to be output to
the user's terminal, so the encoding it should use is that of the
terminal, which is most likely locale-coding-system.

You could also use 'us-ascii', since this particular message is
pure-ASCII.

Encoding a message in cjk-coding will fail if the user's terminal
cannot display some characters encoded with it.

bug#25203: [cjk] Fwd: HELP with emacs 25.1 and cjk-enc.el

[Eli Zaretskii]

> Date: Thu, 15 Dec 2016 23:02:03 +0000 (UTC)
> From: Hin-Tak Leung <htl10@users.sourceforge.net>
> Cc: cjk-list@nongnu.org, by@moscito.org
> 
> Sorry, should be this (note "locale-coding-system"):
> 
> --- a/utils/lisp/emacs/cjk-enc.el
> +++ b/utils/lisp/emacs/cjk-enc.el
> @@ -879,8 +879,9 @@
>          (if (> (- (point) last-pos) 1000)
>              (progn
>                (setq last-pos (point))
> -              (message "Converting: %2d%%"
> -                       (/ (* 100 (point)) (point-max)))))
> +              (let ((coding-system-for-write locale-coding-system))
> +                (message "Converting: %2d%%"
> +                         (/ (* 100 (point)) (point-max))))))
>  
>          ;; Advance to the next character and loop.
>          (forward-char 1))

Sorry, I don't understand the question.  Can you elaborate?

locale-coding-system is a variable whose value is the encoding
supported by the locale in which Emacs was started, and this it should
be appropriate for writing text to the user's terminal.

bug#25203: 25.1; crash during message, infinite recursion

[Eli Zaretskii]

> Date: Thu, 15 Dec 2016 17:48:57 +0100 (CET)
> Cc: hintak.leung@gmail.com, htl10@users.sourceforge.net,
>  25203@debbugs.gnu.org
> From: Werner LEMBERG <wl@gnu.org>
> 
> > In fact, any I/O that might require encoding should be completely
> > avoided in pre-write-conversion.
> 
> Is this documented?  If yes, I've missed it.

Frankly, it should be obvious: you are recursively invoking the same
operation that is being processed by the calling function.

I added a note about this to the doc string of define-coding-system.
However, I really doubt that this will do any tangible good, since we
don't document how to define a coding-system.  The ELisp manual says
just this:

     How to define a coding system is an arcane matter, and is not
  documented here.

Any reasons not to close this bug report, now that all of its aspects
have been addressed?

Thanks.

bug#25203: [cjk] Fwd: HELP with emacs 25.1 and cjk-enc.el

[Werner LEMBERG]

>> Sorry, should be this (note "locale-coding-system"): [...]
>
> Sorry, I don't understand the question.  Can you elaborate?

It's not a question :-)


    Werner

bug#25203: 25.1; crash during message, infinite recursion

[Werner LEMBERG]

>> > In fact, any I/O that might require encoding should be completely
>> > avoided in pre-write-conversion.
>> 
>> Is this documented?  If yes, I've missed it.
> 
> Frankly, it should be obvious: you are recursively invoking the same
> operation that is being processed by the calling function.

Well, `cjk-coding' worked flawlessly for years.  So it's not *that*
obvious.  In other words, there are rather recent changes w.r.t. to
`message' that make the once working code crash.

> I added a note about this to the doc string of define-coding-system.

Thanks.

> Any reasons not to close this bug report, now that all of its
> aspects have been addressed?

Yes, please do.


    Werner

bug#25203: [cjk] Fwd: HELP with emacs 25.1 and cjk-enc.el

[Andreas Schwab]

On Dez 16 2016, Eli Zaretskii <eliz@gnu.org> wrote:

> I don't think using cjk-coding is correct here, you should use
> locale-coding-system instead.  This message is going to be output to
> the user's terminal, so the encoding it should use is that of the
> terminal, which is most likely locale-coding-system.

(terminal-coding-system)

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#25203: 25.1; crash during message, infinite recursion

[Eli Zaretskii]

> Date: Fri, 16 Dec 2016 10:08:55 +0100 (CET)
> Cc: hintak.leung@gmail.com, htl10@users.sourceforge.net,
>  25203@debbugs.gnu.org
> From: Werner LEMBERG <wl@gnu.org>
> 
> >> > In fact, any I/O that might require encoding should be completely
> >> > avoided in pre-write-conversion.
> >> 
> >> Is this documented?  If yes, I've missed it.
> > 
> > Frankly, it should be obvious: you are recursively invoking the same
> > operation that is being processed by the calling function.
> 
> Well, `cjk-coding' worked flawlessly for years.

The problem was triggered by adding the 2 'message' calls, after the
point where coding-system-for-write was bound to this coding-system.
Without those 'message' calls, cjk-coding indeed should still work
flawlessly.

> > Any reasons not to close this bug report, now that all of its
> > aspects have been addressed?
> 
> Yes, please do.

Done.

bug#25203: [cjk] Fwd: HELP with emacs 25.1 and cjk-enc.el

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: Hin-Tak Leung <htl10@users.sourceforge.net>,  cjk-list@nongnu.org,  by@moscito.org,  25203@debbugs.gnu.org
> Date: Fri, 16 Dec 2016 10:28:36 +0100
> 
> On Dez 16 2016, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > I don't think using cjk-coding is correct here, you should use
> > locale-coding-system instead.  This message is going to be output to
> > the user's terminal, so the encoding it should use is that of the
> > terminal, which is most likely locale-coding-system.
> 
> (terminal-coding-system)

Yes, that's probably better, thanks.

bug#25203: [cjk] Fwd: HELP with emacs 25.1 and cjk-enc.el

[Hin-Tak Leung]

--------------------------------------------
On Fri, 16/12/16, Eli Zaretskii <eliz@gnu.org> wrote:
 
> Sorry, I don't understand the question. 
 Can you elaborate?
 
It is not a question - it is a follow-up correction to a proposed (wrong) bug fix to cjk-enc.el.

This thread was started because cjk-enc.el broke with emacs 25. So I was close
enough - and made it worse to crash with two new mesage()'s. The crash surprisingly helps, in the end.
The initial breakage was because there was already a message() (for progress reporting, which
has worked for 10+ years until 24.5) a few lines above the two new ones I added. 

The relevant change in emacs 25 is, as you suggested, this:
http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=c63246628461f748d66a8a07ba008de2e00fd33a
'Obey coding-system-for-write when writing stdout/stderr in batch'

There seems to be another breakage of cjk-enc.el - somehow, `write-region in emacs 25 for big5 input
is calling cjk-encode (pre-write-conversion) more than once, and the 2nd time wrongly. This only
happens with emacs 25 and only with big5 input (i.e. not with earlier emacs nor thai/mule input).
So there is another recursion somewhere - still looking...

I added a (backtrace) to cjk-encode and the 2nd+ calls are also from write-region - so definitely needs
C-level debugging to look further  ( https://debbugs.gnu.org/cgi/bugreport.cgi?bug=8108 )

Yes, the issue with crash with message() in pre-write-conversion is well-understood now,
and thanks a lot for all the help. Bug 25203 can close.

bug#25203: 25.1; crash during message, infinite recursion

[Hin-Tak Leung]

--------------------------------------------
On Fri, 16/12/16, Werner LEMBERG <wl@gnu.org> wrote:
 
> Well, `cjk-coding' worked flawlessly for
 years.  So it's not *that*
 obvious. 
 In other words, there are rather recent changes w.r.t. to
 `message' that make the once working code
 crash.


The relevant change in emacs 25 is this:
http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=c63246628461f748d66a8a07ba008de2e00fd33a
'Obey coding-system-for-write when writing stdout/stderr in batch'

and part of the recent breakage is that, there is already a `message' before the two new ones I tried to add.
There is another recursion somewhere else related specifically to big5 input which I have
not pinned point yet, but I'll just continue in the cjk list.