bug#42088: [feature/native-comp] Lockup on opening TypeScript files

bug#42088: [feature/native-comp] Lockup on opening TypeScript files

[Sebastian Sturm]

Apologies in advance for this being a very vague bug report, but I don't 
know how to properly debug this kind of issue (pointers appreciated!)

I find that, using the latest version of gccemacs (commit 
#801e19d0ba8e048...) and typescript.el (commit #0fc729787007b5111f...) 
Emacs will lock up with 100% cpu usage whenever a TypeScript file is 
opened. Interrupting Emacs using SIGUSR2 produces the following backtrace:

Debugger entered--entering a function:
* #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_10>()
   typescript--ensure-cache(515)
   typescript--class-decl-matcher(515)
   font-lock-fontify-keywords-region(1 515 nil)
   font-lock-default-fontify-region(1 501 nil)
   font-lock-fontify-region(1 501)
   #f(compiled-function (fun) #<bytecode 
0x58bf2cebcbf23b5>)(font-lock-fontify-region)
   jit-lock--run-functions(1 501)
   jit-lock-fontify-now(1 501)
   jit-lock-function(1)
   redisplay_internal\ \(C\ function\)()

I didn't spot any literal lambda within typescript--ensure-cache so I 
assume it's being generated by some macro? Adding the melpa package 
repository to the gccemacs docker image, installing typescript-mode and 
opening a .ts file also causes the Docker version to lock up with 100% 
CPU usage so I assume this is not caused by my config (trying to send 
SIGUSR2 to emacs terminated the entire container).
If there's anything I can try out to debug this issue, I'll gladly do so.

Thanks for your amazing work on gccemacs!
Sebastian

The following is extracted from the template produced by 
report-emacs-bug, though given that the Docker image locks up too, I 
guess most of it isn't of interest anyway:

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 
3.22.30, cairo version 1.15.10)
  of 2020-06-27 built on cartman
Repository revision: 801e19d0ba8e048a9faa5d5169ec4183e41b0148
Repository branch: feature/native-comp
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Linux Mint 19.1

Recent messages:
Doom loaded 235 packages across 53 modules in 0.672s
Waiting for git... [2 times]
Loading /home/sebastian/.emacs.d/.local/cache/recentf...done
+Word-Wrap mode enabled in current buffer
current-kill: Kill ring is empty
Mark set [9 times]

Configured using:
  'configure --prefix=/usr/local/stow/gccemacs CC=gcc-10 'CFLAGS=-O3
  -mtune=native -march=native -DNDEBUG' --with-nativecomp'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS JSON PDUMPER
LCMS2 GMP

Important settings:
   value of $LC_MONETARY: en_US.UTF-8
   value of $LC_NUMERIC: en_US.UTF-8
   value of $LC_TIME: en_US.UTF-8
   value of $LANG: en_US.UTF-8
   locale-coding-system: utf-8

bug#42088: [feature/native-comp] Lockup on opening TypeScript files

[Andrea Corallo]

Sebastian Sturm <mail@sebastian-sturm.de> writes:

> Apologies in advance for this being a very vague bug report, but I
> don't know how to properly debug this kind of issue (pointers
> appreciated!)

Hi Sebastian,

no worries, at this stage we have finished the trivial bugs :)

I think the questions are two:

- Is Emacs just looping forever in
  F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_10 or is does
  something more complex?

- Where is F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_10
  defined?  Being lambdas regular objects is can come from any
  complilaiton unit.

I believe the easiest to answere is to compile using `comp-debug' 1 and
then while is hanging just trap with the gdb and see what is going on.

I managed to reproduce the issue, tomorrow I'll be curious to have a
look.

Thanks!

  Andrea

-- 
akrl@sdf.org

bug#42088: [feature/native-comp] Lockup on opening TypeScript files

[Andrea Corallo]

Okay after some digging I think I've an idea of what is going on:

the code was hanging in `typescript--ensure-cache' in the loop

(cl-loop while (re-search-forward typescript--quick-match-re-func nil t)...

This because typescript--quick-match-re-func is not set correctly going
back and back this is because `typescript--available-frameworks' is set
to nil.

IIUC the reason for that is: cl-macs is expanding cl-loop using various
`--cl-var--', these looks the same but each of this is a separete
uninterned symbol.  The native compiler squash them all toghether having
to pass them through the reader and a simple testcase like this fails to
behave as expected.

===
(require 'cl-lib)

(defun foo ()
  (cl-loop for xxx in '(a b)
           for yyy = xxx
           do (print xxx)))
===

This fails only compiling for dynamic scope because in lexical all
--cl-vars-- are absorbed as slots in the execution stack.

I suspect the solution is to have some renaming performed by the native
compiler not to confuse them.

  Andrea

--
akrl@sdf.org

bug#42088: [feature/native-comp] Lockup on opening TypeScript files

[Andrea Corallo]

Apparently some mail got lost.

Anyway should be fixed by 7f8512765a, please give it a try.

Thanks!

  Andrea

-- 
akrl@sdf.org

bug#42088: [feature/native-comp] Lockup on opening TypeScript files

[Sebastian Sturm]

Hi Andrea,

thanks for your swift fix and for explaining the underlying issue! I can 
confirm that I'm now no longer seeing the typescript-mode bug. I do see 
the --cl-rest-- issue instead, but apparently you already fixed that as 
well so I guess rebuilding Emacs will take care of that.
thanks again for gccemacs,
Sebastian

bug#42088: [feature/native-comp] Lockup on opening TypeScript files

[Stefan Monnier]

> Okay after some digging I think I've an idea of what is going on:
>
> the code was hanging in `typescript--ensure-cache' in the loop
>
> (cl-loop while (re-search-forward typescript--quick-match-re-func nil t)...
>
> This because typescript--quick-match-re-func is not set correctly going
> back and back this is because `typescript--available-frameworks' is set
> to nil.

Hmm... I'm afraid I can't follow this.  Could you provide some details?

> IIUC the reason for that is: cl-macs is expanding cl-loop using various
> `--cl-var--', these looks the same but each of this is a separete
> uninterned symbol.  The native compiler squash them all toghether having
> to pass them through the reader and a simple testcase like this fails to
> behave as expected.

How/where exactly do they get squashed?

The printer is normally able to preserve this info (printing #:<foo>
instead of <foo> for uninterned symbols and using #= and such the refer
to exactly that symbol) when printing code into the .elc file, so I'm
wondering why it gets lost when going through the native compiler.


        Stefan

bug#42088: [feature/native-comp] Lockup on opening TypeScript files

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Okay after some digging I think I've an idea of what is going on:
>>
>> the code was hanging in `typescript--ensure-cache' in the loop
>>
>> (cl-loop while (re-search-forward typescript--quick-match-re-func nil t)...
>>
>> This because typescript--quick-match-re-func is not set correctly going
>> back and back this is because `typescript--available-frameworks' is set
>> to nil.
>
> Hmm... I'm afraid I can't follow this.  Could you provide some details?

Sure,

this is how `typescript--available-frameworks' is computed in
typescript-mode.el.

===
(defconst typescript--available-frameworks
  (cl-loop with available-frameworks
        for style in typescript--class-styles
        for framework = (plist-get style :framework)
        unless (memq framework available-frameworks)
        collect framework into available-frameworks
        finally return available-frameworks)
  "List of available typescript frameworks symbols.")
===

The loop is expanded in:

===
(cl-block nil
    (let*
        ((available-frameworks nil)
         (--cl-var-- typescript--class-styles)
         (style nil)
         (framework nil)
         (available-frameworks nil)
         (--cl-var-- t))
      (while
          (consp --cl-var--)
        (setq style
              (car --cl-var--))
        (setq framework
              (plist-get style :framework))
        (if
            (memq framework available-frameworks)
            (progn)
          (setq available-frameworks
                (nconc available-frameworks
                       (list framework))))
        (setq --cl-var--
              (cdr --cl-var--))
        (setq --cl-var-- nil))
      available-frameworks))
===

If the two --cl-var-- are confused we never iterate and the block
evaluates to nil.  As a result `typescript--available-frameworks' was
(incorrectly) set to nil.

>> IIUC the reason for that is: cl-macs is expanding cl-loop using various
>> `--cl-var--', these looks the same but each of this is a separete
>> uninterned symbol.  The native compiler squash them all toghether having
>> to pass them through the reader and a simple testcase like this fails to
>> behave as expected.
>
> How/where exactly do they get squashed?
>
> The printer is normally able to preserve this info (printing #:<foo>
> instead of <foo> for uninterned symbols and using #= and such the refer
> to exactly that symbol) when printing code into the .elc file, so I'm
> wondering why it gets lost when going through the native compiler.

Yes that's the conclusion I came-up shortly after.  Turned out that the
native compiler was not configuring the printer to handle uninterned
symbols, so the fix I pushed Sunday:

7f8512765a * Setup correctly the printer while dumping objs in native CU (bug#42088)

I corrected myself and discussed the fix in a mail sent into this thread
but unfortunately this got lost.  My sdf mail lost a number of mails in
the last days both incoming and out-coming (possibly including one I sent
you :).

This mail lossage has been extremely annoying sorry.

  Andrea

bug#42088: [feature/native-comp] Lockup on opening TypeScript files

[Stefan Monnier]

>> The printer is normally able to preserve this info (printing #:<foo>
>> instead of <foo> for uninterned symbols and using #= and such the refer
>> to exactly that symbol) when printing code into the .elc file, so I'm
>> wondering why it gets lost when going through the native compiler.
> Yes that's the conclusion I came-up shortly after.  Turned out that the
> native compiler was not configuring the printer to handle uninterned
> symbols, so the fix I pushed Sunday:
> 7f8512765a * Setup correctly the printer while dumping objs in native CU (bug#42088)

Great thanks!

> I corrected myself and discussed the fix in a mail sent into this thread
> but unfortunately this got lost.  My sdf mail lost a number of mails in
> the last days both incoming and out-coming (possibly including one I sent
> you :).

[ Not sure what's the standard euphemism in use in France nowadays but
  back when I lived nearby, "SDF" meant "sans domicile fixe" i.e. what we
  here call "homeless" (and I've heard it referred to disturbingly as
  "rough sleepers").  ]


        Stefan

bug#42088: [feature/native-comp] Lockup on opening TypeScript files

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan wrote:

> [ Not sure what's the standard euphemism in use in France nowadays but
>   back when I lived nearby, "SDF" meant "sans domicile fixe" i.e. what we
>   here call "homeless" (and I've heard it referred to disturbingly as
>   "rough sleepers").  ]

Well... must say this sounds surprisingly accurate! :/

Closing it :)

  Andrea