Re: MPS: native comp

["Gerd Möllmann" <gerd.moellmann@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1909 bytes --]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
> Hello Andrea,
>
> I have a question that probably only you can answer quickly :-).
>
> In the generated code we have things like
>
>   struct Lisp_X *[227] d_reloc;
>
> I would like the get the size of that array (and others that are
> similar), ie. I would like to get the 227 above, and if possible without
> relying on anything like the length of the array returned from
> load_static_obj or computing the size from addresses like &sym1 - &sym2.
>
> Is that possible? Is it somewhere in the C being emitted?
>
> TIA

Kind of a progress report, with the request to anyone understanding
halfway what I say in my debug notes to chime in because I'm stuck. I'm
obviously overlooking something.

Summary of the problem:

Eln files contain an array of Lisp_Objects named d_reloc. Compiled
functions referring to Lisp_Objects load these from d_reloc, say
d_reloc[1]. Igc traces the contents of d_reloc; when objects move,
references are made to point to their new location in memory. So
theoreically the compiled functions should be fine. When a function
loads from d_reloc it gets a traced reference, when a reference is on
its stack, the object doesn't move, so the Lisp_Object doesn't change.

That's the theory. What I see instead is that in one case a compiled
function gets something from d_reloc that has _not_ been traced. In this
case, a symbol has been moved in memory, afaict d_reloc has been traced,
i.e. the symbol ref should be to the new location of the symbol, and
despite that, the compiled function passes the old location to Emacs C
core.

Please add "it looks as if, theoretically" and so on everywhere.

Like I said, I'm overlooking something, abviously.

Debug log with more detailed info attached. I'll let this ripe now for
myself.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Debug log --]
[-- Type: text/x-org, Size: 9604 bytes --]

* set_internal

Changed comp.el to native-compile-debug 3 and native-compile-speed 0.
Build from git clean -xdf, kill after macroexp been native-compiled.
Then

#+begin_src sh
  target create temacs
  settings set -- target.run-args --batch  -l loadup --temacs=pbootstrap --bin-dest /Users/gerd/emacs/savannah/igc/nextstep/Emacs.app/Contents/MacOS/ --eln-dest /Users/gerd/emacs/savannah/igc/nextstep/Emacs.app/Contents/Frameworks/
  command alias go process launch --working-dir .
  settings set target.disable-aslr false
#+end_src

Run N times until symbol in set_internal points to tombstone, and we abort.

#+begin_src sh
  (lldb) p *(struct igc_header *) client_to_base (sym)
  (struct igc_header)  (obj_type = IGC_OBJ_FWD, pvec_type = PVEC_FREE, hash = 1074169, nwords = 7)
  (lldb) p *(struct igc_fwd *) client_to_base (sym)
  (struct igc_fwd) {
    header = (obj_type = IGC_OBJ_FWD, pvec_type = PVEC_FREE, hash = 1074169, nwords = 7)
    new_base_addr = 0x0000000110c6bb18
  }
  (lldb) p *(struct igc_header *) 0x0000000110c6bb18
  (struct igc_header)  (obj_type = IGC_OBJ_SYMBOL, pvec_type = PVEC_FREE, hash = 1074169, nwords = 7)
  (lldb) p *(struct Lisp_Symbol *) base_to_client ((void *) 0x0000000110c6bb18)
  (struct Lisp_Symbol) {
    u = {
      s = {
        gcmarkbit = false
        redirect = SYMBOL_PLAINVAL
        trapped_write = SYMBOL_UNTRAPPED_WRITE
        interned = SYMBOL_INTERNED_IN_INITIAL_OBARRAY
        declared_special = true
        pinned = false
        name = 0x0000000110529d94 (struct Lisp_String *) $8 = 0x0000000110529d90
        val = {
          value = 0x0000000110ca6b63 (struct Lisp_Cons *) $9 = 0x0000000110ca6b60
          alias = 0x0000000110ca6b63
          blv = 0x0000000110ca6b63
          fwd = (fwdptr = 0x0000000110ca6b63)
        }
        function = NULL
        plist = 0x0000000110529dbb (struct Lisp_Cons *) $10 = 0x0000000110529db8
        next = 0x0000000110529dd0
      }
      gcaligned = '\xc0'
    }
  }
  (lldb) xpostmortem
  (lldb) p *$8
  (struct Lisp_String) {
    u = {
      s = {
        size = 23
        size_byte = 23
        intervals = NULL
        data = 0x000000011051b5a8 "byte-compile-form-stack"
      }
      next = 0x0000000000000017
      gcaligned = '\x17'
    }
  }
  ##+end_src

  Comes from specbind with the broken symbol.

  ,#+begin_src sh
  (lldb) xbacktrace
  (unsigned char *) data = 0x000000011051b5c8 "macroexp--expand-all"
  (unsigned char *) data = 0x000000011051b5e8 "macroexp--all-forms"
  (unsigned char *) data = 0x000000011051b5c8 "macroexp--expand-all"
  (unsigned char *) data = 0x000000011051b5c8 "macroexp--expand-all"
  (unsigned char *) data = 0x000000011051b5e8 "macroexp--all-forms"
  (unsigned char *) data = 0x000000011051b5c8 "macroexp--expand-all"
  (unsigned char *) data = 0x000000011051b5e8 "macroexp--all-forms"
  (unsigned char *) data = 0x000000011051b5c8 "macroexp--expand-all"
  (unsigned char *) data = 0x000000011051b5e8 "macroexp--all-forms"
  (unsigned char *) data = 0x000000011051b5c8 "macroexp--expand-all"
  (unsigned char *) data = 0x000000011051b5e8 "macroexp--all-forms"
  (unsigned char *) data = 0x000000011051b5c8 "macroexp--expand-all"
  (unsigned char *) data = 0x000000011051b5e8 "macroexp--all-forms"
  (unsigned char *) data = 0x000000011051b5c8 "macroexp--expand-all"
  (unsigned char *) data = 0x0000000110c60780 "macroexpand--all-toplevel"
  (unsigned char *) data = 0x00000001014df2f0 "internal-macroexpand-for-load"
  (unsigned char *) data = 0x00000001014e104b "eval-buffer"
  (unsigned char *) data = 0x00000001014da312 "if"
  (unsigned char *) data = 0x00000001014da3e2 "let"
  (unsigned char *) data = 0x00000001014da3e2 "let"
  (unsigned char *) data = 0x00000001014da426 "unwind-protect"
  (unsigned char *) data = 0x00000001014da3e2 "let"
  (unsigned char *) data = 0x00000001014da312 "if"
  (unsigned char *) data = 0x0000000110c5b2f8 "load-with-code-conversion"
  (unsigned char *) data = 0x00000001014df6f9 "load"
  (unsigned char *) data = 0x00000001014df6f9 "load"

  (lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 6.1
    frame #0: 0x00000001014ba00c temacs`emacs_abort at sysdep.c:2391:3
    frame #1: 0x00000001011e67a8 temacs`set_internal(symbol=(struct Lisp_Symbol *) $156 = 0x0000000109612230, newval=(struct Lisp_Cons *) $157 = 0x0000000111244110, where=(struct Lisp_Symbol *) $158 = 0x0000000101c13a78, bindflag=SET_INTERNAL_BIND) at data.c:1665:14
    frame #2: 0x00000001012160e8 temacs`do_specbind(sym=0x0000000109612230, bind=0x000000014a032220, value=(struct Lisp_Cons *) $159 = 0x0000000111244110, bindflag=SET_INTERNAL_BIND) at eval.c:3444:9
  * frame #3: 0x000000010120bb44 temacs`specbind(symbol=(struct Lisp_Symbol *) $160 = 0x0000000109612230, value=(struct Lisp_Cons *) $161 = 0x0000000111244110) at eval.c:3531:3
  frame #4: 0x0000000101e39e54 macroexp-2c3e1495-76691d27.eln`F6d6163726f6578702d2d657870616e642d616c6c_macroexp__expand_all_0 + 116
    frame #5: 0x00000001012147f0 temacs`funcall_subr(subr=0x000000010a181528, numargs=1, args=(struct Lisp_Symbol *) $162 = 0x0000000270a8be90) at eval.c:3100:15

#+end_src

frame 3 does the specbind in the eln:

#+begin_src c
extern struct Lisp_X *
F6d6163726f6578702d2d657870616e642d616c6c_macroexp__expand_all_0 (struct Lisp_X * par_0)
{
  struct freloc_link_table * freloc;
  struct Lisp_X *[22] frame;
  struct comp_handler * c;

entry:
  freloc = freloc_link_table;
  /* Lisp function: macroexp--expand-all */
  frame[(int)0] = par_0;
  goto bb_0;

bb_0:
  frame[(int)1] = frame[(int)0];
  /* nil */
  /* const lisp obj: nil */
  /* calling subr: cons */
  frame[(int)1] = freloc->R636f6e73_cons_0 (frame[(int)1], (struct Lisp_X *)NULL);
  frame[(int)2] = frame[(int)1];
  /* calling subr: car-safe */
  frame[(int)2] = freloc->R6361722d73616665_car_safe_0 (frame[(int)2]);
  /* byte-compile-form-stack */
  /* const lisp obj: byte-compile-form-stack */
  /* l-value for lisp obj: byte-compile-form-stack */
  /* calling subr: symbol-value */
  frame[(int)3] = freloc->R73796d626f6c2d76616c7565_symbol_value_0 (d_reloc[(long)1]);
  /* calling subr: cons */
  frame[(int)2] = freloc->R636f6e73_cons_0 (frame[(int)2], frame[(int)3]);
  /* byte-compile-form-stack */
  /* const lisp obj: byte-compile-form-stack */
  /* l-value for lisp obj: byte-compile-form-stack */
  /* calling subr: specbind */
  (void)freloc->R7370656362696e64_specbind_0 (d_reloc[(long)1], frame[(int)2]);
  frame[(int)2] = frame[(int)1];
  /* calling subr: car-safe */
  frame[(int)2] = freloc->R6361722d73616665_car_safe_0 (frame[(int)2]);
#+end_src

Nothing suspicious to see, and it's speed 0. Debugging into this
doesn't work with LLDB OOTB. Have to investigate.

Can see by instrumenting igc that d_reloc[1] is traced and moved in
memory at various points.

#+begin_src sh
  (lldb) disassemble -f
macroexp-2c3e1495-76691d27.eln`F6d6163726f6578702d2d657870616e642d616c6c_macroexp__expand_all_0:
    0x1036c22ec <+0>:     stp    x29, x30, [sp, #-0xf0]!
    0x1036c22f0 <+4>:     mov    x29, sp
    0x1036c22f4 <+8>:     str    x19, [sp, #0x10]
    0x1036c22f8 <+12>:    str    x0, [sp, #0x28]
    0x1036c22fc <+16>:    adrp   x0, 19
    0x1036c2300 <+20>:    add    x0, x0, #0x9d0            ; freloc_link_table
    0x1036c2304 <+24>:    ldr    x0, [x0]
    0x1036c2308 <+28>:    str    x0, [sp, #0xe8]
    0x1036c230c <+32>:    ldr    x0, [sp, #0x28]
    0x1036c2310 <+36>:    str    x0, [sp, #0x38]
    0x1036c2314 <+40>:    nop
    0x1036c2318 <+44>:    ldr    x0, [sp, #0x38]
    0x1036c231c <+48>:    str    x0, [sp, #0x40]
    0x1036c2320 <+52>:    ldr    x0, [sp, #0xe8]
    0x1036c2324 <+56>:    ldr    x2, [x0, #0x2240]
    0x1036c2328 <+60>:    ldr    x0, [sp, #0x40]
    0x1036c232c <+64>:    mov    x1, #0x0                  ; =0
    0x1036c2330 <+68>:    blr    x2
    0x1036c2334 <+72>:    str    x0, [sp, #0x40]
    0x1036c2338 <+76>:    ldr    x0, [sp, #0x40]
    0x1036c233c <+80>:    str    x0, [sp, #0x48]
    0x1036c2340 <+84>:    ldr    x0, [sp, #0xe8]
    0x1036c2344 <+88>:    ldr    x1, [x0, #0x29d8]
    0x1036c2348 <+92>:    ldr    x0, [sp, #0x48]
    0x1036c234c <+96>:    blr    x1
    0x1036c2350 <+100>:   str    x0, [sp, #0x48]
    0x1036c2354 <+104>:   ldr    x0, [sp, #0xe8]
    0x1036c2358 <+108>:   ldr    x1, [x0, #0x2940]
    0x1036c235c <+112>:   adrp   x0, 18
    0x1036c2360 <+116>:   add    x0, x0, #0xee8            ; d_reloc
    0x1036c2364 <+120>:   ldr    x0, [x0, #0x8]
    0x1036c2368 <+124>:   blr    x1
    0x1036c236c <+128>:   str    x0, [sp, #0x50]
    0x1036c2370 <+132>:   ldr    x0, [sp, #0xe8]
    0x1036c2374 <+136>:   ldr    x2, [x0, #0x2240]
    0x1036c2378 <+140>:   ldr    x0, [sp, #0x48]
    0x1036c237c <+144>:   ldr    x1, [sp, #0x50]
    0x1036c2380 <+148>:   blr    x2
    0x1036c2384 <+152>:   str    x0, [sp, #0x48]
    0x1036c2388 <+156>:   ldr    x0, [sp, #0xe8]
    0x1036c238c <+160>:   ldr    x2, [x0, #0x60]
    0x1036c2390 <+164>:   adrp   x0, 18
    0x1036c2394 <+168>:   add    x0, x0, #0xee8            ; d_reloc
    0x1036 c2398 <+172>:   ldr    x0, [x0, #0x8]
    0x1036c239c <+176>:   ldr    x1, [sp, #0x48]
    0x1036c23a0 <+180>:   blr    x2
->  0x1036c23a4 <+184>:   ldr    x0, [sp, #0x40]
    0x1036c23a8 <+188>:   str    x0, [sp, #0x48]
    0x1036c23ac <+192>:   ldr    x0, [sp, #0xe8]
    0x1036c23b0 <+196>:   ldr    x1, [x0, #0x29d8]
    0x1036c23b4 <+200>:   ldr    x0, [sp, #0x48]
    0x1036c23b8 <+204>:   blr    x1
    0x1036c23bc <+208>:   str    x0, [sp, #0x48]

#+end_src

Disassemly looks totally as expectd, too. It uses d_reloc.

=> I'm overlooking something.