bug#56495: 29.0.50; Support for debugging Emacs with LLDB

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Gerd Möllmann]

[-- Attachment #1.1: Type: text/plain, Size: 629 bytes --]

My system, macOS with Apple M1 chip, is currently not supported by GDB.
To quote gdb-devel, "lldb is the way to go" to debug Emacs for me.

Attached patch adds rather limited support for that.  Limited by

- the fact that I don't know LLDB,
- that I don't know LLDB's Python API,
- that I'm not a Python programmer,
- that the Python API documentation is pretty lacking in itself,
- that I didn't implement support for ENABLE_CHECKING and what else
  might change Lisp_Object layout

So please bear with me.

Anyway, at least displaying some Lisp_Objects with 'p obj' seems to
work, and 'xbacktrace' seems to be working.

YMMV.


[-- Attachment #1.2: 0001-Support-for-debugging-Emacs-with-LLDB.patch --]
[-- Type: application/octet-stream, Size: 8978 bytes --]

From c5cfd261a549740a0619eb1854032778204dc51e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gerd=20M=C3=B6llmann?= <gerd@gnu.org>
Date: Sun, 10 Jul 2022 13:35:32 +0200
Subject: [PATCH] Support for debugging Emacs with LLDB

* (src/.lldbinit): New file.
* (etc/emacs_lldb.py): Module loaded from .lldbinit.
---
 etc/emacs_lldb.py | 166 ++++++++++++++++++++++++++++++++++++++++++++++
 src/.lldbinit     |  33 +++++++++
 2 files changed, 199 insertions(+)
 create mode 100644 etc/emacs_lldb.py
 create mode 100644 src/.lldbinit

diff --git a/etc/emacs_lldb.py b/etc/emacs_lldb.py
new file mode 100644
index 0000000000..3a9f17e020
--- /dev/null
+++ b/etc/emacs_lldb.py
@@ -0,0 +1,166 @@
+# Copyright (C) 2022 Free Software Foundation, Inc.
+#
+# This file is part of GNU Emacs.
+#
+# GNU Emacs is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GNU Emacs is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
+
+# Load this module in LLDB with
+#
+# (lldb) command script import emacs_lldb
+#
+# Available commands start with 'x' and can be seen with
+#
+# (lldb) help
+
+import lldb
+
+\f
+########################################################################
+#                              Utilties
+########################################################################
+
+# Return the Lisp_Type of Lisp_Object OBJ.
+def get_lisp_type(obj):
+    int_value = obj.GetValueAsUnsigned()
+    return obj.GetFrame().EvaluateExpression(
+        f"(enum Lisp_Type) ((EMACS_INT) {int_value} "
+        "& (1 << GCTYPEBITS) - 1)")
+
+# Return the Lisp_Type or pseudo-vector type of OBJ.
+def get_lisp_type_or_vectorlike(obj):
+    lisp_type = get_lisp_type(obj)
+    if enumerator_name(lisp_type) == "Lisp_Vectorlike":
+        vector = get_lisp_pointer(obj, "struct Lisp_Vector")
+        header_size = vector.GetValueForExpressionPath(
+            "->header.size").GetValueAsUnsigned()
+        frame = obj.GetFrame()
+        pseudo = frame.EvaluateExpression(
+            f"{header_size} & PSEUDOVECTOR_FLAG")
+        if pseudo.GetValueAsUnsigned() != 0:
+            return frame.EvaluateExpression(
+                f"(enum pvec_type) (({header_size} "
+                "& More_Lisp_Bits::PVEC_TYPE_MASK) "
+                ">> More_Lisp_Bits::PSEUDOVECTOR_AREA_BITS)")
+        return frame.EvaluateExpression("pvec_type::PVEC_NORMAL_VECTOR")
+    return lisp_type
+
+# Return Lisp_Object OBJ as pointer to TYP *.
+def get_lisp_pointer(obj, typ):
+    return obj.GetFrame().EvaluateExpression(
+        f"({typ}*) (((EMACS_INT) {obj.GetValueAsUnsigned()}) & VALMASK)")
+
+# Return Lisp_Object OBJ as pointer to Lisp_Symbol.
+def get_lisp_symbol(obj):
+    ptr = get_lisp_pointer(obj, "char")
+    offset = ptr.GetValueAsUnsigned()
+    return obj.GetFrame().EvaluateExpression(
+        f"(struct Lisp_Symbol *) ((char *) &lispsym + {offset})")
+
+# Return Lisp_Object OBJ as pointer to Lisp_String
+def get_lisp_string(obj):
+    return get_lisp_pointer(obj, "struct Lisp_String")
+
+# Return the string data of Lisp_Object OBJ which denotes a Lisp_String.
+def get_lisp_string_data(obj):
+    string = get_lisp_string(obj)
+    return string.GetValueForExpressionPath("->u.s.data")
+
+# Assuming OBJ denotes a Lisp_Symbol, return the name of the symbol.
+def get_lisp_symbol_name(obj):
+    sym = get_lisp_symbol(obj)
+    name = sym.GetValueForExpressionPath("->u.s.name")
+    return get_lisp_string_data(name)
+
+# Return a string for the enuerator ENUM.
+def enumerator_name(enum):
+    enumerators = enum.GetType().GetEnumMembers()
+    return enumerators[enum.GetValueAsUnsigned()].GetName()
+
+\f
+########################################################################
+#                           LLDB Commands
+########################################################################
+
+def xbacktrace(debugger, command, ctx, result, internal_dict):
+    """Print Emacs Lisp backtrace"""
+    frame = ctx.GetFrame()
+    n = frame.EvaluateExpression(
+        "current_thread->m_specpdl_ptr - current_thread->m_specpdl")
+    for i in reversed(range(0, n.GetValueAsUnsigned())):
+        s = frame.EvaluateExpression(f"current_thread->m_specpdl[{i}]")
+        kind = enumerator_name(s.GetChildMemberWithName("kind"))
+        if kind == "SPECPDL_BACKTRACE":
+            function = s.GetValueForExpressionPath(".bt.function")
+            function_type = enumerator_name(get_lisp_type(function))
+            if function_type == "Lisp_Symbol":
+                sym_name = get_lisp_symbol_name(function)
+                result.AppendMessage(str(sym_name))
+            elif function_type == "Lisp_Vectorlike":
+                subtype = get_lisp_type_or_vectorlike(function)
+                result.AppendMessage(str(subtype))
+            else:
+                result.AppendMessage(function_type)
+
+def xdebug_print(debugger, command, result, internal_dict):
+    """Print Lisp_Objects using safe_debug_print()"""
+    debugger.HandleCommand(f"expr safe_debug_print({command})")
+
+\f
+########################################################################
+#                             Formatters
+########################################################################
+
+# Return a type summary for Lisp_Objects.
+def format_Lisp_Object(obj, internal_dict):
+    lisp_type = get_lisp_type_or_vectorlike(obj)
+    kind = enumerator_name(lisp_type)
+    summary = "-> "
+    if kind == "PVEC_FRAME":
+        ptr = get_lisp_pointer(obj, "struct frame")
+        summary += str(ptr)
+    elif kind == "PVEC_WINDOW":
+        ptr = get_lisp_pointer(obj, "struct window")
+        summary += str(ptr)
+    return summary
+
+\f
+########################################################################
+#                           Initialization
+########################################################################
+
+# Define Python FUNCTION as an LLDB command.
+def define_command (debugger, function):
+    lldb_command = function.__name__
+    python_function = __name__ + "." + function.__name__
+    debugger.HandleCommand(f"command script add "
+                           f"--overwrite "
+                           f"--function {python_function} "
+                           f"{lldb_command}")
+
+# Define Python FUNCTION as an LLDB type formatter.
+def define_formatter(debugger, regex, function):
+    python_function = __name__ + "." + function.__name__
+    debugger.HandleCommand(f"type summary add "
+                           f"--cascade true "
+                           f'--regex "{regex}" '
+                           f"--python-function {python_function}")
+
+# This function is called by LLDB to initialize the module.
+def __lldb_init_module(debugger, internal_dict):
+    define_command(debugger, xbacktrace)
+    define_command(debugger, xdebug_print)
+    define_formatter(debugger, "Lisp_Object", format_Lisp_Object)
+    print('Emacs debugging support has been installed.')
+
+# end.
diff --git a/src/.lldbinit b/src/.lldbinit
new file mode 100644
index 0000000000..617d63958b
--- /dev/null
+++ b/src/.lldbinit
@@ -0,0 +1,33 @@
+# -*- mode: shell-script -*-
+# Copyright (C) 1992-1998, 2000-2022 Free Software Foundation, Inc.
+#
+# This file is part of GNU Emacs.
+#
+# GNU Emacs is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GNU Emacs is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
+#
+# Use 'lldb --local-init' or add to your ~/.lldbinit the line
+#
+# settings set target.load-cwd-lldbinit true
+#
+# Emacs-specific commands start with 'x'.  Type 'help' to see all
+# commands.  Type 'help <command>' to see help for a command
+# <command>.
+
+# Make Python find our files
+script -- sys.path.append('../etc')
+
+# Load our Python files
+command script import emacs_lldb
+
+# end.
-- 
2.37.0


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 874 bytes --]

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Robert Pluim]

>>>>> On Mon, 11 Jul 2022 10:13:35 +0200, Gerd Möllmann <gerd.moellmann@gmail.com> said:

    Gerd> My system, macOS with Apple M1 chip, is currently not supported by GDB.
    Gerd> To quote gdb-devel, "lldb is the way to go" to debug Emacs for me.

    Gerd> Attached patch adds rather limited support for that.  Limited by

    Gerd> - the fact that I don't know LLDB,
    Gerd> - that I don't know LLDB's Python API,
    Gerd> - that I'm not a Python programmer,
    Gerd> - that the Python API documentation is pretty lacking in itself,
    Gerd> - that I didn't implement support for ENABLE_CHECKING and what else
    Gerd>   might change Lisp_Object layout

    Gerd> So please bear with me.

Itʼs better than what we have now, so thanks

    Gerd> Anyway, at least displaying some Lisp_Objects with 'p obj' seems to
    Gerd> work, and 'xbacktrace' seems to be working.

    Gerd> YMMV.

Iʼve got:
    lldb -v
    lldb-1300.0.42.3
    Swift version 5.5.2-dev

(this is an Intel macbook, not an M1)

Iʼm getting this:

    lldb emacs
    error: unknown or ambiguous option
    error: unknown or ambiguous option
    Emacs debugging support has been installed.
    (lldb) target create "emacs"
    Current executable set to ʼ/Users/rpluim/repos/emacs/src/emacsʼ
    (x86_64).

which I can fix by doing this:

diff --git a/etc/emacs_lldb.py b/etc/emacs_lldb.py
index 3a9f17e020..ebf14d44c2 100644
--- a/etc/emacs_lldb.py
+++ b/etc/emacs_lldb.py
@@ -144,7 +144,6 @@ def define_command (debugger, function):
     lldb_command = function.__name__
          python_function = __name__ + "." + function.__name__
               debugger.HandleCommand(f"command script add "
-                           f"--overwrite "
                            f"--function {python_function} "
                            f"{lldb_command}")

Robert
-- 

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Gerd Möllmann]

[-- Attachment #1.1: Type: text/plain, Size: 1850 bytes --]



> On 2022-07-11,, at 11:18 , Robert Pluim <rpluim@gmail.com> wrote:
> 
>>>>>> On Mon, 11 Jul 2022 10:13:35 +0200, Gerd Möllmann <gerd.moellmann@gmail.com> said:
> Iʼve got:
>    lldb -v
>    lldb-1300.0.42.3
>    Swift version 5.5.2-dev

Thanks.

Looks like Apple's LLDB from the Xcode command-line tools.

Mine says "lldb version 14.0.6", and comes from "brew install llvm".  (Which I need for 'llvm-vscode', which I need for 'dap-mode', which I need for debugging with LLDB inside of Emacs, which is because of <known long story here>, and <another, shorter story here>).

I need '--overwrite' because it allows me to reload the code in a running LLDB.  Apple's LLDB seems to '--overwrite' by default, while mine gives an error:

  cannot add command: user command exists and force replace not set

\o/

Maybe I can somehow pythonese if '--overwrite' is supported or not.


> (this is an Intel macbook, not an M1)
> 
> Iʼm getting this:
> 
>    lldb emacs
>    error: unknown or ambiguous option
>    error: unknown or ambiguous option
>    Emacs debugging support has been installed.
>    (lldb) target create "emacs"
>    Current executable set to ʼ/Users/rpluim/repos/emacs/src/emacsʼ
>    (x86_64).
> 
> which I can fix by doing this:
> 
> diff --git a/etc/emacs_lldb.py b/etc/emacs_lldb.py
> index 3a9f17e020..ebf14d44c2 100644
> --- a/etc/emacs_lldb.py
> +++ b/etc/emacs_lldb.py
> @@ -144,7 +144,6 @@ def define_command (debugger, function):
>     lldb_command = function.__name__
>          python_function = __name__ + "." + function.__name__
>               debugger.HandleCommand(f"command script add "
> -                           f"--overwrite "
>                            f"--function {python_function} "
>                            f"{lldb_command}")
> 
> Robert
> --


[-- Attachment #1.2: Type: text/html, Size: 5004 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 874 bytes --]

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Robert Pluim]

>>>>> On Mon, 11 Jul 2022 12:42:04 +0200, Gerd Möllmann <gerd.moellmann@gmail.com> said:

    >> On 2022-07-11,, at 11:18 , Robert Pluim <rpluim@gmail.com> wrote:
    >> 
    >>>>>>> On Mon, 11 Jul 2022 10:13:35 +0200, Gerd Möllmann <gerd.moellmann@gmail.com> said:
    >> Iʼve got:
    >> lldb -v
    >> lldb-1300.0.42.3
    >> Swift version 5.5.2-dev

    Gerd> Thanks.

    Gerd> Looks like Apple's LLDB from the Xcode command-line tools.

    Gerd> Mine says "lldb version 14.0.6", and comes from "brew install llvm".
    Gerd> (Which I need for 'llvm-vscode', which I need for 'dap-mode', which I
    Gerd> need for debugging with LLDB inside of Emacs, which is because of
    Gerd> <known long story here>, and <another, shorter story here>).

I have lldb 13.0.1 from llvm here as well, but it gives me the same
error.

    Gerd> I need '--overwrite' because it allows me to reload the code in a
    Gerd> running LLDB.  Apple's LLDB seems to '--overwrite' by default, while
    Gerd> mine gives an error:

    Gerd>   cannot add command: user command exists and force replace not set

But if I 'command script import emacs_lldb' in my llvm lldb, I get no
errors. Maybe itʼs a version 14 thing.

    Gerd> \o/

🤷 indeed

    Gerd> Maybe I can somehow pythonese if '--overwrite' is supported or not.

I donʼt think itʼs a big deal, and lldb version dependent checks will
invariably be out of date. Maybe just stick in a comment?

Robert
-- 

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Date: Mon, 11 Jul 2022 10:13:35 +0200
> 
> Attached patch adds rather limited support for that.  Limited by
> 
> - the fact that I don't know LLDB,
> - that I don't know LLDB's Python API,
> - that I'm not a Python programmer,
> - that the Python API documentation is pretty lacking in itself,
> - that I didn't implement support for ENABLE_CHECKING and what else
>   might change Lisp_Object layout
> 
> So please bear with me.
> 
> Anyway, at least displaying some Lisp_Objects with 'p obj' seems to
> work, and 'xbacktrace' seems to be working.

Thanks.  Feel free to install when you think this is ready.

Maybe also add a few words in etc/DEBUG about this.

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Gerd Möllmann]

[-- Attachment #1: Type: text/plain, Size: 1155 bytes --]



> On 2022-07-11,, at 12:56 , Robert Pluim <rpluim@gmail.com> wrote:
> 
> But if I 'command script import emacs_lldb' in my llvm lldb, I get no
> errors. Maybe itʼs a version 14 thing.

Looks like it is.

> I donʼt think itʼs a big deal, and lldb version dependent checks will
> invariably be out of date. Maybe just stick in a comment?

This seems to work for me with Apple's LLDB and the one from LLVM:

def define_command (debugger, function):
    lldb_command = function.__name__
    python_function = __name__ + "." + function.__name__
    interpreter = debugger.GetCommandInterpreter()
    def define(overwrite):
        res = lldb.SBCommandReturnObject()
        interpreter.HandleCommand(f"command script add "
                                  f"{overwrite} "
                                  f"--function {python_function} "
                                  f"{lldb_command}",
                                  res)
        return res.Succeeded()
    if not define("--overwrite"):
        define("")

Could you please try it with your LLDBs?

It relies on runtime-availability of the '--overwrite', only.


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 874 bytes --]

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Gerd Möllmann]

[-- Attachment #1: Type: text/plain, Size: 204 bytes --]


> On 2022-07-11,, at 13:19 , Eli Zaretskii <eliz@gnu.org> wrote:
> Thanks.  Feel free to install when you think this is ready.
> 
> Maybe also add a few words in etc/DEBUG about this.

Thanks, will do.


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 874 bytes --]

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Robert Pluim]

>>>>> On Mon, 11 Jul 2022 13:27:22 +0200, Gerd Möllmann <gerd.moellmann@gmail.com> said:

    Gerd> Could you please try it with your LLDBs?

Works for me on the Apple and llvm lldbs, thanks

Robert
-- 

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I an unhappy with duplicating code in our sources for the sake of
supporting a competitor for GDB, that being for the sake of MacOS.
Duplicating code leads to extra maintenance work.

As I recall, GNU Emacs does not support MacOS anyway -- there is a
separate, modified version of Emacs for that.  This duplicate code
should go into that modified version.  The maintainers of that
version presumably think it is worth making an effort to
improve MacOS support -- so let them do the work.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Stefan Kangas]

Richard Stallman <rms@gnu.org> writes:

> As I recall, GNU Emacs does not support MacOS anyway

GNU Emacs supports "MacOS X 10.6 or newer" (etc/MACHINES).

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Gerd Möllmann]

[-- Attachment #1: Type: text/plain, Size: 1357 bytes --]



> On 2022-07-12,, at 5:04 , Richard Stallman <rms@gnu.org> wrote:

> As I recall, GNU Emacs does not support MacOS anyway -- there is a
> separate, modified version of Emacs for that.  This duplicate code
> should go into that modified version.  The maintainers of that
> version presumably think it is worth making an effort to
> improve MacOS support -- so let them do the work.
> 

(As Stefan explained - macOS is supported. EFQ.  You probably think of Aquamacs, which AFAIU is a kind of fork of Emacs that started in the early 2000s.  I don't know much more about it.  They will be assimilated.)

I'd also rather use GDB because of Emacs' .gdbinit.  The problem is GDB: it supports macOS on amd64, but not on arm64 (patches welcome, says gdb-devel), and arm64 is what I have.

So, for now, I've added 'xbacktrace' and 'pr' (safe_debug_print()) for LLDB. Both could be useful for people investigating bugs, I think, as a bare minimum.

For anyone listening and interested:

I've looked at the GDB sources for the first time today.  GDB supports arm64 on other platforms, and it supports macOS with amd64, so it might not be too hard to add something for someone who knows what he is doing.  So far, I'm not that someone, and I don't want to spend a ton of time on programming anyway these days, so I won't promise to do anything.


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 874 bytes --]

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > GNU Emacs supports "MacOS X 10.6 or newer" (etc/MACHINES).

I stand corrected.  Thanks.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#56495: 29.0.50; Support for debugging Emacs with LLDB

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

>     Gerd> Could you please try it with your LLDBs?
>
> Works for me on the Apple and llvm lldbs, thanks

This was two months ago, and it looks like this was added at the time:

commit 7af425f87bf9866c60ac134cbb6aa9eb0c61f8af
Author:     Gerd Möllmann <gerd@gnu.org>
AuthorDate: Sun Jul 10 13:35:32 2022 +0200

    Support for debugging Emacs with LLDB

So I'm closing this bug report now.