bug#30151: Debugger API

bug#30151: Debugger API

[James Nguyen]

I've been meaning to look at how to implement a debugger for Emacs for various languages.

There seems to be various options to go with (realgud, gud/gud-mi?, NIH roll my own) and it seems the community chooses different paths (including not writing one at all).

Some debuggers that come to mind are: edebug, gud-gdb, realgud, cider, indium, jdibug with a varied feature set.

I'm curious if it makes sense (or is doable) to have something similar to flycheck/flymake but for debugging (or like VSCode's https://code.visualstudio.com/docs/extensionAPI/api-debugging) so that there's a common interface to writing a debugger.

gud-def looks to be the closest thing but it seems somewhat low level given it doesn't draw breakpoints on screen (random example) or provide something like a 'locals' view.

If gud-def is the recommended approach, I wonder why the other debuggers (list mentioned above) don't leverage it.


-- 
  James Nguyen
  james@jojojames.com

bug#30151: Debugger API

[Eli Zaretskii]

> From: James Nguyen <james@jojojames.com>
> Date: Wed, 17 Jan 2018 20:56:38 -0800
> 
> I've been meaning to look at how to implement a debugger for Emacs for various languages.
> 
> There seems to be various options to go with (realgud, gud/gud-mi?, NIH roll my own) and it seems the community chooses different paths (including not writing one at all).
> 
> Some debuggers that come to mind are: edebug, gud-gdb, realgud, cider, indium, jdibug with a varied feature set.
> 
> I'm curious if it makes sense (or is doable) to have something similar to flycheck/flymake but for debugging (or like VSCode's https://code.visualstudio.com/docs/extensionAPI/api-debugging) so that there's a common interface to writing a debugger.

Emacs provides an interface to _debuggers_, not to languages.  For
example, you can debug any language supported by GDB using gdb-mi.el,
but will have to use "M-x pdb" (defined in gud.el) for Perl debugging.

IOW, unlike VS, which is a single debugging engine, Emacs supports
several debugging engines.  The closest thing to VSCode's Debugging
API we have is therefore gdb-mi.el -- if you are willing to limit
yourself to those languages currently supported by GDB.

So I'm not sure I understand your idea well enough to answer your
questions in a useful way, at least not if I want to be sure I gave a
complete answer that you can use to decide how to go about this
project.  Can you elaborate on your idea given the above
considerations?

> gud-def looks to be the closest thing but it seems somewhat low level given it doesn't draw breakpoints on screen (random example) or provide something like a 'locals' view.
> 
> If gud-def is the recommended approach, I wonder why the other debuggers (list mentioned above) don't leverage it.

gud-def (and gud.el in general) is supposed to be the extensible
debugging interface, yes.  However, the capabilities it can provide
depend critically on what the is supported by the underlying debugger.
E.g., displaying breakpoints requires that the debugger could be
queried about the location(s) of each breakpoint, and that it returns
the results of the query in a way that Emacs can unequivocally parse.

I cannot tell why packages you mentioned that support debugging roll
their own; perhaps the respective package developers could chime in
and explain.

Thanks.

bug#30151: Debugger API

[James Nguyen]

> Emacs provides an interface to _debuggers_, not to languages.

Thanks for the correction, I've been using the two terms interchangeably.

> IOW, unlike VS, which is a single debugging engine, Emacs supports
> several debugging engines.  The closest thing to VSCode's Debugging
> API we have is therefore gdb-mi.el -- if you are willing to limit
> yourself to those languages currently supported by GDB.

I think the 'limited to languages supported by GDB' might be somewhat limiting.

> So I'm not sure I understand your idea well enough to answer your
> questions in a useful way, at least not if I want to be sure I gave a
> complete answer that you can use to decide how to go about this
> project.

I think what Microsoft has done with their debugger is very interesting in that
most of the UI seems to be a single interface and what interacts (debugger adapters)
with the debugger can do what it wants as long as it can communicate down to that
same interface/UI.

It reminds me of their approach with their language server protocol.

> Can you elaborate on your idea given the above
> considerations?

This topic came out of a few things I've tried in the past.

First I tried using GDB on OSX but it seems GDB can't be used on OSX anymore
due to some system limitation. So I looked for LLDB gud integration. I found one but
setting breakpoints only sets them in the debugger and not visually
(i.e. red breakpoints in the fringe) like in gdb-mi? or realgud.
I was able to hack up the visual breakpoints but stopped
when it seemed to be too messy (straying from the well-trodden path).

Next, I tried pdb (I believe that's the python debugger) but it had the same minor problem
of not drawing breakpoints in the side panel. It seemed like only gdb-mi~ debugger/languages
supported that kind of feature. Taking a look at realgud, the 'visual breakpoint' 'feature'
is available. So from there, from a user-standpoint, I just went with realgud for python debugging.

Fast forward to now, I'm trying to add a debugger integration as I use new languages,
I'm back into looking at whether or not to use realgud or gud-def or roll my own.
Some that come to mind that probably don't use gdb are Typescript, Clojurescript, Elixir/Erlang so
it might be a mismatch with gud. I don't really know. At the moment, I'm interested in
Typescript/Clojurescript support, probably interfacing with Chrome. And/or a working LLDB integration.

Maybe there's just a gap in functionality between gud and gud-mi interface (sorry if I'm butchering the terms).

Ideally, it'd be nice to have something simple that asks me:
1. Where to draw breakpoints in the buffer.
2. What locals exist and displaying them in some kind of 'locals' buffer.
3. Maybe an extra window that will display extraneous information (similar to jdibug's stacktrace buffer)
4. I'm missing some other common functionalities between debuggers.

I think some/all of ^ is very similar to gdb-many-windows but I've only ever seen that for *just* gdb.

> I cannot tell why packages you mentioned that support debugging roll
> their own; perhaps the respective package developers could chime in
> and explain.

Yes! That would be clarifying. I hope those developers read this email chain.

Thanks.


-- 
  James Nguyen
  james@jojojames.com

On Thu, Jan 18, 2018, at 6:41 AM, Eli Zaretskii wrote:
> > From: James Nguyen <james@jojojames.com>
> > Date: Wed, 17 Jan 2018 20:56:38 -0800
> > 
> > I've been meaning to look at how to implement a debugger for Emacs for various languages.
> > 
> > There seems to be various options to go with (realgud, gud/gud-mi?, NIH roll my own) and it seems the community chooses different paths (including not writing one at all).
> > 
> > Some debuggers that come to mind are: edebug, gud-gdb, realgud, cider, indium, jdibug with a varied feature set.
> > 
> > I'm curious if it makes sense (or is doable) to have something similar to flycheck/flymake but for debugging (or like VSCode's https://code.visualstudio.com/docs/extensionAPI/api-debugging) so that there's a common interface to writing a debugger.
> 
> Emacs provides an interface to _debuggers_, not to languages.  For
> example, you can debug any language supported by GDB using gdb-mi.el,
> but will have to use "M-x pdb" (defined in gud.el) for Perl debugging.
> 
> IOW, unlike VS, which is a single debugging engine, Emacs supports
> several debugging engines.  The closest thing to VSCode's Debugging
> API we have is therefore gdb-mi.el -- if you are willing to limit
> yourself to those languages currently supported by GDB.
> 
> So I'm not sure I understand your idea well enough to answer your
> questions in a useful way, at least not if I want to be sure I gave a
> complete answer that you can use to decide how to go about this
> project.  Can you elaborate on your idea given the above
> considerations?
> 
> > gud-def looks to be the closest thing but it seems somewhat low level given it doesn't draw breakpoints on screen (random example) or provide something like a 'locals' view.
> > 
> > If gud-def is the recommended approach, I wonder why the other debuggers (list mentioned above) don't leverage it.
> 
> gud-def (and gud.el in general) is supposed to be the extensible
> debugging interface, yes.  However, the capabilities it can provide
> depend critically on what the is supported by the underlying debugger.
> E.g., displaying breakpoints requires that the debugger could be
> queried about the location(s) of each breakpoint, and that it returns
> the results of the query in a way that Emacs can unequivocally parse.
> 
> I cannot tell why packages you mentioned that support debugging roll
> their own; perhaps the respective package developers could chime in
> and explain.
> 
> Thanks.

bug#30151: Debugger API

[Eli Zaretskii]

> From: James Nguyen <james@jojojames.com>
> Cc: 30151@debbugs.gnu.org
> Date: Fri, 19 Jan 2018 01:25:47 -0800
> 
> Ideally, it'd be nice to have something simple that asks me:
> 1. Where to draw breakpoints in the buffer.
> 2. What locals exist and displaying them in some kind of 'locals' buffer.
> 3. Maybe an extra window that will display extraneous information (similar to jdibug's stacktrace buffer)
> 4. I'm missing some other common functionalities between debuggers.
> 
> I think some/all of ^ is very similar to gdb-many-windows but I've only ever seen that for *just* gdb.

I agree, but as I said, these capabilities need to be supported by the
underlying debuggers.  Currently, only gdb-mi.el provides them because
GDB includes a special interface for debugging front-ends, and that
interface provides the functionality required for the features shown
in gdb-many-windows.  If other debuggers can support similar
functionalities, gud.el could be extended to do that for them as well.

Perhaps the first step would be to look at what realgud does to
provide these features.  But please be aware that if you are going to
read realgud's source with the intent of doing something similar in
Emacs, you are advised to check with realgud's developers whether they
will be willing to sign legal paperwork, in case you end up copying
some of their code.

Thanks.

bug#30151: Debugger API

[James Nguyen]

Thanks, I've submitted an issue here -> https://github.com/realgud/realgud/issues/206.

Looking at some old code I did when adding breakpoints for lldb, it looks like I just used the marker-filter and matched against strings in the filter looking for 'breakpoint print' strings and extracting the info out after.

That might be similar to how something like flymake/flycheck runs a *something* in a separate process and then matches strings against the output to get the error output out.

E.g. http://ix.io/EsZ (meant just for discussion)

I'm not sure how we can better support this flow (or if it's even a good idea) but from what I've seen the marker-filter is the one place where we can 'extract' information from the underlying debugger (using its output).

-- 
  James Nguyen
  james@jojojames.com

On Fri, Jan 19, 2018, at 2:19 AM, Eli Zaretskii wrote:
> > From: James Nguyen <james@jojojames.com>
> > Cc: 30151@debbugs.gnu.org
> > Date: Fri, 19 Jan 2018 01:25:47 -0800
> > 
> > Ideally, it'd be nice to have something simple that asks me:
> > 1. Where to draw breakpoints in the buffer.
> > 2. What locals exist and displaying them in some kind of 'locals' buffer.
> > 3. Maybe an extra window that will display extraneous information (similar to jdibug's stacktrace buffer)
> > 4. I'm missing some other common functionalities between debuggers.
> > 
> > I think some/all of ^ is very similar to gdb-many-windows but I've only ever seen that for *just* gdb.
> 
> I agree, but as I said, these capabilities need to be supported by the
> underlying debuggers.  Currently, only gdb-mi.el provides them because
> GDB includes a special interface for debugging front-ends, and that
> interface provides the functionality required for the features shown
> in gdb-many-windows.  If other debuggers can support similar
> functionalities, gud.el could be extended to do that for them as well.
> 
> Perhaps the first step would be to look at what realgud does to
> provide these features.  But please be aware that if you are going to
> read realgud's source with the intent of doing something similar in
> Emacs, you are advised to check with realgud's developers whether they
> will be willing to sign legal paperwork, in case you end up copying
> some of their code.
> 
> Thanks.

bug#30151: Debugger API

[Stefan Kangas]

James Nguyen <james@jojojames.com> writes:

> Thanks, I've submitted an issue here -> https://github.com/realgud/realgud/issues/206.
>
> Looking at some old code I did when adding breakpoints for lldb, it looks like I just used the marker-filter and matched against strings in the filter looking for 'breakpoint print' strings and extracting the info out after.
>
> That might be similar to how something like flymake/flycheck runs a *something* in a separate process and then matches strings against the output to get the error output out.
>
> E.g. http://ix.io/EsZ (meant just for discussion)
>
> I'm not sure how we can better support this flow (or if it's even a good idea) but from what I've seen the marker-filter is the one place where we can 'extract' information from the underlying debugger (using its output).

Did anything ever come out of your investigations and experiments?  We
sure would welcome contributions and improvements to gud.el.

As it stands however, this is an open issue in our bug tracker, but I
can't really make out what it is concretely about.  Should this issue
perhaps be closed?

Best regards,
Stefan Kangas

bug#30151: Debugger API

[james]

Hi Stefan,

Please close it! I don’t think there is enough interest from any one party to keep it open.

Thanks!

> On Sep 28, 2019, at 3:56 PM, Stefan Kangas <stefan@marxist.se> wrote:
> 
> James Nguyen <james@jojojames.com> writes:
> 
>> Thanks, I've submitted an issue here -> https://github.com/realgud/realgud/issues/206.
>> 
>> Looking at some old code I did when adding breakpoints for lldb, it looks like I just used the marker-filter and matched against strings in the filter looking for 'breakpoint print' strings and extracting the info out after.
>> 
>> That might be similar to how something like flymake/flycheck runs a *something* in a separate process and then matches strings against the output to get the error output out.
>> 
>> E.g. http://ix.io/EsZ (meant just for discussion)
>> 
>> I'm not sure how we can better support this flow (or if it's even a good idea) but from what I've seen the marker-filter is the one place where we can 'extract' information from the underlying debugger (using its output).
> 
> Did anything ever come out of your investigations and experiments?  We
> sure would welcome contributions and improvements to gud.el.
> 
> As it stands however, this is an open issue in our bug tracker, but I
> can't really make out what it is concretely about.  Should this issue
> perhaps be closed?
> 
> Best regards,
> Stefan Kangas

bug#30151: Debugger API

[Stefan Kangas]

<james@jojojames.com> writes:

> Please close it! I don’t think there is enough interest from any one party to keep it open.

Thanks; I'll close it now.  Feel free to submit a new bug report if
you ever come up with any improvements or suggestions in the future.

Best regards,
Stefan Kangas