A Guile debugger workgroup?

A Guile debugger workgroup?

[Olivier Dion via Developers list for Guile, the GNU extensibility library]

Greetings Guilers,

There's been a discussion on the state of debugging with Guile on the
guix-devel mailing list.  Here's the relevant link if you want the full
discussion:
<https://lists.gnu.org/archive/html/guix-devel/2022-11/msg00283.html>
which is a reply to <https://issues.guix.gnu.org/issue/58812#id=33>.

I'm going to resume what I've gather here.  I've mainly cutoff Guix
related stuff.  I'm also putting names here for the sake of clarity.

I think it would be interesting to have more inputs from the Guile users
at large -- outside of Guix -- on that topic.  From my personal
experience and the echo I get back from other users, the debugging
experience in Guile is frustrating.

Things I've gather in this reading and on the IRC with other users.
This is by no way a wish list, but simply ideas to improve the debugging
experience:

  1. Documentation on debugging should be improved.  e.g. The `pk`
     procedure.

  2. A tutorial on how to debug a project with `pk` and the REPL.

  3. A single-step (instruction and line) debugger.

  4. Integration in GDB.  e.g. GDB could insert breakpoints in Scheme
     code.

  5. Maybe not make a debugger with a paradigm for language like C but
     instead take inspiration from other Scheme implementation like
     Racket.

Resume start here:

zimon:

  Preparing some materials for introducing Guile to GuixHPC folk, I am
  trying to collect some tips and, if I am honest, the debugging
  experience with Guile is really poor; compared to others (as Python).
  For example, DrRacket provides an easy and nice user experience

  Well, IMHO, we are somehow suffering from some Guile limitations and
  improvements in this area are an hard task.

Maxim:

  I also agree!  It's hard to convince people to pick Guile for their
  project when there is:

  1. Lack of a debugger that can break and step anywhere in your source
     code
     
  2. Lack of debugger integration to an IDE (it's not even integrated
     into Emacs)

  Perhaps we should assemble a Guile debugger workgroup that'd review
  what's broken, what's missing, what can be borrowed from other Scheme
  or languages for inspiration

Ludo:

  Well, Guile has a debugger that lets you do that [...] Geiser is not
  Visual Studio™ but it does a good job.

  Also, [...] I mentioned before that I almost never use breakpoints on
  Guile code [...]  because it’s rarely the right tool for the job.

  I believe this is largely due to (1) writing functional code, and (2)
  doing live programming at the REPL.  Why would you use breakpoints
  when you can just call the relevant procedures on some input to see
  how they behave?

  So I think you won’t convince people to pick Guile for their project
  by selling it as a C/C++/Python drop-in replacement.  Guile is about
  functional programming and live coding so the set of tools differs.

  Despite what I wrote, I think it’s a good idea.  I suppose inspiration
  would come from other Schemes, in particular Racket, and perhaps from
  other live-coding systems (Common Lisp, Smalltalk, etc.), rather than
  from Python or C.

Zimon:

  Maybe I am wrong or miss some Guile features.  From my experience, the
  issue is not the way that the information is presented or how we
  interact with it (Geiser or else) but, instead, the issue is the
  availability of such information.

  Well, so you are using the good ol’ way putting ’pk’ here or there,
  right?  One thing when debugging is to inspect the current state of
  the program; [...] And, ’pk’ is the poor man breakpoint. :-)

  Racket is an example of functional programming and live coding.
  Haskell is another; it is functional programming and if I might, I
  would recommend to give a look at the interactive GHCi debugger

Maxim:

  When searching for how the debugger work in the Guile Reference info
  manual, I also don't find anything useful: only the gut of the
  debugging API of the Guile VM appears to be documented ("Debugging
  Infrastructure"), so documentation is another place that could be
  improved, with some examples and pro tips for real life, practical
  debugging with Guile.

Ludo:

  I think we should identify scenarios where things don’t work as
  expected, and then turn them into bug reports, documentation issues,
  or any other concrete action we should take.

  [...] that brings us back to Maxim’s suggestion of starting a
  debugger workgroup.

Attila:

  [C]oming from common lisp [...], [I] think the lowest hanging fruit in
  the guile debugging experience is making sure that backtraces are not
  cut short when printed.

  [T]his is regularly causing me frustration when all i need to make
  progress is in the cut off part of the backtrace, and the code in
  question is in a part of the codebase that i can't easily change to
  add some good old printf's.

Joshua:

  Just my 2 cents, I always thought that the elisp debugging experince
  is super user friendly and awesome!

  M-x edebug-defun RET function-name RET

  And you are golden!

  It would be awesome if guile could offer something as seemless.

Csepp:

  Can we also get a profiler like Python's Scalene?  I'm pretty sure
  there are some performance bottlenecks it could help identify, both in
  Guix and in Guile itself.

-- 
Olivier Dion
oldiob.dev

Re: A Guile debugger workgroup?

[Neil Jerram]

I wrote something called GDS, back in 1.8 days, that combined some of
the Emacs interaction features that we now have with Geiser, with
step-by-step debugging and breakpoints.  You can read about that here:
https://www.gnu.org/software/guile/docs/docs-1.8/guile-ref/Using-Guile-in-Emacs.html#Using-Guile-in-Emacs

Unfortunately the debugging side of GDS was based on the low level
traps in the 1.8 interpreter, and that was all blown away in the 2.0
rewrite, so GDS stopped working at that point.  I don't know what
modern equivalent 3.x has to the 1.8 low level traps - perhaps we
would instead advocate for recompiling the relevant code with extra
instrumentation, like edebug does?

On Tue, 29 Nov 2022 at 17:21, Olivier Dion via Developers list for
Guile, the GNU extensibility library <guile-devel@gnu.org> wrote:
>
> Greetings Guilers,
>
> There's been a discussion on the state of debugging with Guile on the
> guix-devel mailing list.  Here's the relevant link if you want the full
> discussion:
> <https://lists.gnu.org/archive/html/guix-devel/2022-11/msg00283.html>
> which is a reply to <https://issues.guix.gnu.org/issue/58812#id=33>.
>
> I'm going to resume what I've gather here.  I've mainly cutoff Guix
> related stuff.  I'm also putting names here for the sake of clarity.
>
> I think it would be interesting to have more inputs from the Guile users
> at large -- outside of Guix -- on that topic.  From my personal
> experience and the echo I get back from other users, the debugging
> experience in Guile is frustrating.
>
> Things I've gather in this reading and on the IRC with other users.
> This is by no way a wish list, but simply ideas to improve the debugging
> experience:
>
>   1. Documentation on debugging should be improved.  e.g. The `pk`
>      procedure.
>
>   2. A tutorial on how to debug a project with `pk` and the REPL.
>
>   3. A single-step (instruction and line) debugger.
>
>   4. Integration in GDB.  e.g. GDB could insert breakpoints in Scheme
>      code.
>
>   5. Maybe not make a debugger with a paradigm for language like C but
>      instead take inspiration from other Scheme implementation like
>      Racket.
>
> Resume start here:
>
> zimon:
>
>   Preparing some materials for introducing Guile to GuixHPC folk, I am
>   trying to collect some tips and, if I am honest, the debugging
>   experience with Guile is really poor; compared to others (as Python).
>   For example, DrRacket provides an easy and nice user experience
>
>   Well, IMHO, we are somehow suffering from some Guile limitations and
>   improvements in this area are an hard task.
>
> Maxim:
>
>   I also agree!  It's hard to convince people to pick Guile for their
>   project when there is:
>
>   1. Lack of a debugger that can break and step anywhere in your source
>      code
>
>   2. Lack of debugger integration to an IDE (it's not even integrated
>      into Emacs)
>
>   Perhaps we should assemble a Guile debugger workgroup that'd review
>   what's broken, what's missing, what can be borrowed from other Scheme
>   or languages for inspiration
>
> Ludo:
>
>   Well, Guile has a debugger that lets you do that [...] Geiser is not
>   Visual Studio™ but it does a good job.
>
>   Also, [...] I mentioned before that I almost never use breakpoints on
>   Guile code [...]  because it’s rarely the right tool for the job.
>
>   I believe this is largely due to (1) writing functional code, and (2)
>   doing live programming at the REPL.  Why would you use breakpoints
>   when you can just call the relevant procedures on some input to see
>   how they behave?
>
>   So I think you won’t convince people to pick Guile for their project
>   by selling it as a C/C++/Python drop-in replacement.  Guile is about
>   functional programming and live coding so the set of tools differs.
>
>   Despite what I wrote, I think it’s a good idea.  I suppose inspiration
>   would come from other Schemes, in particular Racket, and perhaps from
>   other live-coding systems (Common Lisp, Smalltalk, etc.), rather than
>   from Python or C.
>
> Zimon:
>
>   Maybe I am wrong or miss some Guile features.  From my experience, the
>   issue is not the way that the information is presented or how we
>   interact with it (Geiser or else) but, instead, the issue is the
>   availability of such information.
>
>   Well, so you are using the good ol’ way putting ’pk’ here or there,
>   right?  One thing when debugging is to inspect the current state of
>   the program; [...] And, ’pk’ is the poor man breakpoint. :-)
>
>   Racket is an example of functional programming and live coding.
>   Haskell is another; it is functional programming and if I might, I
>   would recommend to give a look at the interactive GHCi debugger
>
> Maxim:
>
>   When searching for how the debugger work in the Guile Reference info
>   manual, I also don't find anything useful: only the gut of the
>   debugging API of the Guile VM appears to be documented ("Debugging
>   Infrastructure"), so documentation is another place that could be
>   improved, with some examples and pro tips for real life, practical
>   debugging with Guile.
>
> Ludo:
>
>   I think we should identify scenarios where things don’t work as
>   expected, and then turn them into bug reports, documentation issues,
>   or any other concrete action we should take.
>
>   [...] that brings us back to Maxim’s suggestion of starting a
>   debugger workgroup.
>
> Attila:
>
>   [C]oming from common lisp [...], [I] think the lowest hanging fruit in
>   the guile debugging experience is making sure that backtraces are not
>   cut short when printed.
>
>   [T]his is regularly causing me frustration when all i need to make
>   progress is in the cut off part of the backtrace, and the code in
>   question is in a part of the codebase that i can't easily change to
>   add some good old printf's.
>
> Joshua:
>
>   Just my 2 cents, I always thought that the elisp debugging experince
>   is super user friendly and awesome!
>
>   M-x edebug-defun RET function-name RET
>
>   And you are golden!
>
>   It would be awesome if guile could offer something as seemless.
>
> Csepp:
>
>   Can we also get a profiler like Python's Scalene?  I'm pretty sure
>   there are some performance bottlenecks it could help identify, both in
>   Guix and in Guile itself.
>
> --
> Olivier Dion
> oldiob.dev
>
>

Re: A Guile debugger workgroup?

[Maxim Cournoyer]

Hi Neil,

Neil Jerram <neiljerram@gmail.com> writes:

> I wrote something called GDS, back in 1.8 days, that combined some of
> the Emacs interaction features that we now have with Geiser, with
> step-by-step debugging and breakpoints.  You can read about that here:
> https://www.gnu.org/software/guile/docs/docs-1.8/guile-ref/Using-Guile-in-Emacs.html#Using-Guile-in-Emacs

Thanks for sharing; unfortunately the page has vanished, it seems, and I
couldn't find an alternative link.  Is the GDS source still available
from somewhere?  Perhaps it could offer hints about how to integrate
Guile with Emacs.

I'd think the lowest friction, most maintainable way though would be to
ensure that the existing Guile debugger covers all the use cases we want
it to, and then bind it to Emacs using GUD, RealGUD or a similar
existing library.  That way it should be also be possible to use it with
other IDEs, which'd help with democratizing Guile, in my opinion.

> Unfortunately the debugging side of GDS was based on the low level
> traps in the 1.8 interpreter, and that was all blown away in the 2.0
> rewrite, so GDS stopped working at that point.  I don't know what
> modern equivalent 3.x has to the 1.8 low level traps - perhaps we
> would instead advocate for recompiling the relevant code with extra
> instrumentation, like edebug does?

Edebug offers a great debugging experience, so if we can mimic any of
what it does, it'd probably be a good thing!

-- 
Thanks,
Maxim

Re: A Guile debugger workgroup?

[Damien Mattei]

[-- Attachment #1: Type: text/plain, Size: 6266 bytes --]

Hello,

i almost never use a debugger but in C as a student.
Never after.
programming in a functional style and if in need print some debug
information, all my program worked and the only hard problem were not due
to implementation but algorithm,and for that the debugger is useless...

http://i-exc.ccm2.net/iex/1280/1809789201/893758.jpg

but this is my personal opinion :-)

Regards,
Damien


On Tue, Nov 29, 2022 at 6:21 PM Olivier Dion via Developers list for Guile,
the GNU extensibility library <guile-devel@gnu.org> wrote:

> Greetings Guilers,
>
> There's been a discussion on the state of debugging with Guile on the
> guix-devel mailing list.  Here's the relevant link if you want the full
> discussion:
> <https://lists.gnu.org/archive/html/guix-devel/2022-11/msg00283.html>
> which is a reply to <https://issues.guix.gnu.org/issue/58812#id=33>.
>
> I'm going to resume what I've gather here.  I've mainly cutoff Guix
> related stuff.  I'm also putting names here for the sake of clarity.
>
> I think it would be interesting to have more inputs from the Guile users
> at large -- outside of Guix -- on that topic.  From my personal
> experience and the echo I get back from other users, the debugging
> experience in Guile is frustrating.
>
> Things I've gather in this reading and on the IRC with other users.
> This is by no way a wish list, but simply ideas to improve the debugging
> experience:
>
>   1. Documentation on debugging should be improved.  e.g. The `pk`
>      procedure.
>
>   2. A tutorial on how to debug a project with `pk` and the REPL.
>
>   3. A single-step (instruction and line) debugger.
>
>   4. Integration in GDB.  e.g. GDB could insert breakpoints in Scheme
>      code.
>
>   5. Maybe not make a debugger with a paradigm for language like C but
>      instead take inspiration from other Scheme implementation like
>      Racket.
>
> Resume start here:
>
> zimon:
>
>   Preparing some materials for introducing Guile to GuixHPC folk, I am
>   trying to collect some tips and, if I am honest, the debugging
>   experience with Guile is really poor; compared to others (as Python).
>   For example, DrRacket provides an easy and nice user experience
>
>   Well, IMHO, we are somehow suffering from some Guile limitations and
>   improvements in this area are an hard task.
>
> Maxim:
>
>   I also agree!  It's hard to convince people to pick Guile for their
>   project when there is:
>
>   1. Lack of a debugger that can break and step anywhere in your source
>      code
>
>   2. Lack of debugger integration to an IDE (it's not even integrated
>      into Emacs)
>
>   Perhaps we should assemble a Guile debugger workgroup that'd review
>   what's broken, what's missing, what can be borrowed from other Scheme
>   or languages for inspiration
>
> Ludo:
>
>   Well, Guile has a debugger that lets you do that [...] Geiser is not
>   Visual Studio™ but it does a good job.
>
>   Also, [...] I mentioned before that I almost never use breakpoints on
>   Guile code [...]  because it’s rarely the right tool for the job.
>
>   I believe this is largely due to (1) writing functional code, and (2)
>   doing live programming at the REPL.  Why would you use breakpoints
>   when you can just call the relevant procedures on some input to see
>   how they behave?
>
>   So I think you won’t convince people to pick Guile for their project
>   by selling it as a C/C++/Python drop-in replacement.  Guile is about
>   functional programming and live coding so the set of tools differs.
>
>   Despite what I wrote, I think it’s a good idea.  I suppose inspiration
>   would come from other Schemes, in particular Racket, and perhaps from
>   other live-coding systems (Common Lisp, Smalltalk, etc.), rather than
>   from Python or C.
>
> Zimon:
>
>   Maybe I am wrong or miss some Guile features.  From my experience, the
>   issue is not the way that the information is presented or how we
>   interact with it (Geiser or else) but, instead, the issue is the
>   availability of such information.
>
>   Well, so you are using the good ol’ way putting ’pk’ here or there,
>   right?  One thing when debugging is to inspect the current state of
>   the program; [...] And, ’pk’ is the poor man breakpoint. :-)
>
>   Racket is an example of functional programming and live coding.
>   Haskell is another; it is functional programming and if I might, I
>   would recommend to give a look at the interactive GHCi debugger
>
> Maxim:
>
>   When searching for how the debugger work in the Guile Reference info
>   manual, I also don't find anything useful: only the gut of the
>   debugging API of the Guile VM appears to be documented ("Debugging
>   Infrastructure"), so documentation is another place that could be
>   improved, with some examples and pro tips for real life, practical
>   debugging with Guile.
>
> Ludo:
>
>   I think we should identify scenarios where things don’t work as
>   expected, and then turn them into bug reports, documentation issues,
>   or any other concrete action we should take.
>
>   [...] that brings us back to Maxim’s suggestion of starting a
>   debugger workgroup.
>
> Attila:
>
>   [C]oming from common lisp [...], [I] think the lowest hanging fruit in
>   the guile debugging experience is making sure that backtraces are not
>   cut short when printed.
>
>   [T]his is regularly causing me frustration when all i need to make
>   progress is in the cut off part of the backtrace, and the code in
>   question is in a part of the codebase that i can't easily change to
>   add some good old printf's.
>
> Joshua:
>
>   Just my 2 cents, I always thought that the elisp debugging experince
>   is super user friendly and awesome!
>
>   M-x edebug-defun RET function-name RET
>
>   And you are golden!
>
>   It would be awesome if guile could offer something as seemless.
>
> Csepp:
>
>   Can we also get a profiler like Python's Scalene?  I'm pretty sure
>   there are some performance bottlenecks it could help identify, both in
>   Guix and in Guile itself.
>
> --
> Olivier Dion
> oldiob.dev
>
>
>

[-- Attachment #2: Type: text/html, Size: 8181 bytes --]

Re: A Guile debugger workgroup?

[Jannneke Nieuwenhuizen]

Olivier Dion via Developers list for Guile, the GNU extensibility library writes:

Hi,

> There's been a discussion on the state of debugging with Guile on the
> guix-devel mailing list.  Here's the relevant link if you want the full
> discussion:
> <https://lists.gnu.org/archive/html/guix-devel/2022-11/msg00283.html>
> which is a reply to <https://issues.guix.gnu.org/issue/58812#id=33>.

In 2014, I wrote patches for Guile to use a debug prompt that GUD can
work with.  The Guile debug patches didn't make it into Guile, but the
Emacs GUD integration is part of Emacs.

   https://lists.gnu.org/archive/html/guile-devel/2014-08/msg00004.html

   https://lists.gnu.org/archive/html/guix-devel/2022-11/msg00343.html

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Re: A Guile debugger workgroup?

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 595 bytes --]


Jannneke Nieuwenhuizen <janneke@gnu.org> writes:
> In 2014, I wrote patches for Guile to use a debug prompt that GUD can
> work with.  The Guile debug patches didn't make it into Guile, but the
> Emacs GUD integration is part of Emacs.
>
>    https://lists.gnu.org/archive/html/guile-devel/2014-08/msg00004.html

What were the reasons these didn’t make it?

I now have push-access, so if these get acked by someone who knows that
part well enough, I can take up the integration.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: A Guile debugger workgroup?

[Janneke Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 952 bytes --]

Dr. Arne Babenhauserheide writes:

Hi Arne,

> Jannneke Nieuwenhuizen <janneke@gnu.org> writes:
>> In 2014, I wrote patches for Guile to use a debug prompt that GUD can
>> work with.  The Guile debug patches didn't make it into Guile, but the
>> Emacs GUD integration is part of Emacs.
>>
>>    https://lists.gnu.org/archive/html/guile-devel/2014-08/msg00004.html
>
> What were the reasons these didn’t make it?

No idea, there was no comment or interest in the patch.  I believe the
consensus was (is?) to use pk for debugging.  FWIW, I'm using my own
"pke", because many of my programs are (pseudo-)filters, writing their
output to stdout, which makes "pk" not so handy.  See attached.
> I now have push-access, so if these get acked by someone who knows that
> part well enough, I can take up the integration.

That's nice, but it probably needs to be updated and it certainly needs
some discussion.

Greetings,
Janneke


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-peek-error-pke.patch --]
[-- Type: text/x-patch, Size: 1142 bytes --]

From 98264afe030eef3b5bbabc03b76808253ce10d14 Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
Date: Fri, 3 Mar 2023 11:26:34 +0100
Subject: [PATCH] Add peek-error, pke.

Debuging using `pk' is popular in Guile, but not really usable if your
program is as (pseudo-)filter, i.e., writing its output to stdout.

* module/ice-9/boot-9.scm (peek-error, pke): New procedures.
---
 module/ice-9/boot-9.scm | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/module/ice-9/boot-9.scm b/module/ice-9/boot-9.scm
index dc3537063..639aaaa79 100644
--- a/module/ice-9/boot-9.scm
+++ b/module/ice-9/boot-9.scm
@@ -183,6 +183,16 @@ This is handy for tracing function calls, e.g.:
 
 (define pk peek)
 
+(define (peek-error . stuff)
+  "Like peek (peok), writing to (CURRENT-ERROR-PORT)."
+  (newline (current-error-port))
+  (display ";;; " (current-error-port))
+  (write stuff (current-error-port))
+  (newline (current-error-port))
+  (car (last-pair stuff)))
+
+(define pke peek-error)
+
 (define (warn . stuff)
   (newline (current-warning-port))
   (display ";;; WARNING " (current-warning-port))
-- 
2.39.1


[-- Attachment #3: Type: text/plain, Size: 164 bytes --]


-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Re: A Guile debugger workgroup?

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 60 bytes --]

Janneke Nieuwenhuizen writes:

Fixing typos in peek-error.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-peek-error-pke.patch --]
[-- Type: text/x-patch, Size: 1140 bytes --]

From bf0f03090d73ea87f1be772010d6209b4372e78b Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
Date: Fri, 3 Mar 2023 11:26:34 +0100
Subject: [PATCH] Add peek-error, pke.

Debuging using `pk' is popular in Guile, but not really usable if your
program is as (pseudo-)filter, i.e., writing its output to stdout.

* module/ice-9/boot-9.scm (peek-error, pke): New procedures.
---
 module/ice-9/boot-9.scm | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/module/ice-9/boot-9.scm b/module/ice-9/boot-9.scm
index dc3537063..4868f6f3c 100644
--- a/module/ice-9/boot-9.scm
+++ b/module/ice-9/boot-9.scm
@@ -183,6 +183,16 @@ This is handy for tracing function calls, e.g.:
 
 (define pk peek)
 
+(define (peek-error . stuff)
+  "Like PEEK (PK), writing to (CURRENT-ERROR-PORT)."
+  (newline (current-error-port))
+  (display ";;; " (current-error-port))
+  (write stuff (current-error-port))
+  (newline (current-error-port))
+  (car (last-pair stuff)))
+
+(define pke peek-error)
+
 (define (warn . stuff)
   (newline (current-warning-port))
   (display ";;; WARNING " (current-warning-port))
-- 
2.39.1


[-- Attachment #3: Type: text/plain, Size: 164 bytes --]


-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com