exception from inside false-if-exception?

exception from inside false-if-exception?

[Attila Lendvai]

dear fellow Guilers,

context:
--------

i'm working on shepherd (with several non-trivial local commits). its test suite runs clean from a shell, but fails when i try to `./pre-inst-env guix build -K shepherd@0.10.99-git`.


the sympthom:
-------------

COLUMNS is not set in the guix build env, and the basic.sh test fails with the following exception/backtrace in the test log:

[...]
In ice-9/boot-9.scm:
  1747:15  8 (with-exception-handler [...])
In shepherd/support.scm:
    613:9  7 (_ . _)
In unknown file:
           6 (display-backtrace [...])
In system/repl/debug.scm:
   148:36  5 (print-frames [...])
In ice-9/boot-9.scm:
   2137:6  4 (_)
  1747:15  3 (with-exception-handler [...])
In system/repl/debug.scm:
    72:40  2 (_)
In ice-9/boot-9.scm:
  1685:16  1 (raise-exception _ #:continuable? _)
  1683:16  0 (raise-exception _ #:continuable? _)
ice-9/boot-9.scm:1683:16: In procedure raise-exception:
In procedure string->number: Wrong type argument in position 1 (expecting string): #f

the expression pointed to by debug.scm,72:40 is this:

(false-if-exception (string->number (getenv "COLUMNS")))

if i paste this into a guile repl, then it behaves as expected.

i have verified that if i set COLUMNS in the basic.sh test, even if i set it to:

COLUMNS=""
export COLUMNS

then the guix package builds fine.

if i add:

unset COLUMNS

to the basic.sh test, then it makes it fail even in my dev shell (after a couple of minutes long timeout):

`make check TESTS="tests/basic.sh"`

there are no WITH-THROW-HANDLER's involved.


my question:
------------

unless i missed something, i seem to be getting an exception *from inside* a false-if-exception? how can that happen?

do i miss something, or is this a guile bug?

if this seems to be a guile bug, then i'll try to set up a simpler reproducer than my current one. in that case, what may be the key difference between the repl and the shepherd test suite? simply compiled vs. evaluated code?


a hypothesis:
-------------

i tried to look around guile's codebase, and STRING->NUMBER seems to be an optimized or otherwise specially treated primitive.

maybe that special treatment interferes with exceptions? maybe it throws a kind of exception that false-if-exception doesn't catch? or in a way that doesn't get caught?

this call of STRING->NUMBER is within the bootstrapped part of guile itself, which again may add an extra layer complexity or special teatment?

note that if i unset the env var then STRING->NUMBER is called with #f instead of an empty string. maybe there's a bug in how STRING->NUMBER handles being called with a non-string?

i didn't notice anything obviously wrong around SCM_VALIDATE_STRING.

any hint is appreciated,

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Watch your thoughts; they become words. Watch your words; they become actions. Watch your actions; they become habit. Watch your habits; they become character. Watch your character; it becomes your destiny.”
	— Lao Tzu (sixth century BC)

bug#46009: exception from inside false-if-exception?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 5793 bytes --]

[Adding Andy Wingo because of the stack shenanigans]

>Subject: exception from inside false-if-exception?

Duplicate of #46009 - (backtrace) crash, string->number: Wrong type argument in position 1 (expecting string): #f - GNU bug report logs

>the expression pointed to by debug.scm,72:40 is this:

>(false-if-exception (string->number (getenv "COLUMNS")))

All uses of false-if-exception are wrong – I haven’t found a single exception (pun intended) to this rule so far.  It shouldn’t be treating stack overflows, out-of-memory, exceptions from asyncs, EPERM, etc., as #false.  – false-if-exception delenda est

[...]

What I think is going on here, is that the exception catching API is a bit of mess, and to a degree, non-composable, leading to bugs like this.

1. raise/raise-continuable + guard: perfectly composable, no problems.
2. throw + catch: that’s fine too, it’s just a historical Guile API for (1), albeit slightly less general. 
3. with-exception-handler – a procedural variant of the ‘guard’ macro – sure ok. Also good for continuable exceptions, IIUC.
4. Pre-unwind handlers / with-throw-handler. / ???.

This is the non-composable stuff (or, difficult to understand and compose, at least). If you are catching an exception to _handle_ it (say, in this case, with false-if-exception), then any throw handler/pre-unwind handler/whatever has no business whatsoever to interfere, and neither to even know something is happening in the first place.

Yet, the documentation says it “is used to be able to intercept an exception that is being thrown before the stack is unwound” – which it has no business of doing in the first place (non-composable). 

Also the description of pre-unwind handlers / with-throw-handler is rather low-level (it’s more described in terms of (un)winding rather than nesting) and it appears to have been grown a bit .. organically, I think it’s time for it to be replaced by a new design that’s (conceptually) closer to raise/guard/...

I suspect the problem is that these throw handlers or whatever are messing things up – whether because they are hard to use, impossible to use correctly or because they are incorrectly implemented in Guile and would propose them to be replaced by something else.

On this something else: according to the documentation, these throw handlers can be used for:

1. Clean up some related state
2. Print a backtrace
3. Pass information about the exception to a debugger

1: IIUC, this is what (dynamic-wind #false thunk clean-up-stuff) is for – this is not entirely correct w.r.t. userspace threading implementation, but can be salvaged with something like

>https://github.com/wingo/fibers/blob/7e29729db7c023c346bc88c859405c978131f27a/fibers/scheduler.scm#L278

to override ‘dynamic-wind’ (actually I think the API ‘rewinding-for-scheduling?’ would need to be adjusted a bit to allow for situations like implementing threads inside threads inside threads inside ..., but it should give the basic idea.)

2. the rationale for this reason, is that when an exception is caught (think ‘catch’ handler), we are not in the same dynamic environment anymore, so by then it’s too late to generate a backtrace. But we can have a cake(= have a backtrace) and eat it too(= post-unwind / in the catch handler), by using ‘with-exception-handler’ with ‘#:unwind? #false’.

That’s not entirely sufficient, because sometimes the exception was already caught and ‘wapped’ to give some extra context in the exception object (but losing some backtrace in the process). OTOH, this ‘losing some backtrace’ already happens in the current implementation IIUC, so it wouldn’t be worse than the current implementation in this respect. I think there is a solution that isn’t just “always record the backtrace and put it in the exception” (which would be terribly inefficient in situation you mentioned), but I haven’t found it yet.

Perhaps the (full) current continuation (think call/cc) could be saved? And then later, when ultimately a backtrace is to be printed, the stack of this continuation can be investigated – This might seem even more inefficient than saving a backtrace, but if you think about it, all those frames were already allocated, so you don’t need to allocate new memory or do much CPU things beyond saving some pointers, you just need to start some new memory branching of an earlier point in the dynamic environment/the frames when/after rewinding.  I’m thinking of spaghetti and cactus stacks.   (The benefit beyond ‘just include backtrace’ is: (1) almost 0 time/memory cost when not used (2) if desired, you can print _more_ information than just the backtrace, e.g. values of certain parameters or whatever.)

And after the backtrace or whatever has been printed, there is no reference to the call/cc thing anymore so it can be garbage collected(*).

(*) there is a bit of a caveat here with respect to user-level threading and pausing computations, where some stuff low(high?) on the stack might be saved for too long, so perhaps it would instead be better to save a delimited continuation – the exception handler that does the backtracing could then set some parameter with a tag that delimits where to stop the delimited continuation – for a comparison: I think there is a procedure with a name like “call-with-stack” or something close to that which does something like that.

(Also, the section “Exceptions” isn’t mentioning ‘raise + guard’, instead there is a separate R6RS section isolated from the rest of the manual – it treats historical Guile idiosyncrasies as the norm and standard SRFI / RnRS as an aberration. But that’s a different thing.)


[-- Attachment #2: Type: text/html, Size: 18473 bytes --]

Re: exception from inside false-if-exception?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1369 bytes --]

Attila Lendvai <attila@lendvai.name> writes:

> dear fellow Guilers,
>
> context:
> --------
>
> i'm working on shepherd (with several non-trivial local commits). its
> test suite runs clean from a shell, but fails when i try to
> `./pre-inst-env guix build -K shepherd@0.10.99-git`.
>
>
> the sympthom:
> -------------
>
> COLUMNS is not set in the guix build env, and the basic.sh test fails with the following exception/backtrace in the test log:
>
> [...]
> In ice-9/boot-9.scm:
>   1747:15  8 (with-exception-handler [...])
> In shepherd/support.scm:
>     613:9  7 (_ . _)
> In unknown file:
>            6 (display-backtrace [...])
> In system/repl/debug.scm:
>    148:36  5 (print-frames [...])
> In ice-9/boot-9.scm:
>    2137:6  4 (_)
>   1747:15  3 (with-exception-handler [...])
> In system/repl/debug.scm:
>     72:40  2 (_)
> In ice-9/boot-9.scm:
>   1685:16  1 (raise-exception _ #:continuable? _)
>   1683:16  0 (raise-exception _ #:continuable? _)
> ice-9/boot-9.scm:1683:16: In procedure raise-exception:
> In procedure string->number: Wrong type argument in position 1 (expecting string): #f
>
> the expression pointed to by debug.scm,72:40 is this:
>
> (false-if-exception (string->number (getenv "COLUMNS")))

I think I've had similar problems in the past, I did fine this IRC
conversation:

  https://logs.guix.gnu.org/guile/2021-01-19.log#204926

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: exception from inside false-if-exception?

[Attila Lendvai]

> I think I've had similar problems in the past, I did fine this IRC
> conversation:
> 
> https://logs.guix.gnu.org/guile/2021-01-19.log#204926

the very same bug was already discussed in 2021?!

from the log:

> you're using a pre-unwind handler here
> whereas false-if-exception only sets a post-unwind handler
> so yours gets to run first

oh my, what a mess!

do i get this right? (with-exception-handler ... #:unwind? #f) installs a so called pre-unwind-handler, which takes priority over a false-if-exception, even if deeper in the stack, because f-i-e installs a post-unwind-handler?

if this is the case, then i'll basically need to review all the scheme code i wrote in the past two years... :)

i came from common lisp, and from a distance i always had this impression of scheme that it is a much more cleaned up lisp... but in CL the condition system is way cleaner/simpler than this. i wrote long running, multi-threaded web services with advanced logging (backtraces, user installable backtrace decorators, etc), error handlers that deal with nested errors, etc... so i'm rather familiar with the problem domain, but apparently i'm lost in the forest here.

maybe the native call/cc primitive brings in a kind of complexity that is not present in CL around exceptions, unwinding, and backtraces.

> All uses of false-if-exception are wrong

i second this sentiment! our CL team had a syntax for IGNORE-ERRORS, its CL equivalent, to be colored bright red in emacs. and it doesn't even have the pre/post-unwind-hook baggage.

thank you both for the quick feedback!

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“A society that puts equality — in the sense of equality of outcome — ahead of freedom will end up with neither equality nor freedom. The use of force to achieve equality will destroy freedom, and the force, introduced for good purposes, will end up in the hands of people who use it to promote their own interests.”
	— Milton Friedman and Rose Friedman

RE: exception from inside false-if-exception?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 3584 bytes --]

>do i get this right? (with-exception-handler ... #:unwind? #f) installs a so called pre-unwind-handler, which takes priority over a false-if-exception, even if deeper in the stack, because f-i-e installs a post-unwind-handler?

It’s not post-unwind? If anything, I would call it pre-unwind. If with-exception-handler were post all the unwinding, then it’s too late to let raise-continuable return anything. 

But other than that (“[it] takes priority over a false-if-exception, [...]), that’s my understanding of things.

I guess what should be done, is it _not_ having priority, and instead it only handling things when there noguard/catch/... inside has caught the exception.

>i came from common lisp, and from a distance i always had this impression of scheme that it is a much more cleaned up lisp... but in CL the condition system is way cleaner/simpler than this. i wrote long running, multi-threaded web services with advanced logging (backtraces, user installable backtrace decorators, etc), error handlers that deal with nested errors, etc... so i'm rather familiar with the problem domain, but apparently i'm lost in the forest here.

First, I want to say that (purely semantically) conditions have nothing to do with exceptions in Guile – you can, technically, raise anything as an exception, not only conditions, and conditions are just a kind of record type that are conventionally used for exceptions (and convenient for that!) but could in principle be used for entirely different things.  So from now on I’ll talk about Guile _exceptions_ instead.

Going by < 9.1 Condition System Concepts | Common Lisp (New) Language Reference (lisp-docs.github.io)>, the API and semantics of CL is pretty much the same as guard/with-exception-handler/raise(-continuable) stuff(*).

(*) Actually, ‘guard’ as documented _always_ handles the exception, it just re-raises them. It’s not mentioned in Guile, but according to RnRS, it’s re-raised in the raise-continuable sense. So this way, it (in theory) has the same semantics as CL stuff.

Looking a bit further, there is also this ‘restart’ stuff in CL. But it seems Scheme already has this in the form or ‘guard’ + ‘raise-continuable’ (if the exception returns a value X, then the (raise-continuable [...]) evaluates to X). 

I don’t think it’s completely equivalent, but it seems to provide similar functionality, and with some effort you could implement something closer to CL with delimited continuations (+ maybe parameters, or their slightly more general API, fluids).

There is ‘warn’ which at first seems not to have an equivalent in Guile, but all that would need to be done is a top-level exception handler

[define a &ignorable-by-default condition]
(define (warn) (raise-continuable [&message stuff + &warning + &ignorable-by-default]))
(guard (c ((ignorable-by-default? c) [print stuff about c] (values)))
  [run main.scm or whatever]).

(I haven’t found search results on what backtrace decorators are in CL.)

>maybe the native call/cc primitive brings in a kind of complexity that is not present in CL around exceptions, unwinding, and backtraces.

There are some implications w.r.t. not misusing parameters for the exception handler stack (IIRC, Guile uses the more general fluid stuff instead because of this), but otherwise, I don’t think so. I think the problem is just a bug (and some of the with-throw-handler stuff that’s making thinking about what’s correct/implementing stuff correctly harder).

Best regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 6904 bytes --]

RE: exception from inside false-if-exception?

[Attila Lendvai]

[-- Attachment #1: Type: text/plain, Size: 4152 bytes --]

i was once again hindered by this, so i digged deeper.

please find a test.scm attached.

looks like (R6RS) WITH-EXCEPTION-HANDLER has such a surprising semantics that i didn't even consider that to be the case. some hours later i found this:

https://practical-scheme.net/gauche/man/gauche-refe/Exceptions.html#index-with_002dexception_002dhandler

"Note: SRFI-18 specifies the exception handler installed with with-exception-handler will be called with exactly the same dynamic environment, including the exception handler settings. It means if an exception is raised within the handler, it will be caught with the same handler."

"However, the bare SRFI-18 semantics turned out to be error prone–users tended to assume errors from a handler would be handled by outer handler, and were perplexed when they ran into infinite recursion of handlers"

now, as surprising as the above was to me, guile behaves in a yet more surprising way, because seeing a stack overflow would have hinted the above semantics to me.

instead, what seems to be happening (?) is that there's some short-circuit somewhere in the guile codebase that aborts this infinite recursion by printing a backtrace and the exception... and then continues who knows where, and with what return value? unfortunately, it doesn't print anything else to explain the situation, or to grep for in the codebase.

context:
--------

in the original context of shepherd, it doesn't seem to just entirely exit the process, because shepherd continues to run, and IIRC it even functions to some extent. and it doesn't seem to just exit the current thread either, because IIUC shepherd is a single-threaded app (using fibers).

for reference, i'm including the output at the end of this mail that i'm seeing in my shepherd branch. note that it says "Uncaught exception in task:" which suggests that this exception seems to have left a fiber and reached fiber's RUN-SCHEDULER/ERROR-HANDLING.

and finally, my ultimate use-case is a CALL-WITH-ERROR-HANDLING function that does not unwind, and has three layers of nested error handlers to log any errors with a backtrace, including errors in the user supplied error handlers, or in the logging subsystem.


i see two actionables:
----------------------

if i'm not wrong in the above, then:

please add some message -- any message -- to this short-circuit code.

please extend the doc of WITH-EXCEPTION-HANDLER with a warning that explains the dynamic extent of the binding of the handler at the time the handler is called, and any possible short circuits installed in guile against that.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“The strength of a person's spirit would then be measured by how much 'truth' he could tolerate, or more precisely, to what extent he needs to have it diluted, disguised, sweetened, muted, falsified.”
	— Friedrich Nietzsche (1844–1900), 'Beyond Good and Evil' (1886)



Uncaught exception in task:
In fibers.scm:
    172:8 13 (_)
In shepherd/service.scm:
    425:9 12 (_)
In ice-9/boot-9.scm:
  1752:10 11 (with-exception-handler _ _ #:unwind? _ # _)
In shepherd/service.scm:
   570:31 10 (service-controller #<service bee-1> #<<channel> getq: ?>)
In /gnu/store/0zd8bv4hmrfqhzb5qv6flslah0a7bplb-shepherd-bee-1.scm:
   1:3579  9 (_ _ . _)
In ice-9/boot-9.scm:
  1685:16  8 (raise-exception _ #:continuable? _)
  1752:10  7 (with-exception-handler _ _ #:unwind? _ # _)
In shepherd/support.scm:
    644:4  6 (log-with-backtrace _ #<&compound-exception componen?> . #)
In unknown file:
           5 (display-backtrace #<stack 7fee5677e9e0> #<output: str?> ?)
In system/repl/debug.scm:
   148:36  4 (print-frames #(#<frame 7fee564e3220 make-stack> #<?> ?) ?)
In ice-9/boot-9.scm:
   2137:6  3 (_)
  1747:15  2 (with-exception-handler #<procedure 7fee566a4780 at ic?> ?)
In system/repl/debug.scm:
    72:40  1 (_)
In ice-9/boot-9.scm:
  1685:16  0 (raise-exception _ #:continuable? _)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
In procedure string->number: Wrong type argument in position 1 (expecting string): #f

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test.scm --]
[-- Type: text/x-scheme; name=test.scm, Size: 1297 bytes --]

#!/usr/bin/env -S guile --no-auto-compile -e main -s
!#

(use-modules (srfi srfi-34))

(define* (test #:key (unwind? #f))
  (with-exception-handler
      (let ((nested #f))
        (lambda (c-level-1)
          (if nested
              (begin
                (format #t "level 1 handler got called recursively~%")
                'level-1-handler-nested)
              (begin
                (set! nested #t)
                (with-exception-handler
                    (lambda (c-level-2)
                      (begin
                        (format #t "level 2 handler got error ~A~%" c-level-2)
                        'level-2-handler))
                  (lambda ()
                    (format #t "level 1 handler~%")
                    (error "let's signal a nested error...")
                    (format #t "level 1 handler is returning~%")
                    'level-1-handler)
                  #:unwind? unwind?)))))
    (lambda ()
      (error "let's signal an error...")
      'thunk)
    #:unwind? unwind?))

(define (main cmd)
  (unsetenv "COLUMNS")
  (format #t "~%~%*** calling with unwind~%")
  (format #t "return value is: ~A~%" (test #:unwind? #t))
  (format #t "~%~%*** calling without unwind~%")
  (format #t "return value is: ~A~%" (test #:unwind? #f))
  (test #:unwind? #f))

RE: exception from inside false-if-exception?

[Attila Lendvai]

> >do i get this right? (with-exception-handler ... #:unwind? #f) installs a so called pre-unwind-handler, which takes priority over a false-if-exception, even if deeper in the stack, because f-i-e installs a post-unwind-handler?
> 
> It’s not post-unwind? If anything, I would call it pre-unwind. If with-exception-handler were post all the unwinding, then it’s too late to let raise-continuable return anything.


i didn't properly understand the meaning of pre-unwind and post-unwind handlers when i wrote the above. i thought they are something more peculiar than what they actually are.


> But other than that (“[it] takes priority over a false-if-exception, [...]), that’s my understanding of things.
> 
> I guess what should be done, is it _not_ having priority, and instead it only handling things when there noguard/catch/... inside has caught the exception.


what i didn't understand is that FALSE-IF-EXCEPTION is implemented with a simple CATCH, so its handler should have shadowed my handler. but what actually happens is more sinister, see my other mail with the test case.


> Going by < 9.1 Condition System Concepts | Common Lisp (New) Language Reference (lisp-docs.github.io)>, the API and semantics of CL is pretty much the same as guard/with-exception-handler/raise(-continuable) stuff(*).


yep. with the addendum that CL has two entirely different macros for unwinding and non-unwinding WITH-EXCEPTION-HANDLER. and the handlers are always uninstalled before they are called. and there's no multitude of duplicate primitives for almost identical functionality due to a single revision of the CL standard.


> (I haven’t found search results on what backtrace decorators are in CL.)


this is user code, sorry. it's something we added to be able to decorate backtraces with domain-specific data.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
<hcf> information we gather and create overflows our means of managing it
<hcf> our ppl r only happenstancely synergistic

RE: exception from inside false-if-exception?

[Attila Lendvai]

[-- Attachment #1: Type: text/plain, Size: 1436 bytes --]

apparently this has been fixed between 3.0.9 and HEAD. i've wasted my time chasing a ghost...

can we please have a new release? :)

as a sidenote, i also had some broken expectations in my test.scm.

it's a tangential, but namely, when #:unwind #t then the handler in a w-e-h returns from the w-e-h block, but with #:unwind #f it tries to return to the RAISE that raised the condition. i.e. a lousy little keyword arg (usually a page down) fundamentally changes the behavior of w-e-h. yet another surprise that violated my expectations regarding APIs.

anyway, i've attached a patch that clarifies what's happening for anyone who stumbles upon this; i.e. be clearer that (?) a backtrace is printed due to reaching a continuation barrier.

this makes it grep'able, and if the user also prints a backtrace, then it makes it clear that it's something printed by guile.

if someone wants to investigate further, then i'm also attaching a new version of my test.scm that behaves in an unexpected way on 3.0.9, but not on HEAD (more specifically on guile-next in guix, which is a rather recent commit).

HTH,

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
All men dream, but not equally. Those who dream by night in the dusty recesses of their minds wake in the day to find that it was vanity. But the dreamers of the day are dangerous men, for they may act their dream with open eyes to make it possible.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-More-specific-error-message-for-exceptions-at-contin.patch --]
[-- Type: text/x-patch; name=0001-More-specific-error-message-for-exceptions-at-contin.patch, Size: 933 bytes --]

From d45817097181d4a5812405474ac7b24af3531f7a Mon Sep 17 00:00:00 2001
From: Attila Lendvai <attila@lendvai.name>
Date: Mon, 17 Jun 2024 19:46:45 +0200
Subject: [PATCH] More specific error message for exceptions at continuation
 barriers

---
 libguile/continuations.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libguile/continuations.c b/libguile/continuations.c
index b8b6e1dca..49f8266dd 100644
--- a/libguile/continuations.c
+++ b/libguile/continuations.c
@@ -405,7 +405,7 @@ print_exception_and_backtrace (SCM port, SCM tag, SCM args)
 
   if (should_print_backtrace (tag, stack))
     {
-      scm_puts ("Backtrace:\n", port);
+      scm_puts ("An exception has reached a continuation barrier:\n", port);
       scm_display_backtrace_with_highlights (stack, port,
                                              SCM_BOOL_F, SCM_BOOL_F,
                                              SCM_EOL);
-- 
2.45.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: test.scm --]
[-- Type: text/x-scheme; name=test.scm, Size: 1431 bytes --]

#!/usr/bin/env -S guile --no-auto-compile -e main -s
!#

(use-modules (ice-9 control))

(define* (test #:key (unwind? #f))
  (let/ec return
    (with-exception-handler
        (let ((nested #f))
          (lambda (c-level-1)
            (if nested
                (begin
                  (format #t "level 1 handler got called recursively~%")
                  'level-1-handler-nested)
                (begin
                  (set! nested #t)
                  (with-exception-handler
                      (lambda (c-level-2)
                        (begin
                          (format #t "level 2 handler got error ~A~%" c-level-2)
                          (format #t "level 2 handler is returning~%")
                          (return 'level-2-handler)))
                    (lambda ()
                      (format #t "level 1 handler~%")
                      (error "let's signal a nested error...")
                      (format #t "level 1 handler is returning~%")
                      (return 'level-1-handler))
                    #:unwind? unwind?)))))
      (lambda ()
        (error "let's signal an error...")
        'thunk)
      #:unwind? unwind?)))

(define (main cmd)
  (unsetenv "COLUMNS")
  (format #t "~%~%*** calling with unwind~%")
  (format #t "return value is: ~A~%" (test #:unwind? #t))
  (format #t "~%~%*** calling without unwind~%")
  (format #t "return value is: ~A~%" (test #:unwind? #f)))

RE: exception from inside false-if-exception?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2846 bytes --]

>it's a tangential, but namely, when #:unwind #t then the handler in a w-e-h returns from the w-e-h block, but with #:unwind #f it tries to return to the RAISE that raised the condition. i.e. a lousy little keyword arg (usually a page down) fundamentally changes the behavior of w-e-h. yet another surprise that violated my expectations regarding APIs.

What is lousy and expectation-violating about a keyword argument doing what it name describes?

If you have unwinded, it’s too late to return return from the raise (unless the implementation is doing delimited continuations shenanigans, which maybe you had in mind?), which explains the behaviour for #:unwind #true.

If you are _not_ unwinding, then the handler can’t be run less deep in the call stack (i.e. “from the w-e-h block”), because to get less deep in the call stack, you need to unwind.

So, there is a direct relation between unwinding/no unwinding, and returning from “the w-e-h block”/”raise(-continuable)”.

If you don’t want Guile to unwind, then maybe don’t ask it to unwind. (Previous sentence N/A if you had above-mentioned delimited continuation shenanigans in mind.)

That said, I would prefer it to be named something like [#:handler-context 'raise]/[#:handler-context 'guard] instead of #:unwind? #false/#true, since ‘unwind’ refers to the implementation instead of the semantics(*). That would reduce the need of roughly knowing how it is implemented (I have a rough idea for an alternate implementation where some unwinding always happens, and the handler is run in the dynamic environment of the ‘guard/with-exception-handler’ instead of the ‘raise’(*), and if raise-continuable is used and the handler returns something, then _re_winding happens via delimited continuations.).

(*)I think this is how it’s supposed to work (?) (including in R6RS), but Guile doesn’t do this. (Except for interactions with dynamic-wind, which might now be incorrect in the case of raise-continuable, but really you shouldn’t be using dynamic-wind in the first place.)

>anyway, i've attached a patch that clarifies what's happening for anyone who stumbles upon this; i.e. be clearer that (?) a backtrace is printed due to reaching a continuation barrier.

Wait where did this happen? You say what’s happening, but you don’t seem to be referring to false-if-exception stuff, and you didn’t mention continuation barriers earlier.

>if someone wants to investigate further, then i'm also attaching a new version of my test.scm that behaves in an unexpected way on 3.0.9, but not on HEAD (more specifically on guile-next in guix, which is a rather recent commit).

It would be helpful to include in test.scm what the expected output would be and what unexpected output is encountered.

Best regards,
Maxime Devos.


[-- Attachment #2: Type: text/html, Size: 5403 bytes --]

RE: exception from inside false-if-exception?

[Attila Lendvai]

[-- Attachment #1: Type: text/plain, Size: 3514 bytes --]

> What is lousy and expectation-violating about a keyword argument
> doing what it name describes?


well, this should have been discussed at the R6RS committee, but...

an API is facilitating efficient reading comprehension when the "verbs" are closer to the beginning of the "sentence" (the sexp), and nothing later in the sentence adjusts the meaning of the verb fundamentally. and especially not a keyword arg at the end of the sentence that is not even mandatory, and defaults to one of the rather different possibilities.

in this case, it's a primitive to catch and handle exceptions. and it's possible to implement both #:unwind? #t and #f so that when the handler returns then the control flow continues at the guard, and never at the RAISE. it's just that one version would call the handler before unwinding.


> If you have unwinded, it’s too late to return return from the raise
> (unless the implementation is doing delimited continuations
> shenanigans, which maybe you had in mind?), which explains the
> behaviour for #:unwind #true.


i didn't have anything on my mind, and that's the point. i'm simply new to scheme. now that i've learned it, i'll learn to live with it.

put another way, my learning curve would have been much steeper if the two, rather different behaviors were defined under two distinct names (or at least mentioned with bold in the documentation).


> That said, I would prefer it to be named something like [#:handler-context 'raise]/[#:handler-context 'guard]


that would be better, but i still wouldn't like the fact that it's focusing on the dynamic context of the handler, instead of the different flow of control.

for reference, in CL they are called HANDLER-CASE and HANDLER-BIND, with completely different syntaxes.



> Wait where did this happen? You say what’s happening, but you don’t
> seem to be referring to false-if-exception stuff, and you didn’t
> mention continuation barriers earlier.


this has caused me layers of confusion, which is sadly reflected in my mails.

guile prints a backtrace without any prelude, which made me think that it's my own code that is printing this backtrace and exception in the logs (remember, i'm working on nested error handling and logging in shepherd). then 3.0.9 does something funny with the control flow, which further made me believe that i'm logging an exception coming from inside FALSE-IF-EXCEPTION and it's somehow flying past my error handler, and reaching fibers.

and i think i'm still confused about a possible continuation barrier being injected somewhere that probably/maybe causes the exception from inside FALSE-IF-EXCEPTION somehow ending up at fibers instead of my error handler. but at this point i'm reaching the frontier of my understanding of scheme, delimited continuations, fibers, etc.

so, one step at a time. once this test.scm is clear, and i'm able to run shepherd on guile-next, then i'll get back to investigate my original issue in its original context.


> It would be helpful to include in test.scm what the expected output
> would be and what unexpected output is encountered.


i've attached a new test.scm that contains the output of two runs, one with 3.0.9, and one with guile HEAD (more specifically, guile-next in guix, which is almost guile HEAD).

HTH,

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Life becomes easier when you learn to accept the apology you never got.”
	— Robert Brault

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test.scm --]
[-- Type: text/x-scheme; name=test.scm, Size: 3921 bytes --]

#!/usr/bin/env -S guile --no-auto-compile -e main -s
!#

(use-modules (ice-9 control))

(define* (test #:key (unwind? #f))
  (let/ec return
    (with-exception-handler
        (let ((nested #f))
          (lambda (c-level-1)
            (if nested
                (begin
                  (format #t "level 1 handler got called recursively~%")
                  'level-1-handler-nested)
                (begin
                  (set! nested #t)
                  (with-exception-handler
                      (lambda (c-level-2)
                        (begin
                          (format #t "level 2 handler got error ~A~%" c-level-2)
                          (format #t "level 2 handler is returning~%")
                          (return 'level-2-handler)))
                    (lambda ()
                      (format #t "level 1 handler~%")
                      (error "let's signal a nested error...")
                      (format #t "level 1 handler is returning~%")
                      (return 'level-1-handler))
                    #:unwind? unwind?)))))
      (lambda ()
        (error "let's signal an error...")
        'thunk)
      #:unwind? unwind?)))

(define (main cmd)
  (unsetenv "COLUMNS")
  (format #t "~%~%*** calling with unwind~%")
  (format #t "return value is: ~A~%" (test #:unwind? #t))
  (format #t "~%~%*** calling without unwind~%")
  (format #t "return value is: ~A~%" (test #:unwind? #f)))

#|
$ guile --version
guile (GNU Guile) 3.0.9
$ guile --no-auto-compile -e main -s ~/workspace/guile/test.scm


*** calling with unwind
level 1 handler
level 2 handler got error #<&compound-exception components: (#<&error> #<&origin origin: #f> #<&message message: "~A"> #<&irritants irritants: ("let's signal a nested error...")> #<&exception-with-kind-and-args kind: misc-error args: (#f "~A" ("let's signal a nested error...") #f)>)>
level 2 handler is returning
return value is: level-2-handler


*** calling without unwind
level 1 handler
Backtrace:
In ice-9/boot-9.scm:
  1752:10 12 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
          11 (apply-smob/0 #<thunk 7f8d73313300>)
In ice-9/boot-9.scm:
    724:2 10 (call-with-prompt ("prompt") #<procedure 7f8d73324f80 …> …)
In ice-9/eval.scm:
    619:8  9 (_ #(#(#<directory (guile-user) 7f8d73316c80>)))
    163:9  8 (_ #(#(#<directory (guile-user) 7f8d73316c80>) ("/ho…")))
In ice-9/boot-9.scm:
    724:2  7 (call-with-prompt (let/ec) #<procedure 7f8d73352900 at…> …)
  1752:10  6 (with-exception-handler _ _ #:unwind? _ # _)
In ice-9/eval.scm:
    619:8  5 (_ #(#(#<directory (guile-user) 7f8d73316c80>)))
In ice-9/boot-9.scm:
   2007:7  4 (error _ . _)
  1685:16  3 (raise-exception _ #:continuable? _)
  1752:10  2 (with-exception-handler _ _ #:unwind? _ # _)
In ice-9/eval.scm:
    619:8  1 (_ #(#(#<directory (guile-user) 7f8d73316c80> #<proc…>)))
In ice-9/boot-9.scm:
   2007:7  0 (error _ . _)

ice-9/boot-9.scm:2007:7: In procedure error:
let's signal a nested error...
$
|#




#|
$ /gnu/store/xm08cp3fz3jxq3zddg9cvj59idy3ri8b-guile-3.0.99-git/bin/guile --no-auto-compile -e main -s ~/workspace/guile/test.scm


*** calling with unwind
level 1 handler
level 2 handler got error #<&compound-exception components: (#<&error> #<&origin origin: #f> #<&message message: "~A"> #<&irritants irritants: ("let's signal a nested error...")> #<&exception-with-kind-and-args kind: misc-error args: (#f "~A" ("let's signal a nested error...") #f)>)>
level 2 handler is returning
return value is: level-2-handler


*** calling without unwind
level 1 handler
level 2 handler got error #<&compound-exception components: (#<&error> #<&origin origin: #f> #<&message message: "~A"> #<&irritants irritants: ("let's signal a nested error...")> #<&exception-with-kind-and-args kind: misc-error args: (#f "~A" ("let's signal a nested error...") #f)>)>
level 2 handler is returning
return value is: level-2-handler
$
|#

RE: exception from inside false-if-exception?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 9443 bytes --]

I don’t think we are going to agree on much here, but some new points of agreement pop up occassionally. Continuing this …

>> What is lousy and expectation-violating about a keyword argument
>> doing what it name describes?
>well, this should have been discussed at the R6RS committee, but...

This is not something to be discussed at the R6RS committee, because #:unwind? is a Guile extension, not part of R6RS. Unless the idea is to (in an alternate past) have introduced keyword arguments to R6RS and this is about keyword argument stuff.

>an API is facilitating efficient reading comprehension when the "verbs" are closer to the beginning of the "sentence" (the sexp), and nothing later in the sentence adjusts the meaning of the verb fundamentally.

Do you have evidence for this? I mean, beyond your own experience, anecdotal evidence doesn’t generalise well. (Not that I don’t use anecdotal evidence or the like myself, but it would be better if at least one of us has something better.)

Also, I don’t see a fundamental change in meaning, except in the sense that anything could be considered fundamental, just like every two topics are related however distantly, etc..  #:unwind? #true/#false is too small a change for me to consider ‘fundamental’ – both #true?/#false result in exception handling, just slightly different exception handling. Neither do I really have an argument to consider it un-fundamental, size isn’t everything. In this context, it’s too meaningless a label to base things on or me.

Also, I disagree – this is not a garden path sentence situation (where, and OV (object-verb) languages exist. If you want to understand a sentence just read the entire sentence?

Now, I did some quick searches for differences in reading comprehension for OV/VO languages, and unfortunately found nothing relevant (neither positive nor negative nor neutral).

If we are talking about reading comprehension, I would like to point out that (Guile) Scheme is case-sensitive. I’d consider it beneficial to reading comprehension if the procedure names that are written in documentation correspond to what’s recognised by the implementation. Someone coming from case-insensitive languages and someone unfamiliar with the upcasing practice might get confused for a moment.

> and especially not a keyword arg at the end of the sentence that is not even mandatory, and defaults to one of the rather different possibilities.

Also, I would rather not have the keyword argument at the front. Usually keyword arguments aren’t verbs and better fit for the end of the procedure call, and even if they were sometimes better suited for the beginning, you now have a situation where sometimes keyword arguments are at the front and other times are at the end, which creates ambiguity when interpreting the arguments.

Another interpretation of this would be to replace the keyword argument by an optional argument, but optional arguments are at the end, so same thing.

A potential way to avoid this is to split things off into a new procedure, say with-exception-handler/unwinding or with-exception-handler/winding (well not these precise name because of issues mentioned earlier). While both provide a similar function (to the point that often you can swap between the two without trouble), if you were to precisely document them or implement them, the documentation and implementation would be rather different, so I think this would be better than an argument (whether (non-)keyword or optional or required) (it’s not like the unwinding-ness varies dynamically in practice, so not really a point in sharing a procedure name).

(I don’t think this really is an improvement in readability though, just a neatness thing.)

>in this case, it's a primitive to catch and handle exceptions. and it's possible to implement both #:unwind? #t and #f so that when the handler returns then the control flow continues at the guard, and never at the RAISE. it's just that one version would call the handler before unwinding.

AFAICT this paragraph as-written is technically true, but rather misleading – there is also raise-continuable, where the control flow needs to continue at raise-continuable.

>> If you have unwinded, it’s too late to return return from the raise
>> (unless the implementation is doing delimited continuations
>> shenanigans, which maybe you had in mind?), which explains the
>> behaviour for #:unwind #true.
>
>i didn't have anything on my mind, and that's the point. i'm simply new to scheme. >now that i've learned it, i'll learn to live with it.

[no replies to this part, I just don’t want to trim context / disrupt flow here]

>put another way, my learning curve would have been much steeper if the two, >rather different behaviors were defined under two distinct names (or at least >mentioned with bold in the documentation).

At first, I was going to write that you probably meant to write the opposite that if the different behaviour were more clearly different (e.g. with distinct names), then the learning would be _less_ steep, but now I read (Wikipedia article on learning curves):

>The common expression "a steep learning curve" is a misnomer suggesting that an
> activity is difficult to learn and that expending much effort does not increase
> proficiency by much, although a learning curve with a steep start actually represents
> rapid progress.[2][3]

(reading comprehension failure).

>> That said, I would prefer it to be named something like [#:handler-context 'raise]/[#:handler-context 'guard]

>that would be better, but i still wouldn't like the fact that it's focusing on the dynamic context of the handler, instead of the different flow of control.

I don’t get the distinction. These are two sides of the same coin? Almost equivalent, though I suppose that flow of control is a bit more general since if you only have info on the dynamic context then multiple flow of controls remain possible (but I don’t know any important change of flow that’s still correct exception handling).

Also, for a hypothetical #: [something flowy], I don’t like the fact that it’s focusing on the flow of control instead of the dynamic context of the handler.

Whenever I do #:unwind? #false/#:true (#:handler-context ‘raise’), it’s to get the dynamic environment/context (I forgot standard terminology) right (think call stack and parameter bindings (either at raise or guard, depending on what you are implementing)). I don’t really care about Guile does the control flow is implemented, as long as the dynamic environment is correct and raise-continuable functions (though TBH I almost never use raise-continuable, and when I do, I always think something along the lines of “neat that this is possible, but there are other methods that are simpler both to implement (whether caller/callee) and to understand”).

>for reference, in CL they are called HANDLER-CASE and HANDLER-BIND, with completely different syntaxes.

Can’t honestly say I like those names (those suffixes -CASE / -BIND don’t really provide information, you have to go into the spec of HANDLER-CASE / HANDLER-BIND itself to get the differences).

Going by this and some earlier text, it seems we are in agreement that with-exception-handler should be split (albeit for different reasons).

>> Wait where did this happen? You say what’s happening, but you don’t
>> seem to be referring to false-if-exception stuff, and you didn’t
>> mention continuation barriers earlier.

>this has caused me layers of confusion, which is sadly reflected in my mails.

>guile prints a backtrace without any prelude, which made me think that it's my own code that is printing this backtrace and exception in the logs (remember, i'm working on nested error handling and logging in shepherd). then 3.0.9 does something funny with the control flow, which further made me believe that i'm logging an exception coming from inside FALSE-IF-EXCEPTION and it's somehow flying past my error handler, and reaching fibers.
>
>and i think i'm still confused about a possible continuation barrier being injected somewhere that probably/maybe causes the exception from inside FALSE-IF-EXCEPTION somehow ending up at fibers instead of my error handler. but at this point i'm reaching the frontier of my understanding of scheme, delimited continuations, fibers, etc.
>
>so, one step at a time. once this test.scm is clear, and i'm able to run shepherd on guile-next, then i'll get back to investigate my original issue in its original context.

Maybe you can install a breakpoint on scm_c_with_continuation_barrier or something, and see when this happens. Or search for with-continuation-barror in Scheme code. I don’t think Shepherd has much use of with-continuation-barrier. If it does, it’s a rather crude tool and can probably be replaced by something else that doesn’t replace the exception handler.

I don’t know what you are investigating precisely, but maybe it is

>https://sources.debian.org/src/guix/1.4.0-6/gnu/build/shepherd.scm/?hl=112#L109

If it is this, surely (@ (fibers) sleep) could be adjusted to _not_ suspend (and instead just, well, sleep) when there is no Fibers task scheduler (similar stuff exists for the (Fibers) channels and conditions implementation).

Best regards,
Maxime Devos.

[-- Attachment #2: Type: text/html, Size: 13234 bytes --]

RE: exception from inside false-if-exception?

[Attila Lendvai]

> Also, I don’t see a fundamental change in meaning


just look at my test.scm:

without the explicit (let/ec return ...) wrapper, and the explicit use of (return ...) in the handlers, the two invocations lead to different flow of control.

it's icing on the cake that if the underlying ERROR were in fact a RAISE-CONTINUABLE, then it wouldn't even lead to a &NON-CONTINUABLE error in the #:unwind? #f case, only to a different return value (different flow of control).

in my vocabulary, a different flow of control when using a control-flow primitive is a fundamental change to its meaning. but your miliage may wary of course.


> If we are talking about reading comprehension, I would like to point
> out that (Guile) Scheme is case-sensitive.


for effective communication we need a syntax to designate when a word is to be interpreted in the scheme/lisp domain (as opposed to plain english). using all-caps in english prose is a somewhat well-spread syntax for that, even in non-CL lisp contexts in my experience.


> Someone coming from case-insensitive languages and someone
> unfamiliar with the upcasing practice might get confused for a
> moment.


for a moment. and that is key.


> >that would be better, but i still wouldn't like the fact that it's
> >focusing on the dynamic context of the handler, instead of the
> >different flow of control.
> 
> I don’t get the distinction. These are two sides of the same coin?


not really, because there are two, orthogonal dimensions here:

1) whether to unwind the stack,

2) where to return the control flow when the handler returns.

(even if one combination is impossible)


> >for reference, in CL they are called HANDLER-CASE and HANDLER-BIND, with completely different syntaxes.
>
> Can’t honestly say I like those names (those suffixes -CASE / -BIND
> don’t really provide information, you have to go into the spec of
> HANDLER-CASE / HANDLER-BIND itself to get the differences).


my point is not whether it's deducible, or whether the names are good. my point is that one API is misleading (i.e. wastes human time), while the other forces the reader to, at worst, look up the documentation. but certainly not conflate the two, thinking that they are doing the same thing -- plus some implementation detail regarding unwinding that he can postpone learning about.

here's a caricature of this situation:

imagine a hypothetical (raise [a potentially larger sexp] #:enabled? #t/#f) control flow primitive that simply returns when #:enabled? is #f.

one can argue that "well, it's disabled, dummy!" (turing tarpit), or try to walk in the shoes of the reader or a newcomer.

yes, you're right when you say something is possible, or deducible. but no, with all due respect, you're wrong when you imply that it doesn't make a difference (in human-brain efficiency).


> Going by this and some earlier text, it seems we are in agreement
> that with-exception-handler should be split (albeit for different
> reasons).


yes.


> Maybe you can install a breakpoint on
> scm_c_with_continuation_barrier or something


it assumes that it's already known that the curlpit is w-c-b, but that was already well into the debugging effort. see my attached patch that intends to greatly shorten the debugging time for anyone else walking down this path after me.

and it's been decades i've done any serious C coding, and i'm not looking forward to learning the ins and outs of gdb when i'm debugging an issue in scheme.


> I don’t think Shepherd has much use of with-continuation-barrier.


IIUC, any time the posix thread enters C code is an implicit continuation barrier. if a fiber in shepherd uses any primitive in guile that is implemented in C, then it's a continuation barrier (and breaks when an attempt is made to suspend that fiber).

changing shepherd not to unconditionally unwind (so that meaningful backtraces can be logged) can lead to subtle issues due to now having some extra C frames on the stack that were previously unwound prior to handling the error.

i even suspect that a continuation barrier may somehow hide, or neutralize my w-e-h higher up on the stack, and the exception from DISPLAY-BACKTRACE ends up at the fibers scheduler instead of my w-e-h due to the above.

but with that i'm again at the frontier of my understanding. to say anything more useful i'll need to test my shepherd branch using the latest guile.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Your head is for having ideas not for holding them.”
	— David Allen (1945–), http://youtu.be/D-3nTl8M44o?t=6m12s

RE: exception from inside false-if-exception?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 14921 bytes --]

> Also, I don’t see a fundamental change in meaning
>>just look at my test.scm:

>>without the explicit (let/ec return ...) wrapper, and the explicit use of (return ...) in the handlers, the two invocations lead to different flow of control.
>>It's icing on the cake that if the underlying ERROR were in fact a RAISE-CONTINUABLE, then >it wouldn't even lead to a &NON-CONTINUABLE error in the #:unwind? #f case, only to a >different return value (different flow of control).

‘error’ is a procedure, not something that would be raise-continuable (I guess you could pass it to ‘raise-continuable’, but seems rather unlikely). I propose replacing ERROR by condition (lack of quotes and capitals intentional, since I’m referring to the concept here, not a Scheme object or macro and in particular not the macro ‘condition’ from SRFI-35

Now I look at the output (somehow when I previously read it I thought it was testing control flow with dynamic-wind, but it turns out it is testing semantics instead (without dynamic-wind) – reading comprehension failure on my part?), the observed behaviour (for v3.0.9) is indeed a significant change.

I don’t think test.scm is a good example though – the output is, AFAICT, just a bug – for the nested error, AFAICT neither of the exception handlers receive the exception, which is plain nonsense.

>>In my vocabulary, a different flow of control when using a control-flow primitive is a >>fundamental change to its meaning. but your miliage may wary of course.

It’s not in mine. What matters to me is (denotational) semantics (in a broad sense, including things like complexity, whether something is constant-time (important for cryptography), etc.). Control-flow is more a operational semantics thing. Often a convenient way of defining the semantics of control structures, but ultimately, if with a different control flow you achieve the same semantics (and aren’t compiling with the equivalent of -Og), that’s fine too.

>> If we are talking about reading comprehension, I would like to point
>> out that (Guile) Scheme is case-sensitive.
>for effective communication we need a syntax to designate when a word is to be
> interpreted in the scheme/lisp domain (as opposed to plain english). using all-caps
> in english prose is a somewhat well-spread syntax for that, even in non-CL lisp
> contexts in my experience.

A simple syntax is to use quotes, e.g. ‘call-with-current-continuation’, which is well-spread (even outside programming to some degree), and does not require mangling the casing. Depending on the medium, there are other options like monotype, putting a small box around it or coloring. For texts (but not e-mail or terminal things), I recommend monotype (and depending on capabilities of the writing software and aesthetic preferences, maybe try out things like coloring and boxes).

So …

>> Someone coming from case-insensitive languages and someone
> unfamiliar with the upcasing practice might get confused for a
> moment.
>for a moment. and that is key.

… why not both eliminate the moment of confusion _and_ use a syntax that avoids the ambiguity?

>> >that would be better, but i still wouldn't like the fact that it's
>> >focusing on the dynamic context of the handler, instead of the
>> >different flow of control.
>> 
> I don’t get the distinction. These are two sides of the same coin?
>
>not really, because there are two, orthogonal dimensions here:
>
>1) whether to unwind the stack,
>
>2) where to return the control flow when the handler returns.
>
>(even if one combination is impossible)

Err, all four combinations are possible with sufficient application of delimited continuations. If you have unwinded you can just rewind (if you have captured continuation things and accept the change in visible behaviour to ‘dynamic-wind’, the first is not a given because of performance considérations and the latter isn’t a given because, well, different behaviour).

There are two components to (1):

   (1)(a): what does dynamic-wind see?
   (1)(b): how does unwinding affect the dynamic environment of the handler.

For (2): first, there is no ‘return the control flow’ happening! The returning is all in tail position (albeit with some extra layers in the dynamic environment, so don’t do tail loops with with-exception-handler or you will use lots of memory) – according to the docs, once the handler is invoked, with-exception-handler stops doing things.

As mentioned in the documentation:

> https://www.gnu.org/software/guile/manual/html_node/Raising-and-Handling-Exceptions.html
>Unless with-exception-handler was invoked with #:unwind? #t, exception handlers are invoked __within the continuation of the error__, without unwinding the stack. The dynamic environment of the handler call will be that of the raise-exception call, with the difference that the current exception handler will be “unwound” to the \"outer\" handler (the one that was in place when the corresponding with-exception-handler was called).
> […]
> if with-exception-handler was invoked with #:unwind? #t is true, raise-exception will first unwind the stack by invoking an escape continuation (see call/ec), and then invoke the handler __with the continuation of the with-exception-handler call__.

(emphasis added with __)

Now, while ‘return the control flow’ is not accurate, there is definitely some control flow manipulation going on (see: ‘call/ec’). But consider this: whether you are in tail-position w.r.t. something, is a property of the dynamic environment! I mean, it definitely is visible in backtraces (which you can make with ‘(backtrace)’, doesn’t need exceptions).

Let’s look at some R6RS definitions :

>For a procedure call, the time between when it is initiated and when it returns is called its dynamic extent. [currently irrelevant things about call/cc]
>Some operations described in the report acquire information in addition to their explicit arguments from the dynamic environment.

(Technically it doesn’t state that the concept of ‘dynamic environment’ includes the ‘dynamic extent’, but it seems quite natural to consider all the dynamicness together, and ‘environment’ seems a sufficiently generic word to also include the ‘extent’.)

(dynamic extent <-> a region of stack, without focus on what the dynamic bindings in that stack are, dynamic environment <-> the full stack upto a point, with a focus at what’s visible at the point]

Summarised: stack stuff is part of the dynamic environment, control flow is stack stuff, so control flow is dynamic environment stuff.

>> >for reference, in CL they are called HANDLER-CASE and HANDLER-BIND, with >completely different syntaxes.
>>
>> Can’t honestly say I like those names (those suffixes -CASE / -BIND
>> don’t really provide information, you have to go into the spec of
>> HANDLER-CASE / HANDLER-BIND itself to get the differences).

>my point is not whether it's deducible, or whether the names are good. my point is that one API is misleading (i.e. wastes human time), while the other forces the reader to, at worst, look up the documentation. but certainly not conflate the two, thinking that they are doing the same thing -- plus some implementation detail regarding unwinding that he can postpone learning about.

It wasn’t my point either, I didn’t think you liked them either, I didn’t think your point was whether it’s deducible, I just interpreted it as an example of splitting stuff up and wanted to make clear those don’t seem good names.

>here's a caricature of this situation:
>
>imagine a hypothetical (raise [a potentially larger sexp] #:enabled? #t/#f) control flow primitive that simply returns when #:enabled? is #f.
>
>one can argue that "well, it's disabled, dummy!" (turing tarpit), or try to walk in the shoes of the reader or a newcomer.
>
>yes, you're right when you say something is possible, or deducible. but no, with all due respect, you're wrong when you imply that it doesn't make a difference (in human-brain efficiency).

With all due respect, that’s a strawman (as expected by a caricature).

I never claimed #:unwind? is a great name, that keyword #:enabled? is terribly named and even worse than #:disable, both you and I previously made clear that naming is important, I proposed renamings of #:unwind? to other things (that you even agreed to were better) or proposals for names for splitting up the procedure (not great names, but still more descriptive than #:enabled? Of all things).

Also, let’s do this walking into the shoes of the newcomer. As a newcomer, one of the things I do is _reading the documentation_ (I have heard this is a rare thing, but I actually do this, and did this).

As a newcomer, I find these references to delimited continuations, dynamic environment and tail call thingies, really interesting and cool, so I read a lot about it (more than I actually use them, even!) and try it out a bit. I also see these mentions of virtual machines and stack frames. I read those too of course, how stack frames are done is really cool and interesting (VM a bit less so, but still ok to skim through).

As a reader, I also think that sometimes the descriptions aren’t done well – it’s not a verbosity thing despite what some would say, but more a structure thing (both local or global). But that’s a property of the documentation, not the API.

I also thing that occassionally the APIs aren’t quite right, making it difficult to understand when to, say, use #:unwind? #false/#true. But as a newcomer, I don’t _need_ to know those distinctions, I can just assume the default is fine and avoid doing fancy dynamic environment stuff inside the exception handler. So, I can just avoid the complicated unwinding-or-not stuff until I’m not a newcomer anymore, and once that’s the case, I can start thinking about how to improve API stuff and utilise the fancy stuff.

Going back to the strawman:

> you're wrong when you imply that it doesn't make a difference (in human-brain efficiency).

I didn’t imply this! Just because I said you need to _read the documentation_ doesn’t mean that naming and the like can’t be improved, the existence of a ‘problem between monitor and chair’ does not exclude the existence of a problem in the software as well. Also that the status quo is ok doesn’t mean it can’t be be improved.

(I think the _name_ itself is fine, rather, I think that the _concept_ that it represents would preferably be changed to something different (and naturally this change in concept induces a change in name and perhaps semantics as well).)

>> Maybe you can install a breakpoint on
>> scm_c_with_continuation_barrier or something
>
>it assumes that it's already known that the curlpit is w-c-b, but that was already well >into the debugging effort. see my attached patch that intends to greatly shorten the >debugging time for anyone else walking down this path after me.
>
>and it's been decades i've done any serious C coding, and i'm not looking forward to >learning the ins and outs of gdb when i'm debugging an issue in scheme.

Of course it assumes that the culprit is ‘with-continuation-barrier’. You provided the information that ‘with-continuation-barrier’ might have something to do with the issue to me. Also, I didn’t mention gdb anywhere. It’s a common debugger, but not the only one. And neither did I mention learning the ins and outs – setting up a breakpoint and asking gdb to print a backtrace is hardly ‘the ins and outs’. (Whether the information of the backtrace will prove useful is another matter.)

Also, difficulty with C debugging is your problem, not mine – this thread isn’t named “how to simplify debugging from C”, as a volunteer I’m just mentioning some options to consider and I’m not your boss. Besides, given that the existence of FFI and the like, and that Guile it partially implemented in C instead of Scheme, it isn’t a given that the issue you are debugging is, in fact, fully in Scheme.

(Also maybe look if Shepherd is doing REPL stuff. The REPL implementation in Guile installs (in Scheme!) a continuation barrier.)

> I don’t think Shepherd has much use of with-continuation-barrier.
>
> IIUC, any time the posix thread enters C code is an implicit continuation barrier. if a
> fiber in shepherd uses any primitive in guile that is implemented in C, then it's a
> continuation barrier (and breaks when an attempt is made to suspend that fiber).

Not in the sense of with-continuation-barrier. C code can throw Scheme exceptions and those exception can in turn be caught by normal Scheme exception (but no raise-continuable stuff from C).

On a related note, there is also some (non-Scheme) dynwind-rewind stuff in Guile’s C code. I don’t have a clue on how that works and if it even works at all.

> changing shepherd not to unconditionally unwind (so that meaningful backtraces
> can be logged) can lead to subtle issues due to now having some extra C frames on
> the stack that were previously unwound prior to handling the error.

That’s not unwinding, that’s rewinding. Unwinding can’t lead to leaving extra C frames on the stack, because removing those C frames is part of the unwinding process and C frames are only generated by C code.  (Except perhaps for that dynwind-rewind stuff, but if it does that stuff, presumably it is prepared to deal with rewinding.)

Hypothetically a Scheme implementation could put C frames back onto the stack, but Guile doesn’t do this – whenever this would happen, it simply refuses(*). (IIRC it does so by throwing an exception.). From the documentation:

(*) besides perhaps that dynwind-rewind stuff.

>Scheme Procedure: suspendable-continuation? tag
>Return #t if a call to abort-to-prompt with the prompt tag tag would produce a >delimited continuation that could be resumed later.
>
>Almost all continuations have this property. The exception is where some code between the call-with-prompt and the abort-to-prompt recursed through C for some reason, the abort-to-prompt will succeed but any attempt to resume the continuation (by calling it) would fail. This is because composing a saved continuation with the current continuation involves relocating the stack frames that were saved from the old stack onto a (possibly) new position on the new stack, and Guile can only do this for stack frames that it created for Scheme code, not stack frames created by the C compiler. It’s a bit gnarly but if you stick with Scheme, you won’t have any problem.
>
>If no prompt is found with the given tag, this procedure just returns #f.

Best regards,
Maxime Devos.

[-- Attachment #2: Type: text/html, Size: 28074 bytes --]