From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id kM2JCSAsCWSgHgAASxT56A (envelope-from ) for ; Thu, 09 Mar 2023 01:45:20 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id gI94CSAsCWSjQwAAauVa8A (envelope-from ) for ; Thu, 09 Mar 2023 01:45:20 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3D570923C for ; Thu, 9 Mar 2023 01:45:19 +0100 (CET) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.de header.s=2017 header.b=P8R2D7NL; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1678322720; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=BR7ZDJEfw5EttRehOAFnve4BGJ4y7osgszOP4mHNuUI=; b=Tmt+/8qRLFw/GzQ57NsOAWV50bXar6F9Z6y7fL+BC3oGUdCL6knNU6hM3L9Lt2NImhoNWm jmhDVB7XjJNWzpDSDdxk45+DXUMqn3zoEvxUbnEBdgJXv1NtuKBR9b1/g1TL1blvL+MVdo lQSn4iM6IjMX1Z+SJXLKfBOEUEMQTTS1X/M4UpFPNnLhGqkNfUMH40TYlcbOvF2nyoYudh 8sbAffoxVagZOI3imav1wsFEqT767cHvpRo6LfVxwIkIP2+IRqr33Sv+Xg2buBjRID5qV8 i4VYi59iMUPmQLa1h3vRfsOKflwjQisA2zdtxOQIA+9Ro7wqHErgaDm3f0iJEQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1678322720; a=rsa-sha256; cv=none; b=cDOSUt9fTZTUd+HULlypmN3Wz/aUEkWZqLMu1NTcCW/wehx8PLqPTAxg8/OPCU9gPmfCdg rfj6VgJ3e9mJ7uxlTbDDzoiCcORXplj3T2VeugLG4NaW6a5trrxRRmp/iwAKvNx4ZUPuwK IJscysuYqb3L1zbFUmyHB1IlZzPRt3J7Www9wcCApuyh6N7+Y8p+Gmbha32S1Y9Dthghwl TQ3RtMV+54ATniqNAA1t58QzzIOwtSLLBAFZyYLNfXzLOgKi7n6Oxues3x7XFv6Z7rMC22 BRqBYI3SlGK6rqbkga63LeBbfj33Hi94MQxDe5D/H6Ec9csb9XgvuHjcieChRA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.de header.s=2017 header.b=P8R2D7NL; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.de Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pa4OO-0006sC-N9; Wed, 08 Mar 2023 19:44:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pa4OM-0006rt-Oy for emacs-orgmode@gnu.org; Wed, 08 Mar 2023 19:44:22 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pa4OJ-00068l-E0 for emacs-orgmode@gnu.org; Wed, 08 Mar 2023 19:44:22 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id A0B06240654 for ; Thu, 9 Mar 2023 01:44:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.de; s=2017; t=1678322655; bh=hSukihPbrn9MgOuYY1AyDrWizkeHoxjmzHaFcjcxqwc=; h=Date:Subject:To:Cc:From:From; b=P8R2D7NLSVR/x7pdr1xlrbp+8mY2cEuooux3UpqSrU0Q/AoeUqGbsKO5dMyrjmg94 6dXwa2URIUFhlqcR5IypDWdayt790XUnR83E1pCk3nnH5apy9txifG4r5L1YMnkY2r VbY1dqpPclojAaot0Wge2ehuPLEVGR0mVOl1NZ3lsb2oXiR1FxTdjorJn4KTEXuNi7 ke8t+V6xGzD6HSRoed8TSVZKzYiNClgVa7riW/A1a7/kD7FFjbfrwpiDOJudqoymqg stXdtLKk3LfZ0Irt/KA12oObtRXu7CDFhoS9m8hRTcDO4EwWwMnCg/QzsAeWB2xSnR sUrymQV/JK5Og== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4PX9Rf3PKBz9rxB; Thu, 9 Mar 2023 01:44:14 +0100 (CET) Content-Type: multipart/alternative; boundary="------------MOU19QOVAvqdPHcrjwDwF0A8" Message-ID: Date: Thu, 9 Mar 2023 00:44:13 +0000 MIME-Version: 1.0 Subject: Re: org-babel guile source block bug in handling multiple values To: Bruno Barbier Cc: emacs-orgmode@gnu.org, Ihor Radchenko References: <9eab60bc-9b82-e037-d63b-3d879573ae32@posteo.de> <87v8jceihi.fsf@localhost> <7fc63848-d6d3-80e0-ae78-00967990813d@posteo.de> <64079614.170a0220.5a0d3.0a23@mx.google.com> <97ee254e-72d2-2bdf-e026-78bde076f1f9@posteo.de> <6408e424.5d0a0220.8862a.2a62@mx.google.com> From: Zelphir Kaltstahl Content-Language: en-US In-Reply-To: <6408e424.5d0a0220.8862a.2a62@mx.google.com> Received-SPF: pass client-ip=185.67.36.66; envelope-from=zelphirkaltstahl@posteo.de; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: X-Migadu-Scanner: scn0.migadu.com X-Migadu-Queue-Id: 3D570923C X-Spam-Score: -11.78 X-Migadu-Spam-Score: -11.78 List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-TUID: WtjBHe7yRuMk This is a multi-part message in MIME format. --------------MOU19QOVAvqdPHcrjwDwF0A8 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 3/8/23 20:38, Bruno Barbier wrote: > Hi Zelphir, > > Zelphir Kaltstahl writes: > >> On 3/7/23 20:52, Bruno Barbier wrote: >> Also thanks for the idea with sessions + separate import source block. I thought >> that should work, but apparently that also has the same error, when running for >> the first time: >> >> ... > Oh, I see. I tested something way simpler :-) > > First, one block to open and configure the guile session. > > #+begin_src scheme :session "!guile" :results silent > (import (except (rnrs base) error vector-map) > (only (guile) > lambda* > λ) > ;; let-values > (srfi srfi-11)) > #+end_src > > Then, you can get to work and evaluate as many blocks as you like in > that session: > > #+begin_src scheme :session "!guile" :results output replace drawer :var x=1 :var y=2 > (let-values ([(a b) (values x y)]) > (simple-format #t "~a ~a\n" a b)) > #+end_src > > #+RESULTS: > :results: > 1 2 > :end: > > Bruno Hello Bruno and hello mailing list! I just tested it a little more: If there is any (import ...) at all in the code block that makes use of the (let-values ...), it seems to somehow disturb already imported libraries. For example the following also does not work: ~~~~START~~~~ #+name: scheme-time-imports #+begin_src scheme :eval query-export :noweb strip-export :session scheme-time :results output replace drawer" (import (except (rnrs base) error vector-map) (only (guile) lambda* λ) ;; let-values (srfi srfi-11)) #+end_src #+RESULTS: scheme-time-imports #+name: scheme-time #+begin_src scheme :eval query-export :noweb strip-export :session scheme-time :results output replace drawer :var x=1 :var y=2 <> (import (srfi srfi-1)) (let-values ([(a b) (values x y)]) (simple-format #t "~a ~a\n" a b)) #+end_src ~~~~~END~~~~~ So that means, even if the import has nothing to do with the actual import which would provide the let-values form, it disturbs it. I am not sure (let ...) is a correct wrapper for noweb included source blocks. What, if I write a (define ...) in my source block and want to use that source block via noweb in another source block? Expected behavior I think would be to be able to access those variables in other source blocks, since they are defined on a top level in an earlier source block, but if they are wrapped in a (let ...), that would make them only available in the (let ...)? It seems to me, that the simple wrapping with a (let ...) might not be the right thing to do. Testing that: ~~~~START~~~~ #+name: scheme-defs #+begin_src scheme :eval query-export :noweb strip-export :session myguile :results output replace drawer :var x=1 :var y=2 (define a x) (define b y) #+end_src #+name: scheme-time #+begin_src scheme :eval query-export :noweb strip-export :session myguile :results output replace drawer <> (simple-format #t "~a ~a\n" a b) #+end_src ~~~~~END~~~~~ Indeed, that also does not work. I guess I did never hit this problem earlier, because I "oursourced" my imports and in imports I do not need any :var header arguments. I've asked on the Guile IRC channel and something interesting is the case here (thanks for clearing it up flatwhatson!) and I understand it as follows: Imports inside (let ...) work. It is just that let-values is a macro and macros are expanded before execution time. However, Guile gets to the body of the wrapping (let ...) at execution time. That means, that when Guile gets to evaluate the body of the let, it does not expand the let-values, because it is already at execution time and no longer at macro expansion time. The import might import the let-values form, or might not, but it is already too late to expand the (let-values ...). What is still a bit weird about it is, that in the original example with `let-values` I don't get an error about `let-values` not being defined, but only about `a` not being defined. And in the example with (define ...) and :var above, I get a message about `x` not being defined, instead of `a` not being defined. Probably a good general workaround is to only have imports at the top level, by moving them into a source block, which does not have any :var header arguments. OK, the question is though, whether org should wrap anything in a (let ...) at all. During discussion on the Guile IRC, some points against let-wrapping were brought up: (1) The presence of a :var header argument currently determines, whether the code in the source block is wrapped with a (let ...). One argument for that was, that this way the variables do not leak. But this also decides, whether other things leak. For example (import ...) or (define ...). Should :var decide, whether bindings created with (define ...) are visible in other source blocks including the source block with the :var header arguments? It seems like a responsibility :var should not have and definitely is unexpected for the user. (2) Wrapping in a (let ...) also moves things from the top level to the inside of expressions. This can make them move from one phase to another. Phases such as described here: https://www.gnu.org/software/guile/manual/html_node/Eval-When.html. This is probably not intended and also unexpected. (3) Not wrapping with a (let ...) expression and instead going with simple (define ...) for :var variables seems a very general solution, that should be portable to all Schemes. (4) To make things easily understandable, bindings defined in a source block that is included in another source block, should either always leak, or never. Not sometimes this way and sometimes that way. But conditionally wrapping with (let ...) if there is a :var header argument introduces a behavior that changes that. (5) Imagine you already have a finished document and want to extend it, by letting one of the source blocks take a variable via :var header argument. Now you must change the source block as well, and not only place the variable's name somewhere, but also need to take care of imports and evaluation and expansion phases. (6) Looking at other languages and how it behaves there. Say Python for example, there are no (let ...) expressions available, so no let-wrapping could happen. Unless org invents an additional `def` to wrap it in that, but only if there are :var header arguments. Lets see how things work there: ~~~~START~~~~ #+name: python-imports #+begin_src python :python /usr/bin/python3 :results output replace drawer :var x=4 import math y = math.sqrt(x) # print(y) #+end_src #+name: python-usage #+begin_src python :python /usr/bin/python3 :return :noweb strip-export :results value replace drawer <> print("y: {}".format(y)) #+end_src ~~~~~END~~~~~ Unfortunately, this example does not seem to work at all, but for a different reason: It seems that using any Python source block with :var header args via :noweb does not work, as it then behaves in the way, that it merely pasted the included source block, without first putting in the :var values into the variables. I get errors about those :var variables being undefined, of course, since they are on the included source block, not on the including one: ~~~~START: *Org-Babel Error Output*~~~~ Traceback (most recent call last): File "", line 10, in File "", line 5, in main NameError: name 'x' is not defined [ Babel evaluation exited with code 1 ] ~~~~~END~~~~~ So it would seem, that :var does not work with :noweb for Python source blocks yet. (7) As an idea, one could still invent a new header argument that determines whether to let-wrap or not, if it is really needed for any other Scheme. OK, to wrap up (ha!), I want to ask: (q1) What is a rationale, if any, behind the let-wrapping? (q2) Any chances of that changing to (define ...)? (q3) How could I change my org-mode's code to not  let-wrap, and instead use (define ...)? Best regards, Zelphir -- repositories:https://notabug.org/ZelphirKaltstahl --------------MOU19QOVAvqdPHcrjwDwF0A8 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
On 3/8/23 20:38, Bruno Barbier wrote:
Hi Zelphir,

Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:


On 3/7/23 20:52, Bruno Barbier wrote:


      

        
Also thanks for the idea with sessions + separate import source block. I thought 
that should work, but apparently that also has the same error, when running for 
the first time:

...


      

      
Oh, I see. I tested something way simpler :-)

First, one block to open and configure the guile session.

     #+begin_src scheme :session "!guile" :results silent
     (import (except (rnrs base) error vector-map)
              (only (guile)
                    lambda*
                    λ)
              ;; let-values
              (srfi srfi-11))
     #+end_src

Then, you can get to work and evaluate as many blocks as you like in
that session:

     #+begin_src scheme :session "!guile" :results output replace drawer :var x=1 :var y=2
     (let-values ([(a b) (values x y)])
        (simple-format #t "~a ~a\n" a b))
     #+end_src

     #+RESULTS:
     :results:
     1 2
     :end:

Bruno

    

    Hello Bruno and hello mailing list!
    
I just tested it a little more:

    
If there is any (import ...) at all in the code block that makes
      use of the (let-values ...), it seems to somehow disturb already
      imported libraries. For example the following also does not work:

    
~~~~START~~~~
#+name: scheme-time-imports
#+begin_src scheme :eval query-export :noweb strip-export :session scheme-time :results output replace drawer"
(import
 (except (rnrs base) error vector-map)
 (only (guile)
       lambda*
       λ)
 ;; let-values
 (srfi srfi-11))
#+end_src

#+RESULTS: scheme-time-imports

#+name: scheme-time
#+begin_src scheme :eval query-export :noweb strip-export :session scheme-time :results output replace drawer :var x=1 :var y=2
<<scheme-time-imports>>
(import (srfi srfi-1))
(let-values ([(a b) (values x y)])
   (simple-format #t "~a ~a\n" a b))
#+end_src
~~~~~END~~~~~


    
So that means, even if the import has nothing to do with the
      actual import which would provide the let-values form, it disturbs
      it.

    
I am not sure (let ...) is a correct wrapper for noweb included
      source blocks. What, if I write a (define ...) in my source block
      and want to use that source block via noweb in another source
      block? Expected behavior I think would be to be able to access
      those variables in other source blocks, since they are defined on
      a top level in an earlier source block, but if they are wrapped in
      a (let ...), that would make them only available in the (let ...)?
      It seems to me, that the simple wrapping with a (let ...) might
      not be the right thing to do. Testing that:

    
~~~~START~~~~
#+name: scheme-defs
#+begin_src scheme :eval query-export :noweb strip-export :session myguile :results output replace drawer :var x=1 :var y=2
(define a x)
(define b y)
#+end_src

#+name: scheme-time
#+begin_src scheme :eval query-export :noweb strip-export :session myguile :results output replace drawer
<<scheme-defs>>
(simple-format #t "~a ~a\n" a b)
#+end_src
~~~~~END~~~~~


    
Indeed, that also does not work.

    
I guess I did never hit this problem earlier, because I
      "oursourced" my imports and in imports I do not need any :var
      header arguments.

    
I've asked on the Guile IRC channel and something interesting is
      the case here (thanks for clearing it up flatwhatson!) and I
      understand it as follows:

    
Imports inside (let ...) work. It is just that let-values is a
      macro and macros are expanded before execution time. However,
      Guile gets to the body of the wrapping (let ...) at execution
      time. That means, that when Guile gets to evaluate the body of the
      let, it does not expand the let-values, because it is already at
      execution time and no longer at macro expansion time. The import
      might import the let-values form, or might not, but it is already
      too late to expand the (let-values ...).

    
What is still a bit weird about it is, that in the original
      example with `let-values` I don't get an error about `let-values`
      not being defined, but only about `a` not being defined. And in
      the example with (define ...) and :var above, I get a message
      about `x` not being defined, instead of `a` not being defined.

    
Probably a good general workaround is to only have imports at the
      top level, by moving them into a source block, which does not have
      any :var header arguments.

    
OK, the question is though, whether org should wrap anything in a
      (let ...) at all. During discussion on the Guile IRC, some points
      against let-wrapping were brought up:

    
(1) The presence of a :var header argument currently determines,
      whether the code in the source block is wrapped with a (let ...).
      One argument for that was, that this way the variables do not
      leak. But this also decides, whether other things leak. For
      example (import ...) or (define ...). Should :var decide, whether
      bindings created with (define ...) are visible in other source
      blocks including the source block with the :var header arguments?
      It seems like a responsibility :var should not have and definitely
      is unexpected for the user.

    
(2) Wrapping in a (let ...) also moves things from the top level
      to the inside of expressions. This can make them move from one
      phase to another. Phases such as described here: https://www.gnu.org/software/guile/manual/html_node/Eval-When.html.
      This is probably not intended and also unexpected.

    
(3) Not wrapping with a (let ...) expression and instead going
      with simple (define ...) for :var variables seems a very general
      solution, that should be portable to all Schemes.

    
(4) To make things easily understandable, bindings defined in a
      source block that is included in another source block, should
      either always leak, or never. Not sometimes this way and sometimes
      that way. But conditionally wrapping with (let ...) if there is a
      :var header argument introduces a behavior that changes that.

    
(5) Imagine you already have a finished document and want to
      extend it, by letting one of the source blocks take a variable via
      :var header argument. Now you must change the source block as
      well, and not only place the variable's name somewhere, but also
      need to take care of imports and evaluation and expansion phases.

    

    
(6) Looking at other languages and how it behaves there. Say
      Python for example, there are no (let ...) expressions available,
      so no let-wrapping could happen. Unless org invents an additional
      `def` to wrap it in that, but only if there are :var header
      arguments. Lets see how things work there:

    
~~~~START~~~~
#+name: python-imports
#+begin_src python :python /usr/bin/python3 :results output replace drawer :var x=4
import math

y = math.sqrt(x)
# print(y)
#+end_src

#+name: python-usage
#+begin_src python :python /usr/bin/python3 :return :noweb strip-export :results value replace drawer
<<python-imports>>

print("y: {}".format(y))
#+end_src
~~~~~END~~~~~


    
Unfortunately, this example does not seem to work at all, but for
      a different reason:

    
It seems that using any Python source block with :var header args
      via :noweb does not work, as it then behaves in the way, that it
      merely pasted the included source block, without first putting in
      the :var values into the variables. I get errors about those :var
      variables being undefined, of course, since they are on the
      included source block, not on the including one:

    
~~~~START: *Org-Babel Error Output*~~~~
Traceback (most recent call last):
  File "<stdin>", line 10, in <module>
  File "<stdin>", line 5, in main
NameError: name 'x' is not defined
[ Babel evaluation exited with code 1 ]
~~~~~END~~~~~


    
So it would seem, that :var does not work with :noweb for Python
      source blocks yet.

    
(7) As an idea, one could still invent a new header argument that
      determines whether to let-wrap or not, if it is really needed for
      any other Scheme.

    

    
OK, to wrap up (ha!), I want to ask:

    
(q1) What is a rationale, if any, behind the let-wrapping?

    
(q2) Any chances of that changing to (define ...)?

    

    
(q3) How could I change my org-mode's code to not  let-wrap, and
      instead use (define ...)?

    

    
Best regards,

      Zelphir

    

    
-- 
repositories: https://notabug.org/ZelphirKaltstahl

  


--------------MOU19QOVAvqdPHcrjwDwF0A8--