babel default header args as functions

babel default header args as functions

[Matt Huszagh]

I've added the ability in my own configuration to use lambda functions
that evaluate to a string as babel default header arguments, instead of
just the plain strings currently allowed. Would anyone else be
interested in this feature? Shall I prepare a patch?

There are a number of use cases for this, but to give you an idea,
here's one I'm using myself.

  (setq org-babel-default-header-args:latex
        `((:file . (lambda ()
                     (concat "img/"
                             (sha1 (org-element-property :value (org-element-at-point)))
                             (by-backend '((html . "-html") (t . "-org")))
                             ".svg")))))

This computes a filename based on the hash of the block contents.

Matt

Re: babel default header args as functions

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 967 bytes --]

Matt Huszagh <huszaghmatt@gmail.com> writes:

> I've added the ability in my own configuration to use lambda functions
> that evaluate to a string as babel default header arguments, instead of
> just the plain strings currently allowed. Would anyone else be
> interested in this feature? Shall I prepare a patch?
>
> There are a number of use cases for this, but to give you an idea,
> here's one I'm using myself.
>
>   (setq org-babel-default-header-args:latex
>         `((:file . (lambda ()
>                      (concat "img/"
>                              (sha1 (org-element-property :value (org-element-at-point)))
>                              (by-backend '((html . "-html") (t . "-org")))
>                              ".svg")))))
>
> This computes a filename based on the hash of the block contents.

I've generated a patch for this. Please let me know your thoughts. I
believe this adds valuable flexibility to default header
arguments.

Thanks!
Matt


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-core.el-Add-ability-to-use-closures-as-default-he.patch --]
[-- Type: text/x-patch, Size: 3058 bytes --]

From 3dfb1066b211fdcc5e3ea1da8d36aa115dde9f9b Mon Sep 17 00:00:00 2001
From: Matt Huszagh <huszaghmatt@gmail.com>
Date: Fri, 28 Aug 2020 11:05:59 -0700
Subject: [PATCH] ob-core.el: Add ability to use closures as default header
 arguments

* lisp/ob-core.el (org-babel-default-header-args): Document ability to
use functions.
(eval-default-headers): New function to generate default header
arguments, which adds the ability to evaluate function arguments at
runtime.
(org-babel-get-src-block-info): Use new header argument evaluate
function when retreiving src block info.

The closures are evaluated at runtime.
---
 lisp/ob-core.el | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 578622232..4a22f17e7 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -473,7 +473,23 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'."
 (defvar org-babel-default-header-args
   '((:session . "none") (:results . "replace") (:exports . "code")
     (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no"))
-  "Default arguments to use when evaluating a source block.")
+  "Default arguments to use when evaluating a source block.
+
+This is a list in which each element is an alist.  Each key
+corresponds to a header argument, and each value to that header's
+value.  The value can either be a string or a closure that
+evaluates to a string at runtime.  For instance, imagine you'd
+like to set the file name output of a latex source block to a
+sha1 of its contents.  We could achieve this with:
+
+(defun org-src-sha ()
+  (let ((elem (org-element-at-point)))
+    (concat (sha1 (org-element-property :value elem)) \".svg\")))
+
+(setq org-babel-default-header-args:latex
+      `((:results . \"file link replace\")
+        (:file . (lambda () (org-src-sha)))))")
+
 (put 'org-babel-default-header-args 'safe-local-variable
      (org-babel-header-args-safe-fn org-babel-safe-header-args))
 
@@ -584,6 +600,18 @@ the outer-most code block.")
 
 (defvar *this*)
 
+(defun eval-default-headers (headers)
+  "Compute default header list set with HEADERS.
+
+  Evaluate all default header arguments set to functions prior to
+  returning the list of header arguments."
+  (let ((lst nil))
+    (dolist (elem (eval headers t))
+      (if (listp (cdr elem))
+          (push `(,(car elem) . ,(funcall (cdr elem))) lst)
+        (push elem lst)))
+    lst))
+
 (defun org-babel-get-src-block-info (&optional light datum)
   "Extract information from a source block or inline source block.
 
@@ -615,7 +643,7 @@ a list with the following pattern:
 	       (apply #'org-babel-merge-params
 		      (if inline org-babel-default-inline-header-args
 			org-babel-default-header-args)
-		      (and (boundp lang-headers) (eval lang-headers t))
+		      (and (boundp lang-headers) (eval-default-headers lang-headers))
 		      (append
 		       ;; If DATUM is provided, make sure we get node
 		       ;; properties applicable to its location within
-- 
2.28.0

Re: babel default header args as functions

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 260 bytes --]

Matt Huszagh <huszaghmatt@gmail.com> writes:

> I've generated a patch for this. Please let me know your thoughts. I
> believe this adds valuable flexibility to default header
> arguments.

I've added an additional fix that makes this work during export too.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-core.el-Add-ability-to-use-closures-as-default-he.patch --]
[-- Type: text/x-patch, Size: 3714 bytes --]

From aec4e905d5d72f9a124adfde877835a783bd637b Mon Sep 17 00:00:00 2001
From: Matt Huszagh <huszaghmatt@gmail.com>
Date: Fri, 28 Aug 2020 11:05:59 -0700
Subject: [PATCH] ob-core.el: Add ability to use closures as default header
 arguments

* lisp/ob-core.el (org-babel-default-header-args): Document ability to
use functions.
(eval-default-headers): New function to generate default header
arguments, which adds the ability to evaluate function arguments at
runtime.
(org-babel-get-src-block-info): Use new header argument evaluate
function when retreiving src block info.

* lisp/ob-exp.el (org-babel-exp-src-block): Must use new
eval-default-headers when exporting as well.

The closures are evaluated at runtime.
---
 lisp/ob-core.el | 32 ++++++++++++++++++++++++++++++--
 lisp/ob-exp.el  |  2 +-
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 578622232..4a22f17e7 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -473,7 +473,23 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'."
 (defvar org-babel-default-header-args
   '((:session . "none") (:results . "replace") (:exports . "code")
     (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no"))
-  "Default arguments to use when evaluating a source block.")
+  "Default arguments to use when evaluating a source block.
+
+This is a list in which each element is an alist.  Each key
+corresponds to a header argument, and each value to that header's
+value.  The value can either be a string or a closure that
+evaluates to a string at runtime.  For instance, imagine you'd
+like to set the file name output of a latex source block to a
+sha1 of its contents.  We could achieve this with:
+
+(defun org-src-sha ()
+  (let ((elem (org-element-at-point)))
+    (concat (sha1 (org-element-property :value elem)) \".svg\")))
+
+(setq org-babel-default-header-args:latex
+      `((:results . \"file link replace\")
+        (:file . (lambda () (org-src-sha)))))")
+
 (put 'org-babel-default-header-args 'safe-local-variable
      (org-babel-header-args-safe-fn org-babel-safe-header-args))
 
@@ -584,6 +600,18 @@ the outer-most code block.")
 
 (defvar *this*)
 
+(defun eval-default-headers (headers)
+  "Compute default header list set with HEADERS.
+
+  Evaluate all default header arguments set to functions prior to
+  returning the list of header arguments."
+  (let ((lst nil))
+    (dolist (elem (eval headers t))
+      (if (listp (cdr elem))
+          (push `(,(car elem) . ,(funcall (cdr elem))) lst)
+        (push elem lst)))
+    lst))
+
 (defun org-babel-get-src-block-info (&optional light datum)
   "Extract information from a source block or inline source block.
 
@@ -615,7 +643,7 @@ a list with the following pattern:
 	       (apply #'org-babel-merge-params
 		      (if inline org-babel-default-inline-header-args
 			org-babel-default-header-args)
-		      (and (boundp lang-headers) (eval lang-headers t))
+		      (and (boundp lang-headers) (eval-default-headers lang-headers))
 		      (append
 		       ;; If DATUM is provided, make sure we get node
 		       ;; properties applicable to its location within
diff --git a/lisp/ob-exp.el b/lisp/ob-exp.el
index 34caf9546..13277f64f 100644
--- a/lisp/ob-exp.el
+++ b/lisp/ob-exp.el
@@ -103,7 +103,7 @@ Assume point is at block opening line."
 		       (apply #'org-babel-merge-params
 			      org-babel-default-header-args
 			      (and (boundp lang-headers)
-				   (symbol-value lang-headers))
+				   (eval-default-headers lang-headers))
 			      (append (org-babel-params-from-properties lang)
 				      (list raw-params)))))))
 	  (setf hash (org-babel-sha1-hash info :export)))
-- 
2.28.0

Re: babel default header args as functions

[Bastien]

Hi Matt,

Matt Huszagh <huszaghmatt@gmail.com> writes:

> Matt Huszagh <huszaghmatt@gmail.com> writes:
>
>> I've generated a patch for this. Please let me know your thoughts. I
>> believe this adds valuable flexibility to default header
>> arguments.
>
> I've added an additional fix that makes this work during export too.

I would like to hear what other think about this feature.

Anyone?

Also, if we integrate the change, `eval-default-headers' would be
better named `org-babel-eval-default-headers'.

-- 
 Bastien

Re: babel default header args as functions

[Tom Gillespie]

I think making the behavior of default arguments consistent with
regular arguments is probably a good thing. I have a number of use
cases that I can imagine would benefit greatly from being able to
define a :header-args: :header (lambda () "yay!") property as a
closure (and actually I assumed that it would just work that way if I
tried to do it, clearly not though). I can't tell for sure if the
patch enables this behavior though or whether I would still get a
Wrong type argument error. I don't see any additional security
implications beyond those already present (or not present) from
allowing closures as arguments to header arguments generally. Looking
at the patch it seems that it preserves the behavior of performing the
evaluation of the closures at the source block, but I'm not 100% sure.
If the default header closures are being evaluated before checking
whether they have been superseded by the headers on a block then that
is incorrect and they should not be evaluated until it is clear that
they are the value of the header for that block and have not been
superseded.
Best!
Tom

Re: babel default header args as functions

[Huszaghmatt]

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

      
  

  
  
  
  <Tom Gillespie (mailto:tgbugs@gmail.com)>  writes:
  
  
  
  >  If the default header closures are being evaluated before checking
  
  >  whether they have been superseded by the headers on a block then that
  
  >  is incorrect and they should not be evaluated until it is clear that
  
  >  they are the value of the header for that block and have not been
  
  >  superseded.   
>   
>   
>   
>     
>   
>     
>   
>     
>   
>     
>   
>     
>   
>     
>   
>         
  
  
 Good point, Tom. I’m back from vacation on Wednesday and can   
  
 take a detailed look at this behavior then.
  
  
  
 Matt
  
     

[-- Attachment #2: Type: text/html, Size: 2456 bytes --]

Re: babel default header args as functions

[Bastien]

Thanks for weighing in into this discussion.

Tom Gillespie <tgbugs@gmail.com> writes:

> I have a number of use
> cases that I can imagine would benefit greatly from being able to
> define a :header-args: :header (lambda () "yay!") property as a
> closure

Can you give some examples? I would love to get a better sense of
the usefulness of this feature.

-- 
 Bastien

Re: babel default header args as functions

[Tom Gillespie]

Hi Bastien,
    Here are some examples that I have in mind. One example would be
to simplify my hacked solution to from
https://lists.gnu.org/archive/html/emacs-orgmode/2020-09/msg00175.html.
I could apply that to all the headings in a subtree by defining it
once in the property drawer. Another example would be to only tangle a
block when a certain value was true, it would require using an
unofficial header convention to store the name of the file per block,
or it could use the name of the block. I do something like that around
line 1336 of https://raw.githubusercontent.com/SciCrunch/sparc-curation/9c08e82ab0b8c497e01d8542ecb1195d40111767/docs/setup.org.
Another extremely common use case I have is detecting that I am
running on a particular host which requires a slightly different value
than some other host that I run regularly on, for example if I am
going to be running a large number of blocks using :dir and on host a
it needs to be /home/my-usual-user-name/working/ and on another it has
to be /home/my-other-user-name/working/ because I wasn't the first
user named tom to get an account on that system. Similar use case
would be switching the database port based on which host I was on
since I often forward a remote database port via ssh while also having
a local database on that port as well. Another case would be to be
able to blanked switch :tangle from yes to no based on whether a
sentinel file was present or any other condition you could imagine
(this is similar to the second example). Best!
Tom


On Sat, Sep 5, 2020 at 10:00 PM Bastien <bzg@gnu.org> wrote:
>
> Thanks for weighing in into this discussion.
>
> Tom Gillespie <tgbugs@gmail.com> writes:
>
> > I have a number of use
> > cases that I can imagine would benefit greatly from being able to
> > define a :header-args: :header (lambda () "yay!") property as a
> > closure
>
> Can you give some examples? I would love to get a better sense of
> the usefulness of this feature.
>
> --
>  Bastien

Re: babel default header args as functions

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 1785 bytes --]

Tom Gillespie <tgbugs@gmail.com> writes:

> [...] I have a number of use
> cases that I can imagine would benefit greatly from being able to
> define a :header-args: :header (lambda () "yay!") property as a
> closure (and actually I assumed that it would just work that way if I
> tried to do it, clearly not though). I can't tell for sure if the
> patch enables this behavior though or whether I would still get a
> Wrong type argument error.

This should work. Do you have reason for believing it might not?

For example, set:

(setq org-babel-default-header-args:bash
      `((:var . (lambda ()
                  "a='yay'"))))

Then in file.org:
#+begin_src bash :results output
echo $a
#+end_src

Executing this will yield:

#+RESULTS:
: yay

> [...] Looking
> at the patch it seems that it preserves the behavior of performing the
> evaluation of the closures at the source block, but I'm not 100% sure.

I'm not sure I completely understand what you mean here. However, the
closures are evaluated when point is at the source block, during the
source block evaluation, not when the default headers are declared. This
allows the closures to use context-dependent functionality (e.g. you can
call `org-element-at-point' inside the closure and retrieve whatever
information you want). Does this address your concern? Please clarify if
I've missed your point.

> If the default header closures are being evaluated before checking
> whether they have been superseded by the headers on a block then that
> is incorrect and they should not be evaluated until it is clear that
> they are the value of the header for that block and have not been
> superseded.

I've fixed my patch (attached) so that now closures are only evaluated
when they are used as part of the final set of headers.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-core.el-Add-ability-to-use-closures-as-default-he.patch --]
[-- Type: text/x-patch, Size: 4619 bytes --]

From 4a461a90ec4f3c5f9634b687a6685ea3ba74f168 Mon Sep 17 00:00:00 2001
From: Matt Huszagh <huszaghmatt@gmail.com>
Date: Fri, 28 Aug 2020 11:05:59 -0700
Subject: [PATCH] ob-core.el: Add ability to use closures as default header
 arguments

* lisp/ob-core.el (org-babel-default-header-args): Document ability to
use closures.
(org-babel-eval-headers): New function to generate header arguments,
which adds the ability to evaluate closures during source block
execution or export.
(org-babel-merge-params): Only evaluate closures when we have our
final list of headers.
---
 lisp/ob-core.el | 60 ++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 55 insertions(+), 5 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 578622232..bef34d7c0 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -473,7 +473,35 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'."
 (defvar org-babel-default-header-args
   '((:session . "none") (:results . "replace") (:exports . "code")
     (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no"))
-  "Default arguments to use when evaluating a source block.")
+  "Default arguments to use when evaluating a source block.
+
+This is a list in which each element is an alist.  Each key
+corresponds to a header argument, and each value to that header's
+value.  The value can either be a string or a closure that
+evaluates to a string.  The closure is evaluated when the source
+block is being evaluated (e.g. during execution or export), with
+point at the source block.  It is not possible to use an
+arbitrary function symbol (e.g. 'some-func), since org uses
+lexical binding.  To achieve the same functionality, call the
+function within a closure (e.g. (lambda () (some-func))).
+
+To understand how closures can be used as default header
+arguments, imagine you'd like to set the file name output of a
+latex source block to a sha1 of its contents.  We could achieve
+this with:
+
+(defun org-src-sha ()
+  (let ((elem (org-element-at-point)))
+    (concat (sha1 (org-element-property :value elem)) \".svg\")))
+
+(setq org-babel-default-header-args:latex
+      `((:results . \"file link replace\")
+        (:file . (lambda () (org-src-sha)))))
+
+Because the closure is evaluated with point at the source block,
+the call to `org-element-at-point' above will always retrieve
+information about the current source block.")
+
 (put 'org-babel-default-header-args 'safe-local-variable
      (org-babel-header-args-safe-fn org-babel-safe-header-args))
 
@@ -584,6 +612,19 @@ the outer-most code block.")
 
 (defvar *this*)
 
+(defun org-babel-eval-headers (headers)
+  "Compute header list set with HEADERS.
+
+Evaluate all header arguments set to functions prior to returning
+the list of header arguments."
+  (let ((lst nil))
+    (dolist (elem headers)
+      (if (and (cdr elem)
+	       (functionp (cdr elem)))
+          (push `(,(car elem) . ,(funcall (cdr elem))) lst)
+        (push elem lst)))
+    lst))
+
 (defun org-babel-get-src-block-info (&optional light datum)
   "Extract information from a source block or inline source block.
 
@@ -2704,12 +2745,21 @@ parameters when merging lists."
 				  results-exclusive-groups
 				  results
 				  (split-string
-				   (if (stringp value) value (eval value t))))))
+				   (if (stringp value)
+				       value
+				     (if (functionp value)
+					 (funcall value)
+				       (eval value t)))))))
 	  (`(:exports . ,value)
 	   (setq exports (funcall merge
 				  exports-exclusive-groups
 				  exports
-				  (split-string (or value "")))))
+				  (split-string (or
+						 (if value
+						     (if (functionp value)
+							 (funcall value)
+						       value)
+						   ""))))))
 	  ;; Regular keywords: any value overwrites the previous one.
 	  (_ (setq params (cons pair (assq-delete-all (car pair) params)))))))
     ;; Handle `:var' and clear out colnames and rownames for replaced
@@ -2724,14 +2774,14 @@ parameters when merging lists."
 			      (cdr (assq param params))))
 	  (setq params
 		(cl-remove-if (lambda (pair) (and (equal (car pair) param)
-					     (null (cdr pair))))
+						  (null (cdr pair))))
 			      params)))))
     ;; Handle other special keywords, which accept multiple values.
     (setq params (nconc (list (cons :results (mapconcat #'identity results " "))
 			      (cons :exports (mapconcat #'identity exports " ")))
 			params))
     ;; Return merged params.
-    params))
+    (org-babel-eval-headers params)))
 
 (defun org-babel-noweb-p (params context)
   "Check if PARAMS require expansion in CONTEXT.
-- 
2.28.0

Re: babel default header args as functions

[Tom Gillespie]

Hi Matt,
   Looking good here. Thanks!
Tom

On Wed, Sep 9, 2020 at 12:06 PM Matt Huszagh <huszaghmatt@gmail.com> wrote:
>
> Tom Gillespie <tgbugs@gmail.com> writes:
>
> > [...] I have a number of use
> > cases that I can imagine would benefit greatly from being able to
> > define a :header-args: :header (lambda () "yay!") property as a
> > closure (and actually I assumed that it would just work that way if I
> > tried to do it, clearly not though). I can't tell for sure if the
> > patch enables this behavior though or whether I would still get a
> > Wrong type argument error.
>
> This should work. Do you have reason for believing it might not?

With the patch applied this is working on my end.

* test header
:PROPERTIES:
:header-args:bash: :tangle (lambda () "./from-header.sh")
:END:

#+begin_src bash :shebang "#!/usr/bin/env bash"
echo yes
#+end_src

> > [...] Looking
> > at the patch it seems that it preserves the behavior of performing the
> > evaluation of the closures at the source block, but I'm not 100% sure.
>
> I'm not sure I completely understand what you mean here. However, the
> closures are evaluated when point is at the source block, during the
> source block evaluation, not when the default headers are declared. This
> allows the closures to use context-dependent functionality (e.g. you can
> call `org-element-at-point' inside the closure and retrieve whatever
> information you want). Does this address your concern? Please clarify if
> I've missed your point.

Yep, you've got it.

> > If the default header closures are being evaluated before checking
> > whether they have been superseded by the headers on a block then that
> > is incorrect and they should not be evaluated until it is clear that
> > they are the value of the header for that block and have not been
> > superseded.
>
> I've fixed my patch (attached) so that now closures are only evaluated
> when they are used as part of the final set of headers.

Great.

Re: babel default header args as functions

[rey-coyrehourcq]

[-- Attachment #1: Type: text/plain, Size: 2365 bytes --]

Hi there,

I'm interested by this functionality, do you know if it was merged or i need to apply patch locally ?

Thanks Matt for your work,

Best 
SR

Le mercredi 09 septembre 2020 à 12:33 -0700, Tom Gillespie a écrit :
> Hi Matt,
>    Looking good here. Thanks!
> Tom
> 
> On Wed, Sep 9, 2020 at 12:06 PM Matt Huszagh <huszaghmatt@gmail.com> wrote:
> > Tom Gillespie <tgbugs@gmail.com> writes:
> > 
> > > [...] I have a number of use
> > > cases that I can imagine would benefit greatly from being able to
> > > define a :header-args: :header (lambda () "yay!") property as a
> > > closure (and actually I assumed that it would just work that way if I
> > > tried to do it, clearly not though). I can't tell for sure if the
> > > patch enables this behavior though or whether I would still get a
> > > Wrong type argument error.
> > 
> > This should work. Do you have reason for believing it might not?
> 
> With the patch applied this is working on my end.
> 
> * test header
> :PROPERTIES:
> :header-args:bash: :tangle (lambda () "./from-header.sh")
> :END:
> 
> #+begin_src bash :shebang "#!/usr/bin/env bash"
> echo yes
> #+end_src
> 
> > > [...] Looking
> > > at the patch it seems that it preserves the behavior of performing the
> > > evaluation of the closures at the source block, but I'm not 100% sure.
> > 
> > I'm not sure I completely understand what you mean here. However, the
> > closures are evaluated when point is at the source block, during the
> > source block evaluation, not when the default headers are declared. This
> > allows the closures to use context-dependent functionality (e.g. you can
> > call `org-element-at-point' inside the closure and retrieve whatever
> > information you want). Does this address your concern? Please clarify if
> > I've missed your point.
> 
> Yep, you've got it.
> 
> > > If the default header closures are being evaluated before checking
> > > whether they have been superseded by the headers on a block then that
> > > is incorrect and they should not be evaluated until it is clear that
> > > they are the value of the header for that block and have not been
> > > superseded.
> > 
> > I've fixed my patch (attached) so that now closures are only evaluated
> > when they are used as part of the final set of headers.
> 
> Great.
> 



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: babel default header args as functions

[Matt Huszagh]

rey-coyrehourcq <sebastien.rey-coyrehourcq@univ-rouen.fr> writes:

> I'm interested by this functionality, do you know if it was merged or i need to apply patch locally ?

Hi SR,

This hasn't been merged yet. I believe it's ready, but we're just
waiting on a maintainer to apply it upstream. If you're able to apply
the patch locally and provide feedback that always helps.

Matt

Re: babel default header args as functions

[rey-coyrehourcq]

[-- Attachment #1.1: Type: text/plain, Size: 1242 bytes --]

Hi Matt,

I'm unfamilliar with patch by mail but i try to apply your patch to my melpa local org 9.4 version used by doom emacs.
Patch hang on Hunk #3, i attach the .rej file.

patch -p1 < 0001-ob-core.el-Add-ability-to-use-closures-as-default-he.patch

File to patch: ob-core.el
patching file ob-core.el
Hunk #1 succeeded at 467 (offset -6 lines).
Hunk #2 succeeded at 596 (offset -16 lines).
Hunk #3 FAILED at 2745.
Hunk #4 succeeded at 2714 (offset -51 lines).
1 out of 4 hunks FAILED -- saving rejects to file ob-core.el.rej
patch unexpectedly ends in middle of line

Best

Le mercredi 14 octobre 2020 à 07:16 -0700, Matt Huszagh a écrit :
> rey-coyrehourcq <sebastien.rey-coyrehourcq@univ-rouen.fr> writes:
> 
> > I'm interested by this functionality, do you know if it was merged or i need to apply patch locally ?
> 
> Hi SR,
> 
> This hasn't been merged yet. I believe it's ready, but we're just
> waiting on a maintainer to apply it upstream. If you're able to apply
> the patch locally and provide feedback that always helps.
> 
> Matt
-- 

Sébastien Rey-Coyrehourcq
Research Engineer UMR IDEES
02.35.14.69.30

{Stronger security for your email, follow EFF tutorial : https://ssd.eff.org/}


[-- Attachment #1.2: ob-core.el.rej --]
[-- Type: text/x-reject, Size: 848 bytes --]

--- lisp/ob-core.el
+++ lisp/ob-core.el
@@ -2745,12 +2786,21 @@ parameters when merging lists."
 				  results-exclusive-groups
 				  results
 				  (split-string
-				   (if (stringp value) value (eval value t))))))
+				   (if (stringp value)
+				       value
+				     (if (functionp value)
+					 (funcall value)
+				       (eval value t)))))))
 	  (`(:exports . ,value)
 	   (setq exports (funcall merge
 				  exports-exclusive-groups
 				  exports
-				  (split-string (or value "")))))
+				  (split-string (or
+						 (if value
+						     (if (functionp value)
+							 (funcall value)
+						       value)
+						   ""))))))
 	  ;; Regular keywords: any value overwrites the previous one.
 	  (_ (setq params (cons pair (assq-delete-all (car pair) params)))))))
     ;; Handle `:var' and clear out colnames and rownames for replaced

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: babel default header args as functions

[Matt Huszagh]

rey-coyrehourcq <sebastien.rey-coyrehourcq@univ-rouen.fr> writes:

> I'm unfamilliar with patch by mail but i try to apply your patch to my melpa local org 9.4 version used by doom emacs.
> Patch hang on Hunk #3, i attach the .rej file.

You might need to start from a different org commit. I believe
6a182b690f works. Otherwise, you could fix the merge conflicts (this is
described online, e.g., https://stackoverflow.com/a/16968982/5710525).

Finally, a very crude solution would be to manually apply the changes
yourself if you're not too conversant with git. The git solutions are
better, but since the changes in this patch are minimal it wouldn't take
too long.

I will at some point fix the conflicts myself and send a new patch, but
I'd prefer to wait until it gets the green light to merge so that I
don't have to do it multiple times.

Matt

Re: babel default header args as functions

[stardiviner]

Bastien <bzg@gnu.org> writes:

> Hi Matt,
>
> Matt Huszagh <huszaghmatt@gmail.com> writes:
>
>> Matt Huszagh <huszaghmatt@gmail.com> writes:
>>
>>> I've generated a patch for this. Please let me know your thoughts. I
>>> believe this adds valuable flexibility to default header
>>> arguments.
>>
>> I've added an additional fix that makes this work during export too.
>
> I would like to hear what other think about this feature.
>
> Anyone?
>
> Also, if we integrate the change, `eval-default-headers' would be
> better named `org-babel-eval-default-headers'.

I like this new feature propose too. It can be used in many situations. Really helpful.

-- 
[ stardiviner ]
       I try to make every word tell the meaning that I want to express.

       Blog: https://stardiviner.github.io/
       IRC(freenode): stardiviner, Matrix: stardiviner
       GPG: F09F650D7D674819892591401B5DF1C95AE89AC3

Re: babel default header args as functions

[Matt Huszagh]

Bastien <bzg@gnu.org> writes:

> Also, if we integrate the change, `eval-default-headers' would be
> better named `org-babel-eval-default-headers'.

I've changed the function name to `org-babel-eval-headers'. The reason
for dropping "default" is that this function is now used as part of
`org-babel-merge-params' on all headers, not just defaults in order to
avoid evaluating closures until we know they are needed. See
https://lists.gnu.org/archive/html/emacs-orgmode/2020-09/msg00464.html
for the new patch and more info.

Matt

Re: babel default header args as functions

[Matt Huszagh]

Bastien,

Any chance you've had the time to look at this? I know how much time you
put into org-mode, so no rush if you need more time! Let me know if
there's anything I can do on my end.

Thanks!
Matt

Re: babel default header args as functions

[Bastien]

Hi Matt,

Matt Huszagh <huszaghmatt@gmail.com> writes:

> Matt Huszagh <huszaghmatt@gmail.com> writes:
>
>> I've generated a patch for this. Please let me know your thoughts. I
>> believe this adds valuable flexibility to default header
>> arguments.
>
> I've added an additional fix that makes this work during export too.

Applied, thanks, and sorry again for the loooong delay.

It deserves an entry in etc/ORG-NEWS for Org 9.5, would you be willing
to submit a patch for this?

Thanks,

-- 
 Bastien

Re: babel default header args as functions

[Matt Huszagh]

Bastien <bzg@gnu.org> writes:

> Applied, thanks, and sorry again for the loooong delay.

Thanks Bastien, and no worries about the delay. However, I hate to say
it but I think you may have applied an old patch. The most recent patch
is

https://lists.gnu.org/archive/html/emacs-orgmode/2020-09/txtzi_PffIaG1.txt

Let me know what I can do.

> It deserves an entry in etc/ORG-NEWS for Org 9.5, would you be willing
> to submit a patch for this?

Yep, I'll send one over.

Matt

Re: babel default header args as functions

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 771 bytes --]

Matt Huszagh <huszaghmatt@gmail.com> writes:

> Thanks Bastien, and no worries about the delay. However, I hate to say
> it but I think you may have applied an old patch. The most recent patch
> is
>
> https://lists.gnu.org/archive/html/emacs-orgmode/2020-09/txtzi_PffIaG1.txt
>
> Let me know what I can do.

I've tested it, and if you revert
78783f4e47901255695031dae0efcbb301a40878 and apply the new patch, it
will apply with conflicts. Let me know if you run into any difficulties,
have any concerns, etc.

>> It deserves an entry in etc/ORG-NEWS for Org 9.5, would you be willing
>> to submit a patch for this?
>
> Yep, I'll send one over.

Here's the patch for the news item. Bear in mind that the last part
about lazy evaluation is only true for the newest patch.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: ORG-NEWS patch --]
[-- Type: text/x-patch, Size: 2167 bytes --]

From ae721d089854e3b6b2d71ab6829b0cace25f0968 Mon Sep 17 00:00:00 2001
From: Matt Huszagh <huszaghmatt@gmail.com>
Date: Tue, 28 Sep 2021 18:26:04 -0700
Subject: [PATCH] etc/ORG-NEWS: Add news item about
 org-babel-default-header-args accepting closures

---
 etc/ORG-NEWS | 29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index b0a893946..db6df83f2 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -151,7 +151,7 @@ enable Calc units simplification mode.
 
 *** Support fontification of inline export snippets
 
-Inline 
+Inline
 
 See [[msg:87im57fh8j.fsf@gmail.com][this thread]].
 
@@ -443,6 +443,33 @@ See [[msg:875z8njaol.fsf@protesilaos.com][this thread]].
 attachment directory at calls of ~org-attach-sync~.  There is
 Never delete, Always delete and Query the user (default).
 
+*** ~org-babel-default-header-args~ can now be specified as closures or strings
+
+~org-babel-default-header-args~ now also accepts closures that
+evaluate to a string. Previously, only direct strings were
+supported. These closures are evaluated when point is at the source
+block, which allows them to make use of contextual information at the
+relevant source block. One example that illustrates the usefulness of
+this addition (also given in the documentation for
+~org-babel-default-header-args~) is:
+
+#+begin_src elisp
+(defun org-src-sha ()
+  (let ((elem (org-element-at-point)))
+    (concat (sha1 (org-element-property :value elem)) \".svg\")))
+
+(setq org-babel-default-header-args:latex
+      `((:results . \"file link replace\")
+        (:file . (lambda () (org-src-sha)))))
+#+end_src
+
+This will set the ~:file~ header argument to the sha1 checksum of the
+contents of the current latex source block.
+
+Finally, the closures are only evaluated if they're not overridden for
+a source block. This improves efficiency in cases where the result of
+a compute-expensive closure would otherwise be discarded.
+
 ** Miscellaneous
 *** =org-bibtex= includes =doi= and =url= entries when exporting to BiBTeX
 =doi= and =url= entries have been made optional for some publication
-- 
2.31.1


[-- Attachment #3: Type: text/plain, Size: 6 bytes --]


Matt

Re: babel default header args as functions

[Matt Huszagh]

Matt Huszagh <huszaghmatt@gmail.com> writes:

> I've tested it, and if you revert
> 78783f4e47901255695031dae0efcbb301a40878 and apply the new patch, it
> will apply with conflicts. Let me know if you run into any difficulties,
> have any concerns, etc.

That "with" should be "without"...

Matt

Re: babel default header args as functions

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 350 bytes --]

Hi  Matt,

> Here’s the patch for the news item. Bear in mind that the last part
> about lazy evaluation is only true for the newest patch.

I just read your docs patch, and that’s lead to a quick question: does this patch
support a header arg function in the form “(:file . #’my-org-file-name-generator)”?

All the best,
Timothy

Re: babel default header args as functions

[Matt Huszagh]

Timothy <tecosaur@gmail.com> writes:

> I just read your docs patch, and that’s lead to a quick question: does this patch
> support a header arg function in the form “(:file . #’my-org-file-name-generator)”?

Unfortunately it doesn't. It's been about a year since I wrote this
patch, so I'm a bit hazy on some of the details, but IIRC it has to deal
with org-mode's use of lexical binding. If you're interested in digging
deeper (or maybe even getting it to work, which would be great) I'd look
at `org-babel-eval-headers'. The `functionp' line in there returns nil
and the function isn't evaluated unless it's a closure.

Matt

Re: babel default header args as functions

[Bastien]

Hi Matt,

Matt Huszagh <huszaghmatt@gmail.com> writes:

> I've tested it, and if you revert
> 78783f4e47901255695031dae0efcbb301a40878 and apply the new patch, it
> will apply with conflicts. Let me know if you run into any difficulties,
> have any concerns, etc.

Done, thanks a lot.

> Here's the patch for the news item. Bear in mind that the last part
> about lazy evaluation is only true for the newest patch.

Applied too, thanks!

-- 
 Bastien