[PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

[PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]

Hi,

The attached patch adds support for two new babel header arguments:
=:noweb-prefix= and =:noweb-trans=.

=:noweb-prefix= can be set to =no= to disable the noweb prefix
behaviour, where prefix characters are repeated when expanding a
multiline noweb reference.

=:noweb-trans= can be set to =prin1-to-string= to insert a lisp string
representing the content of the referenced src block.

The goal is to allow one to use, say, a LaTeX src block to represent
some LaTeX snippet to be tangled into a string in some lisp (or other)
code. This isn't possible currently, and one has to manually string
escape the LaTeX code.

As an example, the following two blocks

#+BEGIN_SRC LaTeX :tangle no :noweb-ref nw
\usepackage{…}
\usepackage{…}
#+END_SRC

#+BEGIN_SRC emacs-lisp :noweb yes :tangle yes :noweb-prefix no 
:noweb-trans prin1-to-string
(setq latex-header <<nw>>)
#+END_SRC

would tangle to

#+BEGIN_SRC emacs-lisp
(setq latex-header "\\usepackage{…}
\\usepackage{…}")
#+END_SRC

I've left undocumented the possibility of setting =:noweb-trans= to
another function. I wonder if anyone can think of some other use.

Regards,

-- 
Sébastien Miquel

[-- Attachment #2: 0001-ob-core.el-Add-noweb-prefix-noweb-trans-babel-header.patch --]
[-- Type: text/x-patch, Size: 5996 bytes --]

From 66f271225767d07e12bcc73a1ddbadf038d245fa Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Mon, 6 Sep 2021 18:45:42 +0200
Subject: [PATCH] ob-core.el: Add `:noweb-prefix`, `:noweb-trans` babel header
 arguments

* lisp/ob-core.el (org-babel-expand-noweb-references): Add support for
`noweb-prefix' header argument, to not repeat the prefix characters
when expanding a noweb reference. Add support for `noweb-trans' header
argument, to apply a function to the noweb content upon
expansion.
(org-babel-common-header-args-w-values):
(org-babel-safe-header-args): Add `noweb-prefix' and `noweb-trans' values.
* doc/org-manual.org: Document `noweb-prefix' and `noweb-trans' babel header
arguments.
* etc/NEWS: Document `:noweb-prefix' and `:noweb-trans'.
---
 doc/org-manual.org | 34 ++++++++++++++++++++++++++++++----
 etc/ORG-NEWS       |  9 +++++++++
 lisp/ob-core.el    | 26 ++++++++++++++++++++------
 3 files changed, 59 insertions(+), 10 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index b4c20f252..d7b1c4203 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -18554,10 +18554,11 @@ Note that the expansion now contains the results of the code block
 : 100
 
 Noweb insertions honor prefix characters that appear before the noweb
-syntax reference.  This behavior is illustrated in the following
-example.  Because the =<<example>>= noweb reference appears behind the
-SQL comment syntax, each line of the expanded noweb reference is
-commented.  With:
+syntax reference. This behavior can be turned off by setting the
+=noweb-prefix= header argument to =no= and is illustrated in the
+following example. Because the =<<example>>= noweb reference appears
+behind the SQL comment syntax, each line of the expanded noweb
+reference is commented. With:
 
 #+begin_example
 ,#+NAME: example
@@ -18626,6 +18627,31 @@ else:
     print('do things when false')
 #+end_example
 
+The header argument =noweb-trans= can be set to =prin1-to-string= to
+insert a lisp string representing the content of the referenced src
+block. With:
+
+#+begin_example
+,#+NAME: latex-header
+,#+BEGIN_SRC latex
+  \usepackage{amsmath}
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+this code block:
+
+#+begin_example
+,#+BEGIN_SRC elisp :noweb yes
+  (setq header <<latex-header>>)
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+expands to:
+
+: (setq header "\\usepackage{amsmath}")
+
 When in doubt about the outcome of a source code block expansion, you
 can preview the results with the following command:
 
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 335db4139..6fa808645 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -63,6 +63,15 @@ list of various table options (between brackets in LaTeX export),
 since certain tabular environments, such as longtblr of the
 tabularray LaTeX package, provides this structure.
 
+*** New =:noweb-prefix= and =:noweb-trans= babel header arguments
+
+=:noweb-prefix= can be set to =no= to prevent the prefix characters
+from being repeated when expanding a multiline noweb reference.
+
+=:noweb-trans= can be set to =prin1-to-string=. Noweb reference
+therein will be expanded to an elisp string representation of their
+content.
+
 ** New functions and changes in function arguments
 
 *** New function ~org-element-cache-map~ for quick mapping across Org elements
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 239a57f96..1d5d1bedc 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -411,6 +411,8 @@ then run `org-babel-switch-to-session'."
     (noweb	. ((yes no tangle no-export strip-export)))
     (noweb-ref	. :any)
     (noweb-sep  . :any)
+    (noweb-prefix . ((no yes)))
+    (noweb-trans  . ((prin1-to-string)))
     (output-dir . :any)
     (padline	. ((yes no)))
     (post       . :any)
@@ -436,9 +438,10 @@ specific header arguments as well.")
 
 (defconst org-babel-safe-header-args
   '(:cache :colnames :comments :exports :epilogue :hlines :noeval
-	   :noweb :noweb-ref :noweb-sep :padline :prologue :rownames
-	   :sep :session :tangle :wrap
+	   :noweb :noweb-ref :noweb-sep :noweb-prefix :padline
+           :prologue :rownames :sep :session :tangle :wrap
 	   (:eval . ("never" "query"))
+           (:noweb-trans . ("prin1-to-string"))
 	   (:results . (lambda (str) (not (string-match "file" str)))))
   "A list of safe header arguments for babel source blocks.
 
@@ -2825,6 +2828,12 @@ block but are passed literally to the \"example-block\"."
          (lang (nth 0 info))
          (body (nth 1 info))
 	 (comment (string= "noweb" (cdr (assq :comments (nth 2 info)))))
+	 (noweb-trans (when (cdr (assq :noweb-trans (nth 2 info)))
+                        (intern (cdr (assq :noweb-trans (nth 2 info))))))
+         (noweb-prefix (let ((v (assq :noweb-prefix (nth 2 info))))
+                         (or (not v)
+                             (and (org-not-nil (cdr v))
+                                  (not (equal (cdr v) "no"))))))
 	 (noweb-re (format "\\(.*?\\)\\(%s\\)"
 			   (with-current-buffer parent-buffer
 			     (org-babel-noweb-wrap))))
@@ -2919,11 +2928,16 @@ block but are passed literally to the \"example-block\"."
 			  (let* ((info (org-babel-get-src-block-info t))
 				 (ref (cdr (assq :noweb-ref (nth 2 info)))))
 			    (push info (gethash ref cache))))))
-		     (funcall expand-references id cache)))))
+		     (funcall expand-references id cache))))
+                  (expansion (if (functionp noweb-trans)
+                                 (funcall noweb-trans expansion)
+                               expansion)))
 	     ;; Interpose PREFIX between every line.
-	     (mapconcat #'identity
-			(split-string expansion "[\n\r]")
-			(concat "\n" prefix))))))
+	     (if noweb-prefix
+                 (mapconcat #'identity
+			    (split-string expansion "[\n\r]")
+			    (concat "\n" prefix))
+               expansion)))))
      body t t 2)))
 
 (defun org-babel--script-escape-inner (str)
-- 
2.34.1

Re: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> The attached patch adds support for two new babel header arguments:
> =:noweb-prefix= and =:noweb-trans=.
>
> =:noweb-prefix= can be set to =no= to disable the noweb prefix
> behaviour, where prefix characters are repeated when expanding a
> multiline noweb reference.

Thanks for the patch! The idea about :noweb-prefix looks useful.

> =:noweb-trans= can be set to =prin1-to-string= to insert a lisp string
> representing the content of the referenced src block.
>
> The goal is to allow one to use, say, a LaTeX src block to represent
> some LaTeX snippet to be tangled into a string in some lisp (or other)
> code. This isn't possible currently, and one has to manually string
> escape the LaTeX code.
> ...
> I've left undocumented the possibility of setting =:noweb-trans= to
> another function. I wonder if anyone can think of some other use.

prin1-to-string is too specific and only solves a single use-case.

Consider the following example:

#+BEGIN_SRC emacs-lisp :noweb yes :tangle yes :noweb-prefix no :noweb-trans prin1-to-string
<<preamble>>
(setq latex-header <<nw>>)
#+END_SRC

There are two noweb references here. Setting source block-wide
:noweb-trans is not helpful because the first reference will be
incorrectly filtered through prin1-to-string.

I'd rather introduce a new syntax to transform the noweb reference
inline. Something like

#+BEGIN_SRC emacs-lisp :noweb yes :tangle yes :noweb-prefix no
<<preamble>>
(setq latex-header <<(prin1-to-string nw)>>)
#+END_SRC

>  Noweb insertions honor prefix characters that appear before the noweb
> -syntax reference.  This behavior is illustrated in the following
> -example.  Because the =<<example>>= noweb reference appears behind the
> -SQL comment syntax, each line of the expanded noweb reference is
> -commented.  With:
> +syntax reference. This behavior can be turned off by setting the
> +=noweb-prefix= header argument to =no= and is illustrated in the
> +following example. Because the =<<example>>= noweb reference appears
> +behind the SQL comment syntax, each line of the expanded noweb
> +reference is commented. With:

This sounds a bit confusing. I would also add an example where it is
useful to set :noweb-prefix to no.
  
Best,
Ihor

Re: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

[Sébastien Miquel]

[-- Attachment #1.1: Type: text/plain, Size: 1788 bytes --]

Hi,

Ihor Radchenko writes:
> prin1-to-string is too specific and only solves a single use-case.
prin1-to-string is actually universal in a way, since any other
manipulation can then be achieved with

: (setq var (do-something <<nw>>))

at least as long as you're tangling to a programming language, that
can read lisp strings.
> Consider the following example:
>
> #+BEGIN_SRC emacs-lisp :noweb yes :tangle yes :noweb-prefix no :noweb-trans prin1-to-string
> <<preamble>>
> (setq latex-header <<nw>>)
> #+END_SRC
>
> There are two noweb references here. Setting source block-wide
> :noweb-trans is not helpful because the first reference will be
> incorrectly filtered through prin1-to-string.
Indeed. Originally I had thought of adding a new syntax <<"nw">> to
insert a string representation. I've attached a new patch, that does
this instead of introducing :noweb-trans. Now that I think of the
universality of prin1-to-string, I actually like it slightly better
than :noweb-trans. It would break existing "nw"-like noweb references.

Of course, one can work around this easily enough by using two blocks.
> I'd rather introduce a new syntax to transform the noweb reference
> inline. Something like
>
> #+BEGIN_SRC emacs-lisp :noweb yes :tangle yes :noweb-prefix no
> <<preamble>>
> (setq latex-header <<(prin1-to-string nw)>>)
> #+END_SRC
You'd need to only allow a single function call with only one
argument, or use something like <<(prin1-to-string <<nw>>)>>. The
change would be much more complex than what I propose, for maybe
little benefit.
>> [...]
> This sounds a bit confusing. I would also add an example where it is
> useful to set :noweb-prefix to no.

I've added such an example in the revised patch attached.

Thanks for the feedback.

Regards,

-- 
Sébastien Miquel

[-- Attachment #1.2: Type: text/html, Size: 3394 bytes --]

[-- Attachment #2: 0001-ob-core.el-Add-noweb-prefix-noweb-trans-babel-header.patch --]
[-- Type: text/x-patch, Size: 6090 bytes --]

From 99d043b9d837a2658e60fb4b4913454d9566519b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Mon, 6 Sep 2021 18:45:42 +0200
Subject: [PATCH] ob-core.el: Add `:noweb-prefix`, `:noweb-trans` babel header
 arguments

* lisp/ob-core.el (org-babel-expand-noweb-references): Add support for
`noweb-prefix' header argument, to not repeat the prefix characters
when expanding a noweb reference. Add support for `noweb-trans' header
argument, to apply a function to the noweb content upon
expansion.
(org-babel-common-header-args-w-values):
(org-babel-safe-header-args): Add `noweb-prefix' and `noweb-trans' values.
* doc/org-manual.org: Document `noweb-prefix' and `noweb-trans' babel header
arguments.
* etc/ORG-NEWS: Document `:noweb-prefix' and `:noweb-trans'.
---
 doc/org-manual.org | 42 ++++++++++++++++++++++++++++++++++++++++++
 etc/ORG-NEWS       | 10 +++++++++-
 lisp/ob-core.el    | 26 ++++++++++++++++++++------
 3 files changed, 71 insertions(+), 7 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 6768ca98d..5ef8e2f8b 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -18760,6 +18760,48 @@ else:
     print('do things when false')
 #+end_example
 
+This prefix behavior can be turned off in a block by setting the
+=noweb-prefix= header argument to =no=, as in:
+
+#+begin_example
+,#+BEGIN_SRC elisp :noweb-prefix no
+  (setq example-data "<<example>>")
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+which expands to:
+
+#+begin_example
+(setq example-data "this is the
+multi-line body of example")
+#+end_example
+
+The header argument =noweb-trans= can be set to =prin1-to-string= to
+insert a lisp string representing the content of the referenced src
+block. With:
+
+#+begin_example
+,#+NAME: latex-header
+,#+BEGIN_SRC latex
+  \usepackage{amsmath}
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+this code block:
+
+#+begin_example
+,#+BEGIN_SRC elisp :noweb yes :noweb-trans prin1-to-string
+  (setq header <<latex-header>>)
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+expands to:
+
+: (setq header "\\usepackage{amsmath}")
+
 When in doubt about the outcome of a source code block expansion, you
 can preview the results with the following command:
 
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 2b539d305..70f7606db 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -150,7 +150,7 @@ The entry points are ~org-persist-register~, ~org-persist-unregister~,
 ~org-persist-read~, and ~org-persist-read-all~.  Storing circular
 structures is supported.  Storing references between different
 variables is also supported (see =:inherit= key in
-~org-persist-register~).  
+~org-persist-register~).
 
 The library permits storing buffer-local variables.  Such variables
 are linked to the buffer text, file =inode=, and file path.
@@ -175,6 +175,14 @@ the =compact-itemx= export option, or globally using the
 Items in a description list that begin with =Function:=, =Variable:=
 or certain related prefixes are converted using Texinfo definition
 commands.
+*** New =:noweb-prefix= and =:noweb-trans= babel header arguments
+
+=:noweb-prefix= can be set to =no= to prevent the prefix characters
+from being repeated when expanding a multiline noweb reference.
+
+=:noweb-trans= can be set to =prin1-to-string=. Noweb reference
+therein will be expanded to an elisp string representation of their
+content.
 
 ** New functions and changes in function arguments
 
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 6590eeee7..b5fb68661 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -413,6 +413,8 @@ then run `org-babel-switch-to-session'."
     (noweb	. ((yes no tangle no-export strip-export)))
     (noweb-ref	. :any)
     (noweb-sep  . :any)
+    (noweb-prefix . ((no yes)))
+    (noweb-trans  . ((prin1-to-string)))
     (output-dir . :any)
     (padline	. ((yes no)))
     (post       . :any)
@@ -438,9 +440,10 @@ specific header arguments as well.")
 
 (defconst org-babel-safe-header-args
   '(:cache :colnames :comments :exports :epilogue :hlines :noeval
-	   :noweb :noweb-ref :noweb-sep :padline :prologue :rownames
-	   :sep :session :tangle :wrap
+	   :noweb :noweb-ref :noweb-sep :noweb-prefix :padline
+           :prologue :rownames :sep :session :tangle :wrap
 	   (:eval . ("never" "query"))
+           (:noweb-trans . ("prin1-to-string"))
 	   (:results . (lambda (str) (not (string-match "file" str)))))
   "A list of safe header arguments for babel source blocks.
 
@@ -2827,6 +2830,12 @@ block but are passed literally to the \"example-block\"."
          (lang (nth 0 info))
          (body (nth 1 info))
 	 (comment (string= "noweb" (cdr (assq :comments (nth 2 info)))))
+	 (noweb-trans (when (cdr (assq :noweb-trans (nth 2 info)))
+                        (intern (cdr (assq :noweb-trans (nth 2 info))))))
+         (noweb-prefix (let ((v (assq :noweb-prefix (nth 2 info))))
+                         (or (not v)
+                             (and (org-not-nil (cdr v))
+                                  (not (equal (cdr v) "no"))))))
 	 (noweb-re (format "\\(.*?\\)\\(%s\\)"
 			   (with-current-buffer parent-buffer
 			     (org-babel-noweb-wrap))))
@@ -2921,11 +2930,16 @@ block but are passed literally to the \"example-block\"."
 			  (let* ((info (org-babel-get-src-block-info t))
 				 (ref (cdr (assq :noweb-ref (nth 2 info)))))
 			    (push info (gethash ref cache))))))
-		     (funcall expand-references id cache)))))
+		     (funcall expand-references id cache))))
+                  (expansion (if (functionp noweb-trans)
+                                 (funcall noweb-trans expansion)
+                               expansion)))
 	     ;; Interpose PREFIX between every line.
-	     (mapconcat #'identity
-			(split-string expansion "[\n\r]")
-			(concat "\n" prefix))))))
+	     (if noweb-prefix
+                 (mapconcat #'identity
+			    (split-string expansion "[\n\r]")
+			    (concat "\n" prefix))
+               expansion)))))
      body t t 2)))
 
 (defun org-babel--script-escape-inner (str)
-- 
2.36.0


[-- Attachment #3: 0001-ob-core.el-noweb-prefix-header-argument-and-nw-synta.patch --]
[-- Type: text/x-patch, Size: 7143 bytes --]

From 8fdd163b3f5c8049c67a6f3eab4587cac4a6d04a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Mon, 6 Sep 2021 18:45:42 +0200
Subject: [PATCH] ob-core.el: `:noweb-prefix` header argument and <<"nw">>
 syntax

* lisp/ob-core.el (org-babel-expand-noweb-references): Add support for
`noweb-prefix' header argument, to not repeat the prefix characters
when expanding a noweb reference. Add support for <<"nw">> noweb
syntax, to insert a lisp string representation of the content.
(org-babel-common-header-args-w-values):
(org-babel-safe-header-args): Add `noweb-prefix' values.
(org-babel-noweb-wrap): Add support for <<"nw">> syntax.
* doc/org-manual.org: Document `noweb-prefix' and <<"nw">> noweb
syntax.
* etc/ORG-NEWS: Document `:noweb-prefix' and <<"nw">> noweb syntax.
---
 doc/org-manual.org | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 etc/ORG-NEWS       | 25 ++++++++++++++++++++++++-
 lisp/ob-core.el    | 34 +++++++++++++++++++++++-----------
 3 files changed, 93 insertions(+), 12 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 6768ca98d..2e1435279 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -18760,6 +18760,52 @@ else:
     print('do things when false')
 #+end_example
 
+This prefix behavior can be turned off in a block by setting the
+=noweb-prefix= header argument to =no=, as in:
+
+#+begin_example
+,#+BEGIN_SRC elisp :noweb-prefix no
+  (setq example-data "<<example>>")
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+which expands to:
+
+#+begin_example
+(setq example-data "this is the
+multi-line body of example")
+#+end_example
+
+Instead of inserting the content of a block, one may insert a lisp
+string representation of the content using the syntax
+
+: <<"CODE-BLOCK-ID">>
+
+#+texinfo: @noindent
+With:
+
+#+begin_example
+,#+NAME: latex-header
+,#+BEGIN_SRC latex
+  \usepackage{amsmath}
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+this code block:
+
+#+begin_example
+,#+BEGIN_SRC elisp :noweb yes
+  (setq header <<"latex-header">>)
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+expands to:
+
+: (setq header "\\usepackage{amsmath}")
+
 When in doubt about the outcome of a source code block expansion, you
 can preview the results with the following command:
 
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 2b539d305..3a8229d73 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -150,7 +150,7 @@ The entry points are ~org-persist-register~, ~org-persist-unregister~,
 ~org-persist-read~, and ~org-persist-read-all~.  Storing circular
 structures is supported.  Storing references between different
 variables is also supported (see =:inherit= key in
-~org-persist-register~).  
+~org-persist-register~).
 
 The library permits storing buffer-local variables.  Such variables
 are linked to the buffer text, file =inode=, and file path.
@@ -176,6 +176,29 @@ Items in a description list that begin with =Function:=, =Variable:=
 or certain related prefixes are converted using Texinfo definition
 commands.
 
+*** New =:noweb-prefix= babel header argument
+
+=:noweb-prefix= can be set to =no= to prevent the prefix characters
+from being repeated when expanding a multiline noweb reference.
+
+*** New syntax to insert a string representation of a noweb reference
+
+Use =<<"foo">>= to insert a lisp string representing the content of
+the =foo= block. For example:
+
+#+BEGIN_SRC org
+  ,#+BEGIN_SRC LaTeX :noweb-ref foo
+    \usepackage{bar}
+  ,#+END_SRC
+
+  ,#+BEGIN_SRC emacs-lisp :noweb yes
+    (setq latex-header <<"foo">>)
+  ,#+END_SRC
+#+END_SRC
+
+will expand to
+: (setq latex-header "\\usepackage{bar}")
+
 ** New functions and changes in function arguments
 
 *** New function ~org-element-cache-map~ for quick mapping across Org elements
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 6590eeee7..cac1a4162 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -187,7 +187,9 @@ Match any reference, or only those matching REGEXP, if non-nil.
 
 When matching, reference is stored in match group 1."
   (concat (regexp-quote org-babel-noweb-wrap-start)
-	  (or regexp "\\([^ \t\n]\\(?:.*?[^ \t\n]\\)?\\)")
+          "\\(?2:\"?\\)"
+	  (or regexp "\\(?1:[^ \t\n]\\(?:.*?[^ \t\n]\\)?\\)")
+          "\\2"
 	  (regexp-quote org-babel-noweb-wrap-end)))
 
 (defvar org-babel-src-name-regexp
@@ -413,6 +415,7 @@ then run `org-babel-switch-to-session'."
     (noweb	. ((yes no tangle no-export strip-export)))
     (noweb-ref	. :any)
     (noweb-sep  . :any)
+    (noweb-prefix . ((no yes)))
     (output-dir . :any)
     (padline	. ((yes no)))
     (post       . :any)
@@ -438,8 +441,8 @@ specific header arguments as well.")
 
 (defconst org-babel-safe-header-args
   '(:cache :colnames :comments :exports :epilogue :hlines :noeval
-	   :noweb :noweb-ref :noweb-sep :padline :prologue :rownames
-	   :sep :session :tangle :wrap
+	   :noweb :noweb-ref :noweb-sep :noweb-prefix :padline
+           :prologue :rownames :sep :session :tangle :wrap
 	   (:eval . ("never" "query"))
 	   (:results . (lambda (str) (not (string-match "file" str)))))
   "A list of safe header arguments for babel source blocks.
@@ -2827,7 +2830,11 @@ block but are passed literally to the \"example-block\"."
          (lang (nth 0 info))
          (body (nth 1 info))
 	 (comment (string= "noweb" (cdr (assq :comments (nth 2 info)))))
-	 (noweb-re (format "\\(.*?\\)\\(%s\\)"
+         (noweb-prefix (let ((v (assq :noweb-prefix (nth 2 info))))
+                         (or (not v)
+                             (and (org-not-nil (cdr v))
+                                  (not (equal (cdr v) "no"))))))
+	 (noweb-re (format "\\(?3:.*?\\)\\(?4:%s\\)"
 			   (with-current-buffer parent-buffer
 			     (org-babel-noweb-wrap))))
 	 (cache nil)
@@ -2878,9 +2885,10 @@ block but are passed literally to the \"example-block\"."
      (lambda (m)
        (with-current-buffer parent-buffer
 	 (save-match-data
-	   (let* ((prefix (match-string 1 m))
-		  (id (match-string 3 m))
+	   (let* ((prefix (match-string 3 m))
+		  (id (match-string 1 m))
 		  (evaluate (string-match-p "(.*)" id))
+                  (print-str (> (length (match-string 2 m)) 0))
 		  (expansion
 		   (cond
 		    (evaluate
@@ -2921,12 +2929,16 @@ block but are passed literally to the \"example-block\"."
 			  (let* ((info (org-babel-get-src-block-info t))
 				 (ref (cdr (assq :noweb-ref (nth 2 info)))))
 			    (push info (gethash ref cache))))))
-		     (funcall expand-references id cache)))))
+		     (funcall expand-references id cache))))
+                  (expansion (if print-str (prin1-to-string expansion)
+                               expansion)))
 	     ;; Interpose PREFIX between every line.
-	     (mapconcat #'identity
-			(split-string expansion "[\n\r]")
-			(concat "\n" prefix))))))
-     body t t 2)))
+	     (if noweb-prefix
+                 (mapconcat #'identity
+			    (split-string expansion "[\n\r]")
+			    (concat "\n" prefix))
+               expansion)))))
+     body t t 4)))
 
 (defun org-babel--script-escape-inner (str)
   (let (in-single in-double backslash out)
-- 
2.36.0

Re: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> at least as long as you're tangling to a programming language, that
> can read lisp strings.
>> Consider the following example:
>>
>> #+BEGIN_SRC emacs-lisp :noweb yes :tangle yes :noweb-prefix no :noweb-trans prin1-to-string
>> <<preamble>>
>> (setq latex-header <<nw>>)
>> #+END_SRC
>>
>> There are two noweb references here. Setting source block-wide
>> :noweb-trans is not helpful because the first reference will be
>> incorrectly filtered through prin1-to-string.
> Indeed. Originally I had thought of adding a new syntax <<"nw">> to
> insert a string representation. I've attached a new patch, that does
> this instead of introducing :noweb-trans. Now that I think of the
> universality of prin1-to-string, I actually like it slightly better
> than :noweb-trans. It would break existing "nw"-like noweb references.

This is too specific, as for me.
Thinking about the whole idea of :noweb-trans more, I see little benefit
compared to something like:

#+name: documentation
This is a sample function documentation.
Because there are "quotes", it must be escaped and cannot be directly
used as noweb-reference.

#+name: doc-escape
#+begin_src emacs-lisp :var str="" :tangle no
(prin1-to-string (string-trim-right str))
#+end_src

#+begin_src emacs-lisp :tangle yes
(defun test ()
<<doc-escape(str=documentation)>>
t)
#+end_src

Maybe we can instead distribute a library ob babel file containing
useful functions together with Org.

>>> [...]
>> This sounds a bit confusing. I would also add an example where it is
>> useful to set :noweb-prefix to no.
>
> I've added such an example in the revised patch attached.

Other than :noweb-trans, the patch looks good for me.

Best,
Ihor

Re: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 729 bytes --]


Ihor Radchenko writes:
> #+name: documentation
> This is a sample function documentation.
> Because there are "quotes", it must be escaped and cannot be directly
> used as noweb-reference.
>
> #+name: doc-escape
> #+begin_src emacs-lisp :var str="" :tangle no
> (prin1-to-string (string-trim-right str))
> #+end_src
>
> #+begin_src emacs-lisp :tangle yes
> (defun test ()
> <<doc-escape(str=documentation)>>
> t)
> #+end_src

Nice ! Quite obscure and brittle (doesn't work if documentation is a 
text src block) but I can use it nonetheless.

> Other than :noweb-trans, the patch looks good for me. 
Here's a patch with only the :noweb-prefix part. If applied, we can mark 
this thread resolved.

Thanks,

-- 
Sébastien Miquel

[-- Attachment #2: 0001-ob-core.el-Add-noweb-prefix-babel-header-argument.patch --]
[-- Type: text/x-patch, Size: 4626 bytes --]

From 3fc3c3557b27026e2cfdb2a1973921c1baf3758a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Mon, 6 Sep 2021 18:45:42 +0200
Subject: [PATCH] ob-core.el: Add `:noweb-prefix` babel header argument

* lisp/ob-core.el (org-babel-expand-noweb-references): Add support for
`noweb-prefix' header argument, to not repeat the prefix characters
when expanding a noweb reference.
(org-babel-common-header-args-w-values):
(org-babel-safe-header-args): Add `noweb-prefix' value.
* doc/org-manual.org: Document `noweb-prefix' babel header argument.
* etc/ORG-NEWS: Document `:noweb-prefix'.
---
 doc/org-manual.org | 17 +++++++++++++++++
 etc/ORG-NEWS       |  6 +++++-
 lisp/ob-core.el    | 17 ++++++++++++-----
 3 files changed, 34 insertions(+), 6 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 6768ca98d..c9c1c1298 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -18760,6 +18760,23 @@ else:
     print('do things when false')
 #+end_example
 
+This prefix behavior can be turned off in a block by setting the
+=noweb-prefix= header argument to =no=, as in:
+
+#+begin_example
+,#+BEGIN_SRC elisp :noweb-prefix no
+  (setq example-data "<<example>>")
+,#+END_SRC
+#+end_example
+
+#+texinfo: @noindent
+which expands to:
+
+#+begin_example
+(setq example-data "this is the
+multi-line body of example")
+#+end_example
+
 When in doubt about the outcome of a source code block expansion, you
 can preview the results with the following command:
 
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 2b539d305..1e8558c7b 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -150,7 +150,7 @@ The entry points are ~org-persist-register~, ~org-persist-unregister~,
 ~org-persist-read~, and ~org-persist-read-all~.  Storing circular
 structures is supported.  Storing references between different
 variables is also supported (see =:inherit= key in
-~org-persist-register~).  
+~org-persist-register~).
 
 The library permits storing buffer-local variables.  Such variables
 are linked to the buffer text, file =inode=, and file path.
@@ -175,6 +175,10 @@ the =compact-itemx= export option, or globally using the
 Items in a description list that begin with =Function:=, =Variable:=
 or certain related prefixes are converted using Texinfo definition
 commands.
+*** New =:noweb-prefix= babel header argument
+
+=:noweb-prefix= can be set to =no= to prevent the prefix characters
+from being repeated when expanding a multiline noweb reference.
 
 ** New functions and changes in function arguments
 
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 6590eeee7..09d6adfe0 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -413,6 +413,7 @@ then run `org-babel-switch-to-session'."
     (noweb	. ((yes no tangle no-export strip-export)))
     (noweb-ref	. :any)
     (noweb-sep  . :any)
+    (noweb-prefix . ((no yes)))
     (output-dir . :any)
     (padline	. ((yes no)))
     (post       . :any)
@@ -438,8 +439,8 @@ specific header arguments as well.")
 
 (defconst org-babel-safe-header-args
   '(:cache :colnames :comments :exports :epilogue :hlines :noeval
-	   :noweb :noweb-ref :noweb-sep :padline :prologue :rownames
-	   :sep :session :tangle :wrap
+	   :noweb :noweb-ref :noweb-sep :noweb-prefix :padline
+           :prologue :rownames :sep :session :tangle :wrap
 	   (:eval . ("never" "query"))
 	   (:results . (lambda (str) (not (string-match "file" str)))))
   "A list of safe header arguments for babel source blocks.
@@ -2827,6 +2828,10 @@ block but are passed literally to the \"example-block\"."
          (lang (nth 0 info))
          (body (nth 1 info))
 	 (comment (string= "noweb" (cdr (assq :comments (nth 2 info)))))
+         (noweb-prefix (let ((v (assq :noweb-prefix (nth 2 info))))
+                         (or (not v)
+                             (and (org-not-nil (cdr v))
+                                  (not (equal (cdr v) "no"))))))
 	 (noweb-re (format "\\(.*?\\)\\(%s\\)"
 			   (with-current-buffer parent-buffer
 			     (org-babel-noweb-wrap))))
@@ -2923,9 +2928,11 @@ block but are passed literally to the \"example-block\"."
 			    (push info (gethash ref cache))))))
 		     (funcall expand-references id cache)))))
 	     ;; Interpose PREFIX between every line.
-	     (mapconcat #'identity
-			(split-string expansion "[\n\r]")
-			(concat "\n" prefix))))))
+	     (if noweb-prefix
+                 (mapconcat #'identity
+			    (split-string expansion "[\n\r]")
+			    (concat "\n" prefix))
+               expansion)))))
      body t t 2)))
 
 (defun org-babel--script-escape-inner (str)
-- 
2.36.0

Re: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

>> Other than :noweb-trans, the patch looks good for me. 
> Here's a patch with only the :noweb-prefix part. If applied, we can mark 
> this thread resolved.

Thanks! Applied on main as 2063596b9.

> Ihor Radchenko writes:
>> #+name: documentation
>> This is a sample function documentation.
>> Because there are "quotes", it must be escaped and cannot be directly
>> used as noweb-reference.
>>
>> #+name: doc-escape
>> #+begin_src emacs-lisp :var str="" :tangle no
>> (prin1-to-string (string-trim-right str))
>> #+end_src
>>
>> #+begin_src emacs-lisp :tangle yes
>> (defun test ()
>> <<doc-escape(str=documentation)>>
>> t)
>> #+end_src
>
> Nice ! Quite obscure and brittle (doesn't work if documentation is a 
> text src block) but I can use it nonetheless.

Well. It looks natural for me. Depends on what you are used to when
working with org babel.

Things are a bit more tricky for latex source blocks, but you can still
do

#+name: nw
#+BEGIN_SRC latex :tangle no
\usepackage{…}
\usepackage{…}
#+END_SRC

#+BEGIN_SRC emacs-lisp :noweb yes :tangle yes :noweb-prefix no
(setq latex-header <<doc-escape(str=nw)>>)
#+END_SRC

Of course, it would only work for source blocks with corresponding
ob-*.el library. But then again, why would you need to wrap things into
text source block when a simple paragraph is enough.

Best,
Ihor

Applied.

Re: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

[Sébastien Miquel]

Hi,

Ihor Radchenko writes:
> Thinking about the whole idea of :noweb-trans more, I see little benefit
> compared to something like:
>
> #+name: documentation
> This is a sample function documentation.
> Because there are "quotes", it must be escaped and cannot be directly
> used as noweb-reference.
>
> #+name: doc-escape
> #+begin_src emacs-lisp :var str="" :tangle no
> (prin1-to-string (string-trim-right str))
> #+end_src
>
> #+begin_src emacs-lisp :tangle yes
> (defun test ()
> <<doc-escape(str=documentation)>>
> t)
> #+end_src
I had converted my uses (tangling code, not text/documentation) to
this but I ended up reverting.

Some drawbacks:
  + doesn't work for all languages (does work for LaTeX)
  + the tangle gets very noisy: not only are the result of execution
    printed in the echo buffer, but emacs visits the tangling buffer
    and moves the point to each block.
    Perhaps this is a bug that can be fixed.
  + src block execution also resets the noweb cache, slowing down
    tangle, though I have not tried to measure the effect.

As stated in the OP, I find it unfortunate that org does not provide
any way to tangle the content of a src block to a string representing
this code. If anyone shows any interest, I've provided two possible
implementations in this thread, that I can rebase.

Regards,

-- 
Sébastien Miquel

Re: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

>> #+name: documentation
>> This is a sample function documentation.
>> Because there are "quotes", it must be escaped and cannot be directly
>> used as noweb-reference.
>>
>> #+name: doc-escape
>> #+begin_src emacs-lisp :var str="" :tangle no
>> (prin1-to-string (string-trim-right str))
>> #+end_src
>>
>> #+begin_src emacs-lisp :tangle yes
>> (defun test ()
>> <<doc-escape(str=documentation)>>
>> t)
>> #+end_src
> I had converted my uses (tangling code, not text/documentation) to
> this but I ended up reverting.

Can you provide more concrete examples?

> Some drawbacks:
>   + doesn't work for all languages (does work for LaTeX)

Which languages do not work?

>   + the tangle gets very noisy: not only are the result of execution
>     printed in the echo buffer, but emacs visits the tangling buffer
>     and moves the point to each block.
>     Perhaps this is a bug that can be fixed.

Did you try to play with :results header argument to disable messages?
What exactly went unexpected?

>   + src block execution also resets the noweb cache, slowing down
>     tangle, though I have not tried to measure the effect.

I am not sure what you are referring to here. Can you elaborate?

Best,
Ihor

Re: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 2178 bytes --]

Hi,

Ihor Radchenko writes:
> Can you provide more concrete examples?
>
>> Some drawbacks:
>>    + doesn't work for all languages (does work for LaTeX)
> Which languages do not work?
Most languages do not work. Using your proposed solution, what I'm
trying to do is

#+name:javascript-header
#+begin_src javascript :tangle no
some javascript, with \ and " to be escaped
#+end_src

#+name: string-escape
#+begin_src emacs-lisp :var str="" :tangle no
(prin1-to-string (string-trim-right str))
#+end_src

#+begin_src emacs-lisp :noweb yes :tangle yes
(setq javascript-header <<string-escape(str=javascript-header)>>)
#+end_src

If you replace javascript with latex, it happens to work, because when
org executes a latex block, it prints its content.

The goal is to tangle to some lisp code whose purpose is to generate
LaTeX/javascript code. Quite niche admittedly, though as you showed,
it could also be used to string-escape documentation.

>>    + the tangle gets very noisy: not only are the result of execution
>>      printed in the echo buffer, but emacs visits the tangling buffer
>>      and moves the point to each block.
>>      Perhaps this is a bug that can be fixed.
> Did you try to play with :results header argument to disable messages?
> What exactly went unexpected?

I did. I might have missed something, but no combination of :results
argument to both the latex block and the string-escape block silences
the tangle (except for :results none, which doesn't tangle the content
of the block). During tangle, the contents of the latex block are
displayed (shortly) in the echo buffer (check *Messages*), and the
point very briefly moves to the latex block. This isn't very
noticeable with a single block.
>>    + src block execution also resets the noweb cache, slowing down
>>      tangle, though I have not tried to measure the effect.
> I am not sure what you are referring to here. Can you elaborate?

Lines 2892-2893 of (my) ob-core.el, in org-babel-expand-noweb-references:

>              ;; Evaluation can potentially modify the buffer
>              ;; and invalidate the cache: reset it.

Regards,

-- 
Sébastien Miquel

[-- Attachment #2: Type: text/html, Size: 4015 bytes --]

[FR] Make :var foo=name-of-src-block assign the source block code instead of currently assigned result of evaluation (was: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments)

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Ihor Radchenko writes:
>> Can you provide more concrete examples?
>>
>>> Some drawbacks:
>>>    + doesn't work for all languages (does work for LaTeX)
>> Which languages do not work?
> Most languages do not work. Using your proposed solution, what I'm
> trying to do is
>
> #+name:javascript-header
> #+begin_src javascript :tangle no
> some javascript, with \ and " to be escaped
> #+end_src
>
> #+name: string-escape
> #+begin_src emacs-lisp :var str="" :tangle no
> (prin1-to-string (string-trim-right str))
> #+end_src
>
> #+begin_src emacs-lisp :noweb yes :tangle yes
> (setq javascript-header <<string-escape(str=javascript-header)>>)
> #+end_src
>
> If you replace javascript with latex, it happens to work, because when
> org executes a latex block, it prints its content.
>
> The goal is to tangle to some lisp code whose purpose is to generate
> LaTeX/javascript code. Quite niche admittedly, though as you showed,
> it could also be used to string-escape documentation.

Thanks for providing an example!

The current behaviour of :var foo=name-of-src-block is assigning result
of evaluation. However, this behaviour is actually not documented.
16.4 Environment of a Code Block section of the manual only documents
uses like
#+BEGIN_SRC emacs-lisp :var length=table-length()
or
#+BEGIN_SRC emacs-lisp :var input=double(input=1)

Currently,
#+BEGIN_SRC emacs-lisp :var length=table-length
is the same with
#+BEGIN_SRC emacs-lisp :var length=table-length()

Given that
#+BEGIN_SRC emacs-lisp :var x=literal-example

inserts the literal contents of the example block, I am voting to change
the current undocumented behaviour for assigning the code block
references.

This will make the discussed example work while not restricting the
existing features.

WDYT?

Best,
Ihor

Re: [FR] Make :var foo=name-of-src-block assign the source block code instead of currently assigned result of evaluation (was: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments)

[Greg Minshall]

Ihor,

> The current behaviour of :var foo=name-of-src-block is assigning result
> of evaluation. However, this behaviour is actually not documented.
> 16.4 Environment of a Code Block section of the manual only documents
> uses like

in fact, the text of the info page (on my system?) says
> code block without arguments
>      A code block name, as assigned by ‘NAME’ keyword from the example
>      above, optionally followed by parentheses.

note the "optionally".

cheers, Greg

Re: [FR] Make :var foo=name-of-src-block assign the source block code instead of currently assigned result of evaluation (was: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments)

[Ihor Radchenko]

Greg Minshall <minshall@umich.edu> writes:

>> The current behaviour of :var foo=name-of-src-block is assigning result
>> of evaluation. However, this behaviour is actually not documented.
>> 16.4 Environment of a Code Block section of the manual only documents
>> uses like
>
> in fact, the text of the info page (on my system?) says
>> code block without arguments
>>      A code block name, as assigned by ‘NAME’ keyword from the example
>>      above, optionally followed by parentheses.
>
> note the "optionally".

Hmm. You are right. I missed "optionally".

Still, I find this idea as the best solution for people who want to
process the source block text during noweb expansion.

Alternative ideas are welcome though. I'd prefer to avoid breaking
change if we can find an equally simple syntax alternative to assign
source block code to a variable.

Best,
Ihor

Re: [FR] Make :var foo=name-of-src-block assign the source block code instead of currently assigned result of evaluation (was: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments)

[Greg Minshall]

Ihor,

> Alternative ideas are welcome though. I'd prefer to avoid breaking
> change if we can find an equally simple syntax alternative to assign
> source block code to a variable.

my two cents would be to avoid the breaking change.  (i notice, for
example, that i use this construct in some of my .org files.)

cheers, Greg

Re: [FR] Make :var foo=name-of-src-block assign the source block code instead of currently assigned result of evaluation (was: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments)

[Sébastien Miquel]

Hi,

Ihor Radchenko writes:
 > Hmm. You are right. I missed "optionally".
 >
 > Still, I find this idea as the best solution for people who want to
 > process the source block text during noweb expansion.
 >
 > Alternative ideas are welcome though. I'd prefer to avoid breaking
 > change if we can find an equally simple syntax alternative to assign
 > source block code to a variable.

The uses are maybe too niche to warrant the breaking change. A syntax
extension like
  : var=block-id[]
seems possible, even though brackets are already overloaded.

One alternative is to only allow the syntax inside noweb brackets
instead of generic variable arguments. I assume there'd be much less
breakage. It would also makes sense to allow noweb references instead
of block ids. We'd add support for
  : <<babel-fn(var=noweb-ref)>>
and <<babel-fn(var=block-id)>> would also insert the contents as a
by-product.

Do you have any example of use in mind, beyond my original one ?

Regards,

-- 
Sébastien Miquel

[PATCH] org-babel: Do not echo output of resolved noweb references

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 2908 bytes --]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

>>>    + the tangle gets very noisy: not only are the result of execution
>>>      printed in the echo buffer, but emacs visits the tangling buffer
>>>      and moves the point to each block.
>>>      Perhaps this is a bug that can be fixed.
>> Did you try to play with :results header argument to disable messages?
>> What exactly went unexpected?
>
> I did. I might have missed something, but no combination of :results
> argument to both the latex block and the string-escape block silences
> the tangle (except for :results none, which doesn't tangle the content
> of the block). During tangle, the contents of the latex block are
> displayed (shortly) in the echo buffer (check *Messages*), and the
> point very briefly moves to the latex block. This isn't very
> noticeable with a single block.

Point movement is normal, but usually invisible because it happens
faster than redisplay cycle. As for displaying the resolved noweb
references, I do think that it should not happen.

Currently, every time Org resolves noweb references, the behaviour is
resembling :results silent. That is, the results of evaluation are
displayed, but not inserted after the executed source blocks.

I propose to change this by using :results none that will display
"results silenced" echo instead of messaging the whole result string.

The proposed behaviour makes more sense compared with the default value
because resolved noweb references are usually used to pipe data
(including table data) between source blocks and to include generated
code during noweb expansion. Such data tends to be fairly bulky and
creates a lot of noise during export/tangling. Not to mention that
displaying large messages also gives a considerable slowdown to the
whole export/tangle process.

Note that changing the current behaviour also triggered the way :results
none are being processed. The de facto implementation in ob-core does
not do any kind of return value post-processing on :results none, which
is not consistent with the manual (16.6 Results of Evaluation). In the
manual, "none" option is a part of the option class consisting of
"replace", "silent", "none", "append", and "prepend". All other
alternative options only influence how the results are inserted after
evaluation and processing. The fact that "none" result is not being
processed according to :file, :post, and other arguments is not
consistent. I changed that. Now, :results "none" are being processed
completely, and only the actual insertion is not performed.

The above change affects the return value of
org-babel-execute-src-block. Previously, it always returned nil with
:results none. Now, it returns the actual post-processed result of
evaluation.

Let me know if you see any potential issues with the proposed change.

Best,
Ihor


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-babel-Do-not-echo-output-of-resolved-noweb-refer.patch --]
[-- Type: text/x-patch, Size: 7234 bytes --]

From f01818ef5bce0a557eb119c58c449d75b28a923e Mon Sep 17 00:00:00 2001
Message-Id: <f01818ef5bce0a557eb119c58c449d75b28a923e.1658401224.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Thu, 21 Jul 2022 18:54:30 +0800
Subject: [PATCH] org-babel: Do not echo output of resolved noweb references

* lisp/ob-core.el (org-babel-execute-src-block): Post-process and
return results even when "none" results parameter is given.  Document
that the result value is returned, as it is assumed by
`org-babel-ref-resolve'.
(org-babel-result-cond): Do return results even when "none" results
parameter is given.

According to the manual ":results none" should only affect how the
results of evaluation are inserted into the buffer.  However, the
results are simply ignored currently.  Fix this.
* lisp/ob-ref.el (org-babel-ref-resolve): Set :results to "none" when
resolving noweb references.  Together with the above changes, this
makes Org not echo the results value yet returning the value
programatically.

Reported in https://orgmode.org/list/7702b511-c289-5688-c64c-fb673324a63a@posteo.eu
---
 lisp/ob-core.el | 118 ++++++++++++++++++++++++------------------------
 lisp/ob-ref.el  |   2 +-
 2 files changed, 59 insertions(+), 61 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index ac9af5d24..714f70c6a 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -712,7 +712,7 @@ (defvar *this*) ; Dynamically bound in `org-babel-execute-src-block'
 
 ;;;###autoload
 (defun org-babel-execute-src-block (&optional arg info params)
-  "Execute the current source code block.
+  "Execute the current source code block and return the result.
 Insert the results of execution into the buffer.  Source code
 execution and the collection and formatting of results can be
 controlled through a variety of header arguments.
@@ -776,51 +776,50 @@ (defun org-babel-execute-src-block (&optional arg info params)
 		     (capitalize lang)
 		     (let ((name (nth 4 info)))
 		       (if name (format " (%s)" name) "")))
-	    (if (member "none" result-params)
-		(progn (funcall cmd body params)
-		       (message "result silenced"))
-	      (setq result
-		    (let ((r (funcall cmd body params)))
-		      (if (and (eq (cdr (assq :result-type params)) 'value)
-			       (or (member "vector" result-params)
-				   (member "table" result-params))
-			       (not (listp r)))
-			  (list (list r))
-			r)))
-	      (let ((file (and (member "file" result-params)
-			       (cdr (assq :file params)))))
-		;; If non-empty result and :file then write to :file.
-		(when file
-		  ;; If `:results' are special types like `link' or
-		  ;; `graphics', don't write result to `:file'.  Only
-		  ;; insert a link to `:file'.
-		  (when (and result
-			     (not (or (member "link" result-params)
-				      (member "graphics" result-params))))
-		    (with-temp-file file
-		      (insert (org-babel-format-result
-			       result
-			       (cdr (assq :sep params)))))
-		    ;; Set file permissions if header argument
-		    ;; `:file-mode' is provided.
-		    (when (assq :file-mode params)
-		      (set-file-modes file (cdr (assq :file-mode params)))))
-		  (setq result file))
-		;; Possibly perform post process provided its
-		;; appropriate.  Dynamically bind "*this*" to the
-		;; actual results of the block.
-		(let ((post (cdr (assq :post params))))
-		  (when post
-		    (let ((*this* (if (not file) result
-				    (org-babel-result-to-file
-				     file
-				     (org-babel--file-desc params result)
-                                     'attachment))))
-		      (setq result (org-babel-ref-resolve post))
-		      (when file
-			(setq result-params (remove "file" result-params))))))
-		(org-babel-insert-result
-		 result result-params info new-hash lang)))
+	    (setq result
+		  (let ((r (funcall cmd body params)))
+		    (if (and (eq (cdr (assq :result-type params)) 'value)
+			     (or (member "vector" result-params)
+				 (member "table" result-params))
+			     (not (listp r)))
+			(list (list r))
+		      r)))
+	    (let ((file (and (member "file" result-params)
+			     (cdr (assq :file params)))))
+	      ;; If non-empty result and :file then write to :file.
+	      (when file
+		;; If `:results' are special types like `link' or
+		;; `graphics', don't write result to `:file'.  Only
+		;; insert a link to `:file'.
+		(when (and result
+			   (not (or (member "link" result-params)
+				  (member "graphics" result-params))))
+		  (with-temp-file file
+		    (insert (org-babel-format-result
+			     result
+			     (cdr (assq :sep params)))))
+		  ;; Set file permissions if header argument
+		  ;; `:file-mode' is provided.
+		  (when (assq :file-mode params)
+		    (set-file-modes file (cdr (assq :file-mode params)))))
+		(setq result file))
+	      ;; Possibly perform post process provided its
+	      ;; appropriate.  Dynamically bind "*this*" to the
+	      ;; actual results of the block.
+	      (let ((post (cdr (assq :post params))))
+		(when post
+		  (let ((*this* (if (not file) result
+				  (org-babel-result-to-file
+				   file
+				   (org-babel--file-desc params result)
+                                   'attachment))))
+		    (setq result (org-babel-ref-resolve post))
+		    (when file
+		      (setq result-params (remove "file" result-params))))))
+	      (if (member "none" result-params)
+		  (message "result silenced")
+	        (org-babel-insert-result
+	         result result-params info new-hash lang)))
 	    (run-hooks 'org-babel-after-execute-hook)
 	    result)))))))
 
@@ -3199,20 +3198,19 @@ (defmacro org-babel-result-cond (result-params scalar-form &rest table-forms)
   (declare (indent 1) (debug t))
   (org-with-gensyms (params)
     `(let ((,params ,result-params))
-       (unless (member "none" ,params)
-	 (if (or (member "scalar" ,params)
-		 (member "verbatim" ,params)
-		 (member "html" ,params)
-		 (member "code" ,params)
-		 (member "pp" ,params)
-		 (member "file" ,params)
-		 (and (or (member "output" ,params)
-			  (member "raw"    ,params)
-			  (member "org"    ,params)
-			  (member "drawer" ,params))
-		      (not (member "table" ,params))))
-	     ,scalar-form
-	   ,@table-forms)))))
+       (if (or (member "scalar" ,params)
+	       (member "verbatim" ,params)
+	       (member "html" ,params)
+	       (member "code" ,params)
+	       (member "pp" ,params)
+	       (member "file" ,params)
+	       (and (or (member "output" ,params)
+			(member "raw"    ,params)
+			(member "org"    ,params)
+			(member "drawer" ,params))
+		    (not (member "table" ,params))))
+	   ,scalar-form
+	 ,@table-forms))))
 
 (defun org-babel-temp-file (prefix &optional suffix)
   "Create a temporary file in the `org-babel-temporary-directory'.
diff --git a/lisp/ob-ref.el b/lisp/ob-ref.el
index d5356f18c..87a7ccf63 100644
--- a/lisp/ob-ref.el
+++ b/lisp/ob-ref.el
@@ -153,7 +153,7 @@ (defun org-babel-ref-resolve (ref)
 	    (setq ref split-ref))
 	  (org-with-wide-buffer
 	   (goto-char (point-min))
-	   (let* ((params (append args '((:results . "silent"))))
+	   (let* ((params (append args '((:results . "none"))))
 		  (regexp (org-babel-named-data-regexp-for-name ref))
 		  (result
 		   (catch :found
-- 
2.35.1

Re: [PATCH] org-babel: Do not echo output of resolved noweb references

[Sébastien Miquel]

Hi,

Ihor Radchenko writes:
 > Let me know if you see any potential issues with the proposed change.

Thanks for looking into this. I think the change looks right, though I
am no great user of org-babel and its header arguments.

Regards,

-- 
Sébastien Miquel

Re: [FR] Make :var foo=name-of-src-block assign the source block code instead of currently assigned result of evaluation (was: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments)

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> The uses are maybe too niche to warrant the breaking change. A syntax
> extension like
>   : var=block-id[]
> seems possible, even though brackets are already overloaded.

This sounds better. Using brackets will actually make a lot of sense.

Currently, we have roughly the following logic in the :var references:

- :var x=reference(<optional parameters>)
  will execute the reference and assign the returned value

- :var x=reference[<optional parameters>]
  will take the reference text representation, possibly process it
  according to the parameters, and return the processed text

  *this only works for tables*

- :var x=reference
  will perform some "default" action depending on the reference type.
  the action may be "execute" or "get text representation" for source
  blocks and others correspondingly.

Extending reference[...] syntax to all the element types will make
things consistent and allow future extensions if we decide to provide
"index" syntax for non-tables (code lines, function definitions?)

> One alternative is to only allow the syntax inside noweb brackets
> instead of generic variable arguments. I assume there'd be much less
> breakage. It would also makes sense to allow noweb references instead
> of block ids. We'd add support for
>   : <<babel-fn(var=noweb-ref)>>
> and <<babel-fn(var=block-id)>> would also insert the contents as a
> by-product.

This would still be a breaking change. And we will introduce
inconsistency between resolving noweb references and resolving :var
header args.

Moreover, resolving references is currently handled by the same code
inside :vars and noweb references. Changing this will complicate the
source code as well, IMHO increasing the maintenance burden too much.

> Do you have any example of use in mind, beyond my original one ?

For example, one may convert .el file commentary to Readme.org:

#+name: my-library
#+begin_src :tangle my-library.el
;; Commentary:
;; A lot of commentary
#+end_src

#+begin_src :exports results :var commentary=my-library[]
(convert-to-org-headlines commentary)
#+end_src

Best,
Ihor

Re: [PATCH] org-babel: Do not echo output of resolved noweb references

[Ihor Radchenko]

Ihor Radchenko <yantar92@gmail.com> writes:

> Currently, every time Org resolves noweb references, the behaviour is
> resembling :results silent. That is, the results of evaluation are
> displayed, but not inserted after the executed source blocks.
>
> I propose to change this by using :results none that will display
> "results silenced" echo instead of messaging the whole result string.

Applied onto main via 2dfdc8953.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=2dfdc895351dd867042a4693a63cc688a7be1fb5

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92

Re: [PATCH] Make :var foo=name-of-src-block assign the source block code instead of currently assigned result of evaluation (was: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments)

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 114 bytes --]

Hi Ihor,

I've implemented this proposal in the patch attached.

Does it look good to you ?

-- 
Sébastien Miquel

[-- Attachment #2: 0001-New-babel-syntax-to-pass-src-block-contents-as-argum.patch --]
[-- Type: text/x-patch, Size: 5096 bytes --]

From b1b783dc80821b07937ac4211ec28df8726fff1c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Sat, 13 Aug 2022 20:49:27 +0200
Subject: [PATCH] New babel syntax to pass src block contents as argument

* lisp/ob-ref.el (org-babel-ref-resolve): Add support for
`named-block[]' syntax, resolving to the contents of a named-block.
* lisp/ob-core.el (org-babel-read-element): Read a code block into its
contents, like other blocks.
* testing/listp/test-ob.el (test-ob/block-content-resolution): Test
block content resolution.
* doc/org-manual.org: Document syntax.
* etc/ORG-NEWS: Document syntax.
---
 doc/org-manual.org      |  7 ++++---
 etc/ORG-NEWS            |  5 +++++
 lisp/ob-core.el         |  2 +-
 lisp/ob-ref.el          | 10 ++++++----
 testing/lisp/test-ob.el | 15 +++++++++++++++
 5 files changed, 31 insertions(+), 8 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 57a57a6fe..794682b49 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -17505,9 +17505,10 @@ a colon, for example: =:var table=other-file.org:example-table=.
   : 4
   #+end_example
 
-- literal example ::
+- literal example, or code block contents ::
 
-  A literal example block named with a =NAME= keyword.
+  A code block or literal example block named with a =NAME= keyword,
+  followed by brackets (optional for example blocks).
 
   #+begin_example
   ,#+NAME: literal-example
@@ -17517,7 +17518,7 @@ a colon, for example: =:var table=other-file.org:example-table=.
   ,#+END_EXAMPLE
 
   ,#+NAME: read-literal-example
-  ,#+BEGIN_SRC emacs-lisp :var x=literal-example
+  ,#+BEGIN_SRC emacs-lisp :var x=literal-example[]
     (concatenate #'string x " for you.")
   ,#+END_SRC
 
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 7dae03dc6..d6d99a64b 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -288,6 +288,11 @@ The =org-md-toplevel-hlevel= customization variable sets the heading
 level used for top level headings, much like how
 =org-html-toplevel-hlevel= sets the heading level used for top level
 headings in HTML export.
+*** Babel: new syntax to pass the contents of a src block as argument
+
+Use the header argument =:var x=code-block[]= or
+: #+CALL: fn(x=code-block[])
+to pass the contents of a named code block as a string argument.
 
 ** New options
 *** A new custom setting =org-hide-drawer-startup= to control initial folding state of drawers
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 68dd5557c..c52ef9ed6 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2156,7 +2156,7 @@ Return nil if ELEMENT cannot be read."
 	(or (org-babel--string-to-number v) v)))
      (`table (org-babel-read-table))
      (`plain-list (org-babel-read-list))
-     (`example-block
+     ((or `example-block `src-block)
       (let ((v (org-element-property :value element)))
 	(if (or org-src-preserve-indentation
 		(org-element-property :preserve-indent element))
diff --git a/lisp/ob-ref.el b/lisp/ob-ref.el
index 87a7ccf63..ee2745e09 100644
--- a/lisp/ob-ref.el
+++ b/lisp/ob-ref.el
@@ -124,12 +124,14 @@ Emacs Lisp representation of the value of the variable."
       (save-excursion
 	(let ((case-fold-search t)
 	      args new-refere new-header-args new-referent split-file split-ref
-	      index)
+	      index contents)
 	  ;; if ref is indexed grab the indices -- beware nested indices
-	  (when (and (string-match "\\[\\([^\\[]+\\)\\]$" ref)
+	  (when (and (string-match "\\[\\([^\\[]*\\)\\]$" ref)
 		     (let ((str (substring ref 0 (match-beginning 0))))
 		       (= (cl-count ?\( str) (cl-count ?\) str))))
-	    (setq index (match-string 1 ref))
+            (if (> (length (match-string 1 ref)) 0)
+	        (setq index (match-string 1 ref))
+              (setq contents t))
 	    (setq ref (substring ref 0 (match-beginning 0))))
 	  ;; assign any arguments to pass to source block
 	  (when (string-match
@@ -171,7 +173,7 @@ Emacs Lisp representation of the value of the variable."
 				(throw :found
 				       (org-babel-execute-src-block
 					nil (org-babel-lob-get-info e) params)))
-			       (`src-block
+			       ((and `src-block (guard (not contents)))
 				(throw :found
 				       (org-babel-execute-src-block
 					nil nil
diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el
index a62bd56bf..c944ccd39 100644
--- a/testing/lisp/test-ob.el
+++ b/testing/lisp/test-ob.el
@@ -178,6 +178,21 @@ should still return the link."
 			    (point-at-bol)
 			    (point-at-eol))))))
 
+(ert-deftest test-ob/block-content-resolution ()
+  "Test block content resolution."
+  (org-test-with-temp-text-in-file "
+
+#+name: four
+#+begin_src emacs-lisp
+  (list 1 2 3 4)
+#+end_src
+
+#+begin_src emacs-lisp :var four=four[]
+  (length (eval (car (read-from-string four))))
+#+end_src"
+                                   (org-babel-next-src-block 2)
+                                   (should (= 4 (org-babel-execute-src-block)))))
+
 (ert-deftest test-ob/cons-cell-as-variable ()
   "Test that cons cell can be assigned as variable."
   (org-test-with-temp-text "
-- 
2.37.2

Re: [PATCH] Make :var foo=name-of-src-block assign the source block code instead of currently assigned result of evaluation (was: [PATCH] Add :noweb-prefix and :noweb-trans babel header arguments)

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> I've implemented this proposal in the patch attached.
>
> Does it look good to you ?

Thanks! The patch looks mostly fine.

Applied onto main via 72f66ca0b.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=72f66ca0b9d336e0da61b17cbe8ce183eef364dd

A small side effect of the patch is that name[] will also work for
lists:

#+name: test
- one
- two
- three


#+begin_src emacs-lisp :var x=test[]
  (message "%S" x)
#+end_src

#+RESULTS:
: (("one") ("two") ("three"))

which is not intentional, but does not break anything and also somewhat
logical.

#+begin_src emacs-lisp :var x=test[1:2]
  (message "%S" x)
#+end_src

#+RESULTS:
: (("two") ("three"))

worked in the past anyway, despite not being documented.

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92