[BUG] Reference chains (C refers to B refers to A) in tangle do not behave as expected [9.5.5 (release_9.5.5 @ /usr/local/share/emacs/28.2/lisp/org/)]

[BUG] Reference chains (C refers to B refers to A) in tangle do not behave as expected [9.5.5 (release_9.5.5 @ /usr/local/share/emacs/28.2/lisp/org/)]

[C. M. Sperberg-McQueen]

Remember to cover the basics, that is, what you expected to happen and
what in fact did happen.  You don't know how to make a good report?  See

     https://orgmode.org/manual/Feedback.html#Feedback

Your bug report will be posted to the Org mailing list.
------------------------------------------------------------------------

My thanks to those who made and maintain org-mode.

Consider the following literate program in org mode:

................ make-abc.org:
#+title: Minimal example
#+date: 20 May 2024

The first letter is A.
#+name: Able
#+begin_src txt
A is for application.
#+end_src

The second letter is B.  It follows A.
#+name: Baker 
#+begin_src txt :noweb tangle
<<Able>>
B is for beta-tester.
#+end_src

The third letter is C.  We'll stop here.
#+name: Charlie
#+begin_src txt :tangle abc.txt :noweb tangle
<<Baker>>
C is for coder.
#+end_src

Bottom line: two-level inclusions work only if the middle layer (here,
/Baker/) has ~:noweb yes~.  Using ~:noweb tangle~ on /Baker/ produces
the same results as using ~:noweb no~.
................ end make-abc.org:

When I run org-babel-tangle (C-c C-v t) to produce abc.txt, what I
expect is a file containing the three lines

  A is for application.
  B is for beta-tester.
  C is for coder.

in that order.  What I get is a file containing:

  <<Able>>
  B is for beta-tester.
  C is for coder.

When I change the noweb header argument on source code block Baker from
:noweb tangle to :noweb yes, I get the expected result when tangling.
Unfortunately, :noweb yes also means that the reference from Baker to
Able is expanded in the HTML export, which is not (here) what I want.

In the org-mode manual [1], the meaning of :noweb tangle is described
as: "Expansion of noweb syntax references in the body of the code block
when tangling. No expansion when evaluating or exporting."  I think that
running org-babel-tangle counts as tangling, so I expect the noweb
syntax references inside of code block Baker to be expanded.

If I've misunderstood the documentation, a clarification would be
welcome.  If there are other work-arounds, I would be glad to hear of
them.

Thanks.

[1] https://orgmode.org/manual/Noweb-Reference-Syntax.html

Emacs  : GNU Emacs 28.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.33, cairo version 1.16.0)
 of 2023-07-17
Package: Org mode version 9.5.5 (release_9.5.5 @ /usr/local/share/emacs/28.2/lisp/org/)

current state:
==============
(setq
 org-link-elisp-confirm-function 'yes-or-no-p
 org-bibtex-headline-format-function #[257 "\300%1\236A\207" [:title] 3 "\n\n(fn ENTRY)"]
 org-log-done 'time
 org-agenda-files '("/home/cmsmcq/2024/tlrr/A/todo-tlrr.org"
                    "/home/cmsmcq/2024/ixml/A/todo-ixml.org"
                    "/home/cmsmcq/2024/mlcd/A/todo-mlcd.org"                    
                    "/home/cmsmcq/2024/A/todo-oop.org")
 org-export-before-parsing-hook '(org-attach-expand-links)
 org-archive-hook '(org-attach-archive-delete-maybe)
 org-odt-format-inlinetask-function 'org-odt-format-inlinetask-default-function
 org-ascii-format-drawer-function #[771 "%1\207" [] 4 "\n\n(fn NAME CONTENTS WIDTH)"]
 org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-hide-drawers
                  org-cycle-show-empty-lines
                  org-optimize-window-after-visibility-change)
 org-link-from-user-regexp "\\<cmsmcq@gemma\\>\\|\\<C\\. M\\. Sperberg-McQueen\\>"
 org-mode-hook '((lambda nil
                  (progn (modify-syntax-entry 60 "_" org-mode-syntax-table)
                   (modify-syntax-entry 62 "_" org-mode-syntax-table))
                  )
                 #[0 "\301\211%10\207"
                   [imenu-create-index-function org-imenu-get-tree] 2]
                 #[0 "\300\301\302\303\304$\207"
                   [add-hook change-major-mode-hook org-show-all append local]
                   5]
                 #[0 "\300\301\302\303\304$\207"
                   [add-hook change-major-mode-hook org-babel-show-result-all
                    append local]
                   5]
                 org-babel-result-hide-spec org-babel-hide-all-hashes)
 org-babel-load-languages '((shell . t) (emacs-lisp . t))
 org-latex-format-drawer-function #[514 "\207" [] 3 "\n\n(fn _ CONTENTS)"]
 org-latex-format-headline-function 'org-latex-format-headline-default-function
 org-confirm-shell-link-function 'yes-or-no-p
 org-reveal-start-hook '(org-decrypt-entry)
 org-html-format-drawer-function #[514 "\207" [] 3 "\n\n(fn NAME CONTENTS)"]
 outline-isearch-open-invisible-function 'outline-isearch-open-invisible
 org-odt-format-headline-function 'org-odt-format-headline-default-function
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-babel-tangle-lang-exts '(("D" . "d") ("C++" . "cpp") ("awk" . "awk")
                              ("clojurescript" . "cljs") ("clojure" . "clj")
                              ("fortran" . "F90") ("groovy" . "groovy")
                              ("haskell" . "hs") ("java" . "java")
                              ("julia" . "jl") ("latex" . "tex")
                              ("LilyPond" . "ly") ("lisp" . "lisp")
                              ("lua" . "lua") ("maxima" . "max")
                              ("ocaml" . "ml") ("perl" . "pl")
                              ("processing" . "pde") ("python" . "py")
                              ("ruby" . "rb") ("sed" . "sed")
                              ("emacs-lisp" . "el") ("elisp" . "el"))
 org-src-mode-hook '(org-src-babel-configure-edit-buffer
                     org-src-mode-configure-edit-buffer)
 org-confirm-elisp-link-function 'yes-or-no-p
 org-todo-keywords '((sequence "TODO(t)" "NEXT(n)" "PROG(p)" "INTERRUPT(i)"
                      "DONE(d)")
                     )
 org-speed-command-hook '(org-speed-command-activate
                          org-babel-speed-command-activate)
 org-html-format-inlinetask-function 'org-html-format-inlinetask-default-function
 org-ascii-format-inlinetask-function 'org-ascii-format-inlinetask-default
 org-odt-format-drawer-function #[514 "\207" [] 3 "\n\n(fn NAME CONTENTS)"]
 org-latex-format-inlinetask-function 'org-latex-format-inlinetask-default-function
 org-emphasis-alist '(("*" bold) ("/" italic) ("=" org-verbatim verbatim)
                      ("~" org-code verbatim) ("+" (:strike-through t)))
 org-tab-first-hook '(org-babel-hide-result-toggle-maybe
                      org-babel-header-arg-expand)
 org-link-shell-confirm-function 'yes-or-no-p
 org-babel-pre-tangle-hook '(save-buffer)
 org-agenda-loop-over-headlines-in-active-region nil
 org-occur-hook '(org-first-headline-recenter)
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-link-parameters '(("attachment" :follow org-attach-follow :complete
                        org-attach-complete-link)
                       ("id" :follow org-id-open)
                       ("eww" :follow org-eww-open :store org-eww-store-link)
                       ("rmail" :follow org-rmail-open :store
                        org-rmail-store-link)
                       ("mhe" :follow org-mhe-open :store org-mhe-store-link)
                       ("irc" :follow org-irc-visit :store org-irc-store-link
                        :export org-irc-export)
                       ("info" :follow org-info-open :export org-info-export
                        :store org-info-store-link)
                       ("gnus" :follow org-gnus-open :store org-gnus-store-link)
                       ("docview" :follow org-docview-open :export
                        org-docview-export :store org-docview-store-link)
                       ("bibtex" :follow org-bibtex-open :store
                        org-bibtex-store-link)
                       ("bbdb" :follow org-bbdb-open :export org-bbdb-export
                        :complete org-bbdb-complete-link :store
                        org-bbdb-store-link)
                       ("w3m" :store org-w3m-store-link)
                       ("doi" :follow org-link-doi-open :export
                        org-link-doi-export)
                       ("mu4e" :follow mu4e-org-open :store mu4e-org-store-link)
                       ("file+sys") ("file+emacs")
                       ("shell" :follow org-link--open-shell)
                       ("news" :follow
                        #[514 "\301\300\302%4Q%2\"\207" ["news" browse-url ":"]
                          6 "\n\n(fn URL ARG)"]
                        )
                       ("mailto" :follow
                        #[514 "\301\300\302%4Q%2\"\207"
                          ["mailto" browse-url ":"] 6 "\n\n(fn URL ARG)"]
                        )
                       ("https" :follow
                        #[514 "\301\300\302%4Q%2\"\207"
                          ["https" browse-url ":"] 6 "\n\n(fn URL ARG)"]
                        )
                       ("http" :follow
                        #[514 "\301\300\302%4Q%2\"\207" ["http" browse-url ":"]
                          6 "\n\n(fn URL ARG)"]
                        )
                       ("ftp" :follow
                        #[514 "\301\300\302%4Q%2\"\207" ["ftp" browse-url ":"]
                          6 "\n\n(fn URL ARG)"]
                        )
                       ("help" :follow org-link--open-help :store
                        org-link--store-help)
                       ("file" :complete org-link-complete-file)
                       ("elisp" :follow org-link--open-elisp))
 org-html-format-headline-function 'org-html-format-headline-default-function
 org-metaup-hook '(org-babel-load-in-session-maybe)
 org-agenda-include-diary t
 )
-- 
C. M. Sperberg-McQueen
Black Mesa Technologies LLC
http://blackmesatech.com

Re: [BUG] Reference chains (C refers to B refers to A) in tangle do not behave as expected [9.5.5 (release_9.5.5 @ /usr/local/share/emacs/28.2/lisp/org/)]

[Ihor Radchenko]

"C. M. Sperberg-McQueen" <cmsmcq@blackmesatech.com> writes:

> Consider the following literate program in org mode:
>
> ................ make-abc.org:
> #+title: Minimal example
> #+date: 20 May 2024
>
> The first letter is A.
> #+name: Able
> #+begin_src txt
> A is for application.
> #+end_src
>
> The second letter is B.  It follows A.
> #+name: Baker 
> #+begin_src txt :noweb tangle
> <<Able>>
> B is for beta-tester.
> #+end_src
>
> The third letter is C.  We'll stop here.
> #+name: Charlie
> #+begin_src txt :tangle abc.txt :noweb tangle
> <<Baker>>
> C is for coder.
> #+end_src
>
> Bottom line: two-level inclusions work only if the middle layer (here,
> /Baker/) has ~:noweb yes~.  Using ~:noweb tangle~ on /Baker/ produces
> the same results as using ~:noweb no~.

Yes, and it is expected.

> ...
> In the org-mode manual [1], the meaning of :noweb tangle is described
> as: "Expansion of noweb syntax references in the body of the code block
> when tangling. No expansion when evaluating or exporting."  I think that
> running org-babel-tangle counts as tangling, so I expect the noweb
> syntax references inside of code block Baker to be expanded.

They are indeed expanded - there is a single noweb reference in Charlie
block - <<Baker>>. It is expanded to the code contained in the Baker
block. When retrieving the code for Baker block, Org mode goes to that
block and cleans up its text according to that block's
expansion. However, for Baker block, when it does not have :noweb
tangle, the <<...>> in the code are treated literally.

In other words, each noweb reference is expanded in its own context,
according to each block's own header arguments.

Not a bug.
Canceled.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Reference chains (C refers to B refers to A) in tangle do not behave as expected [9.5.5 (release_9.5.5 @ /usr/local/share/emacs/28.2/lisp/org/)]

[C. M. Sperberg-McQueen]

Thank you for the response.  I am sorry to say I do not understand it.

You say that non-expansion of references in the code block Baker is
expected "when it does not have :noweb tangle".

> ... When retrieving the code for Baker block, Org mode goes to that
> block and cleans up its text according to that block's
> expansion. However, for Baker block, when it does not have :noweb
> tangle, the <<...>> in the code are treated literally.

The problem I am reporting is that the <<...>> in the code of Baker are
treated literally even when Baker DOES have :noweb tangle.

Thanks.

Michael Sperberg-McQueen



Ihor Radchenko <yantar92@posteo.net> writes:

> "C. M. Sperberg-McQueen" <cmsmcq@blackmesatech.com> writes:
>
>> Consider the following literate program in org mode:
>>
>> ................ make-abc.org:
>> #+title: Minimal example
>> #+date: 20 May 2024
>>
>> The first letter is A.
>> #+name: Able
>> #+begin_src txt
>> A is for application.
>> #+end_src
>>
>> The second letter is B.  It follows A.
>> #+name: Baker 
>> #+begin_src txt :noweb tangle
>> <<Able>>
>> B is for beta-tester.
>> #+end_src
>>
>> The third letter is C.  We'll stop here.
>> #+name: Charlie
>> #+begin_src txt :tangle abc.txt :noweb tangle
>> <<Baker>>
>> C is for coder.
>> #+end_src
>>
>> Bottom line: two-level inclusions work only if the middle layer (here,
>> /Baker/) has ~:noweb yes~.  Using ~:noweb tangle~ on /Baker/ produces
>> the same results as using ~:noweb no~.
>
> Yes, and it is expected.
>
>> ...
>> In the org-mode manual [1], the meaning of :noweb tangle is described
>> as: "Expansion of noweb syntax references in the body of the code block
>> when tangling. No expansion when evaluating or exporting."  I think that
>> running org-babel-tangle counts as tangling, so I expect the noweb
>> syntax references inside of code block Baker to be expanded.
>
> They are indeed expanded - there is a single noweb reference in Charlie
> block - <<Baker>>. It is expanded to the code contained in the Baker
> block. When retrieving the code for Baker block, Org mode goes to that
> block and cleans up its text according to that block's
> expansion. However, for Baker block, when it does not have :noweb
> tangle, the <<...>> in the code are treated literally.
>
> In other words, each noweb reference is expanded in its own context,
> according to each block's own header arguments.
>
> Not a bug.
> Canceled.


-- 
C. M. Sperberg-McQueen
Black Mesa Technologies LLC
http://blackmesatech.com

Re: [BUG] Reference chains (C refers to B refers to A) in tangle do not behave as expected [9.5.5 (release_9.5.5 @ /usr/local/share/emacs/28.2/lisp/org/)]

[Ihor Radchenko]

"C. M. Sperberg-McQueen" <cmsmcq@blackmesatech.com> writes:

>> ... When retrieving the code for Baker block, Org mode goes to that
>> block and cleans up its text according to that block's
>> expansion. However, for Baker block, when it does not have :noweb
>> tangle, the <<...>> in the code are treated literally.
>
> The problem I am reporting is that the <<...>> in the code of Baker are
> treated literally even when Baker DOES have :noweb tangle.

Confirmed.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>