Export tangle filename with source block

Export tangle filename with source block

[Thibault Marin]

Hi list,

I have an org file that I am tangling into multiple files, and exporting
to html.  What I would like to do is to label each source block in the
exported html with the filename used for tangling this specific block.
I don't have a strong opinion about the actual appearance of the label
(adding a comment at the top of the source block, hover in the code
textarea, other?).

I couldn't find any option to do that, so I initially though of using
`org-export-filter-src-block-functions' (following
http://orgmode.org/manual/Advanced-configuration.html).  I was not able
to get the org-element object for the current block from the parameter
received by the filter.  It looks like the third parameter (`info') may
have more information but I currently don't see how to get (1) the
current source block, and (2) the tangling filename.

I then tried to define a derived backend with custom handling of source
blocks as described in the documentation.

I currently have the following, where I try to get the tangle filename
using `org-babel-get-src-block-info', and later insert it into the html
content using an ugly regexp.

(defun my-html-src-block (src-block contents info)
  "Transcode a SRC-BLOCK element from Org to HTML.
     CONTENTS is nil.  INFO is a plist used as a communication
     channel."
  (let* ((lang (org-element-property :language src-block))
         (src-info (org-babel-get-src-block-info t src-block))
         (tang (cdr (assq :tangle (nth 2 src-info))))
         (export-out (org-export-with-backend 'html src-block contents info))
         )
    (if (and (string= lang "lua") (and tang (> (length tang) 0)))
        (progn
          (let ((src-start-pat "\\(<pre class=\"src src-lua\" id=\"[^\"]+*\">\\)"))
            (replace-regexp-in-string src-start-pat
                                      (concat "\\1"
                                              "\n-- ["
                                              tang
                                              "]\n\n") export-out)
            )
          )
      export-out
      )
    )
  )

(org-export-define-derived-backend 'my-html 'html
  :translate-alist '((src-block . my-html-src-block)))

Besides feeling wrong, this always gets the tangle name from the top
level org option ("#+PROPERTY: tangle main-file" at the top of the file)
instead of the one overridden by individual blocks "#+BEGIN_SRC :tangle
other-file".  Moreover the added comment is not syntax highlighted and
this feels really wrong.

Does anybody have any idea on how to achieve this?

Thanks in advance.

thibault

Re: Export tangle filename with source block

[Thibault Marin]

Hi all,

I am following up on a question I posted here a while ago to show my progress in
case it can help others and to ask for a few clarifications.  I would appreciate
any feedback.

My goal is to add a label to source blocks on HTML export to indicate the name
of the file to which the block is tangled.

The issues originally raised were:
1) How to display the label in HTML export?
2) How to get the tangle file name, which may come from
   1) a file property (e.g. #+PROPERTY: header-args :tangle file-main.py in the
      file header),
   2) a section property (e.g. :header-args: :tangle file-prop.py in a property
      drawer),
   3) the source block header line (e.g. #+BEGIN_SRC python :tangle no)?

It appears that the best way to achieve this is to define a derived backend with
special handling for src-block objects as in
http://orgmode.org/manual/Advanced-configuration.html.

* Example file

Here is the example file I am working with.

#+BEGIN_SRC org
,#+TITLE: Test
,#+PROPERTY: header-args :tangle file-main.py

,#+NAME: src-no-1
,#+BEGIN_SRC python :tangle no
# SHOULD NOT BE TANGLED (1)
,#+END_SRC

,#+NAME: src-header
,#+BEGIN_SRC python :tangle file-header.py
# Tangle to file-header.py
,#+END_SRC

,#+NAME: src-main
,#+BEGIN_SRC python
# Tangle to file-main.py
,#+END_SRC

,* section
  :PROPERTIES:
  :header-args: :tangle file-prop.py
  :END:

,#+NAME: src-prop
,#+BEGIN_SRC python :tangle file-prop.py
# Tangle to file-prop.py
,#+END_SRC

,#+NAME: src-no-2
,#+BEGIN_SRC python :tangle no
# SHOULD NOT BE TANGLED (2)
,#+END_SRC
#+END_SRC


* Formatting the label

I think I have a satisfactory (at least for me) solution for this: in my
src-block exporter I first pass the src-block by the regular HTML exporter:
#+BEGIN_SRC emacs-lisp
(let ((export-out (org-export-with-backend 'html src-block contents info)))
  ...)
#+END_SRC

Later, I add a HTML "<div>" block under the "<pre>" block for the source.  I do
this using a regexp:
#+BEGIN_SRC emacs-lisp
(let ((src-start-pat "\\(<pre class=\"src src-[^>]+>\\)"))
  (setq export-out
        (replace-regexp-in-string
         src-start-pat
         (concat "\\1<div style=\"position: absolute; bottom: 0;"
                 " right: 0; text-align:right;\" class=\"src-label\">"
                 tang "</div>") export-out)))
#+END_SRC

This adds the label on the bottom right corner of the "<pre>" block and uses the
"src-label" class, which can be customized via CSS.


* Getting the tangle filename

I am working with the assumption that this will be done from my src-block
handling function in the derived backend:
#+BEGIN_SRC emacs-lisp
(defun my-html-src-block (src-block contents info)
  ...)

(org-export-define-derived-backend 'my-html 'html
  :translate-alist '((src-block . my-html-src-block)))
#+END_SRC

I run the following code to perform the export using my new backend:
#+BEGIN_SRC emacs-lisp
(org-export-to-file 'my-html (buffer-file-name))
#+END_SRC

*Question 1*: Is the `org-element-src-block-parser' function supposed to resolve
 the tangle filename (including inheritance, i.e. to handle the 3 :tangle
 sources listed above)?

From the docstring, it would appear that it shouldn't return a :tangle keyword
and that the tangling functions are responsible for choosing the output tangle
file for each block.  Is this correct?

Hoping that my understanding of *Question 1* is correct, I tried to use the
`org-babel-tangle-single-block' and `org-babel-tangle-collect-blocks' functions,
which are used during the actual tangling, to get the tangle filename.

** Working version

Here is what I was able to get to work on my simple example:

#+BEGIN_SRC emacs-lisp

(defun my-html-src-block (src-block contents info)
  "Transcode a SRC-BLOCK element from Org to HTML.
     CONTENTS is nil.  INFO is a plist used as a communication
     channel."
  (let* ((src-block-name (org-element-property :name src-block))
         (src-blocks
          (cdr (car (org-babel-tangle-collect-blocks))))
         tang
         (export-out (org-export-with-backend 'html src-block contents info)))
    (dolist (src-coll src-blocks)
      (when (string= (nth 3 src-coll) src-block-name)
        (setq tang (cdr (assoc :tangle (nth 4 src-coll))))))
    (if (and tang (> (length tang) 0) (not (string= tang "no")))
        (let ((src-start-pat "\\(<pre class=\"src src-[^>]+>\\)"))
          (setq export-out
                (replace-regexp-in-string
                 src-start-pat
                 (concat "\\1<div style=\"position: absolute; bottom: 0;"
                         " right: 0; text-align:right;\" class=\"src-label\">"
                         tang "</div>") export-out))))
      export-out))

#+END_SRC

To summarize:
- Collect all the tangled blocks
- Among these, find the one currently passed as input (match by source block
  name)
- Get the :tangle property from the matching collected source-block

The main problem I see which this approach is that it collects all the source
blocks for each src-block entry to process.  This makes the exporting process
slow.

Another issue is that this requires named blocks, but this constraint is
acceptable for me.


** Improvements (which I cannot get to work)

- Add collected blocks to `info' in order to perform the collection only once
  and re-use it for subsequent blocks: I couldn't get it to work, function
  arguments seem to be passed by value (?)
- Use `org-babel-tangle-single-block' to process only one block at a time, this
  sounds like the right method.  I believe I must first move point to the
  src-block being processed, which I try to do with
  (org-babel-goto-named-src-block src-block-name): this gives me the correct
  tangle filename when run from a simple code snippet in the scratch buffer
  (using (org-element-map (org-element-parse-buffer) ...)), but results in the
  wrong tangle file when I do that within the `my-html-src-block' function (the
  :tangle options passed on the #+BEGIN_SRC line as in case 3. are ignored and
  are always preempted by the inherited tangle value coming either from the
  ,#+PROPERTY entry or the PROPERTY drawer value).  *Question 2*: How can I use
  `org-babel-tangle-single-block' within the custom src-block handling function?
  I notice the (point) value is different after the
  (org-babel-goto-named-src-block src-block-name) call when used from the
  `my-html-src-block' function (possibly due to some pre-processing performed on
  the buffer, I am not sure).

Sorry for the lengthy post, I hope it is not out of place on this list.  Thanks
in advance for any help.

thibault

* Original message

> Hi list,
>
> I have an org file that I am tangling into multiple files, and exporting
> to html.  What I would like to do is to label each source block in the
> exported html with the filename used for tangling this specific block.
> I don't have a strong opinion about the actual appearance of the label
> (adding a comment at the top of the source block, hover in the code
> textarea, other?).
>
> I couldn't find any option to do that, so I initially though of using
> `org-export-filter-src-block-functions' (following
> http://orgmode.org/manual/Advanced-configuration.html).  I was not able
> to get the org-element object for the current block from the parameter
> received by the filter.  It looks like the third parameter (`info') may
> have more information but I currently don't see how to get (1) the
> current source block, and (2) the tangling filename.
>
> I then tried to define a derived backend with custom handling of source
> blocks as described in the documentation.
>
> I currently have the following, where I try to get the tangle filename
> using `org-babel-get-src-block-info', and later insert it into the html
> content using an ugly regexp.
>
> (defun my-html-src-block (src-block contents info)
>   "Transcode a SRC-BLOCK element from Org to HTML.
>      CONTENTS is nil.  INFO is a plist used as a communication
>      channel."
>   (let* ((lang (org-element-property :language src-block))
>          (src-info (org-babel-get-src-block-info t src-block))
>          (tang (cdr (assq :tangle (nth 2 src-info))))
>          (export-out (org-export-with-backend 'html src-block contents info))
>          )
>     (if (and (string= lang "lua") (and tang (> (length tang) 0)))
>         (progn
>           (let ((src-start-pat "\\(<pre class=\"src src-lua\" id=\"[^\"]+*\">\\)"))
>             (replace-regexp-in-string src-start-pat
>                                       (concat "\\1"
>                                               "\n-- ["
>                                               tang
>                                               "]\n\n") export-out)
>             )
>           )
>       export-out
>       )
>     )
>   )
>
> (org-export-define-derived-backend 'my-html 'html
>   :translate-alist '((src-block . my-html-src-block)))
>
> Besides feeling wrong, this always gets the tangle name from the top
> level org option ("#+PROPERTY: tangle main-file" at the top of the file)
> instead of the one overridden by individual blocks "#+BEGIN_SRC :tangle
> other-file".  Moreover the added comment is not syntax highlighted and
> this feels really wrong.
>
> Does anybody have any idea on how to achieve this?
>
> Thanks in advance.
>
> thibault

Re: Export tangle filename with source block

[Nicolas Goaziou]

Hello,

Thibault Marin <thibault.marin@gmx.com> writes:

> *Question 1*: Is the `org-element-src-block-parser' function supposed to resolve
>  the tangle filename (including inheritance, i.e. to handle the 3 :tangle
>  sources listed above)?

No, it isn't.

> From the docstring, it would appear that it shouldn't return a :tangle keyword
> and that the tangling functions are responsible for choosing the output tangle
> file for each block.  Is this correct?

That's correct.

> ** Improvements (which I cannot get to work)
>
> - Add collected blocks to `info' in order to perform the collection only once
>   and re-use it for subsequent blocks: I couldn't get it to work, function
>   arguments seem to be passed by value (?)

They are not. You may want to see how cache is used with info in, e.g.,
`org-export-get-footnote-definition'.

> - Use `org-babel-tangle-single-block' to process only one block at a time, this
>   sounds like the right method.  I believe I must first move point to the
>   src-block being processed, which I try to do with
>   (org-babel-goto-named-src-block src-block-name): this gives me the correct
>   tangle filename when run from a simple code snippet in the scratch buffer
>   (using (org-element-map (org-element-parse-buffer) ...)), but results in the
>   wrong tangle file when I do that within the `my-html-src-block' function (the
>   :tangle options passed on the #+BEGIN_SRC line as in case 3. are ignored and
>   are always preempted by the inherited tangle value coming either from the
>   ,#+PROPERTY entry or the PROPERTY drawer value).  *Question 2*: How can I use
>   `org-babel-tangle-single-block' within the custom src-block handling function?
>   I notice the (point) value is different after the
>   (org-babel-goto-named-src-block src-block-name) call when used from the
>   `my-html-src-block' function (possibly due to some pre-processing performed on
>   the buffer, I am not sure).

The buffer used during export is unlikely to be the same as the original
one. Macros are expanded, comments are removed and Babel code is
possibly evaluated. The only hook run in an exact copy of the original
buffer is `org-export-before-processing-hook'. You can collect anything
here, but INFO doesn't exist yet.

Regards,

-- 
Nicolas Goaziou

Re: Export tangle filename with source block

[Thibault Marin]

Thanks for your reply, it is very helpful.

Nicolas Goaziou writes:
> They are not. You may want to see how cache is used with info in, e.g.,
> `org-export-get-footnote-definition'.
That looks like what I was trying to achieve (except better), thanks.

> The buffer used during export is unlikely to be the same as the original
> one. Macros are expanded, comments are removed and Babel code is
> possibly evaluated. The only hook run in an exact copy of the original
> buffer is `org-export-before-processing-hook'. You can collect anything
> here, but INFO doesn't exist yet.

OK, then it looks like I may be able to build a list of source block
name/tangle filename pairs on pre-processing, store it in a global
variable and use it when processing source-blocks.  It is probably a
little hack-ish but that would be fine for me.  On my initial attempt,
`org-babel-tangle-collect-blocks' seems to be skipping blocks on one of
my tests, but I haven't spent much time on it yet, I'll report back.

Thanks again for the help.
Best,
thibault

Re: Export tangle filename with source block

[Thibault Marin]

Thibault Marin writes:
> OK, then it looks like I may be able to build a list of source block
> name/tangle filename pairs on pre-processing, store it in a global
> variable and use it when processing source-blocks.  It is probably a
> little hack-ish but that would be fine for me.  On my initial attempt,
> `org-babel-tangle-collect-blocks' seems to be skipping blocks on one of
> my tests, but I haven't spent much time on it yet, I'll report back.
It's now working, thanks again for the guidance.

thibault