Feature Request: Allow Tangling Named INCLUDEs

Feature Request: Allow Tangling Named INCLUDEs

[Nick Daly]

Hi folks, I have a situation where I have many named code snippets
currently available in external files, and would like to tangle them
into an Org mode file.  Is it currently possible to tangle included
files?  If not, could that feature be added to a new version of Org
mode?  I've looked through the mailing list history a bit and couldn't
find this specific request.

- What exactly did you do? :: I created the three files shown below.
     I then tangled the searches.org file (~C-c C-v C-t~).

     For convenience, all three files can be created by tangling this
     email.

- What did you expect to happen? :: I expected the contents of the files
     searchItems(8|12).m to appear in lines [[search-08][(search-08)]]
     and [[search-12][(search-12)]].

- What happened instead? :: The templates on lines
     [[search-08][(search-08)]] and [[search-12][(search-12)]] were
     empty.

Thanks for your time,
Nick

----

If you don't understand the code below, that's because it's MUMPS.  Be
happy that you don't understand.

#+begin_src org :noweb tangle :tangle searches.org -s -n :padline no
  ,# -*- mode: org -*-
  ,* Searches

  ,#+name: search-main
  ,#+begin_src text :noweb tangle :tangle searchAllItems.m
  ,  searches(pId)
  ,    n pDate,i08,i12
  ,    f  pDate=$$idForwardDate(pId,pDate) q:pDate=""  d
  ,    . <<searchItem08>>          ; (ref:search-08)
  ,    . <<searchItem12>>          ; (ref:search-12)
  ,    . ; and about 30 more of these named searches.
  ,#+end_src

  ,#+name: searchItem08
  ,#+INCLUDE: "searchItem08.m" src text

  ,#+name: searchItem12
  ,#+INCLUDE: "searchItem12.m" src text

#+end_src

#+begin_src text :tangle searchItem08.m :padline no
  s i08=$$zgetnp(pId,pDate,08) i i08="" d LogIcErr("i08",i08)
#+end_src

#+begin_src text :tangle searchItem12.m :padline no
  s i12=$$zgetnp(pId,pDate,12) i $$zgetnp(pId,pDate,$$zgetnp(pId,pDate,i12))'=i12 d LogIcErr("i12",i12)
#+end_src

Re: Feature Request: Allow Tangling Named INCLUDEs

[Bastien]

Hi Nick,

Nick Daly <nick.m.daly@gmail.com> writes:

> Hi folks, I have a situation where I have many named code snippets
> currently available in external files, and would like to tangle them
> into an Org mode file.  Is it currently possible to tangle included
> files?  

I just wanted to confirm this is not currently possible...

> If not, could that feature be added to a new version of Org
> mode?

I'm not the one who would undertake this, so my opinion on this should
not matter much... but I think #+INCLUDE is well suited for exporting,
not for tangling.  I'm not able to say why, this is just an intuition.

-- 
 Bastien

Re: Feature Request: Allow Tangling Named INCLUDEs

[Nick Daly]

[-- Attachment #1: Type: text/plain, Size: 826 bytes --]

Hi Bastien,

> Nick Daly wrote:
>
>> Hi folks, I have a situation where I have many named code snippets
>> currently available in external files, and would like to tangle them
>> into an Org mode file.

Bastien writes:
> I'm not the one who would undertake this, so my opinion on this should
> not matter much... but I think #+INCLUDE is well suited for exporting,
> not for tangling.  I'm not able to say why, this is just an intuition.

It seems to me like it's trying to unify exporting and tangling: this
emphasizes that tangling is just another (very specific plain-text) form
of exporting.  Using Eric's function for exporting allows me to remove
lots of boilerplate code from the source org file and keep the org
source better focused on the code's function and structure.

Sure it's weird, but it sure is handy.

Nick

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Feature Request: Allow Tangling Named INCLUDEs

[Rainer M Krug]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On 01/27/14, 01:44 , Nick Daly wrote:
> Hi Bastien,
> 
>> Nick Daly wrote:
>> 
>>> Hi folks, I have a situation where I have many named code
>>> snippets currently available in external files, and would like
>>> to tangle them into an Org mode file.
> 
> Bastien writes:
>> I'm not the one who would undertake this, so my opinion on this
>> should not matter much... but I think #+INCLUDE is well suited
>> for exporting, not for tangling.  I'm not able to say why, this
>> is just an intuition.
> 
> It seems to me like it's trying to unify exporting and tangling:
> this emphasizes that tangling is just another (very specific
> plain-text) form of exporting.

I agree here - I also see tangling as just a different export backend.

I must say I like the idea of having #+INCLUDE expanded before export,
as it would make using code libraries much easier. But one could even
go one step further: not only include the complete file, but only a
named source block - that would be perfect.

Just wondering - wouldn't it be possible, to just put the function
org-export-expand-include-keyword into the pre-tangle-hook?

Cheers,

Rainer

> Using Eric's function for exporting allows me to remove lots of
> boilerplate code from the source org file and keep the org source
> better focused on the code's function and structure.
> 
> Sure it's weird, but it sure is handy.
> 
> Nick
> 

- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :       +33 - (0)9 53 10 27 44
Cell:       +33 - (0)6 85 62 59 98
Fax :       +33 - (0)9 58 10 27 44

Fax (D):    +49 - (0)3 21 21 25 22 44

email:      Rainer@krugs.de

Skype:      RMkrug
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJS5iZfAAoJENvXNx4PUvmCLpQIALHVxBdDxLDfzCvpedCmnWTr
rwMaYhNzfONCeINHuEmBiu3elhsRTAag1VgEIEnPZYatEUBK5gqB19QPKdyFuRmT
J9iYBVQRV6zPmdXq24u/e/ArymXrylov739OMXWV16ITKZaWoMmvWmStGd0HISb3
pID2+HziRUzVMMXRWiGK/uDw1iZrbSE17e5hI9AlUqO+IR9TvFH6aATZAjx0pQ8B
lyXoN4H5QjYGZfLUPCQYJt3GA8TStpTdEBrNC5iIvPmIuBhQrWT1xKV3vWLLv47k
hw/1YIl/nA++20jE3FrstRs4pJFX4OOBgKeuPWR/OUj2T/Jg5xOKAIVLE15V9p4=
=t01U
-----END PGP SIGNATURE-----

Re: Feature Request: Allow Tangling Named INCLUDEs

[Eric Schulte]

Rainer M Krug <Rainer@krugs.de> writes:

> On 01/27/14, 01:44 , Nick Daly wrote:
>> Hi Bastien,
>>
>>> Nick Daly wrote:
>>>
>>>> Hi folks, I have a situation where I have many named code
>>>> snippets currently available in external files, and would like
>>>> to tangle them into an Org mode file.
>>
>> Bastien writes:
>>> I'm not the one who would undertake this, so my opinion on this
>>> should not matter much... but I think #+INCLUDE is well suited
>>> for exporting, not for tangling.  I'm not able to say why, this
>>> is just an intuition.
>>
>> It seems to me like it's trying to unify exporting and tangling:
>> this emphasizes that tangling is just another (very specific
>> plain-text) form of exporting.
>
> I agree here - I also see tangling as just a different export backend.
>
> I must say I like the idea of having #+INCLUDE expanded before export,
> as it would make using code libraries much easier. But one could even
> go one step further: not only include the complete file, but only a
> named source block - that would be perfect.
>
> Just wondering - wouldn't it be possible, to just put the function
> org-export-expand-include-keyword into the pre-tangle-hook?
>

Why don't you try it out and let us know.

>
> Cheers,
>
> Rainer
>
>> Using Eric's function for exporting allows me to remove lots of
>> boilerplate code from the source org file and keep the org source
>> better focused on the code's function and structure.
>>
>> Sure it's weird, but it sure is handy.
>>
>> Nick
>>
>
> --
> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
> Biology, UCT), Dipl. Phys. (Germany)
>
> Centre of Excellence for Invasion Biology
> Stellenbosch University
> South Africa
>
> Tel :       +33 - (0)9 53 10 27 44
> Cell:       +33 - (0)6 85 62 59 98
> Fax :       +33 - (0)9 58 10 27 44
>
> Fax (D):    +49 - (0)3 21 21 25 22 44
>
> email:      Rainer@krugs.de
>
> Skype:      RMkrug

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: Feature Request: Allow Tangling Named INCLUDEs

[Rainer M Krug]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On 01/27/14, 15:37 , Eric Schulte wrote:
> Rainer M Krug <Rainer@krugs.de> writes:
> 
>> On 01/27/14, 01:44 , Nick Daly wrote:
>>> Hi Bastien,
>>> 
>>>> Nick Daly wrote:
>>>> 
>>>>> Hi folks, I have a situation where I have many named code 
>>>>> snippets currently available in external files, and would
>>>>> like to tangle them into an Org mode file.
>>> 
>>> Bastien writes:
>>>> I'm not the one who would undertake this, so my opinion on
>>>> this should not matter much... but I think #+INCLUDE is well
>>>> suited for exporting, not for tangling.  I'm not able to say
>>>> why, this is just an intuition.
>>> 
>>> It seems to me like it's trying to unify exporting and
>>> tangling: this emphasizes that tangling is just another (very
>>> specific plain-text) form of exporting.
>> 
>> I agree here - I also see tangling as just a different export
>> backend.
>> 
>> I must say I like the idea of having #+INCLUDE expanded before
>> export, as it would make using code libraries much easier. But
>> one could even go one step further: not only include the complete
>> file, but only a named source block - that would be perfect.
>> 
>> Just wondering - wouldn't it be possible, to just put the
>> function org-export-expand-include-keyword into the
>> pre-tangle-hook?
>> 
> 
> Why don't you try it out and let us know.

OK - did it.

It expands the #+INCLUDE in the actual org buffer, so they are
expanded after the tangleing, which is not exactly what one wants.

Cheers,

Rainer
> 
>> 
>> Cheers,
>> 
>> Rainer
>> 
>>> Using Eric's function for exporting allows me to remove lots
>>> of boilerplate code from the source org file and keep the org
>>> source better focused on the code's function and structure.
>>> 
>>> Sure it's weird, but it sure is handy.
>>> 
>>> Nick
>>> 
>> 
>> -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc
>> (Conservation Biology, UCT), Dipl. Phys. (Germany)
>> 
>> Centre of Excellence for Invasion Biology Stellenbosch
>> University South Africa
>> 
>> Tel :       +33 - (0)9 53 10 27 44 Cell:       +33 - (0)6 85 62
>> 59 98 Fax :       +33 - (0)9 58 10 27 44
>> 
>> Fax (D):    +49 - (0)3 21 21 25 22 44
>> 
>> email:      Rainer@krugs.de
>> 
>> Skype:      RMkrug
> 

- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :       +33 - (0)9 53 10 27 44
Cell:       +33 - (0)6 85 62 59 98
Fax :       +33 - (0)9 58 10 27 44

Fax (D):    +49 - (0)3 21 21 25 22 44

email:      Rainer@krugs.de

Skype:      RMkrug
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJS52ZsAAoJENvXNx4PUvmCoDMIALThdsiEVeTS+19WAQftcJNu
lW5e210lpMv3KOT4FmP1dKC7Js2sX8U9uc7XsgRfgmz+2jtNKfaCv5ZpqUx2Hwcp
qFtYiojX1N4PrgbTjbJMsbGfM68UaFt9G+p02WtsZRgJY0OSK0TmcJ+OiHx7ejFf
juR/I6ZmJfpgoeLFilKN4RlNk6wVQ2mzvMxFqDMJiTPjdT07zXPCW+RJX9HPZbC2
tCGMrUvaUIM20idVmnN86KEMwtWyw2kqGQYl5yMboVa7mbALOx2SHiDjukmmaVTh
ocwYnAcZxyhPesO7t6jAS7ZVaYZxRpLTGaj2yhShM0XI/T5fCy0jlF07vtvhrcw=
=DXxi
-----END PGP SIGNATURE-----

Re: Feature Request: Allow Tangling Named INCLUDEs

[Eric Schulte]

Nick Daly <nick.m.daly@gmail.com> writes:

> Hi Bastien,
>
>> Nick Daly wrote:
>>
>>> Hi folks, I have a situation where I have many named code snippets
>>> currently available in external files, and would like to tangle them
>>> into an Org mode file.
>
> Bastien writes:
>> I'm not the one who would undertake this, so my opinion on this should
>> not matter much... but I think #+INCLUDE is well suited for exporting,
>> not for tangling.  I'm not able to say why, this is just an intuition.
>
> It seems to me like it's trying to unify exporting and tangling: this
> emphasizes that tangling is just another (very specific plain-text) form
> of exporting.

I'm not sure tangling is naturally an exporting procedure (unlike
weaving which is exactly exporting).  Either way re-implementing
tangling inside of the exporting seems like a large amount of work with
little payoff (unless there are other export features which I'm missing
which would be useful for tangling).

> Using Eric's function for exporting allows me to remove lots of
> boilerplate code from the source org file and keep the org source
> better focused on the code's function and structure.
>

Great.

Best,

>
> Sure it's weird, but it sure is handy.
>
> Nick

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: Feature Request: Allow Tangling Named INCLUDEs

[Eric Schulte]

Nick Daly <nick.m.daly@gmail.com> writes:

> Hi folks, I have a situation where I have many named code snippets
> currently available in external files, and would like to tangle them
> into an Org mode file.  Is it currently possible to tangle included
> files?  If not, could that feature be added to a new version of Org
> mode?  I've looked through the mailing list history a bit and couldn't
> find this specific request.
>
> - What exactly did you do? :: I created the three files shown below.
>      I then tangled the searches.org file (~C-c C-v C-t~).
>
>      For convenience, all three files can be created by tangling this
>      email.
>
> - What did you expect to happen? :: I expected the contents of the files
>      searchItems(8|12).m to appear in lines [[search-08][(search-08)]]
>      and [[search-12][(search-12)]].
>
> - What happened instead? :: The templates on lines
>      [[search-08][(search-08)]] and [[search-12][(search-12)]] were
>      empty.
>

The following function works for me on a simple example file.

    ;; -*- emacs-lisp -*-
    (defun trial-expand-and-tangle ()
      (interactive)
      (save-window-excursion
        (let ((text (buffer-string))
              (org-babel-pre-tangle-hook nil))
          (unwind-protect
              (progn
                (org-export-expand-include-keyword)
                (org-babel-tangle))
            (delete-region (point-min) (point-max))
            (goto-char (point-min))
            (insert text)))))

I'm not sure if this is common enough to have functionality added to the
core.  Maybe adding a `with-expanded-includes' macro would be useful
though.

Best,

>
> Thanks for your time,
> Nick
>
> ----
>
> If you don't understand the code below, that's because it's MUMPS.  Be
> happy that you don't understand.
>
> #+begin_src org :noweb tangle :tangle searches.org -s -n :padline no
>   ,# -*- mode: org -*-
>   ,* Searches
>
>   ,#+name: search-main
>   ,#+begin_src text :noweb tangle :tangle searchAllItems.m
>   ,  searches(pId)
>   ,    n pDate,i08,i12
>   ,    f  pDate=$$idForwardDate(pId,pDate) q:pDate=""  d
>   ,    . <<searchItem08>>          ; (ref:search-08)
>   ,    . <<searchItem12>>          ; (ref:search-12)
>   ,    . ; and about 30 more of these named searches.
>   ,#+end_src
>
>   ,#+name: searchItem08
>   ,#+INCLUDE: "searchItem08.m" src text
>
>   ,#+name: searchItem12
>   ,#+INCLUDE: "searchItem12.m" src text
>
> #+end_src
> #+begin_src text :tangle searchItem08.m :padline no
>   s i08=$$zgetnp(pId,pDate,08) i i08="" d LogIcErr("i08",i08)
> #+end_src
> #+begin_src text :tangle searchItem12.m :padline no
>   s i12=$$zgetnp(pId,pDate,12) i $$zgetnp(pId,pDate,$$zgetnp(pId,pDate,i12))'=i12 d LogIcErr("i12",i12)
> #+end_src

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: Feature Request: Allow Tangling Named INCLUDEs

[Nick Daly]

[-- Attachment #1: Type: text/plain, Size: 754 bytes --]

> Nick Daly wrote:
>
>> Hi folks, I have a situation where I have many named code snippets
>> currently available in external files, and would like to tangle them
>> into an Org mode file.

Eric Schulte <schulte.eric@gmail.com> writes:

> The following function works for me on a simple example file...

Thank you, that function worked perfectly!  It's now found a new home in
my .emacs file.

> I'm not sure if this is common enough to have functionality added to the
> core.  Maybe adding a `with-expanded-includes' macro would be useful
> though.

It probably isn't very common at all, but its incredibly handy to keep
the size of the org-mode file down: it keeps the literate source
readable and free of boilerplate code.

Thanks for your help,
Nick

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Feature Request: Allow Tangling Named INCLUDEs

[Eric Schulte]

Nick Daly <nick.m.daly@gmail.com> writes:

> Hi folks, I have a situation where I have many named code snippets
> currently available in external files, and would like to tangle them
> into an Org mode file.  Is it currently possible to tangle included
> files?  If not, could that feature be added to a new version of Org
> mode?  I've looked through the mailing list history a bit and couldn't
> find this specific request.
>
> - What exactly did you do? :: I created the three files shown below.
>      I then tangled the searches.org file (~C-c C-v C-t~).
>
>      For convenience, all three files can be created by tangling this
>      email.
>
> - What did you expect to happen? :: I expected the contents of the files
>      searchItems(8|12).m to appear in lines [[search-08][(search-08)]]
>      and [[search-12][(search-12)]].
>
> - What happened instead? :: The templates on lines
>      [[search-08][(search-08)]] and [[search-12][(search-12)]] were
>      empty.
>

The following function works for me on a simple example file.

    ;; -*- emacs-lisp -*-
    (defun trial-expand-and-tangle ()
      (interactive)
      (save-window-excursion
        (let ((text (buffer-string))
              (org-babel-pre-tangle-hook nil))
          (unwind-protect
              (progn
                (org-export-expand-include-keyword)
                (org-babel-tangle))
            (delete-region (point-min) (point-max))
            (goto-char (point-min))
            (insert text)))))

I'm not sure if this is common enough to have functionality added to the
core.  Maybe adding a `with-expanded-includes' macro would be useful
though.

Best,

>
> Thanks for your time,
> Nick
>
> ----
>
> If you don't understand the code below, that's because it's MUMPS.  Be
> happy that you don't understand.
>
> #+begin_src org :noweb tangle :tangle searches.org -s -n :padline no
>   ,# -*- mode: org -*-
>   ,* Searches
>
>   ,#+name: search-main
>   ,#+begin_src text :noweb tangle :tangle searchAllItems.m
>   ,  searches(pId)
>   ,    n pDate,i08,i12
>   ,    f  pDate=$$idForwardDate(pId,pDate) q:pDate=""  d
>   ,    . <<searchItem08>>          ; (ref:search-08)
>   ,    . <<searchItem12>>          ; (ref:search-12)
>   ,    . ; and about 30 more of these named searches.
>   ,#+end_src
>
>   ,#+name: searchItem08
>   ,#+INCLUDE: "searchItem08.m" src text
>
>   ,#+name: searchItem12
>   ,#+INCLUDE: "searchItem12.m" src text
>
> #+end_src
> #+begin_src text :tangle searchItem08.m :padline no
>   s i08=$$zgetnp(pId,pDate,08) i i08="" d LogIcErr("i08",i08)
> #+end_src
> #+begin_src text :tangle searchItem12.m :padline no
>   s i12=$$zgetnp(pId,pDate,12) i $$zgetnp(pId,pDate,$$zgetnp(pId,pDate,i12))'=i12 d LogIcErr("i12",i12)
> #+end_src

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D