Literate Programming with Org mode

Literate Programming with Org mode

[Sébastien Vauban]

Hi Eric and all,

Here some promised description of how I'm using Literate Programming with
LaTeX (up to now -- soon directly from Org mode?).

I write an "enhanced LaTeX" file ([1], having the `.nw' -- for Nuweb --
extension). It is enhanced as I can put blocks of code in there (SQL blocks,
for example), and give them a name for further referencing.

Then, at some point in the file, I say how I want the code files to look like,
by assembling the blocks in a certain order, with some glue.

For example:

--8<---------------cut here---------------start------------->8---
<<Enterprise.sql>>=
<<sql-init>>
SELECT abcID, etpID, etpAssurATPolNum
FROM enterprise JOIN record
    ON (etpAbcID_fk = abcID)
WHERE etpAbcID_fk
<<sql-cond>>
@ %def 
--8<---------------cut here---------------end--------------->8---

The above file is made up of two defined blocks (`sql-init' and `sql-cond')
plus some glue in between.

The good thing is I can reuse the same blocks in other context, completely
avoiding copy/paste of code:

--8<---------------cut here---------------start------------->8---
<<Lessons.sql>>=
<<sql-init>>
SELECT abcID, lesAlternNbrSem, lesNbrSem,
       CONVERT(varchar(10), lesDateDeb, @dateFmtStyleOut) AS lesDateDeb,
       CONVERT(varchar(10), lesDateFin, @dateFmtStyleOut) AS lesDateFin
FROM lessons JOIN record
    ON (lesAbcID_fk = abcID)
WHERE lesAbcID_fk
<<sql-cond>>
@ %def 
--8<---------------cut here---------------end--------------->8---

OK. So, my `org-lit-prog.nw' file is ready, and committed under Subversion
(this is the only one I'm committing as it contains both the documentation of
the code, and the source code itself).

Then, I just type (in a terminal):

--8<---------------cut here---------------start------------->8---
noweb org-lit-prog.nw
--8<---------------cut here---------------end--------------->8---

to extract both:

    o   the LaTeX file [2] to be compiled into a PDF [3],
    o   the different source code files (Enterprise.sql [4], Lessons.sql [5],
        [6] and [7])

Just that easy to keep code and doc in sync, and get a very nice-to-read
documentation of the code.

Sometimes, I play an extra trick is made in the noweb file: I put blocks of
static code (for which I don't especially need any noweb feature) after the
end of the LaTeX document. That way, it gets outputted by noweb as a source
file during the Tangle process, and included in my LaTeX document via the
listings package.

Why that trick?  I could directly put the code in the body of the document!
Yes, but I want the code to be highlighted via listings. Otherwise, it's just
black and white display of the code...

For easy access, I've put all the above files on a my Web space:

[1] http://www.mygooglest.com/sva/org-lit-prog.nw
[2] http://www.mygooglest.com/sva/org-lit-prog.tex
[3] http://www.mygooglest.com/sva/org-lit-prog.pdf
[4] http://www.mygooglest.com/sva/Enterprise.sql
[5] http://www.mygooglest.com/sva/Lessons.sql
[6] http://www.mygooglest.com/sva/Payment.sql
[7] http://www.mygooglest.com/sva/prsNumNat.awk

From what I understand, what we both wanna reach is writing all of the `.nw'
file as an Org file directly, and let Org-babel extracts the source code files
and the documentation out of it.

What should be good in the future, when it will be ready for prime time, is
that this becomes complete part of the export process: no tangle operation
needed.

Is the above description worth for you?  Do you understand how I work when
documenting code?  Pay attention: I'm *the* expert in literate programming.
Just an amateur being seduced by some of its features.

Best regards,
  Seb

-- 
Sébastien Vauban



_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Literate Programming with Org mode

[Marcelo de Moraes Serpa]

Hmm, this seems interesting and something I'm looking for, but I'm not
entirely sure what you mean by "literate programming" (I haven't
googled yet either, I'm about to do it once I answer this message).
Anyway, I have a project I'm working on right now that requires me to
set up a "publishing environment", in the sense that I might have
source files (maybe in XML or whatever format makes sense) and be able
to convert them to LaTeX (for pdf), html or whatever format I could
think of, using XSLT, something like the "pragmatic programmers"
publishing toolchain. Not sure if literate programming has anything to
do with that :)

2009/7/28 Sébastien Vauban <wxhgmqzgwmuf@spammotel.com>:
> Hi Eric and all,
>
> Here some promised description of how I'm using Literate Programming with
> LaTeX (up to now -- soon directly from Org mode?).
>
> I write an "enhanced LaTeX" file ([1], having the `.nw' -- for Nuweb --
> extension). It is enhanced as I can put blocks of code in there (SQL blocks,
> for example), and give them a name for further referencing.
>
> Then, at some point in the file, I say how I want the code files to look like,
> by assembling the blocks in a certain order, with some glue.
>
> For example:
>
> --8<---------------cut here---------------start------------->8---
> <<Enterprise.sql>>=
> <<sql-init>>
> SELECT abcID, etpID, etpAssurATPolNum
> FROM enterprise JOIN record
>    ON (etpAbcID_fk = abcID)
> WHERE etpAbcID_fk
> <<sql-cond>>
> @ %def
> --8<---------------cut here---------------end--------------->8---
>
> The above file is made up of two defined blocks (`sql-init' and `sql-cond')
> plus some glue in between.
>
> The good thing is I can reuse the same blocks in other context, completely
> avoiding copy/paste of code:
>
> --8<---------------cut here---------------start------------->8---
> <<Lessons.sql>>=
> <<sql-init>>
> SELECT abcID, lesAlternNbrSem, lesNbrSem,
>       CONVERT(varchar(10), lesDateDeb, @dateFmtStyleOut) AS lesDateDeb,
>       CONVERT(varchar(10), lesDateFin, @dateFmtStyleOut) AS lesDateFin
> FROM lessons JOIN record
>    ON (lesAbcID_fk = abcID)
> WHERE lesAbcID_fk
> <<sql-cond>>
> @ %def
> --8<---------------cut here---------------end--------------->8---
>
> OK. So, my `org-lit-prog.nw' file is ready, and committed under Subversion
> (this is the only one I'm committing as it contains both the documentation of
> the code, and the source code itself).
>
> Then, I just type (in a terminal):
>
> --8<---------------cut here---------------start------------->8---
> noweb org-lit-prog.nw
> --8<---------------cut here---------------end--------------->8---
>
> to extract both:
>
>    o   the LaTeX file [2] to be compiled into a PDF [3],
>    o   the different source code files (Enterprise.sql [4], Lessons.sql [5],
>        [6] and [7])
>
> Just that easy to keep code and doc in sync, and get a very nice-to-read
> documentation of the code.
>
> Sometimes, I play an extra trick is made in the noweb file: I put blocks of
> static code (for which I don't especially need any noweb feature) after the
> end of the LaTeX document. That way, it gets outputted by noweb as a source
> file during the Tangle process, and included in my LaTeX document via the
> listings package.
>
> Why that trick?  I could directly put the code in the body of the document!
> Yes, but I want the code to be highlighted via listings. Otherwise, it's just
> black and white display of the code...
>
> For easy access, I've put all the above files on a my Web space:
>
> [1] http://www.mygooglest.com/sva/org-lit-prog.nw
> [2] http://www.mygooglest.com/sva/org-lit-prog.tex
> [3] http://www.mygooglest.com/sva/org-lit-prog.pdf
> [4] http://www.mygooglest.com/sva/Enterprise.sql
> [5] http://www.mygooglest.com/sva/Lessons.sql
> [6] http://www.mygooglest.com/sva/Payment.sql
> [7] http://www.mygooglest.com/sva/prsNumNat.awk
>
> From what I understand, what we both wanna reach is writing all of the `.nw'
> file as an Org file directly, and let Org-babel extracts the source code files
> and the documentation out of it.
>
> What should be good in the future, when it will be ready for prime time, is
> that this becomes complete part of the export process: no tangle operation
> needed.
>
> Is the above description worth for you?  Do you understand how I work when
> documenting code?  Pay attention: I'm *the* expert in literate programming.
> Just an amateur being seduced by some of its features.
>
> Best regards,
>  Seb
>
> --
> Sébastien Vauban
>
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>

Re: Literate Programming with Org mode

[Sébastien Vauban]

Hi Marcelo,

Marcelo de Moraes Serpa wrote:
> Sébastien Vauban wrote:
>> Hi Eric and all,
>>
>> Here some promised description of how I'm using Literate Programming with
>> LaTeX (up to now -- soon directly from Org mode?).
>>
>> I write an "enhanced LaTeX" file ([1], having the `.nw' -- for Nuweb --
>> extension). It is enhanced as I can put blocks of code in there (SQL blocks,
>> for example), and give them a name for further referencing.
>>
>> Then, at some point in the file, I say how I want the code files to look like,
>> by assembling the blocks in a certain order, with some glue.
>>
>> [...]
>>
>> OK. So, my `org-lit-prog.nw' file is ready, and committed under Subversion
>> (this is the only one I'm committing as it contains both the documentation of
>> the code, and the source code itself).
>>
>> Then, I just type `noweb org-lit-prog.nw' to extract both:
>>
>>    o   the LaTeX file [2] to be compiled into a PDF [3],
>>    o   the different source code files (Enterprise.sql [4], Lessons.sql [5],
>>        [6] and [7])
>>
>> Just that easy to keep code and doc in sync, and get a very nice-to-read
>> documentation of the code.
>>
>> [...]
>>
>> Is the above description worth for you?  Do you understand how I work when
>> documenting code?  Pay attention: I'm *the* expert in literate programming.

Euh... Euh... Euh... I meant: I am _NOT_ *the* expert in literate programming.
Just a user of the approach... Sorry for misbehaving ;-)


>> Just an amateur being seduced by some of its features.
>
> Hmm, this seems interesting and something I'm looking for, but I'm not
> entirely sure what you mean by "literate programming" (I haven't
> googled yet either, I'm about to do it once I answer this message).

If I would have to summarize what it is in a sentence, it would be this: LP
(not logic programming) is about *putting the code in the documentation* -- at
the opposite of the current mainstream approach where one puts the
documentation inside the code (where there is documentation!).

I've heard that Knuth told about it in those words: it's when we will be able
to read the code of a software in our bed, reading a book made of 90% of
documentation and 10% of code. If someone can find this phrase somewhere...


> Anyway, I have a project I'm working on right now that requires me to
> set up a "publishing environment", in the sense that I might have
> source files (maybe in XML or whatever format makes sense) and be able
> to convert them to LaTeX (for pdf), html or whatever format I could
> think of, using XSLT, something like the "pragmatic programmers"
> publishing toolchain. Not sure if literate programming has anything to
> do with that :)

As far as I can understand, yes, as you're talking of inserting source code
(of whatever language) in your big documentation of the project. That's the
philosophy, yes, *iff* you produce the code out of that documentation. The
code is becoming an output of the documentation. Not the opposite.

Seb

-- 
Sébastien Vauban



_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: Literate Programming with Org mode

[sam kleinman]

On Tue, Jul 28, 2009 at 10:41:50PM +0200, Sébastien Vauban wrote:

> I've heard that Knuth told about it in those words: it's when we will be able
> to read the code of a software in our bed, reading a book made of 90% of
> documentation and 10% of code. If someone can find this phrase somewhere...

Here's a literate programing example: 

I talked with a statistician, programer and human rights violation
researcher, who wrote (with his team) reports of statistical studies
of data regarding possible genocide incidents. He wrote the LaTeX
documents which, within the text of the document, all values and
analysis' were called in and generated when LaTeX ran, so that as data
was collected, and the report was recompiled the analysis was
completed with the most up-to-date version of the data, and that the
production of the text was isolated from the collection of data, and
from the analysis of those figures. 

The stack itself, was comprised of Sweave
<http://www.stat.umn.edu/~charlie/Sweave/> R for stats processing,
make, and a little bit of python for glue. I think. 

As an example. I'm not an expert either on this stuff. 

Cheers,
sam
--
tycho(ish) @
garen@tychoish.com
http://www.tychoish.com/
http://www.criticalfutures.com/
"don't get it right, get it written" -- james thurber

Re: Re: Literate Programming with Org mode

[Eric H. Neilsen, Jr.]

Sam,

sam kleinman wrote:

...
> Here's a literate programing example: 
>
> I talked with a statistician, programer and human rights violation
> researcher, who wrote (with his team) reports of statistical studies
> of data regarding possible genocide incidents. He wrote the LaTeX
> documents which, within the text of the document, all values and
> analysis' were called in and generated when LaTeX ran, so that as data
> was collected, and the report was recompiled the analysis was
> completed with the most up-to-date version of the data, and that the
> production of the text was isolated from the collection of data, and
> from the analysis of those figures. 
>
> The stack itself, was comprised of Sweave
> <http://www.stat.umn.edu/~charlie/Sweave/> R for stats processing,
> make, and a little bit of python for glue. I think. 
This is how it is often used in R (or S), and is compatible with the 
original idea, which is a little broader. The idea is to write a full 
software application by first writing a document about its design and 
implementation (in whatever organization is clearest for humans), but at 
a high enough level of detail that *all* code in *all* source files in 
the final application gets included somewhere in code snippets within 
the documentation. To compile your program, you run a program to 
"tangle" your text into C files, makefiles, or whatever is appropriate, 
and then compile that (with no additional editing).

See http://www.literateprogramming.com/

Every time I have tried this, I have given up in frustration at the 
tools. I have put together some org code to do it, and have used it 
successfully for some small projects, but I am still pulling my hair out 
on being able to properly contribute it to org, and it would need to be 
reworked in light of other developments anyway. org-babel now has 
includes literate programming in it as well, but I have not yet 
experimented with it (but am very interested in trying).

    -Eric

-- 
Eric H. Neilsen, Jr.
http://home.fnal.gov/~neilsen

Re: Literate Programming with Org mode

[Eric Schulte]

Hi Sébastien,

Thanks for the description.  This is very interesting and I think
warrants a re-working of the tangle functionality in org-babel.

Given your description below I think I may try to add the Nuweb
link/reference syntax "<<source-block-name>>" to org-babel.  As
source-code blocks are already named in the current org-babel setup this
wouldn't be a very significant change.  Once this is added it shouldn't
be difficult to expand these links/references on export.

Under this new setup the example file you described would be analogous
to the this org-mode file.

Does this sound like it would accomplish most of what you are after?

I will reply to this message if/when I get this implemented.

Cheers -- Eric

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:

> Hi Eric and all,
>
> Here some promised description of how I'm using Literate Programming with
> LaTeX (up to now -- soon directly from Org mode?).
>
> I write an "enhanced LaTeX" file ([1], having the `.nw' -- for Nuweb --
> extension). It is enhanced as I can put blocks of code in there (SQL blocks,
> for example), and give them a name for further referencing.
>
> Then, at some point in the file, I say how I want the code files to look like,
> by assembling the blocks in a certain order, with some glue.
>
> For example:
>
> <<Enterprise.sql>>=
> <<sql-init>>
> SELECT abcID, etpID, etpAssurATPolNum
> FROM enterprise JOIN record
>     ON (etpAbcID_fk = abcID)
> WHERE etpAbcID_fk
> <<sql-cond>>
> @ %def 
>
> The above file is made up of two defined blocks (`sql-init' and `sql-cond')
> plus some glue in between.
>
> The good thing is I can reuse the same blocks in other context, completely
> avoiding copy/paste of code:
>
> <<Lessons.sql>>=
> <<sql-init>>
> SELECT abcID, lesAlternNbrSem, lesNbrSem,
>        CONVERT(varchar(10), lesDateDeb, @dateFmtStyleOut) AS lesDateDeb,
>        CONVERT(varchar(10), lesDateFin, @dateFmtStyleOut) AS lesDateFin
> FROM lessons JOIN record
>     ON (lesAbcID_fk = abcID)
> WHERE lesAbcID_fk
> <<sql-cond>>
> @ %def 
>
> OK. So, my `org-lit-prog.nw' file is ready, and committed under Subversion
> (this is the only one I'm committing as it contains both the documentation of
> the code, and the source code itself).
>
> Then, I just type (in a terminal):
>
> noweb org-lit-prog.nw
>
> to extract both:
>
>     o   the LaTeX file [2] to be compiled into a PDF [3],
>     o   the different source code files (Enterprise.sql [4], Lessons.sql [5],
>         [6] and [7])
>
> Just that easy to keep code and doc in sync, and get a very nice-to-read
> documentation of the code.
>
> Sometimes, I play an extra trick is made in the noweb file: I put blocks of
> static code (for which I don't especially need any noweb feature) after the
> end of the LaTeX document. That way, it gets outputted by noweb as a source
> file during the Tangle process, and included in my LaTeX document via the
> listings package.
>
> Why that trick?  I could directly put the code in the body of the document!
> Yes, but I want the code to be highlighted via listings. Otherwise, it's just
> black and white display of the code...
>
> For easy access, I've put all the above files on a my Web space:
>
> [1] http://www.mygooglest.com/sva/org-lit-prog.nw
> [2] http://www.mygooglest.com/sva/org-lit-prog.tex
> [3] http://www.mygooglest.com/sva/org-lit-prog.pdf
> [4] http://www.mygooglest.com/sva/Enterprise.sql
> [5] http://www.mygooglest.com/sva/Lessons.sql
> [6] http://www.mygooglest.com/sva/Payment.sql
> [7] http://www.mygooglest.com/sva/prsNumNat.awk
>
> From what I understand, what we both wanna reach is writing all of the `.nw'
> file as an Org file directly, and let Org-babel extracts the source code files
> and the documentation out of it.
>
> What should be good in the future, when it will be ready for prime time, is
> that this becomes complete part of the export process: no tangle operation
> needed.
>
> Is the above description worth for you?  Do you understand how I work when
> documenting code?  Pay attention: I'm *the* expert in literate programming.
> Just an amateur being seduced by some of its features.
>
> Best regards,
>   Seb

Re: Re: Literate Programming with Org mode

[Eric Schulte]

Hi Sam,

Not to be overly picky, but I think the scenario you described (while
very engaging) is an instance of Reproducible Research [1].  My
understanding of Reproducible Research and Literate Programming and
their relation is as follows...

- Literate Programming :: A style of programming in which the source
     code is a work of literature (i.e. is intended for human
     consumption) and is structured in such a way that a machine
     readable version can be automatically extracted from the primary
     document.
- Reporducable Research :: A style of publishing information in which
     the raw data and analysis are embedded in the published document
     such that the analysis can be re-run and analyzed by any reader.

I think the difference between these two is that in LP the "product" is
the executable piece of software, where as in RR the "product" is the
document itself.

Cheers -- Eric

PS: The initial goal of the org-babel suite was to implement Sweave type
functionality as part of org-mode.

sam kleinman <garen@tychoish.com> writes:

> On Tue, Jul 28, 2009 at 10:41:50PM +0200, Sébastien Vauban wrote:
>
>> I've heard that Knuth told about it in those words: it's when we will be able
>> to read the code of a software in our bed, reading a book made of 90% of
>> documentation and 10% of code. If someone can find this phrase somewhere...
>
> Here's a literate programing example: 
>
> I talked with a statistician, programer and human rights violation
> researcher, who wrote (with his team) reports of statistical studies
> of data regarding possible genocide incidents. He wrote the LaTeX
> documents which, within the text of the document, all values and
> analysis' were called in and generated when LaTeX ran, so that as data
> was collected, and the report was recompiled the analysis was
> completed with the most up-to-date version of the data, and that the
> production of the text was isolated from the collection of data, and
> from the analysis of those figures. 
>
> The stack itself, was comprised of Sweave
> <http://www.stat.umn.edu/~charlie/Sweave/> R for stats processing,
> make, and a little bit of python for glue. I think. 
>
> As an example. I'm not an expert either on this stuff. 
>
> Cheers,
> sam
> --
> tycho(ish) @
> garen@tychoish.com
> http://www.tychoish.com/
> http://www.criticalfutures.com/
> "don't get it right, get it written" -- james thurber
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Footnotes: 
[1]  http://www.reproducibleresearch.org/

Re: Re: Literate Programming with Org mode

[sam kleinman]

On Fri, Jul 31, 2009 at 11:01:36AM -0600, Eric Schulte wrote:

> I think the difference between these two is that in LP the "product" is
> the executable piece of software, where as in RR the "product" is the
> document itself.

This is true, and I think the explanation suffers somewhat as a result
of this. But I don't think the demarcation you present is as
significant as it seems from your explanation. Perhaps we can think of
the example I provided as being one of "reproducible research by means
of literate programming." 

The Sweave stuff, particularly in cases like the one I describe (and
arguably, when "done right") I think works to transform the "product"
of research from a static document to an "executable document," of
sorts. The analysis (the research?) happens, (at least theoretically)
in real time, and the document is just a snapshot/representation of
this: another output of the program. 

Indeed the programming of the generation of a document, no matter how
complex, is not the "sexiest" expression of the programmer's art, but
I think it still counts. And to be fair, Sweave/LP approaches to
research reporting, doesn't include reproducible data collection
techniques/documentation which is a big part of RR, at least in my
mind. 

I'm interested in you project (a lot, actually) but I must admit that
I'm not much of a numbers guy any more, as if I ever was one. :)

Cheers,
sam

-- 
tycho(ish) @
garen@tychoish.com
http://www.tychoish.com/
http://www.criticalfutures.com/
"don't get it right, get it written" -- james thurber

Re: Literate Programming with Org mode

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 5096 bytes --]

"Eric Schulte" <schulte.eric@gmail.com> writes:

> Hi Sébastien,
>
> Thanks for the description.  This is very interesting and I think
> warrants a re-working of the tangle functionality in org-babel.
>
> Given your description below I think I may try to add the Nuweb
> link/reference syntax "<<source-block-name>>" to org-babel.  As
> source-code blocks are already named in the current org-babel setup this
> wouldn't be a very significant change.  Once this is added it shouldn't
> be difficult to expand these links/references on export.
>
> Under this new setup the example file you described would be analogous
> to the this org-mode file.
>
> Does this sound like it would accomplish most of what you are after?
>
> I will reply to this message if/when I get this implemented.
>

Alright,

I've changed the tangling functionality of org-babel so that it now
follows some of the noweb conventions.

With the newest version of org-babel [1], try opening the attached
org-mode file [2] which is an org-mode translation of Sébastien's noweb
file from his previous email.  Calling `org-babel-tangle' from within
this file will create the Payment.sql, Lessons.sql, and Enterprise.sql
source-code files.  The document can also be exported into html and
latex using org-mode's standard export functionality.

Cheers -- Eric

>
> Cheers -- Eric
>
> Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:
>
>> Hi Eric and all,
>>
>> Here some promised description of how I'm using Literate Programming with
>> LaTeX (up to now -- soon directly from Org mode?).
>>
>> I write an "enhanced LaTeX" file ([1], having the `.nw' -- for Nuweb --
>> extension). It is enhanced as I can put blocks of code in there (SQL blocks,
>> for example), and give them a name for further referencing.
>>
>> Then, at some point in the file, I say how I want the code files to look like,
>> by assembling the blocks in a certain order, with some glue.
>>
>> For example:
>>
>> <<Enterprise.sql>>=
>> <<sql-init>>
>> SELECT abcID, etpID, etpAssurATPolNum
>> FROM enterprise JOIN record
>>     ON (etpAbcID_fk = abcID)
>> WHERE etpAbcID_fk
>> <<sql-cond>>
>> @ %def 
>>
>> The above file is made up of two defined blocks (`sql-init' and `sql-cond')
>> plus some glue in between.
>>
>> The good thing is I can reuse the same blocks in other context, completely
>> avoiding copy/paste of code:
>>
>> <<Lessons.sql>>=
>> <<sql-init>>
>> SELECT abcID, lesAlternNbrSem, lesNbrSem,
>>        CONVERT(varchar(10), lesDateDeb, @dateFmtStyleOut) AS lesDateDeb,
>>        CONVERT(varchar(10), lesDateFin, @dateFmtStyleOut) AS lesDateFin
>> FROM lessons JOIN record
>>     ON (lesAbcID_fk = abcID)
>> WHERE lesAbcID_fk
>> <<sql-cond>>
>> @ %def 
>>
>> OK. So, my `org-lit-prog.nw' file is ready, and committed under Subversion
>> (this is the only one I'm committing as it contains both the documentation of
>> the code, and the source code itself).
>>
>> Then, I just type (in a terminal):
>>
>> noweb org-lit-prog.nw
>>
>> to extract both:
>>
>>     o   the LaTeX file [2] to be compiled into a PDF [3],
>>     o   the different source code files (Enterprise.sql [4], Lessons.sql [5],
>>         [6] and [7])
>>
>> Just that easy to keep code and doc in sync, and get a very nice-to-read
>> documentation of the code.
>>
>> Sometimes, I play an extra trick is made in the noweb file: I put blocks of
>> static code (for which I don't especially need any noweb feature) after the
>> end of the LaTeX document. That way, it gets outputted by noweb as a source
>> file during the Tangle process, and included in my LaTeX document via the
>> listings package.
>>
>> Why that trick?  I could directly put the code in the body of the document!
>> Yes, but I want the code to be highlighted via listings. Otherwise, it's just
>> black and white display of the code...
>>
>> For easy access, I've put all the above files on a my Web space:
>>
>> [1] http://www.mygooglest.com/sva/org-lit-prog.nw
>> [2] http://www.mygooglest.com/sva/org-lit-prog.tex
>> [3] http://www.mygooglest.com/sva/org-lit-prog.pdf
>> [4] http://www.mygooglest.com/sva/Enterprise.sql
>> [5] http://www.mygooglest.com/sva/Lessons.sql
>> [6] http://www.mygooglest.com/sva/Payment.sql
>> [7] http://www.mygooglest.com/sva/prsNumNat.awk
>>
>> From what I understand, what we both wanna reach is writing all of the `.nw'
>> file as an Org file directly, and let Org-babel extracts the source code files
>> and the documentation out of it.
>>
>> What should be good in the future, when it will be ready for prime time, is
>> that this becomes complete part of the export process: no tangle operation
>> needed.
>>
>> Is the above description worth for you?  Do you understand how I work when
>> documenting code?  Pay attention: I'm *the* expert in literate programming.
>> Just an amateur being seduced by some of its features.
>>
>> Best regards,
>>   Seb

Footnotes: 
[1]  http://github.com/eschulte/org-babel/tree/master

[2]  


[-- Attachment #2: org-babel-lit-prog.org --]
[-- Type: application/octet-stream, Size: 2553 bytes --]

#+TITLE: Showing the benefits of Literate Programming
#+AUTHOR: Sebastien Vauban
#+OPTIONS: toc:nil num:nil ^:nil

* Context

I wanna show (or, at least, try to show) some advantages of literate
programming, such as no copy/pasting of code, and being able to
explain the code independently of its final structure as ``source
code''.

This file is a translation of Sebastien's noweb file into an org-mode
document.

* Files with SQL commands

** Common Code

All files have some code in common.
*** Header

#+srcname: sql-init
#+begin_src sql
-- Hey, Emacs! This is a -*- coding: utf-8 -*- file!

-- no longer display the count message
SET NOCOUNT ON

DECLARE @dateFmtStyleIn int; SET @dateFmtStyleIn = 120 -- ODBC canonical
DECLARE @dateFmtStyleOut int; SET @dateFmtStyleOut = 103 -- French dd/mm/yyyy

DECLARE @firstDayOfThisMonth smalldatetime
SET @firstDayOfThisMonth = CONVERT(smalldatetime,
                                   CAST(YEAR(GETDATE()) AS char(4)) + '-'
                                   + CAST(MONTH(GETDATE()) AS char(2)) + '-'
                                   + '01' + ' 00:00:00',
                                   @dateFmtStyleIn)
#+end_src

*** Condition to be satisfied

Condition to select rows will stay the same for all requests.

#+srcname: sql-cond
#+begin_src sql 
      IN (SELECT actAbcID_fk
          FROM status
          WHERE (staID = 338 AND staEtat = 3))
      AND abcSignDate < @firstDayOfThisMonth
  ORDER BY abcID
#+end_src


** Specific code


*** File Enterprise

Specific part to enterprise is:


#+srcname: enterprise
#+begin_src sql :tangle Enterprise
  -- <<sql-init>>
  SELECT abcID, etpID, etpAssurATPolNum
  FROM enterprise JOIN record
      ON (etpAbcID_fk = abcID)
  WHERE etpAbcID_fk
  -- <<sql-cond>>
#+end_src

*** File Lessons

Specific part to lessons is:

#+srcname: lessons
#+begin_src sql :tangle Lessons
  -- <<sql-init>>
  SELECT abcID, lesAlternNbrSem, lesNbrSem,
         CONVERT(varchar(10), lesDateDeb, @dateFmtStyleOut) AS lesDateDeb,
         CONVERT(varchar(10), lesDateFin, @dateFmtStyleOut) AS lesDateFin
  FROM lessons JOIN record
      ON (lesAbcID_fk = abcID)
  WHERE lesAbcID_fk
  -- <<sql-cond>>
#+end_src

*** File Payment

Specific part to payment is:

#+srcname: payment
#+begin_src sql :tangle Payment
  -- <<sql-init>>
  SELECT abcID, payFDNbrKm, payAllocType, payCompens, payFDMontantJ,
         payAllocChomBrute, payAllocChomBruteJM, payAutresAllocsSocBrutes
  FROM payments JOIN record
      ON (payAbcID_fk = abcID)
  WHERE payAbcID_fk
  -- <<sql-cond>>
#+end_src

[-- Attachment #3: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Literate Programming with Org mode

[Sébastien Vauban]

Hi Eric,

"Eric Schulte" wrote:
> "Eric Schulte" <schulte.eric-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>> Thanks for the description.  This is very interesting and I think
>> warrants a re-working of the tangle functionality in org-babel.

I don't know if it warrants a re-working of the tangle functionality, but it
warrants at least having the possibility to do such documents from withing
Org IMHO.

We're both (many more?) convinced...


>> Given your description below I think I may try to add the Nuweb
>> link/reference syntax "<<source-block-name>>" to org-babel.

Good.

Sorry for the (little) mistake I've made, but -- to be exact -- the above
syntax is from noweb.

Nuweb syntax one is with @ such as the following "include" code:

--8<---------------cut here---------------start------------->8---
@< Cursor abcID : Init on records @>
--8<---------------cut here---------------end--------------->8---

where the thing between `@<' and `@>' is the name of the source.


>> As source-code blocks are already named in the current org-babel setup this
>> wouldn't be a very significant change. Once this is added it shouldn't be
>> difficult to expand these links/references on export.

Excellent.


>> Under this new setup the example file you described would be analogous
>> to the this org-mode file.
>>
>> Does this sound like it would accomplish most of what you are after?

I think so, yes. I'll scan all my Nuweb and noweb files over the past years,
but I am positive you cover all of my needs -- with the exception of one, but
I guess it's solvable somehow: the fact that the code should be included via
the LaTeX `listings' package (thus, colored contextually to the programming
language) and not as verbatim.


> With the newest version of org-babel [1]

Done: `git pull'.


> try opening the attached org-mode file [2] which is an org-mode
> translation of Sébastien's noweb file from his previous email.

I wanted to translate it in Org. You've been much quicker than I currently am.
Thanks.


> Calling `org-babel-tangle' from within this file will create the
> Payment.sql, Lessons.sql, and Enterprise.sql source-code files.

Nope...

--8<---------------cut here---------------start------------->8---
Open ~/Personal/Templates/org-babel-lit-prog.org
tangled 0 source-code blocks
--8<---------------cut here---------------end--------------->8---

I don't understand why.

Is it normal that you don't include the extension in the tangle property?
Does it add the language extension automatically?


> The document can also be exported into html and latex using org-mode's
> standard export functionality.

I have problems as well...

LaTeX error:

--8<---------------cut here---------------start------------->8---
ERROR: Missing \endcsname inserted.

--- TeX said ---
<to be read again>
                   \protect
l.47 \begin{\LaTeX{}}

--- HELP ---
From the .log file...

The control sequence marked <to be read again> should
not appear between \csname and \endcsname.
--8<---------------cut here---------------end--------------->8---


(Begin of) exported file is:

--8<---------------cut here---------------start------------->8---
% Created 2009-08-03 Mon 09:58
\documentclass[final]{article}
\usepackage[utf8x]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{longtable}
\usepackage{hyperref}


\title{Showing the benefits of Literate Programming}
\author{Sebastien Vauban}
\date{03 August 2009}

\begin{document}

\maketitle


\section*{Context}
\label{sec-1}


  I wanna show (or, at least, try to show) some advantages of literate
  programming, such as no copy/pasting of code, and being able to
  explain the code independently of its final structure as ``source
  code''.

  This file is a translation of Sebastien's noweb file into an org-mode
  document.


\section*{Files with SQL commands}
\label{sec-2}


\subsection*{Common Code}
\label{sec-2.1}


    All files have some code in common.

\subsubsection*{Header}
\label{sec-2.1.1}



\begin{\LaTeX{}}  % <--- spot of the error <--- <--- <--- <---
\begin{verbatim}
-- Hey, Emacs! This is a -*- coding: utf-8 -*- file!

-- no longer display the count message
SET NOCOUNT ON

DECLARE @dateFmtStyleIn int; SET @dateFmtStyleIn = 120 -- ODBC canonical
DECLARE @dateFmtStyleOut int; SET @dateFmtStyleOut = 103 -- French dd/mm/yyyy

DECLARE @firstDayOfThisMonth smalldatetime
SET @firstDayOfThisMonth = CONVERT(smalldatetime,
CAST(YEAR(GETDATE()) AS char(4)) + '-'
+ CAST(MONTH(GETDATE()) AS char(2)) + '-'
+ '01' + ' 00:00:00',
@dateFmtStyleIn)
\end{verbatim}
\end{\LaTeX{}}
--8<---------------cut here---------------end--------------->8---


Don't understand how this LaTeX environment comes in the play.

I saw a difference between the way you write your levels in Org and the way I
do it: I'm using a odd number of stars.

--8<---------------cut here---------------start------------->8---
;; skip even levels and only use odd levels for the outline
(setq org-odd-levels-only t)
--8<---------------cut here---------------end--------------->8---


So, I've rewritten the Org file as follows:

--8<---------------cut here---------------start------------->8---
#+TITLE: Showing the benefits of Literate Programming
#+AUTHOR: Sebastien Vauban
#+OPTIONS: toc:nil num:nil ^:nil

* Context

  I wanna show (or, at least, try to show) some advantages of literate
  programming, such as no copy/pasting of code, and being able to
  explain the code independently of its final structure as ``source
  code''.

  This file is a translation of Sebastien's noweb file into an org-mode
  document.


* Files with SQL commands

*** Common Code

    All files have some code in common.

***** Header

#+srcname: sql-init
#+begin_src sql
-- Hey, Emacs! This is a -*- coding: utf-8 -*- file!

-- no longer display the count message
SET NOCOUNT ON

DECLARE @dateFmtStyleIn int; SET @dateFmtStyleIn = 120 -- ODBC canonical
DECLARE @dateFmtStyleOut int; SET @dateFmtStyleOut = 103 -- French dd/mm/yyyy

DECLARE @firstDayOfThisMonth smalldatetime
SET @firstDayOfThisMonth = CONVERT(smalldatetime,
                                   CAST(YEAR(GETDATE()) AS char(4)) + '-'
                                   + CAST(MONTH(GETDATE()) AS char(2)) + '-'
                                   + '01' + ' 00:00:00',
                                   @dateFmtStyleIn)
#+end_src
--8<---------------cut here---------------end--------------->8---

But that does not change anything:

    o   Tangle is not extracting any code block;
    o   same LaTeX error.

Any idea?

BTW, what do you think of my proposal to include the Tangle process as a
built-in step of the export to LaTeX (and HTML)?

That way, we only would export and get both flies caught with one stone.

Thanks for all!  This is seriously a hot topic you've almost solved...

Seb

-- 
Sébastien Vauban



_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: Literate Programming with Org mode

[Eric Schulte]

Hi Sébastien,

Thanks for giving this a try, comments below...

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:

> Hi Eric,
>
> "Eric Schulte" wrote:
>> "Eric Schulte" <schulte.eric@gmail.com> writes:

[...]

>>> As source-code blocks are already named in the current org-babel setup this
>>> wouldn't be a very significant change. Once this is added it shouldn't be
>>> difficult to expand these links/references on export.
>
> Excellent.
>
>
>>> Under this new setup the example file you described would be analogous
>>> to the this org-mode file.
>>>
>>> Does this sound like it would accomplish most of what you are after?
>
> I think so, yes. I'll scan all my Nuweb and noweb files over the past years,
> but I am positive you cover all of my needs -- with the exception of one, but
> I guess it's solvable somehow: the fact that the code should be included via
> the LaTeX `listings' package (thus, colored contextually to the programming
> language) and not as verbatim.
>

I agree fontified code would certainly be preferable to the current
approach of inserting source-code as verbatim.  This seems to be a
general org-mode development issue rather than org-babel specific, so
I'm happy to push the buck on this one :).  I don't know much about the
listings package, but would another possibly be easier to use htmlize.el
(used for fontification on html export) along with html -> latex
conversion?

>
>
>> With the newest version of org-babel [1]
>
> Done: `git pull'.
>

Please do another git pull before trying the steps below so we can be
sure we're working off the same code base.

> >
>> try opening the attached org-mode file [2] which is an org-mode
>> translation of Sébastien's noweb file from his previous email.
>
> I wanted to translate it in Org. You've been much quicker than I currently am.
> Thanks.
>
>
>> Calling `org-babel-tangle' from within this file will create the
>> Payment.sql, Lessons.sql, and Enterprise.sql source-code files.
>
> Nope...
>
> Open ~/Personal/Templates/org-babel-lit-prog.org
> tangled 0 source-code blocks
>
> I don't understand why.
>

Me either...  I'm not sure what could be different about our two
environments that could be causing this problem.  Maybe it has to do
with the

--8<---------------cut here---------------start------------->8---
(setq org-odd-levels-only t)
--8<---------------cut here---------------end--------------->8---

that you mention below.  Also note that with org-babel by default
source-code blocks are not tangled, only those with a :tangle header
argument will be tangled to source-code files.  So for example

--8<---------------cut here---------------start------------->8---
#+srcname: sql-init
#+begin_src sql
  -- <<sql-init>>
  SELECT abcID, etpID, etpAssurATPolNum
  FROM enterprise JOIN record
      ON (etpAbcID_fk = abcID)
  WHERE etpAbcID_fk
  -- <<sql-cond>>
#+end_src
--8<---------------cut here---------------end--------------->8---

will not be tangled, while

--8<---------------cut here---------------start------------->8---
#+srcname: enterprise
#+begin_src sql :tangle Enterprise
  -- <<sql-init>>
  SELECT abcID, etpID, etpAssurATPolNum
  FROM enterprise JOIN record
      ON (etpAbcID_fk = abcID)
  WHERE etpAbcID_fk
  -- <<sql-cond>>
#+end_src
--8<---------------cut here---------------end--------------->8---

will be tangled.

I have changed the original org-babel-lit-prog.org so that it should
work with your configuration of odd levels only and posted it up here
[1].

  Note that you can grab all of these referenced files by cloning the
  git repo at git://github.com/eschulte/noweb-example.git.

By calling org-babel-tangle from within this org-mode file I was able to
generate Enterprise.sql [2], Lessons.sql [3], and Payment.sql [4].

>
> Is it normal that you don't include the extension in the tangle property?
> Does it add the language extension automatically?
>
>

Yes, org-babel adds the language specific extension automatically.  This
could be changed/overridden if it proves to limiting.  It grew out of
the fact that originally source-code files were named based upon the
basename of the org-mode file and the language extension.

>
>> The document can also be exported into html and latex using org-mode's
>> standard export functionality.
>
> I have problems as well...
>

Again maybe this is the result of conflicting values of
org-odd-levels-only?  Please try this with the newer org-mode file [1].
I was able to export this to latex resulting in the following tex file
[5] and pdf file [6].  I was also able to export this file to html which
includes code highlighting [7].

[...]

> BTW, what do you think of my proposal to include the Tangle process as a
> built-in step of the export to LaTeX (and HTML)?
>
> That way, we only would export and get both flies caught with one stone.
>

I like that idea, once this is working as two separate processes it
should be relatively trivial to optionally hook org-babel-tangle onto
the other export commands.

>
> Thanks for all!  This is seriously a hot topic you've almost solved...
>
> Seb

Sorry I can't be more specific with answers to your problems tangling
and exporting.  Please do give the new org-mode file [1] a try with the
latest version org-babel and let me know if the problems persist.

Cheers -- Eric


Footnotes: 

[1]  http://github.com/eschulte/noweb-example/raw/a130e165c79c8f6c2f5147330fe4ae5ab6428c17/w-babel/org-babel-lit-prog.org

[2]  http://github.com/eschulte/noweb-example/raw/a130e165c79c8f6c2f5147330fe4ae5ab6428c17/w-babel/Enterprise.sql

[3]  http://github.com/eschulte/noweb-example/raw/a130e165c79c8f6c2f5147330fe4ae5ab6428c17/w-babel/Lessons.sql

[4]  http://github.com/eschulte/noweb-example/raw/a130e165c79c8f6c2f5147330fe4ae5ab6428c17/w-babel/Payment.sql

[5]  http://github.com/eschulte/noweb-example/raw/a130e165c79c8f6c2f5147330fe4ae5ab6428c17/w-babel/org-babel-lit-prog.tex

[6]  http://github.com/eschulte/noweb-example/raw/a130e165c79c8f6c2f5147330fe4ae5ab6428c17/w-babel/org-babel-lit-prog.pdf

[7]  http://github.com/eschulte/noweb-example/raw/a130e165c79c8f6c2f5147330fe4ae5ab6428c17/w-babel/org-babel-lit-prog.html

Re: Literate Programming with Org mode

[Sébastien Vauban]

Hi Eric,

"Eric Schulte" wrote:
> Sébastien Vauban <wxhgmqzgwmuf-geNee64TY+gS+FvcfC7Uqw@public.gmane.org> writes:
>> "Eric Schulte" wrote:
>>> "Eric Schulte" <schulte.eric-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>
> Thanks for giving this a try, comments below...

You're welcome. I'm more than interested into this. And I play the easy role.
Just testing... and commenting ;-)

Sorry for answering late, though.


>> [...] the code should be included via the LaTeX `listings' package (thus,
>> colored contextually to the programming language) and not as verbatim.
>
> I agree fontified code would certainly be preferable to the current approach
> of inserting source-code as verbatim. This seems to be a general org-mode
> development issue rather than org-babel specific, so I'm happy to push the
> buck on this one :). I don't know much about the listings package, but would
> another possibly be easier to use htmlize.el (used for fontification on html
> export) along with html -> latex conversion?

I've tried the following new settings on a freshly pulled Org-mode (this
morning):

--8<---------------cut here---------------start------------->8---
;; tell org to use listings
(setq org-export-latex-listings t)

;; you must include the listings package
(add-to-list 'org-export-latex-packages-alist '("" "listings"))

;; if you want colored source code, then you need to include the
;; (x)color package
(add-to-list 'org-export-latex-packages-alist '("" "xcolor")))
--8<---------------cut here---------------end--------------->8---

though, with no luck, right now.


> Please do another git pull before trying the steps below so we can be sure
> we're working off the same code base.

Done one hour ago.


>>> Calling `org-babel-tangle' from within this file will create the
>>> Payment.sql, Lessons.sql, and Enterprise.sql source-code files.
>>
>> Nope...
>>
>> Open ~/Personal/Templates/org-babel-lit-prog.org
>> tangled 0 source-code blocks
>
> I have changed the original org-babel-lit-prog.org so that it should work
> with your configuration of odd levels only and posted it up here [1].
>
>   Note that you can grab all of these referenced files by cloning the git
>   repo at git://github.com/eschulte/noweb-example.git.

Using your repo.


> By calling org-babel-tangle from within this org-mode file I was able to
> generate Enterprise.sql [2], Lessons.sql [3], and Payment.sql [4].

I still couldn't. Void variable: file.

Now (I've done some changes, explained below), I can...

--8<---------------cut here---------------start------------->8---
Stored: Header
Stored: Condition to be satisfied
Stored: File Enterprise
Stored: File Lessons
Stored: File Payment
Added to /home/sva/Downloads/emacs/site-lisp/noweb-example/w-babel/Enterprise.sql
Added to /home/sva/Downloads/emacs/site-lisp/noweb-example/w-babel/Lessons.sql
Added to /home/sva/Downloads/emacs/site-lisp/noweb-example/w-babel/Payment.sql
tangled 3 source-code blocks
--8<---------------cut here---------------end--------------->8---

Great!

Though, this is not entirely what I expected. Let's take the Enterprise.sql
file:

--8<---------------cut here---------------start------------->8---
-- generated by org-babel-tangle

-- [[file:~/Downloads/emacs/site-lisp/noweb-example/w-babel/org-babel-lit-prog.org::*File%20Enterprise][block-3]]
-- <<sql-init>>
SELECT abcID, etpID, etpAssurATPolNum
FROM enterprise JOIN record
    ON (etpAbcID_fk = abcID)
WHERE etpAbcID_fk
-- <<sql-cond>>
-- block-3 ends here
--8<---------------cut here---------------end--------------->8---

I'm expecting the sql-init and sql-cond blocks to be replaced by their defined
equivalent. So, in this case, I should have the following file:

--8<---------------cut here---------------start------------->8---
-- generated by org-babel-tangle

-- [[file:~/Downloads/emacs/site-lisp/noweb-example/w-babel/org-babel-lit-prog.org::*File%20Enterprise][block-3]]
-- Hey, Emacs! This is a -*- coding: utf-8 -*- file!

-- no longer display the count message
SET NOCOUNT ON

DECLARE @dateFmtStyleIn int; SET @dateFmtStyleIn = 120 -- ODBC canonical
DECLARE @dateFmtStyleOut int; SET @dateFmtStyleOut = 103 -- French dd/mm/yyyy

DECLARE @firstDayOfThisMonth smalldatetime
SET @firstDayOfThisMonth = CONVERT(smalldatetime,
                                   CAST(YEAR(GETDATE()) AS char(4)) + '-'
                                   + CAST(MONTH(GETDATE()) AS char(2)) + '-'
                                   + '01' + ' 00:00:00',
                                   @dateFmtStyleIn)

SELECT abcID, etpID, etpAssurATPolNum
FROM enterprise JOIN record
    ON (etpAbcID_fk = abcID)
WHERE etpAbcID_fk

      IN (SELECT actAbcID_fk
          FROM status
          WHERE (staID = 338 AND staEtat = 3))
      AND abcSignDate < @firstDayOfThisMonth
  ORDER BY abcID
-- block-3 ends here
--8<---------------cut here---------------end--------------->8---

Do we agree on this?


>> Is it normal that you don't include the extension in the tangle property?
>> Does it add the language extension automatically?
>
> Yes, org-babel adds the language specific extension automatically.  This
> could be changed/overridden if it proves to limiting.  It grew out of
> the fact that originally source-code files were named based upon the
> basename of the org-mode file and the language extension.

I find that this is OK right now. Though, people could want to give an
explicit extension, moreover for `.htm' or `.html' files where standard
conventions are not that clear.


>>> The document can also be exported into html and latex using org-mode's
>>> standard export functionality.
>>
>> I have problems as well...
>
> Again maybe this is the result of conflicting values of org-odd-levels-only?
> Please try this with the newer org-mode file [1]. I was able to export this
> to latex resulting in the following tex file [5] and pdf file [6]. I was
> also able to export this file to html which includes code highlighting [7].
>
>> Thanks for all! This is seriously a hot topic you've almost solved...
>
> Sorry I can't be more specific with answers to your problems tangling and
> exporting. Please do give the new org-mode file [1] a try with the latest
> version org-babel and let me know if the problems persist.

I still had the same problem in the beginning: around the `verbatim'
environments, I had wrong `\LaTeX{}' environments created...

Though, I found the problem: it's somehow a conflict with the
`org-special-blocks' package. Un-commenting the following should show you the
problems:

--8<---------------cut here---------------start------------->8---
;; FIXME This is responsible of problems when used with Org-babel
;; (`\LaTeX{}' environments are created around verbatim environments)
;; ;; turn Org blocks into LaTeX environments and HTML divs
;; (try-require 'org-special-blocks)

;; literate programming
(try-require 'org-babel-init)
--8<---------------cut here---------------end--------------->8---

With the above configuration, I could finally export the Org file in LaTeX.
Yeeesss.

I have some comments on the resulting file:

    o   sections (and sub-, and subsub-) should not be starred by default,
        IMHO, and then unnumbered.

    o   indentation of code blocks should be preserved.

    o   name of code blocks should be visible in the produced document
        because, now, I see that the file Enterprise.sql will include the
        sql-init block, but I don't know where it is described (which page,
        and which block).

Except these, the result is already very promising!


>> BTW, what do you think of my proposal to include the Tangle process as a
>> built-in step of the export to LaTeX (and HTML)?
>>
>> That way, we only would export and get both flies caught with one stone.
>
> I like that idea, once this is working as two separate processes it should
> be relatively trivial to optionally hook org-babel-tangle onto the other
> export commands.

Fantastic.

Thanks for all your work.

Best regards,
  Seb

-- 
Sébastien Vauban



_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: Literate Programming with Org mode

[Eric Schulte]

Hi Sébastien,

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:

> Hi Eric,
>
> "Eric Schulte" wrote:
>> Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:
>>> "Eric Schulte" wrote:
>>>> "Eric Schulte" <schulte.eric@gmail.com> writes:

[...]

> Now (I've done some changes, explained below), I can...
>
> Stored: Header
> Stored: Condition to be satisfied
> Stored: File Enterprise
> Stored: File Lessons
> Stored: File Payment
> Added to /home/sva/Downloads/emacs/site-lisp/noweb-example/w-babel/Enterprise.sql
> Added to /home/sva/Downloads/emacs/site-lisp/noweb-example/w-babel/Lessons.sql
> Added to /home/sva/Downloads/emacs/site-lisp/noweb-example/w-babel/Payment.sql
> tangled 3 source-code blocks
>
> Great!
>

Wonderful, so if we're creating source-code files then we're most of the
way there.

>
> Though, this is not entirely what I expected. Let's take the Enterprise.sql
> file:
>
> -- generated by org-babel-tangle
>
> -- [[file:~/Downloads/emacs/site-lisp/noweb-example/w-babel/org-babel-lit-prog.org::*File%20Enterprise][block-3]]
> -- <<sql-init>>
> SELECT abcID, etpID, etpAssurATPolNum
> FROM enterprise JOIN record
>     ON (etpAbcID_fk = abcID)
> WHERE etpAbcID_fk
> -- <<sql-cond>>
> -- block-3 ends here
>
> I'm expecting the sql-init and sql-cond blocks to be replaced by their defined
> equivalent. So, in this case, I should have the following file:
>
> -- generated by org-babel-tangle
>
> -- [[file:~/Downloads/emacs/site-lisp/noweb-example/w-babel/org-babel-lit-prog.org::*File%20Enterprise][block-3]]
> -- Hey, Emacs! This is a -*- coding: utf-8 -*- file!
>
> -- no longer display the count message
> SET NOCOUNT ON
>
> DECLARE @dateFmtStyleIn int; SET @dateFmtStyleIn = 120 -- ODBC canonical
> DECLARE @dateFmtStyleOut int; SET @dateFmtStyleOut = 103 -- French dd/mm/yyyy
>
> DECLARE @firstDayOfThisMonth smalldatetime
> SET @firstDayOfThisMonth = CONVERT(smalldatetime,
>                                    CAST(YEAR(GETDATE()) AS char(4)) + '-'
>                                    + CAST(MONTH(GETDATE()) AS char(2)) + '-'
>                                    + '01' + ' 00:00:00',
>                                    @dateFmtStyleIn)
>
> SELECT abcID, etpID, etpAssurATPolNum
> FROM enterprise JOIN record
>     ON (etpAbcID_fk = abcID)
> WHERE etpAbcID_fk
>
>       IN (SELECT actAbcID_fk
>           FROM status
>           WHERE (staID = 338 AND staEtat = 3))
>       AND abcSignDate < @firstDayOfThisMonth
>   ORDER BY abcID
> -- block-3 ends here
>
> Do we agree on this?
>

Agreed.  This is the contents of my Enterprise.sql (as generated by
org-babel tangle).  I believe this is the contents that you wanted.  I'm
not sure what could be different between our setups...

--8<---------------cut here---------------start------------->8---
-- generated by org-babel-tangle

-- [[file:~/src/noweb-example/w-babel/org-babel-lit-prog.org::*File%20Enterprise][block-3]]
  -- <<sql-init>>
-- Hey, Emacs! This is a -*- coding: utf-8 -*- file!

-- no longer display the count message
SET NOCOUNT ON

DECLARE @dateFmtStyleIn int; SET @dateFmtStyleIn = 120 -- ODBC canonical
DECLARE @dateFmtStyleOut int; SET @dateFmtStyleOut = 103 -- French dd/mm/yyyy

DECLARE @firstDayOfThisMonth smalldatetime
SET @firstDayOfThisMonth = CONVERT(smalldatetime,
                                   CAST(YEAR(GETDATE()) AS char(4)) + '-'
                                   + CAST(MONTH(GETDATE()) AS char(2)) + '-'
                                   + '01' + ' 00:00:00',
                                   @dateFmtStyleIn)

  SELECT abcID, etpID, etpAssurATPolNum
  FROM enterprise JOIN record
      ON (etpAbcID_fk = abcID)
  WHERE etpAbcID_fk
  -- <<sql-cond>>
      IN (SELECT actAbcID_fk
          FROM status
          WHERE (staID = 338 AND staEtat = 3))
      AND abcSignDate < @firstDayOfThisMonth
  ORDER BY abcID
-- block-3 ends here
--8<---------------cut here---------------end--------------->8---

Some changes which I would still like to make to the above are
1) use relative paths when linking back to the org-mode file
2) somehow activate an org-mode minor mode in the source-code file which
   will then "turn on" the org-mode style links in the source code file

> >
>>> Is it normal that you don't include the extension in the tangle property?
>>> Does it add the language extension automatically?
>>
>> Yes, org-babel adds the language specific extension automatically.  This
>> could be changed/overridden if it proves to limiting.  It grew out of
>> the fact that originally source-code files were named based upon the
>> basename of the org-mode file and the language extension.
>
> I find that this is OK right now. Though, people could want to give an
> explicit extension, moreover for `.htm' or `.html' files where standard
> conventions are not that clear.
>

Maybe I can add a simple rule.  *If* there is already an extension
*then* don't add the default extension.  I'll add this to the Org-babel
tasks.

> >
>>>> The document can also be exported into html and latex using org-mode's
>>>> standard export functionality.
>>>
>>> I have problems as well...
>>
>> Again maybe this is the result of conflicting values of org-odd-levels-only?
>> Please try this with the newer org-mode file [1]. I was able to export this
>> to latex resulting in the following tex file [5] and pdf file [6]. I was
>> also able to export this file to html which includes code highlighting [7].
>>
>>> Thanks for all! This is seriously a hot topic you've almost solved...
>>
>> Sorry I can't be more specific with answers to your problems tangling and
>> exporting. Please do give the new org-mode file [1] a try with the latest
>> version org-babel and let me know if the problems persist.
>
> I still had the same problem in the beginning: around the `verbatim'
> environments, I had wrong `\LaTeX{}' environments created...
>
> Though, I found the problem: it's somehow a conflict with the
> `org-special-blocks' package. Un-commenting the following should show you the
> problems:
>
> ;; FIXME This is responsible of problems when used with Org-babel
> ;; (`\LaTeX{}' environments are created around verbatim environments)
> ;; ;; turn Org blocks into LaTeX environments and HTML divs
> ;; (try-require 'org-special-blocks)
>
> ;; literate programming
> (try-require 'org-babel-init)
>
> With the above configuration, I could finally export the Org file in LaTeX.
> Yeeesss.
>
> I have some comments on the resulting file:
>
>     o   sections (and sub-, and subsub-) should not be starred by default,
>         IMHO, and then unnumbered.
>

This should be addressed in the Org-mode core rather than in Org-babel.

>
>     o   indentation of code blocks should be preserved.
>

This should also be addressed in Org-mode proper.

>
>     o   name of code blocks should be visible in the produced document
>         because, now, I see that the file Enterprise.sql will include the
>         sql-init block, but I don't know where it is described (which page,
>         and which block).
>

Excellent that you are now seeing sql-init block in Enterprise.sql, I
agree that it would be nice to see the source code block's name in the
exported file.  I will add this to the Org-babel tasks.

>
> Except these, the result is already very promising!
>

Thanks for you continued help refining this functionality! -- Eric

> >
>>> BTW, what do you think of my proposal to include the Tangle process as a
>>> built-in step of the export to LaTeX (and HTML)?
>>>
>>> That way, we only would export and get both flies caught with one stone.
>>
>> I like that idea, once this is working as two separate processes it should
>> be relatively trivial to optionally hook org-babel-tangle onto the other
>> export commands.
>
> Fantastic.
>
> Thanks for all your work.
>
> Best regards,
>   Seb