[org-babel] Does org-babel needs some simplification?

[org-babel] Does org-babel needs some simplification?

[Torsten Wagner]

Dear All,

as a (quite, but happy) org-bable user of the first hour I followed up
the development process actively.
Nevertheless, some weeks or months pass where I had no need for
org-babel (yes, really strange I know).

Whenever I come back to org-babel, it takes me a huge amount of time to
find myself back again in the syntax. Often I spend a day or two heavily
reading the website and manual again to figure out how to make it working.

There are so many options. tangle files, results, scripting mode,
sessions, noweb, lot, etc.

Just yesterday, I fighted again to make a simple python script running
as desired to generate an automatic report. I did this dozen of times
and even by using some old report as template I still struggle with it.
Comparing old reports I noticed that I did it in many different ways.
Tangeling all snipplets, using noweb syntax, with and without session
support, etc.

Don't get me wrong, I really love org-babel and I think it is really
great. I just wonder wether it has become too complex and too difficult
to use to attract most of the org-mode people. Esp. considering people
who use it not on a regular basis.

Best regards

Torsten

Re: [org-babel] Does org-babel needs some simplification?

[Erik Iverson]

Hello!

<snip>
> 
> Whenever I come back to org-babel, it takes me a huge amount of time to
> find myself back again in the syntax. Often I spend a day or two heavily
> reading the website and manual again to figure out how to make it working.
> 
> There are so many options. tangle files, results, scripting mode,
> sessions, noweb, lot, etc.
> 
> Just yesterday, I fighted again to make a simple python script running
> as desired to generate an automatic report. I did this dozen of times
> and even by using some old report as template I still struggle with it.
> Comparing old reports I noticed that I did it in many different ways.
> Tangeling all snipplets, using noweb syntax, with and without session
> support, etc.

<snip>

I have not used it for Python, but for R coding I've found it incredibly 
intuitive.  However, that might be because R has long supported literate 
programming through Sweave, complete with noweb syntax and code tangling.

I personally don't think it's too complex.  One thing that could help is a 
gentle introduction written by users of each language that babel supports.  I 
have started something like that with R already on my blog, see 
http://blogisticreflections.wordpress.com/2010/05/23/introduction-to-using-r-with-org-babel-part-1/ 


I think the best thing to do is to figure out a system that works for you, and 
document it (in org-mode of course!) so that there are notes the next time you 
come back to the file, or need to produce a new file.

Of course, the authors of babel might have plans to somehow simplify syntax, but 
I just wanted to point out that, like anything, with some repetition, you can 
commit the main ideas to memory and then consult the manual when needed.  I 
found that spending a couple hours running all the possible options in the 
manual with R while taking notes went a long way in helping me understand how it 
it all works.

Best Regards,
Erik

Re: [org-babel] Does org-babel needs some simplification?

[Carsten Dominik]

On Jun 29, 2010, at 5:51 AM, Torsten Wagner wrote:

> Dear All,
>
> as a (quite, but happy) org-bable user of the first hour I followed up
> the development process actively.
> Nevertheless, some weeks or months pass where I had no need for
> org-babel (yes, really strange I know).
>
> Whenever I come back to org-babel, it takes me a huge amount of time  
> to
> find myself back again in the syntax. Often I spend a day or two  
> heavily
> reading the website and manual again to figure out how to make it  
> working.
>
> There are so many options. tangle files, results, scripting mode,
> sessions, noweb, lot, etc.
>
> Just yesterday, I fighted again to make a simple python script running
> as desired to generate an automatic report. I did this dozen of times
> and even by using some old report as template I still struggle with  
> it.
> Comparing old reports I noticed that I did it in many different ways.
> Tangeling all snipplets, using noweb syntax, with and without session
> support, etc.

This is, I think, the problem.  There are several ways to do things,
and the documentation maybe does not give a good guide yet which method
to use.  Maybe it need same addition in the beginning intruducing the
different ways to work with babel, and some guide lines when which  
method
is useful.

Just my 5c.

- Carsten

Re: [org-babel] Does org-babel needs some simplification?

[Thomas S. Dye]

On Jun 28, 2010, at 5:51 PM, Torsten Wagner wrote:

> Dear All,
>
> as a (quite, but happy) org-bable user of the first hour I followed up
> the development process actively.
> Nevertheless, some weeks or months pass where I had no need for
> org-babel (yes, really strange I know).
>
> Whenever I come back to org-babel, it takes me a huge amount of time  
> to
> find myself back again in the syntax. Often I spend a day or two  
> heavily
> reading the website and manual again to figure out how to make it  
> working.
>
> There are so many options. tangle files, results, scripting mode,
> sessions, noweb, lot, etc.
>
> Just yesterday, I fighted again to make a simple python script running
> as desired to generate an automatic report. I did this dozen of times
> and even by using some old report as template I still struggle with  
> it.
> Comparing old reports I noticed that I did it in many different ways.
> Tangeling all snipplets, using noweb syntax, with and without session
> support, etc.
>
> Don't get me wrong, I really love org-babel and I think it is really
> great. I just wonder wether it has become too complex and too  
> difficult
> to use to attract most of the org-mode people. Esp. considering people
> who use it not on a regular basis.
>
> Best regards
>
> Torsten

Hi Torsten,

Part of the difficulty might be that certain default behaviors changed  
in the last several months.  What worked in, say, February might not  
work the same way today.  I've been thrown for a loop more than once  
when old files didn't work as they once did.  So, this could cause  
problems for someone away from babel for a few months.

What would you simplify?  Could the library of babel help with this?

All the best,
Tom

Re: [org-babel] Does org-babel needs some simplification?

[Eric Schulte]

Hi Torsten,

Thanks for bringing this up.  I think you're right that Org-babel does
need to expose some simple points of entry.

However in reviewing the points of complexity,

- tangling
- noweb references
- the profusion of header arguments
- the library of babel

my immersed and subjective perspective is that all of these moving parts
do a pretty good job of being orthogonal, i.e. they don't overlap or
duplicate functionality and each additional piece adds new functionality
which would be otherwise impossible.

So assuming that all of these facets of Org-babel are essential (please
let me know if anyone thinks that there are chunks which could be
re-factored out), then the issue becomes making it straightforward to do
most *common* tasks w/o having to dive into much of the complexity.
This probably means good default values for all configurable parameters,
and better documentation.

We have some example usage documents up on worg [1], however those focus
on showing off all of the bells and whistles.  I like the idea of
compiling some simple language-specific demos which walk through the
basic usage of Org-babel with pointers-to, but no inclusions-of the more
complex features.

Hopefully this is something we can improve in the near future.

Thanks -- Eric

Torsten Wagner <torsten.wagner@gmail.com> writes:

> Dear All,
>
> as a (quite, but happy) org-bable user of the first hour I followed up
> the development process actively.
> Nevertheless, some weeks or months pass where I had no need for
> org-babel (yes, really strange I know).
>
> Whenever I come back to org-babel, it takes me a huge amount of time to
> find myself back again in the syntax. Often I spend a day or two heavily
> reading the website and manual again to figure out how to make it working.
>
> There are so many options. tangle files, results, scripting mode,
> sessions, noweb, lot, etc.
>
> Just yesterday, I fighted again to make a simple python script running
> as desired to generate an automatic report. I did this dozen of times
> and even by using some old report as template I still struggle with it.
> Comparing old reports I noticed that I did it in many different ways.
> Tangeling all snipplets, using noweb syntax, with and without session
> support, etc.
>
> Don't get me wrong, I really love org-babel and I think it is really
> great. I just wonder wether it has become too complex and too difficult
> to use to attract most of the org-mode people. Esp. considering people
> who use it not on a regular basis.
>
> Best regards
>
> Torsten
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Footnotes: 
[1]  http://orgmode.org/worg/org-contrib/babel/uses.php

Re: [org-babel] Does org-babel needs some simplification?

[Eric Schulte]

Hi Erik,

Erik Iverson <eriki@ccbr.umn.edu> writes:

[...]
>
> I have not used it for Python, but for R coding I've found it
> incredibly intuitive.  However, that might be because R has long
> supported literate programming through Sweave, complete with noweb
> syntax and code tangling.
>

Speaking of the Sweave/Babel relation, I've recently re-worked an
existing Sweave tutorial into babel.  See the last section of
http://orgmode.org/worg/org-contrib/babel/uses.php#foo

>
> I personally don't think it's too complex.  One thing that could help
> is a gentle introduction written by users of each language that babel
> supports.

Yes, I couldn't agree more that this would be very helpful.

> I have started something like that with R already on my blog, see
> http://blogisticreflections.wordpress.com/2010/05/23/introduction-to-using-r-with-org-babel-part-1/
>

That's great! I would love to link to your article perhaps as part of a
revamped series of language-specific Org-babel documentation and
demonstrations on Worg.

Cheers -- Eric

Re: [org-babel] Does org-babel needs some simplification?

[Daniel Brunner]

Hello everyone,

thanks for bringing this topic up. I do not want to be misunderstood:
org-mode is one of the greatest things developed the last years. I use
it every day and it is a great help. I also appreciate the work of all
those developers contributing to org-mode / babel.

My setup is: I use the git repository because I like some of the new
features und from time to time I "pull" the new things. Normalley a
"make" does the job and flipping through the log messages helps me to
understand the new things. I use org-mode on my main business machine;
ok, perhaps it is a bit risky. And I try to use as much as possible
"out-of-the-box". 

But then my .emacs reported problems: org-babel has changed its place
and the way it is loaded. I found the reason for that in the news
group. But then the nice patch by Eric Schulte for tangling "org-mode"
files doesn't work any more because "org-babel-tangle-langs" does not
exist any more. And David Maus' addition org-atom for generating atom
feeds is broken because there is something with a
org-export-inbuffer-options-extra variable (which is not a org-babel
problem as I suppose).

I know, org-mode is a growing project and I really really like the
org-babel features. But for me it is now not advisable any more to keep
on the master branch. Today I spent several hours for finding the
reasons why something is not working any more. And that is too much for
a productive environment. 

Today, I am a bit sad about my broken org-mode installation and I am
going back to the 6.36c. 

Kind regards, Daniel. 

Re: Re: [org-babel] Does org-babel needs some simplification?

[Jonathan Arkell]

__________________________________

Jonathan Arkell
Tech Lead
Inspired By Drum & Bass, Scheme, Kawaii

p. 403.206.4377

402 -- 11th Ave SE
Calgary, AB, Canada T2G 0Y4

jonathana@criticalmass.com
criticalmass.com






Hi Daniel

I would say, that if you absolutely need a productive environment, then you should stick to the official releases, rather then the latest and greatest, as sad as that might sound.

Conversely, if things break the next time you do a git pull, you could pull from a previous version to see if it is still broken.

It's bleeding edge for a reason!

On Jun 30, 2010, at 1:30 PM, Daniel Brunner wrote:

> Hello everyone,
>
> thanks for bringing this topic up. I do not want to be misunderstood:
> org-mode is one of the greatest things developed the last years. I use
> it every day and it is a great help. I also appreciate the work of all
> those developers contributing to org-mode / babel.
>
> My setup is: I use the git repository because I like some of the new
> features und from time to time I "pull" the new things. Normalley a
> "make" does the job and flipping through the log messages helps me to
> understand the new things. I use org-mode on my main business machine;
> ok, perhaps it is a bit risky. And I try to use as much as possible
> "out-of-the-box".
>
> But then my .emacs reported problems: org-babel has changed its place
> and the way it is loaded. I found the reason for that in the news
> group. But then the nice patch by Eric Schulte for tangling "org-mode"
> files doesn't work any more because "org-babel-tangle-langs" does not
> exist any more. And David Maus' addition org-atom for generating atom
> feeds is broken because there is something with a
> org-export-inbuffer-options-extra variable (which is not a org-babel
> problem as I suppose).
>
> I know, org-mode is a growing project and I really really like the
> org-babel features. But for me it is now not advisable any more to keep
> on the master branch. Today I spent several hours for finding the
> reasons why something is not working any more. And that is too much for
> a productive environment.
>
> Today, I am a bit sad about my broken org-mode installation and I am
> going back to the 6.36c.
>
> Kind regards, Daniel.
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode


The information contained in this message is confidential. It is intended to be read only by the individual or entity named above or their designee. If the reader of this message is not the intended recipient, you are hereby notified that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and delete or destroy any copy of this message.

Re: [org-babel] Does org-babel needs some simplification?

[Bernt Hansen]

Daniel Brunner <daniel@dbrunner.de> writes:

> I know, org-mode is a growing project and I really really like the
> org-babel features. But for me it is now not advisable any more to keep
> on the master branch. Today I spent several hours for finding the
> reasons why something is not working any more. And that is too much for
> a productive environment. 

I too run my production org-mode stuff from the master branch.  If you
pull and things break you can just step back to the commit you were at
before your pull with

$ git checkout ORIG_HEAD

All of your local commit changes should be available in the reflog so
you can go back to any recent point in time in your repository.

$ git reflog

will show a list of commits you have visited recently and you can return
to any one with a checkout of the SHA1 or by using the HEAD@{n} shortcut
-- just beware that HEAD@{n} changes everytime you move HEAD on your
repository.

Another method is to tag commits you know are good - so you can return
to them easily using whatever-name-you-like.

HTH,
Bernt

Re: [org-babel] Does org-babel needs some simplification?

[Torsten Wagner]

Hi,

many thanks for the nice thoughts and posts.
To sum up, I think it might not be easy to remove parts of org-babel
since it is difficult to determine and a highly personal decision to
define what is important and what is unimportant.

Nevertheless Carten and Eric pointed out that the overhelming feature
set of org-babel, the fact that you could achive the same thing in
different ways and the missing of a "org-babel for dummies" might be a
problem for new org-babel users as well as for infrequent users.

Recently org-mode got his "org-mode for dummies" short manual. I guess
in the case of org-babel it might make more sense to create rather
typical examples for particular languages. This manuals could consist of
a typical example and of a template for this example which makes it easy
for beginners to fill in there own code and text.

Since Eric and the other org-babel and org-mode contributors are already
fully occupied with keeping org-babel and org-mode running, I would
suggest to collect a group of org-babel manual supporters. If possible
for each supported language one. This group could write up standard
situations for the particular language and maintain those manuscripts
whenever org-babel introduces some changes. In fact this group could
also serve as a kind of beta-testers for org-babel by trying on request
from e.g., Eric to compile there examples with the new org-babel
versions. I know there are some standard tests but I guess the do not go
that fare.

I guess, the manual maintainers do NOT have to be experts in both
org-mode resp. org-babel nore they have to be experts in the supported
language. Its more about the kind of standard stuff and maybe, to
complex stuff even scare people. More things like "How to create a
measurement protocol with org-babel and python", How to evaluate and
report data analysis with org-babel and R", etc.

To make it more easy for both the readers and the maintainers a kind of
template for such manuals might be helpful. This would help to find the
same information at the same locations and make a comparison e.g.
between the use of R and python possible.

I'am not an expert for both org-* and python and I'm often very limited
in time. However, I would try to maintain a "python and org-babel" manual.

If there are more people who are interested to act as a kind of manual
maintainers I would like to discuss with you how a template might look like.

Best regards

Torsten



On 06/29/2010 12:51 PM, Torsten Wagner wrote:
> Dear All,
> 
> as a (quite, but happy) org-bable user of the first hour I followed up
> the development process actively.
> Nevertheless, some weeks or months pass where I had no need for
> org-babel (yes, really strange I know).
> 
> Whenever I come back to org-babel, it takes me a huge amount of time to
> find myself back again in the syntax. Often I spend a day or two heavily
> reading the website and manual again to figure out how to make it working.
> 
> There are so many options. tangle files, results, scripting mode,
> sessions, noweb, lot, etc.
> 
> Just yesterday, I fighted again to make a simple python script running
> as desired to generate an automatic report. I did this dozen of times
> and even by using some old report as template I still struggle with it.
> Comparing old reports I noticed that I did it in many different ways.
> Tangeling all snipplets, using noweb syntax, with and without session
> support, etc.
> 
> Don't get me wrong, I really love org-babel and I think it is really
> great. I just wonder wether it has become too complex and too difficult
> to use to attract most of the org-mode people. Esp. considering people
> who use it not on a regular basis.
> 
> Best regards
> 
> Torsten

Re: [org-babel] Does org-babel needs some simplification?

[Eric Schulte]

Hi Torsten,

I love the idea of a "Babel for dummies" manual, and I'm an even bigger
fan of the manual being produced by user's of Babel (i.e. not myself).
I'll be more than happy to support this effort in any way.

Also, the beta-testing role you mention could be extremely helpful.  In
the absence of a comprehensive test suite it can be hard for Dan and I
to exhaustively check new features against all of the possible languages
and header argument combinations.  A filter of language-savvy users
exercising new Org-babel changes could very likely save the "Every day"
babel user (is there such a thing?) from many headaches.  Maybe
something like a sandbox-babel branch of the git repository would be
appropriate as a testing ground for new Babel commits.

That said the rate at which Babel is currently being developed is not
sustainable (at least not if I'm doing all of the development), and the
number of daily changes should drop dramatically in the next couple of
weeks.  So such a group may find itself without much work to do in the
not to distant future -- not that that would necessarily be a bad thing.

Thanks! -- Eric

Torsten Wagner <torsten.wagner@gmail.com> writes:

> Hi,
>
> many thanks for the nice thoughts and posts.
> To sum up, I think it might not be easy to remove parts of org-babel
> since it is difficult to determine and a highly personal decision to
> define what is important and what is unimportant.
>
> Nevertheless Carten and Eric pointed out that the overhelming feature
> set of org-babel, the fact that you could achive the same thing in
> different ways and the missing of a "org-babel for dummies" might be a
> problem for new org-babel users as well as for infrequent users.
>
> Recently org-mode got his "org-mode for dummies" short manual. I guess
> in the case of org-babel it might make more sense to create rather
> typical examples for particular languages. This manuals could consist of
> a typical example and of a template for this example which makes it easy
> for beginners to fill in there own code and text.
>
> Since Eric and the other org-babel and org-mode contributors are already
> fully occupied with keeping org-babel and org-mode running, I would
> suggest to collect a group of org-babel manual supporters. If possible
> for each supported language one. This group could write up standard
> situations for the particular language and maintain those manuscripts
> whenever org-babel introduces some changes. In fact this group could
> also serve as a kind of beta-testers for org-babel by trying on request
> from e.g., Eric to compile there examples with the new org-babel
> versions. I know there are some standard tests but I guess the do not go
> that fare.
>
> I guess, the manual maintainers do NOT have to be experts in both
> org-mode resp. org-babel nore they have to be experts in the supported
> language. Its more about the kind of standard stuff and maybe, to
> complex stuff even scare people. More things like "How to create a
> measurement protocol with org-babel and python", How to evaluate and
> report data analysis with org-babel and R", etc.
>
> To make it more easy for both the readers and the maintainers a kind of
> template for such manuals might be helpful. This would help to find the
> same information at the same locations and make a comparison e.g.
> between the use of R and python possible.
>
> I'am not an expert for both org-* and python and I'm often very limited
> in time. However, I would try to maintain a "python and org-babel" manual.
>
> If there are more people who are interested to act as a kind of manual
> maintainers I would like to discuss with you how a template might look like.
>
> Best regards
>
> Torsten
>
>
>
> On 06/29/2010 12:51 PM, Torsten Wagner wrote:
>> Dear All,
>> 
>> as a (quite, but happy) org-bable user of the first hour I followed up
>> the development process actively.
>> Nevertheless, some weeks or months pass where I had no need for
>> org-babel (yes, really strange I know).
>> 
>> Whenever I come back to org-babel, it takes me a huge amount of time to
>> find myself back again in the syntax. Often I spend a day or two heavily
>> reading the website and manual again to figure out how to make it working.
>> 
>> There are so many options. tangle files, results, scripting mode,
>> sessions, noweb, lot, etc.
>> 
>> Just yesterday, I fighted again to make a simple python script running
>> as desired to generate an automatic report. I did this dozen of times
>> and even by using some old report as template I still struggle with it.
>> Comparing old reports I noticed that I did it in many different ways.
>> Tangeling all snipplets, using noweb syntax, with and without session
>> support, etc.
>> 
>> Don't get me wrong, I really love org-babel and I think it is really
>> great. I just wonder wether it has become too complex and too difficult
>> to use to attract most of the org-mode people. Esp. considering people
>> who use it not on a regular basis.
>> 
>> Best regards
>> 
>> Torsten
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode