Fwd: [RFC] Self-configuring Org-mode files

Fwd: [RFC] Self-configuring Org-mode files

[Thomas S. Dye]

[-- Attachment #1.1: Type: text/plain, Size: 5085 bytes --]

Forwarding off-list replies from Stephen Eglen, most recent at the top.


> One caveat of course with the idea: what software other than emacs
> should be assumed to be available? e.g. if someone has some R code in
> the doc, you'll need R on the system, likewise, perl, python.  (I
> considered whether it was worth setting up a system for processing
> Sweave documents -- but a colleague pointed out that unless you can
> preload R with most of the popular R packages that are likely to be
> needed in a Sweave document, some form of intervention/curation is
> always needed.)

Begin forwarded message:

> From: Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk>
> Date: December 11, 2010 7:18:26 AM HST
> To: "Thomas S. Dye" <tsd@tsdye.com>
> Cc: S.J.Eglen@damtp.cam.ac.uk
> Subject: Re: [RFC] Self-configuring Org-mode files
>
> Hi Tom,
>
> I think you have hit the nail on the head -- it took me ages to get  
> org
> up and running with all the extra config to generate a beamer doc.   
> part
> of the trouble in that case was that i was following emails that  
> were a
> year old, but still I think the problem you outline is clear.  As to  
> the
> solution, your ideas sound sensible.
>
> To add one extra suggestion: how about a web-service of some sort  
> where
> you upload your .org file, and it returns the 'output' (e.g. pdf doc)
> produced by a native Emacs 23.X session running a stable version of  
> org
> with zero extra configuration?  I realise that it may be a lot of work
> to produce such a web service and expensive to maintain, but those
> issues aside, having a "vanilla" system to test RR org documents on  
> would
> be great.
>
> best wishes, Stephen
>
> "Thomas S. Dye" <tsd@tsdye.com> writes:
>
>> Aloha all,
>>
>> I'm eager to learn how to build a self-configuring Org-mode file.
>>
>> I think Org-mode is uniquely positioned to produce reproducible  
>> research
>> documents, but frankly speaking, the author of a reproducible Org- 
>> mode
>> research document would be foolish to release it.   The probability  
>> that it
>> would run out-of-the-box on a reader's computer is quite a bit less  
>> than 1.
>> Multiply this fraction by the number of potential readers and the  
>> probability
>> that the author will receive one or more emails entitled something  
>> like "Your
>> reproducible research isn't reproducible" rapidly approaches 1.
>>
>> A desire to produce reproducible research with Org-mode shouldn't  
>> reduce to an
>> open-ended commitment to solve configuration problems on users'  
>> computers
>> around the world.  An Org-mode file that reliably self-configures  
>> might solve
>> this problem.
>>
>> Based on the kind comments of several list members, I've come up  
>> with what I
>> hope are the rudiments of such a self-configuring Org-mode file.  I'm
>> requesting your comments because I believe, individually and  
>> collectively, you
>> know way more about this than I do.
>>
>> 1) At the end of the Org-mode file are some instructions saying  
>> something like
>> "To use this document, first
>> evaluate this code block", as suggested by Dan Davison:
>>
>> # Local variables:
>> # eval:(sbe "essential-document-config")
>> # End:
>>
>> 2) the source code block "essential-document-config" would consist  
>> of noweb
>> calls to Library of Babel functions:
>>
>> #+source: essential-document-config
>> #+begin_src emacs-lisp :noweb yes
>> <<lob-set-this-variable(x=t)>>
>> <<lob-set-that-variable(x=nil)>>
>> #+end_src
>>
>> 3) The Library of Babel would be
>> populated with little functions, each of which sets a single  
>> variable in a buffer-local way
>> (as suggested several times to me on the list):
>>
>> #+source: lob-set-this-variable
>> #+begin_src emacs-lisp :var x=nil
>> (set (make-local-variable 'this-variable) x)
>> #+end_src
>>
>> 4) A super-function in the Library of Babel would set the buffer- 
>> local
>> instance of every relevant Org-mode variable to its default state:
>>
>> #+source: lob-set-local-defaults
>> #+begin_src emacs-lisp
>> ...
>> #+end_src
>>
>> 5)
>> The Org-mode community would be responsible for populating the  
>> Library of Babel with the relevant little functions--whenever a new  
>> configuration variable is introduced, then a little function for it  
>> would be deposited in the Library of Babel.
>> The name of the configuration function would be part of the  
>> variable's
>> docstring.
>>
>> 6) The author of a reproducible Org-mode document could reasonably  
>> expect to
>> produce a document that self-configures on computers around the  
>> world, in a
>> way that is safe and effective.  The author would call the super- 
>> function,
>> then add noweb calls to little functions until the reproducible  
>> research
>> document works as expected.
>>
>> I'm happy for any comments, positive or negative.
>>
>> All the best,
>> Tom
>>
>> _______________________________________________
>> Emacs-orgmode mailing list
>> Please use `Reply All' to send replies to the list.
>> Emacs-orgmode@gnu.org
>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode


[-- Attachment #1.2: Type: text/html, Size: 10021 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Rsquared for reproductible research

[Stephen Eglen]

Following on from an old thread about self-configuring org files for
reproducible research, R users might be interested to see the following web site
which is exactly what I was thinking of for org mode (but of course, works
"only" for R packages.)

  http://rsquared.stat.uni-muenchen.de/index.rhtml

Stephen

Re: Rsquared for reproductible research

[brian powell]

[-- Attachment #1: Type: text/plain, Size: 1436 bytes --]

Wow! http://rsquared.stat.uni-muenchen.de/index.rhtml looks great--very
interesting--thanks for the heads-up on that link.

Worked with R/S/S-PLUS in grad school--easily my favorite
language/system--especially like its ease of extensibility--reminds me of
EMACS LISP!

I recognized your name, Stephen, as the long-time maintainer of the "Emacs
Lisp List"--thanks for that too.
Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode
implementations and/or Babel examples.
Or maybe optional extensions to OrgMode itself? Both?

Also, I very much agree that a "near exact replica" of the
http://rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great.

Read the 3 papers on the site and came across this reference that may be
interesting to OrgMode/R/Literate Programming persons:

http://cran.r-project.org/web/packages/roxygen/index.html

Maybe that could be a focus of such a site if it were made?: OrgMode <=>
LiterateProgramming <=> R/Bable/whatever other languages.

--where such topics intersect.

On Thu, Jan 12, 2012 at 7:38 AM, Stephen Eglen <sje30@cam.ac.uk> wrote:

> Following on from an old thread about self-configuring org files for
> reproducible research, R users might be interested to see the following
> web site
> which is exactly what I was thinking of for org mode (but of course, works
> "only" for R packages.)
>
>  http://rsquared.stat.uni-muenchen.de/index.rhtml
>
> Stephen
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2296 bytes --]

Re: Rsquared for reproductible research

[Stephen Eglen]

> I recognized your name, Stephen, as the long-time maintainer of the "Emacs
> Lisp List"--thanks for that too.  
> Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode
> implementations and/or Babel examples.

I am interested in maintaining a collection of nice org-babel-R
examples, given my interest in R and Emacs; I made a small start last
year for my tutorial notes on ESS for the useR meeting, but I'd like to
update that.  The key problem I found with learning org-babel was worked
examples, given that the syntax was changing rapidly.  I hope that has
now stabilised, and I should followup with Eric about this.

(I also hope to kill off the Emacs Lisp List once the ELPA has taken
off.)

> Also, I very much agree that a "near exact replica" of the http://
> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great.
Yes!  Any takers?!?

> http://cran.r-project.org/web/packages/roxygen/index.html

Note that ESS has limited support for editing roxygen tags.

Stephen

Re: Rsquared for reproductible research

[Eric Schulte]

Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> writes:

>> I recognized your name, Stephen, as the long-time maintainer of the "Emacs
>> Lisp List"--thanks for that too.  
>> Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode
>> implementations and/or Babel examples.
>
> I am interested in maintaining a collection of nice org-babel-R
> examples, given my interest in R and Emacs; I made a small start last
> year for my tutorial notes on ESS for the useR meeting, but I'd like to
> update that.  The key problem I found with learning org-babel was worked
> examples, given that the syntax was changing rapidly.  I hope that has
> now stabilised, and I should followup with Eric about this.
>

The syntax is now stabilized (we wanted to get this sorted before the
final Emacs24 merge).  That which is currently described in the manual
is and should remain the proper Org-mode code block syntax.

For many small examples, please see [1] which I (unfortunately) haven't
been adding to recently, but I will try to once again start using for
all of my small generally-mailing-list-inspired Babel one-offs.

>
> (I also hope to kill off the Emacs Lisp List once the ELPA has taken
> off.)
>
>> Also, I very much agree that a "near exact replica" of the http://
>> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great.
> Yes!  Any takers?!?
>

From looking at the fairly terse web site for R^2 it is not clear to me
exactly what the system includes (I'm sure I'm missing something
obvious).  It seems to be the addition of a packaging system over-top of
R source files.  What would a potential Org-mode based system provide
which is not already possible with Org-mode text files, Org-mode
publishing and a version control repository.

Perhaps the benefit would simply be a system which eases the integration
of Org, publishing, version control, and possibly automatic Makefile
creation with tasks like "publish", "clean" etc...

Best,

>
>> http://cran.r-project.org/web/packages/roxygen/index.html
>
> Note that ESS has limited support for editing roxygen tags.
>
> Stephen
>
>


Footnotes: 
[1]  http://eschulte.me/org-scraps/

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Rsquared for reproductible research

[Thomas S. Dye]

Eric Schulte <eric.schulte@gmx.com> writes:

> Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> writes:
>
>>> I recognized your name, Stephen, as the long-time maintainer of the "Emacs
>>> Lisp List"--thanks for that too.  
>>> Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode
>>> implementations and/or Babel examples.
>>
>> I am interested in maintaining a collection of nice org-babel-R
>> examples, given my interest in R and Emacs; I made a small start last
>> year for my tutorial notes on ESS for the useR meeting, but I'd like to
>> update that.  The key problem I found with learning org-babel was worked
>> examples, given that the syntax was changing rapidly.  I hope that has
>> now stabilised, and I should followup with Eric about this.
>>
>
> The syntax is now stabilized (we wanted to get this sorted before the
> final Emacs24 merge).  That which is currently described in the manual
> is and should remain the proper Org-mode code block syntax.
>
> For many small examples, please see [1] which I (unfortunately) haven't
> been adding to recently, but I will try to once again start using for
> all of my small generally-mailing-list-inspired Babel one-offs.
>
>>
>> (I also hope to kill off the Emacs Lisp List once the ELPA has taken
>> off.)
>>
>>> Also, I very much agree that a "near exact replica" of the http://
>>> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great.
>> Yes!  Any takers?!?
>>
>
> From looking at the fairly terse web site for R^2 it is not clear to me
> exactly what the system includes (I'm sure I'm missing something
> obvious).  It seems to be the addition of a packaging system over-top of
> R source files.  What would a potential Org-mode based system provide
> which is not already possible with Org-mode text files, Org-mode
> publishing and a version control repository.

One thing that caught my eye was the facility that compared results
across operating systems.

Tom

>
> Perhaps the benefit would simply be a system which eases the integration
> of Org, publishing, version control, and possibly automatic Makefile
> creation with tasks like "publish", "clean" etc...
>
> Best,
>
>>
>>> http://cran.r-project.org/web/packages/roxygen/index.html
>>
>> Note that ESS has limited support for editing roxygen tags.
>>
>> Stephen
>>
>>
>
>
> Footnotes: 
> [1]  http://eschulte.me/org-scraps/

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Rsquared for reproductible research

[Stephen Eglen]

> The syntax is now stabilized (we wanted to get this sorted before the
> final Emacs24 merge).  That which is currently described in the manual
> is and should remain the proper Org-mode code block syntax.

Thanks Eric, this is great news.  I'll ensure my examples from last year
still work with the current org-mode syntax.


> For many small examples, please see [1] which I (unfortunately) haven't

That's a nice site, thanks!

> From looking at the fairly terse web site for R^2 it is not clear to me
> exactly what the system includes (I'm sure I'm missing something
> obvious).  It seems to be the addition of a packaging system over-top of
> R source files.  What would a potential Org-mode based system provide
> which is not already possible with Org-mode text files, Org-mode
> publishing and a version control repository.

The accompanying paper from 2011 goes into detail as to what it does.
But in essence, the website is supposed to be 'neutral', in that it is
not your local system.  A document may compile on my system, but not for
others, because it depends on my local configuration.  Having a neutral
system will avoid such problems.

A neutral system could also have support for all babel languages, so
that I can use it to compile someone's document without e.g. having a
common-lisp compiler on my system.

> Perhaps the benefit would simply be a system which eases the integration
> of Org, publishing, version control, and possibly automatic Makefile
> creation with tasks like "publish", "clean" etc...

yes, that too!

best wishes,
Stephen