all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Fwd: [RFC] Self-configuring Org-mode files
       [not found] <t5xd3p8qlct.fsf@rgc.damtp.cam.ac.uk>
@ 2010-12-12  3:49 ` Thomas S. Dye
  2012-01-12 12:38   ` Rsquared for reproductible research Stephen Eglen
  0 siblings, 1 reply; 9+ messages in thread
From: Thomas S. Dye @ 2010-12-12  3:49 UTC (permalink / raw)
  To: emacs-orgmode ml


[-- Attachment #1.1: Type: text/plain, Size: 5085 bytes --]

Forwarding off-list replies from Stephen Eglen, most recent at the top.


> One caveat of course with the idea: what software other than emacs
> should be assumed to be available? e.g. if someone has some R code in
> the doc, you'll need R on the system, likewise, perl, python.  (I
> considered whether it was worth setting up a system for processing
> Sweave documents -- but a colleague pointed out that unless you can
> preload R with most of the popular R packages that are likely to be
> needed in a Sweave document, some form of intervention/curation is
> always needed.)

Begin forwarded message:

> From: Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk>
> Date: December 11, 2010 7:18:26 AM HST
> To: "Thomas S. Dye" <tsd@tsdye.com>
> Cc: S.J.Eglen@damtp.cam.ac.uk
> Subject: Re: [RFC] Self-configuring Org-mode files
>
> Hi Tom,
>
> I think you have hit the nail on the head -- it took me ages to get  
> org
> up and running with all the extra config to generate a beamer doc.   
> part
> of the trouble in that case was that i was following emails that  
> were a
> year old, but still I think the problem you outline is clear.  As to  
> the
> solution, your ideas sound sensible.
>
> To add one extra suggestion: how about a web-service of some sort  
> where
> you upload your .org file, and it returns the 'output' (e.g. pdf doc)
> produced by a native Emacs 23.X session running a stable version of  
> org
> with zero extra configuration?  I realise that it may be a lot of work
> to produce such a web service and expensive to maintain, but those
> issues aside, having a "vanilla" system to test RR org documents on  
> would
> be great.
>
> best wishes, Stephen
>
> "Thomas S. Dye" <tsd@tsdye.com> writes:
>
>> Aloha all,
>>
>> I'm eager to learn how to build a self-configuring Org-mode file.
>>
>> I think Org-mode is uniquely positioned to produce reproducible  
>> research
>> documents, but frankly speaking, the author of a reproducible Org- 
>> mode
>> research document would be foolish to release it.   The probability  
>> that it
>> would run out-of-the-box on a reader's computer is quite a bit less  
>> than 1.
>> Multiply this fraction by the number of potential readers and the  
>> probability
>> that the author will receive one or more emails entitled something  
>> like "Your
>> reproducible research isn't reproducible" rapidly approaches 1.
>>
>> A desire to produce reproducible research with Org-mode shouldn't  
>> reduce to an
>> open-ended commitment to solve configuration problems on users'  
>> computers
>> around the world.  An Org-mode file that reliably self-configures  
>> might solve
>> this problem.
>>
>> Based on the kind comments of several list members, I've come up  
>> with what I
>> hope are the rudiments of such a self-configuring Org-mode file.  I'm
>> requesting your comments because I believe, individually and  
>> collectively, you
>> know way more about this than I do.
>>
>> 1) At the end of the Org-mode file are some instructions saying  
>> something like
>> "To use this document, first
>> evaluate this code block", as suggested by Dan Davison:
>>
>> # Local variables:
>> # eval:(sbe "essential-document-config")
>> # End:
>>
>> 2) the source code block "essential-document-config" would consist  
>> of noweb
>> calls to Library of Babel functions:
>>
>> #+source: essential-document-config
>> #+begin_src emacs-lisp :noweb yes
>> <<lob-set-this-variable(x=t)>>
>> <<lob-set-that-variable(x=nil)>>
>> #+end_src
>>
>> 3) The Library of Babel would be
>> populated with little functions, each of which sets a single  
>> variable in a buffer-local way
>> (as suggested several times to me on the list):
>>
>> #+source: lob-set-this-variable
>> #+begin_src emacs-lisp :var x=nil
>> (set (make-local-variable 'this-variable) x)
>> #+end_src
>>
>> 4) A super-function in the Library of Babel would set the buffer- 
>> local
>> instance of every relevant Org-mode variable to its default state:
>>
>> #+source: lob-set-local-defaults
>> #+begin_src emacs-lisp
>> ...
>> #+end_src
>>
>> 5)
>> The Org-mode community would be responsible for populating the  
>> Library of Babel with the relevant little functions--whenever a new  
>> configuration variable is introduced, then a little function for it  
>> would be deposited in the Library of Babel.
>> The name of the configuration function would be part of the  
>> variable's
>> docstring.
>>
>> 6) The author of a reproducible Org-mode document could reasonably  
>> expect to
>> produce a document that self-configures on computers around the  
>> world, in a
>> way that is safe and effective.  The author would call the super- 
>> function,
>> then add noweb calls to little functions until the reproducible  
>> research
>> document works as expected.
>>
>> I'm happy for any comments, positive or negative.
>>
>> All the best,
>> Tom
>>
>> _______________________________________________
>> Emacs-orgmode mailing list
>> Please use `Reply All' to send replies to the list.
>> Emacs-orgmode@gnu.org
>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode


[-- Attachment #1.2: Type: text/html, Size: 10021 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Rsquared for reproductible research
  2010-12-12  3:49 ` Fwd: [RFC] Self-configuring Org-mode files Thomas S. Dye
@ 2012-01-12 12:38   ` Stephen Eglen
  2012-01-12 15:27     ` brian powell
  0 siblings, 1 reply; 9+ messages in thread
From: Stephen Eglen @ 2012-01-12 12:38 UTC (permalink / raw)
  To: emacs-orgmode

Following on from an old thread about self-configuring org files for
reproducible research, R users might be interested to see the following web site
which is exactly what I was thinking of for org mode (but of course, works
"only" for R packages.)

  http://rsquared.stat.uni-muenchen.de/index.rhtml

Stephen

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Rsquared for reproductible research
  2012-01-12 12:38   ` Rsquared for reproductible research Stephen Eglen
@ 2012-01-12 15:27     ` brian powell
  2012-01-12 15:40       ` Stephen Eglen
  0 siblings, 1 reply; 9+ messages in thread
From: brian powell @ 2012-01-12 15:27 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 1436 bytes --]

Wow! http://rsquared.stat.uni-muenchen.de/index.rhtml looks great--very
interesting--thanks for the heads-up on that link.

Worked with R/S/S-PLUS in grad school--easily my favorite
language/system--especially like its ease of extensibility--reminds me of
EMACS LISP!

I recognized your name, Stephen, as the long-time maintainer of the "Emacs
Lisp List"--thanks for that too.
Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode
implementations and/or Babel examples.
Or maybe optional extensions to OrgMode itself? Both?

Also, I very much agree that a "near exact replica" of the
http://rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great.

Read the 3 papers on the site and came across this reference that may be
interesting to OrgMode/R/Literate Programming persons:

http://cran.r-project.org/web/packages/roxygen/index.html

Maybe that could be a focus of such a site if it were made?: OrgMode <=>
LiterateProgramming <=> R/Bable/whatever other languages.

--where such topics intersect.

On Thu, Jan 12, 2012 at 7:38 AM, Stephen Eglen <sje30@cam.ac.uk> wrote:

> Following on from an old thread about self-configuring org files for
> reproducible research, R users might be interested to see the following
> web site
> which is exactly what I was thinking of for org mode (but of course, works
> "only" for R packages.)
>
>  http://rsquared.stat.uni-muenchen.de/index.rhtml
>
> Stephen
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2296 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Rsquared for reproductible research
  2012-01-12 15:27     ` brian powell
@ 2012-01-12 15:40       ` Stephen Eglen
  2012-01-12 16:29         ` Eric Schulte
  0 siblings, 1 reply; 9+ messages in thread
From: Stephen Eglen @ 2012-01-12 15:40 UTC (permalink / raw)
  To: emacs-orgmode; +Cc: Stephen Eglen


> I recognized your name, Stephen, as the long-time maintainer of the "Emacs
> Lisp List"--thanks for that too.  
> Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode
> implementations and/or Babel examples.

I am interested in maintaining a collection of nice org-babel-R
examples, given my interest in R and Emacs; I made a small start last
year for my tutorial notes on ESS for the useR meeting, but I'd like to
update that.  The key problem I found with learning org-babel was worked
examples, given that the syntax was changing rapidly.  I hope that has
now stabilised, and I should followup with Eric about this.

(I also hope to kill off the Emacs Lisp List once the ELPA has taken
off.)

> Also, I very much agree that a "near exact replica" of the http://
> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great.
Yes!  Any takers?!?

> http://cran.r-project.org/web/packages/roxygen/index.html

Note that ESS has limited support for editing roxygen tags.

Stephen

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Rsquared for reproductible research
  2012-01-12 15:40       ` Stephen Eglen
@ 2012-01-12 16:29         ` Eric Schulte
  2012-01-12 18:24           ` Thomas S. Dye
  2012-01-12 20:54           ` Stephen Eglen
  0 siblings, 2 replies; 9+ messages in thread
From: Eric Schulte @ 2012-01-12 16:29 UTC (permalink / raw)
  To: Stephen Eglen; +Cc: emacs-orgmode

Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> writes:

>> I recognized your name, Stephen, as the long-time maintainer of the "Emacs
>> Lisp List"--thanks for that too.  
>> Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode
>> implementations and/or Babel examples.
>
> I am interested in maintaining a collection of nice org-babel-R
> examples, given my interest in R and Emacs; I made a small start last
> year for my tutorial notes on ESS for the useR meeting, but I'd like to
> update that.  The key problem I found with learning org-babel was worked
> examples, given that the syntax was changing rapidly.  I hope that has
> now stabilised, and I should followup with Eric about this.
>

The syntax is now stabilized (we wanted to get this sorted before the
final Emacs24 merge).  That which is currently described in the manual
is and should remain the proper Org-mode code block syntax.

For many small examples, please see [1] which I (unfortunately) haven't
been adding to recently, but I will try to once again start using for
all of my small generally-mailing-list-inspired Babel one-offs.

>
> (I also hope to kill off the Emacs Lisp List once the ELPA has taken
> off.)
>
>> Also, I very much agree that a "near exact replica" of the http://
>> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great.
> Yes!  Any takers?!?
>

From looking at the fairly terse web site for R^2 it is not clear to me
exactly what the system includes (I'm sure I'm missing something
obvious).  It seems to be the addition of a packaging system over-top of
R source files.  What would a potential Org-mode based system provide
which is not already possible with Org-mode text files, Org-mode
publishing and a version control repository.

Perhaps the benefit would simply be a system which eases the integration
of Org, publishing, version control, and possibly automatic Makefile
creation with tasks like "publish", "clean" etc...

Best,

>
>> http://cran.r-project.org/web/packages/roxygen/index.html
>
> Note that ESS has limited support for editing roxygen tags.
>
> Stephen
>
>


Footnotes: 
[1]  http://eschulte.me/org-scraps/

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Rsquared for reproductible research
  2012-01-12 16:29         ` Eric Schulte
@ 2012-01-12 18:24           ` Thomas S. Dye
  2012-01-12 20:54           ` Stephen Eglen
  1 sibling, 0 replies; 9+ messages in thread
From: Thomas S. Dye @ 2012-01-12 18:24 UTC (permalink / raw)
  To: Eric Schulte; +Cc: emacs-orgmode, Stephen Eglen

Eric Schulte <eric.schulte@gmx.com> writes:

> Stephen Eglen <S.J.Eglen@damtp.cam.ac.uk> writes:
>
>>> I recognized your name, Stephen, as the long-time maintainer of the "Emacs
>>> Lisp List"--thanks for that too.  
>>> Are you envisioning a repository beyond "Emacs Lisp List" for OrgMode
>>> implementations and/or Babel examples.
>>
>> I am interested in maintaining a collection of nice org-babel-R
>> examples, given my interest in R and Emacs; I made a small start last
>> year for my tutorial notes on ESS for the useR meeting, but I'd like to
>> update that.  The key problem I found with learning org-babel was worked
>> examples, given that the syntax was changing rapidly.  I hope that has
>> now stabilised, and I should followup with Eric about this.
>>
>
> The syntax is now stabilized (we wanted to get this sorted before the
> final Emacs24 merge).  That which is currently described in the manual
> is and should remain the proper Org-mode code block syntax.
>
> For many small examples, please see [1] which I (unfortunately) haven't
> been adding to recently, but I will try to once again start using for
> all of my small generally-mailing-list-inspired Babel one-offs.
>
>>
>> (I also hope to kill off the Emacs Lisp List once the ELPA has taken
>> off.)
>>
>>> Also, I very much agree that a "near exact replica" of the http://
>>> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great.
>> Yes!  Any takers?!?
>>
>
> From looking at the fairly terse web site for R^2 it is not clear to me
> exactly what the system includes (I'm sure I'm missing something
> obvious).  It seems to be the addition of a packaging system over-top of
> R source files.  What would a potential Org-mode based system provide
> which is not already possible with Org-mode text files, Org-mode
> publishing and a version control repository.

One thing that caught my eye was the facility that compared results
across operating systems.

Tom

>
> Perhaps the benefit would simply be a system which eases the integration
> of Org, publishing, version control, and possibly automatic Makefile
> creation with tasks like "publish", "clean" etc...
>
> Best,
>
>>
>>> http://cran.r-project.org/web/packages/roxygen/index.html
>>
>> Note that ESS has limited support for editing roxygen tags.
>>
>> Stephen
>>
>>
>
>
> Footnotes: 
> [1]  http://eschulte.me/org-scraps/

-- 
Thomas S. Dye
http://www.tsdye.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Rsquared for reproductible research
@ 2012-01-12 19:44 brian powell
  2012-01-12 20:33 ` Eric Schulte
  0 siblings, 1 reply; 9+ messages in thread
From: brian powell @ 2012-01-12 19:44 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 3297 bytes --]

>> Also, I very much agree that a "near exact replica" of the http://
>> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great.
> Yes!  Any takers?!?
>
...
Eric questioned:
"From looking at the fairly terse web site for R^2 it is not clear to me
exactly what the system includes (I'm sure I'm missing something
obvious).  It seems to be the addition of a packaging system over-top of
R source files.  What would a potential Org-mode based system provide
which is not already possible with Org-mode text files, Org-mode
publishing and a version control repository."
...

* I mostly agree with your statements. Good challenges. I did more
investigation: This link to the paper that  "Friedrich Leischa, , Manuel
Eugsterb, Torsten Hothornb" put together may make things clearer--this
paper really seems to be the justification/impetus for the R^2 website--it
has made things clearer and more exciting for me:

http://www.sciencedirect.com/science/article/pii/S1877050911001232

<=> "Executable Papers for the R Community: The R2 Platform for
Reproducible Research"

** So papers in R (and maybe other languages--maybe languages run thru
Babel in OrgMode) could be "executed" by people in the community--one could
verify research studies and papers interactively, ad hoc.

*** They mention in the paper that they use several key tools:

"R: the lingua franca of statistics and data analysis
Sweave: the most popular format for executable papers in the R community
CRAN: package building and checking system has been developed for more then
a decade and copes successfully
with the exponential growth of the number of packages"

**** Weave/CWEB/CWEAVE/CTANGLE=>NOWEB (Knuth) comes to mind here--i.e.
Literate Programming

***** http://www-cs-staff.stanford.edu/~uno/cweb.html =>

CTANGLE
converts a source file foo.w to a compilable program file foo.c;
CWEAVE
converts a source file foo.w to a prettily-printable and cross-indexed
document file foo.tex.

* Exactly the paper can be found at:

http://www.sciencedirect.com/science\
?_ob=MiamiImageURL&_cid=280203&_user=10&_pii=S1877050911001232&\
_check=y&_origin=article&_zone=toolbar&_coverDate=\
31-Dec-2011&view=c&originContentFamily=serial&wchp=\
dGLbVlS-zSkWb&md5=4681e5babd7822f321d2a0dd3a9f11cf/\
1-s2.0-S1877050911001232-main.pdf

* I agree Eric that the website is a bit terse; but, for the most part I
was excited about (and I think Stephen is interested in--he suggested it is
something the community might do) the general ideas, the structure of the
website's process "OrgMode=>TeX paper
in"-->...process...-->Executable/verifiable code interaction a user might
experience/stored on-line for researchers (one thing I always enjoy a lot
when working with e.g. R/S-PLUS and PYTHON's interactive CLI, etc.)

** They even publish the minute details of the settings on the
machines--the local environment variables, etc.--the devil is in the
details!

* Thanks for the link Eric to your OrgMode "scraps"--they could be very
useful:

http://eschulte.me/org-scraps/

* Lastly, most importantly I'll repeat the link and query to the community:
...
>> Also, I very much agree that a "near exact replica" of the
http://rsquared.stat.uni-muenchen.de/index.rhtml
>>for OrgMode would be great.
> Yes!  Any takers?!?    ---(Stephen Eglen)

[-- Attachment #2: Type: text/html, Size: 4636 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Rsquared for reproductible research
  2012-01-12 19:44 brian powell
@ 2012-01-12 20:33 ` Eric Schulte
  0 siblings, 0 replies; 9+ messages in thread
From: Eric Schulte @ 2012-01-12 20:33 UTC (permalink / raw)
  To: brian powell; +Cc: emacs-orgmode

brian powell <briangpowellms@gmail.com> writes:

>>> Also, I very much agree that a "near exact replica" of the http://
>>> rsquared.stat.uni-muenchen.de/index.rhtml for OrgMode would be great.
>> Yes!  Any takers?!?
>>
> ...
> Eric questioned:
> "From looking at the fairly terse web site for R^2 it is not clear to me
> exactly what the system includes (I'm sure I'm missing something
> obvious).  It seems to be the addition of a packaging system over-top of
> R source files.  What would a potential Org-mode based system provide
> which is not already possible with Org-mode text files, Org-mode
> publishing and a version control repository."
> ...
>
> * I mostly agree with your statements. Good challenges. I did more
> investigation: This link to the paper that  "Friedrich Leischa, , Manuel
> Eugsterb, Torsten Hothornb" put together may make things clearer--this
> paper really seems to be the justification/impetus for the R^2 website--it
> has made things clearer and more exciting for me:
>
> http://www.sciencedirect.com/science/article/pii/S1877050911001232
>

Ah, thank you for linking to this paper.  It seems I was missing was the
package-management aspect provided by R2 through CRAN.  The instillation
of all software dependencies is a huge benefit exactly as installing
software with apt-get or pacman is simpler than running "./configure &&
make" and manually resolving dependencies.

While such a tool makes sense for a single language system like R, I
fear an Org-mode version of such a system would have too wide of a
scope.  Given that code blocks may contain arbitrary languages, and that
sh blocks can freely call any command-line executable such a system
would turn into a system-wide package management tool.

Perhaps there already exists a portable package management system
designed for local installs which could handle most of the heavy
lifting.

As another option, distributing Virtual Machine images are one solution
which I think work well and are increasingly realistic.  Or similarly
providing the research environment as a cloud server image (e.g., Amazon
EC2).

Certainly an interesting area for further work!

Best,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Rsquared for reproductible research
  2012-01-12 16:29         ` Eric Schulte
  2012-01-12 18:24           ` Thomas S. Dye
@ 2012-01-12 20:54           ` Stephen Eglen
  1 sibling, 0 replies; 9+ messages in thread
From: Stephen Eglen @ 2012-01-12 20:54 UTC (permalink / raw)
  To: Eric Schulte; +Cc: emacs-orgmode, Stephen Eglen


> The syntax is now stabilized (we wanted to get this sorted before the
> final Emacs24 merge).  That which is currently described in the manual
> is and should remain the proper Org-mode code block syntax.

Thanks Eric, this is great news.  I'll ensure my examples from last year
still work with the current org-mode syntax.


> For many small examples, please see [1] which I (unfortunately) haven't

That's a nice site, thanks!

> From looking at the fairly terse web site for R^2 it is not clear to me
> exactly what the system includes (I'm sure I'm missing something
> obvious).  It seems to be the addition of a packaging system over-top of
> R source files.  What would a potential Org-mode based system provide
> which is not already possible with Org-mode text files, Org-mode
> publishing and a version control repository.

The accompanying paper from 2011 goes into detail as to what it does.
But in essence, the website is supposed to be 'neutral', in that it is
not your local system.  A document may compile on my system, but not for
others, because it depends on my local configuration.  Having a neutral
system will avoid such problems.

A neutral system could also have support for all babel languages, so
that I can use it to compile someone's document without e.g. having a
common-lisp compiler on my system.

> Perhaps the benefit would simply be a system which eases the integration
> of Org, publishing, version control, and possibly automatic Makefile
> creation with tasks like "publish", "clean" etc...

yes, that too!

best wishes,
Stephen

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2012-01-12 20:54 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <t5xd3p8qlct.fsf@rgc.damtp.cam.ac.uk>
2010-12-12  3:49 ` Fwd: [RFC] Self-configuring Org-mode files Thomas S. Dye
2012-01-12 12:38   ` Rsquared for reproductible research Stephen Eglen
2012-01-12 15:27     ` brian powell
2012-01-12 15:40       ` Stephen Eglen
2012-01-12 16:29         ` Eric Schulte
2012-01-12 18:24           ` Thomas S. Dye
2012-01-12 20:54           ` Stephen Eglen
2012-01-12 19:44 brian powell
2012-01-12 20:33 ` Eric Schulte

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.