From: Dan Davison <davison@stats.ox.ac.uk>
To: emacs-orgmode@gnu.org
Subject: Re: graphing from org-tables
Date: Thu, 22 Jan 2009 11:27:11 -0500 [thread overview]
Message-ID: <20090122162711.GA11250@stats.ox.ac.uk> (raw)
In-Reply-To: <87r640p2ld.fsf@gmail.com>
On Mon, Dec 22, 2008 at 12:00:14PM -0500, Eric Schulte wrote:
> Hi Dan,
>
<...>
> 2) Also interesting is the idea of referencing a table from a block of R
> code elsewhere in the org file. I've worked some on processing
> blocks of R code in org files in a manner similar to Sweave.
> http://github.com/eschulte/org-contrib/tree/master/org-exp-blocks.el
> If we could implement a simple means of referencing tables (either
> whole tables, or cells/cols/rows of tables) from these R-code blocks
> then that seems like it may be fairly powerful.
I agree with this. So in the case of my code, I think we would like to
be able to do something like
#+TBLR: data:<org-table-reference> columns:(1 2) action:tabulate
This would make the output of the analysis/plot of the table pointed
to by <org-table-reference> appear at the location in the file of the
#+TBLR line. Perhaps one design aim would be to have the same
mechanism work for referencing tables for processing by #TBLFM,
org-plot, org-table-R, org-exp-blocks, etc. What ideas to people have
about implementation (and syntax) for this sort of table referencing?
Dan
>
> Thanks -- Eric
>
> Dan Davison <davison@stats.ox.ac.uk> writes:
>
> > On Mon, Jul 28, 2008 at 04:41:52PM +0100, Dan Davison wrote:
> >> On Mon, Jul 28, 2008 at 08:32:22AM -0700, Eric Schulte wrote:
> >> > Hi Dan,
> >> >
> >> > One way around the sleep(60) hack may be to create the R graph using
> >> > an inferior R process. See
> >>
> >> Yep, I just this minute asked for help on the Emacs Speaks Statistics
> >> (ESS) mailing list! I'll report back if I make progress with that.
> >>
> >> https://stat.ethz.ch/pipermail/ess-help/2008-July/004785.html
> >
> > I think it's worth keeping this avenue alive, despite so much progress
> > having been made with gnuplot in the interim. So below is an initial
> > version of org-table-eval-R, which takes an arbitrary R function, and
> > applies it to an org table. As suggested above, this uses
> > inferior-ess-mode and therefore requires ess (emacs speaks statistics)
> > and R to be installed.
> >
> > At it's simplest, R-function can be the name of a standard R
> > function. So with Eric's first example data set on Worg
> > (http://legito.net/worg/org-tutorials/org-plot.php) i.e. with 2
> > dependent variables
> >
> > (org-table-eval-R "summary") produces (in a new buffer)
> >
> > first.dependent.var second.dependent.var
> > Min. :0.1560 Min. :0.1808
> > 1st Qu.:0.1773 1st Qu.:0.1985
> > Median :0.1900 Median :0.2146
> > Mean :0.2110 Mean :0.2262
> > 3rd Qu.:0.2333 3rd Qu.:0.2375
> > Max. :0.4250 Max. :0.3750
> >
> > and (org-table-eval-R "matplot") produces a rough version of Eric's
> > gnuplot figure. (see below for a closer reproduction of the figure,
> > with the legend etc).
> >
> > It would be simple to provide org-plot/R which would construct a
> > suitable plotting function and pass it to org-table-eval-R, and some
> > ready-made R code could be provided for standard plots -- histograms &
> > barplots, scatter plots, etc. I've given an initial version of
> > org-plot/R below, which gives the option to produce eps output rather
> > than a pop-up window (and it would be easy to use any of the R
> > graphics devices for output: jpeg, png, ps, pdf, tiff, X11, quartz,
> > etc)
> >
> > But the power of this approach is that it provides not just plotting
> > functionality, but the ability to do arbitrary calculations and
> > statistical analyses of the org table, via the base R packages and the
> > hundreds of user-written packages.
> >
> > It would have been better to have developed these ideas along with the
> > development of org-plot, but unfortunately I let that slip. But
> > org-table-eval-R provides a lot of power for anyone that's prepared to
> > write some R code. And if there's interest, then some more
> > user-friendly functionality could be provided for org users that want
> > something easier, but can install ess and R.
> >
> > Dan
> >
> > and here's a rough version of the "O R G" grid plot
> >
> > (org-table-eval-R "(function (x) image(t(as.matrix(x))))" t)
> >
> > Of course, these can be beautified to your heart's content with lots
> > of extra arguments to the R plotting functions. This comes a bit
> > closer to the two dependent-variables graph:
> >
> > (org-table-eval-R "(function(x) { matplot(x, type=\"l\", ylab=\"\") ; legend(\"topright\", legend=c(\"dep var 1\"\,\"dep var 2\"), col=1:2, lty=1:2) })")
> >
> >
> >
> > --------------------------------------------------------------
> > (defun org-table-eval-R (R-function &optional rownames-absent)
> > "Apply an arbitrary R function to the org table at
> > point. R-FUNCTION is an R function (a string) taking one
> > argument, which is the R data frame representation of the org
> > table. It may be the name of an appropriate R
> > function (e.g. \"summary\", \"plot\"), or a user-defined
> > anonymous function of the form \"(function(data.frame)
> > {...})\". Unless ROWNAMES-ABSENT is non-nil the row names of the
> > data frame are taken from the first column of the org table"
> > (interactive)
> > (require 'ess)
> > (let ((csv-file (make-temp-file "org-table-eval-R-")))
> > (org-table-export csv-file "orgtbl-to-csv")
> > (ess-execute (org-table-eval-R-make-expr
> > R-function csv-file rownames-absent))
> > (delete-file csv-file)))
> >
> > (defun org-table-eval-R-make-expr (R-function csv-file rownames-absent)
> > (concat R-function
> > "(read.csv(\"" csv-file "\""
> > (unless rownames-absent ", row.names=1") "))"))
> >
> >
> > (defun org-plot/R (&optional plot-function file rownames-absent)
> > "Use R to create a plot using the org table at point. If FILE
> > is non-nil then the plot will be written to file (currently as
> > eps, in future the format could easily be determined by the file
> > extension.) Unless ROWNAMES-ABSENT is non-nil the row names will
> > be taken from the first column of the org table."
> > (interactive)
> > (require 'ess)
> > (org-table-eval-R (or plot-function "plot") rownames-absent)
> > (if file (ess-execute (concat "dev.copy2eps(\"" file "\")"))))
> >
> >
> >
> >
> >>
> >> Dan
> >>
> >>
> >>
> >>
> >> > http://cran.r-project.org/doc/FAQ/R-FAQ.html#R-and-Emacs for
> >> > information on running R functions from inside of Emacs.
> >> >
> >> > Best -- Eric
> >> >
> >> > On Saturday, July 26, at 19:15, Dan Davison wrote:
> >> > > R (www.r-project.org) is pretty good for data plotting and statistical
> >> > > analyses. Here's my effort at the org-table-plot function, using
> >> > > R. Since R contains a csv importer that can read from stdin, it's
> >> > > pretty simple. I've tried to code it so that you can provide an
> >> > > arbitrary R function as the optional argument, so in principle you can
> >> > > do to your org-table anything that R is capable of in the realms of
> >> > > data analysis and visualisation.
> >> > >
> >> > > (defun org-table-R-plot (&optional R-function)
> >> > > "Plot the current table using R. The table is transformed into a dataframe in R. Optional
> >> > > argument R-function is a string which is either the name of an R
> >> > > function, or an anonymous function definition of the form (function(d) {...}),
> >> > > requiring a single argument (the dataframe). The default is to use
> >> > > the R function 'plot' which produces scatter plots of all pairwise
> >> > > combinations of columns. An example custom plotting function is:
> >> > > * plot column 3 against column 1, adding least-squares linear regression fit in blue (function(df) { plot(x=df[,1], y=df[,3]) ; abline(lm(df[,3] ~ df[,1]), col=\"blue\") }) "
> >> > > (interactive)
> >> > > (unless R-function (set 'R-function "plot"))
> >> > > (let ((file (make-temp-file "org-table-R-plot")))
> >> > > (org-table-export file "orgtbl-to-csv")
> >> > > (set-buffer (find-file-noselect file))
> >> > > (shell-command-on-region
> >> > > (point-min) (point-max)
> >> > > (concat "Rscript -e 'X11() ; " R-function "(read.csv(\"stdin\")) ; system(\"sleep 60\")'"))
> >> > > (delete-file file)))
> >> > >
> >> > >
> >> > > R is at www.r-project.org
> >> > > (package r-base on ubuntu/debian)
> >> > >
> >> > > Rscript is a command line non-interactive scripting utility that is
> >> > > bundled automatically with the R installation. I reckon it'll be OK on
> >> > > OSX but no idea about Windows. My function doesn't have to be used
> >> > > for plotting; the R-function argument can be any function operating on
> >> > > the data from the org-table, producing numerical or graphical output.
> >> > >
> >> > > There's several things that need to be sorted out with my function, e.g.
> >> > >
> >> > > (i) I haven't worked out how to return control to the emacs process
> >> > > while keeping the plot window there. I tried adding an & to the
> >> > > shell-command, but that seemed to result in R receiving nothing on
> >> > > stdin. So I've got that 'sleep 60' hack in there currently; use C-g if
> >> > > you get bored of your plot.
> >> > >
> >> > > (ii) If the R-function isn't doing graphics, then the call to X11()
> >> > > gets in the way. X11() would only work on linux/mac OSX(?) anyway.
> >> > >
> >> > > (iii) I'm afraid I don't even know yet how to pass the optional string
> >> > > argument using M-x org-table-R-plot. Is it possible with some sort of
> >> > > prefix argument, and an (interactive something) declaration? Anyway,
> >> > > it seems to work if you evaluate e.g.
> >> > >
> >> > > (org-table-R-plot "(function(df) { plot(x=df[,1], y=df[,3]) ; abline(lm(df[,3] ~ df[,1]), col=\"blue\") })")
> >> > >
> >> > > Suggestions for improvements welcome!
> >> > >
> >> > > Dan
> >> > >
> >> > >
> >> > >
> >> > > On Fri, Jul 25, 2008 at 12:07:00PM -0700, Eric Schulte wrote:
> >> > > >
> >> > > > I had some time waiting for things to execute, so I condensed your
> >> > > > process into a single command (borrowing heavily from
> >> > > > org-export-table).
> >> > > >
> >> > > > (defun org-table/gnuplot (&optional x-col)
> >> > > > "Plot the current table using gnuplot. Use a prefix argument
> >> > > > to specify a column to use for the x-coordinates, to use the row
> >> > > > number for the x-coordinates provide a prefix argument of 0."
> >> > > > (interactive "p")
> >> > > > (message (format "%S" x-col))
> >> > > > (unless (org-at-table-p)
> >> > > > (error "No table at point"))
> >> > > > (require 'org-exp)
> >> > > > (require 'gnuplot)
> >> > > > (org-table-align) ;; make sure we have everything we need
> >> > > > (let* ((beg (org-table-begin))
> >> > > > (end (org-table-end))
> >> > > > (cols (save-excursion
> >> > > > (goto-char end)
> >> > > > (backward-char 3)
> >> > > > (org-table-current-column)))
> >> > > > (data-beg (if (and
> >> > > > (goto-char beg)
> >> > > > (re-search-forward org-table-dataline-regexp end t)
> >> > > > (re-search-forward org-table-hline-regexp end t)
> >> > > > (re-search-forward org-table-dataline-regexp end t))
> >> > > > (match-beginning 0)
> >> > > > beg))
> >> > > > (skip (- (line-number-at-pos data-beg) (line-number-at-pos beg)))
> >> > > > (exp-format (format "orgtbl-to-tsv :skip %d" skip))
> >> > > > (file (make-temp-file "org-table-plot")))
> >> > > > ;; export table
> >> > > > (org-table-export file exp-format)
> >> > > > (with-temp-buffer
> >> > > > ;; write script
> >> > > > (insert (org-table/gnuplot-script file x-col cols))
> >> > > > ;; graph table
> >> > > > (gnuplot-mode)
> >> > > > (gnuplot-send-buffer-to-gnuplot)
> >> > > > (bury-buffer (get-buffer "*gnuplot*")))
> >> > > > (delete-file file)))
> >> > > >
> >> > > > (defun org-table/gnuplot-script (file x-col num-cols)
> >> > > > (let ((plot-str "'%s' using %s:%d with lines title '%d'");; "\\\n ,"
> >> > > > script)
> >> > > > (dotimes (col (+ 1 num-cols))
> >> > > > (unless (or (and x-col (equal col x-col)) (equal col 0))
> >> > > > (setf script (cons (format plot-str file (or (and x-col (format "%d" x-col)) "") col col) script))))
> >> > > > (concat "plot " (mapconcat 'identity (reverse script) "\\\n ,"))))
> >> > > >
> >> > > > On Friday, July 25, at 17:25, James TD Smith wrote:
> >> > > > > On 2008-07-25 08:53:31(-0700), Eric Schulte wrote:
> >> > > > > >
> >> > > > > > Any advice for quick graphing of a table in org-mode?
> >> > > > > >
> >> > > > >
> >> > > > > I have a setup for plotting data from tables. I'm not sure if it's exactly what
> >> > > > > you want, but yoy may find it useful.
> >> > > > >
> >> > > > > 1. Add the following to your .emacs:
> >> > > > >
> >> > > > > (defun ahkt-plot-table (script)
> >> > > > > "util function to export and plot a table using the supplied
> >> > > > > gnuplot `script'"
> >> > > > > (org-table-export)
> >> > > > > (let ((cbuf (current-buffer))
> >> > > > > (cwin (selected-window)))
> >> > > > > (save-restriction
> >> > > > > (save-excursion
> >> > > > > (find-file script)
> >> > > > > (gnuplot-send-buffer-to-gnuplot)
> >> > > > > (bury-buffer)
> >> > > > > (bury-buffer (get-buffer "*gnuplot*"))))
> >> > > > > (and (window-live-p cwin) (select-window cwin))
> >> > > > > (switch-to-buffer cbuf)
> >> > > > > (delete-other-windows)))
> >> > > > >
> >> > > > > 2. Create a gnuplot script which plots data from a file.
> >> > > > >
> >> > > > > 3. Add the following properties to the headline containing the table.
> >> > > > > TABLE_EXPORT_FILE <filename in the gnuploy script>
> >> > > > > TABLE_EXPORT_FORMAT orgtbl-to-generic :skip 4 :splice t :sep "\t"
> >> > > > >
> >> > > > > 4. Add an org link in the table (it must be in the table otherwise the export
> >> > > > > doesn't work) as below:
> >> > > > > [[elisp:(ahkt-plot-table "<gnuplot script>")][plot table]]
> >> > > > >
> >> > > > > I suggest you put it at the top of the table.
> >> > > > > You will then need to adjust the 'skip' parameter in the export format depending
> >> > > > > on the number of lines at the top of the table which should not be exported
> >> > > > > (hlines, more than one plotting link etc).
> >> > > > >
> >> > > > > 5. You should then be able to open the link, and get a plot of the table
> >> > > > > contents.
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > |-<James TD Smith>-<email/ahktenzero@mohorovi.cc>-|
> >> > > > >
> >> > > > >
> >> > > > > _______________________________________________
> >> > > > > Emacs-orgmode mailing list
> >> > > > > Remember: use `Reply All' to send replies to the list.
> >> > > > > Emacs-orgmode@gnu.org
> >> > > > > http://lists.gnu.org/mailman/listinfo/emacs-orgmode
> >> > > >
> >> > > > --
> >> > > > schulte
> >> > > >
> >> > > >
> >> > > > _______________________________________________
> >> > > > Emacs-orgmode mailing list
> >> > > > Remember: use `Reply All' to send replies to the list.
> >> > > > Emacs-orgmode@gnu.org
> >> > > > http://lists.gnu.org/mailman/listinfo/emacs-orgmode
> >> >
> >> > --
> >> > schulte
> >>
> >>
> >> _______________________________________________
> >> Emacs-orgmode mailing list
> >> Remember: use `Reply All' to send replies to the list.
> >> Emacs-orgmode@gnu.org
> >> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
--
http://www.stats.ox.ac.uk/~davison
next prev parent reply other threads:[~2009-01-22 16:27 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-07-25 15:53 graphing from org-tables Eric Schulte
2008-07-25 16:23 ` Carsten Dominik
2008-07-25 16:25 ` James TD Smith
2008-07-25 16:33 ` Carsten Dominik
2008-07-25 17:14 ` Eric Schulte
2008-07-25 19:07 ` Eric Schulte
2008-07-26 18:15 ` Dan Davison
2008-07-28 15:32 ` Eric Schulte
2008-07-28 15:41 ` Dan Davison
2008-12-21 20:31 ` Dan Davison
2008-12-22 17:00 ` Eric Schulte
2009-01-22 16:27 ` Dan Davison [this message]
2009-01-23 1:37 ` Eric Schulte
2009-01-23 7:30 ` Carsten Dominik
2009-01-26 8:53 ` Carsten Dominik
2009-01-28 3:06 ` Dan Davison
2009-01-28 10:18 ` Carsten Dominik
[not found] ` <C8C13077-8B16-4E8C-8425-5782CE1EDC98@uva.nl>
[not found] ` <488b7c9b.14be600a.11cc.ffff9150@mx.google.com>
[not found] ` <9629760B-7606-42FD-B625-FAC44490221C@uva.nl>
2008-07-28 14:26 ` Eric Schulte
2008-07-28 21:00 ` Carsten Dominik
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090122162711.GA11250@stats.ox.ac.uk \
--to=davison@stats.ox.ac.uk \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.