From: "Eric Schulte" <schulte.eric@gmail.com>
To: Dan Davison <davison@stats.ox.ac.uk>
Cc: Org-mode <emacs-orgmode@gnu.org>
Subject: Re: graphing from org-tables
Date: Mon, 22 Dec 2008 12:00:14 -0500 [thread overview]
Message-ID: <87r640p2ld.fsf@gmail.com> (raw)
In-Reply-To: <20081221203153.GB14389@stats.ox.ac.uk> (Dan Davison's message of "Sun, 21 Dec 2008 15:31:53 -0500")
Hi Dan,
This looks very interesting, and I look forward to playing around with
these functions (when I have some/any free time).
After a quick glance it seems like these functions could be expanded in
two different directions...
1) Adapted to a simple calling mechanism like the one used for org-plot
which relies solely on a couple of parameters stuffed into a #+PLOT
line next to a table. I think this would be a good option for org
users who are more comfortable w/R than w/gnuplot.
2) Also interesting is the idea of referencing a table from a block of R
code elsewhere in the org file. I've worked some on processing
blocks of R code in org files in a manner similar to Sweave.
http://github.com/eschulte/org-contrib/tree/master/org-exp-blocks.el
If we could implement a simple means of referencing tables (either
whole tables, or cells/cols/rows of tables) from these R-code blocks
then that seems like it may be fairly powerful.
Thanks -- Eric
Dan Davison <davison@stats.ox.ac.uk> writes:
> On Mon, Jul 28, 2008 at 04:41:52PM +0100, Dan Davison wrote:
>> On Mon, Jul 28, 2008 at 08:32:22AM -0700, Eric Schulte wrote:
>> > Hi Dan,
>> >
>> > One way around the sleep(60) hack may be to create the R graph using
>> > an inferior R process. See
>>
>> Yep, I just this minute asked for help on the Emacs Speaks Statistics
>> (ESS) mailing list! I'll report back if I make progress with that.
>>
>> https://stat.ethz.ch/pipermail/ess-help/2008-July/004785.html
>
> I think it's worth keeping this avenue alive, despite so much progress
> having been made with gnuplot in the interim. So below is an initial
> version of org-table-eval-R, which takes an arbitrary R function, and
> applies it to an org table. As suggested above, this uses
> inferior-ess-mode and therefore requires ess (emacs speaks statistics)
> and R to be installed.
>
> At it's simplest, R-function can be the name of a standard R
> function. So with Eric's first example data set on Worg
> (http://legito.net/worg/org-tutorials/org-plot.php) i.e. with 2
> dependent variables
>
> (org-table-eval-R "summary") produces (in a new buffer)
>
> first.dependent.var second.dependent.var
> Min. :0.1560 Min. :0.1808
> 1st Qu.:0.1773 1st Qu.:0.1985
> Median :0.1900 Median :0.2146
> Mean :0.2110 Mean :0.2262
> 3rd Qu.:0.2333 3rd Qu.:0.2375
> Max. :0.4250 Max. :0.3750
>
> and (org-table-eval-R "matplot") produces a rough version of Eric's
> gnuplot figure. (see below for a closer reproduction of the figure,
> with the legend etc).
>
> It would be simple to provide org-plot/R which would construct a
> suitable plotting function and pass it to org-table-eval-R, and some
> ready-made R code could be provided for standard plots -- histograms &
> barplots, scatter plots, etc. I've given an initial version of
> org-plot/R below, which gives the option to produce eps output rather
> than a pop-up window (and it would be easy to use any of the R
> graphics devices for output: jpeg, png, ps, pdf, tiff, X11, quartz,
> etc)
>
> But the power of this approach is that it provides not just plotting
> functionality, but the ability to do arbitrary calculations and
> statistical analyses of the org table, via the base R packages and the
> hundreds of user-written packages.
>
> It would have been better to have developed these ideas along with the
> development of org-plot, but unfortunately I let that slip. But
> org-table-eval-R provides a lot of power for anyone that's prepared to
> write some R code. And if there's interest, then some more
> user-friendly functionality could be provided for org users that want
> something easier, but can install ess and R.
>
> Dan
>
> and here's a rough version of the "O R G" grid plot
>
> (org-table-eval-R "(function (x) image(t(as.matrix(x))))" t)
>
> Of course, these can be beautified to your heart's content with lots
> of extra arguments to the R plotting functions. This comes a bit
> closer to the two dependent-variables graph:
>
> (org-table-eval-R "(function(x) { matplot(x, type=\"l\", ylab=\"\") ; legend(\"topright\", legend=c(\"dep var 1\"\,\"dep var 2\"), col=1:2, lty=1:2) })")
>
>
>
> --------------------------------------------------------------
> (defun org-table-eval-R (R-function &optional rownames-absent)
> "Apply an arbitrary R function to the org table at
> point. R-FUNCTION is an R function (a string) taking one
> argument, which is the R data frame representation of the org
> table. It may be the name of an appropriate R
> function (e.g. \"summary\", \"plot\"), or a user-defined
> anonymous function of the form \"(function(data.frame)
> {...})\". Unless ROWNAMES-ABSENT is non-nil the row names of the
> data frame are taken from the first column of the org table"
> (interactive)
> (require 'ess)
> (let ((csv-file (make-temp-file "org-table-eval-R-")))
> (org-table-export csv-file "orgtbl-to-csv")
> (ess-execute (org-table-eval-R-make-expr
> R-function csv-file rownames-absent))
> (delete-file csv-file)))
>
> (defun org-table-eval-R-make-expr (R-function csv-file rownames-absent)
> (concat R-function
> "(read.csv(\"" csv-file "\""
> (unless rownames-absent ", row.names=1") "))"))
>
>
> (defun org-plot/R (&optional plot-function file rownames-absent)
> "Use R to create a plot using the org table at point. If FILE
> is non-nil then the plot will be written to file (currently as
> eps, in future the format could easily be determined by the file
> extension.) Unless ROWNAMES-ABSENT is non-nil the row names will
> be taken from the first column of the org table."
> (interactive)
> (require 'ess)
> (org-table-eval-R (or plot-function "plot") rownames-absent)
> (if file (ess-execute (concat "dev.copy2eps(\"" file "\")"))))
>
>
>
>
>>
>> Dan
>>
>>
>>
>>
>> > http://cran.r-project.org/doc/FAQ/R-FAQ.html#R-and-Emacs for
>> > information on running R functions from inside of Emacs.
>> >
>> > Best -- Eric
>> >
>> > On Saturday, July 26, at 19:15, Dan Davison wrote:
>> > > R (www.r-project.org) is pretty good for data plotting and statistical
>> > > analyses. Here's my effort at the org-table-plot function, using
>> > > R. Since R contains a csv importer that can read from stdin, it's
>> > > pretty simple. I've tried to code it so that you can provide an
>> > > arbitrary R function as the optional argument, so in principle you can
>> > > do to your org-table anything that R is capable of in the realms of
>> > > data analysis and visualisation.
>> > >
>> > > (defun org-table-R-plot (&optional R-function)
>> > > "Plot the current table using R. The table is transformed into a dataframe in R. Optional
>> > > argument R-function is a string which is either the name of an R
>> > > function, or an anonymous function definition of the form (function(d) {...}),
>> > > requiring a single argument (the dataframe). The default is to use
>> > > the R function 'plot' which produces scatter plots of all pairwise
>> > > combinations of columns. An example custom plotting function is:
>> > > * plot column 3 against column 1, adding least-squares linear regression fit in blue (function(df) { plot(x=df[,1], y=df[,3]) ; abline(lm(df[,3] ~ df[,1]), col=\"blue\") }) "
>> > > (interactive)
>> > > (unless R-function (set 'R-function "plot"))
>> > > (let ((file (make-temp-file "org-table-R-plot")))
>> > > (org-table-export file "orgtbl-to-csv")
>> > > (set-buffer (find-file-noselect file))
>> > > (shell-command-on-region
>> > > (point-min) (point-max)
>> > > (concat "Rscript -e 'X11() ; " R-function "(read.csv(\"stdin\")) ; system(\"sleep 60\")'"))
>> > > (delete-file file)))
>> > >
>> > >
>> > > R is at www.r-project.org
>> > > (package r-base on ubuntu/debian)
>> > >
>> > > Rscript is a command line non-interactive scripting utility that is
>> > > bundled automatically with the R installation. I reckon it'll be OK on
>> > > OSX but no idea about Windows. My function doesn't have to be used
>> > > for plotting; the R-function argument can be any function operating on
>> > > the data from the org-table, producing numerical or graphical output.
>> > >
>> > > There's several things that need to be sorted out with my function, e.g.
>> > >
>> > > (i) I haven't worked out how to return control to the emacs process
>> > > while keeping the plot window there. I tried adding an & to the
>> > > shell-command, but that seemed to result in R receiving nothing on
>> > > stdin. So I've got that 'sleep 60' hack in there currently; use C-g if
>> > > you get bored of your plot.
>> > >
>> > > (ii) If the R-function isn't doing graphics, then the call to X11()
>> > > gets in the way. X11() would only work on linux/mac OSX(?) anyway.
>> > >
>> > > (iii) I'm afraid I don't even know yet how to pass the optional string
>> > > argument using M-x org-table-R-plot. Is it possible with some sort of
>> > > prefix argument, and an (interactive something) declaration? Anyway,
>> > > it seems to work if you evaluate e.g.
>> > >
>> > > (org-table-R-plot "(function(df) { plot(x=df[,1], y=df[,3]) ; abline(lm(df[,3] ~ df[,1]), col=\"blue\") })")
>> > >
>> > > Suggestions for improvements welcome!
>> > >
>> > > Dan
>> > >
>> > >
>> > >
>> > > On Fri, Jul 25, 2008 at 12:07:00PM -0700, Eric Schulte wrote:
>> > > >
>> > > > I had some time waiting for things to execute, so I condensed your
>> > > > process into a single command (borrowing heavily from
>> > > > org-export-table).
>> > > >
>> > > > (defun org-table/gnuplot (&optional x-col)
>> > > > "Plot the current table using gnuplot. Use a prefix argument
>> > > > to specify a column to use for the x-coordinates, to use the row
>> > > > number for the x-coordinates provide a prefix argument of 0."
>> > > > (interactive "p")
>> > > > (message (format "%S" x-col))
>> > > > (unless (org-at-table-p)
>> > > > (error "No table at point"))
>> > > > (require 'org-exp)
>> > > > (require 'gnuplot)
>> > > > (org-table-align) ;; make sure we have everything we need
>> > > > (let* ((beg (org-table-begin))
>> > > > (end (org-table-end))
>> > > > (cols (save-excursion
>> > > > (goto-char end)
>> > > > (backward-char 3)
>> > > > (org-table-current-column)))
>> > > > (data-beg (if (and
>> > > > (goto-char beg)
>> > > > (re-search-forward org-table-dataline-regexp end t)
>> > > > (re-search-forward org-table-hline-regexp end t)
>> > > > (re-search-forward org-table-dataline-regexp end t))
>> > > > (match-beginning 0)
>> > > > beg))
>> > > > (skip (- (line-number-at-pos data-beg) (line-number-at-pos beg)))
>> > > > (exp-format (format "orgtbl-to-tsv :skip %d" skip))
>> > > > (file (make-temp-file "org-table-plot")))
>> > > > ;; export table
>> > > > (org-table-export file exp-format)
>> > > > (with-temp-buffer
>> > > > ;; write script
>> > > > (insert (org-table/gnuplot-script file x-col cols))
>> > > > ;; graph table
>> > > > (gnuplot-mode)
>> > > > (gnuplot-send-buffer-to-gnuplot)
>> > > > (bury-buffer (get-buffer "*gnuplot*")))
>> > > > (delete-file file)))
>> > > >
>> > > > (defun org-table/gnuplot-script (file x-col num-cols)
>> > > > (let ((plot-str "'%s' using %s:%d with lines title '%d'");; "\\\n ,"
>> > > > script)
>> > > > (dotimes (col (+ 1 num-cols))
>> > > > (unless (or (and x-col (equal col x-col)) (equal col 0))
>> > > > (setf script (cons (format plot-str file (or (and x-col (format "%d" x-col)) "") col col) script))))
>> > > > (concat "plot " (mapconcat 'identity (reverse script) "\\\n ,"))))
>> > > >
>> > > > On Friday, July 25, at 17:25, James TD Smith wrote:
>> > > > > On 2008-07-25 08:53:31(-0700), Eric Schulte wrote:
>> > > > > >
>> > > > > > Any advice for quick graphing of a table in org-mode?
>> > > > > >
>> > > > >
>> > > > > I have a setup for plotting data from tables. I'm not sure if it's exactly what
>> > > > > you want, but yoy may find it useful.
>> > > > >
>> > > > > 1. Add the following to your .emacs:
>> > > > >
>> > > > > (defun ahkt-plot-table (script)
>> > > > > "util function to export and plot a table using the supplied
>> > > > > gnuplot `script'"
>> > > > > (org-table-export)
>> > > > > (let ((cbuf (current-buffer))
>> > > > > (cwin (selected-window)))
>> > > > > (save-restriction
>> > > > > (save-excursion
>> > > > > (find-file script)
>> > > > > (gnuplot-send-buffer-to-gnuplot)
>> > > > > (bury-buffer)
>> > > > > (bury-buffer (get-buffer "*gnuplot*"))))
>> > > > > (and (window-live-p cwin) (select-window cwin))
>> > > > > (switch-to-buffer cbuf)
>> > > > > (delete-other-windows)))
>> > > > >
>> > > > > 2. Create a gnuplot script which plots data from a file.
>> > > > >
>> > > > > 3. Add the following properties to the headline containing the table.
>> > > > > TABLE_EXPORT_FILE <filename in the gnuploy script>
>> > > > > TABLE_EXPORT_FORMAT orgtbl-to-generic :skip 4 :splice t :sep "\t"
>> > > > >
>> > > > > 4. Add an org link in the table (it must be in the table otherwise the export
>> > > > > doesn't work) as below:
>> > > > > [[elisp:(ahkt-plot-table "<gnuplot script>")][plot table]]
>> > > > >
>> > > > > I suggest you put it at the top of the table.
>> > > > > You will then need to adjust the 'skip' parameter in the export format depending
>> > > > > on the number of lines at the top of the table which should not be exported
>> > > > > (hlines, more than one plotting link etc).
>> > > > >
>> > > > > 5. You should then be able to open the link, and get a plot of the table
>> > > > > contents.
>> > > > >
>> > > > >
>> > > > > --
>> > > > > |-<James TD Smith>-<email/ahktenzero@mohorovi.cc>-|
>> > > > >
>> > > > >
>> > > > > _______________________________________________
>> > > > > Emacs-orgmode mailing list
>> > > > > Remember: use `Reply All' to send replies to the list.
>> > > > > Emacs-orgmode@gnu.org
>> > > > > http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>> > > >
>> > > > --
>> > > > schulte
>> > > >
>> > > >
>> > > > _______________________________________________
>> > > > Emacs-orgmode mailing list
>> > > > Remember: use `Reply All' to send replies to the list.
>> > > > Emacs-orgmode@gnu.org
>> > > > http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>> >
>> > --
>> > schulte
>>
>>
>> _______________________________________________
>> Emacs-orgmode mailing list
>> Remember: use `Reply All' to send replies to the list.
>> Emacs-orgmode@gnu.org
>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
next prev parent reply other threads:[~2008-12-22 17:01 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-07-25 15:53 graphing from org-tables Eric Schulte
2008-07-25 16:23 ` Carsten Dominik
2008-07-25 16:25 ` James TD Smith
2008-07-25 16:33 ` Carsten Dominik
2008-07-25 17:14 ` Eric Schulte
2008-07-25 19:07 ` Eric Schulte
2008-07-26 18:15 ` Dan Davison
2008-07-28 15:32 ` Eric Schulte
2008-07-28 15:41 ` Dan Davison
2008-12-21 20:31 ` Dan Davison
2008-12-22 17:00 ` Eric Schulte [this message]
2009-01-22 16:27 ` Dan Davison
2009-01-23 1:37 ` Eric Schulte
2009-01-23 7:30 ` Carsten Dominik
2009-01-26 8:53 ` Carsten Dominik
2009-01-28 3:06 ` Dan Davison
2009-01-28 10:18 ` Carsten Dominik
[not found] ` <C8C13077-8B16-4E8C-8425-5782CE1EDC98@uva.nl>
[not found] ` <488b7c9b.14be600a.11cc.ffff9150@mx.google.com>
[not found] ` <9629760B-7606-42FD-B625-FAC44490221C@uva.nl>
2008-07-28 14:26 ` Eric Schulte
2008-07-28 21:00 ` Carsten Dominik
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87r640p2ld.fsf@gmail.com \
--to=schulte.eric@gmail.com \
--cc=davison@stats.ox.ac.uk \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.