* Re: Table width cookies become part of data table in R
2024-11-15 4:29 Table width cookies become part of data table in R William Denton
2024-11-15 11:59 ` Rens Oliemans
@ 2024-12-14 17:21 ` Ihor Radchenko
1 sibling, 0 replies; 4+ messages in thread
From: Ihor Radchenko @ 2024-12-14 17:21 UTC (permalink / raw)
To: William Denton; +Cc: Emacs Org mode mailing list
[-- Attachment #1: Type: text/plain, Size: 2571 bytes --]
William Denton <william@williamdenton.org> writes:
> Up until recently, width cookies were ignored when reading a table into an R code block as data. For example, take this test table:
>
> #+name: test_table
> | Date | Weather |
> | <10> | <50> |
> |------------+----------------------|
> | 2024-11-01 | Warm |
> | 2024-11-02 | Warm |
> | 2024-11-03 | Still strangely warm |
>
> This code block turns it into a basic data table:
>
> #+begin_src R :var t=test_table :colnames yes
> t
> #+end_src
>
> #+RESULTS:
> | Date | Weather |
> |------------+----------------------|
> | <10> | <50> |
> | 2024-11-01 | Warm |
> | 2024-11-02 | Warm |
> | 2024-11-03 | Still strangely warm |
>
> That first row, with "<10>", shouldn't be there. It wasn't until
> recently, but I can't see anything in ORG-NEWS or git commits about
> why it might have started. (I'm running from the development tree,
> updated just now.)
>
> Is this a bug or am I missing something?
It is indeed a breaking change.
Among various ob-* backends, we do not have consistency in handling
table width specifiers and other metadata.
For example, ob-python takes the table literally when assigning
variables. ob-R, in the past, removed the width specifications. Not
anymore.
I believe that the end result is more consistency, but, clearly, we got
a regression. Not just in ob-R, but in a number of other babel backends.
I can force the previous behavior in ob-R and similar backends, but I am
not sure if it is the right thing to do.
Pros:
- We will retain historic behavior
Cons:
- We will retain historic behavior with different backends behaving
differently
Here is a list of backends that filtered out special rows from tables in
variable assignments: ob-julia, ob-R, ob-sql, ob-shell, ob-awk, ob-gnuplot&org-plot
6 in total.
Here is a list of backends that did not filter out special rows from tables:
ob-ruby, ob-python, ob-processing, ob-perl, ob-octave, ob-ocaml, ob-lua,
ob-js, ob-java, ob-haskell, ob-eshell
11 in total.
I am not really sure about the best approach here.
Tentatively, I am leaning towards restoring the old behavior for the
time being, until we can come up with something better.
Patch attached.
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-Restore-historic-removal-of-special-rows-in-table.patch --]
[-- Type: text/x-patch, Size: 6225 bytes --]
From 36ddf840db2b6eae3bb25efb228117a8df79a0b8 Mon Sep 17 00:00:00 2001
Message-ID: <36ddf840db2b6eae3bb25efb228117a8df79a0b8.1734196827.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Sat, 14 Dec 2024 18:17:51 +0100
Subject: [PATCH] ob-*: Restore historic removal of special rows in table
assignments
* lisp/ob-R.el (org-babel-R-assign-elisp):
* lisp/ob-awk.el (org-babel-awk-var-to-awk):
* lisp/ob-gnuplot.el (org-babel-gnuplot-table-to-data):
* lisp/ob-julia.el (org-babel-julia-assign-elisp):
* lisp/ob-shell.el (org-babel-sh-var-to-string):
* lisp/ob-sql.el (org-babel-sql-expand-vars):
* lisp/org-plot.el (org-plot/gnuplot-to-data): Restore historic
behavior when special rows are removed before assigning table data to
variables. Note that it is (historically) inconsistent with ob-ruby,
ob-python, ob-processing, ob-perl, ob-octave, ob-ocaml, ob-lua, ob-js,
ob-java, ob-haskell, and ob-eshell where special rows are retained.
Reported-by: William Denton <william@williamdenton.org>
Link: https://orgmode.org/list/_XqzGYdYTbgEql7bdh5hazhuc9DpC5k6pPjWT126KEL_CE5NY-SlkHua-bbPDUsJ_-e4_oopiG7qvGJzCH2iyTZDDkyvtbM-JPqCONyCc8k=@williamdenton.org
---
lisp/ob-R.el | 5 ++++-
lisp/ob-awk.el | 6 +++++-
lisp/ob-gnuplot.el | 1 +
lisp/ob-julia.el | 5 ++++-
lisp/ob-shell.el | 7 +++++--
lisp/ob-sql.el | 8 +++++---
lisp/org-plot.el | 3 ++-
7 files changed, 26 insertions(+), 9 deletions(-)
diff --git a/lisp/ob-R.el b/lisp/ob-R.el
index 1dbbd83f2c..c22f2aa2a3 100644
--- a/lisp/ob-R.el
+++ b/lisp/ob-R.el
@@ -251,7 +251,10 @@ (defun org-babel-R-assign-elisp (name value colnames-p rownames-p)
(min (if lengths (apply 'min lengths) 0)))
;; Ensure VALUE has an orgtbl structure (depth of at least 2).
(unless (listp (car value)) (setq value (mapcar 'list value)))
- (let ((file (orgtbl-to-tsv value '(:fmt org-babel-R-quote-tsv-field)))
+ (let ((file (orgtbl-to-tsv
+ value
+ '( :fmt org-babel-R-quote-tsv-field
+ :with-special-rows nil)))
(header (if (or (eq (nth 1 value) 'hline) colnames-p)
"TRUE" "FALSE"))
(row-names (if rownames-p "1" "NULL")))
diff --git a/lisp/ob-awk.el b/lisp/ob-awk.el
index 0c07aec829..16fbd63f5c 100644
--- a/lisp/ob-awk.el
+++ b/lisp/ob-awk.el
@@ -111,7 +111,11 @@ (defun org-babel-awk-var-to-awk (var &optional sep)
(let ((echo-var (lambda (v) (if (stringp v) v (format "%S" v)))))
(cond
((and (listp var) (listp (car var)))
- (orgtbl-to-generic var (list :sep (or sep "\t") :fmt echo-var)))
+ (orgtbl-to-generic
+ var
+ (list :sep (or sep "\t")
+ :fmt echo-var
+ :with-special-rows nil)))
((listp var)
(mapconcat echo-var var "\n"))
(t (funcall echo-var var)))))
diff --git a/lisp/ob-gnuplot.el b/lisp/ob-gnuplot.el
index 2c53f30664..b7598adf0b 100644
--- a/lisp/ob-gnuplot.el
+++ b/lisp/ob-gnuplot.el
@@ -317,6 +317,7 @@ (defun org-babel-gnuplot-table-to-data (table data-file params)
`( :sep "\t" :fmt org-babel-gnuplot-quote-tsv-field
;; Two setting below are needed to make :fmt work.
:raw t
+ :with-special-rows nil
:backend ,ob-gnuplot-data)
params)))))
data-file)
diff --git a/lisp/ob-julia.el b/lisp/ob-julia.el
index f9da734b37..9ed0ff80a6 100644
--- a/lisp/ob-julia.el
+++ b/lisp/ob-julia.el
@@ -171,7 +171,10 @@ (defun org-babel-julia-assign-elisp (name value)
(min (if lengths (apply #'min lengths) 0)))
;; Ensure VALUE has an orgtbl structure (depth of at least 2).
(unless (listp (car value)) (setq value (list value)))
- (let ((file (orgtbl-to-csv value '(:fmt org-babel-julia-quote-csv-field))))
+ (let ((file (orgtbl-to-csv
+ value
+ '( :fmt org-babel-julia-quote-csv-field
+ :with-special-rows nil))))
(if (= max min)
(format "%s = begin
using CSV
diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
index 7706234919..535729d2bb 100644
--- a/lisp/ob-shell.el
+++ b/lisp/ob-shell.el
@@ -263,8 +263,11 @@ (defun org-babel-sh-var-to-string (var &optional sep hline)
(let ((echo-var (lambda (v) (if (stringp v) v (format "%S" v)))))
(cond
((and (listp var) (or (listp (car var)) (eq (car var) 'hline)))
- (orgtbl-to-generic var (list :sep (or sep "\t") :fmt echo-var
- :hline hline)))
+ (orgtbl-to-generic
+ var
+ (list :sep (or sep "\t") :fmt echo-var
+ :hline hline
+ :with-special-rows nil)))
((listp var)
(mapconcat echo-var var "\n"))
(t (funcall echo-var var)))))
diff --git a/lisp/ob-sql.el b/lisp/ob-sql.el
index d7bcaa0973..8795934520 100644
--- a/lisp/ob-sql.el
+++ b/lisp/ob-sql.el
@@ -418,9 +418,11 @@ (defun org-babel-sql-expand-vars (body vars &optional sqlite)
(insert (orgtbl-to-csv
val (if sqlite
nil
- '(:fmt (lambda (el) (if (stringp el)
- el
- (format "%S" el))))))))
+ '( :fmt (lambda (el)
+ (if (stringp el)
+ el
+ (format "%S" el)))
+ :with-special-rows nil)))))
data-file)
(if (stringp val) val (format "%S" val))))
body t t)))
diff --git a/lisp/org-plot.el b/lisp/org-plot.el
index 836cfaffca..7f21a4cd55 100644
--- a/lisp/org-plot.el
+++ b/lisp/org-plot.el
@@ -140,7 +140,8 @@ (defun org-plot/gnuplot-to-data (table data-file params)
(insert (orgtbl-to-generic
table
(org-combine-plists
- '(:sep "\t" :fmt org-plot-quote-tsv-field)
+ '( :sep "\t" :fmt org-plot-quote-tsv-field
+ :with-special-rows nil)
params)))))
nil)
--
2.47.1
^ permalink raw reply related [flat|nested] 4+ messages in thread