From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id UFDNI4a+XWcBzwAA62LTzQ:P1 (envelope-from ) for ; Sat, 14 Dec 2024 17:21:10 +0000 Received: from aspmx1.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id UFDNI4a+XWcBzwAA62LTzQ (envelope-from ) for ; Sat, 14 Dec 2024 18:21:10 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=kM41CAWx; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1734196870; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=/DG07/cNQekqzbS6gef6TVMEyK2UNSGCuTzUNFIqwW4=; b=LsCOGBRImOQ1WupuJH6D85MHFSxuZ/j4yJuY91/HSDan3huPfu1q/SyqdszGBh5jxATcyp WInGx/QZ9HEm3iyLHY1PDwLXcU28LBoMiiWQ4/nHbskqoAEz5QDTwniTFbpPyZaRSNggCe umyFPDyTi9iSuJvq74rJAYBpGdrbOPZ/CQhMBFdsN38QMK5GJAs3/Uyqgmmg19vLwnnBxC nRkdfYrqHB9ZxV+yXtbqTyxqsRtHw7r3LplgFnbXY7mS7X2tcGJbqzL4keU1lztvk10+jX TaJvHGlP5mDWh6t2gMF/hAvIOXFQVWDp4fyJkRSbyrDwbO5R3BmqEcG8+He1ng== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=kM41CAWx; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Seal: i=1; s=key1; d=yhetil.org; t=1734196870; a=rsa-sha256; cv=none; b=f2TGTf/hK/UNbHwdMl+Zcr20XZAcwtI8DsFFfRZnP6ApsdqHGi4jiwSiySsv+WvEI+Jbwm BO1yljed2hwFfqkr3qFIVaF/jjVQgndhx3niDDv6atb+qqciZ5baeve79kM/Yq1JfCez+4 TOboc+qrf0/Fa/fTtGZ+3UKMrTw/roHldlM57uuER4AW65tdGV2tQoFSNr2h/RKwY34Dt9 LZYV8S1DEk1hluIQlVNVXRYGnUHcXh1vUo63BbPvfol/1EhexO7X9cuKAC26hy+JKUdMyW ps4LYYzHSDLwdhUoXLrsxVmfD3X10Qarlg8LnY9nwBmjho9o2/L3TU0Chlr+lQ== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 6E32A7C574 for ; Sat, 14 Dec 2024 18:21:10 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tMVoN-0006Nu-IV; Sat, 14 Dec 2024 12:20:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tMVoL-0006Nb-5U for emacs-orgmode@gnu.org; Sat, 14 Dec 2024 12:20:13 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tMVoI-0002O8-Eb for emacs-orgmode@gnu.org; Sat, 14 Dec 2024 12:20:12 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 5E3EE240103 for ; Sat, 14 Dec 2024 18:20:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1734196807; bh=xpV/FJbZ303t7ZgxSSy5JT+KLoGqVOy4NRtKjmpqha0=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: From; b=kM41CAWxA0G8KJGHZBLMdE7yqRi6daGXFCXkRzs0EO+pIuQb9L5NOnnHZiHL1hYLQ mjwpgXOWf5QqaVZX0gxIzA/SEdpiqxt3J9c9P51W6b/Qs+/AflsvfYnQPV+0gtQQpe zDVxmD/w+utV9VLgXf/lcISb4UvD9nHPivD3dh43oI3eKgi2oXTXJj1/lxmudkeps0 ygBqboGXTpQk6PEPxPGbDcAwJWRaNtH0vcyv546D0t+vLqAOEz7YAnvw+E93o+qKME yRIx8t9NoAvtqp0Pg3QfQf17I2btNZVCqsWKr/duBVYQPJYfvrEIpv7JetfnUUJXfr vPICE/rfTEU4A== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Y9XyZ3yKpz6txf; Sat, 14 Dec 2024 18:20:06 +0100 (CET) From: Ihor Radchenko To: William Denton Cc: Emacs Org mode mailing list Subject: Re: Table width cookies become part of data table in R In-Reply-To: <_XqzGYdYTbgEql7bdh5hazhuc9DpC5k6pPjWT126KEL_CE5NY-SlkHua-bbPDUsJ_-e4_oopiG7qvGJzCH2iyTZDDkyvtbM-JPqCONyCc8k=@williamdenton.org> References: <_XqzGYdYTbgEql7bdh5hazhuc9DpC5k6pPjWT126KEL_CE5NY-SlkHua-bbPDUsJ_-e4_oopiG7qvGJzCH2iyTZDDkyvtbM-JPqCONyCc8k=@williamdenton.org> Date: Sat, 14 Dec 2024 17:21:35 +0000 Message-ID: <878qsi2na8.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.66; envelope-from=yantar92@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Queue-Id: 6E32A7C574 X-Migadu-Scanner: mx13.migadu.com X-Migadu-Spam-Score: -8.16 X-Spam-Score: -8.16 X-TUID: YW5XE9NpzHma --=-=-= Content-Type: text/plain William Denton writes: > Up until recently, width cookies were ignored when reading a table into an R code block as data. For example, take this test table: > > #+name: test_table > | Date | Weather | > | <10> | <50> | > |------------+----------------------| > | 2024-11-01 | Warm | > | 2024-11-02 | Warm | > | 2024-11-03 | Still strangely warm | > > This code block turns it into a basic data table: > > #+begin_src R :var t=test_table :colnames yes > t > #+end_src > > #+RESULTS: > | Date | Weather | > |------------+----------------------| > | <10> | <50> | > | 2024-11-01 | Warm | > | 2024-11-02 | Warm | > | 2024-11-03 | Still strangely warm | > > That first row, with "<10>", shouldn't be there. It wasn't until > recently, but I can't see anything in ORG-NEWS or git commits about > why it might have started. (I'm running from the development tree, > updated just now.) > > Is this a bug or am I missing something? It is indeed a breaking change. Among various ob-* backends, we do not have consistency in handling table width specifiers and other metadata. For example, ob-python takes the table literally when assigning variables. ob-R, in the past, removed the width specifications. Not anymore. I believe that the end result is more consistency, but, clearly, we got a regression. Not just in ob-R, but in a number of other babel backends. I can force the previous behavior in ob-R and similar backends, but I am not sure if it is the right thing to do. Pros: - We will retain historic behavior Cons: - We will retain historic behavior with different backends behaving differently Here is a list of backends that filtered out special rows from tables in variable assignments: ob-julia, ob-R, ob-sql, ob-shell, ob-awk, ob-gnuplot&org-plot 6 in total. Here is a list of backends that did not filter out special rows from tables: ob-ruby, ob-python, ob-processing, ob-perl, ob-octave, ob-ocaml, ob-lua, ob-js, ob-java, ob-haskell, ob-eshell 11 in total. I am not really sure about the best approach here. Tentatively, I am leaning towards restoring the old behavior for the time being, until we can come up with something better. Patch attached. -- Ihor Radchenko // yantar92, Org mode maintainer, Learn more about Org mode at . Support Org development at , or support my work at --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-ob-Restore-historic-removal-of-special-rows-in-table.patch >From 36ddf840db2b6eae3bb25efb228117a8df79a0b8 Mon Sep 17 00:00:00 2001 Message-ID: <36ddf840db2b6eae3bb25efb228117a8df79a0b8.1734196827.git.yantar92@posteo.net> From: Ihor Radchenko Date: Sat, 14 Dec 2024 18:17:51 +0100 Subject: [PATCH] ob-*: Restore historic removal of special rows in table assignments * lisp/ob-R.el (org-babel-R-assign-elisp): * lisp/ob-awk.el (org-babel-awk-var-to-awk): * lisp/ob-gnuplot.el (org-babel-gnuplot-table-to-data): * lisp/ob-julia.el (org-babel-julia-assign-elisp): * lisp/ob-shell.el (org-babel-sh-var-to-string): * lisp/ob-sql.el (org-babel-sql-expand-vars): * lisp/org-plot.el (org-plot/gnuplot-to-data): Restore historic behavior when special rows are removed before assigning table data to variables. Note that it is (historically) inconsistent with ob-ruby, ob-python, ob-processing, ob-perl, ob-octave, ob-ocaml, ob-lua, ob-js, ob-java, ob-haskell, and ob-eshell where special rows are retained. Reported-by: William Denton Link: https://orgmode.org/list/_XqzGYdYTbgEql7bdh5hazhuc9DpC5k6pPjWT126KEL_CE5NY-SlkHua-bbPDUsJ_-e4_oopiG7qvGJzCH2iyTZDDkyvtbM-JPqCONyCc8k=@williamdenton.org --- lisp/ob-R.el | 5 ++++- lisp/ob-awk.el | 6 +++++- lisp/ob-gnuplot.el | 1 + lisp/ob-julia.el | 5 ++++- lisp/ob-shell.el | 7 +++++-- lisp/ob-sql.el | 8 +++++--- lisp/org-plot.el | 3 ++- 7 files changed, 26 insertions(+), 9 deletions(-) diff --git a/lisp/ob-R.el b/lisp/ob-R.el index 1dbbd83f2c..c22f2aa2a3 100644 --- a/lisp/ob-R.el +++ b/lisp/ob-R.el @@ -251,7 +251,10 @@ (defun org-babel-R-assign-elisp (name value colnames-p rownames-p) (min (if lengths (apply 'min lengths) 0))) ;; Ensure VALUE has an orgtbl structure (depth of at least 2). (unless (listp (car value)) (setq value (mapcar 'list value))) - (let ((file (orgtbl-to-tsv value '(:fmt org-babel-R-quote-tsv-field))) + (let ((file (orgtbl-to-tsv + value + '( :fmt org-babel-R-quote-tsv-field + :with-special-rows nil))) (header (if (or (eq (nth 1 value) 'hline) colnames-p) "TRUE" "FALSE")) (row-names (if rownames-p "1" "NULL"))) diff --git a/lisp/ob-awk.el b/lisp/ob-awk.el index 0c07aec829..16fbd63f5c 100644 --- a/lisp/ob-awk.el +++ b/lisp/ob-awk.el @@ -111,7 +111,11 @@ (defun org-babel-awk-var-to-awk (var &optional sep) (let ((echo-var (lambda (v) (if (stringp v) v (format "%S" v))))) (cond ((and (listp var) (listp (car var))) - (orgtbl-to-generic var (list :sep (or sep "\t") :fmt echo-var))) + (orgtbl-to-generic + var + (list :sep (or sep "\t") + :fmt echo-var + :with-special-rows nil))) ((listp var) (mapconcat echo-var var "\n")) (t (funcall echo-var var))))) diff --git a/lisp/ob-gnuplot.el b/lisp/ob-gnuplot.el index 2c53f30664..b7598adf0b 100644 --- a/lisp/ob-gnuplot.el +++ b/lisp/ob-gnuplot.el @@ -317,6 +317,7 @@ (defun org-babel-gnuplot-table-to-data (table data-file params) `( :sep "\t" :fmt org-babel-gnuplot-quote-tsv-field ;; Two setting below are needed to make :fmt work. :raw t + :with-special-rows nil :backend ,ob-gnuplot-data) params))))) data-file) diff --git a/lisp/ob-julia.el b/lisp/ob-julia.el index f9da734b37..9ed0ff80a6 100644 --- a/lisp/ob-julia.el +++ b/lisp/ob-julia.el @@ -171,7 +171,10 @@ (defun org-babel-julia-assign-elisp (name value) (min (if lengths (apply #'min lengths) 0))) ;; Ensure VALUE has an orgtbl structure (depth of at least 2). (unless (listp (car value)) (setq value (list value))) - (let ((file (orgtbl-to-csv value '(:fmt org-babel-julia-quote-csv-field)))) + (let ((file (orgtbl-to-csv + value + '( :fmt org-babel-julia-quote-csv-field + :with-special-rows nil)))) (if (= max min) (format "%s = begin using CSV diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 7706234919..535729d2bb 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -263,8 +263,11 @@ (defun org-babel-sh-var-to-string (var &optional sep hline) (let ((echo-var (lambda (v) (if (stringp v) v (format "%S" v))))) (cond ((and (listp var) (or (listp (car var)) (eq (car var) 'hline))) - (orgtbl-to-generic var (list :sep (or sep "\t") :fmt echo-var - :hline hline))) + (orgtbl-to-generic + var + (list :sep (or sep "\t") :fmt echo-var + :hline hline + :with-special-rows nil))) ((listp var) (mapconcat echo-var var "\n")) (t (funcall echo-var var))))) diff --git a/lisp/ob-sql.el b/lisp/ob-sql.el index d7bcaa0973..8795934520 100644 --- a/lisp/ob-sql.el +++ b/lisp/ob-sql.el @@ -418,9 +418,11 @@ (defun org-babel-sql-expand-vars (body vars &optional sqlite) (insert (orgtbl-to-csv val (if sqlite nil - '(:fmt (lambda (el) (if (stringp el) - el - (format "%S" el)))))))) + '( :fmt (lambda (el) + (if (stringp el) + el + (format "%S" el))) + :with-special-rows nil))))) data-file) (if (stringp val) val (format "%S" val)))) body t t))) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index 836cfaffca..7f21a4cd55 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -140,7 +140,8 @@ (defun org-plot/gnuplot-to-data (table data-file params) (insert (orgtbl-to-generic table (org-combine-plists - '(:sep "\t" :fmt org-plot-quote-tsv-field) + '( :sep "\t" :fmt org-plot-quote-tsv-field + :with-special-rows nil) params))))) nil) -- 2.47.1 --=-=-=--