* bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines
@ 2021-02-05 14:17 Filipp Gunbin
2021-02-05 14:46 ` bug#46328: additional test case Filipp Gunbin
2021-02-22 23:27 ` bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Peder O. Klingenberg
0 siblings, 2 replies; 4+ messages in thread
From: Filipp Gunbin @ 2021-02-05 14:17 UTC (permalink / raw)
To: 46328
csv-mode 1.14 from elpa. Both csv-separators and csv-field-quotes have
default values.
Create a new csv file with this single line:
--8<---------------cut here---------------start------------->8---
description,"a line, with comma, and another comma"
--8<---------------cut here---------------end--------------->8---
C-c C-t transforms this to:
--8<---------------cut here---------------start------------->8---
description
"a line
with comma
and another comma"
--8<---------------cut here---------------end--------------->8---
The commas inside a (quoted) field were replaced by newlines, this looks
like a bug.
Thanks.
In GNU Emacs 28.0.50 (build 3, x86_64-apple-darwin20.2.0, NS appkit-2022.20 Version 11.1 (Build 20C69))
of 2021-02-05 built on fgunbin.local
Repository revision: f00afb9bb8b5356690e2a785d14aa89995c96f50
Repository branch: master
System Description: macOS 11.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* bug#46328: additional test case
2021-02-05 14:17 bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Filipp Gunbin
@ 2021-02-05 14:46 ` Filipp Gunbin
2021-02-22 23:27 ` bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Peder O. Klingenberg
1 sibling, 0 replies; 4+ messages in thread
From: Filipp Gunbin @ 2021-02-05 14:46 UTC (permalink / raw)
To: 46328
If there're two lines (csv-header-lines is 0, though):
--8<---------------cut here---------------start------------->8---
name,value
description,"a field with comma, and another comma, here"
--8<---------------cut here---------------end--------------->8---
Then the newlines are just prepended before commas inside the quoted
field:
--8<---------------cut here---------------start------------->8---
name,description
value,"a field with comma
, and another comma
, here"
--8<---------------cut here---------------end--------------->8---
^ permalink raw reply [flat|nested] 4+ messages in thread
* bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines
2021-02-05 14:17 bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Filipp Gunbin
2021-02-05 14:46 ` bug#46328: additional test case Filipp Gunbin
@ 2021-02-22 23:27 ` Peder O. Klingenberg
2021-02-23 15:51 ` Lars Ingebrigtsen
1 sibling, 1 reply; 4+ messages in thread
From: Peder O. Klingenberg @ 2021-02-22 23:27 UTC (permalink / raw)
To: 46328
[-- Attachment #1: Type: text/plain, Size: 350 bytes --]
On Fri, 2021-02-05 17:17:39 +0300, Filipp Gunbin wrote:
> The commas inside a (quoted) field were replaced by newlines, this looks
> like a bug.
Caused by split-string not caring about char-syntax ?\". Here's a
patch. If a line has quote chars, use csv-forward-field to fetch each
field, ensuring consistency in what the mode considers a field.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-transposing-csv-files-with-quoted-fields.patch --]
[-- Type: text/x-patch, Size: 2365 bytes --]
From d6b51e2f07d585106ce6ccfe484f12a9ed3fe9dc Mon Sep 17 00:00:00 2001
From: "Peder O. Klingenberg" <peder@klingenberg.no>
Date: Tue, 23 Feb 2021 00:14:35 +0100
Subject: [PATCH] Fix transposing csv files with quoted fields
* csv-mode.el
(csv--collect-fields): New function.
(csv-transpose): Use the new function instead of split-string.
(Fixes Bug#46328)
---
csv-mode.el | 26 ++++++++++++++++++++++----
1 file changed, 22 insertions(+), 4 deletions(-)
diff --git a/csv-mode.el b/csv-mode.el
index eaea881801..ecc33a7bcc 100644
--- a/csv-mode.el
+++ b/csv-mode.el
@@ -4,7 +4,7 @@
;; Author: "Francis J. Wright" <F.J.Wright@qmul.ac.uk>
;; Maintainer: emacs-devel@gnu.org
-;; Version: 1.14
+;; Version: 1.15
;; Package-Requires: ((emacs "24.1") (cl-lib "0.5"))
;; Keywords: convenience
@@ -1264,9 +1264,7 @@ When called non-interactively, BEG and END specify region to process."
(forward-line)
(let ((lep (line-end-position)))
(push
- (split-string
- (buffer-substring-no-properties (point) lep)
- csv-separator-regexp)
+ (csv--collect-fields lep)
rows)
(delete-region (point) lep)
(or (eobp) (delete-char 1)))))
@@ -1305,6 +1303,26 @@ When called non-interactively, BEG and END specify region to process."
;; Re-do soft alignment if necessary:
(if align (csv-align-fields nil (point-min) (point-max)))))))
+(defun csv--collect-fields (row-end-position)
+ "Collect the fields of a row.
+Splits a row into fields, honoring quoted fields, and returns
+the list of fields. ROW-END-POSITION is the end-of-line position.
+point is assumed to be at the beginning of the line."
+ (let ((csv-field-quotes-regexp (apply #'concat `("[" ,@csv-field-quotes "]")))
+ (row-text (buffer-substring-no-properties (point) row-end-position))
+ fields field-start)
+ (if (not (string-match csv-field-quotes-regexp row-text))
+ (split-string row-text csv-separator-regexp)
+ (save-excursion
+ (while (< (setq field-start (point)) row-end-position)
+ (csv-forward-field 1)
+ (push
+ (buffer-substring-no-properties field-start (point))
+ fields)
+ (if (memq (following-char) csv-separator-chars)
+ (forward-char)))
+ (nreverse fields)))))
+
(defvar-local csv--header-line nil)
(defvar-local csv--header-hscroll nil)
(defvar-local csv--header-string nil)
--
2.30.1.windows.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines
2021-02-22 23:27 ` bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Peder O. Klingenberg
@ 2021-02-23 15:51 ` Lars Ingebrigtsen
0 siblings, 0 replies; 4+ messages in thread
From: Lars Ingebrigtsen @ 2021-02-23 15:51 UTC (permalink / raw)
To: Peder O. Klingenberg; +Cc: 46328
"Peder O. Klingenberg" <peder@klingenberg.no> writes:
> Caused by split-string not caring about char-syntax ?\". Here's a
> patch. If a line has quote chars, use csv-forward-field to fetch each
> field, ensuring consistency in what the mode considers a field.
Thanks; applied to the csv-mode GNU ELPA package now. (That is, I hope
-- I'm not 100% sure how the new ELPA layout works after the recent
changes...)
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-02-23 15:51 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-02-05 14:17 bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Filipp Gunbin
2021-02-05 14:46 ` bug#46328: additional test case Filipp Gunbin
2021-02-22 23:27 ` bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Peder O. Klingenberg
2021-02-23 15:51 ` Lars Ingebrigtsen
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.