bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines

bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines

[Filipp Gunbin]

csv-mode 1.14 from elpa.  Both csv-separators and csv-field-quotes have
default values.

Create a new csv file with this single line:

--8<---------------cut here---------------start------------->8---
description,"a line, with comma, and another comma"
--8<---------------cut here---------------end--------------->8---

C-c C-t transforms this to:

--8<---------------cut here---------------start------------->8---
description
"a line
 with comma
 and another comma"
--8<---------------cut here---------------end--------------->8---

The commas inside a (quoted) field were replaced by newlines, this looks
like a bug.

Thanks.

In GNU Emacs 28.0.50 (build 3, x86_64-apple-darwin20.2.0, NS appkit-2022.20 Version 11.1 (Build 20C69))
 of 2021-02-05 built on fgunbin.local
Repository revision: f00afb9bb8b5356690e2a785d14aa89995c96f50
Repository branch: master
System Description:  macOS 11.1

bug#46328: additional test case

[Filipp Gunbin]

If there're two lines (csv-header-lines is 0, though):

--8<---------------cut here---------------start------------->8---
name,value
description,"a field with comma, and another comma, here"
--8<---------------cut here---------------end--------------->8---

Then the newlines are just prepended before commas inside the quoted
field:

--8<---------------cut here---------------start------------->8---
name,description
value,"a field with comma
, and another comma
, here"
--8<---------------cut here---------------end--------------->8---

bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines

[Peder O. Klingenberg]

[-- Attachment #1: Type: text/plain, Size: 350 bytes --]

On Fri, 2021-02-05 17:17:39 +0300, Filipp Gunbin wrote:

> The commas inside a (quoted) field were replaced by newlines, this looks
> like a bug.

Caused by split-string not caring about char-syntax ?\".  Here's a
patch.  If a line has quote chars, use csv-forward-field to fetch each
field, ensuring consistency in what the mode considers a field.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-transposing-csv-files-with-quoted-fields.patch --]
[-- Type: text/x-patch, Size: 2365 bytes --]

From d6b51e2f07d585106ce6ccfe484f12a9ed3fe9dc Mon Sep 17 00:00:00 2001
From: "Peder O. Klingenberg" <peder@klingenberg.no>
Date: Tue, 23 Feb 2021 00:14:35 +0100
Subject: [PATCH] Fix transposing csv files with quoted fields

* csv-mode.el
(csv--collect-fields): New function.
(csv-transpose): Use the new function instead of split-string.

(Fixes Bug#46328)
---
 csv-mode.el | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)

diff --git a/csv-mode.el b/csv-mode.el
index eaea881801..ecc33a7bcc 100644
--- a/csv-mode.el
+++ b/csv-mode.el
@@ -4,7 +4,7 @@
 
 ;; Author: "Francis J. Wright" <F.J.Wright@qmul.ac.uk>
 ;; Maintainer: emacs-devel@gnu.org
-;; Version: 1.14
+;; Version: 1.15
 ;; Package-Requires: ((emacs "24.1") (cl-lib "0.5"))
 ;; Keywords: convenience
 
@@ -1264,9 +1264,7 @@ When called non-interactively, BEG and END specify region to process."
 	      (forward-line)
 	    (let ((lep (line-end-position)))
 	      (push
-	       (split-string
-		(buffer-substring-no-properties (point) lep)
-		csv-separator-regexp)
+	       (csv--collect-fields lep)
 	       rows)
 	      (delete-region (point) lep)
 	      (or (eobp) (delete-char 1)))))
@@ -1305,6 +1303,26 @@ When called non-interactively, BEG and END specify region to process."
 	;; Re-do soft alignment if necessary:
 	(if align (csv-align-fields nil (point-min) (point-max)))))))
 
+(defun csv--collect-fields (row-end-position)
+  "Collect the fields of a row.
+Splits a row into fields, honoring quoted fields, and returns
+the list of fields.  ROW-END-POSITION is the end-of-line position.
+point is assumed to be at the beginning of the line."
+  (let ((csv-field-quotes-regexp (apply #'concat `("[" ,@csv-field-quotes "]")))
+	(row-text (buffer-substring-no-properties (point) row-end-position))
+	fields field-start)
+    (if (not (string-match csv-field-quotes-regexp row-text))
+	(split-string row-text csv-separator-regexp)
+      (save-excursion
+	(while (< (setq field-start (point)) row-end-position)
+	  (csv-forward-field 1)
+	  (push
+	   (buffer-substring-no-properties field-start (point))
+	   fields)
+	  (if (memq (following-char) csv-separator-chars)
+	      (forward-char)))
+	(nreverse fields)))))
+
 (defvar-local csv--header-line nil)
 (defvar-local csv--header-hscroll nil)
 (defvar-local csv--header-string nil)
-- 
2.30.1.windows.1

bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines

[Lars Ingebrigtsen]

"Peder O. Klingenberg" <peder@klingenberg.no> writes:

> Caused by split-string not caring about char-syntax ?\".  Here's a
> patch.  If a line has quote chars, use csv-forward-field to fetch each
> field, ensuring consistency in what the mode considers a field.

Thanks; applied to the csv-mode GNU ELPA package now.  (That is, I hope
-- I'm not 100% sure how the new ELPA layout works after the recent
changes...)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no