* bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines @ 2021-02-05 14:17 Filipp Gunbin 2021-02-05 14:46 ` bug#46328: additional test case Filipp Gunbin 2021-02-22 23:27 ` bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Peder O. Klingenberg 0 siblings, 2 replies; 4+ messages in thread From: Filipp Gunbin @ 2021-02-05 14:17 UTC (permalink / raw) To: 46328 csv-mode 1.14 from elpa. Both csv-separators and csv-field-quotes have default values. Create a new csv file with this single line: --8<---------------cut here---------------start------------->8--- description,"a line, with comma, and another comma" --8<---------------cut here---------------end--------------->8--- C-c C-t transforms this to: --8<---------------cut here---------------start------------->8--- description "a line with comma and another comma" --8<---------------cut here---------------end--------------->8--- The commas inside a (quoted) field were replaced by newlines, this looks like a bug. Thanks. In GNU Emacs 28.0.50 (build 3, x86_64-apple-darwin20.2.0, NS appkit-2022.20 Version 11.1 (Build 20C69)) of 2021-02-05 built on fgunbin.local Repository revision: f00afb9bb8b5356690e2a785d14aa89995c96f50 Repository branch: master System Description: macOS 11.1 ^ permalink raw reply [flat|nested] 4+ messages in thread
* bug#46328: additional test case 2021-02-05 14:17 bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Filipp Gunbin @ 2021-02-05 14:46 ` Filipp Gunbin 2021-02-22 23:27 ` bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Peder O. Klingenberg 1 sibling, 0 replies; 4+ messages in thread From: Filipp Gunbin @ 2021-02-05 14:46 UTC (permalink / raw) To: 46328 If there're two lines (csv-header-lines is 0, though): --8<---------------cut here---------------start------------->8--- name,value description,"a field with comma, and another comma, here" --8<---------------cut here---------------end--------------->8--- Then the newlines are just prepended before commas inside the quoted field: --8<---------------cut here---------------start------------->8--- name,description value,"a field with comma , and another comma , here" --8<---------------cut here---------------end--------------->8--- ^ permalink raw reply [flat|nested] 4+ messages in thread
* bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines 2021-02-05 14:17 bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Filipp Gunbin 2021-02-05 14:46 ` bug#46328: additional test case Filipp Gunbin @ 2021-02-22 23:27 ` Peder O. Klingenberg 2021-02-23 15:51 ` Lars Ingebrigtsen 1 sibling, 1 reply; 4+ messages in thread From: Peder O. Klingenberg @ 2021-02-22 23:27 UTC (permalink / raw) To: 46328 [-- Attachment #1: Type: text/plain, Size: 350 bytes --] On Fri, 2021-02-05 17:17:39 +0300, Filipp Gunbin wrote: > The commas inside a (quoted) field were replaced by newlines, this looks > like a bug. Caused by split-string not caring about char-syntax ?\". Here's a patch. If a line has quote chars, use csv-forward-field to fetch each field, ensuring consistency in what the mode considers a field. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Fix-transposing-csv-files-with-quoted-fields.patch --] [-- Type: text/x-patch, Size: 2365 bytes --] From d6b51e2f07d585106ce6ccfe484f12a9ed3fe9dc Mon Sep 17 00:00:00 2001 From: "Peder O. Klingenberg" <peder@klingenberg.no> Date: Tue, 23 Feb 2021 00:14:35 +0100 Subject: [PATCH] Fix transposing csv files with quoted fields * csv-mode.el (csv--collect-fields): New function. (csv-transpose): Use the new function instead of split-string. (Fixes Bug#46328) --- csv-mode.el | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/csv-mode.el b/csv-mode.el index eaea881801..ecc33a7bcc 100644 --- a/csv-mode.el +++ b/csv-mode.el @@ -4,7 +4,7 @@ ;; Author: "Francis J. Wright" <F.J.Wright@qmul.ac.uk> ;; Maintainer: emacs-devel@gnu.org -;; Version: 1.14 +;; Version: 1.15 ;; Package-Requires: ((emacs "24.1") (cl-lib "0.5")) ;; Keywords: convenience @@ -1264,9 +1264,7 @@ When called non-interactively, BEG and END specify region to process." (forward-line) (let ((lep (line-end-position))) (push - (split-string - (buffer-substring-no-properties (point) lep) - csv-separator-regexp) + (csv--collect-fields lep) rows) (delete-region (point) lep) (or (eobp) (delete-char 1))))) @@ -1305,6 +1303,26 @@ When called non-interactively, BEG and END specify region to process." ;; Re-do soft alignment if necessary: (if align (csv-align-fields nil (point-min) (point-max))))))) +(defun csv--collect-fields (row-end-position) + "Collect the fields of a row. +Splits a row into fields, honoring quoted fields, and returns +the list of fields. ROW-END-POSITION is the end-of-line position. +point is assumed to be at the beginning of the line." + (let ((csv-field-quotes-regexp (apply #'concat `("[" ,@csv-field-quotes "]"))) + (row-text (buffer-substring-no-properties (point) row-end-position)) + fields field-start) + (if (not (string-match csv-field-quotes-regexp row-text)) + (split-string row-text csv-separator-regexp) + (save-excursion + (while (< (setq field-start (point)) row-end-position) + (csv-forward-field 1) + (push + (buffer-substring-no-properties field-start (point)) + fields) + (if (memq (following-char) csv-separator-chars) + (forward-char))) + (nreverse fields))))) + (defvar-local csv--header-line nil) (defvar-local csv--header-hscroll nil) (defvar-local csv--header-string nil) -- 2.30.1.windows.1 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines 2021-02-22 23:27 ` bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Peder O. Klingenberg @ 2021-02-23 15:51 ` Lars Ingebrigtsen 0 siblings, 0 replies; 4+ messages in thread From: Lars Ingebrigtsen @ 2021-02-23 15:51 UTC (permalink / raw) To: Peder O. Klingenberg; +Cc: 46328 "Peder O. Klingenberg" <peder@klingenberg.no> writes: > Caused by split-string not caring about char-syntax ?\". Here's a > patch. If a line has quote chars, use csv-forward-field to fetch each > field, ensuring consistency in what the mode considers a field. Thanks; applied to the csv-mode GNU ELPA package now. (That is, I hope -- I'm not 100% sure how the new ELPA layout works after the recent changes...) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-02-23 15:51 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-02-05 14:17 bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Filipp Gunbin 2021-02-05 14:46 ` bug#46328: additional test case Filipp Gunbin 2021-02-22 23:27 ` bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Peder O. Klingenberg 2021-02-23 15:51 ` Lars Ingebrigtsen
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.