From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: "Peder O. Klingenberg" Newsgroups: gmane.emacs.bugs Subject: bug#46328: 28.0.50; csv-transpose replaces field delimiters in quoted fields with newlines Date: Tue, 23 Feb 2021 00:27:43 +0100 Message-ID: <86mtvv3cpc.fsf@klingenberg.no> References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="13357"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (windows-nt) To: 46328@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Feb 23 00:28:14 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lEKcf-0003M2-S7 for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 23 Feb 2021 00:28:14 +0100 Original-Received: from localhost ([::1]:47268 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lEKce-0002I7-U1 for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 22 Feb 2021 18:28:12 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:53796) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lEKcU-0002Ga-U2 for bug-gnu-emacs@gnu.org; Mon, 22 Feb 2021 18:28:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:46359) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lEKcU-0001qn-Lf for bug-gnu-emacs@gnu.org; Mon, 22 Feb 2021 18:28:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lEKcU-0001jc-HP for bug-gnu-emacs@gnu.org; Mon, 22 Feb 2021 18:28:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: "Peder O. Klingenberg" Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 22 Feb 2021 23:28:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46328 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.16140364766656 (code B ref -1); Mon, 22 Feb 2021 23:28:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 22 Feb 2021 23:27:56 +0000 Original-Received: from localhost ([127.0.0.1]:57905 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lEKcO-0001jG-2D for submit@debbugs.gnu.org; Mon, 22 Feb 2021 18:27:56 -0500 Original-Received: from lists.gnu.org ([209.51.188.17]:48822) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lEKcM-0001j9-4c for submit@debbugs.gnu.org; Mon, 22 Feb 2021 18:27:54 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:53762) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lEKcL-0002Fn-Vm for bug-gnu-emacs@gnu.org; Mon, 22 Feb 2021 18:27:54 -0500 Original-Received: from castor.klingenberg.no ([176.125.234.34]:38894) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lEKcJ-0001l8-8E for bug-gnu-emacs@gnu.org; Mon, 22 Feb 2021 18:27:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=klingenberg.no; s=20200407; h=Content-Type:MIME-Version:Message-ID: In-Reply-To:Date:References:Subject:To:From:Sender:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=Y3nYPkq3AkwAzDEhgFXx5gFuQtfZMbDBIUOLqClTzto=; b=K7Lo06DFr+1ePMtJkdoMpccOh 16eT7ET+Lbwzsk2aD9G1DB/zJ/dsceeHUF3FBqjdMvdVm4iUotIIV+wT+PQsrVWumJHwrTnVtnKz3 qqYYRZhEV7/ibMENjenBUgYyHzPp7e5LB8QjOLYSo3rnwdc4nJJvC/fGAy5RtYiCbRk6Ue06r41Xu XCJRIgUp1a131HCeZwJKMVYm+7SImrYGlWEK/mGltLpsH6i6SBRDzzTzpaSiCRKVzk3qwAMYhu4/H WMEplgZ20Qu0KuVca/uROd0WgAXa8BP7psEvisGL+Ev901PIMOBdvuwOQcdQTB/eDoSFSQDnz4uut xuzLxlfwg==; Original-Received: from ip-239-146-106-77.eidsiva.net ([77.106.146.239] helo=PedersHP) by castor.klingenberg.no with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lEKcE-00086V-6T for bug-gnu-emacs@gnu.org; Tue, 23 Feb 2021 00:27:46 +0100 In-Reply-To: (Filipp Gunbin's message of "Fri, 05 Feb 2021 17:17:39 +0300") Received-SPF: pass client-ip=176.125.234.34; envelope-from=peder@klingenberg.no; helo=castor.klingenberg.no X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:200634 Archived-At: --=-=-= Content-Type: text/plain On Fri, 2021-02-05 17:17:39 +0300, Filipp Gunbin wrote: > The commas inside a (quoted) field were replaced by newlines, this looks > like a bug. Caused by split-string not caring about char-syntax ?\". Here's a patch. If a line has quote chars, use csv-forward-field to fetch each field, ensuring consistency in what the mode considers a field. --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Fix-transposing-csv-files-with-quoted-fields.patch >From d6b51e2f07d585106ce6ccfe484f12a9ed3fe9dc Mon Sep 17 00:00:00 2001 From: "Peder O. Klingenberg" Date: Tue, 23 Feb 2021 00:14:35 +0100 Subject: [PATCH] Fix transposing csv files with quoted fields * csv-mode.el (csv--collect-fields): New function. (csv-transpose): Use the new function instead of split-string. (Fixes Bug#46328) --- csv-mode.el | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/csv-mode.el b/csv-mode.el index eaea881801..ecc33a7bcc 100644 --- a/csv-mode.el +++ b/csv-mode.el @@ -4,7 +4,7 @@ ;; Author: "Francis J. Wright" ;; Maintainer: emacs-devel@gnu.org -;; Version: 1.14 +;; Version: 1.15 ;; Package-Requires: ((emacs "24.1") (cl-lib "0.5")) ;; Keywords: convenience @@ -1264,9 +1264,7 @@ When called non-interactively, BEG and END specify region to process." (forward-line) (let ((lep (line-end-position))) (push - (split-string - (buffer-substring-no-properties (point) lep) - csv-separator-regexp) + (csv--collect-fields lep) rows) (delete-region (point) lep) (or (eobp) (delete-char 1))))) @@ -1305,6 +1303,26 @@ When called non-interactively, BEG and END specify region to process." ;; Re-do soft alignment if necessary: (if align (csv-align-fields nil (point-min) (point-max))))))) +(defun csv--collect-fields (row-end-position) + "Collect the fields of a row. +Splits a row into fields, honoring quoted fields, and returns +the list of fields. ROW-END-POSITION is the end-of-line position. +point is assumed to be at the beginning of the line." + (let ((csv-field-quotes-regexp (apply #'concat `("[" ,@csv-field-quotes "]"))) + (row-text (buffer-substring-no-properties (point) row-end-position)) + fields field-start) + (if (not (string-match csv-field-quotes-regexp row-text)) + (split-string row-text csv-separator-regexp) + (save-excursion + (while (< (setq field-start (point)) row-end-position) + (csv-forward-field 1) + (push + (buffer-substring-no-properties field-start (point)) + fields) + (if (memq (following-char) csv-separator-chars) + (forward-char))) + (nreverse fields))))) + (defvar-local csv--header-line nil) (defvar-local csv--header-hscroll nil) (defvar-local csv--header-string nil) -- 2.30.1.windows.1 --=-=-=--