all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Supported/Bundled CSV  support?
@ 2022-03-14 22:15 T.V Raman
  2022-03-14 22:30 ` Stefan Monnier
  0 siblings, 1 reply; 9+ messages in thread
From: T.V Raman @ 2022-03-14 22:15 UTC (permalink / raw)
  To: emacs-devel

The simple/innocent looking "CSV File Format " AKA Comma Separated
Values, has a surprizingly high number of quirks with respect to
quoting etc.

Languages like Python work around this with a csv module that comes
bundled; Emacs has a couple of modules out on elpa/melpa all of which
look old.

Is it perhaps time that we brought one of those up to snuff (CSV was
finally officially defined a couple of years ago via an RFC).


-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Supported/Bundled CSV  support?
  2022-03-14 22:15 Supported/Bundled CSV support? T.V Raman
@ 2022-03-14 22:30 ` Stefan Monnier
  2022-03-15  7:36   ` Ergus
  2022-03-16 13:43   ` Joost
  0 siblings, 2 replies; 9+ messages in thread
From: Stefan Monnier @ 2022-03-14 22:30 UTC (permalink / raw)
  To: T.V Raman; +Cc: emacs-devel

> Languages like Python work around this with a csv module that comes
> bundled; Emacs has a couple of modules out on elpa/melpa all of which
> look old.

Don't know about Melpa, but in (Non)GNU ELPA, I can only find one such
package and it dates back to Dec 2021.


        Stefan




^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Supported/Bundled CSV  support?
  2022-03-14 22:30 ` Stefan Monnier
@ 2022-03-15  7:36   ` Ergus
  2022-03-15 14:05     ` T.V Raman
  2022-03-16 13:43   ` Joost
  1 sibling, 1 reply; 9+ messages in thread
From: Ergus @ 2022-03-15  7:36 UTC (permalink / raw)
  To: emacs-devel, Stefan Monnier, T.V Raman

[-- Attachment #1: Type: text/plain, Size: 801 bytes --]

Just an idea/question:

With the new Lars package vtable may be possible to support CSV or add a package over it?

I agree it is a bit annoying to depend of an external package for such a simple format and maybe the implementation may be simpler than expected as the format it pretty simple in general.

WDYT?






On March 14, 2022 11:30:43 PM GMT+01:00, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> Languages like Python work around this with a csv module that comes
>> bundled; Emacs has a couple of modules out on elpa/melpa all of which
>> look old.
>
>Don't know about Melpa, but in (Non)GNU ELPA, I can only find one such
>package and it dates back to Dec 2021.
>
>
>        Stefan
>
>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 1295 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Supported/Bundled CSV  support?
  2022-03-15  7:36   ` Ergus
@ 2022-03-15 14:05     ` T.V Raman
  2022-03-15 14:57       ` Stefan Monnier
  0 siblings, 1 reply; 9+ messages in thread
From: T.V Raman @ 2022-03-15 14:05 UTC (permalink / raw)
  To: Ergus; +Cc: emacs-devel, Stefan Monnier

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=gb18030, Size: 1683 bytes --]

Ergus <spacibba@aol.com> writes:


A couple more thoughts around why we may want built-in CSV support:

1. Over the last few years, emacs has incorporated built-in support for:

A. xml --- libxml
B. json --- libjs
C. We're now talking sqlite

Not having CSV builtin feels like ignoring what is in plain-sight.

I think having built-in CSV support that abstracts away the various
details of parsing the various CSV variants will allow the emacs Dev
community to focus on user solutions  e.g.:

A. Seamless data import/export
B. Support the processing of larger amounts of data via org-table and the like
C. Json support vastly sped up modules like lsp and eglot; I'd expect
   the same once we cross the chasm with respect to making CSV support
   a built-in detail that elisp developers can take for granted.
> Just an idea/question:
>
> With the new Lars package vtable may be possible to support CSV or add
> a package over it?
>
> I agree it is a bit annoying to depend of an external package for such
> a simple format and maybe the implementation may be simpler than
> expected as the format it pretty simple in general.
>
> WDYT?
>
> On March 14, 2022 11:30:43 PM GMT+01:00, Stefan Monnier
> <monnier@iro.umontreal.ca> wrote:
>
>     Languages like Python work around this with a csv module that
>     comes
> bundled; Emacs has a couple of modules out on elpa/melpa all of which
> look old.
>
> Don't know about Melpa, but in (Non)GNU ELPA, I can only find one such
> package and it dates back to Dec 2021.
>
>         Stefan

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
7©4 Id: kg:/m/0285kf1  •0Ü8



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Supported/Bundled CSV  support?
  2022-03-15 14:05     ` T.V Raman
@ 2022-03-15 14:57       ` Stefan Monnier
  2022-03-15 15:27         ` T.V Raman
  0 siblings, 1 reply; 9+ messages in thread
From: Stefan Monnier @ 2022-03-15 14:57 UTC (permalink / raw)
  To: T.V Raman; +Cc: Ergus, emacs-devel

> I think having built-in CSV support that abstracts away the various
> details of parsing the various CSV variants will allow the emacs Dev
> community to focus on user solutions  e.g.:
>
> A. Seamless data import/export
> B. Support the processing of larger amounts of data via org-table and the like
> C. Json support vastly sped up modules like lsp and eglot; I'd expect
>    the same once we cross the chasm with respect to making CSV support
>    a built-in detail that elisp developers can take for granted.

You might be right, but if so you're talking about a very different kind
of "CSV support" than the kind I'm familiar with (which is the support
to manually edit/browse a human-generated CSV file, as offered by
`csv-mode`).  You seem to be talking about code to efficiently parse
a whole CSV file into a Lisp representation (list of lists or better) or
to generate a CSV-formatted string from such Lisp representation.

Is that right?  In that case, indeed `csv-mode` is of no
help, basically.


        Stefan


>> Just an idea/question:
>>
>> With the new Lars package vtable may be possible to support CSV or add
>> a package over it?
>>
>> I agree it is a bit annoying to depend of an external package for such
>> a simple format and maybe the implementation may be simpler than
>> expected as the format it pretty simple in general.
>>
>> WDYT?
>>
>> On March 14, 2022 11:30:43 PM GMT+01:00, Stefan Monnier
>> <monnier@iro.umontreal.ca> wrote:
>>
>>     Languages like Python work around this with a csv module that
>>     comes
>> bundled; Emacs has a couple of modules out on elpa/melpa all of which
>> look old.
>>
>> Don't know about Melpa, but in (Non)GNU ELPA, I can only find one such
>> package and it dates back to Dec 2021.
>>
>>         Stefan




^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Supported/Bundled CSV  support?
  2022-03-15 14:57       ` Stefan Monnier
@ 2022-03-15 15:27         ` T.V Raman
  0 siblings, 0 replies; 9+ messages in thread
From: T.V Raman @ 2022-03-15 15:27 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Ergus, emacs-devel

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=gb18030, Size: 2513 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:


Yes, you are correct.

There are modules like pcsv.el and  similarly named ones from the past
that do an approximate job -- when I looked into this approx 20 years
ago, I also ended up writing one of my own within Emacspeak --- in my
experience each of these hand-rolled csv parsers are differently broken
in their own way -- including the one in emacspeak, since as as I
pointed out in my earlier message, parsing various variants of CSV is
surprizingly hard.

>> I think having built-in CSV support that abstracts away the various
>> details of parsing the various CSV variants will allow the emacs Dev
>> community to focus on user solutions  e.g.:
>>
>> A. Seamless data import/export
>> B. Support the processing of larger amounts of data via org-table and the like
>> C. Json support vastly sped up modules like lsp and eglot; I'd expect
>>    the same once we cross the chasm with respect to making CSV support
>>    a built-in detail that elisp developers can take for granted.
>
> You might be right, but if so you're talking about a very different kind
> of "CSV support" than the kind I'm familiar with (which is the support
> to manually edit/browse a human-generated CSV file, as offered by
> `csv-mode`).  You seem to be talking about code to efficiently parse
> a whole CSV file into a Lisp representation (list of lists or better) or
> to generate a CSV-formatted string from such Lisp representation.
>
> Is that right?  In that case, indeed `csv-mode` is of no
> help, basically.
>
>
>         Stefan
>
>
>>> Just an idea/question:
>>>
>>> With the new Lars package vtable may be possible to support CSV or add
>>> a package over it?
>>>
>>> I agree it is a bit annoying to depend of an external package for such
>>> a simple format and maybe the implementation may be simpler than
>>> expected as the format it pretty simple in general.
>>>
>>> WDYT?
>>>
>>> On March 14, 2022 11:30:43 PM GMT+01:00, Stefan Monnier
>>> <monnier@iro.umontreal.ca> wrote:
>>>
>>>     Languages like Python work around this with a csv module that
>>>     comes
>>> bundled; Emacs has a couple of modules out on elpa/melpa all of which
>>> look old.
>>>
>>> Don't know about Melpa, but in (Non)GNU ELPA, I can only find one such
>>> package and it dates back to Dec 2021.
>>>
>>>         Stefan
>

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
7©4 Id: kg:/m/0285kf1  •0Ü8



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Supported/Bundled CSV  support?
  2022-03-14 22:30 ` Stefan Monnier
  2022-03-15  7:36   ` Ergus
@ 2022-03-16 13:43   ` Joost
  2022-03-16 14:46     ` T.V Raman
  2022-03-16 18:49     ` Stefan Monnier
  1 sibling, 2 replies; 9+ messages in thread
From: Joost @ 2022-03-16 13:43 UTC (permalink / raw)
  To: emacs-devel

On Mon, 14 Mar 2022, at 23:30, Stefan Monnier wrote:
[csv-mode]
> Don't know about Melpa, but in (Non)GNU ELPA, I can only find one such
> package and it dates back to Dec 2021.

Speaking of which, I recently had an occasion for using this package and I was somewhat surprised that I did not find a way to specify the field separator on a per-file (or per-buffer) basis. The file I wanted to open used `;` as separator, and some of the fields hat commas in them. Adding `;` to `csv-separators` caused Emacs to recognize *both* `;` and `,` as field separators, meaning that the file wasn't displayed correctly.

I needed to set `csv-separators` to `(";")` to get Emacs to handle the file correctly. But if I make that permatent, I wouldn't be able to open any CSV files with commas as separators anymore.

Did I miss something, or is this indeed how `csv-mode` works? If so, I'm inclined to create a bug report, cause I reckon few (in any) CSV files use more than one separator character.





-- 
Joost Kremers
Life has its moments



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Supported/Bundled CSV  support?
  2022-03-16 13:43   ` Joost
@ 2022-03-16 14:46     ` T.V Raman
  2022-03-16 18:49     ` Stefan Monnier
  1 sibling, 0 replies; 9+ messages in thread
From: T.V Raman @ 2022-03-16 14:46 UTC (permalink / raw)
  To: Joost; +Cc: emacs-devel

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=gb18030, Size: 384 bytes --]

A slightly harder but more scalable in my experience  is to use
forms which has been bundled in emacs for a very, very long time, in
fact for so long that most people have forgotten it. 

See
https://emacspeak.blogspot.com/2020/06/viewing-data-records-as-forms-old-is.html

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
7©4 Id: kg:/m/0285kf1  •0Ü8



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Supported/Bundled CSV  support?
  2022-03-16 13:43   ` Joost
  2022-03-16 14:46     ` T.V Raman
@ 2022-03-16 18:49     ` Stefan Monnier
  1 sibling, 0 replies; 9+ messages in thread
From: Stefan Monnier @ 2022-03-16 18:49 UTC (permalink / raw)
  To: Joost; +Cc: emacs-devel

> I needed to set `csv-separators` to `(";")` to get Emacs to handle the file
> correctly. But if I make that permatent, I wouldn't be able to open any CSV
> files with commas as separators anymore.
>
> Did I miss something, or is this indeed how `csv-mode` works? If so, I'm
> inclined to create a bug report, cause I reckon few (in any) CSV files use
> more than one separator character.

Indeed, there's a missing functionality in the UI to do that.
It would make a lot of sense to have a command to change the (set
of) separators (which could even try and guess based on the buffer's
contents).

I basically only ever use `tsv-mode`, which is why I haven't improved
that part of the code.  Patches welcome.


        Stefan




^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2022-03-16 18:49 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-03-14 22:15 Supported/Bundled CSV support? T.V Raman
2022-03-14 22:30 ` Stefan Monnier
2022-03-15  7:36   ` Ergus
2022-03-15 14:05     ` T.V Raman
2022-03-15 14:57       ` Stefan Monnier
2022-03-15 15:27         ` T.V Raman
2022-03-16 13:43   ` Joost
2022-03-16 14:46     ` T.V Raman
2022-03-16 18:49     ` Stefan Monnier

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.