Re: [PATCH] Add a mechanism for passing unibyte strings from lisp to modules.

["Brennan Vincent" <brennan@umanwizard.com>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <acorallo@gnu.org>
>> Cc: "Brennan Vincent" <brennan@umanwizard.com>,  Stefan Kangas
>>  <stefankangas@gmail.com>,  emacs-devel@gnu.org
>> Date: Sun, 23 Jun 2024 17:15:39 -0400
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: "Brennan Vincent" <brennan@umanwizard.com>
>> >> Date: Sat, 22 Jun 2024 11:22:56 -0400
>> >> 
>> >> Eli Zaretskii <eliz@gnu.org> writes:
>> >> 
>> >> > Why can't you have the module code itself read the file, instead of
>> >> > getting the bytes from Emacs?  Passing large amounts of bytes from
>> >> > Emacs to a module is a very inefficient way of talking to modules
>> >> > anyway, because Emacs is not optimized for moving text to and fro in
>> >> > the shape of Lisp strings.  To say nothing of the GC pressure you will
>> >> > have in your mode, due to a constant consing of strings.  It is best
>> >> > to avoid all that to begin with.
>> >> 
>> >> Of course it's possible to do that, but I wanted to write my mode in
>> >> elisp as much as possible and keep the C side minimal, simply because I
>> >> find elisp a much more enjoyable language to use. But if
>> >> you are opposed to adding this code I can go with that approach.
>> >> 
>> >> Another possibility which would avoid adding specifically
>> >> unibyte-related surface area to the modules API would be to create an
>> >> extended version of copy_string_contents which can take any coding
>> >> system, rather than forcing UTF-8.
>> >> 
>> >> Would you be open to such an approach? If so, I will send an updated patch.
>> >
>> > I very much dislike the idea of letting modules deal with unibyte
>> > strings, for the reasons I explained.  Basically, it will open a large
>> > Pandora box by allowing people who don't know enough about the
>> > subtleties of unibyte strings in Emacs to write buggy modules which
>> > will crash Emacs.
>> >
>> > But let's hear the other co-maintainers.  Stefan and Andrea, what is
>> > your POV on these issues?
>> 
>> I, for one, would be not too much worried.  People writing modules
>> should be already very responsible for what they write as they have
>> already plenty of ways to shoot in their feet 🤷.
>
> The problem is that we get to clean up their mess in too many cases.
> Especially when the package is on ELPA.
>
>> Perhaps we could mitigate the risk with some doc/comment explaining the
>> specific usecase this interface is meant to serve so it's not miss-used?
>
> If we want to allow Emacs to send binary data, I'd rather come up with
> a specialized interface to do just that.  Explaining the subtleties of
> using unibyte text in Emacs is a tough job, since it involves a lot of
> low-level technical details.  When unibyte text comes from encoding
> human-readable text that is at least justified, since that's what
> Emacs was designed to d, among other things.  But using Emacs as a
> handy method of reading binary data, to avoid doing that in the module
> itself, and asking us to add an interface for that use case is too
> much for my palate.

I think it would be great if emacs grew a specialized vector-of-bytes type.

BTW, I have already rewritten my mode to not attempt to pass data with
unibyte strings, and to read/write the file in C. So this is no longer
relevant to me personally. But I think other module writers will hit a
similar issue, and it will be good to have something in place for this
use case.