Re: [PATCH] Add a mechanism for passing unibyte strings from lisp to modules.

[Eli Zaretskii <eliz@gnu.org>]

> From: Andrea Corallo <acorallo@gnu.org>
> Cc: "Brennan Vincent" <brennan@umanwizard.com>,  Stefan Kangas
>  <stefankangas@gmail.com>,  emacs-devel@gnu.org
> Date: Sun, 23 Jun 2024 17:15:39 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: "Brennan Vincent" <brennan@umanwizard.com>
> >> Date: Sat, 22 Jun 2024 11:22:56 -0400
> >> 
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> > Why can't you have the module code itself read the file, instead of
> >> > getting the bytes from Emacs?  Passing large amounts of bytes from
> >> > Emacs to a module is a very inefficient way of talking to modules
> >> > anyway, because Emacs is not optimized for moving text to and fro in
> >> > the shape of Lisp strings.  To say nothing of the GC pressure you will
> >> > have in your mode, due to a constant consing of strings.  It is best
> >> > to avoid all that to begin with.
> >> 
> >> Of course it's possible to do that, but I wanted to write my mode in
> >> elisp as much as possible and keep the C side minimal, simply because I
> >> find elisp a much more enjoyable language to use. But if
> >> you are opposed to adding this code I can go with that approach.
> >> 
> >> Another possibility which would avoid adding specifically
> >> unibyte-related surface area to the modules API would be to create an
> >> extended version of copy_string_contents which can take any coding
> >> system, rather than forcing UTF-8.
> >> 
> >> Would you be open to such an approach? If so, I will send an updated patch.
> >
> > I very much dislike the idea of letting modules deal with unibyte
> > strings, for the reasons I explained.  Basically, it will open a large
> > Pandora box by allowing people who don't know enough about the
> > subtleties of unibyte strings in Emacs to write buggy modules which
> > will crash Emacs.
> >
> > But let's hear the other co-maintainers.  Stefan and Andrea, what is
> > your POV on these issues?
> 
> I, for one, would be not too much worried.  People writing modules
> should be already very responsible for what they write as they have
> already plenty of ways to shoot in their feet 🤷.

The problem is that we get to clean up their mess in too many cases.
Especially when the package is on ELPA.

> Perhaps we could mitigate the risk with some doc/comment explaining the
> specific usecase this interface is meant to serve so it's not miss-used?

If we want to allow Emacs to send binary data, I'd rather come up with
a specialized interface to do just that.  Explaining the subtleties of
using unibyte text in Emacs is a tough job, since it involves a lot of
low-level technical details.  When unibyte text comes from encoding
human-readable text that is at least justified, since that's what
Emacs was designed to d, among other things.  But using Emacs as a
handy method of reading binary data, to avoid doing that in the module
itself, and asking us to add an interface for that use case is too
much for my palate.