Re: Improving the handling of system data (env, users, paths, ...)

[Eli Zaretskii <eliz@gnu.org>]

> Cc: "jean@abou-samra.fr" <jean@abou-samra.fr>, 
> 	"rlb@defaultvalue.org" <rlb@defaultvalue.org>, 
> 	"guile-devel@gnu.org" <guile-devel@gnu.org>
> From: Maxime Devos <maximedevos@telenet.be>
> Date: Sun, 7 Jul 2024 16:59:10 +0200
> 
> >> >> Guile is a Scheme implementation, bound by Scheme standards and compatibility
> >> >> with other Scheme implementations (and backwards compatibility too).
> >> >
> >> >Yes, I understand that.
> >> 
> >> Going by what you are saying below, I think you don’t.
> >
> >Thank you for your vote of confidence.
> 
> That was not a vote of confidence, if anything, it’s the contrary.

You don't say!

> > I’m pretty sure that they weren’t intending to get the 0xb5 byte. Rather, they were using the equivalent of ‘string-ref’ (i.e., ‘aref’) and demonstrating that the result is bogus in Scheme.  In Scheme, ‘(string-ref ...)’ needs to return a character, and there exists no (Unicode) character with codepoint 4194229, so what Emacs returns here would be bogus for (Guile) Scheme.
> 
> >aref in Emacs and string-ref in Guile are not the same, and if Guile
> needs to produce a raw byte in this scenario, it can be easily
> arranged.  In Emacs we have other goals.
> 
> It is the opposite. In Guile, string-ref does not need to produce bytes, but characters – just like aref (modulo difference in how Scheme and Emacs define ‘byte’).

But raw byte is not a character.

> >IOW, I think this argument is pointless, since it is easy to adapt the
> mechanism to what Guile needs.
> 
> No – the argument is about how it is impossible to adapt the mechanism to Guile, since bytes aren’t characters in Unicode.

I'm saying that Guile needs to support raw bytes as well, because they
happen in Real Life, including as part of otherwise legible text.

> > >From the Emacs manual:
> > 
> > >For example, you can access individual characters in a string using the function aref (see Functions that Operate on Arrays).
> > 
> > Thus, (aref the-string index) is the equivalent of (string-ref the-string index).
> 
> >No, because a raw byte is not a character.
> 
> Yes, because characters are characters. Both string-ref and aref return characters. This is documented in both the Emacs and Guile manual:
> 
> Again, from the Emacs manual:
> 
> > A string is a fixed sequence of characters. [...] Since strings are arrays, and therefore sequences as well, you can operate on them with the general array and sequence functions documented in Sequences, Arrays, and Vectors. For example, you can access individual characters in a string using the function aref (see Functions that Operate on Arrays).
> 
> Hence, (aref the-string index) returns (Emacs) characters.

You missed the description of raw bytes and unibyte strings, I guess.

> >If Guile restricts itself to Unicode characters and only them, it will
> lack important features.  So my suggestion is not to have this
> restriction.
> 
> Guile restricting strings to Unicode _is_ an important feature (simplicity, and compatibility).
> 
> Guile extending strings beyond Unicode is a _limitation_ (compatibility and other trickiness for applications).
> 
> I could imagine in the far future there might be too little codepoints left in Unicode, in which case the range of what Guile (and more generally, Scheme and Unicode) considers characters needs to be extended (even if that has some compatibility implicaitons), but that time hasn’t arrived yet.
> 
> The important feature of this thread, is supporting file names (and getenv stuff, etc.) that doesn’t fit properly in the ‘string’ model. As mentioned earlier (in the initial message, even), there are solutions to that do not impose the ‘let characters go beyond Unicode’ limitation.
> 
> >I think the fact that this discussion is held, and that Rob suggested
> to use Latin-1 for the purpose of supporting raw bytes is a clear
> indication that Guile, too, needs to deal with "character-like" data
> that does not fit the Unicode framework. 
> 
> True, and I never claimed otherwise.
> 
> > So I think saying that strings in Guile can only hold Unicode characters will not give you what this discussion attempts to give.
> 
> Sure, and I wasn’t trying to. What I (and IIUC, the other person as well) was doing was mentioning how neither the Emacs’s thing is a solution. (Whether because of backwards compatibility, or whether because of not _wanting_ to conflate bytes with characters (and not wanting to go beyond Unicode) with all the consequences this conflation would imply for applications.)
> 
> > In particular, how will you
> handle the situations described by Rob where a file has a name that is
> not a valid UTF-8 sequence (thus not "characters" as long as you
> interpret text as UTF-8)?
> 
> Scheme does not interpret text as UTF-8, that’s an internal implementation detail and a matter of things like locales. Instead, to Scheme text is (Unicode) characters.
> 
> I have outlined a solution (that does not conflate characters with bytes) in another response. IIRC, it was in a response so Rob. I would propose actually, you know, reading it. I’m not sure, but IIRC Rob also mentioned another solution (i.e., just accept bytevectors in some locations, or do Latin-1).
> 
> Also, this structure makes no sense. Even if I did not provide an alternative solution of my own, that wouldn’t mean Emacs’s thing is the answer. (Negative) criticism can be valid without providing alternatives.

That's fine by me.  I described what we have done in Emacs because I
think it works and works well.  For many years.  So I thought
describing it will be useful to Guile and will allow you to consider
if something like that could solve your problems, which I think are
very similar if not identical.  It is up to you whether to reject that
solution without trying to adapt it to Guile, and in that case I wish
you all the luck in finding your own solutions.