Re: url-retrieve and encoding

[<tomas@tuxteam.de>]

[-- Attachment #1: Type: text/plain, Size: 1827 bytes --]

On Sun, Feb 11, 2024 at 09:21:39PM +0200, Eli Zaretskii wrote:
> > Date: Sun, 11 Feb 2024 18:49:25 +0100
> > From: tomas@tuxteam.de
> > Cc: help-gnu-emacs@gnu.org
> > 
> > > > Yes: decode-coding-region.
> > > 
> > > Ahhh -- thanks a bunch for this one! How could I have missed it.
> > > 
> > > > > (...) But that feels
> > > > > a bit... gross:
> > > > 
> > > > Indeed.  Why didn't you try decoding to begin with?
> > 
> > OK, now I can answer this question more precisely: actually, I'd
> > been there already and was coufused that the function did... nothing.
> > 
> > Now at least I know why: the buffer is unibyte.
> 
> The solution is (quite obviously) not to do that in-place.

I guessed so, thanks for the clarification.

> Alternatively, you could make the buffer multibyte in advance, but
> that's tricky, so I don't recommend that.

If url-retrieve had a "callback interface", as processes have, with
their filters, then one could arrange things for the decoding to happen
there. Actually, that's what's going on in the background, I guess.

> > Its content /is/ utf-8.
> 
> That's not really 100% accurate, although it's close.  If the unibyte
> buffer includes byte sequences that are not valid UTF-8, decoding does
> change the byte stream in those places.

Of course, you are right. The HTTP headers /state/ it to be utf-8. It's
like trusting the label on the bottle :-)

> > But then... I can do things "in buffer" by simply invoking
> > (toggle-enable-multibyte-characters t). At least, it seems to
> > work. But... is it a good idea?
> 
> No.  Always call the decode function, never play with
> multi-uni-byteness, because the latter will eventually surprise (or
> bite) you.

I guessed so. Thanks for your patience (and for helping me learn).

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]