http-client port encoding failed

http-client port encoding failed

[mbcladwell]

Hi,

I am using http-client to submit requests that contain extended Latin  
characters e.g. "Marjanović+Ž".  I believe I have locales set up  
properly as I don't receive the "can't set locales" error.  Also I can  
regex with the extended Latin characters.

scheme@(guile-user)> (locale-encoding)
$11 = "UTF-8"

  When I submit a url containing the above Serbian name, I get the error:

Throw to key `encoding-error' with args `("put-char" "conversion to  
port encoding failed" 84 #<input-output: string 7febcf44c2a0> #\ć)'.

If I paste the URL into my browser I receive the correct response.

Prior to the request I use (set-port-encoding! (current-output-port)  
"UTF-8") without effect.
What should I do?
Thanks
Mortimer

Re: http-client port encoding failed

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]

On Fri, Sep 04, 2020 at 02:35:45PM -0500, mbcladwell@stihie.net wrote:
> 
> Hi,
> 
> I am using http-client to submit requests that contain extended
> Latin characters e.g. "Marjanović+Ž".  I believe I have locales set
> up properly as I don't receive the "can't set locales" error.  Also
> I can regex with the extended Latin characters.
> 
> scheme@(guile-user)> (locale-encoding)
> $11 = "UTF-8"
> 
>  When I submit a url containing the above Serbian name, I get the error:
> 
> Throw to key `encoding-error' with args `("put-char" "conversion to
> port encoding failed" 84 #<input-output: string 7febcf44c2a0> #\ć)'.

I'm not sure and can't double-check at the moment, but: the HTTP header
part (and the URL is part of it) /must/ be 7 bit ASCII. That's why there
is that URL encoding [1] -- basically you take the UTF-8 encoded string
and represent every byte (octet) beyond 0x7f (and some "hot" bytes
beneath that) by %<xx>, where <xx> is the hexadecimal representation
of that byte.

Perhaps you have to URL-encode your URL before "submitting" it.

> If I paste the URL into my browser I receive the correct response.

Your browser does all of the above for you. It just doesn't tell you :)

> Prior to the request I use (set-port-encoding! (current-output-port)
> "UTF-8") without effect.

Hm. I don't know what the "right" encoding for a HTTP connection
is. But the HTTP header part is 7 biz ASCII (which is a subset
of UTF-8, but just that :)

The quintessence is: try url-encoding your URL.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: http-client port encoding failed

[mbcladwell]

Not sure I am doing exactly what you suggest but the following (based  
on your suggestion) seems to work:

(define auth "Marjanović+Ž")
(http-request (string-append "https://......&term=" auth  
"[auth]&retmax=20")) ===> error!!
Throw to key `encoding-error' with args `("put-char" "conversion to  
port encoding failed" 84 #<input-output: string 7f9a247d75b0> #\ć)'.

(define authencode (uri-encode "Marjanović+Ž")) ===>  
"Marjanovi%C4%87%2B%C5%BD"
(http-request (string-append "https://....&term=" authencode  
"[auth]&retmax=20"))  ===>success!!

Thanks
Mortimer

Re: http-client port encoding failed

[tomas]

[-- Attachment #1: Type: text/plain, Size: 224 bytes --]

On Tue, Sep 08, 2020 at 09:26:57AM -0500, mbcladwell@stihie.net wrote:
> 
> Not sure I am doing exactly what you suggest but the following
> (based on your suggestion) seems to work:

Glad it helped :)

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]