TeXmacs on Guile 2.2.4

TeXmacs on Guile 2.2.4

[Massimiliano Gubinelli]

Hi all,

 I’m porting GNU TeXmacs (www.texmacs.org) to Guile 2.2.4 (and hopefully to Guile 3.0). The current status is the following. TeXmacs runs on Guile 2.2.4 with autocompilation turned off: the applications opens and can read and render some files. More work is necessary to switch on compilation. However I’ve a problem with reading some TeXmacs files from the disk. Some of them works OK, but others give errors while reading: the file is first read into a C string by TeXmacs and then converted into a SCM object for further manipulation by Guile. At that point I receive this error message

Backtrace:
           7 (apply-smob/1 #<catch-closure 117caea80>)
           6 (apply-smob/1 #<catch-closure 13de3dfc0>)
           5 (apply-smob/1 #<catch-closure 13de3df40>)
           4 (protected-call #<procedure 13de34380 at kernel/gui/men…>)
           3 (apply-smob/1 #<catch-closure 13de3dee0>)
           2 (apply-smob/1 #<catch-closure 13de3de80>)
In texmacs/texmacs/tm-files.scm:
   413:15  1 (load-buffer-load # ())
In unknown file:
           0 (buffer-load #)

ERROR: In procedure buffer-load:
Throw to key `decoding-error' with args `("scm_from_stringn" "input locale conversion error" 92 #vu8(60 84 101 88 109 97 99 115 124 49 46 57 57 46 57 62 10 10 60 115 116 121 108 101 124 60 116 117 112 108 101 124 97 114 116 105 99 108 101 124 98 114 105 116 105 115 104 124 99 111 109 112 97 99 116 45 108 105 115 116 62 62 10 10 60 92 98 111 100 121 62 10 32 32 60 92 104 105 100 101 45 112 114 101 97 109 98 108 101 62 10 32 32 32 32 92 59 10 10 32 32 32 32 60 97 115 115 105 103 110 124 98 105 98 108 105 111 103 114 97 112 104 121 45 116 101 120 116 124 60 109 97 99 114 111 124 60 108 111 99 97 108 105 122 101 124 66 105 98 108 105 111 103 114 97 112 104 [...cut here...]


Things works well with Guile 1.8, so I do not understand very well what is going on here. Somebody  has some clue as to what could cause this problem?

Thanks!

Masssimiliano Gubinelli

PS: I have the same problem on 2.9.2 (always with autocompilation off).

Re: TeXmacs on Guile 2.2.4

[Mike Gran]

On Sun, May 26, 2019 at 11:17:48PM +0200, Massimiliano Gubinelli wrote:
> Hi all,
> 
>  I???m porting GNU TeXmacs (www.texmacs.org) to Guile 2.2.4 (and
>  hopefully to Guile 3.0). The current status is the
>  following. TeXmacs runs on Guile 2.2.4 with autocompilation turned
>  off: the applications opens and can read and render some
>  files. More work is necessary to switch on compilation. However
>  I???ve a problem with reading some TeXmacs files from the
>  disk. Some of them works OK, but others give errors while reading:
>  the file is first read into a C string by TeXmacs and then
>  converted into a SCM object for further manipulation by Guile. At
>  that point I receive this error message

> Things works well with Guile 1.8, so I do not understand very well what is going on here. Somebody  has some clue as to what could cause this problem?

Hello,

Back in Guile 1.8, it was easy to use strings as ways to read and write
binary data because Guile 1.8 always used 8-bit characters.  It had no
real understanding of UTF-8.  So for 1.8, the concept of strings and
bytevectors could be used more or less interchangeably.

Guile 2.2's string procedures now check that the data the receive
makes sense according to the locale.  You are probably reading in a
file that isn't valid UTF-8 in a context where UTF-8 is expected, if
you locale were something like fr_FR.utf8.  Or maybe you are reading
in a file that isn't pure ASCII when pure ASCII is expected, if your
locale were 'C' or 'POSIX.'

So you have two choices.  You can temporarily set the locale to a
Latin-1 locale, something like 'en_US.iso88591' before you read in the
file, because, Latin-1 is an 8-bit clean locale.  Or, the better idea
is to use a binary port or bytevector to read in your data.

Sorry I don't have time to be more specific.

Regards,
Mike

Re: TeXmacs on Guile 2.2.4

[Massimiliano Gubinelli]

> On 27. May 2019, at 01:49, Mike Gran <spk121@yahoo.com> wrote:
> 
> On Sun, May 26, 2019 at 11:17:48PM +0200, Massimiliano Gubinelli wrote:
>> Hi all,
>> 
>> I???m porting GNU TeXmacs (www.texmacs.org) to Guile 2.2.4 (and
>> hopefully to Guile 3.0). The current status is the...
> 
> Hello,
> 
> Back in Guile 1.8, it was easy to use strings as ways to read and write
> binary data because Guile 1.8 always used 8-bit characters.  It had no
> real understanding of UTF-8.  So for 1.8, the concept of strings and
> bytevectors could be used more or less interchangeably.
> 
> Guile 2.2's string procedures now check that the data the receive
> makes sense according to the locale.  You are probably reading in a
> file that isn't valid UTF-8 in a context where UTF-8 is expected, if
> you locale were something like fr_FR.utf8.  Or maybe you are reading
> in a file that isn't pure ASCII when pure ASCII is expected, if your
> locale were 'C' or 'POSIX.'
> 
> So you have two choices.  You can temporarily set the locale to a
> Latin-1 locale, something like 'en_US.iso88591' before you read in the
> file, because, Latin-1 is an 8-bit clean locale.  Or, the better idea
> is to use a binary port or bytevector to read in your data.
> 
> Sorry I don't have time to be more specific.


Thanks Mike,

  I’m looking right now into the problem taking into account your remarks. While trying some alternative solutions I noticed the following: the function scm_to_latin1_stringn calls out to scm_to_stringn with an argument NULL for the encoding:

    result = scm_to_stringn (str, lenp, NULL,
                             SCM_FAILED_CONVERSION_ERROR);

this causes an error in scm_to_stringn since it contains the check:

   (c_strcasecmp (encoding, "UTF-8") == 0)

and c_strcasecmp does not check for null pointers. Is this a bug? 


Best
Max

> 
> Regards,
> Mike

Re: TeXmacs on Guile 2.2.4

[Mark H Weaver]

Massimiliano Gubinelli <m.gubinelli@gmail.com> writes:

> While trying some alternative solutions I noticed the following: the
> function scm_to_latin1_stringn calls out to scm_to_stringn with an
> argument NULL for the encoding:
>
>     result = scm_to_stringn (str, lenp, NULL,
>                              SCM_FAILED_CONVERSION_ERROR);
>
> this causes an error in scm_to_stringn since it contains the check:
>
>    (c_strcasecmp (encoding, "UTF-8") == 0)
>
> and c_strcasecmp does not check for null pointers. Is this a bug? 

Yes, it is.  I'll fix it soon, thanks.

       Mark