displaying UTF8 characters in and out of a script (guile 2.0.5)

displaying UTF8 characters in and out of a script (guile 2.0.5)

[msematman]

Dear all,

(display "Ćićolina") from the interactive session with the guile
interpreter correctly writes
Ćićolina to the standard output.

However, the same command in a script:

#!/usr/bin/guile \
-s
#!
(display "Ćićolina")

writes this out: ?i?olina

Obviously, there is something wrong with the output of UTF8 characters
in the case of the guile script.

In contrast, a similar bash script on my system (Debian Wheezy
GNU\Linux):

#!/bin/sh
echo "Ćićolina"

writes out the correct output. 

This indicates that there is probably nothing wrong with my locale
setting.

If anybody has any clue what actually went wrong , please let me know.

Cheers





-- 
  
  msematman@myopera.com

Re: displaying UTF8 characters in and out of a script (guile 2.0.5)

[David Pirotte]

Hello,

> However, the same command in a script:
> 
> #!/usr/bin/guile \
> -s
> #!
> (display "Ćićolina")
> 
> writes this out: ?i?olina

you need to set the port encoding first [for further info see section '6.14.1 Ports'
of the manual]:


	#!/bin/sh
	# -*- mode: scheme; coding: utf-8 -*-
	exec guile -e main -s $0 "$@"
	!#

	(define (main args)
	  (let ((port (current-output-port)))
	    (set-port-encoding! port "utf-8")
	    (display "Ćićolina\n" port)))

Cheers,
David

Re: displaying UTF8 characters in and out of a script (guile 2.0.5)

[Mark H Weaver]

David Pirotte <david@altosw.be> writes:

>> However, the same command in a script:
>> 
>> #!/usr/bin/guile \
>> -s
>> #!
>> (display "Ćićolina")
>> 
>> writes this out: ?i?olina
>
> you need to set the port encoding first [for further info see section '6.14.1 Ports'
> of the manual]:

Unfortunately that is only a partial solution (it changes the encoding
of the current-output-port only, not for anything else), and it assumes
that the user wants UTF-8 output.

A more comprehensive solution is to put this:

  (setlocale LC_ALL "")

at the beginning of the script, which will set the locale according to
the user's settings (as specified by the environment variables).  This
is what Guile does when starting an interactive session, and what most
other interactive programs and scripting languages do as well.

Also, since your script contains non-ASCII characters, you should place
a coding declaration in the file so that Guile will know what encoding
to use when reading it.  If your script is in UTF-8, then put this in
the first 500 characters of the file:

  ;;; coding: utf-8

For more details, see:

  http://www.gnu.org/software/guile/manual/html_node/Character-Encoding-of-Source-Files.html

If you do these things, then your script should work properly even when
run by a user who has configured a non-UTF-8 locale.

   Regards,
     Mark

Re: displaying UTF8 characters in and out of a script (guile 2.0.5)

[msematman]

On Sun, Dec 16, 2012, at 09:31 PM, Mark H Weaver wrote:
> David Pirotte <david@altosw.be> writes:
> 
> >> However, the same command in a script:
> >> 
> >> #!/usr/bin/guile \
> >> -s
> >> #!
> >> (display "Ćićolina")
> >> 
> >> writes this out: ?i?olina
> >
> > you need to set the port encoding first [for further info see section '6.14.1 Ports'
> > of the manual]:
> 
> Unfortunately that is only a partial solution (it changes the encoding
> of the current-output-port only, not for anything else), and it assumes
> that the user wants UTF-8 output.
> 
> A more comprehensive solution is to put this:
> 
>   (setlocale LC_ALL "")
> 
> at the beginning of the script, which will set the locale according to
> the user's settings (as specified by the environment variables).  This
> is what Guile does when starting an interactive session, and what most
> other interactive programs and scripting languages do as well.
> 
> Also, since your script contains non-ASCII characters, you should place
> a coding declaration in the file so that Guile will know what encoding
> to use when reading it.  If your script is in UTF-8, then put this in
> the first 500 characters of the file:
> 
>   ;;; coding: utf-8
> 
> For more details, see:
> 
>   http://www.gnu.org/software/guile/manual/html_node/Character-Encoding-of-Source-Files.html
> 
> If you do these things, then your script should work properly even when
> run by a user who has configured a non-UTF-8 locale.
> 
>    Regards,
>      Mark

David, Mark,

Thanks for the detailed explanation, it works now.

Cheers

-- 
  
  msematman@myopera.com

Re: displaying UTF8 characters in and out of a script (guile 2.0.5)

[Andy Wingo]

On Sun 16 Dec 2012 21:31, Mark H Weaver <mhw@netris.org> writes:

> Also, since your script contains non-ASCII characters, you should place
> a coding declaration in the file so that Guile will know what encoding
> to use when reading it.  If your script is in UTF-8, then put this in
> the first 500 characters of the file:
>
>   ;;; coding: utf-8

I think the default is utf-8 now, correct?

Cheers,

Andy
-- 
http://wingolog.org/

Re: displaying UTF8 characters in and out of a script (guile 2.0.5)

[Mark H Weaver]

Andy Wingo <wingo@pobox.com> writes:

> On Sun 16 Dec 2012 21:31, Mark H Weaver <mhw@netris.org> writes:
>
>> Also, since your script contains non-ASCII characters, you should place
>> a coding declaration in the file so that Guile will know what encoding
>> to use when reading it.  If your script is in UTF-8, then put this in
>> the first 500 characters of the file:
>>
>>   ;;; coding: utf-8
>
> I think the default is utf-8 now, correct?

Indeed, you are correct.  I did not know that!

    Thanks,
      Mark

Re: displaying UTF8 characters in and out of a script (guile 2.0.5)

[Ludovic Courtès]

Andy Wingo <wingo@pobox.com> skribis:

> On Sun 16 Dec 2012 21:31, Mark H Weaver <mhw@netris.org> writes:
>
>> Also, since your script contains non-ASCII characters, you should place
>> a coding declaration in the file so that Guile will know what encoding
>> to use when reading it.  If your script is in UTF-8, then put this in
>> the first 500 characters of the file:
>>
>>   ;;; coding: utf-8
>
> I think the default is utf-8 now, correct?

Yes.  I just noticed that it’s only stealthily mentioned under
“Compilation”:

     Each FILE is assumed to be UTF-8-encoded, unless it contains a
     coding declaration as recognized by `file-encoding' (*note
     Character Encoding of Source Files::).

Ludo’.