ruby-mode : ruby-mode-set-encoding

ruby-mode : ruby-mode-set-encoding

[Paul R]

Hello list,

ruby mode is now part of emacs.

There is this particular function :

  (defun ruby-mode-set-encoding ()
    "Insert a magic comment header with the proper encoding if necessary."
    ...

and AFAIK only ruby mode use this type of thing. It is added to saving
functions hooks :

  (add-hook
   (cond ((boundp 'before-save-hook)
          (make-local-variable 'before-save-hook)
          'before-save-hook)
         ((boundp 'write-contents-functions) 'write-contents-functions)
         ((boundp 'write-contents-hooks) 'write-contents-hooks))
   'ruby-mode-set-encoding)

I am not sure what the usecase is for this function but it writes
useless cookies in the headers of my ruby files whenever they contain
something else that ascii chars. Reading the code, it seems that the
definition of "necessary" is not very clear.

Does anybody know why this is necessary please ? If it is really
required, I would suggest to add a customizable variable to disable this
magic behaviour.

thanks,

-- 
  Paul

Re: ruby-mode : ruby-mode-set-encoding

[Leo]

On 2009-03-03 13:46 +0000, Paul R wrote:
>   (defun ruby-mode-set-encoding ()
>     "Insert a magic comment header with the proper encoding if necessary."
>     ...
>
> and AFAIK only ruby mode use this type of thing. It is added to saving
> functions hooks :
>
>   (add-hook
>    (cond ((boundp 'before-save-hook)
>           (make-local-variable 'before-save-hook)
>           'before-save-hook)
>          ((boundp 'write-contents-functions) 'write-contents-functions)
>          ((boundp 'write-contents-hooks) 'write-contents-hooks))
>    'ruby-mode-set-encoding)

I agree. I think this is just too intrusive. Fortunately, I don't use
ruby.

-- 
.:  Leo  :.  [ sdl.web AT gmail.com ]  .: I use Emacs :.

Re: ruby-mode : ruby-mode-set-encoding

[Stefan Monnier]

>> (defun ruby-mode-set-encoding ()
>> "Insert a magic comment header with the proper encoding if necessary."
>> ...
>> 
>> and AFAIK only ruby mode use this type of thing. It is added to saving
>> functions hooks :
>> 
>> (add-hook
>> (cond ((boundp 'before-save-hook)
>> (make-local-variable 'before-save-hook)
>> 'before-save-hook)
>> ((boundp 'write-contents-functions) 'write-contents-functions)
>> ((boundp 'write-contents-hooks) 'write-contents-hooks))
>> 'ruby-mode-set-encoding)

> I agree. I think this is just too intrusive. Fortunately, I don't use ruby.

I don't know if it's too intrusive, but it seems unrelated to Ruby.
Nobuyoshi Nakada, looking at the history of ruby-mode.el at
http://svn.ruby-lang.org/repos/ruby/trunk/misc, it appears the change is
due to you, could you explain what this is intended to do?
If it's not specific to Ruby, then maybe it should be moved elsewhere.


        Stefan

Re: ruby-mode : ruby-mode-set-encoding

[Nobuyoshi Nakada]

Hi,

At Tue, 03 Mar 2009 11:30:51 -0500,
Stefan Monnier wrote:
> I don't know if it's too intrusive, but it seems unrelated to Ruby.
> Nobuyoshi Nakada, looking at the history of ruby-mode.el at
> http://svn.ruby-lang.org/repos/ruby/trunk/misc, it appears the change is
> due to you, could you explain what this is intended to do?
> If it's not specific to Ruby, then maybe it should be moved elsewhere.

It's mandatory for Ruby 1.9 or later.  Ruby 1.9 needs that
"magic comment" to know in what encoding a script is written.
Otherwise, the script is assumed as written with ASCII 7bit
characters only, and non-ASCII character causes an exception if
exists.

Could you elaborate the problem?

-- 
Nobu Nakada

Re: ruby-mode : ruby-mode-set-encoding

[Stefan Monnier]

>> I don't know if it's too intrusive, but it seems unrelated to Ruby.
>> Nobuyoshi Nakada, looking at the history of ruby-mode.el at
>> http://svn.ruby-lang.org/repos/ruby/trunk/misc, it appears the change is
>> due to you, could you explain what this is intended to do?
>> If it's not specific to Ruby, then maybe it should be moved elsewhere.

> It's mandatory for Ruby 1.9 or later.  Ruby 1.9 needs that
> "magic comment" to know in what encoding a script is written.
> Otherwise, the script is assumed as written with ASCII 7bit
> characters only, and non-ASCII character causes an exception if
> exists.

> Could you elaborate the problem?

So you're saying that Ruby uses the same "-*- coding: <foo> -*-" as used
by Emacs?  That makes sense, then, thank you,


        Stefan

Re: ruby-mode : ruby-mode-set-encoding

[Stephen J. Turnbull]

Stefan Monnier writes:

 > So you're saying that Ruby uses the same "-*- coding: <foo> -*-" as used
 > by Emacs?  That makes sense, then, thank you,

As does Python, and it is mandatory in Python for non-ASCII IIRC.  See
PEP 263 for details.  AFAIK Python modes don't try to enforce it
(yet), but they might in the future, for the same reasons that
ruby-mode does.

This should be considered equivalent to an encoding declaration in an
XML fragment.

Re: ruby-mode : ruby-mode-set-encoding

[Stefan Monnier]

>> So you're saying that Ruby uses the same "-*- coding: <foo> -*-" as used
>> by Emacs?  That makes sense, then, thank you,

> As does Python, and it is mandatory in Python for non-ASCII IIRC.  See
> PEP 263 for details.  AFAIK Python modes don't try to enforce it
> (yet), but they might in the future, for the same reasons that
> ruby-mode does.

Interesting.  Why don't they just force the use of some arbitrarily
chosen variant of Unicode?


        Stefan

Re: ruby-mode : ruby-mode-set-encoding

[Stephen J. Turnbull]

Stefan Monnier writes:

 > Interesting.  Why don't they just force the use of some arbitrarily
 > chosen variant of Unicode?

To avoid "arbitrarily" breaking software systems and development
environments that target other character sets.

Specifically, you can automatically add an all-ASCII comment
containing a coding cookie to every Python source file without
breaking anything anywhere.  You cannot simply run all files through
recode, as that will break strings that display on legacy terminals or
have been interned in databases.

Re: ruby-mode : ruby-mode-set-encoding

[Stefan Monnier]

>> Interesting.  Why don't they just force the use of some arbitrarily
>> chosen variant of Unicode?
> To avoid "arbitrarily" breaking software systems and development
> environments that target other character sets.

Sorry, I meant "didn't" instead of "don't": these are recent programming
languages, and since Java was able to do it why couldn't Python do it?

> You cannot simply run all files through recode, as that will break
> strings that display on legacy terminals or have been interned
> in databases.

Nice theoretical problem, but I think that Java proves it to be
a non-issue in practice.  Anyway, it's too late now,


        Stefan

Re: ruby-mode : ruby-mode-set-encoding

[Stephen J. Turnbull]

Stefan Monnier writes:

 > > To avoid "arbitrarily" breaking software systems and development
 > > environments that target other character sets.
 > 
 > Sorry, I meant "didn't" instead of "don't": these are recent programming
 > languages, and since Java was able to do it why couldn't Python do it?

You're showing your age, Stefan, to call these "recent" programming
languages!  And I think your memory may be going, because IIRC Java
did nothing of the kind.  Java supported only Unicode from the get-go
IIRC.  So in that respect Java had no backward compatibility problem.
Python started earlier, when "8-bit clean" code was considered a major
advance and it wasn't clear that Unicode was going to beat ISO 2022.

 > Nice theoretical problem, but I think that Java proves it to be
 > a non-issue in practice.  Anyway, it's too late now,

Actually, the theoretical people (including me) were all in favor of
the One True Encoding approach.  It was the practical people (ie,
people who earn their living programming in Python) who shot it down.

Re: ruby-mode : ruby-mode-set-encoding

[Stefan Monnier]

>> > To avoid "arbitrarily" breaking software systems and development
>> > environments that target other character sets.
>> Sorry, I meant "didn't" instead of "don't": these are recent programming
>> languages, and since Java was able to do it why couldn't Python do it?

> You're showing your age, Stefan, to call these "recent" programming
> languages!  And I think your memory may be going, because IIRC Java
> did nothing of the kind.

My memory isn't completely gone, it seems, since what I meant was indeed:

> Java supported only Unicode from the get-go IIRC.

But indeed, it is on the way out because I thought Java predated Python,
whereas it's the other way around.  This explains that.
OTOH Ruby came out pretty much at the same time as Java, so they clearly
could have done the same as Java, tho its Japanese connection might
explain that Unicode didn't seem like such a great idea to them at
the time.


        Stefan

Re: ruby-mode : ruby-mode-set-encoding

[Paul R]

Hello Nobuyoshi,

Nobuyoshi> It's mandatory for Ruby 1.9 or later. Ruby 1.9 needs that
Nobuyoshi> "magic comment" to know in what encoding a script is written.
Nobuyoshi> Otherwise, the script is assumed as written with ASCII 7bit
Nobuyoshi> characters only, and non-ASCII character causes an exception
Nobuyoshi> if exists.

Fine, I did not relate it to 1.9 (that was recently released) because
I have seen this behaviour for monthes. I think the reason is that I was
using the svn version of ruby-mode.el. After reading the doc, it is
clear and I see indeed why now ruby wants to know file encoding before
parsing it.

Nobuyoshi> Could you elaborate the problem?

The problem simply was that I did not expect this behaviour, and did not
understand the motivation behind it.

Somebody mentionned some similarities with python. Should emacs provide
a standard mechanism to insert this tag for script language modes that
need it ?

Thanks,

-- 
  Paul

Re: ruby-mode : ruby-mode-set-encoding

[Stefan Monnier]

> Somebody mentionned some similarities with python.  Should emacs
> provide a standard mechanism to insert this tag for script language
> modes that need it ?

Don't know about "emacs", but yes, Emacs could provide some help here,
mostly in the form of a function that returns a "good" coding-system
name to put in the tag.


        Stefan

Re: ruby-mode : ruby-mode-set-encoding

[Nobuyoshi Nakada]

Hi,

At Wed, 04 Mar 2009 09:25:24 +0100,
Paul R wrote:
> Somebody mentionned some similarities with python. Should emacs provide
> a standard mechanism to insert this tag for script language modes that
> need it ?

It would be nice, however:

1. Ruby also recognizes Vim style magic comment, such as
   "encoding=utf-8", not only Emacs style.

2. we need ruby-encoding-map because of the difference between
   Ruby's encoding names list and Emacses one.  For instance,
   ruby-mode.el uses CP932 instead of Shift_JIS, since pure
   Shift_JIS is hardly used actually.

-- 
Nobu Nakada