lisp/ChangeLog coding system

lisp/ChangeLog coding system

[Gerd Moellmann]

Michael, somehow you recoded lisp/ChangeLog with your latest change
from its specified coding system iso-2022-7bit to something else.
I've just repaired it...

Re: lisp/ChangeLog coding system

[Richard Stallman]

    Michael, somehow you recoded lisp/ChangeLog with your latest change
    from its specified coding system iso-2022-7bit to something else.

Michael, if you can figure out precisely how this happened, it might
lead us to a way to make the Mule features more robust.  Many users
might benefit from that.  So I think it is worth spending some time to
try to reconstruct how this happened.

Re: lisp/ChangeLog coding system

[Michael Kifer]

>     Michael, somehow you recoded lisp/ChangeLog with your latest change
>     from its specified coding system iso-2022-7bit to something else.
> 
> Michael, if you can figure out precisely how this happened, it might
> lead us to a way to make the Mule features more robust.  Many users
> might benefit from that.  So I think it is worth spending some time to
> try to reconstruct how this happened.

I didn't notice any problems.
However, if there were, it is due to the fact that I had to change X
selection coding system to latin-1, because there is a bug in the default
coding system (which used to be compound-text-with-extensions, but I now
see it no longer is) for X selections. (There was a discussion about this a
week ago and Eli traced it to that X selection coding system bug).

Re: lisp/ChangeLog coding system

[Karl Eichwalder]

Richard Stallman <rms@gnu.org> writes:

> Michael, if you can figure out precisely how this happened, it might
> lead us to a way to make the Mule features more robust.  Many users
> might benefit from that.

Probably the known bug.  The problem is that Emacs sometimes proposes
to use a "save" encoding to save a buffer; if the user follows Emacs'
proposal file corruption starts.

Don't make the user believe the internal Emacs encoding is save.

-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.suse.de/~ke/                                  |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

> From: Karl Eichwalder <ke@gnu.franken.de>
> Date: Sun, 28 Apr 2002 05:06:40 +0200
> 
> Richard Stallman <rms@gnu.org> writes:
> 
> > Michael, if you can figure out precisely how this happened, it might
> > lead us to a way to make the Mule features more robust.  Many users
> > might benefit from that.
> 
> Probably the known bug.  The problem is that Emacs sometimes proposes
> to use a "save" encoding to save a buffer; if the user follows Emacs'
> proposal file corruption starts.

If you can show examples of such corruption, please do.  I'm not aware
of any corruption due to encodings Emacs suggests, although it's true
that the result might surprise sometimes, especially if the user
doesn't know what she is doing.

However, surprise and corruption are two different things...

Re: lisp/ChangeLog coding system

[Stefan Monnier]

> > From: Karl Eichwalder <ke@gnu.franken.de>
> > Date: Sun, 28 Apr 2002 05:06:40 +0200
> > 
> > Richard Stallman <rms@gnu.org> writes:
> > 
> > > Michael, if you can figure out precisely how this happened, it might
> > > lead us to a way to make the Mule features more robust.  Many users
> > > might benefit from that.
> > 
> > Probably the known bug.  The problem is that Emacs sometimes proposes
> > to use a "save" encoding to save a buffer; if the user follows Emacs'
> > proposal file corruption starts.
> 
> If you can show examples of such corruption, please do.  I'm not aware
> of any corruption due to encodings Emacs suggests, although it's true
> that the result might surprise sometimes, especially if the user
> doesn't know what she is doing.

He might be referring to the fact that saving a file with encoding
foo when the first line says "-*- coding: bar -*-" will lead to
corruption.
And more generally if the auto-detection of the coding-system does not
agree with the coding-system used when saving the file.

But of the course the first question (the one RMS asked IIUC) is why
did Emacs not use iso-2202 despite the -*- coding: -*- tag.


	Stefan

Re: lisp/ChangeLog coding system

[Stephen J. Turnbull]

>>>>> "Eli" == Eli Zaretskii <eliz@is.elta.co.il> writes:

    Eli> However, surprise and corruption are two different things...

That's right.  The user can recover trivially from a "surprise"
without help; otherwise it's corruption.

I don't think it's our call to make.  And I think it should be
reasonably easy to protect users substantially better than Emacs
currently does.  (And XEmacs is worse.)

One aspect is making better guesses about desired coding systems.
Another is getting the trash coding systems out of the prompt to the
user.  Eg, I doubt most users _ever_ want to use the -with-esc coding
systems.  (As far as I can tell X Compound Text should serve the
purpose fine, and users can tell for sure that they don't know what it
does.  The ISO-8859-with-esc are tempting for naive users as "the
closest to what I want".)  A sample implementation of these aspects
(for XEmacs only) is latin-unity:

cvs -d pserver:cvs@cvs.xemacs.org:/pack/xemacscvs login  # password "cvs"
cvs -d pserver:cvs@cvs.xemacs.org:/pack/xemacscvs checkout latin-unity

The README describes the features pretty completely.  The prompt
(which the user sees only if one of the user's preferred coding
systems doesn't handle the buffer) is in the function
latin-unity-recommend-representation in file latin-unity.el.  The
user's preferences are captured with the defcustoms in that file.

The two big problems with this package (to the author's mind ;-) are
that it doesn't handle anything but ISO Latin yet, and it doesn't work
on GNU Emacs.  The beta testers seem happy with the functionality.
The UI needs tuning, of course.

Sorry, I don't have time to port to Emacs anytime soon.  However, I'll
be happy to answer questions (cc me, I don't read emacs-devel every
day) and help with factoring out interesting functionality on request.

This is all-new code, contributed to XEmacs, and I have assign.future
on file for XEmacs.  Ie, it's clean for use in GNU Emacs.


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
 My nostalgia for Icon makes me forget about any of the bad things.  I don't
have much nostalgia for Perl, so its faults I remember.  Scott Gilbert c.l.py

Re: lisp/ChangeLog coding system

[Stefan Monnier]

> One aspect is making better guesses about desired coding systems.

I'm not sure what kind of improvements you're thinking about.

> Another is getting the trash coding systems out of the prompt to the user.

I agree that the more "esoteric" systems should be made less
visible.  They are already pushed to the end of the list, but
the list is long (because it always includes the "catch-all"
systems like emacs-mule, iso-2022 and iso8859-with-esc) and looks
just very confusing.  We should probably try to split it into
"likely" and "unlikely" options.  Where the cut-off point should be,
I don't know (non-MIME coding-systems should be in the "unlikely"
list, tho).

>  Eg, I doubt most users _ever_ want to use the -with-esc coding
> systems.  (As far as I can tell X Compound Text should serve the
> purpose fine, and users can tell for sure that they don't know what it
> does.  The ISO-8859-with-esc are tempting for naive users as "the
> closest to what I want").

I don't even know why those systems exist.

> A sample implementation of these aspects (for XEmacs only) is latin-unity:

I'm not sure in which sense this is related.

> The two big problems with this package (to the author's mind ;-) are
> that it doesn't handle anything but ISO Latin yet, and it doesn't work
> on GNU Emacs.  The beta testers seem happy with the functionality.
> The UI needs tuning, of course.

Looking at the README, I have the impression that most of the functionality
is already part of the Emacs CVS code (mostly thanks to Dave's
ucs-tables.el).  Someone should try and figure out the details.


	Stefan

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On Sat, 27 Apr 2002, Michael Kifer wrote:

> However, if there were, it is due to the fact that I had to change X
> selection coding system to latin-1, because there is a bug in the default
> coding system (which used to be compound-text-with-extensions, but I now
> see it no longer is) for X selections.

I doubt that X selection encoding can have any effect on how files are 
encoded.

Gerd, can you tell what exactly was wrong with the ChageLog before you 
fixed it?

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On Sun, 28 Apr 2002, Stefan Monnier wrote:

> He might be referring to the fact that saving a file with encoding
> foo when the first line says "-*- coding: bar -*-" will lead to
> corruption.

FWIW, this is on my TODO, but it's been there for the last year...

> But of the course the first question (the one RMS asked IIUC) is why
> did Emacs not use iso-2202 despite the -*- coding: -*- tag.

Yes, I agree.

Re: lisp/ChangeLog coding system

[Richard Stallman]

    Probably the known bug.  The problem is that Emacs sometimes proposes
    to use a "save" encoding to save a buffer; if the user follows Emacs'
    proposal file corruption starts.

Sorry, I do not understand what scenario you are criticizing.
Could you say it more clearly?

Re: lisp/ChangeLog coding system

[Richard Stallman]

    >     Michael, somehow you recoded lisp/ChangeLog with your latest change
    >     from its specified coding system iso-2022-7bit to something else.
    > 
    > Michael, if you can figure out precisely how this happened, it might
    > lead us to a way to make the Mule features more robust.  Many users
    > might benefit from that.  So I think it is worth spending some time to
    > try to reconstruct how this happened.

    I didn't notice any problems.

The problem was not one you would have noticed within that session.
The problem was that the file was stored in a different coding system.
It may not have caused inconvenience for you, but it did for others.
So we want to track it down.

Can you please help?

    However, if there were, it is due to the fact that I had to change X
    selection coding system to latin-1, because there is a bug in the default
    coding system (which used to be compound-text-with-extensions, 

I don't think that specifying an X selection coding system should have
had this result unless you copied the whole contents of ChangeLog through
the X server.

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On Sun, 28 Apr 2002, Stefan Monnier wrote:

> We should probably try to split it into
> "likely" and "unlikely" options.  Where the cut-off point should be,
> I don't know

The language environment might bring this information somehow.

> (non-MIME coding-systems should be in the "unlikely" list, tho).

Careful: this might not be true on Windows (cpNNN etc.).

> >  Eg, I doubt most users _ever_ want to use the -with-esc coding
> > systems.  (As far as I can tell X Compound Text should serve the
> > purpose fine, and users can tell for sure that they don't know what it
> > does.  The ISO-8859-with-esc are tempting for naive users as "the
> > closest to what I want").
> 
> I don't even know why those systems exist.

Try to force Emacs to encode Latin-2 text as iso8859-1 (yes, this does 
work!), and you will understand, I think ;-)

Re: lisp/ChangeLog coding system

[Michael Kifer]

>     >     Michael, somehow you recoded lisp/ChangeLog with your latest change
>     >     from its specified coding system iso-2022-7bit to something else.
>     > 
>     > Michael, if you can figure out precisely how this happened, it might
>     > lead us to a way to make the Mule features more robust.  Many users
>     > might benefit from that.  So I think it is worth spending some time to
>     > try to reconstruct how this happened.
> 
>     I didn't notice any problems.
> 
> The problem was not one you would have noticed within that session.
> The problem was that the file was stored in a different coding system.
> It may not have caused inconvenience for you, but it did for others.
> So we want to track it down.
> 
> Can you please help?
> 
>     However, if there were, it is due to the fact that I had to change X
>     selection coding system to latin-1, because there is a bug in the default
>     coding system (which used to be compound-text-with-extensions, 
> 
> I don't think that specifying an X selection coding system should have
> had this result unless you copied the whole contents of ChangeLog through
> the X server.

well, the X selection thingie is the only thing that I can think of which
somehow might be related to the coding system. I just created a record in
my own change log and then copy-pasted it to the emacs changelog.
All within the same session.

I normally don't fool around with the coding system. But because there is
some kind of a bug with the default coding system for X selections, I
changed that one.  Can't think of anything else.

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On Mon, 29 Apr 2002, Michael Kifer wrote:

> I just created a record in
> my own change log and then copy-pasted it to the emacs changelog.
> All within the same session.

If you did that in the same Emacs session, X selections should have not 
been involved: Emacs yanks its own selections directly from the kill 
ring, IIRC.

Re: lisp/ChangeLog coding system

[Gerd Moellmann]

Eli Zaretskii <eliz@is.elta.co.il> writes:

> Gerd, can you tell what exactly was wrong with the ChageLog before you 
> fixed it?

Inserting a new change log entry and trying to save the file lead to
the ``choose coding system'' dialog.

The value of buffer-file-coding-system was iso-2022-7bit, and
find-charset-region on the whole buffer returned `(ascii
eight-bit-graphic)', I think.

The change in question is the one from version 1.3763 to 1.3764.

I repaired the file by getting 1.3763, adding Michaels change log
entry, etc.

Re: lisp/ChangeLog coding system

[Stephen J. Turnbull]

>>>>> "Stefan" == Stefan Monnier <monnier+gnu/emacs@rum.cs.yale.edu> writes:

    >> One aspect is making better guesses about desired coding
    >> systems.

    Stefan> I'm not sure what kind of improvements you're thinking
    Stefan> about.

Well, in the version (mid-January, maybe?) of GNU Emacs I have, when I
tried saving a buffer with mixed ascii, latin-1, and latin-2 in it, it
gave me an abominably long list of coding systems including mule
internal, all the -with-esc systems, and iso-2022-jp-2.  But all of
the characters used in the buffer are in ISO-8859-2, it's just Mule
making false distinctions.

At the very least, the defaults in Emacs should be to identify
identical characters (eg, those from the Latin-## subsets) and to
distinguish those where unification is controversial (the Han
ideographs).

    Stefan> non-MIME coding-systems should be in the "unlikely" list, tho.

There is no unique "the unlikely list".

For example, if I were Croatian, I probably would want the buffer
described above saved in ISO-8859-2 without being asked, but a German
would probably want to save it in UTF-8 (or maybe ISO-2022-7 if she
were an Emacs developer), or be queried, defaulting to ISO-8859-2.
And some of the "universal" coding systems (UTF-32, mule internal, all
the -with-esc systems) should probably not even be offered to most
users; they should have to ask for them by name.  But people with
special needs should be able to configure them for regular use.

And what's a "non-MIME coding system"?  AFAIK MIME has nothing to do with
coding systems except that the notation "the preferred MIME name" is a
useful convention.  But KOI8-R and all the Windows-125x sets are MIME
registered.

    Stefan> Looking at the README, I have the impression that most of
    Stefan> the functionality is already part of the Emacs CVS code
    Stefan> (mostly thanks to Dave's ucs-tables.el).  Someone should
    Stefan> try and figure out the details.

As for most functionality being in Emacs, yes, that's why I said I'd
help refactor; relative to ucs-tables.el the contribution is all UI.
My duplication[1] of ucs-tables is straightforward, not terribly
efficient code; all the meat is devoted to the question of "how do we
know which coding systems to offer the user".  Specifically I address
the issues of preferred unibyte systems and preferred universal
systems described above.


Footnotes: 
[1]  XEmacs 21.5 has built-in support for Unicode.  The UCS tables are
loaded at startup from (a local copy of) the Unicode Consortium
tables, and an API is provided to reload if desirable.  The code
predates the release of Emacs 21, and so is different from
ucs-tables.el, unfortunately.  The duplicative parts are for 21.4.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
 My nostalgia for Icon makes me forget about any of the bad things.  I don't
have much nostalgia for Perl, so its faults I remember.  Scott Gilbert c.l.py

Re: lisp/ChangeLog coding system

[Miles Bader]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
> At the very least, the defaults in Emacs should be to identify
> identical characters (eg, those from the Latin-## subsets)

If you turn on `unify-8859-on-encoding-mode', it will do that for the
latin-x stuff.

>     Stefan> non-MIME coding-systems should be in the "unlikely" list, tho.
> 
> There is no unique "the unlikely list".

I think what Stefan was suggesting was to call the tail-end of the
`likely' list (e.g. the list used for decoding priority) `unlikely'.
These are language-environment specific.

-Miles
-- 
Yo mama's so fat when she gets on an elevator it HAS to go down.

Re: lisp/ChangeLog coding system

[Stefan Monnier]

> >>>>> "Stefan" == Stefan Monnier <monnier+gnu/emacs@rum.cs.yale.edu> writes:
> 
>     >> One aspect is making better guesses about desired coding
>     >> systems.
> 
>     Stefan> I'm not sure what kind of improvements you're thinking
>     Stefan> about.
> 
> Well, in the version (mid-January, maybe?) of GNU Emacs I have, when I
> tried saving a buffer with mixed ascii, latin-1, and latin-2 in it, it
> gave me an abominably long list of coding systems including mule
> internal, all the -with-esc systems, and iso-2022-jp-2.  But all of
> the characters used in the buffer are in ISO-8859-2, it's just Mule
> making false distinctions.
> 
> At the very least, the defaults in Emacs should be to identify
> identical characters (eg, those from the Latin-## subsets) and to
> distinguish those where unification is controversial (the Han
> ideographs).

As Miles said, you can call (ucs-unify-8859 t) to unify the latin
charsets when saving a file.  Better yet: you don't need to do it any
more because it's now the default behavior.

>     Stefan> non-MIME coding-systems should be in the "unlikely" list, tho.
> 
> There is no unique "the unlikely list".
> 
> For example, if I were Croatian, I probably would want the buffer
> described above saved in ISO-8859-2 without being asked,

That's what happens right now.

> but a German
> would probably want to save it in UTF-8 (or maybe ISO-2022-7 if she
> were an Emacs developer), or be queried, defaulting to ISO-8859-2.

A German would get a prompt where iso-8859-2 and utf-8 should
be near the beginning of the list of coding-systems she can
choose from.

> And what's a "non-MIME coding system"?

I don't know.  All I know is that it's used when sorting coding-systems
so that the list has more "likely" coding systems at the beginning.
I think it's also used to choose more user-friendly names when a coding-system
has several aliases (such as mule-utf-8 and utf-8.  Don't ask me why the
coding-system is called `mule-utf-8' and why `utf-8' is only an alias, tho).

> As for most functionality being in Emacs, yes, that's why I said I'd
> help refactor; relative to ucs-tables.el the contribution is all UI.
> My duplication[1] of ucs-tables is straightforward, not terribly
> efficient code; all the meat is devoted to the question of "how do we
> know which coding systems to offer the user".  Specifically I address
> the issues of preferred unibyte systems and preferred universal
> systems described above.

I'm beginning to understand, thank you,


	Stefan

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On 29 Apr 2002, Gerd Moellmann wrote:

> Eli Zaretskii <eliz@is.elta.co.il> writes:
> 
> > Gerd, can you tell what exactly was wrong with the ChageLog before you 
> > fixed it?
> 
> Inserting a new change log entry and trying to save the file lead to
> the ``choose coding system'' dialog.
> 
> The value of buffer-file-coding-system was iso-2022-7bit, and
> find-charset-region on the whole buffer returned `(ascii
> eight-bit-graphic)', I think.
> 
> The change in question is the one from version 1.3763 to 1.3764.

Thanks for the info.  As far as I could see, version 1.3764 is encoded in 
Latin-1, not iso-2022-7bit.  But since there's a ``coding: iso-2022-7bit''
cookie there, buffer-file-coding-system gets set to iso-2022-7bit, and 
Emacs then pops the select-safe-coding-system dialog when you try to save.

The question is how did the file get saved on Michael's machine as 
Latin-1.  Ideas, anyone?

I think we should change save-buffer to pay attention to the coding: 
cookie.  This was discussed long ago (more than a year, IIRC), but never 
got coded.  It's a bad mantra, IMHO, that Emacs produces a file whose 
coding: lies through its teeth.

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On 29 Apr 2002, Stephen J. Turnbull wrote:

> And what's a "non-MIME coding system"?

Those that don't have a mime-charset property.  Or, put it in other 
words, those which don't correspond to some MIME charset.

> But KOI8-R and all the Windows-125x sets are MIME registered.

However, some of the more esoteric coding systems, those which usually 
scare users, don't have a mime-charset property.

Re: lisp/ChangeLog coding system

[Karl Eichwalder]

Richard Stallman <rms@gnu.org> writes:

> Sorry, I do not understand what scenario you are criticizing.

Eli and Stefan know about it (the coding cookie trap).

> Could you say it more clearly?

Stephen explained pretty good what I've had in mind -- I can't do it
better than he already did.  As long as Emacs doesn't ask the user in
case of mismatching "codings" something like:

    Mismatching codings (iso-8859-1, iso-8859-2, and iso-8859-15)
    occured.

    Do you want to unify these codings (default "yes")?

And then an additional dialog:

    ... unify as utf-8 or iso-8859-2?

these corruptions will happen again and again.  I strongly vote to
release Dave Love's unification fixes ASAP as Emacs 21.3.

-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.suse.de/~ke/                                  |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)

Re: lisp/ChangeLog coding system

[Stephen J. Turnbull]

>>>>> "Stefan" == Stefan Monnier <monnier+gnu/emacs@rum.cs.yale.edu> writes:

    Stefan> Better yet: you don't need to do it any more because it's
    Stefan> now the default behavior.

OK, so I was working on the basis of (for this purpose) a prehistoric
version.  Sorry.

    [Stefan and Miles explain language environment dependence]

OK.  I personally have never much liked language environments, but
that could very well be because I've never seen one that worked as you
describe.  ;-)  I'll build a more recent CVS and try that for a while,
since the description sounds quite close to what I'd want, except for
some basically cosmetic details.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
 My nostalgia for Icon makes me forget about any of the bad things.  I don't
have much nostalgia for Perl, so its faults I remember.  Scott Gilbert c.l.py

Re: lisp/ChangeLog coding system

[Karl Eichwalder]

"Stefan Monnier" <monnier+gnu/emacs@RUM.cs.yale.edu> writes:

> As Miles said, you can call (ucs-unify-8859 t) to unify the latin
> charsets when saving a file.  Better yet: you don't need to do it any
> more because it's now the default behavior.

Good.  Let's release it as Emacs 21.3 (or 22.x).

-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.suse.de/~ke/                                  |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)

Re: lisp/ChangeLog coding system

[Stefan Monnier]

> > Sorry, I do not understand what scenario you are criticizing.
> Eli and Stefan know about it (the coding cookie trap).

Hmm... so it was indeed what you were talking about.
Just for the record: it was anything but obvious.


	Stefan

Re: lisp/ChangeLog coding system

[Miles Bader]

"Stefan Monnier" <monnier+gnu/emacs@RUM.cs.yale.edu> writes:
> As Miles said, you can call (ucs-unify-8859 t) to unify the latin
> charsets when saving a file.  Better yet: you don't need to do it any
> more because it's now the default behavior.

Is it?  `unify-8859-on-encoding-mode' seems to be nil in my emacs
(which customize says is the default value)...

-Miles
-- 
97% of everything is grunge

Re: lisp/ChangeLog coding system

[Richard Stallman]

    He might be referring to the fact that saving a file with encoding
    foo when the first line says "-*- coding: bar -*-" will lead to
    corruption.

Could you describe the complete scenario in which this might happen?
If the user explicitly asks for it, it might be a mistake but it isn't
Emacs's fault.  If it happens in some other scenario, perhaps Emacs
needs to be changed.

    But of the course the first question (the one RMS asked IIUC) is why
    did Emacs not use iso-2202 despite the -*- coding: -*- tag.

I did not have anything that specific in mind.
I'm just trying to get to the bottom of the situation.

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

> From: Karl Eichwalder <ke@gnu.franken.de>
> Date: Mon, 29 Apr 2002 17:52:45 +0200
> 
> I strongly vote to release Dave Love's unification fixes ASAP as
> Emacs 21.3.

Emacs 21.3 will be released from the branch, so it cannot have Dave's
code.  This will have to wait until the first release from the current
CVS head.

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

> From: "Stefan Monnier" <monnier+gnu/emacs@RUM.cs.yale.edu>
> Date: Mon, 29 Apr 2002 12:37:45 -0400
> 
> > > Sorry, I do not understand what scenario you are criticizing.
> > Eli and Stefan know about it (the coding cookie trap).
> 
> Hmm... so it was indeed what you were talking about.
> Just for the record: it was anything but obvious.

Indeed.  FWIW, when Stefan mentioned this possibility I wondered how
did he make that leap of logic.

Re: lisp/ChangeLog coding system

[Simon Josefsson]

"Eli Zaretskii" <eliz@is.elta.co.il> writes:

>> From: Karl Eichwalder <ke@gnu.franken.de>
>> Date: Mon, 29 Apr 2002 17:52:45 +0200
>> 
>> I strongly vote to release Dave Love's unification fixes ASAP as
>> Emacs 21.3.
>
> Emacs 21.3 will be released from the branch, so it cannot have Dave's
> code.  This will have to wait until the first release from the current
> CVS head.

If this is so, shouldn't the ":version" `defcustom' parameters I just
proposed in the patch to lisp/mail/smtpmail.el and
lisp/mail/sendmail.el be changed to "21.4" or whatever the next
version will be that will include the fixes?  The :version parameter
seem to require that there exists a release plan for future releases
so it is possible to add the correct information.

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On Mon, 29 Apr 2002, Simon Josefsson wrote:

> > Emacs 21.3 will be released from the branch, so it cannot have Dave's
> > code.  This will have to wait until the first release from the current
> > CVS head.
> 
> If this is so, shouldn't the ":version" `defcustom' parameters I just
> proposed in the patch to lisp/mail/smtpmail.el and
> lisp/mail/sendmail.el be changed to "21.4" or whatever the next
> version will be that will include the fixes?

Yes.

> The :version parameter
> seem to require that there exists a release plan for future releases
> so it is possible to add the correct information.

The trouble with release plans is that they tend to change ;-)

Seriously, though: we are just learning how to deal with branches.  I 
guess this issue is part of the learning process.  Suggestions for 
solutions are welcome.

Re: lisp/ChangeLog coding system

[Richard Stallman]

    > I just created a record in
    > my own change log and then copy-pasted it to the emacs changelog.
    > All within the same session.

    If you did that in the same Emacs session, X selections should have not 
    been involved: Emacs yanks its own selections directly from the kill 
    ring, IIRC.

And that by itself should not have affected the rest of the file.
But here's an idea for what might have happened.  If the text that
was yanked used some other character set that looks similar
on the screen, that could have affected how the whole file
got saved afterwards.

Re: lisp/ChangeLog coding system

[Richard Stallman]

    I think we should change save-buffer to pay attention to the coding: 
    cookie.

It is supposed to do that indirectly because buffer-file-coding-system
should get set initially based on the `coding:' specification.
If something legitimately sets buffer-file-coding-system after that,
I think it ought to be obeyed, not ignored.  Don't you?

The question for me is, what changed buffer-file-coding-system to
latin-1?

Re: lisp/ChangeLog coding system

[Richard Stallman]

Can you try to reproduce the failure
so we can figure out what caused it?

Re: lisp/ChangeLog coding system

[Richard Stallman]

    Stephen explained pretty good what I've had in mind -- I can't do it
    better than he already did.

Can someone show me that particular message?  I will then study it
carefully (which I can't afford to do with all the mail on this list
because it is too much for me).

      As long as Emacs doesn't ask the user in
    case of mismatching "codings" something like:

	Mismatching codings (iso-8859-1, iso-8859-2, and iso-8859-15)
	occured.

	Do you want to unify these codings (default "yes")?

Unfortunately I can't make head or tail of this
because I don't understand what the basic issue is.

Re: lisp/ChangeLog coding system

[Michael Kifer]

> Can you try to reproduce the failure
> so we can figure out what caused it?

I really don't know what might have caused it.
As far as I remember all I did was to type in one buffer and then
copy-paste in another. This doesn't cause any problems as far as I can
tell. 
What was the exact symptom of that problem?
Any weird symbols when you visit Changelog?
Does anybody remember what was the exact version# of Changelog that got
screwed up with that coding system?

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On Mon, 29 Apr 2002, Richard Stallman wrote:

>     I think we should change save-buffer to pay attention to the coding: 
>     cookie.
> 
> It is supposed to do that indirectly because buffer-file-coding-system
> should get set initially based on the `coding:' specification.
> If something legitimately sets buffer-file-coding-system after that,
> I think it ought to be obeyed, not ignored.

Yes, but if buffer-file-coding-system is changed, the coding: cookie 
should be rewritten to reflect that, probably after warning the user 
about the presence of the cookie and asking for her permission to modify 
it.

This is what I meant by ``pay attention''.

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On Tue, 30 Apr 2002, Michael Kifer wrote:

> What was the exact symptom of that problem?

The symptoms were that ChangeLog got recoded in Latin-1, but the coding: 
cookie still said iso-2022-7bit.  That's a recipe for trouble.

Re: lisp/ChangeLog coding system

[Michael Kifer]

> On Tue, 30 Apr 2002, Michael Kifer wrote:
> 
> > What was the exact symptom of that problem?
> 
> The symptoms were that ChangeLog got recoded in Latin-1, but the coding: 
> cookie still said iso-2022-7bit.  That's a recipe for trouble.


This confirms my suspicion that this has something to do with me setting
the selection coding system to Latin-1.
But I can't recall anything else that might be relevant.

Re: lisp/ChangeLog coding system

[Simon Josefsson]

On Tue, 30 Apr 2002, Eli Zaretskii wrote:

> On Mon, 29 Apr 2002, Simon Josefsson wrote:
> 
> > > Emacs 21.3 will be released from the branch, so it cannot have Dave's
> > > code.  This will have to wait until the first release from the current
> > > CVS head.
> > 
> > If this is so, shouldn't the ":version" `defcustom' parameters I just
> > proposed in the patch to lisp/mail/smtpmail.el and
> > lisp/mail/sendmail.el be changed to "21.4" or whatever the next
> > version will be that will include the fixes?
> 
> Yes.

Could some change it?  Thanks.

> > The :version parameter
> > seem to require that there exists a release plan for future releases
> > so it is possible to add the correct information.
> 
> The trouble with release plans is that they tend to change ;-)
>
> Seriously, though: we are just learning how to deal with branches.  I 
> guess this issue is part of the learning process.  Suggestions for 
> solutions are welcome.

The only solution I can think of is that :version contains the alpha
release version in CVS, and whenever someone changes the release version
he has to replace the version in the elisp as well.  It could be done by a
script.  But I'm not sure it is an ideal solution.

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On Tue, 30 Apr 2002, Michael Kifer wrote:

> > The symptoms were that ChangeLog got recoded in Latin-1, but the coding: 
> > cookie still said iso-2022-7bit.  That's a recipe for trouble.
> 
> This confirms my suspicion that this has something to do with me setting
> the selection coding system to Latin-1.

Is it possible that you've pasted the new entries from outside of your 
Emacs session, or from another application?

Re: lisp/ChangeLog coding system

[Michael Kifer]

> On Tue, 30 Apr 2002, Michael Kifer wrote:
> 
> > > The symptoms were that ChangeLog got recoded in Latin-1, but the coding: 
> > > cookie still said iso-2022-7bit.  That's a recipe for trouble.
> > 
> > This confirms my suspicion that this has something to do with me setting
> > the selection coding system to Latin-1.
> 
> Is it possible that you've pasted the new entries from outside of your 
> Emacs session, or from another application?

I don't think so.

Re: lisp/ChangeLog coding system

[Karl Eichwalder]

Richard Stallman <rms@gnu.org> writes:

> Can someone show me that particular message?

I'll forward it separately.

-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.suse.de/~ke/                                  |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)

Re: lisp/ChangeLog coding system

[Richard Stallman]

    Yes, but if buffer-file-coding-system is changed, the coding: cookie 
    should be rewritten to reflect that, probably after warning the user 
    about the presence of the cookie and asking for her permission to modify 
    it.

    This is what I meant by ``pay attention''.

Actually changing the file seems rather drastic.
I think we should start with what Dave Love just sent us,
which is just to ask for extra confirmation.

Re: lisp/ChangeLog coding system

[Eli Zaretskii]

On Wed, 1 May 2002, Richard Stallman wrote:

> Actually changing the file seems rather drastic.
> I think we should start with what Dave Love just sent us,
> which is just to ask for extra confirmation.

I don't have Dave's changes handy to look--did they actually allow to 
produce a file with "coding: foo" and encoded in something that isn't 
`foo'?  If so, I think it's not a good idea to produce such a file, as it 
almost certainly will create problems when it is visited.