Compilation to native

Compilation to native

[Matthew Mundell]

>From etc/TODO:

    * Investigate using GNU Lightning or similar system for incremental
      compilation of selected bytecode functions to subrs.  Converting CCL
      programs to native code is probably the first thing to try, though.

Is there any work towards this?

Re: Compilation to native

[Matthew Mundell]

Matthew Mundell <matt@mundell.ukfsn.org> writes:

> >From etc/TODO:
>
>     * Investigate using GNU Lightning or similar system for incremental
>       compilation of selected bytecode functions to subrs.  Converting CCL
>       programs to native code is probably the first thing to try, though.
>
> Is there any work towards this?

Well, I have a start at generating native code from byte code at run
time, and invoking the code produced.  It uses the GNU Lightning
macros.  The generation is done in a modified copy of Fbyte_code,
called Fnative_compile_byte_code, by generating the code for each byte
code operation instead of performing the operation.

Enough is implemented to run the example function from the "Speed of
Byte-Code" node in the Elisp manual.  The function is:

(defun silly-loop (n)
  "Return time before and after N iterations of a loop."
  (let ((t1 (current-time-string)))
    (while (> (setq n (1- n))
			  0))
	(list t1 (current-time-string))))

Here is a set of results:

(silly-loop 100000000)
=> ("Sat Feb 28 10:04:39 2004" "Sat Feb 28 10:05:30 2004")     ; 51 secs

(byte-compile 'silly-loop)
(silly-loop 100000000)
=> ("Sat Feb 28 10:06:37 2004" "Sat Feb 28 10:06:53 2004")     ; 16 secs

(native-compile 'silly-loop)
(silly-loop 100000000)
=> ("Sat Feb 28 10:17:13 2004" "Sat Feb 28 10:17:22 2004")     ; 9 secs


The changes come to more than 25000 characters, so they're here:
    http://www.mundell.ukfsn.org/native/bytecode.c.diff
    http://www.mundell.ukfsn.org/native/eval.c.diff
    http://www.mundell.ukfsn.org/native/lisp.h.diff
Alternatively, the full files are available:
    http://www.mundell.ukfsn.org/native/bytecode.c
    http://www.mundell.ukfsn.org/native/eval.c
    http://www.mundell.ukfsn.org/native/lisp.h
Either way, this file is also required:
    http://www.mundell.ukfsn.org/native/native.el

The changes require NO_UNION_TYPE to be set, USE_LSB_TAG to be clear,
and EMACS_INT to be an int.  For now a fixed amount of memory is
allocated for the generated code.


Is this effort good enough to continue?

Re: Compilation to native

[Richard Stallman]

    (byte-compile 'silly-loop)
    (silly-loop 100000000)
    => ("Sat Feb 28 10:06:37 2004" "Sat Feb 28 10:06:53 2004")     ; 16 secs

    (native-compile 'silly-loop)
    (silly-loop 100000000)
    => ("Sat Feb 28 10:17:13 2004" "Sat Feb 28 10:17:22 2004")     ; 9 secs

I don't think that a speedup of less than a factor of 2 would be worth
installing something that might take substantial maintenance effort.

However, it could be that this test is a bad test and doesn't really
show the benefit of compilation.  Maybe this loop spends most of the
time inside current-time-string.  How about trying something purely
computational?

    The changes require NO_UNION_TYPE to be set, USE_LSB_TAG to be clear,
    and EMACS_INT to be an int.

It will be necessary to remove these restrictions to make it ready to
install.  However, there's no harm working initially on this case
and handling other cases later.

Re: Compilation to native

[Juri Linkov]

Richard Stallman <rms@gnu.org> writes:
> I don't think that a speedup of less than a factor of 2 would be worth
> installing something that might take substantial maintenance effort.

Hmm, I expected more speedup given that it compiles to machine code.

Anyhow, while jit compilation of Emacs bytecode to native code would
be a far-reaching goal, the most urgent issue is with CCL programs.
Currently, CCL programs are unbearable slow.  For example, opening
a 10MB UTF-8 file on a fast machine takes 5 min, while opening it
without conversion is performed instantly.

I thought about a different approach: predefined CCL programs
could be statically converted into C code and compiled by a C
compiler into Emacs core.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Compilation to native

[Stefan Monnier]

>> I don't think that a speedup of less than a factor of 2 would be worth
>> installing something that might take substantial maintenance effort.
> Hmm, I expected more speedup given that it compiles to machine code.

Maybe Richard is right, but I wouldn't be surprised if we can't go much
faster than that.  A lot of time is spent in C primitives including
binding variables and looking them up.  Also a lot of the time spent in
function call is cleverly spread and duplicated between the caller and the
callee which are at two different places, so there's a lot of opportunities
for optimization but it's very difficult to do it at all, let
alone do it automatically.

> Anyhow, while jit compilation of Emacs bytecode to native code would
> be a far-reaching goal, the most urgent issue is with CCL programs.
> Currently, CCL programs are unbearable slow.  For example, opening
> a 10MB UTF-8 file on a fast machine takes 5 min, while opening it
> without conversion is performed instantly.

And compiling CCL to native code is likely to bring much more impressive
speed ups because less time is spent in C primitives.


        Stefan

Re: Compilation to native

[Juri Linkov]

> Maybe Richard is right, but I wouldn't be surprised if we can't go much
> faster than that.  A lot of time is spent in C primitives including
> binding variables and looking them up.  Also a lot of the time spent in
> function call is cleverly spread and duplicated between the caller and the
> callee which are at two different places, so there's a lot of opportunities
> for optimization but it's very difficult to do it at all, let
> alone do it automatically.

Yes, this would require implementing an optimizing Lisp compiler
which is a tremendous task.

> And compiling CCL to native code is likely to bring much more impressive
> speed ups because less time is spent in C primitives.

Since CCL is a much simpler and quite independent language
implementing its compilation is more manageable task and it should
considerably improve the performance of code conversion which
currently is very poor.

One of the main questions that should be decided now is how to
implement CCL compilation: dynamically with the help of GNU Lightning,
or statically by generating C code which will be compiled by a C
compiler into the Emacs executable.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Compilation to native

[Matthew Mundell]

Juri Linkov <juri@jurta.org> writes:

> > Maybe Richard is right, but I wouldn't be surprised if we can't go much
> > faster than that.  A lot of time is spent in C primitives including
> > binding variables and looking them up.  Also a lot of the time spent in
> > function call is cleverly spread and duplicated between the caller and the
> > callee which are at two different places, so there's a lot of opportunities
> > for optimization but it's very difficult to do it at all, let
> > alone do it automatically.
>
> Yes, this would require implementing an optimizing Lisp compiler
> which is a tremendous task.

The byte compiler already does some optimisation.  Surely the
combination of byte compiler and compilation to native heads towards
an optimising Lisp compiler?

Re: Compilation to native

[Miles Bader]

On Wed, Mar 24, 2004 at 11:50:39PM +0000, Matthew Mundell wrote:
> > Yes, this would require implementing an optimizing Lisp compiler
> > which is a tremendous task.
> 
> The byte compiler already does some optimisation.  Surely the
> combination of byte compiler and compilation to native heads towards
> an optimising Lisp compiler?

I don't think the current byte compiler would be a practical base from which
to build a real optimizing compiler, it's a huge mess.

[And of course, for an optimizing compiler, you _really_ want to use lexical
binding, so help me with my emacs--lexbind branch!]

-Miles
-- 
I'm beginning to think that life is just one long Yoko Ono album; no rhyme
or reason, just a lot of incoherent shrieks and then it's over.  --Ian Wolff

Re: Compilation to native

[Matthew Mundell]

Juri Linkov <juri@jurta.org> writes:

> Richard Stallman <rms@gnu.org> writes:
> > I don't think that a speedup of less than a factor of 2 would be worth
> > installing something that might take substantial maintenance effort.
>
> Hmm, I expected more speedup given that it compiles to machine code.

I expected less, given that the generated code does mostly what the
compiled byte interpreter does, saving only on the time taken to loop
through the byte code.

> Anyhow, while jit compilation of Emacs bytecode to native code would
> be a far-reaching goal, the most urgent issue is with CCL programs.
> Currently, CCL programs are unbearable slow.  For example, opening
> a 10MB UTF-8 file on a fast machine takes 5 min, while opening it
> without conversion is performed instantly.

I only wondered why the TODO entry favoured CCL, as it was simpler to
try the byte code first.  The changes to function calling required for
the compiled byte code would count towards a similar effort for CCL.

Re: Compilation to native

[Juri Linkov]

> I only wondered why the TODO entry favoured CCL, as it was simpler to
> try the byte code first.  The changes to function calling required for
> the compiled byte code would count towards a similar effort for CCL.

As far as this TODO entry speaks about compilation, it is much
simpler to compile CCL than Emacs Lisp.  And AFAIU, it suggests
first to try compiling CCL bytecode to native code as opposed
to compiling Emacs Lisp bytecode to native code.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Compilation to native

[Matthew Mundell]

Juri Linkov <juri@jurta.org> writes:

> > I only wondered why the TODO entry favoured CCL, as it was simpler to
> > try the byte code first.  The changes to function calling required for
> > the compiled byte code would count towards a similar effort for CCL.
>
> As far as this TODO entry speaks about compilation, it is much
> simpler to compile CCL than Emacs Lisp.

Modifying the byte interpreter offered a simpler way to get going.

> And AFAIU, it suggests first to try compiling CCL bytecode to native
> code as opposed to compiling Emacs Lisp bytecode to native code.

It does: "Converting CCL programs to native code is probably the first
thing to try, though."

Re: Compilation to native

[Richard Stallman]

    I thought about a different approach: predefined CCL programs
    could be statically converted into C code and compiled by a C
    compiler into Emacs core.

It is worth a try.  On the other hand, I have to wonder
if the CCL interpreter could be faster.

Re: Compilation to native

[Kenichi Handa]

In article <E1B5q0C-00039m-FD@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:

>     I thought about a different approach: predefined CCL programs
>     could be statically converted into C code and compiled by a C
>     compiler into Emacs core.

> It is worth a try.  On the other hand, I have to wonder
> if the CCL interpreter could be faster.

I think there's not much room of improvement in the CCL
interpreter (i.e. the function ccl_driver).

In addtion, in emacs-unicode, CCL is, by default, used only
for Ethiopic font encoding, and it can easily be changed not
to use CCL.

So, I think it's not that worth working on CCL interpreter.

---
Ken'ichi HANDA
handa@m17n.org

Re: Compilation to native

[David Kastrup]

Kenichi Handa <handa@m17n.org> writes:

> In article <E1B5q0C-00039m-FD@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:
> 
> >     I thought about a different approach: predefined CCL programs
> >     could be statically converted into C code and compiled by a C
> >     compiler into Emacs core.
> 
> > It is worth a try.  On the other hand, I have to wonder
> > if the CCL interpreter could be faster.
> 
> I think there's not much room of improvement in the CCL
> interpreter (i.e. the function ccl_driver).
> 
> In addtion, in emacs-unicode, CCL is, by default, used only
> for Ethiopic font encoding, and it can easily be changed not
> to use CCL.
> 
> So, I think it's not that worth working on CCL interpreter.

If we have a development plan to switch to emacs-unicode soon.  We
really need to get a grip about what should be in the next feature
release.

If the unicode and bidi branches are considerable usable, what are we
waiting for?

We are in the situation that currently for many purposes one has to
tell people "try using CVS".  People get more and more to rely on it
for daily work.  This situation is unhealthy.  If things like
emacs-unicode and emacs-bidi are expected to cause longer-lasting
trouble, then we should crank out something like a full-featured 21.5
or so just before merging them.  If the merge phase leads to longer
problems, we at least have a somewhat stable release to refer people
to while we are sorting the problems out.

If, on the other hand, users and developers of the unicode and bidi
branches are confident enough that under _normal_ use (i.e., if one
does not use bidi texts) stability should not be affected much, then
I'd say "what the heck, give it to us".  Maintaining separate branches
for longer always leads to merging headaches.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Compilation to native

[Kenichi Handa]

In article <x5lll8554h.fsf@lola.goethe.zz>, David Kastrup <dak@gnu.org> writes:

> Kenichi Handa <handa@m17n.org> writes:
>>  In article <E1B5q0C-00039m-FD@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:
>>  
>>  >     I thought about a different approach: predefined CCL programs
>>  >     could be statically converted into C code and compiled by a C
>>  >     compiler into Emacs core.
>>  
>>  > It is worth a try.  On the other hand, I have to wonder
>>  > if the CCL interpreter could be faster.
>>  
>>  I think there's not much room of improvement in the CCL
>>  interpreter (i.e. the function ccl_driver).
>>  
>>  In addtion, in emacs-unicode, CCL is, by default, used only
>>  for Ethiopic font encoding, and it can easily be changed not
>>  to use CCL.
>>  
>>  So, I think it's not that worth working on CCL interpreter.

> If we have a development plan to switch to emacs-unicode soon.  We
> really need to get a grip about what should be in the next feature
> release.

> If the unicode and bidi branches are considerable usable, what are we
> waiting for?

I think Unicode branch is fairly usable in normal use.  At
least it's stable enough for my daily work.  But if it is
used with third party packages, I think some of them must be
adjusted for emacs-unicode.

Bidi branch is far from usable.  I created that branch
mainly for that the other people can contribute.  It's very
difficult to find a time to work on it for me.

> We are in the situation that currently for many purposes one has to
> tell people "try using CVS".  People get more and more to rely on it
> for daily work.  This situation is unhealthy.  If things like
> emacs-unicode and emacs-bidi are expected to cause longer-lasting
> trouble, then we should crank out something like a full-featured 21.5
> or so just before merging them.  If the merge phase leads to longer
> problems, we at least have a somewhat stable release to refer people
> to while we are sorting the problems out.

> If, on the other hand, users and developers of the unicode and bidi
> branches are confident enough that under _normal_ use (i.e., if one
> does not use bidi texts) stability should not be affected much, then
> I'd say "what the heck, give it to us".  Maintaining separate branches
> for longer always leads to merging headaches.

I fully agree.

When I synchronized emacs-unicode branch to HEAD a half year
ago, it took about 10 days concentrated work.  I think it
will need the same amount of work to merge emacs-unicode to
HEAD.  If Richard says "go ahead", I'll manage to make that
time.

---
Ken'ichi HANDA
handa@m17n.org

Re: Compilation to native

[Alex Schroeder]

Kenichi Handa <handa@m17n.org> writes:

> I think Unicode branch is fairly usable in normal use.  At
> least it's stable enough for my daily work.  But if it is
> used with third party packages, I think some of them must be
> adjusted for emacs-unicode.

I used HEAD for several months, and have since switched to the unicode
branch for several months.  I still experience a crash or two a month,
which is bad, but I cannot say I noticed any difference between the
two.

As for third party packages, I'm using the unicode branch with Gnus
and ERC (IRC client) for heavy duty work every day, so clearly I think
the Unicode branch is ready for merging, feature freeze, testing, and
release as soon as possible.

I must confess, however, that I have hoped all this time that the bidi
branch would be merged into the unicode branch, and that we'd release
the stuff as Emacs 22...  Even fickle Arabic and Hebrew support is
better than no support at all...  And for all it's worth, I've used
the bidi branch for IRC as well!  :)

Alex.
-- 
.O.  http://www.emacswiki.org/alex/
..O  Schroeder's fourth law:
OOO  None of your friends and coworkers share your taste in music.

Re: Compilation to native

[Kenichi Handa]

In article <87zn9nqras.fsf@emacswiki.org>, Alex Schroeder <alex@emacswiki.org> writes:
> I used HEAD for several months, and have since switched to the unicode
> branch for several months.  I still experience a crash or two a month,
> which is bad, but I cannot say I noticed any difference between the
> two.

Could you please report the detail of the crash?  Aren't
there any backtrace information?

> As for third party packages, I'm using the unicode branch with Gnus
> and ERC (IRC client) for heavy duty work every day, so clearly I think
> the Unicode branch is ready for merging, feature freeze, testing, and
> release as soon as possible.

> I must confess, however, that I have hoped all this time that the bidi
> branch would be merged into the unicode branch, and that we'd release
> the stuff as Emacs 22...  Even fickle Arabic and Hebrew support is
> better than no support at all...  And for all it's worth, I've used
> the bidi branch for IRC as well!  :)

When I set enable-bidi-display to t in Emacs of emacs-bidi
branch, it crashes as soon as it tries to display Hebrew or
Arabic.  I have not yet been able to find a time to
investigate it.  :-(

---
Ken'ichi HANDA
handa@m17n.org

It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Alex Schroeder <alex@emacswiki.org> writes:

> Kenichi Handa <handa@m17n.org> writes:
> 
> > I think Unicode branch is fairly usable in normal use.  At
> > least it's stable enough for my daily work.  But if it is
> > used with third party packages, I think some of them must be
> > adjusted for emacs-unicode.
> 
> I used HEAD for several months, and have since switched to the unicode
> branch for several months.  I still experience a crash or two a month,
> which is bad, but I cannot say I noticed any difference between the
> two.
> 
> I must confess, however, that I have hoped all this time that the bidi
> branch would be merged into the unicode branch, and that we'd release
> the stuff as Emacs 22...  Even fickle Arabic and Hebrew support is
> better than no support at all...  And for all it's worth, I've used
> the bidi branch for IRC as well!  :)

I also think that unicode and bidi belongs in 22.1 => which is why
they should NOT be merged at this time.

I suggest that we make a complete feature freeze on HEAD for 21.4 NOW !!!

Thus, we allow just bug fixes, and minor adjustments to new features
(in case current interface/implementation is sub-optimal or unclean).

Eg. if there are still some issues with the new compile.el, do 
whatever is necessary to fix them.

New packages that we have already decided to add may also still go in.

And of course, we should get the remaining documentation tasks done
(getting new stuff in NEWS into the manuals).

Besides that, our efforts should concentrate on ironing out the (few?)
remaning unsolved reports about crashes in CVS head.

We should aim at doing the first pre-test in approx. 1 month (no later
than June 1st).

WHY?

Current HEAD is rock-solid for normal use, and I don't think it will
can be much better in that respect (but we should try!).  I suppose we
have all used it in our daily work for [too] many months already, and
so have numerous anonymous users who need - or just like - the new
features of 21.4.

Also, people (notably Luc) have spent quite some time proof-reading
the current documentation on HEAD, so docs are also in a pretty
consistent state.


Consequently, we should be able to have an expedited pre-test phase,
as many parts of the code has already received extensive testing by a
much broader audience than any previous pre-test versions of emacs.


If we don't do it now, and start merging unicode and bidi to HEAD, I
fear that a 22.1 release is at least 1 year further into the future --
and that CVS HEAD will go through a phase of lesser stable code than
we have been used to for quite some time (I'm not judging the quality
of the code on those branches, it's just that it hasn't been tested to
the same extent as HEAD, so we should expect problems).


It's time!!!

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <m3lll7b3j0.fsf_-_@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:
> I suggest that we make a complete feature freeze on HEAD for 21.4 NOW !!!

I have thought that we are going to take the following
schedule.

(1) Release the current RC branch as 21.4.
(2) Feature freeze on HEAD.
(3) Merge HEAD into RC branch, and make it 21.4.50.
(4) Merge Unicode branch into HEAD, and make it 22.0.0.
(5) Start pretest of 21.5 based on RC.
(6) Release RC branch as 21.5.
...
(7) Feature freeze on HEAD (i.e. Unicode version)
(8) Merge HEAD into RC branch, and make it 22.0.50.

Do you intend to skip (1)?  Perhaps, that will be better
because we can avoid disappointing users who exepect new
features in the release of this long delay.

> If we don't do it now, and start merging unicode and bidi to HEAD, I
> fear that a 22.1 release is at least 1 year further into the future --
> and that CVS HEAD will go through a phase of lesser stable code than
> we have been used to for quite some time (I'm not judging the quality
> of the code on those branches, it's just that it hasn't been tested to
> the same extent as HEAD, so we should expect problems).

I agree.  But, I think bug-fixing work for 21.5 (or 21.4)
must be done on RC branch so that (4) can be done promptly
and we can have more users of Unicode version, which boosts
the release of 22.1.

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[John Wiegley]

Kenichi Handa <handa@m17n.org> writes:

> I have thought that we are going to take the following schedule.
>
> (1) Release the current RC branch as 21.4.

I would note too that I'm currently trying to identify a frequent
crash that occurs on Mac OS/X (1-4 times daily).  This is a new
breakage, as earlier versions of Emacs were quite stable.

I'm still watching for another crash, to find if it's all the same
source.  The last one was at:

  SetupOffscreenGDevice()
  PortToNQDPixMap()
  PortToNQDPixMap()
  StdText()
  XDrawString (display=0xbfffd6a4, w=0xffffffff, gc=0xbfffd69c, x=1, y=-1441792, buf=0xffea0000 <Address 0xffea0000 out of bounds>, nchars=25559642) at macterm.c:779
  779       mac_draw_string_common (display, w, gc, x, y, buf, nchars, srcOr, 1);

It seems to happen more frequently if I have more than one frame open.

John

Re: It is time for a feature freeze (it is NOW or never).

[YAMAMOTO Mitsuharu]

>>>>> On Wed, 07 Apr 2004 20:13:04 -0700, John Wiegley <johnw@gnu.org> said:

> I would note too that I'm currently trying to identify a frequent
> crash that occurs on Mac OS/X (1-4 times daily).  This is a new
> breakage, as earlier versions of Emacs were quite stable.

I guess some of them are related to repeated C-g.  Is that right?  I
think mac_check_for_quit_char (in macterm.c) needs some changes so
that it reflects the recent changes on kbd_buffer_store_event_hold (in
keyboard.c).

> I'm still watching for another crash, to find if it's all the same
> source.  The last one was at:

>  XDrawString (display=0xbfffd6a4, w=0xffffffff, gc=0xbfffd69c, x=1, y=-1441792, buf=0xffea0000 <Address 0xffea0000 out of bounds>, nchars=25559642) at macterm.c:779
>  779       mac_draw_string_common (display, w, gc, x, y, buf, nchars, srcOr, 1);

Seems like you compiled with `-O2 -g'.  In my experience, gdb in Mac
OS X sometimes shows wrong values about function arguments if emacs is
compiled with optimization options.  Maybe you can obtain more correct
information if compiled without `-O2'.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: It is time for a feature freeze (it is NOW or never).

[John Wiegley]

YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

> I guess some of them are related to repeated C-g.  Is that right?  I
> think mac_check_for_quit_char (in macterm.c) needs some changes so
> that it reflects the recent changes on kbd_buffer_store_event_hold (in
> keyboard.c).

No C-g pressing, I don't recall...  I think it happened after waking
up the screen.  Hasn't recurred today, although somebody on #emacs
triggered the same bug.  He thought it might be a font issue.

John

Re: It is time for a feature freeze (it is NOW or never).

[Piet van Oostrum]

>>>>> YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> (YM) wrote:

>>>>> On Wed, 07 Apr 2004 20:13:04 -0700, John Wiegley <johnw@gnu.org> said:
>> I would note too that I'm currently trying to identify a frequent
>> crash that occurs on Mac OS/X (1-4 times daily).  This is a new
>> breakage, as earlier versions of Emacs were quite stable.

YM> I guess some of them are related to repeated C-g.  Is that right?  I
YM> think mac_check_for_quit_char (in macterm.c) needs some changes so
YM> that it reflects the recent changes on kbd_buffer_store_event_hold (in
YM> keyboard.c).

The crashes on my machine are (almost) invariably tied to the following
sequence:

I am reading news/mail with gnus.
I switch a couple of times between emacs and other applications
I go back to gnus and continue reading or request new news with the 'g'
command in *Groups*.
Emacs hangs
I try to get it out of the hang by hitting C-g once or several times.
Crash.

These crashes happen sometimes several times a day.

It was my impression that it could be related to something like screen
redrawing or redrawing the menu, that corrupts some memory or doesn't
cooperate properly with GC.
-- 
Piet van Oostrum <piet@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP]
Private email: P.van.Oostrum@hccnet.nl

Re: It is time for a feature freeze (it is NOW or never).

[Jason Rumney]

Kenichi Handa wrote:

> I have thought that we are going to take the following
> schedule.
> 
> (1) Release the current RC branch as 21.4.

That was discussed some weeks ago, but no pretest has begun yet, so 
maybe it is better to skip this if it is going to take a long time anyway.

> (2) Feature freeze on HEAD.
> (3) Merge HEAD into RC branch, and make it 21.4.50.

I think making a new branch would be easier than merging. I doubt there 
are important bugfixes in the RC branch that are not already in HEAD.

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Jason Rumney <jasonr@gnu.org> writes:

> Kenichi Handa wrote:
> 
> > I have thought that we are going to take the following
> > schedule.
> > (1) Release the current RC branch as 21.4.
> 
> That was discussed some weeks ago, but no pretest has begun yet, so
> maybe it is better to skip this if it is going to take a long time
> anyway.

I would say that depends on how long we expect the 21.5 pretest to take.
If THAT takes a long time, let's go for a 21.4 pretest now, and release
quickly if the pretest doesn't reveal anything major (compared to 21.3).

> 
> > (2) Feature freeze on HEAD.
> > (3) Merge HEAD into RC branch, and make it 21.4.50.
> 
> I think making a new branch would be easier than merging. I doubt
> there are important bugfixes in the RC branch that are not already in
> HEAD.

I agree that we should leave the old RC branch as is (ending with 21.4)
and create a new RC branch for 21.5 _asap_!!.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Kenichi Handa <handa@m17n.org> writes:

> In article <m3lll7b3j0.fsf_-_@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:
> > I suggest that we make a complete feature freeze on HEAD for 21.4 NOW !!!
> 
> I have thought that we are going to take the following
> schedule.
> 
> (1) Release the current RC branch as 21.4.

I have no opinion on this.

> (2) Feature freeze on HEAD.

Yes.

> (3) Merge HEAD into RC branch, and make it 21.4.50.

There is no need to merge to the existing RC branch.

We just create a new RC branch from HEAD for 21.5 named EMACS_21_5_RC.

> (4) Merge Unicode branch into HEAD, and make it 22.0.0.

Yes.

I suggest we also merge the multi-tty branch to HEAD quickly.


> (5) Start pretest of 21.5 based on RC.

(With RC = 21_5_RC):

Fix all 21.4 => 21.5 references on RC.
Then start pretest from RC.

> (6) Release RC branch as 21.5.

Yes.

> ...
> (7) Feature freeze on HEAD (i.e. Unicode version)

I suppose that bidi support is not mature enough for 22.1 --

actually IMO, bidi support would warrent another major number,
i.e. 23.1.

> (8) Merge HEAD into RC branch, and make it 22.0.50.

No, again just make a 22_1_RC branch from HEAD.

> 
> Do you intend to skip (1)?  Perhaps, that will be better
> because we can avoid disappointing users who exepect new
> features in the release of this long delay.

Since there is some confusion as to what 21.4 is or would be, I
suggest we aim for using 21.5 as the version for the release from
the new RC branch.   If things take too long, we still have the option
to release 21.4 from the old RC branch.


> 
> > If we don't do it now, and start merging unicode and bidi to HEAD, I
> > fear that a 22.1 release is at least 1 year further into the future --
> > and that CVS HEAD will go through a phase of lesser stable code than
> > we have been used to for quite some time (I'm not judging the quality
> > of the code on those branches, it's just that it hasn't been tested to
> > the same extent as HEAD, so we should expect problems).
> 
> I agree.  But, I think bug-fixing work for 21.5 (or 21.4)
> must be done on RC branch so that (4) can be done promptly
> and we can have more users of Unicode version, which boosts
> the release of 22.1.

Yes, lets branch now for 21.5 and fix bugs on the RC branch.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

storm@cua.dk (Kim F. Storm) writes:

> Kenichi Handa <handa@m17n.org> writes:
> 
> > In article <m3lll7b3j0.fsf_-_@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:
> > > I suggest that we make a complete feature freeze on HEAD for 21.4 NOW !!!
> > 
> > I have thought that we are going to take the following
> > schedule.
> > 
> > (1) Release the current RC branch as 21.4.
> 
> I have no opinion on this.

I also don't know how important that is.  I don't know the fixes as
opposed to 21.3.

But it _is_ important that we get the four years of development since
21.0 feature freeze out of the door sometimes.  People increasingly
switch to Emacs CVS since that is the only sane option to do, and that
means that they have to use something with a stability completely
based on timing and good luck.  If we don't release from HEAD before
potentially large merges, they will be screwed without an alternative.

> > (2) Feature freeze on HEAD.
> 
> Yes.
> 
> > (3) Merge HEAD into RC branch, and make it 21.4.50.
> 
> There is no need to merge to the existing RC branch.

Right.

> We just create a new RC branch from HEAD for 21.5 named
> EMACS_21_5_RC.
> 
> > (4) Merge Unicode branch into HEAD, and make it 22.0.0.
> 
> Yes.

Whatever.

> I suggest we also merge the multi-tty branch to HEAD quickly.

After 21_5_RC has been split off.  If it turns out that post-release
changes leave HEAD stable enough, we can still surprise the world by
following up with 22.0 on short notice.  But at the current point of
time, the tester base for the branches is not so large that we should
take the risk.

> > (7) Feature freeze on HEAD (i.e. Unicode version)
> 
> I suppose that bidi support is not mature enough for 22.1 --
> 
> actually IMO, bidi support would warrent another major number,
> i.e. 23.1.

Actually, I'd want to see both.  Anyhow, after the unicode merge, the
bidi branch will need to synch up, and one can then see how it fares.

Unicode makes somewhat more sense with bidi (and I want to be able to
offer Emacs at one time for creation of philological texts, and
vocalized Hebrew is not exactly irrelevant in that respect).  But if
it is apparent that 22.0 would be help up by it considerably, I'd say
to go without it.

> Since there is some confusion as to what 21.4 is or would be, I
> suggest we aim for using 21.5 as the version for the release from
> the new RC branch.  If things take too long, we still have the
> option to release 21.4 from the old RC branch.

Sounds reasonable.  Also, if we happen to skip a number, people will
notice that there is some larger change.

> > I agree.  But, I think bug-fixing work for 21.5 (or 21.4)
> > must be done on RC branch so that (4) can be done promptly
> > and we can have more users of Unicode version, which boosts
> > the release of 22.1.
> 
> Yes, lets branch now for 21.5 and fix bugs on the RC branch.

Sounds reasonable.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

> (1) Release the current RC branch as 21.4.

I don't consider it very useful, but if people want to spend their time on
it I can't stop them.

> (2) Feature freeze on HEAD.
> (3) Merge HEAD into RC branch, and make it 21.4.50.
> (4) Merge Unicode branch into HEAD, and make it 22.0.0.
> (5) Start pretest of 21.5 based on RC.
> (6) Release RC branch as 21.5.
> ...
> (7) Feature freeze on HEAD (i.e. Unicode version)
> (8) Merge HEAD into RC branch, and make it 22.0.50.

I'd offer a very slightly different schedule:
(2) Feature freeze on the trunk NOW
(3) Start pretest from the trunk (maybe after fixing known bugs, but there
    doesn't seem to be many of those, so it seems unnecessary).
(4) When bug-fixing starts to slow down such that some developers feel bored,
    make a new RC branch and move the bug-fixing and pretesting there.
    Those people who are bored can start playing on the trunk again.
(5a) Finish bug-fixing RC and finally Release.
(5b) Merge unicode, multi-tty, and bidi (in this order, but quickly) to the
    trunk.  Branches basically don't get any testing and waste too much of
    our energy with merging, so we should only keep them for code that's
    really unstable or that we don't know whether we'll ever include.
    E.g. if bidi only crashed when you set enable-bidi-display, then it's
    ready for inclusion.
    I'd also like to see the emacs-xft (anti-aliasing) merged in, but AFAIK
    it's not ready yet.
(6) Almost immediately after that: goto number (2).

Note that the plan above tries to reduce the amount of "parallel
development" for two reasons:
- we need to make sure people focus on fixing bugs during the pretest.
- we don't have the manpower to work and test many branches at the same time,
  let alone keep them uptodate w.r.t the trunk.

> Do you intend to skip (1)?  Perhaps, that will be better
> because we can avoid disappointing users who exepect new
> features in the release of this long delay.

Agreed.

> I agree.  But, I think bug-fixing work for 21.5 (or 21.4)
> must be done on RC branch so that (4) can be done promptly
> and we can have more users of Unicode version, which boosts
> the release of 22.1.

You have a point but I think we should only do it when the bug-fixing
frenzy slows down (hopefully this will happen very quickly).
But of course, as soon as the bugs left over are "none of your business",
you'd better start merging rather than twiddle your thumbs.


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> (5b) Merge unicode, multi-tty, and bidi (in this order, but quickly) to the
>     trunk.

I'd like to (post-release) merge my tiling branch too; I don't seen any
particularly good reason to keep it separate.  I think the only probable
conflict is with multi-tty (at the least because of RIF changes).

-Miles
-- 
"I distrust a research person who is always obviously busy on a task."
   --Robert Frosch, VP, GM Research

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Miles Bader <miles@lsi.nec.co.jp> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> > (5b) Merge unicode, multi-tty, and bidi (in this order, but quickly) to the
> >     trunk.
> 
> I'd like to (post-release) merge my tiling branch too; I don't seen any
> particularly good reason to keep it separate.  I think the only probable
> conflict is with multi-tty (at the least because of RIF changes).

That is ok with me -- IMO, it would be nice to get all feature
branches merged to the trunk soon after branching for 21.5 so we can
get everything (and everybody) synchronized.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

On Sat, Apr 10, 2004 at 01:43:32AM +0200, Kim F. Storm wrote:
> IMO, it would be nice to get all feature branches merged to the trunk soon
> after branching for 21.5 so we can get everything (and everybody)
> synchronized.

There's also the `lexbind' branch

It seems to work OK in normal use (i.e. it doesn't seem to adversely affect
the normal case of dynamic binding), but until recently I haven't used it for
everyday use; I'm now running the "miles" branch which is the sum of the
tiling and lexbind branches, so hopefully if there are any problems I'll see
them.

-Miles
-- 
Quidquid latine dictum sit, altum viditur.

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

I wrote:
> I'd like to (post-release) merge my tiling branch too; I don't seen any
> particularly good reason to keep it separate.  I think the only probable
> conflict is with multi-tty (at the least because of RIF changes).

I did a test merge of multi-tty with tiling, and there are actually very few
patch conflicts, though the RIF changes probably do mean there are few
semantic errors in the result.

I'd love see the multi-tty changes go in -- not just because it's a great
feature (and it is), but because I like the code cleanup.  Compared to the
current "everything global all the time" code, it should make things a bit
easier to understand, which is no small thing given the confusingness of the
redisplay code.

-Miles
-- 
`There are more things in heaven and earth, Horatio,
 Than are dreamt of in your philosophy.'

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Miles Bader <miles@gnu.org> writes:

> I wrote:
> > I'd like to (post-release) merge my tiling branch too; I don't seen any
> > particularly good reason to keep it separate.  I think the only probable
> > conflict is with multi-tty (at the least because of RIF changes).
> 
> I did a test merge of multi-tty with tiling, and there are actually very few
> patch conflicts, though the RIF changes probably do mean there are few
> semantic errors in the result.
> 
> I'd love see the multi-tty changes go in -- not just because it's a great
> feature (and it is), but because I like the code cleanup.  Compared to the
> current "everything global all the time" code, it should make things a bit
> easier to understand, which is no small thing given the confusingness of the
> redisplay code.

Yes, I also look forward to the RIF cleanup from the multi-tty patch.
I had plans to do this cleanup myself sometime, but Romain beat me to it.

It should definitely go in soon after 21.5 is branched from trunk.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Romain Francoise]

storm@cua.dk (Kim F. Storm) writes:

> Yes, I also look forward to the RIF cleanup from the multi-tty patch.
> I had plans to do this cleanup myself sometime, but Romain beat me to it.

Just to give credit where it's due, the person behind the multi-tty
branch is Károly Lorentey, not myself.  I'm just a convinced and
sometimes vocal user.

-- 
Romain Francoise <romain@orebokech.com> | It was fourteen degrees below
it's a miracle -- http://orebokech.com/ | on a screeching march 23.

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Romain Francoise <romain@orebokech.com> writes:

> storm@cua.dk (Kim F. Storm) writes:
> 
> > Yes, I also look forward to the RIF cleanup from the multi-tty patch.
> > I had plans to do this cleanup myself sometime, but Romain beat me to it.
> 
> Just to give credit where it's due, the person behind the multi-tty
> branch is Károly Lorentey, not myself.  I'm just a convinced and
> sometimes vocal user.

Sorry for the mixup.  So it was Károly who beat me to it :-)

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <m365c7vg5v.fsf@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:

>>  I did a test merge of multi-tty with tiling, and there are actually very few
>>  patch conflicts, though the RIF changes probably do mean there are few
>>  semantic errors in the result.
>>  
>>  I'd love see the multi-tty changes go in -- not just because it's a great
>>  feature (and it is), but because I like the code cleanup.  Compared to the
>>  current "everything global all the time" code, it should make things a bit
>>  easier to understand, which is no small thing given the confusingness of the
>>  redisplay code.

> Yes, I also look forward to the RIF cleanup from the multi-tty patch.
> I had plans to do this cleanup myself sometime, but Romain beat me to it.

> It should definitely go in soon after 21.5 is branched from trunk.

I don't object to it, but if it changes some internal
interfaces, I'd like to ask to merge it after emacs-unicode
merge is done.  At least emacs-unicode doesn't change RIF.

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

Kenichi Handa <handa@m17n.org> writes:

> In article <m365c7vg5v.fsf@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:
> 
> >>  I did a test merge of multi-tty with tiling, and there are
> >>  actually very few patch conflicts, though the RIF changes
> >>  probably do mean there are few semantic errors in the result.
> >>  
> >>  I'd love see the multi-tty changes go in -- not just because
> >>  it's a great feature (and it is), but because I like the code
> >>  cleanup.  Compared to the current "everything global all the
> >>  time" code, it should make things a bit easier to understand,
> >>  which is no small thing given the confusingness of the redisplay
> >>  code.
> 
> > Yes, I also look forward to the RIF cleanup from the multi-tty
> > patch.  I had plans to do this cleanup myself sometime, but Romain
> > beat me to it.
> 
> > It should definitely go in soon after 21.5 is branched from trunk.
> 
> I don't object to it, but if it changes some internal interfaces,
> I'd like to ask to merge it after emacs-unicode merge is done.  At
> least emacs-unicode doesn't change RIF.

In my opinion, we should first merge the unicode branch into the
trunk.  Once unicode is merged, multi-tty can merge the current
trunk.  Once this is done, one can judge the respective stabilities.

If it turns out that the unicode branch merge left us with a rather
stable system on the trunk, we should aim for releasing 22.1 as fast
as possible.  This means that multi-tty should only be merged if its
HEAD (after merge in of the trunk's HEAD) seems similarly stable.

Since we can't just right now judge how the stability is affected
with the combined branches, I would postpone the decision of when to
merge multi-tty until we have the actual merge candidates.

When in doubt, there is no harm in releasing 22.0, noticing
afterwards that the multi-head branch leaves us with a quite stable
Emacs, too, and release 22.1 soon after that.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Lőrentey Károly]

Kenichi Handa <handa@m17n.org> writes:
> In article <m365c7vg5v.fsf@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:
>>>  I did a test merge of multi-tty with tiling, and there are actually very few
>>>  patch conflicts, though the RIF changes probably do mean there are few
>>>  semantic errors in the result.
>>>  
>>>  I'd love see the multi-tty changes go in -- not just because it's a great
>>>  feature (and it is), but because I like the code cleanup.  Compared to the
>>>  current "everything global all the time" code, it should make things a bit
>>>  easier to understand, which is no small thing given the confusingness of the
>>>  redisplay code.
>
>> Yes, I also look forward to the RIF cleanup from the multi-tty patch.
>> I had plans to do this cleanup myself sometime, but Romain beat me to it.
>
>> It should definitely go in soon after 21.5 is branched from trunk.
>
> I don't object to it, but if it changes some internal
> interfaces, I'd like to ask to merge it after emacs-unicode
> merge is done.  At least emacs-unicode doesn't change RIF.

That's OK with me; my branch follows HEAD, and I will happily merge in
the unicode changes as soon as they appear there.  (I looked at the
unicode branch a few months ago and it seemed the merge could be done
without too much work.)

-- 
Károly

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

    I'd like to (post-release) merge my tiling branch too; I don't seen any
    particularly good reason to keep it separate.

I seem to recall that code had serious flaws and that it was not clear
whether they could be fixed at all.  I don't remember the details
after all this time, but we cannot schedule installation ofn this code.

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

On Sat, Apr 10, 2004 at 10:34:55PM -0400, Richard Stallman wrote:
>     I'd like to (post-release) merge my tiling branch too; I don't seen any
>     particularly good reason to keep it separate.
> 
> I seem to recall that code had serious flaws and that it was not clear
> whether they could be fixed at all.

If so, nobody told _me_ about them...

-Miles
-- 
In New York, most people don't have cars, so if you want to kill a person, you
have to take the subway to their house.  And sometimes on the way, the train
is delayed and you get impatient, so you have to kill someone on the subway.
  [George Carlin]

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

On Sun, Apr 11, 2004 at 05:06:05AM -0400, Miles Bader wrote:
> On Sat, Apr 10, 2004 at 10:34:55PM -0400, Richard Stallman wrote:
> >     I'd like to (post-release) merge my tiling branch too; I don't seen any
> >     particularly good reason to keep it separate.
> > 
> > I seem to recall that code had serious flaws and that it was not clear
> > whether they could be fixed at all.
> 
> If so, nobody told _me_ about them...

To restate that in a less glib manner, to the best of my knowledge, my tiling
changes have never been reviewed by anybody, and such discussion as has
occurred has mostly been off on various tangents, not about the core
functionality.

For instance Dave Love commented that would nice if a background image could
be synchronized with the text (so it acted like "paper"), as that's what web
browsers do, and the ensuing discussion was mostly about why it was hard to
do that in emacs -- but I don't think that detracts from the basic
functionality, it's just an interesting possible feature.

Are you sure you're not thinking of some other patch?

-Miles
-- 
/\ /\
(^.^)
(")")
*This is the cute kitty virus, please copy this into your sig so it can spread.

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Miles Bader <miles@gnu.org> writes:

> On Sun, Apr 11, 2004 at 05:06:05AM -0400, Miles Bader wrote:
> > On Sat, Apr 10, 2004 at 10:34:55PM -0400, Richard Stallman wrote:
> > >     I'd like to (post-release) merge my tiling branch too; I don't seen any
> > >     particularly good reason to keep it separate.
> > > 
> > > I seem to recall that code had serious flaws and that it was not clear
> > > whether they could be fixed at all.
> > 
> > If so, nobody told _me_ about them...

IIRC, it was yourself who said that my various consolidation changes
broke your tiling patches.  I'm pleased to hear that it wasn't the
case after all (or that you managed to recover from my "de-messing"
efforts :-)

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

On Mon, Apr 12, 2004 at 11:10:48PM +0200, Kim F. Storm wrote:
> IIRC, it was yourself who said that my various consolidation changes
> broke your tiling patches.  I'm pleased to hear that it wasn't the
> case after all (or that you managed to recover from my "de-messing"
> efforts :-)

No, in fact though I've been a bit scared everytime you re-arrange redisplay,
merging's never been a real problem even then.

-Miles
-- 
Quidquid latine dictum sit, altum viditur.

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

    > > I seem to recall that code had serious flaws and that it was not clear
    > > whether they could be fixed at all.
    > 
    > If so, nobody told _me_ about them...

    To restate that in a less glib manner, to the best of my knowledge, my tiling
    changes have never been reviewed by anybody, and such discussion as has
    occurred has mostly been off on various tangents, not about the core
    functionality.

I think we had a discussion about this right when you wrote it.
They were conceptual-level issues, not little bugs.

But my memory is vague.  I don't remember what the tiling patch does,
I only have an associated memory that there was some problem which
wasn't obvious how to fix.  It is possible I'm confusing this with
some other patch.

Can you tell me the range of dates where I could find the first discussion
of the tiling patch?

Re: It is time for a feature freeze (it is NOW or never).

[Juri Linkov]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> I'd offer a very slightly different schedule:

I think this is the best plan for Emacs users and for Emacs developers.
I have only a few comments.

> (2) Feature freeze on the trunk NOW

Please announce a day for feature freeze ~1-2 weeks from now,
to give a chance to submit latest features.

> (3) Start pretest from the trunk (maybe after fixing known bugs, but there
>     doesn't seem to be many of those, so it seems unnecessary).

This shouldn't take too long given the fact that CVS Emacs is already
under constant testing.

> (4) When bug-fixing starts to slow down such that some developers feel bored,
>     make a new RC branch and move the bug-fixing and pretesting there.
>     Those people who are bored can start playing on the trunk again.
> (5a) Finish bug-fixing RC and finally Release.
> (5b) Merge unicode, multi-tty, and bidi (in this order, but quickly) to the
>     trunk.  Branches basically don't get any testing and waste too much of
>     our energy with merging, so we should only keep them for code that's
>     really unstable or that we don't know whether we'll ever include.
>     E.g. if bidi only crashed when you set enable-bidi-display, then it's
>     ready for inclusion.
>     I'd also like to see the emacs-xft (anti-aliasing) merged in, but AFAIK
>     it's not ready yet.
> (6) Almost immediately after that: goto number (2).

After merging unicode, multi-tty, and bidi, when most developers will
start to use them, new bugs will be discovered and new related features
will be proposed, so most probably the next release will be postponed
for more than a half-year.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

On Thu, Apr 08, 2004 at 04:35:15AM +0200, Kim F. Storm wrote:
> Current HEAD is rock-solid for normal use, and I don't think it will
> can be much better in that respect (but we should try!).

This may be a false warning for HEAD (I run my own branch which has a fair
number of redisplay changes), but just in case:

I've had several crashes recently related to i-search faces, and haven't been
able to track down the cause; it appears that the face cache ids are being
used even after the face cache has been cleared.

-Miles
-- 
`Suppose Korea goes to the World Cup final against Japan and wins,' Moon said.
`All the past could be forgiven.'   [NYT]

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <20040408020537.GA22508@fencepost>, Miles Bader <miles@gnu.org> writes:

> On Thu, Apr 08, 2004 at 04:35:15AM +0200, Kim F. Storm wrote:
>>  Current HEAD is rock-solid for normal use, and I don't think it will
>>  can be much better in that respect (but we should try!).

> This may be a false warning for HEAD (I run my own branch which has a fair
> number of redisplay changes), but just in case:

> I've had several crashes recently related to i-search faces, and haven't been
> able to track down the cause; it appears that the face cache ids are being
> used even after the face cache has been cleared.

In emacs-unicode, I also met the similar bug.  Emacs crashed
at the end of get_glyph_face_and_encoding.  In this
function, FACE_FROM_ID (f, glyph->face_id) returned NULL,
thus, the macro call PREPARE_FACE_FOR_DISPLAY at the end led
to the crash.  Though, this happens very very rarely.

As emacs-unicode-2 was branched after a big change in
display code, it's possible that we are seeing the same bug.

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

Kenichi Handa <handa@m17n.org> writes:
> In emacs-unicode, I also met the similar bug.  Emacs crashed at the
> end of get_glyph_face_and_encoding.  In this function, FACE_FROM_ID
> (f, glyph->face_id) returned NULL, thus, the macro call
> PREPARE_FACE_FOR_DISPLAY at the end led to the crash.  Though, this
> happens very very rarely.

Yes that sounds like the same symptoms I saw.

-Miles
-- 
A zen-buddhist walked into a pizza shop and
said, "Make me one with everything."

Redisplay crash

[Stefan Monnier]

>>> Current HEAD is rock-solid for normal use, and I don't think it will
>>> can be much better in that respect (but we should try!).

>> This may be a false warning for HEAD (I run my own branch which has a fair
>> number of redisplay changes), but just in case:

>> I've had several crashes recently related to i-search faces, and haven't been
>> able to track down the cause; it appears that the face cache ids are being
>> used even after the face cache has been cleared.

> In emacs-unicode, I also met the similar bug.  Emacs crashed
> at the end of get_glyph_face_and_encoding.  In this
> function, FACE_FROM_ID (f, glyph->face_id) returned NULL,
> thus, the macro call PREPARE_FACE_FOR_DISPLAY at the end led
> to the crash.  Though, this happens very very rarely.

I've had such problems in the past.
There was a discussion about it with Gerd which led to

2002-08-27  Gerd Moellmann  <gerd.moellmann@t-online.de>

	* xdisp.c (redisplay_updating_p): Variable removed.
	(inhibit_free_realized_faces, Qinhibit_free_realized_faces):
	New variables.
	(init_iterator): Don't free realized faces if
	inhibit_free_realized_faces is set.
	(redisplay_internal): Bind Qinhibit_free_realized_faces to nil.
	(syms_of_xdisp): DEFVAR_BOOL inhibit-free-realized-faces,
	initialize Qinhibit_free_realized_faces.

	* dispextern.h (PRODUCE_GLYPHS): Set inhibit_free_realized_faces
	when iterator is adding glyphs to a glyph matrix.

I can't find trace of this discussion, tho.
I've always had the impression that the above patch didn't really fix the
problem, tho it made it less frequent, so maybe it only hid the problem
more than fix it, I don't know.  In any case I figure maybe you'd like
to know.


        Stefan

Re: Redisplay crash

[Kenichi Handa]

In article <87r7uyfoz9.fsf-monnier+emacs@alfajor.local>, Stefan Monnier <monnier@iro.umontreal.ca> writes:
> There was a discussion about it with Gerd which led to

> 2002-08-27  Gerd Moellmann  <gerd.moellmann@t-online.de>

> 	* xdisp.c (redisplay_updating_p): Variable removed.
> 	(inhibit_free_realized_faces, Qinhibit_free_realized_faces):
> 	New variables.
> 	(init_iterator): Don't free realized faces if
> 	inhibit_free_realized_faces is set.
> 	(redisplay_internal): Bind Qinhibit_free_realized_faces to nil.
> 	(syms_of_xdisp): DEFVAR_BOOL inhibit-free-realized-faces,
> 	initialize Qinhibit_free_realized_faces.

> 	* dispextern.h (PRODUCE_GLYPHS): Set inhibit_free_realized_faces
> 	when iterator is adding glyphs to a glyph matrix.

> I can't find trace of this discussion, tho.
> I've always had the impression that the above patch didn't really fix the
> problem, tho it made it less frequent, so maybe it only hid the problem
> more than fix it, I don't know.  In any case I figure maybe you'd like
> to know.

Thank you for the info.  I found at least one way to crash
Emacs constantly.

At first, evaluate this in *scratch* buffer.

(put-text-property 2 3 'display '(when (clear-face-cache t)))

Then, type C-h h C-x C-k (i.e. view HELLO file and kill that
buffer).

Then, Emacs crashes in get_next_display_element at:

      /* Adjust face id for a multibyte character.  There are no
         multibyte character in unibyte text.  */
      if (it->multibyte_p
	  && success_p
	  && FRAME_WINDOW_P (it->f))
	{
	  struct face *face = FACE_FROM_ID (it->f, it->face_id);
here->	  it->face_id = FACE_FOR_CHAR (it->f, face, it->c);
	}

because FACE_FROM_ID returns NULL.

But, it seems that this is a different bug.

---
Ken'ichi HANDA
handa@m17n.org

Re: Redisplay crash

[Piet van Oostrum]

>>>>> Kenichi Handa <handa@m17n.org> (KH) wrote:

KH> Thank you for the info.  I found at least one way to crash
KH> Emacs constantly.

KH> At first, evaluate this in *scratch* buffer.

KH> (put-text-property 2 3 'display '(when (clear-face-cache t)))

KH> Then, type C-h h C-x C-k (i.e. view HELLO file and kill that
KH> buffer).

For me, on Mac OS X, the first step alone will already crash emacs.
This is the backtrace:

Program received signal EXC_BAD_ACCESS, Could not access memory.
0x00021054 in get_next_display_element (it=0xbfffd430) at xdisp.c:4948
4948              it->face_id = FACE_FOR_CHAR (it->f, face, it->c);
(gdb) bt
#0  0x00021054 in get_next_display_element (it=0xbfffd430) at xdisp.c:4948
#1  0x00021970 in next_element_from_buffer (it=0xbfffd430) at xdisp.c:5436
#2  0x00020cc0 in get_next_display_element (it=0xbfffd430) at xdisp.c:4787
#3  0x00031520 in display_line (it=0xbfffd430) at xdisp.c:14411
#4  0x0002d6c4 in try_window (window=10719952, pos={charpos = 1, bytepos = 1}) at xdisp.c:12140
#5  0x0002c820 in redisplay_window (window=-2110188656, just_this_one_p=0) at xdisp.c:11783
#6  0x00029978 in redisplay_window_0 (window=10719952) at xdisp.c:10538
#7  0x000e0cc0 in internal_condition_case_1 (bfun=0x2993c <redisplay_window_0>, arg=-2110188656, handlers=-1604078312, hfun=0x29908 <redisplay_window_error>) at eval.c:1374
#8  0x000298e0 in redisplay_windows (window=0) at xdisp.c:10517
#9  0x000298c0 in redisplay_windows (window=0) at xdisp.c:10511
#10 0x00028f20 in redisplay_internal (preserve_echo_area=10719952) at xdisp.c:10102
#11 0x0007e24c in read_char (commandflag=1, nmaps=3, maps=0xbfffecc0, prev_event=595592192, used_mouse_menu=0xbfffedc4) at keyboard.c:2472
#12 0x0008686c in read_key_sequence (keybuf=0x34b6f4, bufsize=3505968, prompt=-2110188656, dont_downcase_last=-1821406164, can_return_switch_frame=1, fix_current_buffer=1) at keyboard.c:8770
#13 0x0007c01c in command_loop_1 () at keyboard.c:1476
#14 0x000e0b54 in internal_condition_case (bfun=0x7bb60 <command_loop_1>, handlers=595637240, hfun=0x7b560 <cmd_error>) at eval.c:1333
#15 0x0007b988 in command_loop_2 () at keyboard.c:1264
#16 0x000e05bc in internal_catch (tag=10719952, func=0x7b948 <command_loop_2>, arg=595592192) at eval.c:1094
#17 0x0007b8e0 in command_loop () at keyboard.c:1243
#18 0x0007b2fc in recursive_edit_1 () at keyboard.c:959
#19 0x0007b484 in Frecursive_edit () at keyboard.c:1015
#20 0x00079fac in main (argc=3510216, argv=0x3565ac) at emacs.c:1692
(gdb) quit

-- 
Piet van Oostrum <piet@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP]
Private email: P.van.Oostrum@hccnet.nl

Re: Redisplay crash

[Richard Stallman]

    (put-text-property 2 3 'display '(when (clear-face-cache t)))

    Then, type C-h h C-x C-k (i.e. view HELLO file and kill that
    buffer).

    Then, Emacs crashes in get_next_display_element at:

Has anyone investigated this bug?

Re: Redisplay crash

[Richard Stallman]

    I've had such problems in the past.
    There was a discussion about it with Gerd which led to

    2002-08-27  Gerd Moellmann  <gerd.moellmann@t-online.de>

Perhaps we need to record history information about occurrences
within redisplay, so that after it crashes we can investigate
what happened without needing to find a reproducible test case.

Re: Redisplay crash

[Kim F. Storm]

Richard Stallman <rms@gnu.org> writes:

>     I've had such problems in the past.
>     There was a discussion about it with Gerd which led to
> 
>     2002-08-27  Gerd Moellmann  <gerd.moellmann@t-online.de>
> 
> Perhaps we need to record history information about occurrences
> within redisplay, so that after it crashes we can investigate
> what happened without needing to find a reproducible test case.

I don't think it will be worth the efforts.

There is just so much going on during redisplay, that it is almost
impossible to know what and how much information to put into the
history.  I have used various methods of collecting history during
redisplay to track down display errors, even when doing a targeted
information collection, it is very difficult to actually use that
history to find bugs.

It is always better to have a specific test case and debug the code to
find the cause of the error.


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Redisplay crash

[Richard Stallman]

    > Perhaps we need to record history information about occurrences
    > within redisplay, so that after it crashes we can investigate
    > what happened without needing to find a reproducible test case.

    I don't think it will be worth the efforts.

    It is always better to have a specific test case and debug the code to
    find the cause of the error.

It is surely better to have a specific test case; the problem is
that sometimes we don't have one and nobody can find one.

Re: It is time for a feature freeze (it is NOW or never).

[Romain Francoise]

storm@cua.dk (Kim F. Storm) writes:

> I suggest that we make a complete feature freeze on HEAD for 21.4 NOW
> !!!

Please consider merging the multi-tty branch.  It brings a set of _very_
useful features, and postponing the merge until Emacs 22 sounds like a
real waste.  I have been using this branch for some months as my main
Emacs environment for heavy usage and it's very solid.  The only
show-stopper is that Mac, Windows and DOS support is missing, but maybe
this could be left aside temporarily...

<URL: http://lorentey.web.elte.hu/project/emacs.html.en>

-- 
Romain Francoise <romain@orebokech.com> | They're nothing but scared
it's a miracle -- http://orebokech.com/ | little mice.

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Romain Francoise <romain@orebokech.com> writes:

> Please consider merging the multi-tty branch.  It brings a set of _very_
> useful features, and postponing the merge until Emacs 22 sounds like a
> real waste.  

IMHO, postponing currently available features that have already
been in HEAD for years any further would be an even bigger waste.

>              I have been using this branch for some months as my main
> Emacs environment for heavy usage and it's very solid.  

One or a few user's impression here doesn't convince me that it can be
merged without hazzle.

I have looked briefly at the multi-tty pathces, and although it looks
very sensible and systematic in its approach, it touches on many files
and interfaces!  

Since it probably hasn't received much attention (yet) from the core
developers, I would think that there are still some issues which need
to be ironed out.  For example , what is the value of window-system
variable if you have both a window frame and a non-window frame open
at the same time (I haven't looked, so I don't know how this issue has
been resolved, but there could be other issues like it).

Also, is the documentation on the multi-tty branch updated to reflect
changes in interfaces etc. (where it matters).

I would fear that even minor issues with it would inhibit a 21.5
release in 2004, and no matter how useful it is, I think it will have
to wait.  

Others may (and will) disagree of course...

>                                                         The only
> show-stopper is that Mac, Windows and DOS support is missing, but maybe
> this could be left aside temporarily...
> 

We are talking about a major new release here, so I don't like to have
"temporary show-stoppers" that will generate noise in the pretest.

There are enough new features for people to test already...

But I also think that as soon as we have branched from HEAD for 21.5,
it would be a good time to merge the multi-tty stuff to HEAD (with the
unicode branch).

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Romain Francoise]

storm@cua.dk (Kim F. Storm) writes:

> IMHO, postponing currently available features that have already
> been in HEAD for years any further would be an even bigger waste.

True.

> I have looked briefly at the multi-tty pathces, and although it looks
> very sensible and systematic in its approach, it touches on many files
> and interfaces!  

Yes; it is quite intrusive.

> For example , what is the value of window-system variable if you have
> both a window frame and a non-window frame open at the same time (I
> haven't looked, so I don't know how this issue has been resolved, but
> there could be other issues like it).

It's what it's supposed to be: if you eval it in an X frame, you get 'x,
in a console frame you get nil.

> Also, is the documentation on the multi-tty branch updated to reflect
> changes in interfaces etc. (where it matters).

I don't know about that, but it doesn't look like it.

> I would fear that even minor issues with it would inhibit a 21.5
> release in 2004, and no matter how useful it is, I think it will have
> to wait.  

You're right, apparently a large amount of work is needed before this
branch can go in HEAD, your plan for an immediate release is the most
reasonable course of action.  I will have to wait some more before
seeing these features in mainline Emacs...

Cheers,

-- 
Romain Francoise <romain@orebokech.com> | It was fourteen degrees below
it's a miracle -- http://orebokech.com/ | on a screeching march 23.

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

I agree that we should not include the multi-tty code in the next HEAD
release.  However, it sounds like maybe the unicode branch is ready to
install now.  If so, maybe we should install that as soon as
practical, then start pretesting from HEAD, and skip making another
release from RC.

However, we need to remove the files that relate specifically to
encryption.

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Richard Stallman <rms@gnu.org> writes:

> I agree that we should not include the multi-tty code in the next HEAD
> release.  However, it sounds like maybe the unicode branch is ready to
> install now.  

I still think this is a bad idea -- it cannot be done without potentially
destabilizing the trunk which is currently in a very stable state, and
thus in a pretty unique state of "readiness for release".

>               If so, maybe we should install that as soon as
> practical, then start pretesting from HEAD, and skip making another
> release from RC.
> 
> However, we need to remove the files that relate specifically to
> encryption.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <m3smfcbol5.fsf@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:

> Richard Stallman <rms@gnu.org> writes:
>>  I agree that we should not include the multi-tty code in the next HEAD
>>  release.  However, it sounds like maybe the unicode branch is ready to
>>  install now.  

> I still think this is a bad idea -- it cannot be done without potentially
> destabilizing the trunk which is currently in a very stable state, and
> thus in a pretty unique state of "readiness for release".

I agree with Kim.  Unicode branch is stable for me, but I
think we'll meet various problems when used more widely.  In
addition, we must work on mule-related manuals.  I dared not
work on manuals in Unicode branch to avoid conflict with
frequent changes in HEAD.

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

      In
    addition, we must work on mule-related manuals.  I dared not
    work on manuals in Unicode branch to avoid conflict with
    frequent changes in HEAD.

In that case, I guess we should aim for a release without the unicode
branch.  And certainly without the multi-tty branch.  (As for the
tiling and lexical branches, I am not convinced I want to make those
changes.)

However, this reminds me that we need to update the Emacs Manual.  I
updated the Emacs Lisp manual myself some 6 months ago, but aside from
Luc Teirlink who has done a lot of work, most of you have not helped
me check it.  Now it is out of print.  But I don't know if I can
update the Emacs Manual.  I can barely keep up with my work, and I am
injured.

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Richard Stallman <rms@gnu.org> writes:

>       In
>     addition, we must work on mule-related manuals.  I dared not
>     work on manuals in Unicode branch to avoid conflict with
>     frequent changes in HEAD.
> 
> In that case, I guess we should aim for a release without the unicode
> branch.  And certainly without the multi-tty branch.  (As for the
> tiling and lexical branches, I am not convinced I want to make those
> changes.)

So when do we declare "feature freeze" ?  May 1st ?


> 
> However, this reminds me that we need to update the Emacs Manual.  I
> updated the Emacs Lisp manual myself some 6 months ago, but aside from
> Luc Teirlink who has done a lot of work, most of you have not helped
> me check it.  Now it is out of print.  But I don't know if I can
> update the Emacs Manual.  I can barely keep up with my work, and I am
> injured.

Once we have "feature freeze", it will be a good time for everyone to
update the manuals to reflect their additions (hopefully documented in
etc/NEWS).

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

>> In that case, I guess we should aim for a release without the unicode
>> branch.  And certainly without the multi-tty branch.  (As for the
>> tiling and lexical branches, I am not convinced I want to make those
>> changes.)

> So when do we declare "feature freeze" ?  May 1st ?

May 1st sounds very good to me,


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> >> In that case, I guess we should aim for a release without the unicode
> >> branch.  And certainly without the multi-tty branch.  (As for the
> >> tiling and lexical branches, I am not convinced I want to make those
> >> changes.)
> 
> > So when do we declare "feature freeze" ?  May 1st ?
> 
> May 1st sounds very good to me,

Too good to be true ?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

On Sat, May 01, 2004 at 10:16:25PM +0200, Kim F. Storm wrote:
> > > So when do we declare "feature freeze" ?  May 1st ?
> > 
> > May 1st sounds very good to me,
> 
> Too good to be true ?

Um, as you seem to be adding many of the new features these days, why don't
you tell us?

-Miles
-- 
`The suburb is an obsolete and contradictory form of human settlement'

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Miles Bader <miles@gnu.org> writes:

> On Sat, May 01, 2004 at 10:16:25PM +0200, Kim F. Storm wrote:
> > > > So when do we declare "feature freeze" ?  May 1st ?
> > > 
> > > May 1st sounds very good to me,
> > 
> > Too good to be true ?
> 
> Um, as you seem to be adding many of the new features these days, why don't
> you tell us?

I was busy doing it before May 1st :-)

I'll concentrate on bug-fixing + documentation from now on !!

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

Miles Bader <miles@gnu.org> writes:

> On Sat, May 01, 2004 at 10:16:25PM +0200, Kim F. Storm wrote:
> > > > So when do we declare "feature freeze" ?  May 1st ?
> > > 
> > > May 1st sounds very good to me,
> > 
> > Too good to be true ?
> 
> Um, as you seem to be adding many of the new features these days,
> why don't you tell us?

Hey, that was _before_ May 1, to get a nice feature set to iron out.
One that will make people happy for the next three years we need to
get a feature release out.

Ok, the image stuff seemingly has a few discussions with possible
interface changes pending, but the features themselves were in before
the feature freeze.  And that's the point of a freeze: gather things
that really should get it before that all at once, so that one can
the solidify on everything afterwards.

With regard to solidification, I think that many of Emacs' scrolling
functions should get more comfortable with large-height glyphs.  While
we should avoid last-minute before-release changes (which might
introduce deadlocks and similar), quite a bit of changes in that area
would qualify not as "features" but as bug fixes.  The feature is tall
lines, and it needs work to be considered close to proper.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

>> > > So when do we declare "feature freeze" ?  May 1st ?
>> > May 1st sounds very good to me,
>> Too good to be true ?
> Um, as you seem to be adding many of the new features these days, why don't
> you tell us?

As a perpertrator of "last minute feature addition", I can't complain.
But in any case I think people should from now on concentrate on bug-fixing
(which might include some interface-changes, admittedly).


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Nick Roberts]

 > >> > > So when do we declare "feature freeze" ?  May 1st ?
 > >> > May 1st sounds very good to me,
 > >> Too good to be true ?
 > > Um, as you seem to be adding many of the new features these days, why don't
 > > you tell us?
 > 
 > As a perpertrator of "last minute feature addition", I can't complain.
 > But in any case I think people should from now on concentrate on bug-fixing
 > (which might include some interface-changes, admittedly).

IMHO, any feature freeze is only meaningful if it is accompanied by a branch.
Otherwise new features will have no place to go and will quite likely get
bundled in with bug-fixes.

Also, from the mailing list archives, I see that a year elapsed between the
first pretest and the release of Emacs 21.1. So I would like to see a target
date for the release so that the "freeze" doesn't drag on indefinitely.

Well, one can only hope.

Nick

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

Nick Roberts <nick@nick.uklinux.net> writes:

>  > >> > > So when do we declare "feature freeze" ?  May 1st ?
>  > >> > May 1st sounds very good to me,
>  > >> Too good to be true ?

>  > > Um, as you seem to be adding many of the new features these
>  > > days, why don't you tell us?
>  > 
>  > As a perpertrator of "last minute feature addition", I can't
>  > complain.  But in any case I think people should from now on
>  > concentrate on bug-fixing (which might include some
>  > interface-changes, admittedly).
> 
> IMHO, any feature freeze is only meaningful if it is accompanied by
> a branch.

No.

> Otherwise new features will have no place to go and will quite
> likely get bundled in with bug-fixes.

New features can go into branches if really necessary.  And we already
_have_ some things like lexical, bidi, multi-tty, unicode and so on in
branches.  So it is not that this is a novelty.  But developers should
get their head wrapped around getting Emacs into releasable state
rather than bothering with new features now, if possible.

> Also, from the mailing list archives, I see that a year elapsed
> between the first pretest and the release of Emacs 21.1. So I would
> like to see a target date for the release so that the "freeze"
> doesn't drag on indefinitely.

Nope.  There is no point in releasing it before it is ready.  And
there is no sense in delaying further once it is ready.  And there is
no point in telling a lot of voluntary that they have to deliver on a
certain date or lose their non-job.

> Well, one can only hope.

And work.  Proofreading the manual is something that needs not be
restricted to core developers.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Nick Roberts]

 > > Otherwise new features will have no place to go and will quite
 > > likely get bundled in with bug-fixes.
 > 
 > New features can go into branches if really necessary.

By new features, I just mean patches that might break functionality. They
might be too small to justify a separate branch.

 > > Also, from the mailing list archives, I see that a year elapsed
 > > between the first pretest and the release of Emacs 21.1. So I would
 > > like to see a target date for the release so that the "freeze"
 > > doesn't drag on indefinitely.
 >
 > Nope.  There is no point in releasing it before it is ready.  And
 > there is no sense in delaying further once it is ready.

Well, Confucius might say something like that. If there was a nominal date,
developers like Kai, could make a judgement as to whether or not there was
enough time to merge new features of Tramp.

 >                                                         And there is
 > no point in telling a lot of voluntary that they have to deliver on a
 > certain date or lose their non-job.

It would just be something to aim for. I don't remember talking about penalty
clauses. Currently, for all I know, features of GDB needed by gdb-ui.el may
be obsolete before the next Emacs release.

Nick

Re: It is time for a feature freeze (it is NOW or never).

[Jason Rumney]

Nick Roberts <nick@nick.uklinux.net> writes:

>  > > Otherwise new features will have no place to go and will quite
>  > > likely get bundled in with bug-fixes.
>  > 
>  > New features can go into branches if really necessary.
> 
> By new features, I just mean patches that might break functionality. They
> might be too small to justify a separate branch.

Put them into the unicode branch, since that is the target of the
next feature release after this one.

If we branch now, everyone will continue to work on HEAD, and we will
never get a release out. We need to concentrate on the release now,
and worry about branching for bugfixes after we have made a release.

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

    New features can go into branches if really necessary.  And we already
    _have_ some things like lexical, bidi, multi-tty, unicode and so on in
    branches.  So it is not that this is a novelty.  But developers should
    get their head wrapped around getting Emacs into releasable state
    rather than bothering with new features now, if possible.

I agree--it is little use having a "freeze" in the trunk if most
developers turn their attention away from it.  To aim for a release,
we should focus on the work that needs to be done for a release.

The biggest separate jobs remaining are updating and checking the
manuals.

Re: It is time for a feature freeze (it is NOW or never).

[Jan D.]

>
> The biggest separate jobs remaining are updating and checking the
> manuals.

Should the icons for the toolbar be updated for this release?  When 
running
on Gnome, it would be an advantage if Emacs used the same icons as other
Gnome/GTK applications.  I think Emacs now use GTK 1.2 icons.
We could just convert the GTK 2.x icons to xpm.  Alternative I have a
patch to use GTK stock icons for Emacs compiled with GTK.  It involves
setting the property :stock on tool bar items.  It is just for GTK
though, so it is not general.

	Jan D.

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

    Should the icons for the toolbar be updated for this release?  When 
    running
    on Gnome, it would be an advantage if Emacs used the same icons as other
    Gnome/GTK applications.  I think Emacs now use GTK 1.2 icons.
    We could just convert the GTK 2.x icons to xpm.

I think that someone drew us new icons for Emacs, but I don't
remember if they ever got installed, nor where to find them now.

Re: It is time for a feature freeze (it is NOW or never).

[Jan D.]

>     Should the icons for the toolbar be updated for this release?  When
>     running
>     on Gnome, it would be an advantage if Emacs used the same icons as 
> other
>     Gnome/GTK applications.  I think Emacs now use GTK 1.2 icons.
>     We could just convert the GTK 2.x icons to xpm.
>
> I think that someone drew us new icons for Emacs, but I don't
> remember if they ever got installed, nor where to find them now.

I've put up some pictures at http://mywebpage.netscape.com/jhdns/ so
you can view them easily.  There where two suggestions, Daniel Pfeiffer 
made
changes to the speedbar icons, and someone whos name escapes me (jefbed)
made Gnome icons in xpm formats.  None of these are installed in CVS.

The speedbar icons where marginally preferred by me and Miles Bader, I 
don't
think there where any other comments.  The Gnome icons where not 
commented
much either.

I slightly prefer the new icons, but it is not that much of a 
difference.
I do think we should change though.

	Jan D.

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

"Jan D." <jan.h.d@swipnet.se> writes:

> >     Should the icons for the toolbar be updated for this release?
> >     When running on Gnome, it would be an advantage if Emacs used
> >     the same icons as other Gnome/GTK applications.  I think Emacs
> >     now use GTK 1.2 icons.  We could just convert the GTK 2.x
> >     icons to xpm.
> >
> > I think that someone drew us new icons for Emacs, but I don't
> > remember if they ever got installed, nor where to find them now.
> 
> I've put up some pictures at http://mywebpage.netscape.com/jhdns/ so
> you can view them easily.  There where two suggestions, Daniel
> Pfeiffer made changes to the speedbar icons, and someone whos name
> escapes me (jefbed) made Gnome icons in xpm formats.  None of these
> are installed in CVS.
> 
> The speedbar icons where marginally preferred by me and Miles Bader,
> I don't think there where any other comments.  The Gnome icons where
> not commented much either.

The speedbar icon changes are marginal, mainly being less glaring
(which is not the worst idea, considering that they are merely
indicators beside the main text).

With the other icons, the changes are more significant: one thing
that is apparent is that they are done with much more focus on
getting them to rasterize well.  With the exception of the scissor, I
consider them a definite improvement.

I am not sure the "help" icon is an improvement.  The "?" icon is
just that, iconic, whereas the life-saver ring looks like a pretty
toy at first sight.

But in general, I don't think it worth the trouble to do an Emacs-only
policy for single icons: such details should rather be discussed
upstream; they are not Emacs-specific.

> I slightly prefer the new icons, but it is not that much of a
> difference.  I do think we should change though.

Agreed.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

We can use the GMOME icons on the same basis that we use things
such as oldxlib: saying that they are not part of Emacs, but
used by Emacs and distributed with Emacs.

B&W icons (was It is time for a feature freeze (it is NOW or never).)

[Jan D.]

Hello.

I've noticed that the GTK 2.0 icons look much worse than the current 
ones
when rendered as black and white.  I was thinking of keeping the old 
ones
for black and white (as there should not be that many B&W displays 
compared
to colour displays).  Is that feasible?  Alternatively, does anyone know
how to do a better conversion than xpmtoppm|ppmdist|pgmtopbm|pnminvert?

	Jan D.

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

> IMHO, any feature freeze is only meaningful if it is accompanied by a branch.
> Otherwise new features will have no place to go and will quite likely get
> bundled in with bug-fixes.

No, we want people to stop working on features and work on
bugfixing instead.  When we create the branch is when we say "alright
people, if you're bored of bugfixing you can add features again".

> Also, from the mailing list archives, I see that a year elapsed between the
> first pretest and the release of Emacs 21.1. So I would like to see a target
> date for the release so that the "freeze" doesn't drag on indefinitely.

It just depends on how much time it takes before the stream of bug-reports
slows down enough that the code can be considered "stable".


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Juanma Barranquero]

On 02 May 2004 15:16:56 -0400, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> No, we want people to stop working on features and work on
> bugfixing instead.

I've been trying to finish the help-highlight-argument stuff in time to
install it before the freeze. It's tiny and it can be removed easily if
some problem arises.

I'd like an OK or three before install it, though.

                                                           /L/e/k/t/u

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Juanma Barranquero <lektu@mi.madritel.es> writes:

> On 02 May 2004 15:16:56 -0400, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
> > No, we want people to stop working on features and work on
> > bugfixing instead.
> 
> I've been trying to finish the help-highlight-argument stuff in time to
> install it before the freeze. It's tiny and it can be removed easily if
> some problem arises.
> 
> I'd like an OK or three before install it, though.

In my book "feature freeze" does not completely exclude tiny
enhancements (like this one) going in; rather it is a "state of mind"
on the developers concentrating on _finalizing_ what's already there
rather than _adding_ new stuff.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Juanma Barranquero]

On 03 May 2004 07:44:18 +0200
storm@cua.dk (Kim F. Storm) wrote:

> In my book "feature freeze" does not completely exclude tiny
> enhancements (like this one) going in; rather it is a "state of mind"
> on the developers concentrating on _finalizing_ what's already there
> rather than _adding_ new stuff.

I agree, but I'd rather ask than step on somebody's toes.

Thanks,

                                                                Juanma

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> >> In that case, I guess we should aim for a release without the unicode
> >> branch.  And certainly without the multi-tty branch.  (As for the
> >> tiling and lexical branches, I am not convinced I want to make those
> >> changes.)
> 
> > So when do we declare "feature freeze" ?  May 1st ?
> 
> May 1st sounds very good to me,

BTW, what about USE_LSB_TAG ?  

Should we make it the default on some platforms?  Which ones?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

> BTW, what about USE_LSB_TAG ?  
> Should we make it the default on some platforms?  Which ones?

The USE_LSB_TAG code is ready in the sense that it should work just as
well as the default (or as USE_LISP_UNION_TYPE).  Even .gdbinit has been
updated to automatically know how to do xbacktrace and stuff with it.

So we could make it the default on some platforms (I've used it extensively
on MacOSX and GNU/Linux).

Note that there is still an open bug (by David Ponce) that I have not been
able to track down.


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

Let's make USE_LSB_TAG the default for now, and if the bug you
have not found does not get fixed, we can back it out later.

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

> Let's make USE_LSB_TAG the default for now, and if the bug you
> have not found does not get fixed, we can back it out later.

I have now made it the default on systems where I know it should work.
I.e. it checks that it is compiled with gcc and it also checks that the
malloc library is either the one that comes with glibc or macosx's
libc, or is doug lea's malloc or gnu_malloc.
I don't know if that covers *BSD systems.

You'll need to rm src/*.o before rebuilding, since the required
dependencies on lisp.h are missing from src/Makefile.


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > Let's make USE_LSB_TAG the default for now, and if the bug you
> > have not found does not get fixed, we can back it out later.
> 
> I have now made it the default on systems where I know it should work.

I have installed a fix to mark_kboards to avoid marking x and y members
of an input_event when it is overloaded by a selection_input_event.

Without USE_LSB_TAG, it didn't really matter as x and y were still
just integers, but with USE_LSB_TAG, they become random objects when
overloaded.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

>> > Let's make USE_LSB_TAG the default for now, and if the bug you
>> > have not found does not get fixed, we can back it out later.
>> I have now made it the default on systems where I know it should work.
> I have installed a fix to mark_kboards to avoid marking x and y members
> of an input_event when it is overloaded by a selection_input_event.

Eww.... I didn't know about this overloading.  Pretty ugly.
Thanks for figuring it out.  I strongly suspect that it is the problem that
David Ponce had and that I couldn't reproduce (his backtraces always passed
through mark_kboards).


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[YAMAMOTO Mitsuharu]

>>>>> On 13 May 2004 13:14:14 -0400, Stefan Monnier <monnier@iro.umontreal.ca> said:

>> Let's make USE_LSB_TAG the default for now, and if the bug you have
>> not found does not get fixed, we can back it out later.

> I have now made it the default on systems where I know it should
> work.

Fccl_execute_on_string fails to set the values to the status vector.
The following change should fix that.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

*** src/ccl.c.~1.84.~	Mon Nov 17 01:17:09 2003
--- src/ccl.c	Fri May 14 20:57:30 2004
***************
*** 2196,2202 ****
    produced = ccl_driver (&ccl, SDATA (str), outbuf,
  			 SBYTES (str), outbufsize, (int *) 0);
    for (i = 0; i < 8; i++)
!     XSET (AREF (status, i), Lisp_Int, ccl.reg[i]);
    XSETINT (AREF (status, 8), ccl.ic);
    UNGCPRO;
  
--- 2196,2202 ----
    produced = ccl_driver (&ccl, SDATA (str), outbuf,
  			 SBYTES (str), outbufsize, (int *) 0);
    for (i = 0; i < 8; i++)
!     XSETINT (AREF (status, i), ccl.reg[i]);
    XSETINT (AREF (status, 8), ccl.ic);
    UNGCPRO;

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <wlekpnxkv3.wl@church.math.s.chiba-u.ac.jp>, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
>>  I have now made it the default on systems where I know
>> it should work.

> Fccl_execute_on_string fails to set the values to the
> status vector.  The following change should fix that.

Thank you.  I've just installed this slightly different fix.

Index: ccl.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/ccl.c,v
retrieving revision 1.84
retrieving revision 1.85
diff -u -c -r1.84 -r1.85
cvs server: conflicting specifications of output style
*** ccl.c	16 Nov 2003 16:17:09 -0000	1.84
--- ccl.c	14 May 2004 12:41:32 -0000	1.85
***************
*** 2196,2203 ****
    produced = ccl_driver (&ccl, SDATA (str), outbuf,
  			 SBYTES (str), outbufsize, (int *) 0);
    for (i = 0; i < 8; i++)
!     XSET (AREF (status, i), Lisp_Int, ccl.reg[i]);
!   XSETINT (AREF (status, 8), ccl.ic);
    UNGCPRO;
  
    if (NILP (unibyte_p))
--- 2196,2203 ----
    produced = ccl_driver (&ccl, SDATA (str), outbuf,
  			 SBYTES (str), outbufsize, (int *) 0);
    for (i = 0; i < 8; i++)
!     ASET (status, i, make_number (ccl.reg[i]));
!   ASET (status, 8, make_number (ccl.ic));
    UNGCPRO;
  
    if (NILP (unibyte_p))

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> >> In that case, I guess we should aim for a release without the unicode
> >> branch.  And certainly without the multi-tty branch.  (As for the
> >> tiling and lexical branches, I am not convinced I want to make those
> >> changes.)
> 
> > So when do we declare "feature freeze" ?  May 1st ?
> 
> May 1st sounds very good to me,


Did we decide on naming the next release 21.5 ?  

It's simpler just to stick with 21.4 and do some real work, but if we
must change it, who will make an effort to change 21.4 -> 21.5
throughout the sources.

To speed up things even further, should we make a pretest now?

We could announce to the pretesters that it is an early pre-test, but
believed to be stable and ask the pretesters to actively start using
it so we can get feedback asap.

I suggest that we set the USE_LSB_TAG for GNU/Linux and Mac/OS for the
first pretest; we can then enable it for more ports in the next
pretest.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

storm@cua.dk (Kim F. Storm) writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> > >> In that case, I guess we should aim for a release without the
> > >> unicode branch.  And certainly without the multi-tty branch.
> > >> (As for the tiling and lexical branches, I am not convinced I
> > >> want to make those changes.)
> > 
> > > So when do we declare "feature freeze" ?  May 1st ?
> > 
> > May 1st sounds very good to me,
> 
> 
> Did we decide on naming the next release 21.5 ?
> 
> It's simpler just to stick with 21.4 and do some real work, but if we
> must change it, who will make an effort to change 21.4 -> 21.5
> throughout the sources.

I think that 21.4 has been mentioned frequently as probably another
bug fix release, and 21.5 is "half-way" to 22.0.

Since we will need quite some time of consolidation before we can
hope to get out the new feature release, reserving 21.4 for a
possible bug fix release in case something horrible turns up that
needs immediate fixing during that time might not do harm.

And if we don't need to release a 21.4 in the mean time, the version
number jump will then perhaps be somewhat of an indicator that there
was a larger change.

It's annoying if you write things like "will be available in version
21.4 and later" in some manual and then have to admit that you were
lying.  And 21.4 has already been earmarked as a non-trunk release in
some announcements or comments.

So if we skip this bugfix release, let us also skip the number.

> To speed up things even further, should we make a pretest now?
> 
> We could announce to the pretesters that it is an early pre-test,
> but believed to be stable and ask the pretesters to actively start
> using it so we can get feedback asap.

Believed to be stable?  With several unresolved lockups and display
problems and last-minute additions?  I think that would be unfair.
"Workable" and "worth it" would be more like it.

> I suggest that we set the USE_LSB_TAG for GNU/Linux and Mac/OS for
> the first pretest; we can then enable it for more ports in the next
> pretest.

I suggest that before doing a formal pretest release, we let the last
changes shake down a few weeks among developers.

Well, at least till Mother's Day.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Juanma Barranquero]

On 04 May 2004 09:14:26 +0200
David Kastrup <dak@gnu.org> wrote:

> So if we skip this bugfix release, let us also skip the number.

I fully agree...

> reserving 21.4 for a
> possible bug fix release in case something horrible turns up that
> needs immediate fixing during that time might not do harm.

...but this doesn't seem very sensible. Getting out a 21.5 and
*afterwards* a bug-fix 21.4 would be confusing, to say the least. If
we're afraid it can really happen, we would be best served by bumping up
21.5 to 22.0 so 21.X, X > 3 can be presented as "new releases of the
stable 21.1 series".

But in fact I think that if something horrible happens after 21.5, the
thing to do would be to fix it as quickly as posible, and meanwhile
recommend users to stand with the stable 21.3.

                                                                Juanma

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

Juanma Barranquero <jmbarranquero@wke.es> writes:

> On 04 May 2004 09:14:26 +0200
> David Kastrup <dak@gnu.org> wrote:
> 
> > So if we skip this bugfix release, let us also skip the number.
> 
> I fully agree...
> 
> > reserving 21.4 for a
> > possible bug fix release in case something horrible turns up that
> > needs immediate fixing during that time might not do harm.
> 
> ...but this doesn't seem very sensible. Getting out a 21.5 and
> *afterwards* a bug-fix 21.4 would be confusing, to say the least.

No, no.  The 21.4 was supposed to be reserved as a bug fix release to
21.3.  Once 21.5 is out, 21.4 should not appear: then people have to
jump the gun and help us get the bug out from 21.5 instead.

I just wanted to take 21.4 as a reserve when we know that 21.5 can't
be out soon, but we would very much need a particular bug fix to 21.3
to be out before that.

> But in fact I think that if something horrible happens after 21.5,
> the thing to do would be to fix it as quickly as posible, and
> meanwhile recommend users to stand with the stable 21.3.

Right.  I was talking about reserving 21.4 for _before_ 21.5 is
released.  Once we consider the trunk stable enough for an 21.5
release to be imminent, expending work on an in-between bug fix
release would be inappropriate.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

David Kastrup <dak@gnu.org> writes:

> > > So if we skip this bugfix release, let us also skip the number.
> > 
> > I fully agree...

I agree too -- so who will take on the task of changing 21.4 -> 21.5 ?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Juanma Barranquero]

On 04 May 2004 08:34:31 +0200
storm@cua.dk (Kim F. Storm) wrote:

> I agree too -- so who will take on the task of changing 21.4 -> 21.5 ?

I'll do tonight (in about ten hours).


                                                                Juanma

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

>> I agree too -- so who will take on the task of changing 21.4 -> 21.5 ?
> I'll do tonight (in about ten hours).

Please don't.


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Juanma Barranquero]

On 04 May 2004 09:45:02 -0400
Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> Please don't.

Ok, but why?  Are you disagreeing with releasing 21.5, or just have many
uncommited changes right now?


                                                                Juanma

Re: It is time for a feature freeze (it is NOW or never).

[Juanma Barranquero]

On 04 May 2004 10:17:49 +0200
David Kastrup <dak@gnu.org> wrote:

> No, no.  The 21.4 was supposed to be reserved as a bug fix release to
> 21.3.  Once 21.5 is out, 21.4 should not appear: then people have to
> jump the gun and help us get the bug out from 21.5 instead.

Sorry, I misunderstood you.

I agree with your plan.


                                                                Juanma

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

> I think that 21.4 has been mentioned frequently as probably another
> bug fix release, and 21.5 is "half-way" to 22.0.

Huh?  I think it's been much more mentioned as "the next big thing".
As in "will only be included in 21.4".

And all the code currently refersw to 21.4.

So I suggest we don't change anything for now (i.e. aim for 21.4).
If something bad happens we can always release a 21.4 bugfix and go through
the CVS code to fix the refs from 21.4 to 21.5.


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Juanma Barranquero]

On 04 May 2004 09:44:36 -0400
Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> So I suggest we don't change anything for now (i.e. aim for 21.4).

We'll have to change the places where 21.5 is already mentioned:

  lisp/cus-start.el
  lisp/descr-text.el
  lisp/log-edit.el
  lisp/international/latin1-disp.el
  lispref/modes.texi

(At least.)

                                                                Juanma

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > I think that 21.4 has been mentioned frequently as probably another
> > bug fix release, and 21.5 is "half-way" to 22.0.
> 
> Huh?  I think it's been much more mentioned as "the next big thing".
> As in "will only be included in 21.4".

That was once.  And then there was a time where it was "21.4 will
likely be another bug fix release".

There was once talk to have another RC_1 branch release, before we
decided to bite the bullet and go directly for a trunk release next.

> And all the code currently refersw to 21.4.
> 
> So I suggest we don't change anything for now (i.e. aim for 21.4).
> If something bad happens we can always release a 21.4 bugfix and go through
> the CVS code to fix the refs from 21.4 to 21.5.

The problem is that I saw quite a few announcements of the "will be
fixed/available in 21.4 or 21.5" by now.  Since talk was made about
another RC_1 release, people have amended the lines that once talked
about the feature being in 21.2, 21.3 and 21.4 in succession.

And if indeed people expect 21.4 to be the next great thing,
releasing as 21.5 will do no harm.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

I think we may as well skip making another release from RC.  The
question is whether we should call the next release 21.4 or 22.0.

There has been so much change that maybe the next release should be
22.0.

I don't think we are ready for a pretest now.

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

> I think we may as well skip making another release from RC.  The
> question is whether we should call the next release 21.4 or 22.0.
> 
> There has been so much change that maybe the next release should be
> 22.0.

While there have been lots of small to medium changes and lots of
additions across the board, I don't think that there have been really
fundamental changes as compared to 21.0.  So the problem is not as
much that we have a big change warranting 22.0, but that we have
missed out on releasing anything between 21.4 and 21.20.

>From what we have planned right now, immediately after getting the
next release out, the Unicode branch will get merged and
consolidated, and the multi tty branch merged as well, if feasible.
Those are fundamental changes.  The Unicode branch is believed to be
in pretty good shape, but we want to have a consolidated _released_
version to merge it into.

Unicode, in particular with multi tty, will definitely warrant a
changed major release number.  It would appear awkward to change to
22.0, and right in the next step (after at most 22.1) to 23.0.
Merging the bidi branch, in contrast, will probably not warrant a
major release step, even though it will mean major work.

So in order to avoid version overkill, I think that we would stay at
21.x as long as we have "merely" extensive, but not fundamental
changes.

> I don't think we are ready for a pretest now.

Same here.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

David Kastrup <dak@gnu.org> writes:

> Richard Stallman <rms@gnu.org> writes:
> 
> > I think we may as well skip making another release from RC.  The
> > question is whether we should call the next release 21.4 or 22.0.
> > 
> > There has been so much change that maybe the next release should be
> > 22.0.
> 
> So in order to avoid version overkill, I think that we would stay at
> 21.x as long as we have "merely" extensive, but not fundamental
> changes.

Well, there are SOME fundamental changes from 21.3 to "this" release:

** Leim is now part of the Emacs distribution.
** The Emacs Lisp Reference Manual is now part of the distribution.
** The max size of buffers and integers has been doubled.
** GTK support

Besides, a lot of external lisp packages are now included with emacs
(tramp, ses, cua, ...)

If you look at NEWS, 3500 lines have been added since 21.3.

Compare this to 4800 lines describing the changes from 20.7 to 21.3

But I don't really case if we call this 21.4, 21.5, or 22.0 --
as long as we release something in 2004 !!

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

storm@cua.dk (Kim F. Storm) writes:

> David Kastrup <dak@gnu.org> writes:
> 
> > Richard Stallman <rms@gnu.org> writes:
> > 
> > > I think we may as well skip making another release from RC.  The
> > > question is whether we should call the next release 21.4 or 22.0.
> > > 
> > > There has been so much change that maybe the next release should be
> > > 22.0.
> > 
> > So in order to avoid version overkill, I think that we would stay at
> > 21.x as long as we have "merely" extensive, but not fundamental
> > changes.
> 
> Well, there are SOME fundamental changes from 21.3 to "this" release:
> 
> ** Leim is now part of the Emacs distribution.

A difference in packaging is not fundamental.

> ** The Emacs Lisp Reference Manual is now part of the distribution.

A difference in packaging is not fundamental.

> ** The max size of buffers and integers has been doubled.

This is just a minor detail for the user.

> ** GTK support

Probably more relevant.  Also image support has been extended to Mac
and Windows, but I consider this more a bugfix than a feature
enhancement: image support never was intended to be X-only.  It was
just a tolerated shortcoming in order to get something out.

> Besides, a lot of external lisp packages are now included with emacs
> (tramp, ses, cua, ...)

A difference in packaging...  but I repeat myself.  It is convenient,
but not fundamental.

> But I don't really case if we call this 21.4, 21.5, or 22.0 -- as
> long as we release something in 2004 !!

Yes.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Luc Teirlinck]

I personally believe that the most logical thing would be to call the
next release "21.4".  The NEWS has always described all changes as to
be made in 21.4.  Skipping a number like "21.5" might confuse people
and make them believe that there was a "21.4" which they somehow
"missed".  If a sysadmin updates to 21.5, I am sure plenty of people
are going to believe there was a 21.4, which just never got updated
to, especially given the large amount of elapsed time.  "22.0" might
be OK, but there is the lack of a really super-significant individual
change.

   > But I don't really case if we call this 21.4, 21.5, or 22.0 -- as
   > long as we release something in 2004 !!

   Yes.

Of course, I would not really mind if it were called "21.5" either,
even though that would be my least favorite choice of the three.

Sincerely,

Luc.

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> I personally believe that the most logical thing would be to call the
> next release "21.4".  The NEWS has always described all changes as to
> be made in 21.4.

The NEWS in CVS is read by developers.  Nobody ever claimed that it
was a realiable document before the release.

> Skipping a number like "21.5" might confuse people and make them
> believe that there was a "21.4" which they somehow "missed".  If a
> sysadmin updates to 21.5, I am sure plenty of people are going to
> believe there was a 21.4, which just never got updated to,

Well, if they believe that, where is the problem?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

On Wed, May 05, 2004 at 12:21:18AM +0200, David Kastrup wrote:
> So in order to avoid version overkill, I think that we would stay at
> 21.x as long as we have "merely" extensive, but not fundamental
> changes.

I agree with this; I think unicode should be 22.0, at least nominally.

-Miles
-- 
`Life is a boundless sea of bitterness'

Re: It is time for a feature freeze (it is NOW or never).

[Robert J. Chassell]

   >From what we have planned right now, immediately after getting the
   next release out, the Unicode branch will get merged and
   consolidated, and the multi tty branch merged as well, if feasible.
   Those are fundamental changes.  

Right.  I think the release that includes these should be called 22.

Anything before that, such as the one currently planned, should stay
in 21.

David Kastrup is right when he says:

   ...  in order to avoid version overkill, I think that we would stay
   at 21.x as long as we have "merely" extensive, but not fundamental
   changes.

-- 
    Robert J. Chassell                         Rattlesnake Enterprises
    As I slowly update it,                     bob@rattlesnake.com
        I rewrite a "What's New" segment for   http://www.rattlesnake.com

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

    I still think this is a bad idea -- it cannot be done without potentially
    destabilizing the trunk which is currently in a very stable state, and
    thus in a pretty unique state of "readiness for release".

We could urge all the pretesters to try out the unicode branch now.
That way we would get a picture of how far it is from readiness.

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

On Sun, Apr 11, 2004 at 11:50:57PM -0400, Richard Stallman wrote:
> We could urge all the pretesters to try out the unicode branch now.
> That way we would get a picture of how far it is from readiness.

I was under the impression that the unicode-2 branch was fairly old (I looked
at a few files, and it seems to have last been updated from the trunk around
last summer), which might make such testing a bit questionable.

I think a merge one direction or another should take place.

BTW, I wonder if it might be a slightly easier to use arch to manage the
unicode branch, at least the merging duties.  I keep my branches up-to-date
by merging regularly from the trunk, and it's quite painless in general (some
of this is that I now have a fair amount of experience with arch, of course
-- but I was pretty good with CVS too, and it was always a pain).

Since unicode-2 was branched before I added taglines, it would take a bit of
preparation, but once done it would be simple for me to keep the unicode
branch synchronized between CVS and arch as I do with the trunk, and the
unicode hackers could continue to use CVS for daily hacking if they wished,
with arch used to keep the unicode branch updated with respect to the CVS
trunk (this latter is probably best done by a unicode hacker, as they would
know how to best resolve merge conflicts).

-Miles
-- 
Ich bin ein Virus. Mach' mit und kopiere mich in Deine .signature.

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

    BTW, I wonder if it might be a slightly easier to use arch to manage the
    unicode branch, at least the merging duties.

Handa is the main developer of that branch, so let's do this whatever
way he prefers.

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <20040412041506.GA31211@fencepost>, Miles Bader <miles@gnu.org> writes:

> On Sun, Apr 11, 2004 at 11:50:57PM -0400, Richard Stallman wrote:
>>  We could urge all the pretesters to try out the unicode branch now.
>>  That way we would get a picture of how far it is from readiness.

> I was under the impression that the unicode-2 branch was fairly old (I looked
> at a few files, and it seems to have last been updated from the trunk around
> last summer), which might make such testing a bit questionable.

> I think a merge one direction or another should take place.

I've just synchronized emacs-unicode-2 branch to the trunk,
i.e, I've done something like this:

% cd .../emacs-trunk
% cvs tag handa-temp-tag
% cd .../emacs-unicode-2
% cvs update -j emcas-unicode-2-base -j handa-temp-tag
% cvs commit -m 'Sync to HEAD'
% cd .../emacs-trunk
% cvs tag -F -r handa-temp-tag emacs-unicode-2

> Since unicode-2 was branched before I added taglines, it would take a bit of
> preparation, but once done it would be simple for me to keep the unicode
> branch synchronized between CVS and arch as I do with the trunk, and the
> unicode hackers could continue to use CVS for daily hacking if they wished,
> with arch used to keep the unicode branch updated with respect to the CVS
> trunk (this latter is probably best done by a unicode hacker, as they would
> know how to best resolve merge conflicts).

So, now all files in emacs-unicode-2 branch have arch tags.
But, I've never used arch.  Could you tell me the exact
procedure to use arch for synchronizing?  Is it surely
easier than the above procedure for CVS?

By the way, the last step above aborted as below.

[...]
T src/s/vms4-4.h
T src/s/vms5-5.h
T src/s/windows95.h
T src/s/xenix.h
cvs server: Tagging vms
T vms/README
T vms/make-mms-derivative.el
cvs [server aborted]: "tag" requires write access to the repository

Could someone tell me what it means?

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

> % cd .../emacs-trunk
> % cvs tag handa-temp-tag
> % cd .../emacs-unicode-2
> % cvs update -j emcas-unicode-2-base -j handa-temp-tag
> % cvs commit -m 'Sync to HEAD'
> % cd .../emacs-trunk
> % cvs tag -F -r handa-temp-tag emacs-unicode-2

Seems right (except for the `emcas' typo and the fact that the last command
should set emacs-unicode-2-base rather than emacs-unicode-2, but I expect
these were copy-errors and you use the right commands ;-).

> So, now all files in emacs-unicode-2 branch have arch tags.
> But, I've never used arch.  Could you tell me the exact
> procedure to use arch for synchronizing?

Well, you'll have to create an Arch branch first.

> Is it surely easier than the above procedure for CVS?

Once the branch is setup, merging is much easier because the management of
emacs-unicode-2-base is basically all done for you.  You'll just do

   tla star-merge --three-way miles@gnu.org--gnu-2004/emacs--cvs-trunk--0

to merge in changes from the cvs-trunk.

> By the way, the last step above aborted as below.
> [...]
> T src/s/vms4-4.h
> T src/s/vms5-5.h
> T src/s/windows95.h
> T src/s/xenix.h
> cvs server: Tagging vms
> T vms/README
> T vms/make-mms-derivative.el
> cvs [server aborted]: "tag" requires write access to the repository
> Could someone tell me what it means?

No idea.  Can you try again?
Since `cvs tag handa-temp-tag' worked, I'd assume that it was just
a transient bug.  Otherwise, it means that some permissions are messed up,
which seems odd.


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <jwvu0zkc7ue.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>  % cd .../emacs-trunk
>>  % cvs tag handa-temp-tag
>>  % cd .../emacs-unicode-2
>>  % cvs update -j emcas-unicode-2-base -j handa-temp-tag
>>  % cvs commit -m 'Sync to HEAD'
>>  % cd .../emacs-trunk
>>  % cvs tag -F -r handa-temp-tag emacs-unicode-2

> Seems right (except for the `emcas' typo and the fact that the last command
> should set emacs-unicode-2-base rather than emacs-unicode-2, but I expect
> these were copy-errors and you use the right commands ;-).

Ah, yes.  They are my typo (I manually typed above, not
copy&past).

>>  So, now all files in emacs-unicode-2 branch have arch tags.
>>  But, I've never used arch.  Could you tell me the exact
>>  procedure to use arch for synchronizing?

> Well, you'll have to create an Arch branch first.

>>  Is it surely easier than the above procedure for CVS?

> Once the branch is setup, merging is much easier because the management of
> emacs-unicode-2-base is basically all done for you.  You'll just do

>    tla star-merge --three-way miles@gnu.org--gnu-2004/emacs--cvs-trunk--0

> to merge in changes from the cvs-trunk.

Thank you.  I don't know what is "Arch branch", how to
create it.  In addition, I couldn't install "tla-1.2" on my
Debian.  I downloaded it from
http://ftp.gnu.org/gnu/gnu-arch/.

The configure script showed this error:
----------------------------------------------------------------------
% ../configure --prefix /usr/local/work/tmp

The configured versions of diff and diff3 do not handle files
not ending in newline correctly.

  configured diff = diff.
  configured diff3 = diff3.


Use config options
  --with-gnu-diff PROG
  --with-gnu-diff3 PROG

  NOTE: especially on BSD-derived systems, you will want
   to grab a recent version of GNU diff and compile it for use
   with those config options.   You don't need to replace
   the native diff tools on your system, but you do need to
   configure tla specially.   Sorry about that -- some BSDs
   have made a poor decision about their native diffs.

   (Example systems: some NetBSD, FreeBSD, and MacOS versions.)
----------------------------------------------------------------------

At least, as I found a manual in it, I'm reading it now.

>>  By the way, the last step above aborted as below.
>>  [...]
>>  T src/s/vms4-4.h
>>  T src/s/vms5-5.h
>>  T src/s/windows95.h
>>  T src/s/xenix.h
>>  cvs server: Tagging vms
>>  T vms/README
>>  T vms/make-mms-derivative.el
>>  cvs [server aborted]: "tag" requires write access to the repository
>>  Could someone tell me what it means?

> No idea.  Can you try again?
> Since `cvs tag handa-temp-tag' worked, I'd assume that it was just
> a transient bug.  Otherwise, it means that some permissions are messed up,
> which seems odd.

I found that "cvs tag handa-temp-tag" also showed this at
the tail.

[...]
T src/s/xenix.h
cvs server: Tagging vms
T vms/README
T vms/make-mms-derivative.el
cvs [server aborted]: "tag" requires write access to the repository

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

> Thank you.  I don't know what is "Arch branch", how to
> create it.  In addition, I couldn't install "tla-1.2" on my
> Debian.  I downloaded it from
> http://ftp.gnu.org/gnu/gnu-arch/.

Have you tried `apt-get install tla' ?  Worked for me (it's in testing,
not in stable).

> The configure script showed this error:
> ----------------------------------------------------------------------
> % ../configure --prefix /usr/local/work/tmp

> The configured versions of diff and diff3 do not handle files
> not ending in newline correctly.

>   configured diff = diff.
>   configured diff3 = diff3.

You need to upgrade your diff program.  GNU diff 2.7 (and non-GNU diffs)
have various bugs related to handling of files without terminating newline.

> I found that "cvs tag handa-temp-tag" also showed this at
> the tail.

> [...]
> T src/s/xenix.h
> cvs server: Tagging vms
> T vms/README
> T vms/make-mms-derivative.el
> cvs [server aborted]: "tag" requires write access to the repository

By any chance, do you have a CVS/Root file somewhere that points to
some other repository ?  Do a

        cat **/CVS/Root

in zsh (or eshell) or

        find . -type d -name CVS --exec cat {}/Root


-- Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <m165byy1p4.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>  Thank you.  I don't know what is "Arch branch", how to
>>  create it.  In addition, I couldn't install "tla-1.2" on my
>>  Debian.  I downloaded it from
>>  http://ftp.gnu.org/gnu/gnu-arch/.

> Have you tried `apt-get install tla' ?  Worked for me (it's in testing,
> not in stable).

It couldn't find the package "tla".  :-(

>>  The configure script showed this error:
>>  ----------------------------------------------------------------------
>>  % ../configure --prefix /usr/local/work/tmp

>>  The configured versions of diff and diff3 do not handle files
>>  not ending in newline correctly.

>>    configured diff = diff.
>>    configured diff3 = diff3.

> You need to upgrade your diff program.  GNU diff 2.7 (and non-GNU diffs)
> have various bugs related to handling of files without terminating newline.

Thank you for the info.

apt-get tells me that `diff' on my machine is the newest
version.  So, I installed diffutils-2.8.1 manually, then tla
was successfully built.

>>  I found that "cvs tag handa-temp-tag" also showed this at
>>  the tail.

>>  [...]
>>  T src/s/xenix.h
>>  cvs server: Tagging vms
>>  T vms/README
>>  T vms/make-mms-derivative.el
>>  cvs [server aborted]: "tag" requires write access to the repository

> By any chance, do you have a CVS/Root file somewhere that points to
> some other repository ?  Do a

>         cat **/CVS/Root

> in zsh (or eshell) or

>         find . -type d -name CVS --exec cat {}/Root

Bingo!  I recalled I built `gnulib' directory and checked
out regex.[ch] in it.  CVS/Root in it is:

:ext:anoncvs@subversions.gnu.org:/cvsroot/gnulib

I'm so sorry for bothering you.

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

>>> Thank you.  I don't know what is "Arch branch", how to
>>> create it.  In addition, I couldn't install "tla-1.2" on my
>>> Debian.  I downloaded it from
>>> http://ftp.gnu.org/gnu/gnu-arch/.
>> Have you tried `apt-get install tla' ?  Worked for me (it's in testing,
>> not in stable).
> It couldn't find the package "tla".  :-(

Do you have `testing' or `unstable' in your sources.list?

> apt-get tells me that `diff' on my machine is the newest version.

That's because you're using `stable'.

   apt-get install diffutils/testing

should get you 2.8.1 or somesuch (provided you have `testing' in your
sources.list).

> So, I installed diffutils-2.8.1 manually, then tla
> was successfully built.

Good.


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <jwvr7uk6pnw.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>>  Have you tried `apt-get install tla' ?  Worked for me (it's in testing,
>>>  not in stable).
>>  It couldn't find the package "tla".  :-(

> Do you have `testing' or `unstable' in your sources.list?

Ah! No.  But I'd like to avoid installing such versions by
apt-get.  I always install testing version in
/usr/local/... by getting source and compiling it.

Anyway, thank you for the tip.

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[Miles Bader]

On Tue, Apr 20, 2004 at 08:44:58AM +0900, Kenichi Handa wrote:
> > Do you have `testing' or `unstable' in your sources.list?
> 
> Ah! No.  But I'd like to avoid installing such versions by
> apt-get.  I always install testing version in
> /usr/local/... by getting source and compiling it.

I wouldn't worry too much about it -- "unstable" is actually quite reliable
generally (failures are almost always of the form "package won't upgrade",
which of course leaves you wait the old version still installed).

"testing" is even better: it's like unstable without the occasional problems
unstable has, albeit a tiny bit older.

My vague rules are:

  * For a personal machine with a fast net connection (so downloading fixes
    is fast), use "unstable" -- the frequent upgrades and quick bugfixes are
    more than worth the occasional (and usually minor) problem.

  * For a personal machine with a slow net connection, or where you don't
    want to do frequent updates, use "testing".  Weighing the (small)
    _potential_ for unstability in "testing" versus the rather old (and often
    buggy) software in "stable", testing almost always wins.

  * For a big department server, where changes can affect many people, maybe
    "stable" is justified -- but even here, you probably want to download
    both package lists, and install some packages from testing as well,
    where the stable version is simply too stupidly old.

-Miles
-- 
`Cars give people wonderful freedom and increase their opportunities.
 But they also destroy the environment, to an extent so drastic that
 they kill all social life' (from _A Pattern Language_)

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <20040419235950.GA12584@fencepost>, Miles Bader <miles@gnu.org> writes:
> I wouldn't worry too much about it -- "unstable" is actually quite reliable
> generally (failures are almost always of the form "package won't upgrade",
> which of course leaves you wait the old version still installed).

> "testing" is even better: it's like unstable without the occasional problems
> unstable has, albeit a tiny bit older.

Thank you for the info.  It seems "testing" is good enough
for my purpose.

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[Stefan Monnier]

>     I still think this is a bad idea -- it cannot be done without potentially
>     destabilizing the trunk which is currently in a very stable state, and
>     thus in a pretty unique state of "readiness for release".

> We could urge all the pretesters to try out the unicode branch now.
> That way we would get a picture of how far it is from readiness.

What's the problem with releasing from the current trunk ASAP instead?
Based on feedback from the few non-latin-only users I know, the trunk's
unicode support is "good enough" and is a major improvement over
Emacs-21.3.

There's no shortage of improvements in the current trunk.  Trying to merge
the unicode branch before the next release would just delay the release of
those improvements.  Maybe it would end up releasing the unicode-branch
a bit earlier, but I'm wondering why the unicode branch should be
considered so much more important than all those other features.
What concrete improvement does it bring to the user that's so valuable
compared to all the rest that's already in the trunk (i.e. utf-8 CJK
support, customizable fringes, many new packages, extra bundled manuals,
the list in NEWS is already much too long).


        Stefan

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Richard Stallman <rms@gnu.org> writes:

>     I still think this is a bad idea -- it cannot be done without potentially
>     destabilizing the trunk which is currently in a very stable state, and
>     thus in a pretty unique state of "readiness for release".
> 
> We could urge all the pretesters to try out the unicode branch now.
> That way we would get a picture of how far it is from readiness.

But the unicode branch is not synchronized with the trunk, so it wouldn't
really tell us anything.

Even Handa-san says he is in favour of not merging unicode before we
have made a major release from trunk.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Kenichi Handa]

In article <m3r7usnbad.fsf@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:

> Richard Stallman <rms@gnu.org> writes:
>>      I still think this is a bad idea -- it cannot be done without potentially
>>      destabilizing the trunk which is currently in a very stable state, and
>>      thus in a pretty unique state of "readiness for release".
>>  
>>  We could urge all the pretesters to try out the unicode branch now.
>>  That way we would get a picture of how far it is from readiness.

> But the unicode branch is not synchronized with the trunk, so it wouldn't
> really tell us anything.

I'm gradually synchronizing unicode branch with the trunk by
the method Stephen taught me (i.e. moving
emacs-unicode-2-base tag), but the progress is slow.

> Even Handa-san says he is in favour of not merging unicode before we
> have made a major release from trunk.

Yes, but, what do you mean by "major release"?  I thought
releasing from the trunk with the version number 21.5 (or
21.4) is a "minor release" because it changes only the minor
version.

---
Ken'ichi HANDA
handa@m17n.org

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Kenichi Handa <handa@m17n.org> writes:

> > Even Handa-san says he is in favour of not merging unicode before we
> > have made a major release from trunk.
> 
> Yes, but, what do you mean by "major release"?  I thought
> releasing from the trunk with the version number 21.5 (or
> 21.4) is a "minor release" because it changes only the minor
> version.

IMO a release from trunk _is_ a major release -- even if we only
increment the minor number...

Sorry to be imprecise.

-- 
Kim F. Storm  http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Lőrentey Károly writes:

> The multi-tty branch changes some internal interfaces which break
> non-UN*X ports of Emacs.  The changes are not complex at all, but I
> can not fix the broken ports without help from Mac/Windows/DOS gurus.
> Because of this, the multi-tty branch is certainly not yet ready for
> merging.

Do you know that the ports are broken with your changes (have you made
the changes to those ports?), or do you just suspect they are broken
because you haven't been able to test your changes.

I usually manage to change this sort of thing (primarily textual,
rather than functional changes) on the W32/DOS/MAC ports even though I
don't use (or even compile on) them myself.

I just make the changes (very carefully), commit my changes, and then
leave it to the maintainers of those ports to verify my changes.  So
far, this procedure hasn't caused much trouble.

> > I have looked briefly at the multi-tty pathces, and although it looks
> > very sensible and systematic in its approach, it touches on many files
> > and interfaces!  
> >
> > Since it probably hasn't received much attention (yet) from the core
> > developers, I would think that there are still some issues which need
> > to be ironed out.  For example , what is the value of window-system
> > variable if you have both a window frame and a non-window frame open
> > at the same time (I haven't looked, so I don't know how this issue has
> > been resolved, but there could be other issues like it).
> 
> The window-system variable is frame-local in the multi-tty branch.
> Actually, I think this is the most important change on Lisp level;
> multi-tty support is otherwise mostly transparent to Lisp code.

Indeed.  

Still, one problem with that variable is that it is no longer safe to
base other (global) settings on the value of that variable.  I think
some defcustoms test this variable to select the default setting.

But I suppose it works ok in practice.

> Of course, the Lisp-level display/frame initialization code has
> changed quite a bit, and a few libraries (first and foremost
> server.el) have been enhanced for multi-tty support.
> 
> > Also, is the documentation on the multi-tty branch updated to reflect
> > changes in interfaces etc. (where it matters).
> 
> I think it matters surprisingly little.  Most of the things I changed
> are (unfortunately) :-) undocumented.  

That is good.

>                                        That said, a few nodes in the
> Emacs manuals need to be updated to reflect the changes.

Have you identified the sections of the emacs and elisp
manuals that need to be changed?

> 
> I think it would be useful if Emacs had display-local variables, and a
> display type that is accessible from Lisp code.  The Elisp manual will
> have to be updated when (if) these features are implemented.

That sounds useful, especially with multi-tty support.

Richard -- is this something we should add?

> I agree that the next release should be from the current HEAD,
> without multi-tty.
> 
> -- 
> Károly
> 
> 

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

    > I think it would be useful if Emacs had display-local variables, and a
    > display type that is accessible from Lisp code.  The Elisp manual will
    > have to be updated when (if) these features are implemented.

    That sounds useful, especially with multi-tty support.

    Richard -- is this something we should add?

If it is necessary, we could add it with the multi-tty support.
However, if we can do without it, let's do without it.  These would
surely not be used very often.  It could be a feature not worth the
effort it costs.

When would you use it?

Re: It is time for a feature freeze (it is NOW or never).

[Kim F. Storm]

Richard Stallman <rms@gnu.org> writes:

>     > I think it would be useful if Emacs had display-local variables, and a
>     > display type that is accessible from Lisp code.  The Elisp manual will
>     > have to be updated when (if) these features are implemented.
> 
>     That sounds useful, especially with multi-tty support.
> 
>     Richard -- is this something we should add?
> 
> If it is necessary, we could add it with the multi-tty support.
> However, if we can do without it, let's do without it.  These would
> surely not be used very often.  It could be a feature not worth the
> effort it costs.
> 
> When would you use it?

I like to have a menu-bar on a window system, but don't like it on
a terminal.  Maybe a display-local variable could do that.

There could also be a need for different keymaps to co-exist.

Some modes may also initialize things differently based on the display
type, e.g. depending on whether images are supported or not.

-- 
Kim F. Storm  http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Lőrentey Károly]

Kim F. Storm <storm@cua.dk> writes:
> Richard Stallman <rms@gnu.org> writes:
>
>>     > I think it would be useful if Emacs had display-local variables, and a
>>     > display type that is accessible from Lisp code.  The Elisp manual will
>>     > have to be updated when (if) these features are implemented.
>> 
>>     That sounds useful, especially with multi-tty support.
>> 
>>     Richard -- is this something we should add?
>> 
>> If it is necessary, we could add it with the multi-tty support.
>> However, if we can do without it, let's do without it.  These would
>> surely not be used very often.  It could be a feature not worth the
>> effort it costs.
>> 
>> When would you use it?
>
> I like to have a menu-bar on a window system, but don't like it on
> a terminal.  Maybe a display-local variable could do that.
>
> There could also be a need for different keymaps to co-exist.

There are a few special display-local variables even in the current
Emacs CVS trunk:

	File: elisp,  Node: Multiple Displays
        [...]
	   A few Lisp variables are "terminal-local"; that is, they have a
	separate binding for each terminal.  The binding in effect at any time
	is the one for the terminal that the currently selected frame belongs
	to.  These variables include `default-minibuffer-frame',
	`defining-kbd-macro', `last-kbd-macro', and `system-key-alist'.  They
	are always terminal-local, and can never be buffer-local (*note
	Buffer-Local Variables::) or frame-local.

(The implementation involves calculating offsets into struct kboard,
and other such fun; see defvar_kboard in lread.c and the DEFVAR_KBOARD
macro.)

My other proposal, making a primitive Lisp type for displays would
greatly simplify a few things, for example implementing support for
two separate Emacs displays on the same device.  I know that sounds
like a stupid thing to do, but I (and judging from the bug reports,
others) frequently want to run more than one separate emacsclient tty
sessions on the same terminal.  (Think classical UNIX shell-centric
vi-style: you start editing a file by `emacsclient foo', remember
something, so you press C-z to get back to the shell.  A few commands
later you need to make a quick edit in another file, so naturally you
type `emacsclient bar', only to find that it fails because Lisp code
can not currently distinguish between two separate displays on the
same device.  It's annoying.)

> Some modes may also initialize things differently based on the display
> type, e.g. depending on whether images are supported or not.

I think that is not a good idea to do in new Lisp code, as the same
buffer may later be displayed on another device, possibly even
simultaneously.

-- 
Károly

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

    I like to have a menu-bar on a window system, but don't like it on
    a terminal.  Maybe a display-local variable could do that.

It would be just as easy to do this job with frame-local variables
(i.e. frame parameters), and they already exist.

    There could also be a need for different keymaps to co-exist.

I don't want to add a feature like this just because maybe
someone could use it sometimes.  Emacs is already quite complex;
let's not add more complexity to its basic mechanisms
unless that is really important.

    Some modes may also initialize things differently based on the display
    type, e.g. depending on whether images are supported or not.

On the contrary-- when the same Emacs can display on both ttys and
graphic displays, it is clearly better to set up the buffer in a way
that can work ok on both kinds of displays.

Re: It is time for a feature freeze (it is NOW or never).

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     I like to have a menu-bar on a window system, but don't like it on
>     a terminal.  Maybe a display-local variable could do that.
> 
> It would be just as easy to do this job with frame-local variables
> (i.e. frame parameters), and they already exist.

I don't see how.  After all, we are not talking about the
_possibility_ of having a menu-bar or not (that's trivial), but about
the default for creation of new frames.  And as far as I can tell,
you can't set different defaults for different displays.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Display-local settings (was: It is time for a feature freeze)

[Stefan Monnier]

> I don't see how.  After all, we are not talking about the
> _possibility_ of having a menu-bar or not (that's trivial), but about
> the default for creation of new frames.  And as far as I can tell,
> you can't set different defaults for different displays.

But it's like mode-local variables.  We don't have those, but we can
simulate them with buffer-local variables.
So frame-local settings are sufficient as long as we get a chance to
dynamically setup those parameters whenever a new frame is created.


        Stefan

Re: Display-local settings (was: It is time for a feature freeze)

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > I don't see how.  After all, we are not talking about the
> > _possibility_ of having a menu-bar or not (that's trivial), but about
> > the default for creation of new frames.  And as far as I can tell,
> > you can't set different defaults for different displays.
> 
> But it's like mode-local variables.  We don't have those, but we can
> simulate them with buffer-local variables.

Almost every mode has a startup hook.

> So frame-local settings are sufficient as long as we get a chance to
> dynamically setup those parameters whenever a new frame is created.

We currently only have

frame-creation-function's value is 
x-create-frame-with-faces

Window-system dependent function to call to create a new frame.
The window system startup file should set this to its frame creation
function, which should take an alist of parameters as its argument.

Defined in `frame'.

[back]

That's hardly sufficient.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Display-local settings (was: It is time for a feature freeze)

[Stefan Monnier]

>> So frame-local settings are sufficient as long as we get a chance to
>> dynamically setup those parameters whenever a new frame is created.

> We currently only have
[...]
> That's hardly sufficient.

True enough.  My only point is that we can get the effect of display-local
settings without complexifying the C code too much: we just need to add
a hook at the right place.


        Stefan

Re: Display-local settings

[Lőrentey Károly]

David Kastrup <dak@gnu.org> writes:
>> So frame-local settings are sufficient as long as we get a chance to
>> dynamically setup those parameters whenever a new frame is created.
>
> We currently only have
>
> frame-creation-function's value is 
> x-create-frame-with-faces
>
> Window-system dependent function to call to create a new frame.
> The window system startup file should set this to its frame creation
> function, which should take an alist of parameters as its argument.
>
> Defined in `frame'.
>
> [back]
>
> That's hardly sufficient.

I think I should mention that the behaviour of make-frame has changed
a bit in the multi-tty branch: `frame-creation-function' has been
replaced by a window system-dependent `frame-creation-function-alist'.
(I know it's an ugly incompatible change--providing an equivalent
compatible solution is on my todo list.)

But changing the frame creation method is perhaps not the best way to
approach this particular problem: if a Lisp package really does need
to do window-system specific frame initialization, then I think it
should simply hook into after-make-frame-functions, like this:

(defun xyzzy-after-make-frame-function (frame)
  (let ((w (window-system frame)))
    (cond
     ((eq w 'x) (xyzzyfy-x-frame frame))
     ((eq w 'nil) (xyzzyfy-tty-frame frame))
     ;; etc.
     (t (error "Can not xyzzyfy frame: window-system %s unknown" w)))))

(add-hook 'after-make-frame-functions 'xyzzy-after-make-frame-function)

Probably most packages only need to set special frame parameters; we
could introduce a simple window-system specific default-frame-alist
mechanism to make that easier.

-- 
Károly

Re: Display-local settings

[Stefan Monnier]

> I think I should mention that the behaviour of make-frame has changed
> a bit in the multi-tty branch: `frame-creation-function' has been
> replaced by a window system-dependent `frame-creation-function-alist'.
> (I know it's an ugly incompatible change--providing an equivalent
> compatible solution is on my todo list.)

I'm not sure how important is compatibility here: I always thought of this
variable as an internal one used to communicate between the C and Elisp
part of the frame management code.

> But changing the frame creation method is perhaps not the best way to
> approach this particular problem: if a Lisp package really does need
> to do window-system specific frame initialization, then I think it
> should simply hook into after-make-frame-functions, like this:

Sounds right.  As long as this is used early enough that modifying the
size/position/etc... does not cause any user-visible
resizing/repositioning/...


        Stefan

Re: Display-local settings

[Lőrentey Károly]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> I think I should mention that the behaviour of make-frame has changed
>> a bit in the multi-tty branch: `frame-creation-function' has been
>> replaced by a window system-dependent `frame-creation-function-alist'.
>> (I know it's an ugly incompatible change--providing an equivalent
>> compatible solution is on my todo list.)
>
> I'm not sure how important is compatibility here: I always thought of this
> variable as an internal one used to communicate between the C and Elisp
> part of the frame management code.

I see; that's good news. :-)

>> But changing the frame creation method is perhaps not the best way to
>> approach this particular problem: if a Lisp package really does need
>> to do window-system specific frame initialization, then I think it
>> should simply hook into after-make-frame-functions, like this:
>
> Sounds right.  As long as this is used early enough that modifying the
> size/position/etc... does not cause any user-visible
> resizing/repositioning/...

Good point; that does not seem to be true for after-make-frame-functions:

(defun my-flicker-test (frame)
  (let ((i 0))
    (while (< (setq i (1+ i)) 20)
      (modify-frame-parameters frame `((top . ,(* 10 i)) (left . ,(* 10 i))))
      (sleep-for 0.01))))

(add-hook 'after-make-frame-functions 'my-flicker-test)
(make-frame-command)

The new window-system-default-frame-alist
support could be easily fixed to prevent flickering, but if packages
need dynamic window-system dependent frame parameter initialization,
then I guess that needs a new hook.

-- 
Károly

Re: It is time for a feature freeze (it is NOW or never).

[Richard Stallman]

    I don't see how.  After all, we are not talking about the
    _possibility_ of having a menu-bar or not (that's trivial), but about
    the default for creation of new frames.

I guess I don't understand your idea--I don't see how display-local
variables would have anything to do with this.

Display-local variables (Re: It is time for a feature freeze)

[Lőrentey Károly]

David Kastrup <dak@gnu.org> writes:
> Richard Stallman <rms@gnu.org> writes:
>
>>     I like to have a menu-bar on a window system, but don't like it on
>>     a terminal.  Maybe a display-local variable could do that.
>> 
>> It would be just as easy to do this job with frame-local variables
>> (i.e. frame parameters), and they already exist.
>
> I don't see how.  After all, we are not talking about the
> _possibility_ of having a menu-bar or not (that's trivial), but about
> the default for creation of new frames.  And as far as I can tell,
> you can't set different defaults for different displays.

I think Richard is right; it seems to be easy to create an interface
for setting different default frame parameters for different displays.
For example, a variable like this would be trivial to implement:

,----
| ;; Enable the menu bar and the toolbar under X, but disable them on tty frames.
| (setq window-system-default-frame-alist
|       '((x   (menu-bar-lines . 1) (tool-bar-lines . 1))
|         (nil (menu-bar-lines . 0) (tool-bar-lines . 0))))
`----

In fact, I found that idea so useful that I have just implemented it :-)
(see patch-150). Apart from the new variable definition, it was a
one-line patch.  What do you think?

I am now convinced that display-local variables are not really
necessary.  What about making displays first-level Lisp objects?  Are
there any objections against that?

-- 
Károly

Re: Display-local variables (Re: It is time for a feature freeze)

[Kim F. Storm]

lorentey@elte.hu (Lőrentey Károly) writes:

> ,----
> | ;; Enable the menu bar and the toolbar under X, but disable them on tty frames.
> | (setq window-system-default-frame-alist
> |       '((x   (menu-bar-lines . 1) (tool-bar-lines . 1))
> |         (nil (menu-bar-lines . 0) (tool-bar-lines . 0))))
> `----
> 
> In fact, I found that idea so useful that I have just implemented it :-)
> (see patch-150). Apart from the new variable definition, it was a
> one-line patch.  What do you think?

Nice idea.

> 
> I am now convinced that display-local variables are not really
> necessary.  What about making displays first-level Lisp objects?  Are
> there any objections against that?

Where is this needed ?


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display-local variables (Re: It is time for a feature freeze)

[Lőrentey Károly]

Kim F. Storm <storm@cua.dk> writes:
>> I am now convinced that display-local variables are not really
>> necessary.  What about making displays first-level Lisp objects?  Are
>> there any objections against that?
>
> Where is this needed ?

First, it would make it easy to implement support for having more than
one display on the same device.  I'd like to be able to have more than
one emacsclient tty sessions on the same terminal.  The only problem is
that Lisp code can not currently distinguish between two displays that
are on the same device.  I think exposing the display list to Lisp
would solve that problem nicely.

Plus, having a display type would make it easier in general to
manipulate displays from Lisp code. E.g. for checking that two frames
are on the same display, I'd like to simply say:

	(eq (frame-display a) (frame-display b))

Do you know an easy way to provide this functionality without
Lisp-level display types?  (Well, I guess I could assign an id number
for each display, but that does not sound like the Emacs Way.
For example, handling deleted displays would be hard to do right.)

-- 
Károly

Re: Display-local variables (Re: It is time for a feature freeze)

[Kim F. Storm]

lorentey@elte.hu (Lőrentey Károly) writes:

> Do you know an easy way to provide this functionality without
> Lisp-level display types?  (Well, I guess I could assign an id number
> for each display, but that does not sound like the Emacs Way.


Well, I don't see that it would be wrong to use e.g. an integer to
identify a display in emacs.  For example, characters and keymaps are
not 1st level lisp objects either.

But in some cases it would probably be nice to identify a display as
e.g. #<display 5 on 1.2.3.4:0.1>.  But you could do (display-name id)
to retrieve that information when needed.

In any case, you must find a way to do this generically for the non-X
ports.  An integer would probably work fine there too?

>                                                              
> For example, handling deleted displays would be hard to do right.)

I don't quite follow.

The functions that operate on a display object vs. display id must
still ensure that DISPLAY_LIVE_P or some such -- probably by
traversing some list of active displays.  Whether it does that by
looking for a display object or a display id doesn't seem to make much
difference to me.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: It is time for a feature freeze (it is NOW or never).

[Lőrentey Károly]

[Resend to list]
Kim F. Storm <storm@cua.dk> writes:
> Lőrentey Károly writes:
>> The multi-tty branch changes some internal interfaces which break
>> non-UN*X ports of Emacs.  The changes are not complex at all, but I
>> can not fix the broken ports without help from Mac/Windows/DOS gurus.
>> Because of this, the multi-tty branch is certainly not yet ready for
>> merging.
>
> Do you know that the ports are broken with your changes (have you made
> the changes to those ports?), or do you just suspect they are broken
> because you haven't been able to test your changes.

I am positive they are broken, and won't even compile.  I did not even
attempt to update them while working, because the branch was initially
very unstable (source reorganization, variable renamings, second
thoughts etc.), and it would have been impossible to keep the ports
that I can not compile in shape.  I decided to update them all in one
go, after the base code stabilized a bit.  The branch is quite stable
now, so I think it is time for me to begin that update.

> I usually manage to change this sort of thing (primarily textual,
> rather than functional changes) on the W32/DOS/MAC ports even though I
> don't use (or even compile on) them myself.
>
> I just make the changes (very carefully), commit my changes, and then
> leave it to the maintainers of those ports to verify my changes.  So
> far, this procedure hasn't caused much trouble.

I am planning to do exactly that. :-)

>> > Since it probably hasn't received much attention (yet) from the core
>> > developers, I would think that there are still some issues which need
>> > to be ironed out.  For example , what is the value of window-system
>> > variable if you have both a window frame and a non-window frame open
>> > at the same time (I haven't looked, so I don't know how this issue has
>> > been resolved, but there could be other issues like it).
>> 
>> The window-system variable is frame-local in the multi-tty branch.
>> Actually, I think this is the most important change on Lisp level;
>> multi-tty support is otherwise mostly transparent to Lisp code.
>
> Indeed.  
>
> Still, one problem with that variable is that it is no longer safe to
> base other (global) settings on the value of that variable.  I think
> some defcustoms test this variable to select the default setting.
>
> But I suppose it works ok in practice.

Yes; in theory, Elisp packages that refer to window-system need to be
updated for multi-tty support, but I found that in practice most of
them work fine without any changes.  Emacs behaves the same way as it
used to until the user actually creates simultaneous tty and X frames,
so the degradation (if any) is graceful.

>>                                        That said, a few nodes in the
>> Emacs manuals need to be updated to reflect the changes.
>
> Have you identified the sections of the emacs and elisp
> manuals that need to be changed?

Not yet.  I will leaf through the manuals sometime and take notes.

-- 
Károly

Re: Compilation to native

[Richard Stallman]

    I think Unicode branch is fairly usable in normal use.  At
    least it's stable enough for my daily work.

Is there any Emacs feature that does not work, to your knowledge,
in that branch?

						 But if it is
    used with third party packages, I think some of them must be
    adjusted for emacs-unicode.

That will always be true, so it is no argument for installation now or
for installation later.  Please don't concern yourself with it.

What matters is that the features included in Emacs itself work.

Re: Compilation to native

[Kenichi Handa]

In article <E1BBO1v-0002HU-JZ@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:
>     I think Unicode branch is fairly usable in normal use.  At
>     least it's stable enough for my daily work.

> Is there any Emacs feature that does not work, to your knowledge,
> in that branch?

No, as far as I know, but, I usually don't use that many
variety of Emacs features.

---
Ken'ichi HANDA
handa@m17n.org

Re: Compilation to native

[Richard Stallman]

    > Is there any Emacs feature that does not work, to your knowledge,
    > in that branch?

    No, as far as I know, but, I usually don't use that many
    variety of Emacs features.

How much have you heard of others' using that branch?

Re: Compilation to native

[Richard Stallman]

    In addtion, in emacs-unicode, CCL is, by default, used only
    for Ethiopic font encoding, and it can easily be changed not
    to use CCL.

    So, I think it's not that worth working on CCL interpreter.

Ok.

Re: Compilation to native

[Juri Linkov]

Kenichi Handa <handa@m17n.org> writes:
> In addtion, in emacs-unicode, CCL is, by default, used only
> for Ethiopic font encoding, and it can easily be changed not
> to use CCL.

I tested the unicode branch a little.  Visiting a UTF-8 file
is very fast: a 20MB file under 2 sec!  (Actually, no wonder if
it's Emacs internal encoding).  Visiting a non-ASCII non-UTF-8
file is faster than on the trunk, but still is too slow:
a 7MB file in 30 sec.  Even though the unicode branch doesn't
use CCL programs, visiting a file in map-encoded charsets
needs more improvement.

Generally, the unicode branch seems stable on GNU/Linux.
I have noticed only a few bugs like, for example, sometimes
displaying a row of ^@^@^@^@^@^@^@^@ characters instead
of file names in dired buffers, and some minor problems,
for example, not finding fonts that the trunk version
is able to find.

Sorry, I can't test it by everyday use because it lacks
new features added to the trunk.  I hope it will be merged
with the trunk soon (after preserving the current trunk
on a separate branch, of course).

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Compilation to native

[Kenichi Handa]

In article <87oepvo4jz.fsf@mail.jurta.org>, Juri Linkov <juri@jurta.org> writes:
> I tested the unicode branch a little.  Visiting a UTF-8 file
> is very fast: a 20MB file under 2 sec!  (Actually, no wonder if
> it's Emacs internal encoding).  Visiting a non-ASCII non-UTF-8
> file is faster than on the trunk, but still is too slow:
> a 7MB file in 30 sec.  Even though the unicode branch doesn't
> use CCL programs, visiting a file in map-encoded charsets
> needs more improvement.

As Emacs-unicode has to map every character in a legacy
charset to Unicode, it's inevitable that such kind of code
version gets slower.  For instance, on reading iso-8859-2
file, Emacs 21 only have to add some leading byte to each
character.  But emacs-unicode has to lookup a table to get a
Unicode character code, and then has to get an UTF-8
byte-sequence for that character code.

> Generally, the unicode branch seems stable on GNU/Linux.
> I have noticed only a few bugs like, for example, sometimes
> displaying a row of ^@^@^@^@^@^@^@^@ characters instead
> of file names in dired buffers, and some minor problems,
> for example, not finding fonts that the trunk version
> is able to find.

Do you remember the name of fonts that emacs-unicode failed
to find?

> Sorry, I can't test it by everyday use because it lacks
> new features added to the trunk.  I hope it will be merged
> with the trunk soon (after preserving the current trunk
> on a separate branch, of course).

I'm now synchronizing emacs-unicode to trunk version
gradually.  But, it will take more days to finish
synchronizing.

---
Ken'ichi HANDA
handa@m17n.org

Re: Compilation to native

[Juri Linkov]

Kenichi Handa <handa@m17n.org> writes:
> As Emacs-unicode has to map every character in a legacy
> charset to Unicode, it's inevitable that such kind of code
> version gets slower.  For instance, on reading iso-8859-2
> file, Emacs 21 only have to add some leading byte to each
> character.  But emacs-unicode has to lookup a table to get a
> Unicode character code, and then has to get an UTF-8
> byte-sequence for that character code.

I don't know how lookups are implemented in emacs-unicode,
but generally lookups would be very fast when implemented
on arrays.  Surely, it requires memory, but for small-sized
alphabets this is acceptable.

>> Generally, the unicode branch seems stable on GNU/Linux.
>> I have noticed only a few bugs like, for example, sometimes
>> displaying a row of ^@^@^@^@^@^@^@^@ characters instead
>> of file names in dired buffers, and some minor problems,
>> for example, not finding fonts that the trunk version
>> is able to find.
>
> Do you remember the name of fonts that emacs-unicode failed
> to find?

I'm not sure this list is the right place to report bugs on the
emacs-unicode branch, but here are the symptoms:

Calling the function

(set-frame-font "-rfx-fixed-medium-r-normal--10-*-*-*-c-60-*-*")

doesn't report an error and doesn't set the correct font either.

Calling the `set-frame-font' after creating a fontset reports an error:

(create-fontset-from-ascii-font "-rfx-fixed-medium-r-normal--10-*-*-*-c-60-*-*")
"-rfx-fixed-medium-r-normal--10-100-75-75-c-60-fontset-iso10646_1"

(set-frame-font "-rfx-fixed-medium-r-normal--10-100-75-75-c-60-fontset-iso10646_1")
Font `-rfx-fixed-medium-r-normal--10-100-75-75-c-60-fontset-iso10646_1' is not defined

Whereas in Emacs 21.3.50 `create-fontset-from-ascii-font' with the
same font name returns some other value:

(create-fontset-from-ascii-font "-rfx-fixed-medium-r-normal--10-*-*-*-c-60-*-*")
"-*-*-medium-r-normal--10-*-*-*-c-60-fontset-iso10646_1_10"



BTW, I also noticed the *Message* buffer often reports such errors:

Error during redisplay: (error Attempt to modify read-only object)

but I don't see where this error manifests itself.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Compilation to native

[Kenichi Handa]

In article <87u0zniciw.fsf@mail.jurta.org>, Juri Linkov <juri@jurta.org> writes:
> Kenichi Handa <handa@m17n.org> writes:
>>  As Emacs-unicode has to map every character in a legacy
>>  charset to Unicode, it's inevitable that such kind of code
>>  version gets slower.  For instance, on reading iso-8859-2
>>  file, Emacs 21 only have to add some leading byte to each
>>  character.  But emacs-unicode has to lookup a table to get a
>>  Unicode character code, and then has to get an UTF-8
>>  byte-sequence for that character code.

> I don't know how lookups are implemented in emacs-unicode,
> but generally lookups would be very fast when implemented
> on arrays.  Surely, it requires memory, but for small-sized
> alphabets this is acceptable.

It's surely fast, but not as first as this code:

  unsigined char byte = *src++;
  *dst++ = _SOME_LEADING_BYTE_;
  *dst++ = byte;

And please consider the procedure for generating UTF-8 byte
sequence from a character code which contains two or more
conditionals.

Anyway, I have some idea to improve the performance, but
don't want to work on it at the moment.  Synchronizing to
the trunk should be done first.

>>>  Generally, the unicode branch seems stable on GNU/Linux.
>>>  I have noticed only a few bugs like, for example, sometimes
>>>  displaying a row of ^@^@^@^@^@^@^@^@ characters instead
>>>  of file names in dired buffers, and some minor problems,
>>>  for example, not finding fonts that the trunk version
>>>  is able to find.
>> 
>>  Do you remember the name of fonts that emacs-unicode failed
>>  to find?

> I'm not sure this list is the right place to report bugs on the
> emacs-unicode branch, but here are the symptoms:

Ok, I'll reply in emacs-unicode mailing list including you
in CC:.

---
Ken'ichi HANDA
handa@m17n.org

Pure strings (Re: Compilation to native)

[Juri Linkov]

Juri Linkov <juri@jurta.org> writes:
> BTW, I also noticed the *Message* buffer often reports such errors:
>
> Error during redisplay: (error Attempt to modify read-only object)

This error occurs when the function `put-text-property' tries to
put text properties on the top menu item "Buffers".  But the string
"Buffers" is read-only.  This is true as for the unicode branch
as well as for the trunk.

For example, with the HEAD of the trunk, started by `emacs -Q',
the following code was evaluated:

*** Welcome to IELM ***  Type (describe-mode) for help.
ELISP> (cadr (nth 3 (global-key-binding [menu-bar])))
"Options"
ELISP> (put-text-property 0 1 'auto-composed t (cadr (nth 3 (global-key-binding [menu-bar]))))
nil
ELISP> (cadr (nth 3 (global-key-binding [menu-bar])))
#("Options" 0 1
  (auto-composed t)
  1 7 nil)

;; so far so good: `put-text-property' puts the `auto-composed' property
;; on all top menu items except "Buffers":

ELISP> (cadr (nth 4 (global-key-binding [menu-bar])))
"Buffers"
ELISP> (put-text-property 0 1 'auto-composed t (cadr (nth 4 (global-key-binding [menu-bar]))))
*** Eval error ***  Attempt to modify read-only object
ELISP> (cadr (nth 4 (global-key-binding [menu-bar])))
"Buffers"

This error is raised on the string "Buffers" by the macro CHECK_IMPURE.

Could someone explain why should this string be "impure"?

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Pure strings (Re: Compilation to native)

[Richard Stallman]

    This error is raised on the string "Buffers" by the macro CHECK_IMPURE.

    Could someone explain why should this string be "impure"?

The problem occurs because the string is pure.  It looks like it
is pure because it comes from menu-bar-update-buffers,
which is in a preloaded file.

Does this patch fix it?

*** menu-bar.el	11 Sep 2003 09:45:49 -0400	1.244
--- menu-bar.el	18 Apr 2004 14:34:39 -0400	
***************
*** 1557,1563 ****
  
  	 (setq buffers-menu (cons 'keymap (cons "Select Buffer" buffers-menu)))
  	 (define-key (current-global-map) [menu-bar buffer]
! 	   (cons "Buffers" buffers-menu)))))
  
  (add-hook 'menu-bar-update-hook 'menu-bar-update-buffers)
  
--- 1557,1563 ----
  
  	 (setq buffers-menu (cons 'keymap (cons "Select Buffer" buffers-menu)))
  	 (define-key (current-global-map) [menu-bar buffer]
! 	   (cons (copy-sequence "Buffers") buffers-menu)))))
  
  (add-hook 'menu-bar-update-hook 'menu-bar-update-buffers)

Re: Pure strings (Re: Compilation to native)

[Kenichi Handa]

In article <E1BFK88-0004ja-Mb@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:
>     This error is raised on the string "Buffers" by the macro CHECK_IMPURE.
>     Could someone explain why should this string be "impure"?

> The problem occurs because the string is pure.  It looks like it
> is pure because it comes from menu-bar-update-buffers,
> which is in a preloaded file.

> Does this patch fix it?

Yes.  Isn't it better to do copy-sequence on the fly, for
instance, in parse_menu_item (in keyboard.c) to avoid the
similar problem for the future?

---
Ken'ichi HANDA
handa@m17n.org

Re: Pure strings (Re: Compilation to native)

[Stefan Monnier]

> Yes.  Isn't it better to do copy-sequence on the fly, for
> instance, in parse_menu_item (in keyboard.c) to avoid the
> similar problem for the future?

Sounds like a costly alternative, constantly allocating new strings.


        Stefan

Re: Pure strings (Re: Compilation to native)

[Kenichi Handa]

In article <jwv1xmk6od9.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>  Yes.  Isn't it better to do copy-sequence on the fly, for
>>  instance, in parse_menu_item (in keyboard.c) to avoid the
>>  similar problem for the future?

> Sounds like a costly alternative, constantly allocating new strings.

What I thought was to change the slot of the string to the
copied one if the string is read-only (as the attached
code).  Then it seems that the count of copying is the same
as the case that we call copy-sequence in
menu-bar-update-buffers.  But, as I'm not familiar with the
code around parse_menu_item, I may be wrong.

---
Ken'ichi HANDA
handa@m17n.org

*** keyboard.c	19 Apr 2004 08:43:56 +0900	1.770
--- keyboard.c	20 Apr 2004 08:08:53 +0900	
***************
*** 7150,7155 ****
--- 7150,7160 ----
    AREF (item_properties, ITEM_PROPERTY_ITEM) = item;
  
    item_string = XCAR (item);
+   if (STRINGP (item_string) && PURE_P (item_string))
+     {
+       item_string = Fcopy_sequence (item_string);
+       XSETCAR (item, item_string);
+     }
  
    start = item;
    item = XCDR (item);

Re: Pure strings (Re: Compilation to native)

[Richard Stallman]

    Yes.  Isn't it better to do copy-sequence on the fly, for
    instance, in parse_menu_item (in keyboard.c) to avoid the
    similar problem for the future?

This problem can only occur in very special circumstances,
when there is a preloaded function that contains the menu
string as a constant.

Re: Pure strings (Re: Compilation to native)

[Kenichi Handa]

In article <E1BFdNK-0003FU-Kd@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:

>     Yes.  Isn't it better to do copy-sequence on the fly, for
>     instance, in parse_menu_item (in keyboard.c) to avoid the
>     similar problem for the future?

> This problem can only occur in very special circumstances,
> when there is a preloaded function that contains the menu
> string as a constant.

Ok, I don't insist on my idea that much.

---
Ken'ichi HANDA
handa@m17n.org

Re: Compilation to native

[Matthew Mundell]

Richard Stallman <rms@gnu.org> writes:

>     (byte-compile 'silly-loop)
>     (silly-loop 100000000)
>     => ("Sat Feb 28 10:06:37 2004" "Sat Feb 28 10:06:53 2004")     ; 16 secs
>
>     (native-compile 'silly-loop)
>     (silly-loop 100000000)
>     => ("Sat Feb 28 10:17:13 2004" "Sat Feb 28 10:17:22 2004")     ; 9 secs
>
> I don't think that a speedup of less than a factor of 2 would be worth
> installing something that might take substantial maintenance effort.

The example may come within a factor of 2 with Fsub1 and Fgtr inlined
instead of called.  This depends on the proportion of time being spent
in set_internal.  I'll try it.

> However, it could be that this test is a bad test and doesn't really
> show the benefit of compilation.  Maybe this loop spends most of the
> time inside current-time-string.  How about trying something purely
> computational?

The loop _is_ purely computational.  The calls to current-time-string
are only to record the time before and after the loop.

(defun silly-loop (n)
  "Return time before and after N iterations of a loop."
  (let ((t1 (current-time-string)))
    (while (> (setq n (1- n))
			  0))
	(list t1 (current-time-string))))

I think that this example favours the native compilation, as the
operations in the loop are relatively short.

>     The changes require NO_UNION_TYPE to be set, USE_LSB_TAG to be clear,
>     and EMACS_INT to be an int.
>
> It will be necessary to remove these restrictions to make it ready to
> install.  However, there's no harm working initially on this case
> and handling other cases later.

The change adds macros to lisp.h to deal with the differences
controlled by these settings.  Adding the other cases should simply
entail completing the macros for them.

Re: Compilation to native

[Richard Stallman]

    The loop _is_ purely computational.

I was fooled by the indentation, which makes it appear
that the call to current-time-string is inside the loop.
See?

    (defun silly-loop (n)
      "Return time before and after N iterations of a loop."
      (let ((t1 (current-time-string)))
	(while (> (setq n (1- n))
			      0))
	    (list t1 (current-time-string))))

Anyway, such a small speedup is not worth the trouble.
Compilation of CCL may be worth while.  Or optimization
of the CCL interpreter may be possible.

Re: Compilation to native

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     The loop _is_ purely computational.
> 
> I was fooled by the indentation, which makes it appear
> that the call to current-time-string is inside the loop.
> See?
> 
>     (defun silly-loop (n)
>       "Return time before and after N iterations of a loop."
>       (let ((t1 (current-time-string)))
> 	(while (> (setq n (1- n))
> 			      0))
> 	    (list t1 (current-time-string))))
> 
> Anyway, such a small speedup is not worth the trouble.

A factor of 2 for a first attempt?  I don't think that is something
one should toss lightly.

I have jobs here where Emacs spends close to two hours formatting some
text in the size of an order of 1MB.  If things could be cut into
half, that would help a lot.  Of course, the solution in the end is
to replace the ad-hoc code by something optimized very thoroughly and
with all O(n^2) elements removed, but don't forget one thing:

Emacs, among other things, is a grand platform for solving problems
ad-hoc, and fast prototypes often happen to be inefficient.

Cutting times in half, if it turns out that this will be more or less
a sustained gain, helps a lot in that ballpark.

> Compilation of CCL may be worth while.  Or optimization of the CCL
> interpreter may be possible.

You bet.  Saving such a text file takes minutes on my machine, which
is plainly ridiculous (600MHz laptop).  It is also disconcerting when
you have to save because your laptop batteries are running out, and
you don't know whether the (computationally intensive) save will
finish before things break down.  Of course, the autosave (which is
done unconverted) should be up to par, but it still is a nuisance if
the box hangs saving.

I also have other things like preparing large Usenet groups where
cutting the time in half would be quite an advantage.  Yes, many
things might probably be done by substituting elaborate algorithms,
but that improves things only at one place.  Overall speedups benefit
everything.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Compilation to native

[Matthew Mundell]

Richard Stallman <rms@gnu.org> writes:

>     The loop _is_ purely computational.
>
> I was fooled by the indentation, which makes it appear
> that the call to current-time-string is inside the loop.
> See?
>
>     (defun silly-loop (n)
>       "Return time before and after N iterations of a loop."
>       (let ((t1 (current-time-string)))
> 	(while (> (setq n (1- n))
> 			      0))
> 	    (list t1 (current-time-string))))
>
> Anyway, such a small speedup is not worth the trouble.

OK, thanks for considering it.

Re: Compilation to native

[Matthew Mundell]

Richard Stallman <rms@gnu.org> writes:

>     The loop _is_ purely computational.
>
> I was fooled by the indentation, which makes it appear
> that the call to current-time-string is inside the loop.
> See?
>
>     (defun silly-loop (n)
>       "Return time before and after N iterations of a loop."
>       (let ((t1 (current-time-string)))
> 	(while (> (setq n (1- n))
> 			      0))
> 	    (list t1 (current-time-string))))
>
> Anyway, such a small speedup is not worth the trouble.
> Compilation of CCL may be worth while.  Or optimization
> of the CCL interpreter may be possible.

This is for the record, at least.  The speedup is a little better with
Fgtr inlined into the native, a few excess memory instructions saved,
and, where possible, objects passed between byte operations using
registers instead of the stack.

Byte compiled:
  ("Tue Mar 30 21:54:05 2004" "Tue Mar 30 21:54:21 2004")  16 s
  ("Tue Mar 30 21:54:26 2004" "Tue Mar 30 21:54:42 2004")  16 s
  ("Tue Mar 30 21:54:45 2004" "Tue Mar 30 21:55:01 2004")  16 s

Compiled from byte code to native:
  ("Tue Mar 30 21:55:43 2004" "Tue Mar 30 21:55:49 2004")  6 s
  ("Tue Mar 30 21:55:51 2004" "Tue Mar 30 21:55:58 2004")  7 s
  ("Tue Mar 30 21:56:01 2004" "Tue Mar 30 21:56:07 2004")  6 s

There is the possibility of further improvement, especially for this
example, by moving Lisp object referencing and setting out of loops.
This will surely be tricky, though, if only because the functions
called by the byte code operations could change these objects.

There are also two extra function calls inside the variable setting
operation (Bvarset), which is used inside the loop in the example.
They're a work around to modify struct bit fields, which can probably
be done quicker in place.

The original figures:

> (silly-loop 100000000)
> => ("Sat Feb 28 10:04:39 2004" "Sat Feb 28 10:05:30 2004")     ; 51 secs
>
> (byte-compile 'silly-loop)
> (silly-loop 100000000)
> => ("Sat Feb 28 10:06:37 2004" "Sat Feb 28 10:06:53 2004")     ; 16 secs
>
> (native-compile 'silly-loop)
> (silly-loop 100000000)
> => ("Sat Feb 28 10:17:13 2004" "Sat Feb 28 10:17:22 2004")     ; 9 secs

Re: Compilation to native

[David Kastrup]

Matthew Mundell <matt@mundell.ukfsn.org> writes:

> Richard Stallman <rms@gnu.org> writes:
> 
> > Anyway, such a small speedup is not worth the trouble.
> > Compilation of CCL may be worth while.  Or optimization of the CCL
> > interpreter may be possible.
> 
> This is for the record, at least.  The speedup is a little better
> with Fgtr inlined into the native, a few excess memory instructions
> saved, and, where possible, objects passed between byte operations
> using registers instead of the stack.
> 
> Byte compiled:
>   ("Tue Mar 30 21:54:05 2004" "Tue Mar 30 21:54:21 2004")  16 s
>   ("Tue Mar 30 21:54:26 2004" "Tue Mar 30 21:54:42 2004")  16 s
>   ("Tue Mar 30 21:54:45 2004" "Tue Mar 30 21:55:01 2004")  16 s
> 
> Compiled from byte code to native:
>   ("Tue Mar 30 21:55:43 2004" "Tue Mar 30 21:55:49 2004")  6 s
>   ("Tue Mar 30 21:55:51 2004" "Tue Mar 30 21:55:58 2004")  7 s
>   ("Tue Mar 30 21:56:01 2004" "Tue Mar 30 21:56:07 2004")  6 s

This sounds impressive, but of course in real-life tasks the amount of
work done by the Lisp interpreter as opposed to C primitives should be
less.  One would have to see the memory impact as well to find out
whether the overall gains in processing speed would remain as
impressive.  One thing that is nice about the byte code is that it is
pretty hard to terminally crash the machine just with Lisp (yes, I am
aware that there are byte code sequences that can cause segfaults).

Perhaps a CVS branch for playing around with the concept some more
would be worthwhile?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Compilation to native

[Matthew Mundell]

David Kastrup <dak@gnu.org> writes:

> Matthew Mundell <matt@mundell.ukfsn.org> writes:
[...]
>
> > This is for the record, at least.  The speedup is a little better
> > with Fgtr inlined into the native, a few excess memory instructions
> > saved, and, where possible, objects passed between byte operations
> > using registers instead of the stack.
> >
> > Byte compiled:
> >   ("Tue Mar 30 21:54:05 2004" "Tue Mar 30 21:54:21 2004")  16 s
> >   ("Tue Mar 30 21:54:26 2004" "Tue Mar 30 21:54:42 2004")  16 s
> >   ("Tue Mar 30 21:54:45 2004" "Tue Mar 30 21:55:01 2004")  16 s
> >
> > Compiled from byte code to native:
> >   ("Tue Mar 30 21:55:43 2004" "Tue Mar 30 21:55:49 2004")  6 s
> >   ("Tue Mar 30 21:55:51 2004" "Tue Mar 30 21:55:58 2004")  7 s
> >   ("Tue Mar 30 21:56:01 2004" "Tue Mar 30 21:56:07 2004")  6 s
>
> This sounds impressive, but of course in real-life tasks the amount of
> work done by the Lisp interpreter as opposed to C primitives should be
> less.  One would have to see the memory impact as well to find out
> whether the overall gains in processing speed would remain as
> impressive.

Well, compiling to native creates a primitive.  Inlining in the
generated code causes a size increase in the same way as inlining does
in the static code.

The dynamic native example is about 1000 bytes.  An equivalent
primitive runs in under a second and is about 200 bytes.

Some of the differences in speed and size will be due to the way the
Lightning macros immediately write out instructions, compared to the c
compiler's use of an intermediate representation.

I think the rest comes down to how much work the dynamic compiler puts
into the compilation, and the information available at compile time
about the context of the Lisp code.  Perhaps passing variable type and
binding information to the compilers will improve the differences.  I
think Juri's reference to the CL declarations suggests this.

Re: Compilation to native

[Juri Linkov]

Matthew Mundell <matt@mundell.ukfsn.org> writes:
> There is the possibility of further improvement, especially for this
> example, by moving Lisp object referencing and setting out of loops.

There are several declaration forms in the CL package
(info "(cl)Declarations") which are intended for optimising
compilers, but currently unused.  They could be used for the
native code compiler.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Compilation to native

[Richard Stallman]

    There are several declaration forms in the CL package
    (info "(cl)Declarations") which are intended for optimising
    compilers, but currently unused.  They could be used for the
    native code compiler.

CL is an optional extension package which is present in
Emacs for the sake of those who want to load it.
It is not part of Emacs Lisp.  It would be erroneous
for the Emacs Lisp compiler to expect people to use CL constructs.

We can consider proposals to add some sort of optimization declaration
construct to the Emacs Lisp language.  Compatibility with Common Lisp
is not a primary design goal for this, but all else being equal
it is better to be compatible than not.

Re: Compilation to native

[Richard Stallman]

    There is the possibility of further improvement, especially for this
    example, by moving Lisp object referencing and setting out of loops.
    This will surely be tricky, though, if only because the functions
    called by the byte code operations could change these objects.

Maybe optimize the bytecode instead.

Re: Compilation to native

[Matthew Mundell]

Richard Stallman <rms@gnu.org> writes:

>     There is the possibility of further improvement, especially for this
>     example, by moving Lisp object referencing and setting out of loops.
>     This will surely be tricky, though, if only because the functions
>     called by the byte code operations could change these objects.
>
> Maybe optimize the bytecode instead.

This seems to be very much like the second item on the byte-opt.el TO
DO list.  The item is as follows.


;; maintain a list of functions known not to access any global variables
;; (actually, give them a 'dynamically-safe property) and then
;;   (let ( v1 v2 ... vM vN ) <...dynamically-safe...> )  ==>
;;   (let ( v1 v2 ... vM ) vN <...dynamically-safe...> )
;; by recursing on this, we might be able to eliminate the entire let.
;; However certain variables should never have their bindings optimized
;; away, because they affect everything.
;;   (put 'debug-on-error 'binding-is-magic t)
;;   (put 'debug-on-abort 'binding-is-magic t)
;;   (put 'debug-on-next-call 'binding-is-magic t)
;;   (put 'inhibit-quit 'binding-is-magic t)
;;   (put 'quit-flag 'binding-is-magic t)
;;   (put 't 'binding-is-magic t)
;;   (put 'nil 'binding-is-magic t)
;; possibly also
;;   (put 'gc-cons-threshold 'binding-is-magic t)
;;   (put 'track-mouse 'binding-is-magic t)
;; others?

Re: Compilation to native

[Matthew Mundell]

Matthew Mundell <matt@mundell.ukfsn.org> writes:

> Richard Stallman <rms@gnu.org> writes:
> > [...]
> > Anyway, such a small speedup is not worth the trouble.
> > Compilation of CCL may be worth while.  Or optimization
> > of the CCL interpreter may be possible.
>
> This is for the record, at least.  The speedup is a little better with
> Fgtr inlined into the native, a few excess memory instructions saved,
> and, where possible, objects passed between byte operations using
> registers instead of the stack.
>
> Byte compiled:
>   ("Tue Mar 30 21:54:05 2004" "Tue Mar 30 21:54:21 2004")  16 s
>   ("Tue Mar 30 21:54:26 2004" "Tue Mar 30 21:54:42 2004")  16 s
>   ("Tue Mar 30 21:54:45 2004" "Tue Mar 30 21:55:01 2004")  16 s
>
> Compiled from byte code to native:
>   ("Tue Mar 30 21:55:43 2004" "Tue Mar 30 21:55:49 2004")  6 s
>   ("Tue Mar 30 21:55:51 2004" "Tue Mar 30 21:55:58 2004")  7 s
>   ("Tue Mar 30 21:56:01 2004" "Tue Mar 30 21:56:07 2004")  6 s
>
> There is the possibility of further improvement, especially for this
> example, by moving Lisp object referencing and setting out of loops.
> This will surely be tricky, though, if only because the functions
> called by the byte code operations could change these objects.

This optimisation more than doubles the speed of the generated
function.  The optimisation adds two new byte operations.  As a start
the new operations always take parameters that are four bytes wide.  I
think the loading of these four bytes is the cause of the slower byte
interpretation.

Byte compiled:
   ("Mon Apr 12 18:03:01 2004" "Mon Apr 12 18:03:20 2004")  19 s
   ("Mon Apr 12 18:03:50 2004" "Mon Apr 12 18:04:09 2004")  19 s

Compiled from byte code to native:
   ("Mon Apr 12 18:05:24 2004" "Mon Apr 12 18:05:26 2004")  2 s
   ("Mon Apr 12 18:05:45 2004" "Mon Apr 12 18:05:48 2004")  3 s

An equivalent primitive runs in under a second.  I think the speed
difference between the primitive and the generated native is largely
due to type checking and type extraction done in the loop of the
generated native.  Perhaps this could be also be optimised out of the
loop.  I think that may be fairly difficult.

> [...]
> The original figures:
>
> > (silly-loop 100000000)
> > => ("Sat Feb 28 10:04:39 2004" "Sat Feb 28 10:05:30 2004")     ; 51 secs
> >
> > (byte-compile 'silly-loop)
> > (silly-loop 100000000)
> > => ("Sat Feb 28 10:06:37 2004" "Sat Feb 28 10:06:53 2004")     ; 16 secs
> >
> > (native-compile 'silly-loop)
> > (silly-loop 100000000)
> > => ("Sat Feb 28 10:17:13 2004" "Sat Feb 28 10:17:22 2004")     ; 9 secs


The test function (aka silly-loop) is below.

(defun test-simple (n)
  "Return time before and after N iterations of a loop."
  (let ((t1 (current-time-string)))
	(while (> (setq n (1- n))
			  0))
	(list t1 (current-time-string))))

Re: Compilation to native

[Stefan Monnier]

> This optimisation more than doubles the speed of the generated
> function.  The optimisation adds two new byte operations.  As a start
> the new operations always take parameters that are four bytes wide.  I
> think the loading of these four bytes is the cause of the slower byte
> interpretation.

I don't understand.  Could you give us more info about what those new byte
ops are and what they're used for (and maybe why you think they make things
slower in the byte-interpreter, tho I should be able to figure that out on
my own at that point).


        Stefan

Re: Compilation to native

[Matthew Mundell]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > This optimisation more than doubles the speed of the generated
> > function.  The optimisation adds two new byte operations.  As a start
> > the new operations always take parameters that are four bytes wide.  I
> > think the loading of these four bytes is the cause of the slower byte
> > interpretation.
>
> I don't understand.  Could you give us more info about what those new byte
> ops are and what they're used for (and maybe why you think they make things
> slower in the byte-interpreter, tho I should be able to figure that out on
> my own at that point).

The optimisation moves variable referencing and setting out of loops.
The Lisp objects are instead referenced onto the stack before the
loop, and set from the stack afterwards.  The new operations are used
inside the loop to access or set an object on the stack.  The
operations both take a parameter which indicates how far below the top
of the stack the object is.  The parameter is the four bytes in the
byte stream which follow the operation byte code.  In the byte
interpreter this means four more loads for each reference and for each
set, on every iteration of the optimised example.  I think this is the
cause of the 3 extra seconds in the interpretation of the optimised
byte code.

The purpose of doing the optimisation was to speed up the native code
generated for the test example.  Always using four bytes was the
quickest way to do it.  Now that it is done I'll try for a better
parameter mechanism.  A possibility is to do it like the varset and
varref operations where the operation byte code identifies the number
of parameter bytes to follow.  I'm guessing that afterwards the byte
interpretation of the test example will be at most a few seconds
faster than before the optimisation.  It'll be interesting to see.

Re: Compilation to native

[Stefan Monnier]

>> > This optimisation more than doubles the speed of the generated
>> > function.  The optimisation adds two new byte operations.  As a start
>> > the new operations always take parameters that are four bytes wide.  I
>> > think the loading of these four bytes is the cause of the slower byte
>> > interpretation.
>> I don't understand.  Could you give us more info about what those new byte
>> ops are and what they're used for (and maybe why you think they make things
>> slower in the byte-interpreter, tho I should be able to figure that out on
>> my own at that point).
> The optimisation moves variable referencing and setting out of loops.

Feel free to use precise words like `varref' or `varset'.  We want/need
technical details here.

> The Lisp objects are instead referenced onto the stack before the
> loop, and set from the stack afterwards.  The new operations are used

You mean you added two byteops: one to read the Nth word on the stack
(counting from the top) and the other to set that word?  I expect you
called them something like `sset' and `sref'?
Sounds generally useful (something similar is used in the `lexical' branch
of Miles, you might want to check that out).


        Stefan

Re: Compilation to native

[Matthew Mundell]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> >> > This optimisation more than doubles the speed of the generated
> >> > function.  The optimisation adds two new byte operations.  As a start
> >> > the new operations always take parameters that are four bytes wide.  I
> >> > think the loading of these four bytes is the cause of the slower byte
> >> > interpretation.
> >> I don't understand.  Could you give us more info about what those new byte
> >> ops are and what they're used for (and maybe why you think they make things
> >> slower in the byte-interpreter, tho I should be able to figure that out on
> >> my own at that point).
> > The optimisation moves variable referencing and setting out of loops.
>
> Feel free to use precise words like `varref' or `varset'.  We want/need
> technical details here.

Thanks.  varrefs and varsets are moved out of loops.

>
> > The Lisp objects are instead referenced onto the stack before the
> > loop, and set from the stack afterwards.  The new operations are used
>
> You mean you added two byteops: one to read the Nth word on the stack
> (counting from the top) and the other to set that word? I expect you
> called them something like `sset' and `sref'?

Yes.  They're called stkref and stkset.  stkref pushes a copy of the
Nth Lisp_Object from the top of the stack onto the top of the stack.
stkset sets the Nth Lisp_Object from the top of the stack.

> Sounds generally useful (something similar is used in the `lexical' branch
> of Miles, you might want to check that out).

I've been verging on checking out his branch, but have been putting
off learning arch, partly because the documentation is excluded from
Debian, mostly because it looks like it'll take a while.

Re: Compilation to native

[Miles Bader]

On Tue, Apr 13, 2004 at 08:03:41PM +0100, Matthew Mundell wrote:
> > Sounds generally useful (something similar is used in the `lexical' branch
> > of Miles, you might want to check that out).
> 
> I've been verging on checking out his branch, but have been putting
> off learning arch, partly because the documentation is excluded from
> Debian, mostly because it looks like it'll take a while.

A simple `check out a tree' usage with tla is only two commands (basically
the same as CVS, but the archive location is set with a command instead of an
envvar/option).

However, I also maintain a CVS branch for this, called `lexbind'.

BTW I noticed that one difference between our commands is that my stack
ref/set ops use the _absolute_ position on the stack (counted from the base),
not a TOS-relative position.

-Miles
-- 
`To alcohol!  The cause of, and solution to,
 all of life's problems' --Homer J. Simpson

Re: Compilation to native

[Miles Bader]

On Tue, Apr 13, 2004 at 05:00:28PM -0400, Miles Bader wrote:
> However, I also maintain a CVS branch for this, called `lexbind'.

BTW, a word of warning: the `lexbind-base' tag in CVS is _not_ accurate -- if
you do a `cvs diff -rlexbind-base -rlexbind' the resulting diff will contain
many months of changes on the trunk too!

-Miles
-- 
P.S.  All information contained in the above letter is false,
      for reasons of military security.

Re: Compilation to native

[Matthew Mundell]

Miles Bader <miles@gnu.org> writes:

> On Tue, Apr 13, 2004 at 08:03:41PM +0100, Matthew Mundell wrote:
> > > Sounds generally useful (something similar is used in the `lexical' branch
> > > of Miles, you might want to check that out).
> >
> > I've been verging on checking out his branch, but have been putting
> > off learning arch, partly because the documentation is excluded from
> > Debian, mostly because it looks like it'll take a while.
>
> A simple `check out a tree' usage with tla is only two commands (basically
> the same as CVS, but the archive location is set with a command instead of an
> envvar/option).

tla my-id "Matthew Mundell <matt@mundell.ukfsn.org>"
tla register-archive miles@gnu.org--gnu-2004 http://sourcecontrol.net/~miles/miles@gnu.org--gnu-2004
tla get miles@gnu.org--gnu-2004/emacs--lexbind--0 emacs-lexbind

That's three!  It seems quite similar to CVS in general.  Racing
through the tutorial did take a while though.

Re: Compilation to native

[Matthew Mundell]

(nobody) writes:

> Miles Bader <miles@gnu.org> writes:
>
> > BTW I noticed that one difference between our commands is that my stack
> > ref/set ops use the _absolute_ position on the stack (counted from the base),
> > not a TOS-relative position.

Yes.  The optimisation could use the absolute versions.  That could
save the byte compiler from having to count the change in stack depth
when replacing the loop's varsets and varrefs.

However, for native compilation the relative positions may be
required, to save having to record the position of the bottom of the
stack.

Re: Compilation to native

[Miles Bader]

On Tue, Apr 13, 2004 at 01:32:37PM -0400, Stefan Monnier wrote:
> You mean you added two byteops: one to read the Nth word on the stack
> (counting from the top) and the other to set that word?  I expect you
> called them something like `sset' and `sref'?
> Sounds generally useful (something similar is used in the `lexical' branch
> of Miles, you might want to check that out).

Yes; here are the new  byte codes in the lexical-binding branch:

   #define Bstack_ref 0
   #define Bstack_set  0262
   #define Bstack_set2 0263
   #define Bvec_ref    0264
   #define Bvec_set    0265
   #define BdiscardN   0266

stack_ref is a `1 byte with embedded constant' opcode; stack_set2 uses a
following 2-byte constant, the rest use a following 1-byte constant (as they
have lisp-level equivalents, unusually large offsets can be handled by
calling an out-of-line function).  discardN uses a hack that allows the TOS
to optionally be preserved.

-Miles
-- 
Love is a snowmobile racing across the tundra.  Suddenly it flips over,
pinning you underneath.  At night the ice weasels come.  --Nietzsche

Re: Compilation to native

[Matthew Mundell]

(nobody) writes:

> Miles Bader <miles@gnu.org> writes:
>
> > On Tue, Apr 13, 2004 at 01:32:37PM -0400, Stefan Monnier wrote:
> > > You mean you added two byteops: one to read the Nth word on the stack
> > > (counting from the top) and the other to set that word?  I expect you
> > > called them something like `sset' and `sref'?
> > > Sounds generally useful (something similar is used in the `lexical' branch
> > > of Miles, you might want to check that out).
> >
> > Yes; here are the new  byte codes in the lexical-binding branch:
> >
> >    #define Bstack_ref 0
> >    #define Bstack_set  0262
> >    #define Bstack_set2 0263
> >    #define Bvec_ref    0264
> >    #define Bvec_set    0265
> >    #define BdiscardN   0266
> >
> > stack_ref is a `1 byte with embedded constant' opcode; stack_set2 uses a
> > following 2-byte constant,
> > the rest use a following 1-byte constant (as they
> > have lisp-level equivalents, unusually large offsets can be handled by
> > calling an out-of-line function).  discardN uses a hack that allows the TOS
> > to optionally be preserved.

Do the 1 and 2 byte constants for stack_ref, stack_set and stack_set2
cater for the maximum size of the stack?  Fbyte_code's maxdepth
parameter is a Lisp_Int, which suggests that the stack can be at least
as large as can be held in 29 bits.  Is there perhaps some other
restriction on the stack size?

Re: Compilation to native

[Stefan Monnier]

> Do the 1 and 2 byte constants for stack_ref, stack_set and stack_set2
> cater for the maximum size of the stack?  Fbyte_code's maxdepth
> parameter is a Lisp_Int, which suggests that the stack can be at least
> as large as can be held in 29 bits.  Is there perhaps some other
> restriction on the stack size?

The "stack" in Fbyte_code is really just the activation frame: every time
you enter a new byte-coded function, a new "stack" is created (allocated on
the C stack via alloca).
So it's extremely unlikely that maxdepth will ever be anywhere near 2^29.


        Stefan

Re: Compilation to native

[Matthew Mundell]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > Do the 1 and 2 byte constants for stack_ref, stack_set and stack_set2
> > cater for the maximum size of the stack?  Fbyte_code's maxdepth
> > parameter is a Lisp_Int, which suggests that the stack can be at least
> > as large as can be held in 29 bits.  Is there perhaps some other
> > restriction on the stack size?
> 
> The "stack" in Fbyte_code is really just the activation frame: every time
> you enter a new byte-coded function, a new "stack" is created (allocated on
> the C stack via alloca).
> So it's extremely unlikely that maxdepth will ever be anywhere near
> 2^29.

OK, so only something like inlined recursion would use those depths,
and even stack_ref's one byte is likely to cover the depths used in
every function.

Re: Compilation to native

[Matthew Mundell]

Matthew Mundell <matt@mundell.ukfsn.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> > > This optimisation more than doubles the speed of the generated
> > > function.  The optimisation adds two new byte operations.  As a start
> > > the new operations always take parameters that are four bytes wide.  I
> > > think the loading of these four bytes is the cause of the slower byte
> > > interpretation.
> >
> > I don't understand.  Could you give us more info about what those new byte
> > ops are and what they're used for (and maybe why you think they make things
> > slower in the byte-interpreter, tho I should be able to figure that out on
> > my own at that point).
>
> The optimisation moves variable referencing and setting out of loops.
> The Lisp objects are instead referenced onto the stack before the
> loop, and set from the stack afterwards.  The new operations are used
> inside the loop to access or set an object on the stack.  The
> operations both take a parameter which indicates how far below the top
> of the stack the object is.  The parameter is the four bytes in the
> byte stream which follow the operation byte code.  In the byte
> interpreter this means four more loads for each reference and for each
> set, on every iteration of the optimised example.  I think this is the
> cause of the 3 extra seconds in the interpretation of the optimised
> byte code.
>
> The purpose of doing the optimisation was to speed up the native code
> generated for the test example.  Always using four bytes was the
> quickest way to do it.  Now that it is done I'll try for a better
> parameter mechanism.  A possibility is to do it like the varset and
> varref operations where the operation byte code identifies the number
> of parameter bytes to follow.  I'm guessing that afterwards the byte
> interpretation of the test example will be at most a few seconds
> faster than before the optimisation.  It'll be interesting to see.

Using varset-like op code parameters for the stkset and stkref
operations makes the byte interpretation of the optimised example a
second faster than usual.  So the loading of the four bytes was the
cause of the extra 3 seconds.