Release update

Release update

[Chong Yidong]

Hi fellas,

Here's a short update on the Emacs 23 release process.  We are getting
pretty close, and currently there are two main things to do before
starting the pretest: adding ruby-mode to the CVS repository, and making
rmail-mbox/pmail ready for use.

The papers for ruby-mode are already in the mail; many thanks to Phil
Hagelberg for helping get them signed.  Hopefully, we'll get the
go-ahead from the FSF to add ruby-mode to the repository within a week
or so.  The version of ruby-mode that we hope to commit to CVS is
currently hosted by Phil at

  http://github.com/technomancy/rinari/tree/master%2Futil%2Fruby-mode.el?raw=true

so take a look and see if you can spot any problems.

Rmail-mbox/pmail is going less well, as RMS does not think the new code
is ready for use.  If any of you have the time, please offer to help
Paul Reilly fix bugs in the pmail code.  Much appreciated.

Hopefully, we are still on track to start pretest sometime in December;
let's see.

Re: Release update

[Yavor Doganov]

At Mon, 01 Dec 2008 21:43:01 -0500,
Chong Yidong wrote:
> 
> We are getting pretty close, and currently there are two main things
> to do before starting the pretest: adding ruby-mode to the CVS
> repository, and making rmail-mbox/pmail ready for use.

I don't know what is the Emacs maintainers' opinion about the GNUstep
port wrt release status/goals, but I'd like to mention that in its
current form it is not usable as a replacement of GTK+/Lucid/nox.
Maybe this is not a problem in general, as probably the main goal of
the port is to make Emacs available for users of the Muck OS X system
(as a replacement of the Carbon port).

I don't see it this way, though, although I realize I'm in the
minority.  As a GNUstep user my goal is to use Emacs.app as a
replacement of the Lucid build (or the GTK+ build with the GTK-Step
GTK theme).  This is not possible now, even with my extremely low
Emacs usage requirements.

Whether this is considered release-critical is up to the Emacs
maintainers to decide, naturally.

The trouble is that if GNUstep users don't find this flavor usable to
run it by default, bugs will not be even discovered, let alone fixed.
Of course, it is impossible to fix all bugs in time for the 23
release, but at least the most critical ones should be nailed.

Here is a short summary of what I consider important for the GNUstep
port (in my capacity as a humble user).  I am still experimenting with
some of the bugs to which Adrian sent useful comments.

* #1333: Emacs.app does not load ~/.emacs
  The consequences of this are a real disaster.  Emacs also doesn't
  inherit the environment, which in many cases makes usage of external
  processes a PITA.

* #984: Emacs.app segfaults on startup with the cairo backend
  The cairo backend is still considered experimental in GNUstep Back,
  although probably 90% of the people are using it, because rendering
  is faster and much more beautiful than art.  Not being able to use
  Emacs with cairo means "downgrading" to art, as it is not possible
  to define the backend on a per-app basis (and that's how it should
  be).  This is a GNUstep problem, I think.

* #620: Bootstrapping with the GNUstep port impossible
  Distro unfriendliness.  This is not a problem for regular users, as
  released Emacs already contains byte-compiled files, and there is an
  easy workaround for bootstrapping anyway.  However, nowadays the
  majority of the users prefer distro-provided packages.  Among the
  distros who care about GNUstep and ship it, Emacs.app currently
  cannot be provided for Gentoo because of this, and for Debian
  (gNewSense actually) I am doing a horrible hack/workaround that 
  no sane (Debian) maintainer would upload as an official package.
  As a result, I suspect that at the end this port will have even less
  testing.

* Some GNU Coding Standards violations that I'm going to report soon
  (with patches, eventually).  The problem, basically, is that the
  Emacs.app build system breaks certain user expectations that were in
  place since about forever.

I hope that even intrusive patches for the GNUstep port will be
accepted in the pretest period.

Re: Release update

[Ted Zlatanov]

On Thu, 04 Dec 2008 19:08:29 +0200 Yavor Doganov <yavor@gnu.org> wrote: 

YD> Here is a short summary of what I consider important for the GNUstep
YD> port (in my capacity as a humble user).  I am still experimenting with
YD> some of the bugs to which Adrian sent useful comments.

I wanted to add:

- Emacs.app on Mac OS X crashes at random times and the error is not
  logged and can't be debugged.  This needs to be fixed or it's simply
  not possible to produce a viable environment for real work.

Because of these random crashes and some other issues I've reported,
I've had to switch to the X11 Emacs in Mac OS X for coding and Gnus
(temporarily, I hope, for its fonts are nasty and it's slow).

Thanks
Ted

Re: Release update

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> - Emacs.app on Mac OS X crashes at random times and the error is not
>   logged and can't be debugged.  This needs to be fixed or it's simply
>   not possible to produce a viable environment for real work.

Have you tried running Emacs in a debugger?

Adrian, have you observed these crashes?  Do you have any idea what
causes them?

Re: Release update

[Stefan Monnier]

> I don't know what is the Emacs maintainers' opinion about the GNUstep
> port wrt release status/goals, but I'd like to mention that in its
> current form it is not usable as a replacement of GTK+/Lucid/nox.
> Maybe this is not a problem in general, as probably the main goal of
> the port is to make Emacs available for users of the Muck OS X system
> (as a replacement of the Carbon port).

The GNUstep port is not release-critical for Emacs-23.1.  I do think
it's an important target, so it might become release-ciritcal for
Emacs-23.2, but I just don't think it's worth delaying a release for it,
given its current status.

> * #1333: Emacs.app does not load ~/.emacs
>   The consequences of this are a real disaster.  Emacs also doesn't
>   inherit the environment, which in many cases makes usage of external
>   processes a PITA.

I have the nagging feeling that this is linked to the
CANNOT_DUMP problem.  So it's probably better to spend time on getting
DUMP to work.

> * #984: Emacs.app segfaults on startup with the cairo backend
>   The cairo backend is still considered experimental in GNUstep Back,
>   although probably 90% of the people are using it, because rendering
>   is faster and much more beautiful than art.  Not being able to use
>   Emacs with cairo means "downgrading" to art, as it is not possible
>   to define the backend on a per-app basis (and that's how it should
>   be).  This is a GNUstep problem, I think.

No idea about this.

> * Some GNU Coding Standards violations that I'm going to report soon
>   (with patches, eventually).  The problem, basically, is that the
>   Emacs.app build system breaks certain user expectations that were in
>   place since about forever.

Awaiting your patches.

> I hope that even intrusive patches for the GNUstep port will be
> accepted in the pretest period.

Patches that only affect GNUstep can be about as intrusive as they want,
yes (as long as they move forward, obviously), since the port is barely
usable anyway.


        Stefan

Re: Release update

[Dan Nicolaescu]

Yavor Doganov <yavor@gnu.org> writes:

  > At Mon, 01 Dec 2008 21:43:01 -0500,
  > Chong Yidong wrote:
  > > 
  > > We are getting pretty close, and currently there are two main things
  > > to do before starting the pretest: adding ruby-mode to the CVS
  > > repository, and making rmail-mbox/pmail ready for use.
  > 
  > I don't know what is the Emacs maintainers' opinion about the GNUstep
  > port wrt release status/goals, but I'd like to mention that in its
  > current form it is not usable as a replacement of GTK+/Lucid/nox.
  > Maybe this is not a problem in general, as probably the main goal of
  > the port is to make Emacs available for users of the Muck OS X system
  > (as a replacement of the Carbon port).
  > 
  > I don't see it this way, though, although I realize I'm in the
  > minority.  As a GNUstep user my goal is to use Emacs.app as a
  > replacement of the Lucid build (or the GTK+ build with the GTK-Step
  > GTK theme).  This is not possible now, even with my extremely low
  > Emacs usage requirements.
  > 
  > Whether this is considered release-critical is up to the Emacs
  > maintainers to decide, naturally.

At the time the nextstep code was checked in Stefan said that it won't
delay a release.  And it probably was a good decision, although A LOT of
people report bugs, unfortunately only the maintainer works on fixing
them.  Not sure why it's so hard to attract developers...

Re: Release update

[Ted Zlatanov]

On Thu, 04 Dec 2008 14:29:06 -0500 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> Ted Zlatanov <tzz@lifelogs.com> writes:
>> - Emacs.app on Mac OS X crashes at random times and the error is not
>> logged and can't be debugged.  This needs to be fixed or it's simply
>> not possible to produce a viable environment for real work.

CY> Have you tried running Emacs in a debugger?

Yes, unfortunately it didn't help.

CY> Adrian, have you observed these crashes?  Do you have any idea what
CY> causes them?

I reported the issues to emacs-devel and the Emacs.app mailing list in
<m2y702mdfw.fsf@getconnected.com>

Adrian replied only to the Emacs.app mailing list with
<AE3F6761-560A-4113-A84F-BB2D5781BCCD@gmail.com> and the thread stayed
there.  I can summarize here if it's necessary, although Adrian can
probably do a better job.

Ted

Re: Release update

[Dan Nicolaescu]

Chong Yidong <cyd@stupidchicken.com> writes:

  > Hi fellas,
  > 
  > Here's a short update on the Emacs 23 release process.  We are getting

May I suggest that writing a release announcement be part of the release
process?  Something short that summarizes the most important features
would be good to give people reasons to look further in NEWS and to
upgrade, and it's good for PR.  Not everyone wants to read the 2000
lines NEWS file...

Re: Release update

[Chong Yidong]

Dan Nicolaescu <dann@ics.uci.edu> writes:

> May I suggest that writing a release announcement be part of the
> release process?  Something short that summarizes the most important
> features would be good to give people reasons to look further in NEWS
> and to upgrade, and it's good for PR.  Not everyone wants to read the
> 2000 lines NEWS file...

We do include a feature summary in the release announcement; see, for
instance, the 21.1 and 22.1 release announcements.  This is written up
shortly before the release.  We also put a feature summary on the
webpage.

Re: Release update

[Dan Nicolaescu]

Chong Yidong <cyd@stupidchicken.com> writes:

  > Dan Nicolaescu <dann@ics.uci.edu> writes:
  > 
  > > May I suggest that writing a release announcement be part of the
  > > release process?  Something short that summarizes the most important
  > > features would be good to give people reasons to look further in NEWS
  > > and to upgrade, and it's good for PR.  Not everyone wants to read the
  > > 2000 lines NEWS file...
  > 
  > We do include a feature summary in the release announcement; see, for
  > instance, the 21.1 and 22.1 release announcements.  This is written up

The 22.1 announcement is the one I had in mind, and it could be improved
a lot.  Something like:

   - New modes and packages, including Calc, Grep, TRAMP, URL, IDO,
     CUA, ERC, rcirc, Table, Image-Dired, SES, Ruler, Org, PGG,
     Flymake, Password, Printing, Reveal, wdired, t-mouse,
     longlines, savehist, Conf mode, Python mode, DNS mode, etc.

   - Abbrev definitions are read automatically at startup.


is not too helpful for a lot of users.  IMHO the announcement should be
readable for people that are not intimately familiar with the emacs internals.

Re: Release update

[Adrian Robert]

On Dec 4, 2008, at 2:29 PM, Chong Yidong wrote:

> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>> - Emacs.app on Mac OS X crashes at random times and the error is not
>>  logged and can't be debugged.  This needs to be fixed or it's simply
>>  not possible to produce a viable environment for real work.
>
> Have you tried running Emacs in a debugger?
>
> Adrian, have you observed these crashes?  Do you have any idea what
> causes them?

I use Emacs.app heavily on a daily basis on 10.4 and 10.5 for English  
and Chinese text editing and program editing/compilation and don't get  
crashes.  However I don't use more complex modes like gud, gnus, and  
image-viewing.  (Though I was using ecb for a while.)  I don't know  
what the problem would be but if it is intermittent all I can suggest  
is building with no optimization, just -g, to get a better stack  
trace, and running from gdb (cd emacs/src ; gdb ../nextstep/Emacs.app/ 
Contents/MacOS/Emacs).

Re: Release update

[Randal L. Schwartz]

>>>>> "Ted" == Ted Zlatanov <tzz@lifelogs.com> writes:

Ted> - Emacs.app on Mac OS X crashes at random times and the error is not
Ted>   logged and can't be debugged.  This needs to be fixed or it's simply
Ted>   not possible to produce a viable environment for real work.

And has incremental building on "make install" been fixed?  Last I checked
a few weeks ago, it still didn't work.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

Re: Release update

[Randal L. Schwartz]

>>>>> "Randal" == Randal L Schwartz <merlyn@stonehenge.com> writes:

Randal> And has incremental building on "make install" been fixed?  Last I checked
Randal> a few weeks ago, it still didn't work.

Yup.  Just verified.  Still broken.  Repeat by:

make bootstrap install # works
touch src/*.m # to force some changes
make install # fails to build a working .app, since things like .app/etc are broken


-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

Re: Release update

[Richard M Stallman]

Are people saying that Emacs works better with MacOS than with
GNUstep?  That is not good.  But in order to change this, we need some
GNUstep users to fix the bugs.  Can we recruit some to help, perhaps
by posting on some GNUstep list?

Re: Re: Release update

[richardeng]

Because my PC is slow, I've used WindowMaker(GNUStep) about 4 years.
What kind of skills is needed to solve bugs under GNUStep(module name in bug description is NS?)
It seem that only bugs related to Emacs.app belongs to NS, right?

It's better, if somebody can provide some brief introduction.

Emacs on GNUstep (was: Release update)

[Stefan Monnier]

> Because my PC is slow, I've used WindowMaker(GNUStep) about 4 years.
> What kind of skills is needed to solve bugs under GNUStep(module name
> in bug description is NS?)  It seem that only bugs related to
> Emacs.app belongs to NS, right?

IIUC the main problem with it right now is that it cannot be dumped.
I.e. we cannot perform the compilation step where we load `temacs', then
all the core Emacs files, and then dump the result into a new executable
`emacs'.
This means that Emacs/GNUstep has to load all the fils like subr.el,
simple.el, text-mode.el, ... over and over again at each startup, and it
has several other nasty consequences.

Fixing this requires someone who can delve into the details of how
GNUstep binaries are layed out and how the ObjC runtime is linked and
initialized and how all this interacts.


        Stefan

Emacs on GNUstep (was: Release update)

[Stephen J. Turnbull]

Stefan Monnier writes:

 > Fixing this requires someone who can delve into the details of how
 > GNUstep binaries are layed out and how the ObjC runtime is linked and
 > initialized and how all this interacts.

Or a portable dumper which reloads the Lisp environemnt into a new
malloc'd area, and doesn't care about all that.

Re: Emacs on GNUstep (was: Release update)

[richardeng]

Stephen J. Turnbull wrote:

>>Stefan Monnier writes:
>> Fixing this requires someone who can delve into the details of how
>> GNUstep binaries are layed out and how the ObjC runtime is linked and
>> initialized and how all this interacts.

> Or a portable dumper which reloads the Lisp environemnt into a new
> malloc'd area, and doesn't care about all that.

It doesn't seem a easy job.
I always "./configure && make && make install" in GNUStep.
At now, I don't know the difference between with/without --with-ns
I'll try "./configure --with-ns" to reproduce #620(Bootstrapping with the GNUstep port impossible)

Re: Emacs on GNUstep (was: Release update)

[Stephen J. Turnbull]

richardeng writes:

 > > Or a portable dumper which reloads the Lisp environemnt into a new
 > > malloc'd area, and doesn't care about all that.
 > 
 > It doesn't seem a easy job.

It isn't, and I don't know if the one that has been written for Emacs
would serve as well as the one XEmacs has; Emacs might need to start
from scratch.

I'm just reminding Stefan that there's a long-term solution with lots
of benefits that go beyond making GNUStep a first-class citizen of the
community.

Re: Re: Emacs on GNUstep (was: Release update)

[richardeng]

Stephen J. Turnbull writes:

>richardeng writes:

>> > Or a portable dumper which reloads the Lisp environemnt into a new
> > > malloc'd area, and doesn't care about all that.
> > 
> > It doesn't seem a easy job.

> It isn't, and I don't know if the one that has been written for Emacs
> would serve as well as the one XEmacs has; Emacs might need to start
> from scratch.

> I'm just reminding Stefan that there's a long-term solution with lots
> of benefits that go beyond making GNUStep a first-class citizen of the
> community.

I spent some time on it. Too sad, I was blocked on(apt-get install what-package?): 
checking AppKit/AppKit.h usability... no
checking AppKit/AppKit.h presence... no
checking for AppKit/AppKit.h... no
configure: error: `--with-ns' was specified, but the include

Maybe, apple users should pay more attention on NS release. 
I feel emacs configured without --with-ns well. 
If build without --with-ns, emacs won't work on MacOS GUI? I guess yes.

Re: Emacs on GNUstep

[Chong Yidong]

"richardeng" <richardeng@foxmail.com> writes:

> I spent some time on it. Too sad, I was blocked on(apt-get install what-package?): 
> checking AppKit/AppKit.h usability... no
> checking AppKit/AppKit.h presence... no
> checking for AppKit/AppKit.h... no
> configure: error: `--with-ns' was specified, but the include

You need gnustep-devel and libgnustep-gui-dev.

Re: Release update

[Ted Zlatanov]

On Thu, 4 Dec 2008 17:55:59 -0500 Adrian Robert <adrian.b.robert@gmail.com> wrote: 

AR> On Dec 4, 2008, at 2:29 PM, Chong Yidong wrote:

>> Ted Zlatanov <tzz@lifelogs.com> writes:
>> 
>>> - Emacs.app on Mac OS X crashes at random times and the error is not
>>> logged and can't be debugged.  This needs to be fixed or it's simply
>>> not possible to produce a viable environment for real work.
>> 
>> Have you tried running Emacs in a debugger?
>> 
>> Adrian, have you observed these crashes?  Do you have any idea what
>> causes them?

AR> I use Emacs.app heavily on a daily basis on 10.4 and 10.5 for English
AR> and Chinese text editing and program editing/compilation and don't get
AR> crashes.  However I don't use more complex modes like gud, gnus, and
AR> image-viewing.  (Though I was using ecb for a while.)  I don't know
AR> what the problem would be but if it is intermittent all I can suggest
AR> is building with no optimization, just -g, to get a better stack
AR> trace, and running from gdb (cd emacs/src ; gdb ../nextstep/Emacs.app/
AR> Contents/MacOS/Emacs).

I ran Emacs.app for a day under the debugger, using Gnus and other
tools, and it worked all right without crashes.  It may be a difference
between my command-line setup and starting it from the MacOS Finder.
I'll keep testing and write back if I find anything useful.

The part where "the error can't be logged or debugged" is the bigger
concern IMO.  Adrian, what needs to happen to have the Emacs.app bundle
(the one users will open) produce usable core files and log messages?
I'll help any way I can.

Ted

Release update

[Chong Yidong]

A quick note about the release.  Currently, the main thing we are
waiting for before we can begin the pretest is a copyright assignment
from one of the previous contributors to the Nextstep port (he seems to
have gone incommunicado, but we are trying to get back in contact with
him).

The pmail/rmail-mbox code will not be included in the Emacs 23.1
release.  However, I encourage everyone to test the code if possible,
because it will probably replace rmail in CVS as soon as 23.1 is
released.

In the meantime, let's continue fixing the open bugs.  Thanks.

Re: Release update

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Mon, 22 Dec 2008 09:32:53 -0500
> 
> A quick note about the release.  Currently, the main thing we are
> waiting for before we can begin the pretest is a copyright assignment
> from one of the previous contributors to the Nextstep port (he seems to
> have gone incommunicado, but we are trying to get back in contact with
> him).

What about proofreading the documentation?  The manuals update
according to NEWS is almost done (see below), but I'm quite sure that
some changes are not in there, and that some of the docs is outdated.
Without proofreading all the 3 manuals in their entirety, I don't see
how we can ensure these inaccuracies are fixed.

If that is supposed to be part of the pretest, either it will be a
very long pretest or we will not be able to finish the review in time
for the release.

Also, the multi-tty and the new font backend are still not documented
in the ELisp manual.  I'm (slowly) working on the former, and will
probably get it done soon enough, but I can surely use some help with
the latter.

Re: Release update

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Mon, 22 Dec 2008 09:32:53 -0500
> 
> A quick note about the release.  Currently, the main thing we are
> waiting for before we can begin the pretest is a copyright assignment
> from one of the previous contributors to the Nextstep port (he seems to
> have gone incommunicado, but we are trying to get back in contact with
> him).

Also, what about bug #970?  I don't think we can release Emacs that
messes up the tty display like that.

RE: Release update

[Drew Adams]

> Also, what about bug #970?  I don't think we can release Emacs that
> messes up the tty display like that.

If we're reminding about specific bugs and talking about possible release, the
following two bugs are very long-standing. They are regressions specific to
Emacs 23.

#117 -

* Fringe (on first frame only), when no-fringe was specified.

* Tool bar, when no-tool-bar was specified.

* Duplicate and contradictory frame parameters.

* Frames not redisplayed until manual C-l. Frame goes blank
  when another frame or window-mgr window is dragged over it
  or otherwise obscures it - and frame stays blank until C-l.

#1562 aka #119 -

* modify-frame-parameters, given a `font' parameter argument ending
  with "-iso8859-1" changes the `font' frame parameter to a font
  name that ends with "-fontset-auto1" (where `1' seems to be an
  index of the number of such calls, so the second uses `auto2'...).

In both cases, I sent code to reproduce the problem.

Those bugs are, to me, quite major - they (esp. #117) make Emacs 23 unusable for
me, sad to say. And there are of course other open bugs.

Another noticeable problem is the time it takes Emacs to start up. I know that
people have tried to improve this, but for me this startup time has not changed,
as of this build:

In GNU Emacs 23.0.60.1 (i386-mingw-nt5.1.2600)
 of 2008-12-19 on LENNART-69DE564
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4) --no-opt --cflags -Ic:/g/include
-fno-crossjumping'

Whether I use my own setup of a standalone minibuffer frame and one other,
principal frame or I use emacs -Q, the startup time is about the same. And
that's true whether I start by editing a file or Dired or just start with
*scratch*.

With my own setup, it takes on average 15-20 seconds for the initial frame to
appear, and another 10 seconds for my two-frame setup to finish. emacs -Q takes
15-20 seconds on average for the frame to appear. The command I use is this:
runemacs.exe -Q --debug-init. This is on a 1-year-old laptop with 2G RAM and
pretty good processors.

The startup time is variable, for some reason. Sometimes it is much less (as if
something were cached), sometimes it is a little more. Usually it is about 15-20
sec.

To compare: for my two-frame setup, Emacs 22.3 takes about 4 sec total, and
Emacs 22.3 -Q takes less than a second. (These are all wall-clock durations.)

Personally, the long startup time does not prevent me from using Emacs 23. The
bugs do.

I don't know whether this kind of performance and these kinds of bugs are normal
for a pretest. Perhaps pretesting will lead to fixes. But I hope Emacs 23 will
not be released with these problems.

Re: Release update

[Leo]

On 2008-12-22 23:04 +0000, Drew Adams wrote:
> With my own setup, it takes on average 15-20 seconds for the initial
> frame to appear, and another 10 seconds for my two-frame setup to
> finish. emacs -Q takes 15-20 seconds on average for the frame to
> appear. The command I use is this: runemacs.exe -Q --debug-init. This
> is on a 1-year-old laptop with 2G RAM and pretty good processors.

Do you still experience the slowness with the one from:
http://code.google.com/p/emacs-for-windows/?

-- 
.:  Leo  :.  [ sdl.web AT gmail.com ]  .: I use Emacs :.

Re: Release update

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

> What about proofreading the documentation?  The manuals update
> according to NEWS is almost done (see below), but I'm quite sure that
> some changes are not in there, and that some of the docs is outdated.
> Without proofreading all the 3 manuals in their entirety, I don't see
> how we can ensure these inaccuracies are fixed.
>
> If that is supposed to be part of the pretest, either it will be a
> very long pretest or we will not be able to finish the review in time
> for the release.

I think it is OK to perform the documentation improvements/proofreading
in parallel with the pretest.  For one thing, progress will probably
speed up as the number of serious bugs and other issues decreases,
freeing up developer time.

We won't make the final release without a review.

Re: Release update

[Chong Yidong]

"Drew Adams" <drew.adams@oracle.com> writes:

> #117 -
>
> * Fringe (on first frame only), when no-fringe was specified.
>
> * Tool bar, when no-tool-bar was specified.
>
> * Duplicate and contradictory frame parameters.
>
> * Frames not redisplayed until manual C-l. Frame goes blank
>   when another frame or window-mgr window is dragged over it
>   or otherwise obscures it - and frame stays blank until C-l.

A good way to get this bug fixed is to try and reduce the size of your
testcase.

Re: Release update

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

> Also, what about bug #970?  I don't think we can release Emacs that
> messes up the tty display like that.

Handa-san, could you take a look at this bug?

http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=970

RE: Release update

[Drew Adams]

> > With my own setup, it takes on average 15-20 seconds for the initial
> > frame to appear, and another 10 seconds for my two-frame setup to
> > finish. emacs -Q takes 15-20 seconds on average for the frame to
> > appear. The command I use is this: runemacs.exe -Q 
> --debug-init. This
> > is on a 1-year-old laptop with 2G RAM and pretty good processors.
> 
> Do you still experience the slowness with the one from:
> http://code.google.com/p/emacs-for-windows/?

Yes, unfortunately. 18 sec for emacs -Q.

But, as I said, the time is variable. Testing only emacs -Q:
First time: 18 sec. 
Second time, < 1 sec. 
Third time, 16 sec. 
Fourth time, < 1 sec. 
Fifth time, < 1 sec. 
Sixth time, 15 sec. 
Seventh time, 15 sec. 
Eighth time, 1 sec.
Ninth time, 16 sec.
Tenth time, 1 sec

That's exactly the behavior I see in Lennart's vanilla builds. It either takes
quite a long time or it comes up immediately. I was thinking that it usually
takes a long time, but trying now several times in a row, it looks like it's
about 50-50. Same thing for Lennart's build.

I'm not running anything else at the moment, besides a mail client and Web
browser, and during this test no mail went in or out (and no mail polling
AFAICT), and I didn't use the browser (static Web page).

Re: Release update

[Lennart Borgman]

On Tue, Dec 23, 2008 at 1:17 AM, Drew Adams <drew.adams@oracle.com> wrote:
> That's exactly the behavior I see in Lennart's vanilla builds. It either takes
> quite a long time or it comes up immediately. I was thinking that it usually
> takes a long time, but trying now several times in a row, it looks like it's
> about 50-50. Same thing for Lennart's build.

FYI I do not see anything like this. The first time it takes a little
bit longer, but then it is very quick. My pc is probably slower.

Something with disk access, caching? Page file size? (Could that
really matter in this situation??) My page file is 1.5GB and 3GB max
(approx). But, hm, the time delay must be too long for such things.
Some network problem?

Re: Release update

[Kenichi Handa]

In article <87prjjzryu.fsf@cyd.mit.edu>, Chong Yidong <cyd@stupidchicken.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
> > Also, what about bug #970?  I don't think we can release Emacs that
> > messes up the tty display like that.

> Handa-san, could you take a look at this bug?

> http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=970

Ok, I'll check what is wrong.  But, I can't reproduce the
following problem:

> Type C-n several times, and you will see some very strange behavior:
> for example, some lines are skipped and point never enters them.

---
Kenichi Handa
handa@m17n.org

RE: Release update

[Drew Adams]

> > That's exactly the behavior I see in Lennart's vanilla 
> > builds. It either takes quite a long time or it comes up
> > immediately. I was thinking that it usually
> > takes a long time, but trying now several times in a row, 
> > it looks like it's about 50-50. Same thing for Lennart's build.
> 
> FYI I do not see anything like this. The first time it takes a little
> bit longer, but then it is very quick. My pc is probably slower.
> 
> Something with disk access, caching? Page file size? (Could that
> really matter in this situation??) My page file is 1.5GB and 3GB max
> (approx). But, hm, the time delay must be too long for such things.
> Some network problem?

I'm working locally - no network (but Internet connection).
Page file is 2G (max 4G); single hard disk. RAM is 2G.

RE: Release update

[Drew Adams]

> > > That's exactly the behavior I see in Lennart's vanilla 
> > > builds. It either takes quite a long time or it comes up
> > > immediately. I was thinking that it usually
> > > takes a long time, but trying now several times in a row, 
> > > it looks like it's about 50-50. Same thing for Lennart's build.
> > 
> > FYI I do not see anything like this. The first time it 
> takes a little
> > bit longer, but then it is very quick. My pc is probably slower.
> > 
> > Something with disk access, caching? Page file size? (Could that
> > really matter in this situation??) My page file is 1.5GB and 3GB max
> > (approx). But, hm, the time delay must be too long for such things.
> > Some network problem?
> 
> I'm working locally - no network (but Internet connection).
> Page file is 2G (max 4G); single hard disk. RAM is 2G.

And no visible change in PF usage or CPU usage when I start Emacs 23. No
noticeable change in CPU usage as Emacs starts up - stays ~0% to 5%. PF usage
remains ~922 MB throughout startup. No difference noticeable in Task Mgr whether
Emacs 23 happens to start in 1 sec or 15 sec.

The elapsed time (e.g. 15 sec) occurs before the frame starts to appear. The
frame is displayed, and startup finishes, at the normal speed. It seems almost
as if Emacs waits before it decides to start up.

I suppose it could be something particular to my machine, but I don't see such a
wait/delay for any other apps or other Emacs versions (20, 21, 22.x).

Re: Release update

[Eli Zaretskii]

> From: Kenichi Handa <handa@m17n.org>
> CC: eliz@gnu.org, emacs-devel@gnu.org
> Date: Tue, 23 Dec 2008 10:42:35 +0900
> 
> In article <87prjjzryu.fsf@cyd.mit.edu>, Chong Yidong <cyd@stupidchicken.com> writes:
> 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > > Also, what about bug #970?  I don't think we can release Emacs that
> > > messes up the tty display like that.
> 
> > Handa-san, could you take a look at this bug?
> 
> > http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=970
> 
> Ok, I'll check what is wrong.  But, I can't reproduce the
> following problem:
> 
> > Type C-n several times, and you will see some very strange behavior:
> > for example, some lines are skipped and point never enters them.

This part of the problem is already solved, see my last email in the
bug.

Re: Release update

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Mon, 22 Dec 2008 15:04:05 -0800
> Cc: emacs-devel@gnu.org
> 
> Another noticeable problem is the time it takes Emacs to start up. I know that
> people have tried to improve this, but for me this startup time has not changed,
> as of this build:
> 
> In GNU Emacs 23.0.60.1 (i386-mingw-nt5.1.2600)
>  of 2008-12-19 on LENNART-69DE564
> Windowing system distributor `Microsoft Corp.', version 5.1.2600
> configured using `configure --with-gcc (3.4) --no-opt --cflags -Ic:/g/include
> -fno-crossjumping'
> 
> Whether I use my own setup of a standalone minibuffer frame and one other,
> principal frame or I use emacs -Q, the startup time is about the same. And
> that's true whether I start by editing a file or Dired or just start with
> *scratch*.
> 
> With my own setup, it takes on average 15-20 seconds for the initial frame to
> appear, and another 10 seconds for my two-frame setup to finish. emacs -Q takes
> 15-20 seconds on average for the frame to appear. The command I use is this:
> runemacs.exe -Q --debug-init. This is on a 1-year-old laptop with 2G RAM and
> pretty good processors.
> 
> The startup time is variable, for some reason. Sometimes it is much less (as if
> something were cached), sometimes it is a little more. Usually it is about 15-20
> sec.

Does the disk LED flicker considerably during the long startups?

Re: Release update

[Kenichi Handa]

In article <uhc4vcynm.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:
> > Ok, I'll check what is wrong.  But, I can't reproduce the
> > following problem:
> > 
> > > Type C-n several times, and you will see some very strange behavior:
> > > for example, some lines are skipped and point never enters them.

> This part of the problem is already solved, see my last email in the
> bug.

Ah, ok, I see.

---
Kenichi Handa
handa@m17n.org

RE: Release update

[Drew Adams]

> Does the disk LED flicker considerably during the long startups?

No, not at all. I even closed my mail client, to minimise other disk accesses
etc. No flicker, same delay.

RE: Release update

[Stephen J. Turnbull]

Drew Adams writes:

 > > I'm working locally - no network (but Internet connection).

It's possible that when Emacs starts up it uses getaddrinfo to get the
hostname, and the IPv6 spec requires that you get your canonical
hostname from a nameserver.  XEmacs currently disobeys the spec by
default because on most networks IPv6 is hardly used even today, and
startup delays can be substantial especially if your Internet
connection is "on-demand" (as it typically was in the days of POTS and
time-metered modem connections, when that decision was made).

Re: Release update

[Giorgos Keramidas]

On Mon, 22 Dec 2008 22:35:26 -0800, "Drew Adams" <drew.adams@oracle.com> wrote:
>> Does the disk LED flicker considerably during the long startups?
>
> No, not at all. I even closed my mail client, to minimise other disk accesses
> etc. No flicker, same delay.

Hi Drew,

Is there any possibility Emacs is trying DNS requests, i.e. to resolve
parts of user-mail-address or something related?  Startup delays are
often caused by waiting for DNS and then timing out after a while.

IIRC you are running the Windows build of Emacs.  Can you try wireshark
or another network monitoring tool, and watch for network activity while
Emacs is starting?

RE: Release update

[Drew Adams]

> Is there any possibility Emacs is trying DNS requests, i.e. to resolve
> parts of user-mail-address or something related?  Startup delays are
> often caused by waiting for DNS and then timing out after a while.
> 
> IIRC you are running the Windows build of Emacs.  Can you try 
> wireshark or another network monitoring tool, and watch for network 
> activity while Emacs is starting?

No, sorry. But I will say that there is no network activity shown by the task
manager.

If no one else sees this problem, you needn't worry about it on my account. I
already said the startup delay is not a problem for me. The bugs I mentioned are
a problem.

Re: Release update

[Werner LEMBERG]

> > Do you still experience the slowness with the one from:
> > http://code.google.com/p/emacs-for-windows/?
>
> Yes, unfortunately. 18 sec for emacs -Q.
>
> But, as I said, the time is variable. Testing only emacs -Q:
> First time: 18 sec.
> Second time, < 1 sec.
> Third time, 16 sec.
> Fourth time, < 1 sec.
> Fifth time, < 1 sec.
> Sixth time, 15 sec.
> Seventh time, 15 sec.
> Eighth time, 1 sec.
> Ninth time, 16 sec.
> Tenth time, 1 sec
>
> That's exactly the behavior I see in Lennart's vanilla builds. It
> either takes quite a long time or it comes up immediately. I was
> thinking that it usually takes a long time, but trying now several
> times in a row, it looks like it's about 50-50. Same thing for
> Lennart's build.

A wild guess: We have experienced a similar slowness at startup for
lilypond, caused by fontconfig which rebuilt the cache files each
time.  The solution was to untick `Automatically adjust clock for
daylight saving changes'.  More can be found in this thread:

  http://www.mail-archive.com/lilypond-user@gnu.org/msg42261.html


      Werner

RE: Release update

[Drew Adams]

> > That's exactly the behavior I see in Lennart's vanilla builds. It
> > either takes quite a long time or it comes up immediately. I was
> > thinking that it usually takes a long time, but trying now several
> > times in a row, it looks like it's about 50-50. Same thing for
> > Lennart's build.
> 
> A wild guess: We have experienced a similar slowness at startup for
> lilypond, caused by fontconfig which rebuilt the cache files each
> time.  The solution was to untick `Automatically adjust clock for
> daylight saving changes'.  More can be found in this thread:
> 
>   http://www.mail-archive.com/lilypond-user@gnu.org/msg42261.html

Wow. Interesting. I tried that, and the problem immediately "went away". And
it's still gone after checking the box again.

For the moment, I have "Automatically adjust..." checked again, and there is
still no problem. I'll try again some other day, and see if the problem really
went away. Before now, I've seen the problem systematically - every day. 

Maybe just unchecking, then rechecking that check box took care of the problem.
That would be good. If only it worked for the bugs too. ;-)

In the thread you cite, I see this:

"forcing a rebuild results in the same delay you're experiencing
every time unless the automatic clock adjustment is turned off for the
first run."

That makes me think that unchecking the check box, then starting Emacs 23, then
checking the box again will keep the problem fixed (which is what I seem to
see). That seems to say that it's enough that the automatic adjustment be turned
off "for the first run", which I guess in my case is for the first time that I
start Emacs 23. And later posts in that thread also suggest that it's OK to turn
the check box on again. BTW, I'm running XP (SP3), FWIW, but apparently Vista
also has this problem.

Veritable voodoo. If this truly is the problem and the fix needed, then perhaps
it should be cited in the FAQ, if it can't be prevented altogether. The thread
mentions a patch for fontconfig that fixes the problem - perhaps Emacs could use
that too.

Here is a summary of the problem, from that thread:

"seems that the problem occurs if the cache is first built
during a daylight saving period when the system is set to automatically
adjust the clock for daylight saving."

Thx! Very cool. The whole thread is interesting, BTW.

Re: Release update

[Richard M Stallman]

    I think it is OK to perform the documentation improvements/proofreading
    in parallel with the pretest.

We have done them in parallel in the past.

Re: Release update

[Richard M Stallman]

Reading the manuals checking for errors is a standard part of making a
major release.  It is very important.  It should be done after
all the new features have been written up for the manual.

Usually we ask people to divide up this job, file by file.
I will try to do some of it.

I am sorry if this disappoints some people who hoped for an Emacs 23 release
very soon.  But I think that hope was unrealistic.  An Emacs major release
is a big job, which is part of the reason for not doing it so often.

RE: Release update

[Drew Adams]

> > A wild guess: We have experienced a similar slowness at startup for
> > lilypond, caused by fontconfig which rebuilt the cache files each
> > time.  The solution was to untick `Automatically adjust clock for
> > daylight saving changes'.  More can be found in this thread:
> > 
> >   http://www.mail-archive.com/lilypond-user@gnu.org/msg42261.html
> 
> Wow. Interesting. I tried that, and the problem immediately 
> "went away". And it's still gone after checking the box again.
> 
> For the moment, I have "Automatically adjust..." checked 
> again, and there is still no problem. I'll try again some other
> day, and see if the problem really went away. Before now, I've
> seen the problem systematically - every day. 
> 
> Maybe just unchecking, then rechecking that check box took 
> care of the problem. That would be good. If only it worked for
> the bugs too. ;-)
> 
> In the thread you cite, I see this:
> 
> "forcing a rebuild results in the same delay you're experiencing
> every time unless the automatic clock adjustment is turned off for the
> first run."
> 
> That makes me think that unchecking the check box, then 
> starting Emacs 23, then checking the box again will keep the
> problem fixed (which is what I seem to see). That seems to say
> that it's enough that the automatic adjustment be turned
> off "for the first run", which I guess in my case is for the 
> first time that I start Emacs 23. And later posts in that
> thread also suggest that it's OK to turn
> the check box on again. BTW, I'm running XP (SP3), FWIW, but 
> apparently Vista also has this problem.
> 
> Veritable voodoo. If this truly is the problem and the fix 
> needed, then perhaps it should be cited in the FAQ, if it
> can't be prevented altogether. The thread
> mentions a patch for fontconfig that fixes the problem - 
> perhaps Emacs could use that too.
> 
> Here is a summary of the problem, from that thread:
> 
> "seems that the problem occurs if the cache is first built
> during a daylight saving period when the system is set to 
> automatically adjust the clock for daylight saving."
> 
> Thx! Very cool. The whole thread is interesting, BTW.

Sad to say, I spoke too soon. I see the problem again this morning, exactly as I
described it before.

And repeating the operation - uncheck the box, open & close Emacs 23, close the
box - doesn't seem to fix it (at first, I thought it did).

Re: Release update

[Werner LEMBERG]

> Sad to say, I spoke too soon. I see the problem again this morning,
> exactly as I described it before.
> 
> And repeating the operation - uncheck the box, open & close Emacs
> 23, close the box - doesn't seem to fix it (at first, I thought it
> did).

You probably have to delete all fontconfig cache files.


    Werner

RE: Release update

[Drew Adams]

> > Sad to say, I spoke too soon. I see the problem again this morning,
> > exactly as I described it before.
> > 
> > And repeating the operation - uncheck the box, open & close Emacs
> > 23, close the box - doesn't seem to fix it (at first, I thought it
> > did).
> 
> You probably have to delete all fontconfig cache files.

Can you tell where to do that? I'm on MS Windows - where do these files reside?
Are they only for Emacs? (Searching for `cache', `fontconfig', and `font-config'
turns up nothing in the Elisp manual.)

RE: Release update

[Drew Adams]

> Can you tell where to do that? I'm on MS Windows - where do 
> these files reside? Are they only for Emacs? (Searching for
> `cache', `fontconfig', and `font-config' turns up nothing in
> the Elisp manual.)

I found node Font X in the Emacs manual, which mentions Fontconfig but not the
cache files. It references this URL: http://fontconfig.org/fontconfig-user.html,
but the files mentioned there have UNIX or Gnu/Linux locations: /etc/fonts/*. It
mentions also ~/.fonts.conf, which could be on Windows too, but I have no such
file. I searched my hard drive for a file named .fonts.conf, but there is none
anywhere.

I have Cygwin installed, so I did locate a fonts.conf file (no initial dot) in
~/cygwin/etc/fonts.

The Fontconfig manual says that the config file has a <cache> element that names
the cache file. The manual says that the default cache file is
~/.fonts.cache-<version>, but I have no such file(s).

In my file fonts.conf, there is no <cache> element. There are two <cachedir>
elements, however, one of which is /var/cache/fontconfig</cachedir>, and the
other of which is ~/.fontconfig</cachedir>.

In ~/cygwin/var/cache/fontconfig/, I do see some files with system-generated
names. Should I just delete those?

What if I didn't have Cygwin installed? Where would the cache files be on MS
Windows? Is it Emacs that creates these files? If so, how does it know whether I
have Cygwin installed etc.?

Thx.

RE: Release update

[Drew Adams]

The Fontconfig manual (at the cited URL) also says this: "~/.fonts.cache-* is
the conventional repository of font information that isn't found in the
per-directory caches. This file is automatically maintained by fontconfig."

There is no such file on my machine.

Re: Release update

[Werner LEMBERG]

> > You probably have to delete all fontconfig cache files.
>
> Can you tell where to do that? I'm on MS Windows - where do these
> files reside?  Are they only for Emacs? (Searching for `cache',
> `fontconfig', and `font-config' turns up nothing in the Elisp
> manual.)

Sorry, I don't use Windows at all.  Perhaps running

  fc-cache.exe -v -r

does the trick to rebuild the cache files, showing the locations of
the cache directories.


    Werner

RE: Release update

[Drew Adams]

> > > You probably have to delete all fontconfig cache files.
> >
> > Can you tell where to do that? I'm on MS Windows - where do these
> > files reside?  Are they only for Emacs? (Searching for `cache',
> > `fontconfig', and `font-config' turns up nothing in the Elisp
> > manual.)
> 
> Sorry, I don't use Windows at all.  Perhaps running
> 
>   fc-cache.exe -v -r
> 
> does the trick to rebuild the cache files, showing the locations of
> the cache directories.

In what directory would I run that? (Again, I have Cygwin, in case that changes
something.)

RE: Release update

[Drew Adams]

> In ~/cygwin/var/cache/fontconfig/, I do see some files with 
> system-generated names. Should I just delete those?

Anyway, I tried that. Deleting those files had no effect on the startup time.
 
> What if I didn't have Cygwin installed? Where would the cache 
> files be on MS Windows? Is it Emacs that creates these files?
> If so, how does it know whether I have Cygwin installed etc.?

Re: Release update

[Werner LEMBERG]

> > In ~/cygwin/var/cache/fontconfig/, I do see some files with 
> > system-generated names. Should I just delete those?
> 
> Anyway, I tried that. Deleting those files had no effect on the
> > startup time.

Well, the first time you call emacs after deleting those file will be
slow since fontconfig rebuilds the cache files.  However, starting
with the second call it should be fast.  In case this isn't so my
guess of the problem's cause is probably wrong.


    Werner

RE: Release update

[Drew Adams]

> > > In ~/cygwin/var/cache/fontconfig/, I do see some files with 
> > > system-generated names. Should I just delete those?
> > 
> > > Anyway, I tried that. Deleting those files had no effect on the
> > > startup time.
> 
> Well, the first time you call emacs after deleting those file will be
> slow since fontconfig rebuilds the cache files.  However, starting
> with the second call it should be fast.  In case this isn't so my
> guess of the problem's cause is probably wrong.

Right; I understood. No, it's not just the first time that it's slow.

Since then, Lennart pointed me to this:
http://www.ascendercorp.com/support/#cache,
which points to file C:\Windows\System32\FNTCACHE.DAT as a font cache.

Not sure deleting that will help either. 
I will try it later - don't want to reboot now.

FWIW, this area doesn't seem to be very well documented for Emacs users. Maybe
if things were working well it would not need to be, but the Fontconfig manual
referenced in the Emacs manual seems to be platform-specific, mentioning only
cache file locations on UNIX or GNU/Linux.

Re: Emacs on GNUstep (was: Release update)

[YAMAMOTO Mitsuharu]

>>>>> On Fri, 05 Dec 2008 10:49:42 -0500, Stefan Monnier <monnier@iro.umontreal.ca> said:

>> Because my PC is slow, I've used WindowMaker(GNUStep) about 4
>> years.  What kind of skills is needed to solve bugs under
>> GNUStep(module name in bug description is NS?)  It seem that only
>> bugs related to Emacs.app belongs to NS, right?

> IIUC the main problem with it right now is that it cannot be dumped.
> I.e. we cannot perform the compilation step where we load `temacs',
> then all the core Emacs files, and then dump the result into a new
> executable `emacs'.  This means that Emacs/GNUstep has to load all
> the fils like subr.el, simple.el, text-mode.el, ... over and over
> again at each startup, and it has several other nasty consequences.

> Fixing this requires someone who can delve into the details of how
> GNUstep binaries are layed out and how the ObjC runtime is linked
> and initialized and how all this interacts.

I just tried copying ObjC-related information in the .data section not
from the dumping process but from the original temacs file so as to
avoid some confusion during the startup time of the dumped executable.
It seems to work for me at least on GNU/Linux (Ubuntu 9.04).

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Index: src/unexelf.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/unexelf.c,v
retrieving revision 1.71
diff -c -p -r1.71 unexelf.c
*** src/unexelf.c	7 Feb 2009 13:07:39 -0000	1.71
--- src/unexelf.c	2 May 2009 06:23:18 -0000
*************** unexec (new_name, old_name, data_start, 
*** 1206,1216 ****
        symendp = (ElfW(Sym) *) ((byte *)symp + NEW_SECTION_H (n).sh_size);
  
        for (; symp < symendp; symp ++)
! 	if (strcmp ((char *) (symnames + symp->st_name), "_end") == 0
! 	    || strcmp ((char *) (symnames + symp->st_name), "end") == 0
! 	    || strcmp ((char *) (symnames + symp->st_name), "_edata") == 0
! 	    || strcmp ((char *) (symnames + symp->st_name), "edata") == 0)
! 	  memcpy (&symp->st_value, &new_bss_addr, sizeof (new_bss_addr));
      }
  
    /* This loop seeks out relocation sections for the data section, so
--- 1206,1242 ----
        symendp = (ElfW(Sym) *) ((byte *)symp + NEW_SECTION_H (n).sh_size);
  
        for (; symp < symendp; symp ++)
! 	{
! 	  if (strcmp ((char *) (symnames + symp->st_name), "_end") == 0
! 	      || strcmp ((char *) (symnames + symp->st_name), "end") == 0
! 	      || strcmp ((char *) (symnames + symp->st_name), "_edata") == 0
! 	      || strcmp ((char *) (symnames + symp->st_name), "edata") == 0)
! 	    memcpy (&symp->st_value, &new_bss_addr, sizeof (new_bss_addr));
! 
! 	  /* ObjC runtime modifies the values of some data structures
! 	     such as classes and selectors in the .data section after
! 	     loading.  As the dump process copies the .data section
! 	     from the current process, that causes problems when the
! 	     modified classes are reinitialized in the dumped
! 	     executable.  We copy such data from the old file, not
! 	     from the current process.  */
! 	  if (strncmp ((char *) (symnames + symp->st_name),
! 		       "_OBJC_", sizeof ("_OBJC_") - 1) == 0)
! 	    {
! 	      caddr_t old, new;
! 
! 	      /* XXX: Check the section name of symp->st_shndx? */
! 	      new = ((symp->st_value - NEW_SECTION_H (symp->st_shndx).sh_addr)
! 		     + NEW_SECTION_H (symp->st_shndx).sh_offset + new_base);
! 	      /* "Unpatch" index.  */
! 	      nn = symp->st_shndx;
! 	      if (nn > old_bss_index)
! 		nn--;
! 	      old = ((symp->st_value - NEW_SECTION_H (symp->st_shndx).sh_addr)
! 		     + OLD_SECTION_H (nn).sh_offset + old_base);
! 	      memcpy (new, old, symp->st_size);
! 	    }
! 	}
      }
  
    /* This loop seeks out relocation sections for the data section, so

Re: Emacs on GNUstep

[Stefan Monnier]

> I just tried copying ObjC-related information in the .data section not
> from the dumping process but from the original temacs file so as to
> avoid some confusion during the startup time of the dumped executable.
> It seems to work for me at least on GNU/Linux (Ubuntu 9.04).

Great news, thank you.  I'll try it out as soon as possible.
If you wrap it in appropriate #ifdef, you can install it on the trunk
(and please add a comment indicating that the ifdef shouldn't be
necessary anyway).  Of course, remove the CANNOT_DUMP setting for
GNUStep at the same.


        Stefan

Re: Emacs on GNUstep (was: Release update)

[Yavor Doganov]

YAMAMOTO Mitsuharu wrote:
> I just tried copying ObjC-related information in the .data section
> not from the dumping process but from the original temacs file so as
> to avoid some confusion during the startup time of the dumped
> executable.

Thank you very much!  This looks like a much simpler and more
straightforward approach than trying to duplicate the whole logic
using NSZone functions.

> It seems to work for me at least on GNU/Linux (Ubuntu 9.04).

I confirm it works fine on gNewSense DeltaH (with fairly old
GCC/GNUstep versions) and Debian GNU/kFreeBSD.

What a relief, I bet this fixes a lot of issues in the GNUstep port.

Re: Emacs on GNUstep

[YAMAMOTO Mitsuharu]

>>>>> On Sun, 03 May 2009 15:38:49 -0400, Stefan Monnier <monnier@iro.umontreal.ca> said:

>> I just tried copying ObjC-related information in the .data section
>> not from the dumping process but from the original temacs file so
>> as to avoid some confusion during the startup time of the dumped
>> executable.  It seems to work for me at least on GNU/Linux (Ubuntu
>> 9.04).

> Great news, thank you.  I'll try it out as soon as possible.  If you
> wrap it in appropriate #ifdef, you can install it on the trunk (and
> please add a comment indicating that the ifdef shouldn't be
> necessary anyway).  Of course, remove the CANNOT_DUMP setting for
> GNUStep at the same.

We can't simply remove CANNOT_DUMP because dumping requires unexelf.c
currently.  The macro __ELF__ cannot be used here because it doesn't
necessarily imply the use of unexelf.c (e.g., Solaris 10).

How about the patch below?  I also removed UNEXEC_SRC because it is no
longer used anywhere.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Index: configure.in
===================================================================
RCS file: /sources/emacs/emacs/configure.in,v
retrieving revision 1.594
diff -c -p -r1.594 configure.in
*** configure.in	1 May 2009 15:32:01 -0000	1.594
--- configure.in	4 May 2009 07:59:51 -0000
*************** AC_DEFINE_UNQUOTED(C_SWITCH_X_SITE,  ${C
*** 2536,2543 ****
   HAVE_X_WINDOWS above and your X include files aren't in a place
   that your compiler can find on its own, you might want to add
   "-I/..." or something similar.])
! AC_DEFINE_UNQUOTED(UNEXEC_SRC,       ${UNEXEC_SRC},
! 		   [Define to the unexec source file name.])
  
  if test "${HAVE_X_WINDOWS}" = "yes" ; then
    AC_DEFINE(HAVE_X_WINDOWS, 1,
--- 2536,2546 ----
   HAVE_X_WINDOWS above and your X include files aren't in a place
   that your compiler can find on its own, you might want to add
   "-I/..." or something similar.])
! case "${unexec}" in
!   unexelf.o)
!     AC_DEFINE(UNEXEC_SUPPORT_OBJC, 1, [Define to 1 if unexec supports ObjC.])
!   ;;
! esac
  
  if test "${HAVE_X_WINDOWS}" = "yes" ; then
    AC_DEFINE(HAVE_X_WINDOWS, 1,
*************** AH_BOTTOM([
*** 2607,2614 ****
  #define HAVE_MOUSE
  #endif
  
! /* Sadly for now, GNUstep dump does not work.  */
! #ifdef NS_IMPL_GNUSTEP
  #define CANNOT_DUMP
  #endif
  
--- 2610,2617 ----
  #define HAVE_MOUSE
  #endif
  
! /* Sadly for now, GNUstep dump does not work with all unexecs.  */
! #if defined (NS_IMPL_GNUSTEP) && !defined (UNEXEC_SUPPORT_OBJC)
  #define CANNOT_DUMP
  #endif
  

Re: Emacs on GNUstep (was: Release update)

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 04 May 2009 01:45:51 +0300, Yavor Doganov <yavor@gnu.org> said:

>> I just tried copying ObjC-related information in the .data section
>> not from the dumping process but from the original temacs file so
>> as to avoid some confusion during the startup time of the dumped
>> executable.

> Thank you very much!  This looks like a much simpler and more
> straightforward approach than trying to duplicate the whole logic
> using NSZone functions.

Perhaps the latter wouldn't help the GNUstep dumping problem because
the use of zones on unexmacosx.c is primarily for the problem that the
dumped heap area cannot be used as an ordinary heap in the dumped
executable, IIUC.  The crucial difference would rather be how
ObjC-related data is located in the executable: Mac OS X uses a
dedicated segment, but GNU/Linux uses the .data section that also
contains other read-write data.

>> It seems to work for me at least on GNU/Linux (Ubuntu 9.04).

> I confirm it works fine on gNewSense DeltaH (with fairly old
> GCC/GNUstep versions) and Debian GNU/kFreeBSD.

Thanks for testing.

> What a relief, I bet this fixes a lot of issues in the GNUstep port.

I hope so.  But it is quite unstable in my environment regardless of
dumping.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Emacs on GNUstep

[Stefan Monnier]

> We can't simply remove CANNOT_DUMP because dumping requires unexelf.c
> currently.  The macro __ELF__ cannot be used here because it doesn't
> necessarily imply the use of unexelf.c (e.g., Solaris 10).

I think it's OK to say that the GNUStep port currently only "works" with
ELF systems.  In any case, experience has shown that CANNOT_DUMP doesn't
really work, so if people want to use GNUStep under Solaris, they're
just going to have to debug the corresponding dump process.  In this
context, it's worthwhile to keep/put the "#ifdef NS_IMPL_GNUSTEP" in
unexelf.c as a document that can be useful for hackers trying to make it
work on other unex*.c.


        Stefan

Re: Emacs on GNUstep

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 04 May 2009 09:45:40 -0400, Stefan Monnier <monnier@iro.umontreal.ca> said:

>> We can't simply remove CANNOT_DUMP because dumping requires
>> unexelf.c currently.  The macro __ELF__ cannot be used here because
>> it doesn't necessarily imply the use of unexelf.c (e.g., Solaris
>> 10).

> I think it's OK to say that the GNUStep port currently only "works"
> with ELF systems.  In any case, experience has shown that
> CANNOT_DUMP doesn't really work, so if people want to use GNUStep
> under Solaris, they're just going to have to debug the corresponding
> dump process.  In this context, it's worthwhile to keep/put the
> "#ifdef NS_IMPL_GNUSTEP" in unexelf.c as a document that can be
> useful for hackers trying to make it work on other unex*.c.

Dumping "works", but the GNUstep port itself doesn't at least for me.
Maybe you want to show some warning message at the configure time not
only for non-ELF systems but also for the GNUstep port in general.
I've installed a change only for unexelf.c.  Please make remaining
changes as you think appropriate.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Emacs on GNUstep

[Stefan Monnier]

> Dumping "works", but the GNUstep port itself doesn't at least for me.

Same for me.

> Maybe you want to show some warning message at the configure time not
> only for non-ELF systems but also for the GNUstep port in general.
> I've installed a change only for unexelf.c.  Please make remaining
> changes as you think appropriate.

Thanks.  I've removed the CANNOT_DUMP.


        Stefan