New emacs maintainer for cygwin

New emacs maintainer for cygwin

[Ken Brown]

I have just taken over as the emacs maintainer for cygwin.  My hope is 
to have an up-to-date emacs package for cygwin when cygwin 1.7 
(currently in beta testing) is released.  To get started, I built the 
emacs 23.0.92 pretest for cygwin 1.7 and made it available as an 
experimental package:

    http://cygwin.com/ml/cygwin-announce/2009-05/msg00016.html

If there are any cygwin users on this list who want to try it out, I 
would welcome any feedback.  But you have to also be willing to try out 
cygwin 1.7 (see http://cygwin.com/#beta-test).

Ken

Re: New emacs maintainer for cygwin

[Eli Zaretskii]

> Date: Fri, 15 May 2009 09:41:37 -0400
> From: Ken Brown <kbrown@cornell.edu>
> 
> I have just taken over as the emacs maintainer for cygwin.

Good news!  Welcome on board.

Maybe you could look at Cygwin-related bugs filed with the Emacs
bug-tracker

  http://emacsbugs.donarmstrong.com/emacs

and see which ones of them could be fixed before the release.

There are also a few Cygwin-related problems in etc/PROBLEMS.

TIA

> To get started, I built the emacs 23.0.92 pretest for cygwin 1.7 and
> made it available as an experimental package:

The latest pretest version is 23.0.92, to be followed by 23.0.94 in a
few days.

Re: New emacs maintainer for cygwin

[Ken Brown]

On 5/15/2009 10:16 AM, Eli Zaretskii wrote:
> Maybe you could look at Cygwin-related bugs filed with the Emacs
> bug-tracker
> 
>   http://emacsbugs.donarmstrong.com/emacs
> 
> and see which ones of them could be fixed before the release.

I'm on my way out of town for a few days, but I'll take a look when I 
get back.  As you know from previous correspondence, I'm not a 
programmer, and I have no debugging experience.  But I'll do my best.

> There are also a few Cygwin-related problems in etc/PROBLEMS.

This is way out of date.  For example, the information about which gcc 
versions are needed is wrong.  I'll take a closer look and submit a 
patch.  If there are any remaining problems, I'll see what I can do.

> The latest pretest version is 23.0.92, to be followed by 23.0.94 in a
> few days.

You meant to say the latest pretest is 23.0.93.  I built that as soon as 
it came out and have been using it since then, just to make sure there 
were no regressions from 23.0.92.  I'll also test 23.0.94 and any future 
pretests.  My reason for using 23.0.92 for the "official" cygwin package 
was that I had built it a month ago and knew that it worked well (at 
least for me), so I thought I should just stick with that.

Ken

P.S.  I subscribe to the emacs-devel list so that I can keep up with 
what's going on.  Are there any other lists I should watch?  I'm not 
interested in being inundated with a lot of email, but I do want to know 
about it when cygwin issues arise.

Re: New emacs maintainer for cygwin

[Lennart Borgman]

On Fri, May 15, 2009 at 4:35 PM, Ken Brown <kbrown@cornell.edu> wrote:
> P.S.  I subscribe to the emacs-devel list so that I can keep up with what's
> going on.  Are there any other lists I should watch?  I'm not interested in
> being inundated with a lot of email, but I do want to know about it when
> cygwin issues arise.

It would be nice if you kept an eye on EmacsWiki too. There are some
pages that mentions cygwin. You may also want to announce your builds
there, for example on the page
http://www.emacswiki.org/emacs/CategoryWThirtyTwo

Re: New emacs maintainer for cygwin

[Chong Yidong]

Ken Brown <kbrown@cornell.edu> writes:

> P.S.  I subscribe to the emacs-devel list so that I can keep up with
> what's going on.  Are there any other lists I should watch?  I'm not
> interested in being inundated with a lot of email, but I do want to
> know about it when cygwin issues arise.

You should subscribe to the bug-gnu-emacs mailing list, where Emacs bug
reports are sent (either that, or read them on by web on
emacs.donarmstrong.com).

Re: New emacs maintainer for cygwin

[Stefan Monnier]

> My reason for using 23.0.92 for the "official" cygwin package was that
> I had built it a month ago and knew that it worked well (at least for
> me), so I thought I should just stick with that.

Actually, it's much better to always use the very latest pretest.
If you want stability, then use 22.3.


        Stefan

Re: New emacs maintainer for cygwin

[Ken Brown]

On 5/15/2009 10:16 AM, Eli Zaretskii wrote:
> There are also a few Cygwin-related problems in etc/PROBLEMS.

I've started looking at this, and it's not clear to me exactly what is
supposed to be in etc/PROBLEMS.  If a problem is solved, should the
discussion be updated to indicate what the solution was or should it
just be deleted?  Here's a specific example.  There is a discussion in
etc/PROBLEMS of a rebasing problem, followed by this workaround:

> To work around this, build Emacs like this:
> 
>   LDFLAGS='-Wl,--enable-auto-import -Wl,--enable-auto-image-base' ./configure
>   make LD='$(CC)'
>   make LD='$(CC)' install
> 
> This produces an Emacs binary that is independent of rebasing.
> 
> Note that you _must_ use LD='$(CC)' in the last two commands above, to
> prevent GCC from passing the "--image-base 0x20000000" option to the
> linker, which is what it does by default.  That option produces an
> Emacs binary with the base address 0x20000000, which will cause Emacs
> to hang after Cygwin DLLs are rebased.

But a much more straightforward solution to the problem is to apply the
following patch to src/s/cygwin.h:

--- origsrc/emacs-23.0.92/src/s/cygwin.h	2009-01-08 06:46:27.000000000 -0500
+++ src/emacs-23.0.92/src/s/cygwin.h	2009-05-17 11:40:55.812500000 -0400
@@ -108,8 +108,12 @@ along with GNU Emacs.  If not, see <http
  /* force the emacs image to start high in memory, so dll relocation
     can put things in low memory without causing all sorts of grief for
     emacs lisp pointers */
-#define DATA_SEG_BITS 0x20000000
-#define LINKER $(CC) -Wl,--image-base,DATA_SEG_BITS
+/* but this can cause problems if the user later rebases; so I'm
+   changing it (KB) */
+
+/* #define DATA_SEG_BITS 0x20000000 */
+/* #define LINKER $(CC) -Wl,--image-base,DATA_SEG_BITS */
+#define LINKER $(CC)

  /* Use terminfo instead of termcap.  Fewer environment variables to
     go wrong, more terminal types. */

If the emacs developers accept this patch, then we can just remove the
whole thing from etc/PROBLEMS.  Or is it better to just report that the
workaround is no longer needed as a result of the patch?

While I'm on the subject of cygwin.h, I asked one of the cygwin 
developers to look at that file and see if it seems reasonable.  He had 
a couple of comments:

 > #define PTY_OPEN					\
 > >  do							\
 > >    {							\
 > >      int dummy;					\
 > >      SIGMASKTYPE mask;					\
 > >      mask = sigblock (sigmask (SIGCHLD));		\
 > >      if (-1 == openpty (&fd, &dummy, pty_name, 0, 0))	\
 > >	fd = -1;					\
 > >      sigsetmask (mask);				\
 > >      emacs_close (dummy);				\

 > This will close a random number if openpty fails.

 > > #define vfork fork

 > This is a no-op since vfork == fork.

Ken

Re: New emacs maintainer for cygwin

[Stefan Monnier]

> I've started looking at this, and it's not clear to me exactly what is
> supposed to be in etc/PROBLEMS.

Known problems that can show up.

> If a problem is solved, should the discussion be updated to indicate
> what the solution was or should it just be deleted?

PROBLEMS describes real current problems, not problems that used to
appear in older Emacsen, so if the problem is fixed, it should be removed.

> --- origsrc/emacs-23.0.92/src/s/cygwin.h	2009-01-08 06:46:27.000000000 -0500
> +++ src/emacs-23.0.92/src/s/cygwin.h	2009-05-17 11:40:55.812500000 -0400
> @@ -108,8 +108,12 @@ along with GNU Emacs.  If not, see <http
>  /* force the emacs image to start high in memory, so dll relocation
>     can put things in low memory without causing all sorts of grief for
>     emacs lisp pointers */
> -#define DATA_SEG_BITS 0x20000000
> -#define LINKER $(CC) -Wl,--image-base,DATA_SEG_BITS
> +/* but this can cause problems if the user later rebases; so I'm
> +   changing it (KB) */
> +
> +/* #define DATA_SEG_BITS 0x20000000 */
> +/* #define LINKER $(CC) -Wl,--image-base,DATA_SEG_BITS */
> +#define LINKER $(CC)

If that can be used (which requires the use of USE_LSB_TAG), it's
a better solution indeed.

> If the emacs developers accept this patch, then we can just remove the
> whole thing from etc/PROBLEMS.

Yes.


        Stefan

Re: New emacs maintainer for cygwin

[Ken Brown]

On 5/19/2009 10:14 PM, Stefan Monnier wrote:
>> --- origsrc/emacs-23.0.92/src/s/cygwin.h	2009-01-08 06:46:27.000000000 -0500
>> +++ src/emacs-23.0.92/src/s/cygwin.h	2009-05-17 11:40:55.812500000 -0400
>> @@ -108,8 +108,12 @@ along with GNU Emacs.  If not, see <http
>>  /* force the emacs image to start high in memory, so dll relocation
>>     can put things in low memory without causing all sorts of grief for
>>     emacs lisp pointers */
>> -#define DATA_SEG_BITS 0x20000000
>> -#define LINKER $(CC) -Wl,--image-base,DATA_SEG_BITS
>> +/* but this can cause problems if the user later rebases; so I'm
>> +   changing it (KB) */
>> +
>> +/* #define DATA_SEG_BITS 0x20000000 */
>> +/* #define LINKER $(CC) -Wl,--image-base,DATA_SEG_BITS */
>> +#define LINKER $(CC)
> 
> If that can be used (which requires the use of USE_LSB_TAG), it's
> a better solution indeed.

Could you elaborate on this?  I don't know anything about USE_LSB_TAG. 
All I know is that I applied the patch and built emacs, and so far it 
seems to work fine.  But I only did this a day or two ago, so maybe a 
problem could still show up.  (Previously I've always used the 
workaround described in etc/PROBLEMS.)  Is there some test I should try 
to see if my build is OK?

Thanks.

Ken

Re: New emacs maintainer for cygwin

[Stefan Monnier]

> Could you elaborate on this?  I don't know anything about USE_LSB_TAG. All

grep for USE_LSB_TAG in src/lisp.h.  It's quite likely that Cygwin uses
it already (ideally, all systems should use it).  What it does is it
makes Emacs use the lower 3bits for tags rather than the higher 3bits.
The advantage being that ELisp pointers can reach any part of the memory
rather than only the lower 512MB.  The downside is that all objects need
to be aligned on an 8byte boundary, and on some systems it's difficult
to get this guarantee.


        Stefan

Re: New emacs maintainer for cygwin

[Ken Brown]

On 5/20/2009 11:22 AM, Stefan Monnier wrote:
>> Could you elaborate on this?  I don't know anything about USE_LSB_TAG. All
> 
> grep for USE_LSB_TAG in src/lisp.h.  It's quite likely that Cygwin uses
> it already (ideally, all systems should use it).

That does seem to be the case, if I understand src/lisp.h correctly. 
I'm using gcc, so __GNUC__ is defined, and src/config.h contains #define 
GNU_MALLOC 1.  This should guarantee that USE_LSB_TAG is being used.  Right?

I therefore propose the following patch to src/s/cygwin.h, which is a 
slightly modified version of the one I sent yesterday:

--- cygwin.h.orig	2009-05-20 13:30:27.062500000 -0400
+++ cygwin.h	2009-05-20 13:31:17.140625000 -0400
@@ -105,11 +105,7 @@
  #define SYSV_SYSTEM_DIR 1
  #define UNEXEC unexcw.o
  #define POSIX_SIGNALS 1
-/* force the emacs image to start high in memory, so dll relocation
-   can put things in low memory without causing all sorts of grief for
-   emacs lisp pointers */
-#define DATA_SEG_BITS 0x20000000
-#define LINKER $(CC) -Wl,--image-base,DATA_SEG_BITS
+#define LINKER $(CC)

  /* Use terminfo instead of termcap.  Fewer environment variables to
     go wrong, more terminal types. */

I realize that this patch does not fix a regression from emacs 22.3, so 
maybe it should wait until after 23.1 is released.  But it affect only 
the cygwin build, and it would simplify my task of patching etc/PROBLEMS 
if it could be applied soon.

Ken

Re: New emacs maintainer for cygwin

[Angelo Graziosi]

Ken Brown wrote:

> followed by this workaround:

>>     To work around this, build Emacs like this:
>> 
>> 
>>       LDFLAGS='-Wl,--enable-auto-import -Wl,--enable-auto-image-base' ./configure
>>       make LD='$(CC)'
>>       make LD='$(CC)' install
>> 
>> 
>>     This produces an Emacs binary that is independent of rebasing.
>> 
>> 
>>     Note that you _must_ use LD='$(CC)' in the last two commands above, to
>>     prevent GCC from passing the "--image-base 0x20000000" option to the
>>     linker, which is what it does by default.  That option produces an
>>     Emacs binary with the base address 0x20000000, which will cause Emacs
>>     to hang after Cygwin DLLs are rebased.

That workaround was born when I started to build Emacs. I found that the
resulting build where more stable. Really it does not help in rebasing,
which regards only DLL and not EXE, but it helps to build ALL the EXE
(not only emacs.exe etc.) with those flags (-Wl,--enable...). To be
honest, it is some time that in my build I use only:

auto_import="-Wl,--enable-auto-import"
pseudo_reloc="-Wl,--enable-runtime-pseudo-reloc"

which should be recommended. Indeed, if I have well understood the
discussion on Cygwin lists, in a future release of binutils, the new
ld.exe should add automatically those flags in building exe file.

Obviosly, those flags should be applied via cigwin.h, but the patch I
found added them only for emacs.exe so I omitted to propose the patch.

Cheers,
    Angelo.

---
Don't know much about geography
Don't know much trigonometry
Don't know much about algebra
...

    Sam Cooke, Wonderful World

Re: New emacs maintainer for cygwin

[Ken Brown]

On 5/20/2009 4:05 AM, Angelo Graziosi wrote:
> Ken Brown wrote:
>>>     Note that you _must_ use LD='$(CC)' in the last two commands 
>>> above, to
>>>     prevent GCC from passing the "--image-base 0x20000000" option to the
>>>     linker, which is what it does by default.
[...]
> To be
> honest, it is some time that in my build I use only:
> 
> auto_import="-Wl,--enable-auto-import"
> pseudo_reloc="-Wl,--enable-runtime-pseudo-reloc"
> 
> which should be recommended.

I wasn't talking about the linker flags, which I agree are recommended. 
  I was talking about the need to use LD='$(CC)'.  The patch I proposed 
makes that unnecessary.

Ken

Re: New emacs maintainer for cygwin

[Angelo Graziosi]

Ken Brown ha scritto:

> 
> I wasn't talking about the linker flags, which I agree are recommended. 
>  I was talking about the need to use LD='$(CC)'.  The patch I proposed 
> makes that unnecessary.

At that time (Emacs-cvs-22.0.50), I did not know much about patches
etc., so that basic workaround looked the best and simple thing to do...


Cheers,
    Angelo.

---
Facesti come quei che va di notte,
che porta il lume dietro e se' non giova,
ma dopo se' fa le persone dotte.

                   DANTE, Purgatorio, xxii 67-69

Re: New emacs maintainer for cygwin

[Angelo Graziosi]

Ken Brown wrote:
> I realize that this patch does not fix a regression from emacs 22.3,

Gulp! Which regression I am missing?

Anyway...

> so maybe it should wait until after 23.1 is released. But it affect only the cygwin build, and it would simplify my task of patching etc/PROBLEMS if it could be applied soon.

I have tested the patch which Ken sent yesterday and it produces the 
same build log I obtain with my workaround cited in etc/PROBLEMS...

So +1 to be applied soon...

Cheers,
    Angelo.

Re: New emacs maintainer for cygwin

[Ken Brown]

On 5/20/2009 6:19 PM, Angelo Graziosi wrote:
> Ken Brown wrote:
>> I realize that this patch does not fix a regression from emacs 22.3,
> 
> Gulp! Which regression I am missing?

Sorry, I wasn't clear.  There's a policy that, in general, no new 
patches will be applied unless their purpose is to fix a regression from 
22.3.  There is no regression that's being fixed by this patch.  Its 
purpose is just to simplify the build.

Ken