[patch] combined build

[patch] combined build

[Dmitry Antipov]

[-- Attachment #1: Type: text/plain, Size: 1977 bytes --]

This patch provides a minimal support for compiling (the most of) emacs C code at once using
-combine option supported by never versions of GCC. Since an almost everything is compiled
each time when the compiler is invoked, this mode is not suitable for developers. But it might
be useful for package builders and other peoples who just wants to do './configure && make &&
make install' sequence. This feature also gives GCC a chance to optimize something during
inter-module analysis.

Notes:

1. Compiling with -combine implies quite strict function declarations vs. definitions checking,
    so C code is touched in some places. In particular, current 'fatal' implementation bombs
    out combining compilation, and I've redone it with va_args. This might be non-portable or
    may break some ancient compiler.

2. For me, it doesn't work when using GTK toolkit. When using both GTK and -combine, I've got
    the following nasty error messages:

    In file included from /usr/include/gtk-2.0/gtk/gtk.h:74,
                     from xterm.h:46,
                     from fns.c:53:
    /usr/include/gtk-2.0/gtk/gtkcurve.h:104: error: conflicting types for 'gtk_curve_get_vector'
    /usr/include/gtk-2.0/gtk/gtkcurve.h:104: error: previous declaration of 'gtk_curve_get_vector' was here
    /usr/include/gtk-2.0/gtk/gtkcurve.h:106: error: conflicting types for 'gtk_curve_set_vector'
    /usr/include/gtk-2.0/gtk/gtkcurve.h:106: error: previous declaration of 'gtk_curve_set_vector' was here

    I suspect this is a bug in GCC. So I've installed an additional check in configure.in
    to avoid using both GTK and -combine simultaneously.

3. On i386, combined compilation might eat up to 400M of memory, so it shouldn't be used on
    low-memory machines.

4. Dependencies are broken - everything.o depends only on .c files, and .h files included from
    them are ignored. But this should be safe if you just want to compile and install from clean
    sources.

Dmitry


[-- Attachment #2: combined-build.patch --]
[-- Type: text/x-patch, Size: 10118 bytes --]

Index: configure.in
===================================================================
RCS file: /sources/emacs/emacs/configure.in,v
retrieving revision 1.479
diff -u -r1.479 configure.in
--- configure.in	7 Nov 2007 04:00:24 -0000	1.479
+++ configure.in	15 Nov 2007 11:50:05 -0000
@@ -1326,6 +1326,23 @@
 unset has_option
 unset SAVE_CFLAGS
 
+AC_ARG_ENABLE(combine,
+[AS_HELP_STRING([--enable-combine],
+		[compile with -combine option (GCC only)])],
+  AC_MSG_CHECKING([whether combined compilation is supported])
+  combine=no
+  if test $(basename ${CC}) = "gcc"; then
+    LANG=C ${CC} -combine 2>&1 | grep -q unrecognized
+    test $? -eq 1 && combine=yes
+  fi
+  if test "$combine" = "yes"; then
+    AC_DEFINE(COMBINE, [1], [Define to build with -combine GCC option.])
+    AC_MSG_RESULT(yes)
+  else
+    AC_MSG_RESULT(no)
+    AC_MSG_ERROR([Reconfigure without --enable-combine option.])
+  fi)
+
 #### Some other nice autoconf tests.
 
 dnl checks for programs
@@ -2192,6 +2209,11 @@
 
 fi
 
+dnl Using both GTK and -combine doesn't work for now.
+if test "${HAVE_GTK}" = "yes" && test "${enable_combine+set}" = set; then
+   AC_MSG_ERROR([(Due to GCC bugs ?) GTK toolkit may not be used with -combine option.
+Reconfigure either with another X toolkit or without --enable-combine.])
+fi
 
 if test "${HAVE_GTK}" = "yes"; then
 
Index: src/Makefile.in
===================================================================
RCS file: /sources/emacs/emacs/src/Makefile.in,v
retrieving revision 1.359
diff -u -r1.359 Makefile.in
--- src/Makefile.in	1 Nov 2007 07:16:09 -0000	1.359
+++ src/Makefile.in	15 Nov 2007 11:50:06 -0000
@@ -978,7 +978,13 @@
    for the first time, this prevents any variation between configurations
    in the contents of the DOC file.
    Likewise for ${SOME_MACHINE_LISP}.  */
+
+#ifdef COMBINE
+combobj = everything.o
+${etc}DOC: ${libsrc}make-docfile${EXEEXT} ${combobj} ${shortlisp} ${SOME_MACHINE_LISP}
+#else 
 ${etc}DOC: ${libsrc}make-docfile${EXEEXT} ${obj} ${shortlisp} ${SOME_MACHINE_LISP}
+#endif /* COMBINE */
 	-rm -f ${etc}DOC
 	${libsrc}make-docfile -d ${srcdir} ${SOME_MACHINE_OBJECTS} ${obj} > ${etc}DOC
 	${libsrc}make-docfile -a ${etc}DOC -d ${srcdir} ${SOME_MACHINE_LISP} ${shortlisp}
@@ -991,11 +997,21 @@
 #define MAKE_PARALLEL
 #endif
 
+#ifdef COMBINE
+${combobj}: $(obj:.o=.c) $(otherobj:.o=.c) 
+	$(CC) -c $(CPPFLAGS) $(ALL_CFLAGS) -combine $? -o $@
+
+temacs${EXEEXT}: MAKE_PARALLEL $(LOCALCPP) $(STARTFILES) stamp-oldxmenu ${combobj} OBJECTS_MACHINE prefix-args${EXEEXT}
+	echo "${obj} ${otherobj} " OBJECTS_MACHINE > buildobj.lst
+	$(LD) YMF_PASS_LDFLAGS (${STARTFLAGS} ${TEMACS_LDFLAGS}) $(LDFLAGS) \
+    -o temacs ${STARTFILES} ${combobj} OBJECTS_MACHINE ${LIBES}
+#else
 temacs${EXEEXT}: MAKE_PARALLEL $(LOCALCPP) $(STARTFILES) stamp-oldxmenu ${obj} ${otherobj} OBJECTS_MACHINE prefix-args${EXEEXT}
 	echo "${obj} ${otherobj} " OBJECTS_MACHINE > buildobj.lst
 	$(LD) YMF_PASS_LDFLAGS (${STARTFLAGS} ${TEMACS_LDFLAGS}) $(LDFLAGS) \
     -o temacs ${STARTFILES} ${obj} ${otherobj}  \
     OBJECTS_MACHINE ${LIBES}
+#endif /* COMBINE */
 
 /* We do not use ALL_LDFLAGS because LD_SWITCH_SYSTEM and LD_SWITCH_MACHINE
    often contain options that have to do with using Emacs''s crt0,
Index: src/cm.c
===================================================================
RCS file: /sources/emacs/emacs/src/cm.c,v
retrieving revision 1.24
diff -u -r1.24 cm.c
--- src/cm.c	29 Aug 2007 05:27:53 -0000	1.24
+++ src/cm.c	15 Nov 2007 11:50:06 -0000
@@ -30,18 +30,6 @@
 #include "termhooks.h"
 #include "termchar.h"
 
-
-/* For now, don't try to include termcap.h.  On some systems,
-   configure finds a non-standard termcap.h that the main build
-   won't find.  */
-
-#if defined HAVE_TERMCAP_H && 0
-#include <termcap.h>
-#else
-extern void tputs P_ ((const char *, int, int (*)(int)));
-extern char *tgoto P_ ((const char *, int, int));
-#endif
-
 #define	BIG	9999		/* 9999 good on VAXen.  For 16 bit machines
 				   use about 2000.... */
 
Index: src/cm.h
===================================================================
RCS file: /sources/emacs/emacs/src/cm.h,v
retrieving revision 1.23
diff -u -r1.23 cm.h
--- src/cm.h	29 Aug 2007 05:27:54 -0000	1.23
+++ src/cm.h	15 Nov 2007 11:50:06 -0000
@@ -162,6 +162,17 @@
 extern int cost;
 extern int evalcost ();
 
+/* For now, don't try to include termcap.h.  On some systems,
+   configure finds a non-standard termcap.h that the main build
+   won't find.  */
+
+#if defined HAVE_TERMCAP_H && 0
+#include <termcap.h>
+#else
+extern int tputs P_ ((const char *, int, int (*)(int)));
+extern char *tgoto P_ ((const char *, int, int));
+#endif
+
 #define emacs_tputs(tty, str, affcnt, putc) (current_tty = (tty), tputs (str, affcnt, putc))
 
 extern struct tty_display_info *current_tty;
Index: src/config.in
===================================================================
RCS file: /sources/emacs/emacs/src/config.in,v
retrieving revision 1.236
diff -u -r1.236 config.in
--- src/config.in	1 Nov 2007 07:38:58 -0000	1.236
+++ src/config.in	15 Nov 2007 11:50:06 -0000
@@ -32,6 +32,9 @@
 /* Define to 1 if the mktime function is broken. */
 #undef BROKEN_MKTIME
 
+/* Define to build with -combine GCC option. */
+#undef COMBINE
+
 /* Define to one of `_getb67', `GETB67', `getb67' for Cray-2 and Cray-YMP
    systems. This function is required for `alloca.c' support on those systems.
    */
Index: src/fringe.c
===================================================================
RCS file: /sources/emacs/emacs/src/fringe.c,v
retrieving revision 1.48
diff -u -r1.48 fringe.c
--- src/fringe.c	14 Sep 2007 07:33:27 -0000	1.48
+++ src/fringe.c	15 Nov 2007 11:50:06 -0000
@@ -1659,7 +1659,7 @@
 /* Garbage collection hook */
 
 void
-mark_fringe_data ()
+mark_fringe_data (void)
 {
   int i;
 
Index: src/keyboard.c
===================================================================
RCS file: /sources/emacs/emacs/src/keyboard.c,v
retrieving revision 1.929
diff -u -r1.929 keyboard.c
--- src/keyboard.c	9 Nov 2007 12:19:21 -0000	1.929
+++ src/keyboard.c	15 Nov 2007 11:50:06 -0000
@@ -12519,7 +12519,7 @@
 /* Mark the pointers in the kboard objects.
    Called by the Fgarbage_collector.  */
 void
-mark_kboards ()
+mark_kboards (void)
 {
   KBOARD *kb;
   Lisp_Object *p;
Index: src/lisp.h
===================================================================
RCS file: /sources/emacs/emacs/src/lisp.h,v
retrieving revision 1.599
diff -u -r1.599 lisp.h
--- src/lisp.h	29 Oct 2007 18:15:02 -0000	1.599
+++ src/lisp.h	15 Nov 2007 11:50:06 -0000
@@ -2361,6 +2361,7 @@
 void hash_remove P_ ((struct Lisp_Hash_Table *, Lisp_Object));
 void hash_clear P_ ((struct Lisp_Hash_Table *));
 void remove_hash_entry P_ ((struct Lisp_Hash_Table *, int));
+extern void seed_random P_ ((long));
 extern void init_fns P_ ((void));
 EXFUN (Fsxhash, 1);
 EXFUN (Fmake_hash_table, MANY);
@@ -3227,6 +3228,11 @@
 extern int emacs_close P_ ((int));
 extern int emacs_read P_ ((int, char *, unsigned int));
 extern int emacs_write P_ ((int, const char *, unsigned int));
+extern void fatal (const char *, ...) NO_RETURN;
+
+/* Defined in unexec implementation.  */
+extern void unexec P_ ((char *, char *, unsigned int,
+			unsigned int, unsigned int));
 
 /* defined in filelock.c */
 EXFUN (Funlock_buffer, 0);
@@ -3255,7 +3261,6 @@
 
 /* Defined in term.c */
 extern void syms_of_term P_ ((void));
-extern void fatal () NO_RETURN;
 
 /* Defined in terminal.c */
 extern void syms_of_terminal P_ ((void));
Index: src/sysdep.c
===================================================================
RCS file: /sources/emacs/emacs/src/sysdep.c,v
retrieving revision 1.288
diff -u -r1.288 sysdep.c
--- src/sysdep.c	27 Sep 2007 21:20:06 -0000	1.288
+++ src/sysdep.c	15 Nov 2007 11:50:06 -0000
@@ -26,6 +26,8 @@
 #include <signal.h>
 #include <stdio.h>
 #include <setjmp.h>
+#include <stdarg.h>
+
 #ifdef HAVE_UNISTD_H
 #include <unistd.h>
 #endif
@@ -2421,7 +2423,7 @@
  */
 
 #ifndef start_of_data
-char *
+POINTER_TYPE *
 start_of_data ()
 {
 #ifdef DATA_START
@@ -5496,5 +5498,19 @@
 }
 #endif /* HAVE_STRSIGNAL */
 
+void
+fatal (const char *fmt, ...)
+{
+  va_list ap;
+  
+  va_start (ap, fmt);
+  fprintf (stderr, "emacs: ");
+  vfprintf (stderr, fmt, ap);
+  va_end (ap);
+  fprintf (stderr, "\n");
+  fflush (stderr);
+  exit (1);
+}
+
 /* arch-tag: edb43589-4e09-4544-b325-978b5b121dcf
    (do not change this comment) */
Index: src/term.c
===================================================================
RCS file: /sources/emacs/emacs/src/term.c,v
retrieving revision 1.209
diff -u -r1.209 term.c
--- src/term.c	25 Oct 2007 02:38:37 -0000	1.209
+++ src/term.c	15 Nov 2007 11:50:06 -0000
@@ -54,20 +54,6 @@
 #include "syssignal.h"
 #include "systty.h"
 #include "intervals.h"
-
-/* For now, don't try to include termcap.h.  On some systems,
-   configure finds a non-standard termcap.h that the main build
-   won't find.  */
-
-#if defined HAVE_TERMCAP_H && 0
-#include <termcap.h>
-#else
-extern void tputs P_ ((const char *, int, int (*)(int)));
-extern int tgetent P_ ((char *, const char *));
-extern int tgetflag P_ ((char *id));
-extern int tgetnum P_ ((char *id));
-#endif
-
 #include "cm.h"
 #ifdef HAVE_X_WINDOWS
 #include "xterm.h"
@@ -3754,18 +3740,6 @@
   abort ();
 }
 
-/* VARARGS 1 */
-void
-fatal (str, arg1, arg2)
-     char *str, *arg1, *arg2;
-{
-  fprintf (stderr, "emacs: ");
-  fprintf (stderr, str, arg1, arg2);
-  fprintf (stderr, "\n");
-  fflush (stderr);
-  exit (1);
-}
-
 \f
 
 /* Delete the given tty terminal, closing all frames on it. */
Index: src/terminfo.c
===================================================================
RCS file: /sources/emacs/emacs/src/terminfo.c,v
retrieving revision 1.22
diff -u -r1.22 terminfo.c
--- src/terminfo.c	26 Jul 2007 05:27:57 -0000	1.22
+++ src/terminfo.c	15 Nov 2007 11:50:06 -0000
@@ -42,7 +42,7 @@
      int len, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9;
 {
   char *temp;
-  extern char *tparm();
+  extern char *tparm (char *, ...);
 
   temp = tparm (string, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9);
   if (outstring == 0)

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: [patch] combined build

[Dan Nicolaescu]

Dmitry Antipov <dmantipov@yandex.ru> writes:

  > This patch provides a minimal support for compiling (the most of) emacs C code at once using
  > -combine option supported by never versions of GCC. Since an almost everything is compiled
  > each time when the compiler is invoked, this mode is not suitable for developers. But it might
  > be useful for package builders and other peoples who just wants to do './configure && make &&
  > make install' sequence. This feature also gives GCC a chance to optimize something during
  > inter-module analysis.
  > 
  > Notes:
  > 
  > 1. Compiling with -combine implies quite strict function declarations vs. definitions checking,
  >    so C code is touched in some places. In particular, current 'fatal' implementation bombs
  >    out combining compilation, and I've redone it with va_args. This might be non-portable or
  >    may break some ancient compiler.

I went through exactly this exercise 2 years ago, trying to fix all the
combine problems in the tree at that time. I had to undo a similar
"fatal" change for these concerns.

Now is a good time to reconsider, IMHO we should not support building
without varargs.

Please note that to get the full benefit of interprocedural optimization
you'd also need -fwhole-program, and that implies that the files in
lwlib should be added to the temacs compile line.


  > 2. For me, it doesn't work when using GTK toolkit. When using both GTK and -combine, I've got
  >    the following nasty error messages:
  > 
  >    In file included from /usr/include/gtk-2.0/gtk/gtk.h:74,
  >                     from xterm.h:46,
  >                     from fns.c:53:
  >    /usr/include/gtk-2.0/gtk/gtkcurve.h:104: error: conflicting types for 'gtk_curve_get_vector'
  >    /usr/include/gtk-2.0/gtk/gtkcurve.h:104: error: previous declaration of 'gtk_curve_get_vector' was here
  >    /usr/include/gtk-2.0/gtk/gtkcurve.h:106: error: conflicting types for 'gtk_curve_set_vector'
  >    /usr/include/gtk-2.0/gtk/gtkcurve.h:106: error: previous declaration of 'gtk_curve_set_vector' was here
  > 
  >    I suspect this is a bug in GCC. So I've installed an additional check in configure.in
  >    to avoid using both GTK and -combine simultaneously.

What version of Gtk is this?
If you are certain this is a gcc bug, please report it.

Re: [patch] combined build

[Stefan Monnier]

>> This patch provides a minimal support for compiling (the most of)
>> emacs C code at once using -combine option supported by never
>> versions of GCC. Since an almost everything is compiled each time
>> when the compiler is invoked, this mode is not suitable for
>> developers. But it might be useful for package builders and other
>> peoples who just wants to do './configure && make && make install'
>> sequence. This feature also gives GCC a chance to optimize something
>> during inter-module analysis.

BTW, what kind of benefit are we looking at?
Is the resulting executable smaller?  If so by how much?
Is it faster?  If so by how much?
Is the compilation quicker?

I understand this depends on other optimization options and the machine
on which you test it, but does someone have at least some vague
measurements to get an idea if the effort is worthwhile or not?


        Stefan

Re: [patch] combined build

[Dan Nicolaescu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

  > >> This patch provides a minimal support for compiling (the most of)
  > >> emacs C code at once using -combine option supported by never
  > >> versions of GCC. Since an almost everything is compiled each time
  > >> when the compiler is invoked, this mode is not suitable for
  > >> developers. But it might be useful for package builders and other
  > >> peoples who just wants to do './configure && make && make install'
  > >> sequence. This feature also gives GCC a chance to optimize something
  > >> during inter-module analysis.
  > 
  > BTW, what kind of benefit are we looking at?
  > Is the resulting executable smaller?  If so by how much?
  > Is it faster?  If so by how much?
  > Is the compilation quicker?

I did this 2 years ago, so I don't have any numbers handy. Maybe Dmitry
has some. 
It should help a bit both the code size (if I remember well about 5%),
and the speed, especially when using -fwhole-program on x86 (in that
case the calling convention for functions that don't have their address
taken can be changed from stack based to register based -- there are
other things too, but AFAIR this is one of the important ones).

Size is easy to look at, but how would you test the speed?

Re: [patch] combined build

[Richard Stallman]

    BTW, what kind of benefit are we looking at?

I agree.  I don't think we should undertake to support more modes of
building just for the sake of supporting them.  Each additional mode
we do support means more work for us in the future.

We should add this if it provides substantial benefit, but otherwise
let's not.

Re: [patch] combined build

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 15 Nov 2007 11:59:11 -0500
> Cc: Dmitry Antipov <dmantipov@yandex.ru>, emacs-devel@gnu.org
> 
> BTW, what kind of benefit are we looking at?
> Is the resulting executable smaller?  If so by how much?
> Is it faster?  If so by how much?
> Is the compilation quicker?

And what about debuggability of the resulting binary?  If it's harder
to debug (e.g., if there are problems to get a meaningful backtrace
from a core file of a crashed Emacs), then I'd actually urge package
maintainers _not_ to use this while preparing binary distros, because
that would make our lives as Emacs maintainers harder when a user
reports a crash.

Re: [patch] combined build

[Dmitry Antipov]

Richard Stallman wrote:

> We should add this if it provides substantial benefit, but otherwise
> let's not.

For now, size benefits are marginal - ~11k if compiled with -O2 and
~7k if compiled with -Os.

It's not so easy to measure speed of interactive application, but
the speed of byte-compilation looks exactly the same (up to the
errors of measurement).

The worst news is that I was unable to compile both with '-combine'
and '-fwhole-program' - temacs just crashes silently with huge core,
which probably means C stack overflow due to deep recursion.
This applies to both 4.2.1 and 4.2.2 versions of GCC. I'm trying
to understand why this happens, but still have no ideas.

So, it looks like this technology doesn't give substantial benefits
for today. But I believe it's features should be revised periodically,
according to the state and progress of GCC development.

Dmitry

Re: [patch] combined build

[Dan Nicolaescu]

Richard Stallman <rms@gnu.org> writes:

  >     BTW, what kind of benefit are we looking at?
  > 
  > I agree.  I don't think we should undertake to support more modes of
  > building just for the sake of supporting them.  Each additional mode
  > we do support means more work for us in the future.
  > 
  > We should add this if it provides substantial benefit, but otherwise
  > let's not.

IMO we should check in the C code changes because they are technically
correct and TRTD.

The build changes are probably not worth it as the benefit is not
great. There's work in progress in gcc to make using interprocedural
optimizations easier (just adding a flag on the command line, no other
build changes), but it is not ready yet.

Re: [patch] combined build

[Richard Stallman]

    IMO we should check in the C code changes because they are technically
    correct and TRTD.

Please don't write `(void)' as argument list (in fringe.c, keyboard.c,
etc).  As for using stdard for `fatal', I will go along with it.