The 'cross' directory

The 'cross' directory

[Eli Zaretskii]

The file cross/README, though somewhat terse, seems to say that this
directory is for cross-building Emacs.  AFAIK the only target for
which Emacs can currently be cross-built is Android.  But the portion
of configure.ac that creates subdirectories in this directory and
copies (links) files there is conditioned by

  if test "$XCONFIGURE" != "android"; then

So this seems to run in any build but Android?

In any case, it sounds like creating those subdirectories and linking
files there is only needed in the Android build.  And if so, can we
please avoid doing this unnecessary stuff in other builds?

Thanks.

Re: The 'cross' directory

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> The file cross/README, though somewhat terse, seems to say that this
> directory is for cross-building Emacs.  AFAIK the only target for
> which Emacs can currently be cross-built is Android.  But the portion
> of configure.ac that creates subdirectories in this directory and
> copies (links) files there is conditioned by
>
>   if test "$XCONFIGURE" != "android"; then
>
> So this seems to run in any build but Android?

XCONFIGURE is only set within the second invocation of `configure'
during the Android configuration process, which creates Makefiles that
are subsequently copied into the cross directory.

This code is only meant to run inside the initial configuration for the
build machine.

> In any case, it sounds like creating those subdirectories and linking
> files there is only needed in the Android build.  And if so, can we
> please avoid doing this unnecessary stuff in other builds?

Yes, will do.

Re: The 'cross' directory

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Is it true that cross-compiling Emacs does not work
except when the target is Emacs?

It is supposed to work.  Normally autoconf does that automatically.
What broke it?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: The 'cross' directory

[Po Lu]

Richard Stallman <rms@gnu.org> writes:

> Is it true that cross-compiling Emacs does not work
> except when the target is Emacs?

I presume you meant Android by Emacs.  I view this as a feature, not a
problem, because:

> It is supposed to work.  Normally autoconf does that automatically.
> What broke it?

Nothing broke it.  Emacs never supported cross compilation in the past,
since Emacs is needed on the build machine to compile Lisp and dump
itself.

This arrangement has not changed under Android.  Instead, Emacs is
configured once and run on the build machine for the purpose of
byte-compiling Lisp code, before `configure' is rerun with the Android
cross-compiler to generate an undumped C binary that is copied into the
Android installation package.  Dumping subsequently takes place when
that package is installed.

Re: The 'cross' directory

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Nothing broke it.  Emacs never supported cross compilation in the past,
  > since Emacs is needed on the build machine to compile Lisp and dump
  > itself.

In the past, building Emacs was cleanly separated into building
`temacs' and then using it to dump.  We included .elc files in the tar
ball so that users did not need to compile any Lisp code unless they
changed Lisp source.  (Byte-compiling Lisp code was slow and the .elc
files were portable.)

So I think it should have worked to make `temacs' cross-compiling
and then run the rest on the target machine.  That should have been
easy because it did not need to compile or link any C code.

That may still be workable.  `make temacs' ought to work ok under
cross-compilation, and the rest could be done on the target machine
without a C compiler or linker if `make temacs' has done its job.

Does this work?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: The 'cross' directory

[Po Lu]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Nothing broke it.  Emacs never supported cross compilation in the past,
>   > since Emacs is needed on the build machine to compile Lisp and dump
>   > itself.
>
> In the past, building Emacs was cleanly separated into building
> `temacs' and then using it to dump.  We included .elc files in the tar
> ball so that users did not need to compile any Lisp code unless they
> changed Lisp source.  (Byte-compiling Lisp code was slow and the .elc
> files were portable.)
>
> So I think it should have worked to make `temacs' cross-compiling
> and then run the rest on the target machine.  That should have been
> easy because it did not need to compile or link any C code.
>
> That may still be workable.  `make temacs' ought to work ok under
> cross-compilation, and the rest could be done on the target machine
> without a C compiler or linker if `make temacs' has done its job.
>
> Does this work?

I don't know.  That being said, the cross directory is not used for this
form of cross-compilation.

Re: The 'cross' directory

[Emanuel Berg]

Richard Stallman wrote:

> In the past, building Emacs was cleanly separated into
> building `temacs' and then using it to dump. We included
> .elc files in the tar ball so that users did not need to
> compile any Lisp code unless they changed Lisp source.
> (Byte-compiling Lisp code was slow and the .elc files
> were portable.)

OT1H everything was more easy then, OTOH tools are much more
powerful now ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: The 'cross' directory

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: eliz@gnu.org, emacs-devel@gnu.org
> Date: Sat, 12 Aug 2023 21:43:31 -0400
> 
>   > Nothing broke it.  Emacs never supported cross compilation in the past,
>   > since Emacs is needed on the build machine to compile Lisp and dump
>   > itself.
> 
> In the past, building Emacs was cleanly separated into building
> `temacs' and then using it to dump.  We included .elc files in the tar
> ball so that users did not need to compile any Lisp code unless they
> changed Lisp source.  (Byte-compiling Lisp code was slow and the .elc
> files were portable.)
> 
> So I think it should have worked to make `temacs' cross-compiling
> and then run the rest on the target machine.  That should have been
> easy because it did not need to compile or link any C code.

This should still work, although I doubt if anyone tried it lately.
Emacs with native-compilation cannot be built that way, because the
native-code *.eln files are produced at build time, and cannot be
included in the tarball (they depend on the architecture of the target
machine and include signatures of the Emacs binary and the location of
the source *.el files).  But Emacs without native-compilation could be
produced in such two steps, except that the target environment still
needs some support tools: GNU Make, cp and rm.  (Alternatively, they
could invoke the final build commands by hand, if they know how.)

But I don't believe this is relevant to the Android build, since the
Android development tools don't run on Android, and the Android port
produces an undumped Emacs anyway.

Re: The 'cross' directory

[Helmut Eller]

On Sun, Aug 13 2023, Eli Zaretskii wrote:

>> From: Richard Stallman <rms@gnu.org>
>> So I think it should have worked to make `temacs' cross-compiling
>> and then run the rest on the target machine.  That should have been
>> easy because it did not need to compile or link any C code.
>
> This should still work, although I doubt if anyone tried it lately.
> Emacs with native-compilation cannot be built that way, because the
> native-code *.eln files are produced at build time, and cannot be
> included in the tarball (they depend on the architecture of the target
> machine and include signatures of the Emacs binary and the location of
> the source *.el files).  But Emacs without native-compilation could be
> produced in such two steps, except that the target environment still
> needs some support tools: GNU Make, cp and rm.  (Alternatively, they
> could invoke the final build commands by hand, if they know how.)

Is it possible/easy to cross-compile Emacs for Windows on a Unix
machine?  Possibly with the help of Wine for dumping.

Debian has a package gcc-mingw-w64-x86-64, so I suppose installing the
cross compiler itself is fairly easy.

Helmut

Re: The 'cross' directory

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: rms@gnu.org,  luangruo@yahoo.com,  emacs-devel@gnu.org
> Date: Tue, 15 Aug 2023 10:05:45 +0200
> 
> On Sun, Aug 13 2023, Eli Zaretskii wrote:
> 
> >> From: Richard Stallman <rms@gnu.org>
> >> So I think it should have worked to make `temacs' cross-compiling
> >> and then run the rest on the target machine.  That should have been
> >> easy because it did not need to compile or link any C code.
> >
> > This should still work, although I doubt if anyone tried it lately.
> > Emacs with native-compilation cannot be built that way, because the
> > native-code *.eln files are produced at build time, and cannot be
> > included in the tarball (they depend on the architecture of the target
> > machine and include signatures of the Emacs binary and the location of
> > the source *.el files).  But Emacs without native-compilation could be
> > produced in such two steps, except that the target environment still
> > needs some support tools: GNU Make, cp and rm.  (Alternatively, they
> > could invoke the final build commands by hand, if they know how.)
> 
> Is it possible/easy to cross-compile Emacs for Windows on a Unix
> machine?  Possibly with the help of Wine for dumping.

Maybe, I don't know.  I never tried, and neither I think has anyone
else.

You'd still need some GNU tools on the target, as indicated above, or
you will have to type the relevant commands manually on the target.

> Debian has a package gcc-mingw-w64-x86-64, so I suppose installing the
> cross compiler itself is fairly easy.

Yes, it is.  (You need Binutils as well, and the headers of the
optional libraries, like XPM, JPEG, GnuTLS, etc.).

Re: The 'cross' directory

[Helmut Eller]

On Tue, Aug 15 2023, Eli Zaretskii wrote:

>> From: Helmut Eller <eller.helmut@gmail.com>
>> Is it possible/easy to cross-compile Emacs for Windows on a Unix
>> machine?  Possibly with the help of Wine for dumping.
>
> Maybe, I don't know.  I never tried, and neither I think has anyone
> else.

I tried it now.  It kinda works, but it's not easy.  What follows is a
summary of the steps that I did, for the unlikely case that somebody is
interested in these things:

1. Run configure with:
   configure --host=x86_64-w64-mingw32 --without-gnutls
    --without-libsystemd --without-xml2 --without-dbus
    --prefix=/z/opt/emacs -C CFLAGS="-g -O1"

   The crucial bit here is the --host argument.  The --without-X
   switches are only needed because the configure script incorrectly
   detects those features.  The --prefix option is probably not needed,
   as I didn't actually call "make install".

2. This produced errors like:
   ./build-aux/msys-to-w32: line 79: pwd: -W: invalid option
  
   The -W option seems to be a cygwin/msys specific option, for printing
   Windows filenames in a form like c:/foo/bar.  I tried to replace this
   with "winepath -w" but that uses the form c:\foo\bar and so it didn't
   work either.  In the end I removed the -W option and edited the
   src/epaths.h file manually.  In particular

   #define PATH_DUMPLOADSEARCH "Z:/emacs-mingw/lisp"

   must be set correctly, otherwise temacs will abort in init_charset.
   
3. Executing "make" goes a bit further, but while building emacsclient
   an error like

   can't open icon file `icons\emacs.ico': No such file or directory

   occurs.  In emacsclient.rc.in, I replaced icons\emacs.ico with
   icons/emacs.ico.  This is probably safe, as emacs.rc.in also uses
   forward slashes.

4. Executing "make" now wants to execute make-docfile.exe, but can't.

   In autoconf terminology, make-docfile is executed on the build
   machine and not on the host machine.  However the Makefiles aren't
   prepared for such a distinction.  For a while I tried to introduce
   CC_FOR_BUILD and CLFAGS_FOR_BUILD variables, patterned after the
   configure scripts of the binutits.  That kinda worked for
   make-docfile, but not well enough make-fingerprint, which is needed a
   bit later and has the same problem.

   So I gave up on CC_FOR_BUILD and instead copied make-docfile and
   make-fingerprint from a normal, non-cross build directory, like

   cp ../emacs-git/lib-src/make-docfile lib-src/make-docfile.exe
   cp ../emacs-git/lib-src/make-fingerprint lib-src/make-fingerprint.exe

5. After that, "make" continues until it wants to run temacs the first
   time.

   To pass this hurdle, I created a script, wine-temacs.sh, that
   basically executes: wine $emacsdir/src/temacs.exe "$@"

   It's a bit more complicated than that, because wine, mysteriously,
   can't print to stderr and instead produces helpful messages of the
   form:

   Z:\emacs-mingw\src\emacs.exe: Write error to standard output: Success

   Maybe my wine installation is just botched.

   Anyway, running temacs with wine actually produces
   bootstrap-emacs.pdmp.

6. Now "make RUN_TEMACS=$(pwd)/src/wine-temacs.sh" wants run bootstrap-emacs
   to produce all the .elc files.

   For that I wrote another script, wine-bootstrap-emacs.sh, that just
   like wine-temacs.sh, runs bootstrap-emacs.exe in wine.

7. Now "make RUN_TEMACS=$(pwd)/src/wine-temacs.sh \
             bootstrap_exe=$(pwd)/src/wine-bootstrap-emacs.sh"
   wants to compile the .elc files.

   This would be slow, so I aborted and instead copied all the .elc
   files from the non-cross build and touched them like

   find . -name '*.elc' -exec touch {} +

   so that Make would consider them new.

8. Even with all the .elc files in place, make still wants to execute
   bootstrap-emacs for things like

   GEN ../lisp/leim/quail/CCDOSPY.el
   or
   GEN ../../lisp/cedet/semantic/bovine/c-by.el
   or
   ELC ../lisp/loaddefs.elc

   I just waited and let it proceed, as I didn't know any shortcuts.

9. Eventually it creates the all important emacs.pdmp.

   At that point make, still wants to continue, this time executing
   emacs.exe.  For which I wrote the thirt script, wine-emacs.sh, which
   does again the usual.

10. Now "make RUN_TEMACS=$(pwd)/src/wine-temacs.sh \
              bootstrap_exe=$(pwd)/src/wine-bootstrap-emacs.sh \
              EMACS=$(pwd)/src/wine-emacs.sh"
    still doesn't stop and produces things like

    GEN      autoloads
    ...
    ELC      cedet/semantic/grammar-wy.elc
    ...
    GEN      org.texi
    ...

11. Eventually it aborts with

    ***  "make all" succeeded, but Emacs is not functional.

    The filename used in the sanity-check can't be easily overriden so I
    stopped here.

Executing "wine src/emacs.exe" does however start an GUI Emacs and at
least the basic things seem to work.  "wine src/emacs.exe -nw" mostly
works, but some keys like, C-c, '(' and ')' seem to get lost somewhere.
In "wineconsole src/emacs.exe -nw" all keys seem to work as expected,
but it's a "choppy" experience as wine (or emacs) seems to sleep for a
moment after every few keystrokes.

It also starts on a real Windows laptop.  I used sshfs to mount the
build directory and executed emacs.exe there.  So that wasn't a proper
installation as "make install" would, of course, not have worked on
Windows.

Helmut

Re: The 'cross' directory

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: rms@gnu.org,  luangruo@yahoo.com,  emacs-devel@gnu.org
> Date: Wed, 16 Aug 2023 23:32:46 +0200
> 
> On Tue, Aug 15 2023, Eli Zaretskii wrote:
> 
> >> From: Helmut Eller <eller.helmut@gmail.com>
> >> Is it possible/easy to cross-compile Emacs for Windows on a Unix
> >> machine?  Possibly with the help of Wine for dumping.
> >
> > Maybe, I don't know.  I never tried, and neither I think has anyone
> > else.
> 
> I tried it now.  It kinda works, but it's not easy.  What follows is a
> summary of the steps that I did, for the unlikely case that somebody is
> interested in these things:

Thanks.  The use of Posix configury for building Emacs for Windows was
designed and tested only for native (host = target) builds, and was
never tested otherwise, so some hidden assumptions and other similar
issues are not surprising.

> 4. Executing "make" now wants to execute make-docfile.exe, but can't.

I believe this is not a Windows specific issue.

> 6. Now "make RUN_TEMACS=$(pwd)/src/wine-temacs.sh" wants run bootstrap-emacs
>    to produce all the .elc files.

Only if you build from Git.  If you build from the release tarball,
all the *.elc files are supposed to be already present in the tree.

> 8. Even with all the .elc files in place, make still wants to execute
>    bootstrap-emacs for things like
> 
>    GEN ../lisp/leim/quail/CCDOSPY.el
>    or
>    GEN ../../lisp/cedet/semantic/bovine/c-by.el
>    or
>    ELC ../lisp/loaddefs.elc
> 
>    I just waited and let it proceed, as I didn't know any shortcuts.

Why is this a problem?  If it runs on the host (which it should),
that's okay.  If it runs only after Emacs was dumped, that's indeed a
problem, but again: in a release tarball all those *.elc files should
already be present.

Re: The 'cross' directory

[Helmut Eller]

On Thu, Aug 17 2023, Eli Zaretskii wrote:

>> 8. Even with all the .elc files in place, make still wants to execute
>>    bootstrap-emacs for things like
>> 
>>    GEN ../lisp/leim/quail/CCDOSPY.el
>>    or
>>    GEN ../../lisp/cedet/semantic/bovine/c-by.el
>>    or
>>    ELC ../lisp/loaddefs.elc
>> 
>>    I just waited and let it proceed, as I didn't know any shortcuts.
>
> Why is this a problem?  If it runs on the host (which it should),
> that's okay.  If it runs only after Emacs was dumped, that's indeed a
> problem, but again: in a release tarball all those *.elc files should
> already be present.

It's not a problem.  It was just surprising to me; but I only have
superficial and probably incorrect understanding of the build process.
It might me a problem if switching frequently between build and host
machine is slow or impractical.

Helmut

Re: The 'cross' directory

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > So I think it should have worked to make `temacs' cross-compiling
  > > and then run the rest on the target machine.  That should have been
  > > easy because it did not need to compile or link any C code.

  > This should still work, although I doubt if anyone tried it lately.


Would someone please try this?  It is not tremendously important,
but if it has minor bugs we may as well fix them.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: The 'cross' directory

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Oops.  I wrote

  > Is it true that cross-compiling Emacs does not work
  > except when the target is Emacs?

I meant to write

  > Is it true that cross-compiling Emacs does not work
  > except when the target is Android>

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: The 'cross' directory

[Jose E. Marchesi]

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> Oops.  I wrote
>
>   > Is it true that cross-compiling Emacs does not work
>   > except when the target is Emacs?
>
> I meant to write
>
>   > Is it true that cross-compiling Emacs does not work
>   > except when the target is Android>

Pity.  I thought you were doing some sort of higher-level zen-like
philosophy and was eagerly waiting for the answer!

RE: [External] : Re: The 'cross' directory

[Drew Adams]

> > Oops.  I wrote
> >
> >   > Is it true that cross-compiling Emacs does not work
> >   > except when the target is Emacs?
> >
> > I meant to write
> >
> >   > Is it true that cross-compiling Emacs does not work
> >   > except when the target is Android>
> 
> Pity.  I thought you were doing some sort of higher-level zen-like
> philosophy and was eagerly waiting for the answer!

It's higher higher-level.
And that _is_ the answer. ;-)