Re: Interest in a Cosmopolitan build?

Re: Interest in a Cosmopolitan build?

[Alex Corcoles]

On Wed, 2023-11-08 at 12:00 -0500, emacs-devel-request@gnu.org wrote:
> From: Emanuel Berg <incal@dataswamp.org>
> Message-ID: <87edh0k2jf.fsf@dataswamp.org>
>    Cosmopolitan Libc makes C a build-once run-anywhere
>    language, like Java, except it doesn't need an interpreter
>    or virtual machine. [1]
> Sounds interesting!
> I wonder how they do that?

If I understand correctly, they found a way to make something that fits
all executable formats that major OSs support, plus they embed a zip
with binaries and dependencies (e.g. parts of /usr/ for Emacs), plus
they even convinced the POSIX stewards to loosen up the definition of
what's an executable. Plus a new libc...

> > Even though Cosmopolitan can be considered controversial
> 
> Why is it controversial?

I'm not entirely sure their means to achieve their objectives are
"clean". Also, it's yet another libc. Some people I've discussed the
idea with were a bit hostile towards it.

Cheers,

Álex

Re: Interest in a Cosmopolitan build?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If I understand correctly, they found a way to make something that fits
  > all executable formats that major OSs support, plus they embed a zip
  > with binaries and dependencies (e.g. parts of /usr/ for Emacs), plus
  > they even convinced the POSIX stewards to loosen up the definition of
  > what's an executable. Plus a new libc...

Could you possibly explain concretely what "a new libc" means in
this context?  In what sense is it "new"?  In what ways is it
different from the existing libc that any POSIX system would have?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Interest in a Cosmopolitan build?

[Alex Corcoles]

Hi,

>   > If I understand correctly, they found a way to make something
> that fits
>   > all executable formats that major OSs support, plus they embed a
> zip
>   > with binaries and dependencies (e.g. parts of /usr/ for Emacs),
> plus
>   > they even convinced the POSIX stewards to loosen up the
> definition of
>   > what's an executable. Plus a new libc...
> 
> Could you possibly explain concretely what "a new libc" means in
> this context?  In what sense is it "new"?  In what ways is it
> different from the existing libc that any POSIX system would have?

I am afraid I am a novice about systems programming, C, and
portability. I can quote parts from their README:

> Please note that the Cosmopolitan build configuration doesn't link
> any C/C++ library dependencies by default, so you still have the 
> flexibility to choose the one provided by your system. If you'd 
> prefer Cosmopolitan, just add $(LIBC) and $(CRT) to your linker
>arguments.

> The Cosmopolitan Library exports only the most stable canonical
> system calls for all supported operating systems, regardless of which
> platform is used for compilation. We polyfill many of the APIs, e.g.
> read(), write() so they work consistently everywhere while other 
> apis, e.g. CreateWindowEx(), might only work on one platform, in
> which case they become no-op functions on others.

> Please note that, unlike Cygwin or MinGW, Cosmopolitan does not
> achieve broad support by bolting on a POSIX emulation layer. We do
> nothing more than (in most cases) stateless API translations that get
> you 90% of the way there in a fast lightweight manner. We therefore 
> can't address some of the subtle differences, such as the nuances of
> absolute paths on Windows.

I can provide you further links, although they host their code on
GitHub. I'm happy to push it to some other Git forge, or host a tarball
of the source on a system of my own. I'm also happy to answer any other
question, although really I don't feel qualified for this discussion.

Not sure, though, if this remains on-topic for this list. Happy to
continue this discussion in some other place if needed.

Cheers,

Álex

Interest in a Cosmopolitan build?

[Alex Corcoles]

Hi,

I submitted this [1] to the help list, but I was suggested this list
might be better.

I have been able to build Emacs 29.1 using Cosmopolitan. This solves
neatly my issue of running the latest version of Emacs in all my
systems. Cosmopolitan generates a self-contained binary that can run in
different OS and CPU combinations.

Is anyone interested in such a build?

Currently this requires some hacky patches [2]. Not sure if these are
gaps in Cosmopolitan (perhaps I could file some bugs if I knew what to
ask), or maybe this is something that could be applied in Emacs itself
(at least the enum stuff. The enums I think need to be fixed in a
different way)?

Even though Cosmopolitan can be considered controversial, I think it
can be a nice way to distribute Emacs.

Cheers,

Álex


[1]
https://lists.gnu.org/archive/html/help-gnu-emacs/2023-11/msg00010.html
[2]
https://github.com/alexpdp7/superconfigure/blob/emacs_29.1/emacs-29.1/minimal.diff

Re: Interest in a Cosmopolitan build?

[Emanuel Berg]

Alex Corcoles wrote:

> I have been able to build Emacs 29.1 using Cosmopolitan.
> This solves neatly my issue of running the latest version of
> Emacs in all my systems.

   Cosmopolitan Libc makes C a build-once run-anywhere
   language, like Java, except it doesn't need an interpreter
   or virtual machine. [1]

Sounds interesting!

I wonder how they do that?

> Even though Cosmopolitan can be considered controversial

Why is it controversial?

[1] https://github.com/jart/cosmopolitan

-- 
underground experts united
https://dataswamp.org/~incal

Re: Interest in a Cosmopolitan build?

[Eli Zaretskii]

> From: Alex Corcoles <alex@corcoles.net>
> Date: Tue, 07 Nov 2023 22:23:21 +0100
> 
> I have been able to build Emacs 29.1 using Cosmopolitan. This solves
> neatly my issue of running the latest version of Emacs in all my
> systems. Cosmopolitan generates a self-contained binary that can run in
> different OS and CPU combinations.
> 
> Is anyone interested in such a build?

What is the license of Cosmopolitan?

> Currently this requires some hacky patches [2].

Can you post the patches here?  I'd prefer not to point people to
GitHub.

Thanks.

Re: Interest in a Cosmopolitan build?

[Björn Bidar]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Alex Corcoles <alex@corcoles.net>
>> Date: Tue, 07 Nov 2023 22:23:21 +0100
>> 
>> I have been able to build Emacs 29.1 using Cosmopolitan. This solves
>> neatly my issue of running the latest version of Emacs in all my
>> systems. Cosmopolitan generates a self-contained binary that can run in
>> different OS and CPU combinations.
>> 
>> Is anyone interested in such a build?
>
> What is the license of Cosmopolitan?

ISC LICENSE:
https://raw.githubusercontent.com/jart/cosmopolitan/master/LICENSE

Re: Interest in a Cosmopolitan build?

[Alex Corcoles]

Hi,


> Can you post the patches here?  I'd prefer not to point people to
> GitHub.

Not sure if this will come out OK:

8< 8< 8< 8< 8< 8< 8< 8< 8< 8<
--- ./emacs-29.1/src/config.in	2023-07-23 09:58:50.000000000 +0200
+++ ./emacs-29.1/src/config.in.new	2023-11-05 10:42:18.274387041
+0100
@@ -1790,7 +1790,7 @@
 #undef INTERNAL_TERMINAL
 
 /* Define to read input using SIGIO. */
-#undef INTERRUPT_INPUT
+#undef INTERRUPT_INPUT0
 
 /* Returns true if character is any form of separator. */
 #undef IS_ANY_SEP
@@ -2094,7 +2094,7 @@
 #undef UNIX98_PTYS
 
 /* Define to 1 if FIONREAD is usable. */
-#undef USABLE_FIONREAD
+#undef USABLE_FIONREAD0
 
 /* Define to 1 if SIGIO is usable. */
 #undef USABLE_SIGIO

--- ./emacs-29.1/src/filelock.c	2023-01-01 14:46:43.000000000 +0100
+++ ./emacs-29.1/src/filelock.c.new	2023-11-04 17:53:19.163803983
+0100
@@ -298,7 +298,7 @@
    There is no way to tell whether a symlink call fails due to
    permissions issues or because links are not supported, but luckily
    the lock file code should work either way.  */
-enum { LINKS_MIGHT_NOT_WORK = EPERM };
+#define LINKS_MIGHT_NOT_WORK EPERM
 
 /* Rename OLD to NEW.  If FORCE, replace any existing NEW.
    It is OK if there are temporarily two hard links to OLD.
@@ -498,17 +498,14 @@
 
 /* True if errno values are negative.  Although the C standard
    requires them to be positive, they are negative in Haiku.  */
-enum { NEGATIVE_ERRNO = EDOM < 0 };
+#define NEGATIVE_ERRNO (EDOM < 0)
 
 /* Nonzero values that are not errno values.  */
-enum
-  {
-    /* Another process on this machine owns it.  */
-    ANOTHER_OWNS_IT = NEGATIVE_ERRNO ? 1 : -1,
-
-    /* This Emacs process owns it.  */
-    I_OWN_IT = 2 * ANOTHER_OWNS_IT
-  };
+/* Another process on this machine owns it.  */
+#define ANOTHER_OWNS_IT (NEGATIVE_ERRNO ? 1 : -1)
+
+/* This Emacs process owns it.  */
+#define I_OWN_IT (2 * ANOTHER_OWNS_IT)
 
 /* Return 0 if nobody owns the lock file LFNAME or the lock is
obsolete,
    ANOTHER_OWNS_IT if another process owns it
8< 8< 8< 8< 8< 8< 8< 8< 8< 8<

The undef's I think clearly need to be solved in Cosmopolitan. The
enums... I'm not so sure.

Cheers,

Álex

Re: Interest in a Cosmopolitan build?

[Po Lu]

Alex Corcoles <alex@corcoles.net> writes:

> Hi,
>
>
>> Can you post the patches here?  I'd prefer not to point people to
>> GitHub.
>
> Not sure if this will come out OK:
>
> 8< 8< 8< 8< 8< 8< 8< 8< 8< 8<
> --- ./emacs-29.1/src/config.in	2023-07-23 09:58:50.000000000 +0200
> +++ ./emacs-29.1/src/config.in.new	2023-11-05 10:42:18.274387041
> +0100
> @@ -1790,7 +1790,7 @@
>  #undef INTERNAL_TERMINAL
>  
>  /* Define to read input using SIGIO. */
> -#undef INTERRUPT_INPUT
> +#undef INTERRUPT_INPUT0
>  
>  /* Returns true if character is any form of separator. */
>  #undef IS_ANY_SEP
> @@ -2094,7 +2094,7 @@
>  #undef UNIX98_PTYS
>  
>  /* Define to 1 if FIONREAD is usable. */
> -#undef USABLE_FIONREAD
> +#undef USABLE_FIONREAD0

Changing config.h is not correct, and it is automatically generated from
configure.ac at any rate.  Corrections to macros defined in config.h
should be applied in the configury.

>  /* Define to 1 if SIGIO is usable. */
>  #undef USABLE_SIGIO
>
> --- ./emacs-29.1/src/filelock.c	2023-01-01 14:46:43.000000000 +0100
> +++ ./emacs-29.1/src/filelock.c.new	2023-11-04 17:53:19.163803983
> +0100
> @@ -298,7 +298,7 @@
>     There is no way to tell whether a symlink call fails due to
>     permissions issues or because links are not supported, but luckily
>     the lock file code should work either way.  */
> -enum { LINKS_MIGHT_NOT_WORK = EPERM };
> +#define LINKS_MIGHT_NOT_WORK EPERM
>  
>  /* Rename OLD to NEW.  If FORCE, replace any existing NEW.
>     It is OK if there are temporarily two hard links to OLD.
> @@ -498,17 +498,14 @@
>  
>  /* True if errno values are negative.  Although the C standard
>     requires them to be positive, they are negative in Haiku.  */
> -enum { NEGATIVE_ERRNO = EDOM < 0 };
> +#define NEGATIVE_ERRNO (EDOM < 0)
>  
>  /* Nonzero values that are not errno values.  */
> -enum
> -  {
> -    /* Another process on this machine owns it.  */
> -    ANOTHER_OWNS_IT = NEGATIVE_ERRNO ? 1 : -1,
> -
> -    /* This Emacs process owns it.  */
> -    I_OWN_IT = 2 * ANOTHER_OWNS_IT
> -  };
> +/* Another process on this machine owns it.  */
> +#define ANOTHER_OWNS_IT (NEGATIVE_ERRNO ? 1 : -1)
> +
> +/* This Emacs process owns it.  */
> +#define I_OWN_IT (2 * ANOTHER_OWNS_IT)
>  
>  /* Return 0 if nobody owns the lock file LFNAME or the lock is
> obsolete,
>     ANOTHER_OWNS_IT if another process owns it
> 8< 8< 8< 8< 8< 8< 8< 8< 8< 8<
>
> The undef's I think clearly need to be solved in Cosmopolitan. The
> enums... I'm not so sure.

Why is that?  errno values are constant expressions in Standard C, so
this is a deviation from it on the part of Cosmopolitan.

Now, furthermore, I see a more general shortcoming to this:

Emacs employs a statically linked library names gnulib that supplies
substitutes for operating system functions suffering from deficiencies.
To establish which substitutes must be applied, configure executes tests
for all known defects in various system functions.  Since Cosmopolitan
binaries run on a variety of systems at once, these tests should be run
on all those systems, and the failures aggregated into a single
configuration file sourced from configure.ac.  Absent these precautions,
the generated Emacs binary will not function properly.

Re: Interest in a Cosmopolitan build?

[Eli Zaretskii]

> From: Alex Corcoles <alex@corcoles.net>
> Cc: emacs-devel@gnu.org
> Date: Wed, 08 Nov 2023 21:11:51 +0100
> 
> > Can you post the patches here?  I'd prefer not to point people to
> > GitHub.
> 
> Not sure if this will come out OK:

Thanks, but can you explain the reasons for these changes?  They look
strange to me.  E.g., why do you care which symbol is #undef'ed here:

>  /* Define to read input using SIGIO. */
> -#undef INTERRUPT_INPUT
> +#undef INTERRUPT_INPUT0

Or why do you need to avoid using enum here:

> --- ./emacs-29.1/src/filelock.c	2023-01-01 14:46:43.000000000 +0100
> +++ ./emacs-29.1/src/filelock.c.new	2023-11-04 17:53:19.163803983
> +0100
> @@ -298,7 +298,7 @@
>     There is no way to tell whether a symlink call fails due to
>     permissions issues or because links are not supported, but luckily
>     the lock file code should work either way.  */
> -enum { LINKS_MIGHT_NOT_WORK = EPERM };
> +#define LINKS_MIGHT_NOT_WORK EPERM
> +/* Another process on this machine owns it.  */
> +#define ANOTHER_OWNS_IT (NEGATIVE_ERRNO ? 1 : -1)
> +
> +/* This Emacs process owns it.  */
> +#define I_OWN_IT (2 * ANOTHER_OWNS_IT)

This sounds like a kludgey solution fopr some problem, but what is the
problem?

Are these all the changes you need, or just a sample?

Re: Interest in a Cosmopolitan build?

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> This sounds like a kludgey solution fopr some problem, but what is the
> problem?

errno values are not constant under Cosmopolitan, since they vary by the
underlying OS the binary runs on.

Re: Interest in a Cosmopolitan build?

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Alex Corcoles <alex@corcoles.net>,  emacs-devel@gnu.org
> Date: Thu, 09 Nov 2023 16:32:19 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > This sounds like a kludgey solution fopr some problem, but what is the
> > problem?
> 
> errno values are not constant under Cosmopolitan, since they vary by the
> underlying OS the binary runs on.

Then the change must be specific to Cosmopolitan, as I don't think we
want to punish other systems on this behalf.  Macros make debugging
harder when people forget to compile with -g3, which they almost
always do.

Re: Interest in a Cosmopolitan build?

[Alex Corcoles]

Hi,

> 
> > errno values are not constant under Cosmopolitan, since they vary
> > by the
> > underlying OS the binary runs on.
> 
> Then the change must be specific to Cosmopolitan, as I don't think we
> want to punish other systems on this behalf.  Macros make debugging
> harder when people forget to compile with -g3, which they almost
> always do.

Ah, thanks for the insight. Now I understand the enum thing. I don't
know how severe would this punishment be, but I trust others in this
thread.

Then I suspect the current approach is fine in that regard, and
probably maintaining the build out of tree (or other means) is better
than upstreaming the changes.

Right now, the binary is usable, but I have stumbled into some issues
(e.g. no file modification support, so Helm is slightly impeded). As I
mentioned somewhere else in the thread, an out-of-tree solution meets
my needs. Still, I'd like to bait someone more knowledgeable that sees
this as a useful feature to help :)

(Of course, someone with those skills is likely to not mind building
Emacs. But I still think this might be a nice way for some novices like
me to get the latest Emacs in an easy to use manner.)

Cheers,

Álex

Re: Interest in a Cosmopolitan build?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I have been able to build Emacs 29.1 using Cosmopolitan. This solves
  > neatly my issue of running the latest version of Emacs in all my
  > systems. Cosmopolitan generates a self-contained binary that can run in
  > different OS and CPU combinations.

This could in principle be a good feature to support, but it also
raises possible moral issues.  Whet software would get linked, or
packages, dith Emacs in this kind of binary?  We may have boith a
legal issue and a moral issue to address:

* The legal issue: would all that software satisfy the "system library
  exception" in the GNU GPL?

* The moral issue: would any of that software be nonfree, and would it
include any material that a straightforward byuild would NOT package
with Emacs?

Before supporting this, we need to establish that there is no moral
problem with it.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Interest in a Cosmopolitan build?

[Alex Corcoles]

Hi,

> This could in principle be a good feature to support, but it also
> raises possible moral issues.  Whet software would get linked, or
> packages, dith Emacs in this kind of binary?  We may have boith a
> legal issue and a moral issue to address:
> 
> * The legal issue: would all that software satisfy the "system
> library
>   exception" in the GNU GPL?
> 
> * The moral issue: would any of that software be nonfree, and would
> it
> include any material that a straightforward byuild would NOT package
> with Emacs?
> 
> Before supporting this, we need to establish that there is no moral
> problem with it.

I share your concern. I believe Cosmopolitan might offer some benefits
to the distribution of "moral issues-free" software, which might make
it interesting to have a conversation broader than the Emacs scope
about it. Unfortunately, I don't feel equipped to have this
conversation, although I would have great interest in it. I feel
Cosmopolitan might be an important project.

About using it for Emacs, to be clear, I'm interested in using Emacs
distributed as a Cosmopolitan binary, but I don't even intend
*anything* to be done in Emacs itself, if it poses a problem. Perhaps
*something* can help Emacs be more portable or improve it, but I'm not
even sure of that. I expect a Cosmopolitan build could be maintained
out of tree (as it is now), if necessary, and be strictly opt-in.

My intention with this thread was a bit selfish- to see if any Emacs
build expert was interested in the advantages of Cosmopolitan, because
I managed to get it to a working state for me, but I fear maintenance
of this might be beyond my means- and this is already giving me some
benefits I'd like to enjoy in the future :)

About the license details, it seems it uses GCC, clang, and its own
libc. I believe there's ISC, BSD, MIT, and Apache 2.0 software being
used, however I am not an expert in licensing, and perhaps they are not
well documented. As I mentioned, I would be interested in an auditing
of Cosmopolitan from a moral perspective.

Thanks,

Álex