unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / Atom feed
* [feature/dll-only-windows] A new windows build, comments wanted
@ 2021-01-09 19:57 Phillip Lord
  2021-01-09 20:18 ` Eli Zaretskii
                   ` (2 more replies)
  0 siblings, 3 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-09 19:57 UTC (permalink / raw)
  To: emacs-devel



In preparation for making a native-comp build of Emacs on Windows, I
have revisited the windows packaging process.

The current version figures out dependencies using msys2 and as a result
has an awful lot of dependencies. Even with some efforts to reduce this
size (e.g. python isn't included any more) it's still pretty large.

So I have tried a new technique which is now on
feature/dll-only-windows. Essentially, I just name all of the DLLs that
Emacs uses directly and then figure out any dependencies of these using
`ntldd`. I've tried `ntldd` directly on emacs.exe which does not
work. The practical upshot of this is that the dependencies file is just
a pile of DLLs. The installer version of Emacs (which is the smallest
because it is most compressed) is now only 45Mb (vs 63) in size, with
zip being 130Mb (vs 160Mb).

The source download still using the same technique as before; i.e. the
source bundle will produce a lot more than just the DLLs included.


Some questions for now:

1) Does this seem reasonable? I can some upload some binaries to alpha
if anyone is prepared to try them.

2) harfbuzz is currently not working on i686. Before I fix this, I think
it is worth asking whether I still need to produce a i686 binary. It
doubles the build time, obviously.

3) Currently the "no-deps" version actually includes libXpm. Emacs
starts without it, but looks ugly. I would like to no longer special
case libXpm and just make the "no-deps" download really include no
deps. I think this is reasonable, because this download is now, really
special purpose and "with-deps" is the default.


Some questions for when I add a native-comp build

4) Currently, native-comp is an "--with-nativecomp" option even on the
native-comp branch. It is likely to be merged this way to master?
Currently, I build the Windows distribution of Emacs with all the
default options. I can make an alternative release "--with-nativecomp"
but needs some effort; it also does not address the question of how I
should build Emacs for Windows when a full release of Emacs-28
happens. If, the full release of Emacs-28 will be native-comp, I'd
rather start building snapshots with it as soon as it is merged.

5) Why is it "--with-nativecomp", shouldn't it be "--with-native-comp".

Thoughts welcome.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 19:57 [feature/dll-only-windows] A new windows build, comments wanted Phillip Lord
@ 2021-01-09 20:18 ` Eli Zaretskii
  2021-01-09 21:31   ` Phillip Lord
                     ` (3 more replies)
  2021-01-09 20:47 ` Alan Third
  2021-01-10 15:43 ` Phillip Lord
  2 siblings, 4 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-09 20:18 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Date: Sat, 09 Jan 2021 19:57:00 +0000
> 
> So I have tried a new technique which is now on
> feature/dll-only-windows. Essentially, I just name all of the DLLs that
> Emacs uses directly and then figure out any dependencies of these using
> `ntldd`. I've tried `ntldd` directly on emacs.exe which does not
> work.

Of course, it won't: we load all the DLLs dynamically at run time, we
don't link against their import libraries.  The only exceptions I know
of are (1) GMP, and (2) libgccjit (in the native-comp build).

> The practical upshot of this is that the dependencies file is just
> a pile of DLLs.

Did you verify that the DLLs include all of _their_ dependencies?  If
yes, how did you do that?

> 2) harfbuzz is currently not working on i686.

Is that a bug in the MinGW64 HarfBuzz port, or is that a bug in Emacs?
I'm using a 32-bit Emacs build with HarfBuzz all the time, but it's
HarfBuzz I built myself (it's available from the ezwinports site).

> Before I fix this, I think it is worth asking whether I still need
> to produce a i686 binary.

I cannot tell you what to do, but it would be nice to have binaries
that can be run on older Windows versions.  So if the build supports
XP and older Windows, keeping the 32-bit build would be an advantage.

> 3) Currently the "no-deps" version actually includes libXpm. Emacs
> starts without it, but looks ugly. I would like to no longer special
> case libXpm and just make the "no-deps" download really include no
> deps. I think this is reasonable, because this download is now, really
> special purpose and "with-deps" is the default.

If we believe no one will want the no-deps download, why have it at
all?  If we think someone will want it, I don't think they should be
punished by having BW icons on the tool bar.

> 4) Currently, native-comp is an "--with-nativecomp" option even on the
> native-comp branch. It is likely to be merged this way to master?

Yes, I think so.

> Currently, I build the Windows distribution of Emacs with all the
> default options.

Which non-default options of practical importance does that leave out?

> I can make an alternative release "--with-nativecomp" but needs some
> effort; it also does not address the question of how I should build
> Emacs for Windows when a full release of Emacs-28 happens. If, the
> full release of Emacs-28 will be native-comp, I'd rather start
> building snapshots with it as soon as it is merged.

I think you should build --with-nativecomp.  People can always
uninstall libgccjit or rename it if they don't want to use native
compilation.

> 5) Why is it "--with-nativecomp", shouldn't it be "--with-native-comp".

Probably, but we didn't yet get to splitting such thin hair in that
branch ;-)



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 19:57 [feature/dll-only-windows] A new windows build, comments wanted Phillip Lord
  2021-01-09 20:18 ` Eli Zaretskii
@ 2021-01-09 20:47 ` Alan Third
  2021-01-09 21:33   ` Phillip Lord
  2021-01-10  3:28   ` Eli Zaretskii
  2021-01-10 15:43 ` Phillip Lord
  2 siblings, 2 replies; 78+ messages in thread
From: Alan Third @ 2021-01-09 20:47 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

On Sat, Jan 09, 2021 at 07:57:00PM +0000, Phillip Lord wrote:
> 
> 3) Currently the "no-deps" version actually includes libXpm. Emacs
> starts without it, but looks ugly. I would like to no longer special
> case libXpm and just make the "no-deps" download really include no
> deps. I think this is reasonable, because this download is now, really
> special purpose and "with-deps" is the default.

Do you even really need libXpm? We manage without it on NS and Cairo
builds by using the built-in functions at src/image.c:4671. Perhaps
that could be extended to Windows as well?
-- 
Alan Third



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 20:18 ` Eli Zaretskii
@ 2021-01-09 21:31   ` Phillip Lord
  2021-01-10  8:49     ` Arash Esbati
  2021-01-10 16:45     ` Eli Zaretskii
  2021-01-09 21:36   ` Óscar Fuentes
                     ` (2 subsequent siblings)
  3 siblings, 2 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-09 21:31 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Date: Sat, 09 Jan 2021 19:57:00 +0000
>> 
>> So I have tried a new technique which is now on
>> feature/dll-only-windows. Essentially, I just name all of the DLLs that
>> Emacs uses directly and then figure out any dependencies of these using
>> `ntldd`. I've tried `ntldd` directly on emacs.exe which does not
>> work.
>
> Of course, it won't: we load all the DLLs dynamically at run time, we
> don't link against their import libraries.  The only exceptions I know
> of are (1) GMP, and (2) libgccjit (in the native-comp build).

Indeed.

>> The practical upshot of this is that the dependencies file is just
>> a pile of DLLs.
>
> Did you verify that the DLLs include all of _their_ dependencies?  If
> yes, how did you do that?

It's totally dependent on ntldd being correct, of course. It comes with
a --recursive option which I used rather than reimplementing it
myself. As ever, the only tests I run on the windows package for Emacs
are to unpack, start and then run "w32-feature.el" from etc. These all
pass.



>> 2) harfbuzz is currently not working on i686.
>
> Is that a bug in the MinGW64 HarfBuzz port, or is that a bug in Emacs?
> I'm using a 32-bit Emacs build with HarfBuzz all the time, but it's
> HarfBuzz I built myself (it's available from the ezwinports site).

I don't know, I haven't debugged it yet:-)

>> Before I fix this, I think it is worth asking whether I still need
>> to produce a i686 binary.
>
> I cannot tell you what to do, but it would be nice to have binaries
> that can be run on older Windows versions.  So if the build supports
> XP and older Windows, keeping the 32-bit build would be an advantage.

I know that you use a very old windows, but I doubt you use my packages!
I have no other data of how many people this would impact, but my guess
would be not many, so I am asking for opinions.

One solution could be to stop making the 32-bit build and see if anyone
complains. It could be restored, either before or after the Emacs-28
release.


>> 3) Currently the "no-deps" version actually includes libXpm. Emacs
>> starts without it, but looks ugly. I would like to no longer special
>> case libXpm and just make the "no-deps" download really include no
>> deps. I think this is reasonable, because this download is now, really
>> special purpose and "with-deps" is the default.
>
> If we believe no one will want the no-deps download, why have it at
> all?  If we think someone will want it, I don't think they should be
> punished by having BW icons on the tool bar.

Indeed. My guess is that currently if people use -no-deps they do so
because they want the smaller download. This change, of course, we
reduce the difference and regardless the installer version is smaller
still (by a half).

Given all that, the only real audience for the -no-deps version would be
those who have a mingw64 installation already. They won't get BW icons
(assuming that libXpm is installed). Of course, that audience is likely
to be technically skilled and they could just delete the DLLs in the
"with-deps" download.

On balance, therefore, I would say we don't need the -no-deps version
and it would clean up the download site and reduce confusion.


>> 4) Currently, native-comp is an "--with-nativecomp" option even on the
>> native-comp branch. It is likely to be merged this way to master?
>
> Yes, I think so.

Are you worried about stability or newness? I guess it will become the
default at some point. It seems to have few disadvantages, other than
hammering the CPU at bit at initial start up.


>
>> Currently, I build the Windows distribution of Emacs with all the
>> default options.
>
> Which non-default options of practical importance does that leave out?

Currently, none. If --with-nativecomp is not default, then, I think it
would become the example.

>> I can make an alternative release "--with-nativecomp" but needs some
>> effort; it also does not address the question of how I should build
>> Emacs for Windows when a full release of Emacs-28 happens. If, the
>> full release of Emacs-28 will be native-comp, I'd rather start
>> building snapshots with it as soon as it is merged.
>
> I think you should build --with-nativecomp.  People can always
> uninstall libgccjit or rename it if they don't want to use native
> compilation.

Okay. I guess "make NATIVE_FULL_AOT=1" would be the thing as well,
depending on how long it takes.


>> 5) Why is it "--with-nativecomp", shouldn't it be "--with-native-comp".
>
> Probably, but we didn't yet get to splitting such thin hair in that
> branch ;-)

Yeah, can't help it. Splitting hairs is my job.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 20:47 ` Alan Third
@ 2021-01-09 21:33   ` Phillip Lord
  2021-01-10  0:04     ` Alan Third
  2021-01-10  3:28   ` Eli Zaretskii
  1 sibling, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-09 21:33 UTC (permalink / raw)
  To: Alan Third; +Cc: emacs-devel

Alan Third <alan@idiocy.org> writes:

> On Sat, Jan 09, 2021 at 07:57:00PM +0000, Phillip Lord wrote:
>> 
>> 3) Currently the "no-deps" version actually includes libXpm. Emacs
>> starts without it, but looks ugly. I would like to no longer special
>> case libXpm and just make the "no-deps" download really include no
>> deps. I think this is reasonable, because this download is now, really
>> special purpose and "with-deps" is the default.
>
> Do you even really need libXpm? We manage without it on NS and Cairo
> builds by using the built-in functions at src/image.c:4671. Perhaps
> that could be extended to Windows as well?

I don't think we do need it in the version with deps. AFAIU, it's just
there as the smallest dependency that can give colour icons.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 20:18 ` Eli Zaretskii
  2021-01-09 21:31   ` Phillip Lord
@ 2021-01-09 21:36   ` Óscar Fuentes
  2021-01-10 16:46     ` Eli Zaretskii
  2021-01-09 21:51   ` Andrea Corallo via Emacs development discussions.
  2021-01-10 15:14   ` Phillip Lord
  3 siblings, 1 reply; 78+ messages in thread
From: Óscar Fuentes @ 2021-01-09 21:36 UTC (permalink / raw)
  To: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> Before I fix this, I think it is worth asking whether I still need
>> to produce a i686 binary.
>
> I cannot tell you what to do, but it would be nice to have binaries
> that can be run on older Windows versions.  So if the build supports
> XP and older Windows, keeping the 32-bit build would be an advantage.

MSYS2 MinGW-w64 binaries do not support XP, they officially decided this
some years ago and now some runtime dlls use APIs not present on XP.




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 20:18 ` Eli Zaretskii
  2021-01-09 21:31   ` Phillip Lord
  2021-01-09 21:36   ` Óscar Fuentes
@ 2021-01-09 21:51   ` Andrea Corallo via Emacs development discussions.
  2021-01-10  3:33     ` Eli Zaretskii
  2021-01-10 15:09     ` Phillip Lord
  2021-01-10 15:14   ` Phillip Lord
  3 siblings, 2 replies; 78+ messages in thread
From: Andrea Corallo via Emacs development discussions. @ 2021-01-09 21:51 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel, Phillip Lord

Eli Zaretskii <eliz@gnu.org> writes:

[...]

> I think you should build --with-nativecomp.  People can always
> uninstall libgccjit or rename it if they don't want to use native
> compilation.

ATM if using --with-nativecomp libgccjit is not present (or non
functional) we stop at configure time with an error.  I guess that's one
of the hairs we'll have to split then :)

  Andrea



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 21:33   ` Phillip Lord
@ 2021-01-10  0:04     ` Alan Third
  0 siblings, 0 replies; 78+ messages in thread
From: Alan Third @ 2021-01-10  0:04 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

On Sat, Jan 09, 2021 at 09:33:56PM +0000, Phillip Lord wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > Do you even really need libXpm? We manage without it on NS and Cairo
> > builds by using the built-in functions at src/image.c:4671. Perhaps
> > that could be extended to Windows as well?
> 
> I don't think we do need it in the version with deps. AFAIU, it's just
> there as the smallest dependency that can give colour icons.

Yes, my point is we have colour icons with no deps at all by using
functions already in image.c.

-- 
Alan Third



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 20:47 ` Alan Third
  2021-01-09 21:33   ` Phillip Lord
@ 2021-01-10  3:28   ` Eli Zaretskii
  1 sibling, 0 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-10  3:28 UTC (permalink / raw)
  To: Alan Third; +Cc: emacs-devel, phillip.lord

> Date: Sat, 9 Jan 2021 20:47:41 +0000
> From: Alan Third <alan@idiocy.org>
> Cc: emacs-devel@gnu.org
> 
> On Sat, Jan 09, 2021 at 07:57:00PM +0000, Phillip Lord wrote:
> > 
> > 3) Currently the "no-deps" version actually includes libXpm. Emacs
> > starts without it, but looks ugly. I would like to no longer special
> > case libXpm and just make the "no-deps" download really include no
> > deps. I think this is reasonable, because this download is now, really
> > special purpose and "with-deps" is the default.
> 
> Do you even really need libXpm? We manage without it on NS and Cairo
> builds by using the built-in functions at src/image.c:4671. Perhaps
> that could be extended to Windows as well?

Emacs supports native image display on Windows, but that is still
experimental.  In time, we could indeed try to get rid of the libXpm
dependency.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 21:51   ` Andrea Corallo via Emacs development discussions.
@ 2021-01-10  3:33     ` Eli Zaretskii
  2021-01-10 15:09     ` Phillip Lord
  1 sibling, 0 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-10  3:33 UTC (permalink / raw)
  To: Andrea Corallo; +Cc: emacs-devel, phillip.lord

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Phillip Lord <phillip.lord@russet.org.uk>, emacs-devel@gnu.org
> Date: Sat, 09 Jan 2021 21:51:16 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> [...]
> 
> > I think you should build --with-nativecomp.  People can always
> > uninstall libgccjit or rename it if they don't want to use native
> > compilation.
> 
> ATM if using --with-nativecomp libgccjit is not present (or non
> functional) we stop at configure time with an error.

The situation relevant here is that libgccjit is present at build
time, but not at ruin time (on another system).



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 21:31   ` Phillip Lord
@ 2021-01-10  8:49     ` Arash Esbati
  2021-01-10 15:19       ` Phillip Lord
  2021-01-10 16:45     ` Eli Zaretskii
  1 sibling, 1 reply; 78+ messages in thread
From: Arash Esbati @ 2021-01-10  8:49 UTC (permalink / raw)
  To: Phillip Lord; +Cc: Eli Zaretskii, emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Did you verify that the DLLs include all of _their_ dependencies?  If
>> yes, how did you do that?
>
> It's totally dependent on ntldd being correct, of course. It comes with
> a --recursive option which I used rather than reimplementing it
> myself.

cygcheck.exe has this capability as well.  Try

    cygcheck libjansson-4.dll

in a mingw64 console.  Maybe you can use this to double check the
dependencies reported by ntldd.

Best, Arash



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 21:51   ` Andrea Corallo via Emacs development discussions.
  2021-01-10  3:33     ` Eli Zaretskii
@ 2021-01-10 15:09     ` Phillip Lord
  2021-01-10 19:06       ` Andrea Corallo via Emacs development discussions.
  1 sibling, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-10 15:09 UTC (permalink / raw)
  To: Andrea Corallo; +Cc: Eli Zaretskii, emacs-devel

Andrea Corallo <akrl@sdf.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> [...]
>
>> I think you should build --with-nativecomp.  People can always
>> uninstall libgccjit or rename it if they don't want to use native
>> compilation.
>
> ATM if using --with-nativecomp libgccjit is not present (or non
> functional) we stop at configure time with an error.  I guess that's one
> of the hairs we'll have to split then :)


I think there are two hairs here. Having an emacs compiled with
nativecomp behave cleanly if libgccjit is not available at runtime. And
having a runtime mechanism for switching nativecomp of totally even if
it is compiled in.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 20:18 ` Eli Zaretskii
                     ` (2 preceding siblings ...)
  2021-01-09 21:51   ` Andrea Corallo via Emacs development discussions.
@ 2021-01-10 15:14   ` Phillip Lord
  2021-01-10 17:23     ` Eli Zaretskii
  3 siblings, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-10 15:14 UTC (permalink / raw)
  To: emacs-devel


My test for GMP is also currently failing. It is this:

(ert-deftest feature-gmp ()
  (should
   (string-match-p "GMP" system-configuration-features)))

This test also fails on Emacs-28 master on my Ubuntu machine, as well as
on my Windows package, although all of them report that libgmp support
is available at configure time. And the test succeeds on Emacs-27 on the
same machines. So it looks like the test is wrong, rather libgmp support
broken.

Any thoughts on why GMP is no longer in system-configuration-features
for Emacs-28. Is there a better test that I can use? Can we add
`gmp-available-p' or equivalent?

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10  8:49     ` Arash Esbati
@ 2021-01-10 15:19       ` Phillip Lord
  0 siblings, 0 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-10 15:19 UTC (permalink / raw)
  To: Arash Esbati; +Cc: Eli Zaretskii, emacs-devel

Arash Esbati <arash@gnu.org> writes:

> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>> Did you verify that the DLLs include all of _their_ dependencies?  If
>>> yes, how did you do that?
>>
>> It's totally dependent on ntldd being correct, of course. It comes with
>> a --recursive option which I used rather than reimplementing it
>> myself.
>
> cygcheck.exe has this capability as well.  Try
>
>     cygcheck libjansson-4.dll
>
> in a mingw64 console.  Maybe you can use this to double check the
> dependencies reported by ntldd.


Thanks, I will have a poke.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 19:57 [feature/dll-only-windows] A new windows build, comments wanted Phillip Lord
  2021-01-09 20:18 ` Eli Zaretskii
  2021-01-09 20:47 ` Alan Third
@ 2021-01-10 15:43 ` Phillip Lord
  2021-01-12  6:01   ` Corwin Brust
  2 siblings, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-10 15:43 UTC (permalink / raw)
  To: emacs-devel



I have pushed some binaries up to alpha in case anyone wants to give
them a trial.

Directory:

https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-28/

Two download links: as zip or installer version.


https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-28/emacs-28.0.50-snapshot-feature_dll-only-windows-2021-01-09-x86_64.zip
https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-28/emacs-28.0.50-snapshot-feature_dll-only-windows-2021-01-09-x86_64-installer.exe

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 21:31   ` Phillip Lord
  2021-01-10  8:49     ` Arash Esbati
@ 2021-01-10 16:45     ` Eli Zaretskii
  2021-01-10 18:23       ` Phillip Lord
  1 sibling, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-10 16:45 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: emacs-devel@gnu.org
> Date: Sat, 09 Jan 2021 21:31:45 +0000
> 
> > Did you verify that the DLLs include all of _their_ dependencies?  If
> > yes, how did you do that?
> 
> It's totally dependent on ntldd being correct, of course. It comes with
> a --recursive option which I used rather than reimplementing it
> myself.

You can use objdump:

  objdump -p *.exe *.dll | fgrep "DLL Name:" | gawk " {print $3, $4, $5}" | sort -u

Keep adding DLLs and running the above command until all the DLLs it
shows are in your staging directory.

> >> 4) Currently, native-comp is an "--with-nativecomp" option even on the
> >> native-comp branch. It is likely to be merged this way to master?
> >
> > Yes, I think so.
> 
> Are you worried about stability or newness? I guess it will become the
> default at some point. It seems to have few disadvantages, other than
> hammering the CPU at bit at initial start up.

Usually, such radical features start as optional, but maybe we will
decide otherwise.

Why does it matter?



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-09 21:36   ` Óscar Fuentes
@ 2021-01-10 16:46     ` Eli Zaretskii
  2021-01-10 18:34       ` Phillip Lord
  0 siblings, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-10 16:46 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sat, 09 Jan 2021 22:36:36 +0100
> 
> > I cannot tell you what to do, but it would be nice to have binaries
> > that can be run on older Windows versions.  So if the build supports
> > XP and older Windows, keeping the 32-bit build would be an advantage.
> 
> MSYS2 MinGW-w64 binaries do not support XP, they officially decided this
> some years ago and now some runtime dlls use APIs not present on XP.

That's what I knew.  So in that case, I see no reason for us to
produce official 32-bit binaries, as the chances for them being used
are likely nil.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10 15:14   ` Phillip Lord
@ 2021-01-10 17:23     ` Eli Zaretskii
  0 siblings, 0 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-10 17:23 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Date: Sun, 10 Jan 2021 15:14:03 +0000
> 
> Any thoughts on why GMP is no longer in system-configuration-features
> for Emacs-28.

It's a bug, please report it.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10 16:45     ` Eli Zaretskii
@ 2021-01-10 18:23       ` Phillip Lord
  2021-01-10 19:05         ` Eli Zaretskii
  0 siblings, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-10 18:23 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> >> 4) Currently, native-comp is an "--with-nativecomp" option even on the
>> >> native-comp branch. It is likely to be merged this way to master?
>> >
>> > Yes, I think so.
>> 
>> Are you worried about stability or newness? I guess it will become the
>> default at some point. It seems to have few disadvantages, other than
>> hammering the CPU at bit at initial start up.
>
> Usually, such radical features start as optional, but maybe we will
> decide otherwise.
>
> Why does it matter?

I'd like to keep the Windows package using the default options as far as
possible, mostly because I am lazy and because it means that the windows
package updates when ever the defaults do.

Currently, we have

--without-dbus --without-compress-install

The latter is so the documentation works without gzip. The latter is
because we don't normally have dbus available. I am not sure why either
of these needed (i.e. why they are not the default on windows).

Adding --with-nativecomp isn't massively problematic, but would need
removing when and if it becomes default. Not a big problem, the less
difference between the windows package and the defaults, the happier I
would be.

Phil





^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10 16:46     ` Eli Zaretskii
@ 2021-01-10 18:34       ` Phillip Lord
  0 siblings, 0 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-10 18:34 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Óscar Fuentes, emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Sat, 09 Jan 2021 22:36:36 +0100
>> 
>> > I cannot tell you what to do, but it would be nice to have binaries
>> > that can be run on older Windows versions.  So if the build supports
>> > XP and older Windows, keeping the 32-bit build would be an advantage.
>> 
>> MSYS2 MinGW-w64 binaries do not support XP, they officially decided this
>> some years ago and now some runtime dlls use APIs not present on XP.
>
> That's what I knew.  So in that case, I see no reason for us to
> produce official 32-bit binaries, as the chances for them being used
> are likely nil.


Then I shall disable the code that produces it for Emacs-28. I will
leave the code and the "-x86_64-" names in place, I think, just in case
we get any complaints from Windows 7/10 32-bit users. Then, that can go
for Emacs-29, so binaries will become simply "emacs-29.1.zip" or
"emacs-29.1-installer.exe".

Assuming that this smaller DLL only distribution tests once I start
producing snapshots, I will also remove the "-no-deps" zip from
Emacs-28. At this point, I no longer need the independent copies of the
dependencies, so they can go as well. That will reduce the total number
of files for releases from 8 binary + 1 source (for the deps), to 2
binary + 1 source.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10 18:23       ` Phillip Lord
@ 2021-01-10 19:05         ` Eli Zaretskii
  2021-01-10 19:20           ` Óscar Fuentes
                             ` (2 more replies)
  0 siblings, 3 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-10 19:05 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Date: Sun, 10 Jan 2021 18:23:06 +0000
> Cc: emacs-devel@gnu.org
> 
> I'd like to keep the Windows package using the default options as far as
> possible, mostly because I am lazy and because it means that the windows
> package updates when ever the defaults do.

It's your call.  I just envision many users to expect native-comp
support and asking why isn't it there.

> Currently, we have
> 
> --without-dbus --without-compress-install
> 
> The latter is so the documentation works without gzip. The latter is
> because we don't normally have dbus available. I am not sure why either
> of these needed (i.e. why they are not the default on windows).

The gzip thing is again your call (having gzip in the package is no
big deal, IMO).  Dbus is not really useful on MS-Windows, so IMO it
makes no sense building with it.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10 15:09     ` Phillip Lord
@ 2021-01-10 19:06       ` Andrea Corallo via Emacs development discussions.
  2021-01-11  9:47         ` Phillip Lord
  0 siblings, 1 reply; 78+ messages in thread
From: Andrea Corallo via Emacs development discussions. @ 2021-01-10 19:06 UTC (permalink / raw)
  To: Phillip Lord; +Cc: Eli Zaretskii, emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> Andrea Corallo <akrl@sdf.org> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> [...]
>>
>>> I think you should build --with-nativecomp.  People can always
>>> uninstall libgccjit or rename it if they don't want to use native
>>> compilation.
>>
>> ATM if using --with-nativecomp libgccjit is not present (or non
>> functional) we stop at configure time with an error.  I guess that's one
>> of the hairs we'll have to split then :)
>
>
> I think there are two hairs here. Having an emacs compiled with
> nativecomp behave cleanly if libgccjit is not available at runtime.

Right, at this stage this should be easy to implement (on Windows).

> And having a runtime mechanism for switching nativecomp of totally
> even if it is compiled in.

We should define "switching nativecomp of" and the triggering mechanism,
this might be already implmentented.

  Andrea



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10 19:05         ` Eli Zaretskii
@ 2021-01-10 19:20           ` Óscar Fuentes
  2021-01-10 19:37             ` Eli Zaretskii
  2021-01-10 20:52           ` `gzip` dependency (was: [feature/dll-only-windows] A new windows build, comments wanted) Stefan Monnier
  2021-01-11  9:59           ` [feature/dll-only-windows] A new windows build, comments wanted Phillip Lord
  2 siblings, 1 reply; 78+ messages in thread
From: Óscar Fuentes @ 2021-01-10 19:20 UTC (permalink / raw)
  To: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> I'd like to keep the Windows package using the default options as far as
>> possible, mostly because I am lazy and because it means that the windows
>> package updates when ever the defaults do.
>
> It's your call.  I just envision many users to expect native-comp
> support and asking why isn't it there.

I agree. In fact, some popular packages already recommend using
native-comp.

>> Currently, we have
>> 
>> --without-dbus --without-compress-install
>> 
>> The latter is so the documentation works without gzip. The latter is
>> because we don't normally have dbus available. I am not sure why either
>> of these needed (i.e. why they are not the default on windows).
>
> The gzip thing is again your call (having gzip in the package is no
> big deal, IMO).  Dbus is not really useful on MS-Windows, so IMO it
> makes no sense building with it.

I think that Phillip's problem with that is that gzip does not exist as
a MinGW package in the MSYS2 ecosystem.




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10 19:20           ` Óscar Fuentes
@ 2021-01-10 19:37             ` Eli Zaretskii
  0 siblings, 0 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-10 19:37 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sun, 10 Jan 2021 20:20:22 +0100
> 
> I think that Phillip's problem with that is that gzip does not exist as
> a MinGW package in the MSYS2 ecosystem.

I'm surprised, but then it's very easy to build it.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* `gzip` dependency (was: [feature/dll-only-windows] A new windows build, comments wanted)
  2021-01-10 19:05         ` Eli Zaretskii
  2021-01-10 19:20           ` Óscar Fuentes
@ 2021-01-10 20:52           ` Stefan Monnier
  2021-01-11  3:27             ` Eli Zaretskii
  2021-01-20 19:29             ` [feature/internal-msys] thoughts of a more function windows package Phillip Lord
  2021-01-11  9:59           ` [feature/dll-only-windows] A new windows build, comments wanted Phillip Lord
  2 siblings, 2 replies; 78+ messages in thread
From: Stefan Monnier @ 2021-01-10 20:52 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel, Phillip Lord

> The gzip thing is again your call (having gzip in the package is no
> big deal, IMO).

BTW, we don't really need a `gzip` executable as long as we have the
`zlib-decompress-region` function, right?


        Stefan




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: `gzip` dependency (was: [feature/dll-only-windows] A new windows build, comments wanted)
  2021-01-10 20:52           ` `gzip` dependency (was: [feature/dll-only-windows] A new windows build, comments wanted) Stefan Monnier
@ 2021-01-11  3:27             ` Eli Zaretskii
  2021-01-11 10:00               ` `gzip` dependency Phillip Lord
  2021-01-11 14:59               ` Stefan Monnier
  2021-01-20 19:29             ` [feature/internal-msys] thoughts of a more function windows package Phillip Lord
  1 sibling, 2 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-11  3:27 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel, phillip.lord

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Phillip Lord <phillip.lord@russet.org.uk>,  emacs-devel@gnu.org
> Date: Sun, 10 Jan 2021 15:52:07 -0500
> 
> > The gzip thing is again your call (having gzip in the package is no
> > big deal, IMO).
> 
> BTW, we don't really need a `gzip` executable as long as we have the
> `zlib-decompress-region` function, right?

Not necessarily, because Info files can be read outside Emacs.

Did we completely replaced jka-compr in Emacs?



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10 19:06       ` Andrea Corallo via Emacs development discussions.
@ 2021-01-11  9:47         ` Phillip Lord
  2021-01-11 11:01           ` Andrea Corallo via Emacs development discussions.
  0 siblings, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-11  9:47 UTC (permalink / raw)
  To: Andrea Corallo; +Cc: Eli Zaretskii, emacs-devel

Andrea Corallo <akrl@sdf.org> writes:

> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>>
>> I think there are two hairs here. Having an emacs compiled with
>> nativecomp behave cleanly if libgccjit is not available at runtime.
>
> Right, at this stage this should be easy to implement (on Windows).
>
>> And having a runtime mechanism for switching nativecomp of totally
>> even if it is compiled in.
>
> We should define "switching nativecomp of" and the triggering mechanism,
> this might be already implmentented.


From a user perspective, I think that there are two reasons we might
want to switch off native-comp.

The first is because it's broken or providing buggy behaviour. In the
extreme, and if I understand the build process fully, I guess this is
impossible to switch of in a binary distribution, because some
native-comp happens before dumping. Other than that, we can "switch off"
native-comp by just deleting the *eln files, right, or otherwise
preventing their loading. In practice, I think this is a minor
motivation; if we discover bugs in the native-comp, they would be fixed
by making another release.

The second reason is that the initial compilation takes quite a lot of
CPU. I wouldn't like that to happen while my laptop where in battery
power, for instance. At the moment, it's possible to drop the number of
jobs that native-comp uses. I don't think that there is a way to drop
this to zero at the moment. Probably, we need that.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10 19:05         ` Eli Zaretskii
  2021-01-10 19:20           ` Óscar Fuentes
  2021-01-10 20:52           ` `gzip` dependency (was: [feature/dll-only-windows] A new windows build, comments wanted) Stefan Monnier
@ 2021-01-11  9:59           ` Phillip Lord
  2021-01-11 15:21             ` Eli Zaretskii
  2 siblings, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-11  9:59 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Date: Sun, 10 Jan 2021 18:23:06 +0000
>> Cc: emacs-devel@gnu.org
>> 
>> I'd like to keep the Windows package using the default options as far as
>> possible, mostly because I am lazy and because it means that the windows
>> package updates when ever the defaults do.
>
> It's your call.  I just envision many users to expect native-comp
> support and asking why isn't it there.

No, you misunderstand me. I would plan to make Emacs-28
--with-nativecomp. But if I am doing this for the windows build, it is
reasonable to assume that other packagers will be doing for their
builds. If all that is true, it would make sense to me for it to be the
default.



>
>> Currently, we have
>> 
>> --without-dbus --without-compress-install
>> 
>> The latter is so the documentation works without gzip. The latter is
>> because we don't normally have dbus available. I am not sure why either
>> of these needed (i.e. why they are not the default on windows).
>
> The gzip thing is again your call (having gzip in the package is no
> big deal, IMO).  Dbus is not really useful on MS-Windows, so IMO it
> makes no sense building with it.

We've been down this road before. I could add gzip and I am happy to
consider this. But what other tools do we add. What about aspell
(or equivalent), so we can spell check? Git so we can version? If I add
them, then we get more "out-of-the-box" behaviour. But where do we stop.

Actually, for both of those, I'd rather see Emacs depend on their
respective libraries. Magit is widely reported to be usuable on Windows;
libgit integration might work around this. Anyway, getting off-topic.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: `gzip` dependency
  2021-01-11  3:27             ` Eli Zaretskii
@ 2021-01-11 10:00               ` Phillip Lord
  2021-01-11 15:22                 ` Eli Zaretskii
  2021-01-11 14:59               ` Stefan Monnier
  1 sibling, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-11 10:00 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Stefan Monnier, emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: Phillip Lord <phillip.lord@russet.org.uk>,  emacs-devel@gnu.org
>> Date: Sun, 10 Jan 2021 15:52:07 -0500
>> 
>> > The gzip thing is again your call (having gzip in the package is no
>> > big deal, IMO).
>> 
>> BTW, we don't really need a `gzip` executable as long as we have the
>> `zlib-decompress-region` function, right?
>
> Not necessarily, because Info files can be read outside Emacs.


This is not an issue for the Windows package of Emacs, as far as I can tell.


Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-11  9:47         ` Phillip Lord
@ 2021-01-11 11:01           ` Andrea Corallo via Emacs development discussions.
  2021-01-11 16:29             ` Phillip Lord
  0 siblings, 1 reply; 78+ messages in thread
From: Andrea Corallo via Emacs development discussions. @ 2021-01-11 11:01 UTC (permalink / raw)
  To: Phillip Lord; +Cc: Eli Zaretskii, emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> Andrea Corallo <akrl@sdf.org> writes:
>
>> Phillip Lord <phillip.lord@russet.org.uk> writes:
>>
>>>
>>> I think there are two hairs here. Having an emacs compiled with
>>> nativecomp behave cleanly if libgccjit is not available at runtime.
>>
>> Right, at this stage this should be easy to implement (on Windows).
>>
>>> And having a runtime mechanism for switching nativecomp of totally
>>> even if it is compiled in.
>>
>> We should define "switching nativecomp of" and the triggering mechanism,
>> this might be already implmentented.
>
>
>>From a user perspective, I think that there are two reasons we might
> want to switch off native-comp.
>
> The first is because it's broken or providing buggy behaviour. In the
> extreme, and if I understand the build process fully, I guess this is
> impossible to switch of in a binary distribution, because some
> native-comp happens before dumping. Other than that, we can "switch off"
> native-comp by just deleting the *eln files, right, or otherwise
> preventing their loading. In practice, I think this is a minor
> motivation; if we discover bugs in the native-comp, they would be fixed
> by making another release.

Hi Phillip,

ATM we can prevent Emacs loading .eln file in place of .elc
automatically setting `load-no-native' to t.

> The second reason is that the initial compilation takes quite a lot of
> CPU. I wouldn't like that to happen while my laptop where in battery
> power, for instance. At the moment, it's possible to drop the number of
> jobs that native-comp uses. I don't think that there is a way to drop
> this to zero at the moment. Probably, we need that.

Setting `comp-deferred-compilation' to nil should stop any automatic
attempt of compiling asycronousy.

Maybe these two settings are already covering most of these needs?

Regards

  Andrea



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: `gzip` dependency
  2021-01-11  3:27             ` Eli Zaretskii
  2021-01-11 10:00               ` `gzip` dependency Phillip Lord
@ 2021-01-11 14:59               ` Stefan Monnier
  2021-01-11 15:15                 ` Phillip Lord
  2021-01-11 15:46                 ` Eli Zaretskii
  1 sibling, 2 replies; 78+ messages in thread
From: Stefan Monnier @ 2021-01-11 14:59 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel, phillip.lord

>> > The gzip thing is again your call (having gzip in the package is no
>> > big deal, IMO).
>> BTW, we don't really need a `gzip` executable as long as we have the
>> `zlib-decompress-region` function, right?
> Not necessarily, because Info files can be read outside Emacs.

I don't think we should worry about that: those who want to read Info
files with something else than Emacs may need to have `gzip` installed,
and I think it's perfectly acceptable.

My question was instead about whether or not Emacs does indeed make use
of `zlib-decompress-region` when it tries to read a gzipped Info file,
and whether the Windows build does always provide
`zlib-decompress-region` or whether that function also requires an
external dependency that may or may not be available (in which case
it's no better than using the `gzip` executable in this respect).


        Stefan




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: `gzip` dependency
  2021-01-11 14:59               ` Stefan Monnier
@ 2021-01-11 15:15                 ` Phillip Lord
  2021-01-11 15:46                 ` Eli Zaretskii
  1 sibling, 0 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-11 15:15 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, emacs-devel

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> > The gzip thing is again your call (having gzip in the package is no
>>> > big deal, IMO).
>>> BTW, we don't really need a `gzip` executable as long as we have the
>>> `zlib-decompress-region` function, right?
>> Not necessarily, because Info files can be read outside Emacs.
>
> I don't think we should worry about that: those who want to read Info
> files with something else than Emacs may need to have `gzip` installed,
> and I think it's perfectly acceptable.
>
> My question was instead about whether or not Emacs does indeed make use
> of `zlib-decompress-region` when it tries to read a gzipped Info file,
> and whether the Windows build does always provide
> `zlib-decompress-region` or whether that function also requires an
> external dependency that may or may not be available (in which case
> it's no better than using the `gzip` executable in this respect).



zlib is not given as an explicit dependency of Emacs on Windows at the
moment, although perhaps it should be. However, it is included as a
transitive dependency.

If I gzip all the info files then do M-x info and look into a node, I
get an error with message "Unpression program `gzip' not found". If I
open a gzip node with find-file-literally then do eval
(zlib-decompress-region (point-min) (point-max)), I get the contents of
the info file.

Conclusions:

1) Emacs uses gzip to decompress but doesn't need to.

2) zlib should probably be added an explicit dependency regardless,
since it's a feature of Emacs that can be disabled.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-11  9:59           ` [feature/dll-only-windows] A new windows build, comments wanted Phillip Lord
@ 2021-01-11 15:21             ` Eli Zaretskii
  2021-01-11 18:29               ` Phillip Lord
  0 siblings, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-11 15:21 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: emacs-devel@gnu.org
> Date: Mon, 11 Jan 2021 09:59:29 +0000
> 
> > It's your call.  I just envision many users to expect native-comp
> > support and asking why isn't it there.
> 
> No, you misunderstand me. I would plan to make Emacs-28
> --with-nativecomp. But if I am doing this for the windows build, it is
> reasonable to assume that other packagers will be doing for their
> builds.

Not necessarily: on platforms other than MS-Windows, if Emacs was
linked against some optional library, that library must be installed
for Emacs to run without crashing.

So on MS-Windows you could build Emacs with all the possible
extensions, and let users decide which ones they want to install and
use.

> > The gzip thing is again your call (having gzip in the package is no
> > big deal, IMO).  Dbus is not really useful on MS-Windows, so IMO it
> > makes no sense building with it.
> 
> We've been down this road before.

Did we reach any conclusions?

> I could add gzip and I am happy to consider this. But what other
> tools do we add. What about aspell (or equivalent), so we can spell
> check? Git so we can version? If I add them, then we get more
> "out-of-the-box" behaviour. But where do we stop.

We stop where you decide to stop.  (And the utility of aspell is much
smaller than that of gzip, btw.  And the complexity of aspell
installation is OTOH significantly greater.)  There's no need for any
consistency here.  "Consistency is the hobgoblin", and all that.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: `gzip` dependency
  2021-01-11 10:00               ` `gzip` dependency Phillip Lord
@ 2021-01-11 15:22                 ` Eli Zaretskii
  0 siblings, 0 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-11 15:22 UTC (permalink / raw)
  To: Phillip Lord; +Cc: monnier, emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
> Date: Mon, 11 Jan 2021 10:00:32 +0000
> 
> >> BTW, we don't really need a `gzip` executable as long as we have the
> >> `zlib-decompress-region` function, right?
> >
> > Not necessarily, because Info files can be read outside Emacs.
> 
> This is not an issue for the Windows package of Emacs, as far as I can tell.

Emacs installs Info files that can be used outside of Emacs.  So I
think it could be an issue for Emacs.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: `gzip` dependency
  2021-01-11 14:59               ` Stefan Monnier
  2021-01-11 15:15                 ` Phillip Lord
@ 2021-01-11 15:46                 ` Eli Zaretskii
  1 sibling, 0 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-11 15:46 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel, phillip.lord

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: phillip.lord@russet.org.uk,  emacs-devel@gnu.org
> Date: Mon, 11 Jan 2021 09:59:19 -0500
> 
> My question was instead about whether or not Emacs does indeed make use
> of `zlib-decompress-region` when it tries to read a gzipped Info file,

No, I don't think we do.

> and whether the Windows build does always provide
> `zlib-decompress-region`

Only if Emacs was built with zlib.

> or whether that function also requires an external dependency that
> may or may not be available

One needs both an Emacs compiled with zlib, and the zlib DLL
available.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-11 11:01           ` Andrea Corallo via Emacs development discussions.
@ 2021-01-11 16:29             ` Phillip Lord
  2021-01-11 17:21               ` Andrea Corallo via Emacs development discussions.
  0 siblings, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-11 16:29 UTC (permalink / raw)
  To: Andrea Corallo; +Cc: Eli Zaretskii, emacs-devel

Andrea Corallo <akrl@sdf.org> writes:

>> The first is because it's broken or providing buggy behaviour. In the
>> extreme, and if I understand the build process fully, I guess this is
>> impossible to switch of in a binary distribution, because some
>> native-comp happens before dumping. Other than that, we can "switch off"
>> native-comp by just deleting the *eln files, right, or otherwise
>> preventing their loading. In practice, I think this is a minor
>> motivation; if we discover bugs in the native-comp, they would be fixed
>> by making another release.
>
> Hi Phillip,
>
> ATM we can prevent Emacs loading .eln file in place of .elc
> automatically setting `load-no-native' to t.
>
>> The second reason is that the initial compilation takes quite a lot of
>> CPU. I wouldn't like that to happen while my laptop where in battery
>> power, for instance. At the moment, it's possible to drop the number of
>> jobs that native-comp uses. I don't think that there is a way to drop
>> this to zero at the moment. Probably, we need that.
>
> Setting `comp-deferred-compilation' to nil should stop any automatic
> attempt of compiling asycronousy.
>
> Maybe these two settings are already covering most of these needs?


Yes, I think so, but it's probably an issue of thinking about the user
interface. Having this a bit more consistent (i.e. a single name space,
rather than two (native-comp and comp) and none (i.e. load-no-native))
and perhaps a single option that does both ("comp-disable").

Just thoughts.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-11 16:29             ` Phillip Lord
@ 2021-01-11 17:21               ` Andrea Corallo via Emacs development discussions.
  0 siblings, 0 replies; 78+ messages in thread
From: Andrea Corallo via Emacs development discussions. @ 2021-01-11 17:21 UTC (permalink / raw)
  To: Phillip Lord; +Cc: Eli Zaretskii, emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> Andrea Corallo <akrl@sdf.org> writes:
>
>>> The first is because it's broken or providing buggy behaviour. In the
>>> extreme, and if I understand the build process fully, I guess this is
>>> impossible to switch of in a binary distribution, because some
>>> native-comp happens before dumping. Other than that, we can "switch off"
>>> native-comp by just deleting the *eln files, right, or otherwise
>>> preventing their loading. In practice, I think this is a minor
>>> motivation; if we discover bugs in the native-comp, they would be fixed
>>> by making another release.
>>
>> Hi Phillip,
>>
>> ATM we can prevent Emacs loading .eln file in place of .elc
>> automatically setting `load-no-native' to t.
>>
>>> The second reason is that the initial compilation takes quite a lot of
>>> CPU. I wouldn't like that to happen while my laptop where in battery
>>> power, for instance. At the moment, it's possible to drop the number of
>>> jobs that native-comp uses. I don't think that there is a way to drop
>>> this to zero at the moment. Probably, we need that.
>>
>> Setting `comp-deferred-compilation' to nil should stop any automatic
>> attempt of compiling asycronousy.
>>
>> Maybe these two settings are already covering most of these needs?
>
>
> Yes, I think so, but it's probably an issue of thinking about the user
> interface. Having this a bit more consistent (i.e. a single name space,
> rather than two (native-comp and comp) and none (i.e. load-no-native))
> and perhaps a single option that does both ("comp-disable").
>
> Just thoughts.

Welcome thoughts from my side.

I did my best to fit current name conventions but indeed this might be
not the best arrangement, especially looking at it as single UI
interface towards the "native compiler" seen as single entity.

Looking forward to discuss and tackle this in the code review.

  Andrea



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-11 15:21             ` Eli Zaretskii
@ 2021-01-11 18:29               ` Phillip Lord
  0 siblings, 0 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-11 18:29 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Cc: emacs-devel@gnu.org
>> Date: Mon, 11 Jan 2021 09:59:29 +0000
>> 
>> > It's your call.  I just envision many users to expect native-comp
>> > support and asking why isn't it there.
>> 
>> No, you misunderstand me. I would plan to make Emacs-28
>> --with-nativecomp. But if I am doing this for the windows build, it is
>> reasonable to assume that other packagers will be doing for their
>> builds.
>
> Not necessarily: on platforms other than MS-Windows, if Emacs was
> linked against some optional library, that library must be installed
> for Emacs to run without crashing.
>
> So on MS-Windows you could build Emacs with all the possible
> extensions, and let users decide which ones they want to install and
> use.

The installer version could support that kind of thing, but I don't want
to provide too many options for the sake of confusion. The sort of
people who do not want features can probably build Emacs for themselves.


>
>> > The gzip thing is again your call (having gzip in the package is no
>> > big deal, IMO).  Dbus is not really useful on MS-Windows, so IMO it
>> > makes no sense building with it.
>> 
>> We've been down this road before.
>
> Did we reach any conclusions?

Yes, we concluded that it was hard to know where to stop!


>
>> I could add gzip and I am happy to consider this. But what other
>> tools do we add. What about aspell (or equivalent), so we can spell
>> check? Git so we can version? If I add them, then we get more
>> "out-of-the-box" behaviour. But where do we stop.
>
> We stop where you decide to stop.  (And the utility of aspell is much
> smaller than that of gzip, btw.  And the complexity of aspell
> installation is OTOH significantly greater.)  There's no need for any
> consistency here.  "Consistency is the hobgoblin", and all that.

I think that this is wrong. Gzip is minimal use on Windows, except for
info files, and the total install file size is 18Mb vs 5Mb.

Compare to aspell (or hunspell) for the people installing Emacs to use
Org mode? Or git for those who want to use magit. I think it would be
even better (i.e. potentially quicker!) if we could access these as
libraries.

Still, if the number of total downloads has decreased, adding a set of
command line utilities as extra options in the installer might be worth
considering. I will cogitate.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-10 15:43 ` Phillip Lord
@ 2021-01-12  6:01   ` Corwin Brust
  2021-01-12  9:48     ` Phillip Lord
  0 siblings, 1 reply; 78+ messages in thread
From: Corwin Brust @ 2021-01-12  6:01 UTC (permalink / raw)
  To: Phillip Lord; +Cc: Emacs developers

Thank you much for working on this!

On Sun, Jan 10, 2021 at 9:43 AM Phillip Lord <phillip.lord@russet.org.uk> wrote:
>
> I have pushed some binaries up to alpha in case anyone wants to give
> them a trial.
>

I found a problem and thought I'd been check in case it is not
situated safely between my keyboard and chair   Also, the emacsclient
issue I mentioned on r/emacs was a self-inflicted wound; if I run it
with a property shortcut incantation (including -Q) it works great!

When I start emacs using the same configuration I use daily with Emacs
27.1 and Emacs 26.3, I get a backtrace.  Staring at this in wonder, it
looks like it could be related to use-package, which features heavily
in my setup.

tia and please do let me know what additional help I might be.

Here's the backtrace:

Debugger entered--Lisp error: (invalid-read-syntax "Invalid byte-code object")
  read(get-file-char)
  require(use-package-core)
  byte-code("\300\301!\210\300\302!\210\300\303!\210\300\304!\210\300\305!\210\306\307\310\311\312$\210\313\314!\207"
[require use-package-core use-package-bind-key use-package-diminish
use-package-delight use-package-ensure autoload
use-package-jump-to-package-form "use-package-jump" nil t provide
use-package] 5)
  require(use-package)
  (progn (require 'cl) (require 'use-package))
  eval((progn (require 'cl) (require 'use-package)) t)
  #f(compiled-function (&rest body) "Like `progn', but evaluates the
body at compile time if you're compiling.\nThus, the result of the
body appears to the compiler as a quoted\nconstant.  In interpreted
code, this is entirely equivalent to\n`progn', except that the value
of the expression may be (but is\nnot necessarily) computed at load
time if eager macro expansion\nis enabled." #<bytecode
-0x173efc045404a52b>)((require 'cl) (require 'use-package))
  (eval-when-compile (require 'cl) (require 'use-package))
  eval-buffer(#<buffer  *load*-573669> nil
"d:/projects/dotfiles/elisp/init-package-managment...." nil t)  ;
Reading at buffer position 924
  load-with-code-conversion("d:/projects/dotfiles/elisp/init-package-managment...."
"d:/projects/dotfiles/elisp/init-package-managment...." nil t)
  require(init-package-managment)
  (let ((pkg (car --dolist-tail--))) (require pkg) (setq
--dolist-tail-- (cdr --dolist-tail--)))
  (while --dolist-tail-- (let ((pkg (car --dolist-tail--))) (require
pkg) (setq --dolist-tail-- (cdr --dolist-tail--))))
  (let ((--dolist-tail-- '(init-local init-package-managment
init-editor init-org init-elisp init-projectile init-vc init-company
init-fun init-bindkey init-theme))) (while --dolist-tail-- (let ((pkg
(car --dolist-tail--))) (require pkg) (setq --dolist-tail-- (cdr
--dolist-tail--)))))
  eval-buffer(#<buffer  *load*-904898> nil
"d:/projects/dotfiles/elisp/init.el" nil t)  ; Reading at buffer
position 3546
  load-with-code-conversion("d:/projects/dotfiles/elisp/init.el"
"d:/projects/dotfiles/elisp/init.el" nil nil)
  load("d:/projects/dotfiles/elisp/init")
  eval-buffer(#<buffer  *load*> nil "c:/Users/corwi/.emacs" nil t)  ;
Reading at buffer position 414
  load-with-code-conversion("c:/Users/corwi/.emacs" "c:/Users/corwi/.emacs" t t)
  load("~/.emacs" noerror nomessage)
  startup--load-user-init-file(#f(compiled-function () #<bytecode
0x5a5c7670ff81e72>) #f(compiled-function () #<bytecode
0xc085514c3215d0c>) t)
  command-line()
  normal-top-level()


Regards,
Corwin
corwin.bru.st



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-12  6:01   ` Corwin Brust
@ 2021-01-12  9:48     ` Phillip Lord
  2021-01-12 10:27       ` Corwin Brust
  0 siblings, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-12  9:48 UTC (permalink / raw)
  To: Corwin Brust; +Cc: Emacs developers



Can you do a compare and contrast with this snapshot version?

https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-28/emacs-28.0.50-snapshot-2021-01-04-x86_64.zip

This was built at about the same time as the dll-only version. I doubt
that this problem is caused by the new package, but this will tell us
for sure.

Phil

Corwin Brust <corwin@bru.st> writes:

> Thank you much for working on this!
>
> On Sun, Jan 10, 2021 at 9:43 AM Phillip Lord
> <phillip.lord@russet.org.uk> wrote:
>>
>> I have pushed some binaries up to alpha in case anyone wants to give
>> them a trial.
>>
>
> I found a problem and thought I'd been check in case it is not
> situated safely between my keyboard and chair   Also, the emacsclient
> issue I mentioned on r/emacs was a self-inflicted wound; if I run it
> with a property shortcut incantation (including -Q) it works great!
>
> When I start emacs using the same configuration I use daily with Emacs
> 27.1 and Emacs 26.3, I get a backtrace.  Staring at this in wonder, it
> looks like it could be related to use-package, which features heavily
> in my setup.
>
> tia and please do let me know what additional help I might be.
>
> Here's the backtrace:
>
> Debugger entered--Lisp error: (invalid-read-syntax "Invalid byte-code
> object")
>   read(get-file-char)
>   require(use-package-core)
>   byte-code("\300\301!\210\300\302!\210\300\303!\210\300\304!\210\300\305!\210\306\307\310\311\312$\210\313\314!\207"
> [require use-package-core use-package-bind-key use-package-diminish
> use-package-delight use-package-ensure autoload
> use-package-jump-to-package-form "use-package-jump" nil t provide
> use-package] 5)
>   require(use-package)
>   (progn (require 'cl) (require 'use-package))
>   eval((progn (require 'cl) (require 'use-package)) t)
>   #f(compiled-function (&rest body) "Like `progn', but evaluates the
> body at compile time if you're compiling.\nThus, the result of the
> body appears to the compiler as a quoted\nconstant.  In interpreted
> code, this is entirely equivalent to\n`progn', except that the value
> of the expression may be (but is\nnot necessarily) computed at load
> time if eager macro expansion\nis enabled." #<bytecode
> -0x173efc045404a52b>)((require 'cl) (require 'use-package))
>   (eval-when-compile (require 'cl) (require 'use-package))
>   eval-buffer(#<buffer  *load*-573669> nil
> "d:/projects/dotfiles/elisp/init-package-managment...." nil t)  ;
> Reading at buffer position 924
>   load-with-code-conversion("d:/projects/dotfiles/elisp/init-package-managment...."
> "d:/projects/dotfiles/elisp/init-package-managment...." nil t)
>   require(init-package-managment)
>   (let ((pkg (car --dolist-tail--))) (require pkg) (setq
> --dolist-tail-- (cdr --dolist-tail--)))
>   (while --dolist-tail-- (let ((pkg (car --dolist-tail--))) (require
> pkg) (setq --dolist-tail-- (cdr --dolist-tail--))))
>   (let ((--dolist-tail-- '(init-local init-package-managment
> init-editor init-org init-elisp init-projectile init-vc init-company
> init-fun init-bindkey init-theme))) (while --dolist-tail-- (let ((pkg
> (car --dolist-tail--))) (require pkg) (setq --dolist-tail-- (cdr
> --dolist-tail--)))))
>   eval-buffer(#<buffer  *load*-904898> nil
> "d:/projects/dotfiles/elisp/init.el" nil t)  ; Reading at buffer
> position 3546
>   load-with-code-conversion("d:/projects/dotfiles/elisp/init.el"
> "d:/projects/dotfiles/elisp/init.el" nil nil)
>   load("d:/projects/dotfiles/elisp/init")
>   eval-buffer(#<buffer  *load*> nil "c:/Users/corwi/.emacs" nil t)  ;
> Reading at buffer position 414
>   load-with-code-conversion("c:/Users/corwi/.emacs"
> "c:/Users/corwi/.emacs" t t)
>   load("~/.emacs" noerror nomessage)
>   startup--load-user-init-file(#f(compiled-function () #<bytecode
> 0x5a5c7670ff81e72>) #f(compiled-function () #<bytecode
> 0xc085514c3215d0c>) t)
>   command-line()
>   normal-top-level()
>
>
> Regards,
> Corwin
> corwin.bru.st



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/dll-only-windows] A new windows build, comments wanted
  2021-01-12  9:48     ` Phillip Lord
@ 2021-01-12 10:27       ` Corwin Brust
  0 siblings, 0 replies; 78+ messages in thread
From: Corwin Brust @ 2021-01-12 10:27 UTC (permalink / raw)
  To: Phillip Lord; +Cc: Emacs developers

You are right!

On Tue, Jan 12, 2021 at 3:48 AM Phillip Lord <phillip.lord@russet.org.uk> wrote:
> Can you do a compare and contrast with this snapshot version?
>
> https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-28/emacs-28.0.50-snapshot-2021-01-04-x86_64.zip
>
> This was built at about the same time as the dll-only version. I doubt
> that this problem is caused by the new package, but this will tell us
> for sure.

Same backtrace, at a glance.  Open a new bug for this, then?

Debugger entered--Lisp error: (invalid-read-syntax "Invalid byte-code object")
  read(get-file-char)
  require(use-package-core)
  byte-code("\300\301!\210\300\302!\210\300\303!\210\300\304!\210\300\305!\210\306\307\310\311\312$\210\313\314!\207"
[require use-package-core use-package-bind-key use-package-diminish
use-package-delight use-package-ensure autoload
use-package-jump-to-package-form "use-package-jump" nil t provide
use-package] 5)
  require(use-package)
  (progn (require 'cl) (require 'use-package))
  eval((progn (require 'cl) (require 'use-package)) t)
  #f(compiled-function (&rest body) "Like `progn', but evaluates the
body at compile time if you're compiling.\nThus, the result of the
body appears to the compiler as a quoted\nconstant.  In interpreted
code, this is entirely equivalent to\n`progn', except that the value
of the expression may be (but is\nnot necessarily) computed at load
time if eager macro expansion\nis enabled." #<bytecode
-0x173efc045404914c>)((require 'cl) (require 'use-package))
  (eval-when-compile (require 'cl) (require 'use-package))
  eval-buffer(#<buffer  *load*-275982> nil
"d:/projects/dotfiles/elisp/init-package-managment...." nil t)  ;
Reading at buffer position 924
  load-with-code-conversion("d:/projects/dotfiles/elisp/init-package-managment...."
"d:/projects/dotfiles/elisp/init-package-managment...." nil t)
  require(init-package-managment)
  (let ((pkg (car --dolist-tail--))) (require pkg) (setq
--dolist-tail-- (cdr --dolist-tail--)))
  (while --dolist-tail-- (let ((pkg (car --dolist-tail--))) (require
pkg) (setq --dolist-tail-- (cdr --dolist-tail--))))
  (let ((--dolist-tail-- '(init-local init-package-managment
init-editor init-org init-elisp init-projectile init-vc init-company
init-fun init-bindkey init-theme))) (while --dolist-tail-- (let ((pkg
(car --dolist-tail--))) (require pkg) (setq --dolist-tail-- (cdr
--dolist-tail--)))))
  eval-buffer(#<buffer  *load*-477369> nil
"d:/projects/dotfiles/elisp/init.el" nil t)  ; Reading at buffer
position 3546
  load-with-code-conversion("d:/projects/dotfiles/elisp/init.el"
"d:/projects/dotfiles/elisp/init.el" nil nil)
  load("d:/projects/dotfiles/elisp/init")
  eval-buffer(#<buffer  *load*> nil "c:/Users/corwi/.emacs" nil t)  ;
Reading at buffer position 414
  load-with-code-conversion("c:/Users/corwi/.emacs" "c:/Users/corwi/.emacs" t t)
  load("~/.emacs" noerror nomessage)
  startup--load-user-init-file(#f(compiled-function () #<bytecode
0x5a5c782f8c782d2>) #f(compiled-function () #<bytecode
0xc085514c3215d0c>) t)
  command-line()
  normal-top-level()


> > Regards,
> > Corwin
> > corwin.bru.st



^ permalink raw reply	[flat|nested] 78+ messages in thread

* [feature/internal-msys] thoughts of a more function windows package
  2021-01-10 20:52           ` `gzip` dependency (was: [feature/dll-only-windows] A new windows build, comments wanted) Stefan Monnier
  2021-01-11  3:27             ` Eli Zaretskii
@ 2021-01-20 19:29             ` Phillip Lord
  2021-01-21 12:36               ` Stephen Leake
  2021-01-21 14:11               ` Eli Zaretskii
  1 sibling, 2 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-20 19:29 UTC (permalink / raw)
  To: emacs-devel

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> The gzip thing is again your call (having gzip in the package is no
>> big deal, IMO).
>
> BTW, we don't really need a `gzip` executable as long as we have the
> `zlib-decompress-region` function, right?




Following on from this discussion, I have had a thought about a better
way of producing a more function windows download and have come up with
this idea, which is just to install msys inside the Emacs install
directory.

The plan would be during the installation run the newly installed
Emacs. This would download, unpack and configure msys2, as well as
install a variety of related packages (git, find, gzip, hunspell,
aspell, what ever). For those using the zip package of Emacs, they would
achieve the same thing by running a command.

I have given this a quick bash. It could be achieved without bloating
the Emacs download; I'd have to had xz to the Emacs installer to allow
me to unpack msys, but otherwise it would have no implications. I would
not need to host either msys2 nor its source on the Gnu FTP because,
this would all come from msys2. The installation process would be Emacs
driven, which would be easier for me to write. Emacs would remain using
the DLLs we ship with it and so stable, while the underlying msys2 would
be independently updateable. And, to all intents and purposes, it would
look like a normal windows app; packaged, bundled and installed in
directory.

What I have so far:

1) w32-msys-install

which updates msys and installs git as well as adds a loader to site-start.el

2) w32-msys-site-start.el

which updates the path and so

To be fully functional 1) would need to download msys and unpack it and
I'd need to hook it into the installer.

It would solve the question of what should I package -- I'd add some
basic utilities, and thereafter, the user could add what ever they
wanted.


Thoughts? Does this seem worth following up?

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-20 19:29             ` [feature/internal-msys] thoughts of a more function windows package Phillip Lord
@ 2021-01-21 12:36               ` Stephen Leake
  2021-01-21 16:11                 ` Phillip Lord
  2021-01-21 18:22                 ` Stephen Leake
  2021-01-21 14:11               ` Eli Zaretskii
  1 sibling, 2 replies; 78+ messages in thread
From: Stephen Leake @ 2021-01-21 12:36 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> Following on from this discussion, I have had a thought about a better
> way of producing a more function windows download and have come up with
> this idea, which is just to install msys inside the Emacs install
> directory.

Works for me, as long as it detects my current msys install and does not
modify it without asking my permission.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-20 19:29             ` [feature/internal-msys] thoughts of a more function windows package Phillip Lord
  2021-01-21 12:36               ` Stephen Leake
@ 2021-01-21 14:11               ` Eli Zaretskii
  2021-01-21 16:44                 ` Phillip Lord
  1 sibling, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-21 14:11 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Date: Wed, 20 Jan 2021 19:29:58 +0000
> 
> The plan would be during the installation run the newly installed
> Emacs. This would download, unpack and configure msys2, as well as
> install a variety of related packages (git, find, gzip, hunspell,
> aspell, what ever). For those using the zip package of Emacs, they would
> achieve the same thing by running a command.

Could you please elaborate on how will this work for users?  E.g., how
about describing the steps a user who installs Emacs should do to
install these additions?  I don't think I have a clear picture of what
you intend to do, and the files you committed are just the starting
point, so they don't answer this question.

Btw, I don't think I understand why you insist on using 'concat' to
generate files names from 2 components: that's what expand-file-name
is for, and it does that better.

One other not about the code you committed is that modifying PATH from
within Emacs is not generally a good idea, it has some subtleties.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-21 12:36               ` Stephen Leake
@ 2021-01-21 16:11                 ` Phillip Lord
  2021-01-21 18:22                 ` Stephen Leake
  1 sibling, 0 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-21 16:11 UTC (permalink / raw)
  To: Stephen Leake; +Cc: emacs-devel

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> Following on from this discussion, I have had a thought about a better
>> way of producing a more function windows download and have come up with
>> this idea, which is just to install msys inside the Emacs install
>> directory.
>
> Works for me, as long as it detects my current msys install and does not
> modify it without asking my permission.

I have tested this; you can have two msys installations on a machine and
they just ignore each other. In practice, someone who already has an
msys installation wouldn't want this, although there is no reason why
any technology that we put in place for managing an internal msys
shouldn't also work for a preinstalled one.

For example, there might be a command "install recommended packages"
which would just update your local msys an add those packages that work
with Emacs. Likewise, if there is an expectation that Emacs will be used
in this way, we might modify Emacs to work "properly"; for example, M-x
find tends to break on Windows because it finds the wrong "find"
command.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-21 14:11               ` Eli Zaretskii
@ 2021-01-21 16:44                 ` Phillip Lord
  2021-01-21 20:17                   ` Eli Zaretskii
  0 siblings, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-21 16:44 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Date: Wed, 20 Jan 2021 19:29:58 +0000
>> 
>> The plan would be during the installation run the newly installed
>> Emacs. This would download, unpack and configure msys2, as well as
>> install a variety of related packages (git, find, gzip, hunspell,
>> aspell, what ever). For those using the zip package of Emacs, they would
>> achieve the same thing by running a command.
>
> Could you please elaborate on how will this work for users?  E.g., how
> about describing the steps a user who installs Emacs should do to
> install these additions?  I don't think I have a clear picture of what
> you intend to do, and the files you committed are just the starting
> point, so they don't answer this question.

Considering just the .exe installer, at the end of the installation
process, we would run Emacs with menu bar, scroll bar and everything
switch off and run the command "w32-msys-internal-install-dialog".

Emacs would say "do you want do install a handy collection of
tools". User says "y", then Emacs downloads the latest msys2-date.xz,
then unpacks it with xz. Emacs will run msys2.exe to initialize
(hopefully without poping up windows if this will work). Then Emacs will
run pacman to update and install. The user will see Emacs saying

Would you like to install msys, and a handy set of tools (Y/n)
Installing msys...done
Updating msys...done
Installing git...done

When it's finished, the user starts Emacs and does vc-dir and it works,
instead of seeing "git not found" errors. For those using the zip file,
the user would have to launch "M-x w32-msys-install" manually.

At a later date, I might add some cuter things. So, for example we might
install:

https://github.com/brotzeit/arch-packer

or equivalent. We might create "internal-msys" package on ELPA which
contains a standard set of packages and Emacs fixes to make it work, so
it would be freely updatable. At which point the w32-msys-install
command would install the ELPA package before doing anything else.

There are also issues that when Emacs is removed msys (and any
configuration) would get blitzed, so I'd have to think about how to
persist msys configuration between two installs of Emacs. Currently the
file organisation looks like this:

c:/Program Files/Emacs/emacs-28.1/bin
c:/Program Files/Emacs/emacs-28.1/share
c:/Program Files/Emacs/emacs-28.1/msys

(etc)

but 

c:/Program Files/Emacs/emacs-28.1/bin
c:/Program Files/Emacs/emacs-28.1/share
c:/Program Files/Emacs/msys

would mean that installing Emacs-28.2 would work naturally.

There are also some issues about user persmissions; I only ever run
Windows as administrator, so there will be some thought given as to how
this would work. Neither Emacs or msys require this AFAICT.



> Btw, I don't think I understand why you insist on using 'concat' to
> generate files names from 2 components: that's what expand-file-name
> is for, and it does that better.

Because in my own code, I tend to use f.el and I have got used to having
a clean, consistent and comprehensive API, so I tend to forget the
underlying Emacs primitives.


> One other not about the code you committed is that modifying PATH from
> within Emacs is not generally a good idea, it has some subtleties.

Any alternative that achieves the same thing?

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-21 12:36               ` Stephen Leake
  2021-01-21 16:11                 ` Phillip Lord
@ 2021-01-21 18:22                 ` Stephen Leake
  2021-01-21 18:44                   ` phillip.lord
  2021-01-21 18:53                   ` Óscar Fuentes
  1 sibling, 2 replies; 78+ messages in thread
From: Stephen Leake @ 2021-01-21 18:22 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> Following on from this discussion, I have had a thought about a better
>> way of producing a more function windows download and have come up with
>> this idea, which is just to install msys inside the Emacs install
>> directory.
>
> Works for me, as long as it detects my current msys install and does not
> modify it without asking my permission.

Actually, a better way to proceed in this direction is to update the
current emacs mingw64 package; it is currently at 26.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-21 18:22                 ` Stephen Leake
@ 2021-01-21 18:44                   ` phillip.lord
  2021-01-23  2:51                     ` Stephen Leake
  2021-01-21 18:53                   ` Óscar Fuentes
  1 sibling, 1 reply; 78+ messages in thread
From: phillip.lord @ 2021-01-21 18:44 UTC (permalink / raw)
  To: Stephen Leake; +Cc: emacs-devel

On 2021-01-21 18:22, Stephen Leake wrote:
> Stephen Leake <stephen_leake@stephe-leake.org> writes:
> 
>> Phillip Lord <phillip.lord@russet.org.uk> writes:
>> 
>>> Following on from this discussion, I have had a thought about a 
>>> better
>>> way of producing a more function windows download and have come up 
>>> with
>>> this idea, which is just to install msys inside the Emacs install
>>> directory.
>> 
>> Works for me, as long as it detects my current msys install and does 
>> not
>> modify it without asking my permission.
> 
> Actually, a better way to proceed in this direction is to update the
> current emacs mingw64 package; it is currently at 26.

Is it?

https://packages.msys2.org/package/mingw-w64-x86_64-emacs?repo=mingw64

report 27.1

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-21 18:22                 ` Stephen Leake
  2021-01-21 18:44                   ` phillip.lord
@ 2021-01-21 18:53                   ` Óscar Fuentes
  1 sibling, 0 replies; 78+ messages in thread
From: Óscar Fuentes @ 2021-01-21 18:53 UTC (permalink / raw)
  To: emacs-devel

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>
>> Phillip Lord <phillip.lord@russet.org.uk> writes:
>>
>>> Following on from this discussion, I have had a thought about a better
>>> way of producing a more function windows download and have come up with
>>> this idea, which is just to install msys inside the Emacs install
>>> directory.
>>
>> Works for me, as long as it detects my current msys install and does not
>> modify it without asking my permission.
>
> Actually, a better way to proceed in this direction is to update the
> current emacs mingw64 package; it is currently at 26.

The MSYS2/MinGW64 Emacs package? It was updated to 27.1 one day after
the release.




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-21 16:44                 ` Phillip Lord
@ 2021-01-21 20:17                   ` Eli Zaretskii
  2021-01-21 21:37                     ` Phillip Lord
  0 siblings, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-21 20:17 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: emacs-devel@gnu.org
> Date: Thu, 21 Jan 2021 16:44:02 +0000
> 
> > Could you please elaborate on how will this work for users?  E.g., how
> > about describing the steps a user who installs Emacs should do to
> > install these additions?  I don't think I have a clear picture of what
> > you intend to do, and the files you committed are just the starting
> > point, so they don't answer this question.
> 
> Considering just the .exe installer, at the end of the installation
> process, we would run Emacs with menu bar, scroll bar and everything
> switch off and run the command "w32-msys-internal-install-dialog".

"We" being the installation process?  IOW, this will start
automatically at the end of the normal installation?

> Emacs would say "do you want do install a handy collection of
> tools". User says "y", then Emacs downloads the latest msys2-date.xz,
> then unpacks it with xz. Emacs will run msys2.exe to initialize
> (hopefully without poping up windows if this will work). Then Emacs will
> run pacman to update and install. The user will see Emacs saying
> 
> Would you like to install msys, and a handy set of tools (Y/n)
> Installing msys...done
> Updating msys...done
> Installing git...done

First, why Git?  This is a "normal" user, not an Emacs developer,
right?  Git is a huge package (and comes with Vim on top of that), so
maybe reconsider that part.

Next, by "msys" you mean all those non-native programs that depend on
msys-1.0.dll?  That's again meant for MinGW developers, not "normal"
users.

And finally, which packages will pacman install? are you going to
provide some list of packages, and if so, what will be there?

(Btw, pacman can ask questions and prompt the user for confirmation,
but the way you invoke it in w32-msys-run doesn't seem to be prepared
for such interaction?)

One other aspect that bothers me is that if the user already has some
parts of MSYS/MinGW64 installed, they will now have two places with
DLLs, which is a step towards "DLL hell".

> We might create "internal-msys" package on ELPA which
> contains a standard set of packages and Emacs fixes to make it work, so
> it would be freely updatable. At which point the w32-msys-install
> command would install the ELPA package before doing anything else.

Is it really possible for us to distribute MSYS, given its license?

> > Btw, I don't think I understand why you insist on using 'concat' to
> > generate files names from 2 components: that's what expand-file-name
> > is for, and it does that better.
> 
> Because in my own code, I tend to use f.el and I have got used to having
> a clean, consistent and comprehensive API, so I tend to forget the
> underlying Emacs primitives.

Then I think f.el has a big problem.

> > One other not about the code you committed is that modifying PATH from
> > within Emacs is not generally a good idea, it has some subtleties.
> 
> Any alternative that achieves the same thing?

I think the only good idea here is to tell the user to amend PATH by
adding such-and-such directories to it.  I don't like installers that
futz with my PATH, and would hate it if Emacs did that to others.
It's very easy to get that wrong, especially on Windows.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-21 20:17                   ` Eli Zaretskii
@ 2021-01-21 21:37                     ` Phillip Lord
  2021-01-22  7:24                       ` Eli Zaretskii
  0 siblings, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-21 21:37 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> Considering just the .exe installer, at the end of the installation
>> process, we would run Emacs with menu bar, scroll bar and everything
>> switch off and run the command "w32-msys-internal-install-dialog".
>
> "We" being the installation process?  IOW, this will start
> automatically at the end of the normal installation?


Yep.


>> Emacs would say "do you want do install a handy collection of
>> tools". User says "y", then Emacs downloads the latest msys2-date.xz,
>> then unpacks it with xz. Emacs will run msys2.exe to initialize
>> (hopefully without poping up windows if this will work). Then Emacs will
>> run pacman to update and install. The user will see Emacs saying
>> 
>> Would you like to install msys, and a handy set of tools (Y/n)
>> Installing msys...done
>> Updating msys...done
>> Installing git...done
>
> First, why Git?  This is a "normal" user, not an Emacs developer,
> right?  Git is a huge package (and comes with Vim on top of that), so
> maybe reconsider that part.

That's open for question. I'd have to think about the packages I would
want to use. However, for me, git is one of the first commands that
Emacs complains about the absence of; this is because I use Emacs on my
windows build machine, of course, which contains the Emacs git
repo. But, also because most IDEs come with the ability to interact with
git off the shelf. I think Windows users will expect it.

Yep, it's big. I get this before installation:

319 MB (335,214,637 bytes)

and this after installation.

826 MB (866,624,644 bytes)

My guess is that it's the co-install of perl that is causing this issue
rather than vim. But, openssh comes for free with this. And "huge" is a
relative term. An installation under 1Gb seems reasonable to me.

> Next, by "msys" you mean all those non-native programs that depend on
> msys-1.0.dll?  That's again meant for MinGW developers, not "normal"
> users.

Yes. Because it's got all the packages and tools and as far as I can
see, they work with Emacs.


> And finally, which packages will pacman install? are you going to
> provide some list of packages, and if so, what will be there?

Yes. Initially hard coded into the Emacs install, we might turn the list
of msys packages into an ELPA package. That way, we could update the
list of tools supported, without updating the Emacs distribution.


> (Btw, pacman can ask questions and prompt the user for confirmation,
> but the way you invoke it in w32-msys-run doesn't seem to be prepared
> for such interaction?)

That's why I use "--no-confirm". I'm looking for the minimum here that
does something usable.


> One other aspect that bothers me is that if the user already has some
> parts of MSYS/MinGW64 installed, they will now have two places with
> DLLs, which is a step towards "DLL hell".

Yes, it is. And indeed, the DLLs distributed in the Emacs distribution
might well get duplicated in the msys install. But they would be in one
place. My presumption is that people who use msys already would not want
this, and it would not be forced upon them.


>> We might create "internal-msys" package on ELPA which
>> contains a standard set of packages and Emacs fixes to make it work, so
>> it would be freely updatable. At which point the w32-msys-install
>> command would install the ELPA package before doing anything else.
>
> Is it really possible for us to distribute MSYS, given its license?

You misunderstand, my poor wording. The ELPA package would not contain
msys, but would contain list of msys package names and supporting
code. Msys would come from the msys2 repo.


>> > Btw, I don't think I understand why you insist on using 'concat' to
>> > generate files names from 2 components: that's what expand-file-name
>> > is for, and it does that better.
>> 
>> Because in my own code, I tend to use f.el and I have got used to having
>> a clean, consistent and comprehensive API, so I tend to forget the
>> underlying Emacs primitives.
>
> Then I think f.el has a big problem.

I was being slightly provocative; I shall divert the discussion no
further.



>> > One other not about the code you committed is that modifying PATH from
>> > within Emacs is not generally a good idea, it has some subtleties.
>> 
>> Any alternative that achieves the same thing?
>
> I think the only good idea here is to tell the user to amend PATH by
> adding such-and-such directories to it.  I don't like installers that
> futz with my PATH, and would hate it if Emacs did that to others.
> It's very easy to get that wrong, especially on Windows.


There has to be a better way that this. Editing environment variables by
hand is so last century. Of course, I do not want to edit the path
globally. If I can do it in Emacs, then I would have to think of a
different solution.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-21 21:37                     ` Phillip Lord
@ 2021-01-22  7:24                       ` Eli Zaretskii
  2021-01-22 16:14                         ` Phillip Lord
  0 siblings, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-22  7:24 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: emacs-devel@gnu.org
> Date: Thu, 21 Jan 2021 21:37:57 +0000
> 
> Yep, it's big. I get this before installation:
> 
> 319 MB (335,214,637 bytes)
> 
> and this after installation.
> 
> 826 MB (866,624,644 bytes)
> 
> My guess is that it's the co-install of perl that is causing this issue
> rather than vim.

It's Vim, and Perl, and Tcl.  And at some point I'd expect them to
include Python as well, to support those Git commands which rely on
it.

> But, openssh comes for free with this. And "huge" is a
> relative term. An installation under 1Gb seems reasonable to me.

I thought this started as an attempt to make the Emacs installation
smaller...

If the rationale is to provide a full development environment on top
of MS-Windows, then indeed the size will be much larger, and you will
need to include a lot of packages there.  Just be sure to spell this
out (including the size, perhaps) when you ask the user whether he or
she wants to install that.

> > Next, by "msys" you mean all those non-native programs that depend on
> > msys-1.0.dll?  That's again meant for MinGW developers, not "normal"
> > users.
> 
> Yes. Because it's got all the packages and tools and as far as I can
> see, they work with Emacs.

They mostly work, until they don't.  Like with Cygwin, there are
subtle incompatibilities, mainly in file names and in communications
with subprocesses and response to "signals".  Encoding defaults are
also different.

> > (Btw, pacman can ask questions and prompt the user for confirmation,
> > but the way you invoke it in w32-msys-run doesn't seem to be prepared
> > for such interaction?)
> 
> That's why I use "--no-confirm". I'm looking for the minimum here that
> does something usable.

What happens when pacman wants to replace Bash from which it was
launched, or update itself or some of its component DLLs?

> > I think the only good idea here is to tell the user to amend PATH by
> > adding such-and-such directories to it.  I don't like installers that
> > futz with my PATH, and would hate it if Emacs did that to others.
> > It's very easy to get that wrong, especially on Windows.
> 
> There has to be a better way that this.

The only way I know of is to distribute a program that writes into the
Registry.  Don't forget that there are system-wide variables and
variables specific to the current user.  And some systems have the
former locked down and the latter requires a UAC elevation.  Good luck
(you will need it) with successfully negotiating all these obstacles.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-22  7:24                       ` Eli Zaretskii
@ 2021-01-22 16:14                         ` Phillip Lord
  2021-01-22 17:03                           ` Eli Zaretskii
  0 siblings, 1 reply; 78+ messages in thread
From: Phillip Lord @ 2021-01-22 16:14 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Cc: emacs-devel@gnu.org
>> Date: Thu, 21 Jan 2021 21:37:57 +0000
>> 
>> Yep, it's big. I get this before installation:
>> 
>> 319 MB (335,214,637 bytes)
>> 
>> and this after installation.
>> 
>> 826 MB (866,624,644 bytes)
>> 
>> My guess is that it's the co-install of perl that is causing this issue
>> rather than vim.
>
> It's Vim, and Perl, and Tcl.  And at some point I'd expect them to
> include Python as well, to support those Git commands which rely on
> it.

Yep, they don't ship a "git-core" without all this baggage
unfortunately.


>> But, openssh comes for free with this. And "huge" is a
>> relative term. An installation under 1Gb seems reasonable to me.
>
> I thought this started as an attempt to make the Emacs installation
> smaller...
>
> If the rationale is to provide a full development environment on top
> of MS-Windows, then indeed the size will be much larger, and you will
> need to include a lot of packages there.  Just be sure to spell this
> out (including the size, perhaps) when you ask the user whether he or
> she wants to install that.

Yes, a small install for easy downloads, but with the option for a
bigger one. The small install is still important, though, because it
will also mean a small upgrade from one emacs to the next.


>> > Next, by "msys" you mean all those non-native programs that depend on
>> > msys-1.0.dll?  That's again meant for MinGW developers, not "normal"
>> > users.
>> 
>> Yes. Because it's got all the packages and tools and as far as I can
>> see, they work with Emacs.
>
> They mostly work, until they don't.  Like with Cygwin, there are
> subtle incompatibilities, mainly in file names and in communications
> with subprocesses and response to "signals".  Encoding defaults are
> also different.

That's true for the msys2 commands but not the mingw64 ones? In the end,
not all of the tools need for Emacs are in mingw64 part of msys2 and
msys2 comes with pacman.

>> > (Btw, pacman can ask questions and prompt the user for confirmation,
>> > but the way you invoke it in w32-msys-run doesn't seem to be prepared
>> > for such interaction?)
>> 
>> That's why I use "--no-confirm". I'm looking for the minimum here that
>> does something usable.
>
> What happens when pacman wants to replace Bash from which it was
> launched, or update itself or some of its component DLLs?

That happens straight away on first launch. I don't know yet, but am
working on it.


>
>> > I think the only good idea here is to tell the user to amend PATH by
>> > adding such-and-such directories to it.  I don't like installers that
>> > futz with my PATH, and would hate it if Emacs did that to others.
>> > It's very easy to get that wrong, especially on Windows.
>> 
>> There has to be a better way that this.
>
> The only way I know of is to distribute a program that writes into the
> Registry.  Don't forget that there are system-wide variables and
> variables specific to the current user.  And some systems have the
> former locked down and the latter requires a UAC elevation.  Good luck
> (you will need it) with successfully negotiating all these obstacles.

Would it be easier to have Emacs allow me to successfully update PATH
during run?

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-22 16:14                         ` Phillip Lord
@ 2021-01-22 17:03                           ` Eli Zaretskii
  2021-01-24 22:13                             ` Phillip Lord
  0 siblings, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-22 17:03 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: emacs-devel@gnu.org
> Date: Fri, 22 Jan 2021 16:14:33 +0000
> 
> >> > Next, by "msys" you mean all those non-native programs that depend on
> >> > msys-1.0.dll?  That's again meant for MinGW developers, not "normal"
> >> > users.
> >> 
> >> Yes. Because it's got all the packages and tools and as far as I can
> >> see, they work with Emacs.
> >
> > They mostly work, until they don't.  Like with Cygwin, there are
> > subtle incompatibilities, mainly in file names and in communications
> > with subprocesses and response to "signals".  Encoding defaults are
> > also different.
> 
> That's true for the msys2 commands but not the mingw64 ones?

Yes.  That's why I asked about msys-1.0.dll: the programs that depend
on that aren't mingw64 (native) programs.

> > The only way I know of is to distribute a program that writes into the
> > Registry.  Don't forget that there are system-wide variables and
> > variables specific to the current user.  And some systems have the
> > former locked down and the latter requires a UAC elevation.  Good luck
> > (you will need it) with successfully negotiating all these obstacles.
> 
> Would it be easier to have Emacs allow me to successfully update PATH
> during run?

Update how?



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-21 18:44                   ` phillip.lord
@ 2021-01-23  2:51                     ` Stephen Leake
  0 siblings, 0 replies; 78+ messages in thread
From: Stephen Leake @ 2021-01-23  2:51 UTC (permalink / raw)
  To: phillip.lord; +Cc: emacs-devel

phillip.lord@russet.org.uk writes:

> On 2021-01-21 18:22, Stephen Leake wrote:
>> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>> 
>>> Phillip Lord <phillip.lord@russet.org.uk> writes:
>>> 
>>>> Following on from this discussion, I have had a thought about a
>>>> better
>>>> way of producing a more function windows download and have come up
>>>> with
>>>> this idea, which is just to install msys inside the Emacs install
>>>> directory.
>>> Works for me, as long as it detects my current msys install and
>>> does not
>>> modify it without asking my permission.
>> Actually, a better way to proceed in this direction is to update the
>> current emacs mingw64 package; it is currently at 26.
>
> Is it?
>
> https://packages.msys2.org/package/mingw-w64-x86_64-emacs?repo=mingw64
>
> report 27.1

ah, my local mingw64 cache is out of date. Sorry.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-22 17:03                           ` Eli Zaretskii
@ 2021-01-24 22:13                             ` Phillip Lord
  2021-01-24 22:56                               ` Óscar Fuentes
  2021-01-25 15:20                               ` Eli Zaretskii
  0 siblings, 2 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-24 22:13 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> > They mostly work, until they don't.  Like with Cygwin, there are
>> > subtle incompatibilities, mainly in file names and in communications
>> > with subprocesses and response to "signals".  Encoding defaults are
>> > also different.
>> 
>> That's true for the msys2 commands but not the mingw64 ones?
>
> Yes.  That's why I asked about msys-1.0.dll: the programs that depend
> on that aren't mingw64 (native) programs.


But, mingw64 does not have all the packages I need.

How do people use Emacs on windows? I mean, do they install find, ls,
git, aspell and all the rest by hand? It's been a long time since I have
done it.


>> > The only way I know of is to distribute a program that writes into the
>> > Registry.  Don't forget that there are system-wide variables and
>> > variables specific to the current user.  And some systems have the
>> > former locked down and the latter requires a UAC elevation.  Good luck
>> > (you will need it) with successfully negotiating all these obstacles.
>> 
>> Would it be easier to have Emacs allow me to successfully update PATH
>> during run?
>
> Update how?

Using some magic that doesn't exist at the current time. I mean a way
like did (by fiddling with setenv) that doesn't suffer the problems that
it causes.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-24 22:13                             ` Phillip Lord
@ 2021-01-24 22:56                               ` Óscar Fuentes
  2021-01-24 23:34                                 ` Phillip Lord
  2021-01-25 15:20                               ` Eli Zaretskii
  1 sibling, 1 reply; 78+ messages in thread
From: Óscar Fuentes @ 2021-01-24 22:56 UTC (permalink / raw)
  To: emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> > They mostly work, until they don't.  Like with Cygwin, there are
>>> > subtle incompatibilities, mainly in file names and in communications
>>> > with subprocesses and response to "signals".  Encoding defaults are
>>> > also different.
>>> 
>>> That's true for the msys2 commands but not the mingw64 ones?
>>
>> Yes.  That's why I asked about msys-1.0.dll: the programs that depend
>> on that aren't mingw64 (native) programs.
>
>
> But, mingw64 does not have all the packages I need.

Ok, here goes a data point:

> How do people use Emacs on windows?

I install MSYS2 and execute a shell script that installs this packages:

universal-ctags-git ag putty-ssh aspell aspell-en aspell-es diffutils

(the scripts prepends the names listed above with the corresponding
mingw-w64-i686/x86_64 prefix). Then I install emacs with `pacman`,
either from MSYS2 binary repository or from the packages I build.

> I mean, do they install find,

I don't use it. I don't use it on GNU/Linux either (I work 99% of the
time on GNU/Linux).

> ls,

Is it necessary?

> git,

Git for Windows.

> aspell

See above.

> and all the rest by hand?

On my case, only Git for Windows and MSYS2 are installed by hand.

BTW, Git for Windows is also based on MSYS2: it installs the required
pieces that are required for running the parts of git that still depend
on POSIX.

It theory, instead of Git for Windows I could also run MSYS2's own
git.exe setting magit-git-executable and vc-git-program, without adding
MSYS2 /bin directory to PATH, but last time I tried it was somewhat
slower than Git for Windows, which is slow enough itself. Also, the
later comes with some goodies built-in, such as git-svn. MSYS2 git
package requires about 30 MB, while Git for Windows 240.

I execute runemacs.exe from wathever/msys64/mingw64/bin. emacs.el sets
PATH and exec-path:

(defun anade-a-path (path)
  (setenv "PATH" (concat (getenv "PATH") path-separator dir))
  (setq exec-path (append exec-path (list dir))))

(anade-a-path (file-name-directory (car command-line-args)))

With this I can execute programs on emacs.exe directory (that is,
whatever/msys64/mingw64/bin) with no problems.




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-24 22:56                               ` Óscar Fuentes
@ 2021-01-24 23:34                                 ` Phillip Lord
  2021-01-25  0:12                                   ` Óscar Fuentes
  2021-01-25 15:24                                   ` Eli Zaretskii
  0 siblings, 2 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-24 23:34 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

Óscar Fuentes <ofv@wanadoo.es> writes:

> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> But, mingw64 does not have all the packages I need.
>
> Ok, here goes a data point:
>
>> How do people use Emacs on windows?
>
> I install MSYS2 and execute a shell script that installs this
> packages:
>
> universal-ctags-git ag putty-ssh aspell aspell-en aspell-es diffutils
>
> (the scripts prepends the names listed above with the corresponding
> mingw-w64-i686/x86_64 prefix). Then I install emacs with `pacman`,
> either from MSYS2 binary repository or from the packages I build.

In your case, you are using msys effectively as an installer, to install
mingw64 packages. Likewise, I presume, with Emacs? This seems to come in
both an msys2 and mingw64 package.

Any reason for putty rather than openssh?


>> I mean, do they install find,
>
> I don't use it. I don't use it on GNU/Linux either (I work 99% of the
> time on GNU/Linux).

Oh, I'm a find junkie.

>> ls,
>
> Is it necessary?

Not sure. I thought dired used it, at least if it's there but maybe not.


>> git,
>
> Git for Windows.
>
>> aspell
>
> See above.
>
>> and all the rest by hand?
>
> On my case, only Git for Windows and MSYS2 are installed by hand.
>
> BTW, Git for Windows is also based on MSYS2: it installs the required
> pieces that are required for running the parts of git that still depend
> on POSIX.
>
> It theory, instead of Git for Windows I could also run MSYS2's own
> git.exe setting magit-git-executable and vc-git-program, without adding
> MSYS2 /bin directory to PATH, but last time I tried it was somewhat
> slower than Git for Windows, which is slow enough itself. Also, the
> later comes with some goodies built-in, such as git-svn. MSYS2 git
> package requires about 30 MB, while Git for Windows 240.

Yes, I hear that magit is difficult to use on Windows. I don't know if
that is fixable within Emacs, or if it is fundamental to Windows.



> I execute runemacs.exe from wathever/msys64/mingw64/bin. emacs.el sets
> PATH and exec-path:
>
> (defun anade-a-path (path)
>   (setenv "PATH" (concat (getenv "PATH") path-separator dir))
>   (setq exec-path (append exec-path (list dir))))
>
> (anade-a-path (file-name-directory (car command-line-args)))
>
> With this I can execute programs on emacs.exe directory (that is,
> whatever/msys64/mingw64/bin) with no problems.

That's a good data point! Thank you.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-24 23:34                                 ` Phillip Lord
@ 2021-01-25  0:12                                   ` Óscar Fuentes
  2021-01-25 15:24                                   ` Eli Zaretskii
  1 sibling, 0 replies; 78+ messages in thread
From: Óscar Fuentes @ 2021-01-25  0:12 UTC (permalink / raw)
  To: emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> Óscar Fuentes <ofv@wanadoo.es> writes:
>
>> Phillip Lord <phillip.lord@russet.org.uk> writes:
>>
>>> But, mingw64 does not have all the packages I need.
>>
>> Ok, here goes a data point:
>>
>>> How do people use Emacs on windows?
>>
>> I install MSYS2 and execute a shell script that installs this
>> packages:
>>
>> universal-ctags-git ag putty-ssh aspell aspell-en aspell-es diffutils
>>
>> (the scripts prepends the names listed above with the corresponding
>> mingw-w64-i686/x86_64 prefix). Then I install emacs with `pacman`,
>> either from MSYS2 binary repository or from the packages I build.
>
> In your case, you are using msys effectively as an installer, to install
> mingw64 packages. Likewise, I presume, with Emacs? This seems to come in
> both an msys2 and mingw64 package.
>
> Any reason for putty rather than openssh?

There is no mingw64 package of openssh and putty is "made for Windows",
which means that everything works out of the box (I'm not talking about
Tramp, which I don't use.)

>>> I mean, do they install find,
>>
>> I don't use it. I don't use it on GNU/Linux either (I work 99% of the
>> time on GNU/Linux).
>
> Oh, I'm a find junkie.

I also use find quite often while using a shell. For Emacs, ag (the
silver searcher) makes rgrep & co. irrelevant and my Dired usage is
quite basic.

>>> ls,
>>
>> Is it necessary?
>
> Not sure. I thought dired used it, at least if it's there but maybe
> not.

If there is no ls available, Emacs uses its own facilities. That applies
to Eshell, Dired, etc.

>>> git,
>>
>> Git for Windows.
>>
>>> aspell
>>
>> See above.
>>
>>> and all the rest by hand?
>>
>> On my case, only Git for Windows and MSYS2 are installed by hand.
>>
>> BTW, Git for Windows is also based on MSYS2: it installs the required
>> pieces that are required for running the parts of git that still depend
>> on POSIX.
>>
>> It theory, instead of Git for Windows I could also run MSYS2's own
>> git.exe setting magit-git-executable and vc-git-program, without adding
>> MSYS2 /bin directory to PATH, but last time I tried it was somewhat
>> slower than Git for Windows, which is slow enough itself. Also, the
>> later comes with some goodies built-in, such as git-svn. MSYS2 git
>> package requires about 30 MB, while Git for Windows 240.
>
> Yes, I hear that magit is difficult to use on Windows.

No, not difficult, just slow.

> I don't know if that is fixable within Emacs, or if it is fundamental
> to Windows.

Launching processes is slower in Windows than in GNU/Linux, plus Git is
heavily optimized for GNU/Linux, plus (and this is not really a "plus"
but a "times") Magit does a lot of git calls, about 50 runs of git
whenever the status buffer is updated, last time I checked.

>> I execute runemacs.exe from wathever/msys64/mingw64/bin. emacs.el sets
>> PATH and exec-path:
>>
>> (defun anade-a-path (path)
>>   (setenv "PATH" (concat (getenv "PATH") path-separator dir))
>>   (setq exec-path (append exec-path (list dir))))
>>
>> (anade-a-path (file-name-directory (car command-line-args)))
>>
>> With this I can execute programs on emacs.exe directory (that is,
>> whatever/msys64/mingw64/bin) with no problems.
>
> That's a good data point! Thank you.

You're welcome.




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-24 22:13                             ` Phillip Lord
  2021-01-24 22:56                               ` Óscar Fuentes
@ 2021-01-25 15:20                               ` Eli Zaretskii
  2021-01-25 20:01                                 ` Richard Copley
  2021-01-26 10:43                                 ` Phillip Lord
  1 sibling, 2 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-25 15:20 UTC (permalink / raw)
  To: Phillip Lord; +Cc: emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: emacs-devel@gnu.org
> Date: Sun, 24 Jan 2021 22:13:07 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > They mostly work, until they don't.  Like with Cygwin, there are
> >> > subtle incompatibilities, mainly in file names and in communications
> >> > with subprocesses and response to "signals".  Encoding defaults are
> >> > also different.
> >> 
> >> That's true for the msys2 commands but not the mingw64 ones?
> >
> > Yes.  That's why I asked about msys-1.0.dll: the programs that depend
> > on that aren't mingw64 (native) programs.
> 
> But, mingw64 does not have all the packages I need.

Are we talking about you personally, or are we talking about Emacs
users?

If the former, then MinGW64 are not the only source of good ports of
Free Software to Windows, far from that.  If they don't have some
package, you just go out and look for it elsewhere.

If we are talking about Emacs users who will download Emacs from the
GNU sites, then I'd say give them only what MinGW64 provides, so that
they could use pacman to easily update that.  Those who need more will
have to find and install whatever they need on their own.  Telling
them to install MSYS ports instead risks exposing them to subtle
problems, so I wouldn't recommend it.

> How do people use Emacs on windows? I mean, do they install find, ls,
> git, aspell and all the rest by hand?

If MinGW doesn't provide those, what else can you do?  Me, I ported
some of the packages myself (where I found no ports that were good
enough or new for me), and installed others where I found good ports.
Almost all of my ports are available from the ezwinports site.

> >> Would it be easier to have Emacs allow me to successfully update PATH
> >> during run?
> >
> > Update how?
> 
> Using some magic that doesn't exist at the current time. I mean a way
> like did (by fiddling with setenv) that doesn't suffer the problems that
> it causes.

I don't think this magic can exist.  But if someone knows, let them
speak up.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-24 23:34                                 ` Phillip Lord
  2021-01-25  0:12                                   ` Óscar Fuentes
@ 2021-01-25 15:24                                   ` Eli Zaretskii
  2021-01-25 19:49                                     ` chad
  1 sibling, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-25 15:24 UTC (permalink / raw)
  To: Phillip Lord; +Cc: ofv, emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Date: Sun, 24 Jan 2021 23:34:29 +0000
> Cc: emacs-devel@gnu.org
> 
> Oh, I'm a find junkie.

Me too.  Which is why I ported Findutils long ago (the available ports
were abysmally slow).  I have an updatedb job running every week until
this day.

> >> ls,
> >
> > Is it necessary?
> 
> Not sure. I thought dired used it, at least if it's there but maybe not.

I do use ls, regardless of Dired (which doesn't need it on Windows).
That's about the only Coreutils program I ported myself; for the rest
I use an old GnuWin32 port, which is quite adequate -- ls was the
single exception.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 15:24                                   ` Eli Zaretskii
@ 2021-01-25 19:49                                     ` chad
  2021-01-25 19:57                                       ` Eli Zaretskii
  2021-01-25 20:42                                       ` Stefan Monnier
  0 siblings, 2 replies; 78+ messages in thread
From: chad @ 2021-01-25 19:49 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: ofv, EMACS development team, Phillip Lord

[-- Attachment #1: Type: text/plain, Size: 313 bytes --]

On Mon, Jan 25, 2021 at 7:24 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > Oh, I'm a find junkie.
>
> Me too.
>

Tangent, but: you might be interested in fd-find, which is an
implementation of  (paraphrasing here) about 80% of find that's usually
several times faster.

   https://github.com/sharkdp/fd

Hth,
~Chad

[-- Attachment #2: Type: text/html, Size: 752 bytes --]

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 19:49                                     ` chad
@ 2021-01-25 19:57                                       ` Eli Zaretskii
  2021-01-25 20:42                                       ` Stefan Monnier
  1 sibling, 0 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-25 19:57 UTC (permalink / raw)
  To: chad; +Cc: ofv, emacs-devel, phillip.lord

> From: chad <yandros@gmail.com>
> Date: Mon, 25 Jan 2021 11:49:24 -0800
> Cc: Phillip Lord <phillip.lord@russet.org.uk>, ofv@wanadoo.es, 
> 	EMACS development team <emacs-devel@gnu.org>
> 
> Tangent, but: you might be interested in fd-find, which is an implementation of  (paraphrasing here) about
> 80% of find that's usually several times faster.
>  
>    https://github.com/sharkdp/fd

I know about it, but it isn't a 100% functional replacement of 'find',
and uses a confusingly different mini-language on top of that.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 15:20                               ` Eli Zaretskii
@ 2021-01-25 20:01                                 ` Richard Copley
  2021-01-25 21:17                                   ` Óscar Fuentes
  2021-01-26 16:35                                   ` Stephen Leake
  2021-01-26 10:43                                 ` Phillip Lord
  1 sibling, 2 replies; 78+ messages in thread
From: Richard Copley @ 2021-01-25 20:01 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Emacs Development, Phillip Lord

On Mon, 25 Jan 2021 at 15:21, Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Phillip Lord <phillip.lord@russet.org.uk>
> > Eli Zaretskii <eliz@gnu.org> writes:

> > >> Would it be easier to have Emacs allow me to successfully update PATH
> > >> during run?
> > >
> > > Update how?
> >
> > Using some magic that doesn't exist at the current time. I mean a way
> > like did (by fiddling with setenv) that doesn't suffer the problems that
> > it causes.
>
> I don't think this magic can exist.  But if someone knows, let them
> speak up.

No magic is needed. Just change exec-path and the PATH environment
variable matching values, not forgetting to add Emacs' libexec dir to
exec-path. I have code in my init file to select different paths,
because different toolchains that I want to run in Emacs require
different incompatible paths. It doesn't cause any problems, and
everything behaves as you expect.

This works for me because I install emacs into the /mingw64/bin
directory of MSYS2 (which is what "make install" does by default, in
the MSYS2/MINGW64 shell), so emacs can never fail to find its required
DLLs at runtime, nor find the wrong ones. (The LoadLibrary function
always searches the exe's directory and the System32 directory.)
The installer should make sure all the necessary DLLs are in the same
directory as emacs.exe, too, for the same reason.

When running emacs from the /src directory, features that load DLLs at
runtime don't work properly unless the correct DLLs are found on the
path, but there's nothing mysterious, magical or unexpected about
that.

By the way, the MSYS2 builds of Emacs no longer apply any significant
patches. There's just one, which is intended to disable ImageMagick,
and which is superfluous but harmless. I hope we don't do anything
that breaks the ability to build and install Emacs into an existing
MSYS2 tree using "./autogen.sh && ./configure && make && make
install", as their PKGBUILD script does.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 19:49                                     ` chad
  2021-01-25 19:57                                       ` Eli Zaretskii
@ 2021-01-25 20:42                                       ` Stefan Monnier
  2021-01-25 22:13                                         ` chad
                                                           ` (2 more replies)
  1 sibling, 3 replies; 78+ messages in thread
From: Stefan Monnier @ 2021-01-25 20:42 UTC (permalink / raw)
  To: chad; +Cc: ofv, Eli Zaretskii, Phillip Lord, EMACS development team

> Tangent, but: you might be interested in fd-find, which is an
> implementation of  (paraphrasing here) about 80% of find that's usually
> several times faster.

Any hint *why* it's faster?


        Stefan




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 20:01                                 ` Richard Copley
@ 2021-01-25 21:17                                   ` Óscar Fuentes
  2021-01-26  3:29                                     ` Eli Zaretskii
  2021-01-26 16:35                                   ` Stephen Leake
  1 sibling, 1 reply; 78+ messages in thread
From: Óscar Fuentes @ 2021-01-25 21:17 UTC (permalink / raw)
  To: emacs-devel

Richard Copley <rcopley@gmail.com> writes:

[snip]

> By the way, the MSYS2 builds of Emacs no longer apply any significant
> patches. There's just one, which is intended to disable ImageMagick,
> and which is superfluous but harmless.

That patch is not used. I'm not sure why I didn't delete it.

> I hope we don't do anything
> that breaks the ability to build and install Emacs into an existing
> MSYS2 tree using "./autogen.sh && ./configure && make && make
> install", as their PKGBUILD script does.

+1, although I don't expect that this is checked regularly by the Emacs
developers. Currently my Windows machines are using 27.1 as distributed
by MSYS2, but eventually I'll switch to master, building it every few
months, so when 28 is released the build should be fine.




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 20:42                                       ` Stefan Monnier
@ 2021-01-25 22:13                                         ` chad
  2021-01-25 22:28                                         ` Dmitry Gutov
  2021-01-26  3:26                                         ` Eli Zaretskii
  2 siblings, 0 replies; 78+ messages in thread
From: chad @ 2021-01-25 22:13 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: ofv, Eli Zaretskii, Phillip Lord, EMACS development team

[-- Attachment #1: Type: text/plain, Size: 1903 bytes --]

On Mon, Jan 25, 2021 at 12:42 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > Tangent, but: you might be interested in fd-find, which is an
> > implementation of  (paraphrasing here) about 80% of find that's usually
> > several times faster.
>
> Any hint *why* it's faster?
>

At this point I'm just passing along what I've read on the project pages,
but it claims most of the speed comes from leveraging the speed efforts of
ripgrep's component pieces (it's rust-based regex and ignore packages).
There are two relevant chunks from ripgrep's project page; a blog post with
details and benchmarks, and a summary on ripgrep's project page.

Blog post: https://blog.burntsushi.net/ripgrep/

Summarizing, ripgrep is fast because:
>
>    - It is built on top of Rust's regex engine
>    <https://github.com/rust-lang/regex>. Rust's regex engine uses finite
>    automata, SIMD and aggressive literal optimizations to make searching very
>    fast. (PCRE2 support can be opted into with the -P/--pcre2 flag.)
>
>
>    - Rust's regex library maintains performance with full Unicode support
>    by building UTF-8 decoding directly into its deterministic finite automaton
>    engine.
>
>
>    - It supports searching with either memory maps or by searching
>    incrementally with an intermediate buffer. The former is better for single
>    files and the latter is better for large directories. ripgrep chooses the
>    best searching strategy for you automatically.
>
>
>    - Applies your ignore patterns in .gitignore files using a RegexSet
>    <https://docs.rs/regex/1/regex/struct.RegexSet.html>. That means a
>    single file path can be matched against multiple glob patterns
>    simultaneously.
>
>
>    - It uses a lock-free parallel recursive directory iterator, courtesy
>    of crossbeam <https://docs.rs/crossbeam> and ignore
>    <https://docs.rs/ignore>.
>
>
Hope that helps,
~Chad

[-- Attachment #2: Type: text/html, Size: 5487 bytes --]

^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 20:42                                       ` Stefan Monnier
  2021-01-25 22:13                                         ` chad
@ 2021-01-25 22:28                                         ` Dmitry Gutov
  2021-01-26  3:26                                         ` Eli Zaretskii
  2 siblings, 0 replies; 78+ messages in thread
From: Dmitry Gutov @ 2021-01-25 22:28 UTC (permalink / raw)
  To: Stefan Monnier, chad
  Cc: ofv, Eli Zaretskii, EMACS development team, Phillip Lord

On 25.01.2021 22:42, Stefan Monnier wrote:
>> Tangent, but: you might be interested in fd-find, which is an
>> implementation of  (paraphrasing here) about 80% of find that's usually
>> several times faster.
> Any hint*why*  it's faster?

In my benchmarks, it isn't. But there is at least one report with 
numbers that says otherwise (on your bug mailing list).



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 20:42                                       ` Stefan Monnier
  2021-01-25 22:13                                         ` chad
  2021-01-25 22:28                                         ` Dmitry Gutov
@ 2021-01-26  3:26                                         ` Eli Zaretskii
  2 siblings, 0 replies; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-26  3:26 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: ofv, yandros, emacs-devel, phillip.lord

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 25 Jan 2021 15:42:13 -0500
> Cc: ofv@wanadoo.es, Eli Zaretskii <eliz@gnu.org>,
>  Phillip Lord <phillip.lord@russet.org.uk>,
>  EMACS development team <emacs-devel@gnu.org>
> 
> > Tangent, but: you might be interested in fd-find, which is an
> > implementation of  (paraphrasing here) about 80% of find that's usually
> > several times faster.
> 
> Any hint *why* it's faster?

It uses several threads, and uses PCRE instead of fnmatch.  The latter
is a subtle difference from what Find does, and it explains the
different results of the search.



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 21:17                                   ` Óscar Fuentes
@ 2021-01-26  3:29                                     ` Eli Zaretskii
  2021-01-26  5:43                                       ` Óscar Fuentes
  0 siblings, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-26  3:29 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 25 Jan 2021 22:17:15 +0100
> 
> > I hope we don't do anything
> > that breaks the ability to build and install Emacs into an existing
> > MSYS2 tree using "./autogen.sh && ./configure && make && make
> > install", as their PKGBUILD script does.
> 
> +1, although I don't expect that this is checked regularly by the Emacs
> developers.

??? How else do you think I build my Emacs?



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-26  3:29                                     ` Eli Zaretskii
@ 2021-01-26  5:43                                       ` Óscar Fuentes
  2021-01-26  6:56                                         ` Eli Zaretskii
  0 siblings, 1 reply; 78+ messages in thread
From: Óscar Fuentes @ 2021-01-26  5:43 UTC (permalink / raw)
  To: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> > I hope we don't do anything
>> > that breaks the ability to build and install Emacs into an existing
>> > MSYS2 tree using "./autogen.sh && ./configure && make && make
>> > install", as their PKGBUILD script does.
>> 
>> +1, although I don't expect that this is checked regularly by the Emacs
>> developers.
>
> ??? How else do you think I build my Emacs?

I thought you used MSYS, not MSYS2.

Do you also use the MinGW-w64 libraries provided by MSYS2?




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-26  5:43                                       ` Óscar Fuentes
@ 2021-01-26  6:56                                         ` Eli Zaretskii
  2021-01-26  7:37                                           ` Óscar Fuentes
  0 siblings, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-26  6:56 UTC (permalink / raw)
  To: emacs-devel, Óscar Fuentes

On January 26, 2021 7:43:23 AM GMT+02:00, "Óscar Fuentes" <ofv@wanadoo.es> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > I hope we don't do anything
> >> > that breaks the ability to build and install Emacs into an
> existing
> >> > MSYS2 tree using "./autogen.sh && ./configure && make && make
> >> > install", as their PKGBUILD script does.
> >> 
> >> +1, although I don't expect that this is checked regularly by the
> Emacs
> >> developers.
> >
> > ??? How else do you think I build my Emacs?
> 
> I thought you used MSYS, not MSYS2.

That's right, I do use MSYS.  But how is that relevant to breaking the build procedure?  What am I missing?

> Do you also use the MinGW-w64 libraries provided by MSYS2?

No.





^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-26  6:56                                         ` Eli Zaretskii
@ 2021-01-26  7:37                                           ` Óscar Fuentes
  2021-01-26  9:57                                             ` Eli Zaretskii
  2021-01-27 14:55                                             ` Stephen Leake
  0 siblings, 2 replies; 78+ messages in thread
From: Óscar Fuentes @ 2021-01-26  7:37 UTC (permalink / raw)
  To: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

> On January 26, 2021 7:43:23 AM GMT+02:00, "Óscar Fuentes" <ofv@wanadoo.es> wrote:
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> > I hope we don't do anything
>> >> > that breaks the ability to build and install Emacs into an
>> existing
>> >> > MSYS2 tree using "./autogen.sh && ./configure && make && make
>> >> > install", as their PKGBUILD script does.
>> >> 
>> >> +1, although I don't expect that this is checked regularly by the
>> Emacs
>> >> developers.
>> >
>> > ??? How else do you think I build my Emacs?
>> 
>> I thought you used MSYS, not MSYS2.
>
> That's right, I do use MSYS.

That's why I wrote "although I don't expect that this [building on
MSYS2] is checked regularly by the Emacs developers."

> But how is that relevant to breaking the build procedure? What am I
> missing?

Whatever works in MSYS does not necessarily works on MSYS2.




^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-26  7:37                                           ` Óscar Fuentes
@ 2021-01-26  9:57                                             ` Eli Zaretskii
  2021-01-26 15:58                                               ` martin rudalics
  2021-01-27 14:55                                             ` Stephen Leake
  1 sibling, 1 reply; 78+ messages in thread
From: Eli Zaretskii @ 2021-01-26  9:57 UTC (permalink / raw)
  To: emacs-devel, Óscar Fuentes

On January 26, 2021 9:37:48 AM GMT+02:00, "Óscar Fuentes" <ofv@wanadoo.es> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > That's right, I do use MSYS.
> 
> That's why I wrote "although I don't expect that this [building on
> MSYS2] is checked regularly by the Emacs developers."

But everybody else who builds on Windows here uses MSYS2, so I still don't understand the problem you had in mind.

> Whatever works in MSYS does not necessarily works on MSYS2.

Do you mean MSYS2 or do you mean MinGW64?  The former is supposed to work like MSYS, for the latter I always check in their sources to make sure nothing would break.  And again, others build with those tools and report (rare) problems when they happen.  And if people stop using our build procedure with MinGW64, there will be no reason to worry about it not working, right?





^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 15:20                               ` Eli Zaretskii
  2021-01-25 20:01                                 ` Richard Copley
@ 2021-01-26 10:43                                 ` Phillip Lord
  1 sibling, 0 replies; 78+ messages in thread
From: Phillip Lord @ 2021-01-26 10:43 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Cc: emacs-devel@gnu.org
>> Date: Sun, 24 Jan 2021 22:13:07 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> > They mostly work, until they don't.  Like with Cygwin, there are
>> >> > subtle incompatibilities, mainly in file names and in communications
>> >> > with subprocesses and response to "signals".  Encoding defaults are
>> >> > also different.
>> >> 
>> >> That's true for the msys2 commands but not the mingw64 ones?
>> >
>> > Yes.  That's why I asked about msys-1.0.dll: the programs that depend
>> > on that aren't mingw64 (native) programs.
>> 
>> But, mingw64 does not have all the packages I need.
>
> Are we talking about you personally, or are we talking about Emacs
> users?

Emacs users in general. I don't use Emacs for Windows myself, at least
not in any routine way.


> If the former, then MinGW64 are not the only source of good ports of
> Free Software to Windows, far from that.  If they don't have some
> package, you just go out and look for it elsewhere.

I am sure that this is true. But I have to scrape together tools from
several different locations, particularly if I have to host it (and thus
have the responsibilty of getting all the source), it's not going to
happen.

I might be able to use a different package manager (like chocolaty); I'd
rather use msys2 because I already use it for my build machine.

> If we are talking about Emacs users who will download Emacs from the
> GNU sites, then I'd say give them only what MinGW64 provides, so that
> they could use pacman to easily update that.  Those who need more will
> have to find and install whatever they need on their own.  Telling
> them to install MSYS ports instead risks exposing them to subtle
> problems, so I wouldn't recommend it.

Okay. In which case I'll only add the mingw64 to the path. This excludes
git which was one of my main aims to be honest. But at least we can add
a spell checker to Emacs.



>
>> How do people use Emacs on windows? I mean, do they install find, ls,
>> git, aspell and all the rest by hand?
>
> If MinGW doesn't provide those, what else can you do?  Me, I ported
> some of the packages myself (where I found no ports that were good
> enough or new for me), and installed others where I found good ports.
> Almost all of my ports are available from the ezwinports site.

Give up on these tools till a better solution appears.


>> >> Would it be easier to have Emacs allow me to successfully update PATH
>> >> during run?
>> >
>> > Update how?
>> 
>> Using some magic that doesn't exist at the current time. I mean a way
>> like did (by fiddling with setenv) that doesn't suffer the problems that
>> it causes.
>
> I don't think this magic can exist.  But if someone knows, let them
> speak up.

I don't know what the problems are that it causes, but again, if it's
not simple and we cannot provide a 90/10 solution for changing path
within a running Emacs, then I will try something that doesn't need
this.

Phil



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-26  9:57                                             ` Eli Zaretskii
@ 2021-01-26 15:58                                               ` martin rudalics
  0 siblings, 0 replies; 78+ messages in thread
From: martin rudalics @ 2021-01-26 15:58 UTC (permalink / raw)
  To: Eli Zaretskii, emacs-devel, Óscar Fuentes

 > But everybody else who builds on Windows here uses MSYS2, so I still
 > don't understand the problem you had in mind.

Not everybody else.

martin



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-25 20:01                                 ` Richard Copley
  2021-01-25 21:17                                   ` Óscar Fuentes
@ 2021-01-26 16:35                                   ` Stephen Leake
  1 sibling, 0 replies; 78+ messages in thread
From: Stephen Leake @ 2021-01-26 16:35 UTC (permalink / raw)
  To: Richard Copley; +Cc: Eli Zaretskii, Phillip Lord, Emacs Development

Richard Copley <rcopley@gmail.com> writes:

> On Mon, 25 Jan 2021 at 15:21, Eli Zaretskii <eliz@gnu.org> wrote:
>> > From: Phillip Lord <phillip.lord@russet.org.uk>
>> > Eli Zaretskii <eliz@gnu.org> writes:
>
>> > >> Would it be easier to have Emacs allow me to successfully update PATH
>> > >> during run?
>> > >
>> > > Update how?
>> >
>> > Using some magic that doesn't exist at the current time. I mean a way
>> > like did (by fiddling with setenv) that doesn't suffer the problems that
>> > it causes.
>>
>> I don't think this magic can exist.  But if someone knows, let them
>> speak up.
>
> No magic is needed. Just change exec-path and the PATH environment
> variable matching values, not forgetting to add Emacs' libexec dir to
> exec-path. I have code in my init file to select different paths,
> because different toolchains that I want to run in Emacs require
> different incompatible paths. It doesn't cause any problems, and
> everything behaves as you expect.

I do the same; I sometimes change PATH after init when I change
projects.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 78+ messages in thread

* Re: [feature/internal-msys] thoughts of a more function windows package
  2021-01-26  7:37                                           ` Óscar Fuentes
  2021-01-26  9:57                                             ` Eli Zaretskii
@ 2021-01-27 14:55                                             ` Stephen Leake
  1 sibling, 0 replies; 78+ messages in thread
From: Stephen Leake @ 2021-01-27 14:55 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

Óscar Fuentes <ofv@wanadoo.es> writes:

> That's why I wrote "although I don't expect that this [building on
> MSYS2] is checked regularly by the Emacs developers."

For what it's worth, I build on MSYS2.

I only build once a month or so, and I don't install.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 78+ messages in thread

end of thread, other threads:[~2021-01-27 14:55 UTC | newest]

Thread overview: 78+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-09 19:57 [feature/dll-only-windows] A new windows build, comments wanted Phillip Lord
2021-01-09 20:18 ` Eli Zaretskii
2021-01-09 21:31   ` Phillip Lord
2021-01-10  8:49     ` Arash Esbati
2021-01-10 15:19       ` Phillip Lord
2021-01-10 16:45     ` Eli Zaretskii
2021-01-10 18:23       ` Phillip Lord
2021-01-10 19:05         ` Eli Zaretskii
2021-01-10 19:20           ` Óscar Fuentes
2021-01-10 19:37             ` Eli Zaretskii
2021-01-10 20:52           ` `gzip` dependency (was: [feature/dll-only-windows] A new windows build, comments wanted) Stefan Monnier
2021-01-11  3:27             ` Eli Zaretskii
2021-01-11 10:00               ` `gzip` dependency Phillip Lord
2021-01-11 15:22                 ` Eli Zaretskii
2021-01-11 14:59               ` Stefan Monnier
2021-01-11 15:15                 ` Phillip Lord
2021-01-11 15:46                 ` Eli Zaretskii
2021-01-20 19:29             ` [feature/internal-msys] thoughts of a more function windows package Phillip Lord
2021-01-21 12:36               ` Stephen Leake
2021-01-21 16:11                 ` Phillip Lord
2021-01-21 18:22                 ` Stephen Leake
2021-01-21 18:44                   ` phillip.lord
2021-01-23  2:51                     ` Stephen Leake
2021-01-21 18:53                   ` Óscar Fuentes
2021-01-21 14:11               ` Eli Zaretskii
2021-01-21 16:44                 ` Phillip Lord
2021-01-21 20:17                   ` Eli Zaretskii
2021-01-21 21:37                     ` Phillip Lord
2021-01-22  7:24                       ` Eli Zaretskii
2021-01-22 16:14                         ` Phillip Lord
2021-01-22 17:03                           ` Eli Zaretskii
2021-01-24 22:13                             ` Phillip Lord
2021-01-24 22:56                               ` Óscar Fuentes
2021-01-24 23:34                                 ` Phillip Lord
2021-01-25  0:12                                   ` Óscar Fuentes
2021-01-25 15:24                                   ` Eli Zaretskii
2021-01-25 19:49                                     ` chad
2021-01-25 19:57                                       ` Eli Zaretskii
2021-01-25 20:42                                       ` Stefan Monnier
2021-01-25 22:13                                         ` chad
2021-01-25 22:28                                         ` Dmitry Gutov
2021-01-26  3:26                                         ` Eli Zaretskii
2021-01-25 15:20                               ` Eli Zaretskii
2021-01-25 20:01                                 ` Richard Copley
2021-01-25 21:17                                   ` Óscar Fuentes
2021-01-26  3:29                                     ` Eli Zaretskii
2021-01-26  5:43                                       ` Óscar Fuentes
2021-01-26  6:56                                         ` Eli Zaretskii
2021-01-26  7:37                                           ` Óscar Fuentes
2021-01-26  9:57                                             ` Eli Zaretskii
2021-01-26 15:58                                               ` martin rudalics
2021-01-27 14:55                                             ` Stephen Leake
2021-01-26 16:35                                   ` Stephen Leake
2021-01-26 10:43                                 ` Phillip Lord
2021-01-11  9:59           ` [feature/dll-only-windows] A new windows build, comments wanted Phillip Lord
2021-01-11 15:21             ` Eli Zaretskii
2021-01-11 18:29               ` Phillip Lord
2021-01-09 21:36   ` Óscar Fuentes
2021-01-10 16:46     ` Eli Zaretskii
2021-01-10 18:34       ` Phillip Lord
2021-01-09 21:51   ` Andrea Corallo via Emacs development discussions.
2021-01-10  3:33     ` Eli Zaretskii
2021-01-10 15:09     ` Phillip Lord
2021-01-10 19:06       ` Andrea Corallo via Emacs development discussions.
2021-01-11  9:47         ` Phillip Lord
2021-01-11 11:01           ` Andrea Corallo via Emacs development discussions.
2021-01-11 16:29             ` Phillip Lord
2021-01-11 17:21               ` Andrea Corallo via Emacs development discussions.
2021-01-10 15:14   ` Phillip Lord
2021-01-10 17:23     ` Eli Zaretskii
2021-01-09 20:47 ` Alan Third
2021-01-09 21:33   ` Phillip Lord
2021-01-10  0:04     ` Alan Third
2021-01-10  3:28   ` Eli Zaretskii
2021-01-10 15:43 ` Phillip Lord
2021-01-12  6:01   ` Corwin Brust
2021-01-12  9:48     ` Phillip Lord
2021-01-12 10:27       ` Corwin Brust

unofficial mirror of emacs-devel@gnu.org 

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://yhetil.org/emacs-devel/0 emacs-devel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 emacs-devel emacs-devel/ https://yhetil.org/emacs-devel \
		emacs-devel@gnu.org
	public-inbox-index emacs-devel

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.yhetil.org/yhetil.emacs.devel
	nntp://news.gmane.io/gmane.emacs.devel


AGPL code for this site: git clone http://ou63pmih66umazou.onion/public-inbox.git