In preparation for making a native-comp build of Emacs on Windows, I have revisited the windows packaging process. The current version figures out dependencies using msys2 and as a result has an awful lot of dependencies. Even with some efforts to reduce this size (e.g. python isn't included any more) it's still pretty large. So I have tried a new technique which is now on feature/dll-only-windows. Essentially, I just name all of the DLLs that Emacs uses directly and then figure out any dependencies of these using `ntldd`. I've tried `ntldd` directly on emacs.exe which does not work. The practical upshot of this is that the dependencies file is just a pile of DLLs. The installer version of Emacs (which is the smallest because it is most compressed) is now only 45Mb (vs 63) in size, with zip being 130Mb (vs 160Mb). The source download still using the same technique as before; i.e. the source bundle will produce a lot more than just the DLLs included. Some questions for now: 1) Does this seem reasonable? I can some upload some binaries to alpha if anyone is prepared to try them. 2) harfbuzz is currently not working on i686. Before I fix this, I think it is worth asking whether I still need to produce a i686 binary. It doubles the build time, obviously. 3) Currently the "no-deps" version actually includes libXpm. Emacs starts without it, but looks ugly. I would like to no longer special case libXpm and just make the "no-deps" download really include no deps. I think this is reasonable, because this download is now, really special purpose and "with-deps" is the default. Some questions for when I add a native-comp build 4) Currently, native-comp is an "--with-nativecomp" option even on the native-comp branch. It is likely to be merged this way to master? Currently, I build the Windows distribution of Emacs with all the default options. I can make an alternative release "--with-nativecomp" but needs some effort; it also does not address the question of how I should build Emacs for Windows when a full release of Emacs-28 happens. If, the full release of Emacs-28 will be native-comp, I'd rather start building snapshots with it as soon as it is merged. 5) Why is it "--with-nativecomp", shouldn't it be "--with-native-comp". Thoughts welcome. Phil
> From: Phillip Lord <phillip.lord@russet.org.uk> > Date: Sat, 09 Jan 2021 19:57:00 +0000 > > So I have tried a new technique which is now on > feature/dll-only-windows. Essentially, I just name all of the DLLs that > Emacs uses directly and then figure out any dependencies of these using > `ntldd`. I've tried `ntldd` directly on emacs.exe which does not > work. Of course, it won't: we load all the DLLs dynamically at run time, we don't link against their import libraries. The only exceptions I know of are (1) GMP, and (2) libgccjit (in the native-comp build). > The practical upshot of this is that the dependencies file is just > a pile of DLLs. Did you verify that the DLLs include all of _their_ dependencies? If yes, how did you do that? > 2) harfbuzz is currently not working on i686. Is that a bug in the MinGW64 HarfBuzz port, or is that a bug in Emacs? I'm using a 32-bit Emacs build with HarfBuzz all the time, but it's HarfBuzz I built myself (it's available from the ezwinports site). > Before I fix this, I think it is worth asking whether I still need > to produce a i686 binary. I cannot tell you what to do, but it would be nice to have binaries that can be run on older Windows versions. So if the build supports XP and older Windows, keeping the 32-bit build would be an advantage. > 3) Currently the "no-deps" version actually includes libXpm. Emacs > starts without it, but looks ugly. I would like to no longer special > case libXpm and just make the "no-deps" download really include no > deps. I think this is reasonable, because this download is now, really > special purpose and "with-deps" is the default. If we believe no one will want the no-deps download, why have it at all? If we think someone will want it, I don't think they should be punished by having BW icons on the tool bar. > 4) Currently, native-comp is an "--with-nativecomp" option even on the > native-comp branch. It is likely to be merged this way to master? Yes, I think so. > Currently, I build the Windows distribution of Emacs with all the > default options. Which non-default options of practical importance does that leave out? > I can make an alternative release "--with-nativecomp" but needs some > effort; it also does not address the question of how I should build > Emacs for Windows when a full release of Emacs-28 happens. If, the > full release of Emacs-28 will be native-comp, I'd rather start > building snapshots with it as soon as it is merged. I think you should build --with-nativecomp. People can always uninstall libgccjit or rename it if they don't want to use native compilation. > 5) Why is it "--with-nativecomp", shouldn't it be "--with-native-comp". Probably, but we didn't yet get to splitting such thin hair in that branch ;-)
On Sat, Jan 09, 2021 at 07:57:00PM +0000, Phillip Lord wrote:
>
> 3) Currently the "no-deps" version actually includes libXpm. Emacs
> starts without it, but looks ugly. I would like to no longer special
> case libXpm and just make the "no-deps" download really include no
> deps. I think this is reasonable, because this download is now, really
> special purpose and "with-deps" is the default.
Do you even really need libXpm? We manage without it on NS and Cairo
builds by using the built-in functions at src/image.c:4671. Perhaps
that could be extended to Windows as well?
--
Alan Third
Eli Zaretskii <eliz@gnu.org> writes: >> From: Phillip Lord <phillip.lord@russet.org.uk> >> Date: Sat, 09 Jan 2021 19:57:00 +0000 >> >> So I have tried a new technique which is now on >> feature/dll-only-windows. Essentially, I just name all of the DLLs that >> Emacs uses directly and then figure out any dependencies of these using >> `ntldd`. I've tried `ntldd` directly on emacs.exe which does not >> work. > > Of course, it won't: we load all the DLLs dynamically at run time, we > don't link against their import libraries. The only exceptions I know > of are (1) GMP, and (2) libgccjit (in the native-comp build). Indeed. >> The practical upshot of this is that the dependencies file is just >> a pile of DLLs. > > Did you verify that the DLLs include all of _their_ dependencies? If > yes, how did you do that? It's totally dependent on ntldd being correct, of course. It comes with a --recursive option which I used rather than reimplementing it myself. As ever, the only tests I run on the windows package for Emacs are to unpack, start and then run "w32-feature.el" from etc. These all pass. >> 2) harfbuzz is currently not working on i686. > > Is that a bug in the MinGW64 HarfBuzz port, or is that a bug in Emacs? > I'm using a 32-bit Emacs build with HarfBuzz all the time, but it's > HarfBuzz I built myself (it's available from the ezwinports site). I don't know, I haven't debugged it yet:-) >> Before I fix this, I think it is worth asking whether I still need >> to produce a i686 binary. > > I cannot tell you what to do, but it would be nice to have binaries > that can be run on older Windows versions. So if the build supports > XP and older Windows, keeping the 32-bit build would be an advantage. I know that you use a very old windows, but I doubt you use my packages! I have no other data of how many people this would impact, but my guess would be not many, so I am asking for opinions. One solution could be to stop making the 32-bit build and see if anyone complains. It could be restored, either before or after the Emacs-28 release. >> 3) Currently the "no-deps" version actually includes libXpm. Emacs >> starts without it, but looks ugly. I would like to no longer special >> case libXpm and just make the "no-deps" download really include no >> deps. I think this is reasonable, because this download is now, really >> special purpose and "with-deps" is the default. > > If we believe no one will want the no-deps download, why have it at > all? If we think someone will want it, I don't think they should be > punished by having BW icons on the tool bar. Indeed. My guess is that currently if people use -no-deps they do so because they want the smaller download. This change, of course, we reduce the difference and regardless the installer version is smaller still (by a half). Given all that, the only real audience for the -no-deps version would be those who have a mingw64 installation already. They won't get BW icons (assuming that libXpm is installed). Of course, that audience is likely to be technically skilled and they could just delete the DLLs in the "with-deps" download. On balance, therefore, I would say we don't need the -no-deps version and it would clean up the download site and reduce confusion. >> 4) Currently, native-comp is an "--with-nativecomp" option even on the >> native-comp branch. It is likely to be merged this way to master? > > Yes, I think so. Are you worried about stability or newness? I guess it will become the default at some point. It seems to have few disadvantages, other than hammering the CPU at bit at initial start up. > >> Currently, I build the Windows distribution of Emacs with all the >> default options. > > Which non-default options of practical importance does that leave out? Currently, none. If --with-nativecomp is not default, then, I think it would become the example. >> I can make an alternative release "--with-nativecomp" but needs some >> effort; it also does not address the question of how I should build >> Emacs for Windows when a full release of Emacs-28 happens. If, the >> full release of Emacs-28 will be native-comp, I'd rather start >> building snapshots with it as soon as it is merged. > > I think you should build --with-nativecomp. People can always > uninstall libgccjit or rename it if they don't want to use native > compilation. Okay. I guess "make NATIVE_FULL_AOT=1" would be the thing as well, depending on how long it takes. >> 5) Why is it "--with-nativecomp", shouldn't it be "--with-native-comp". > > Probably, but we didn't yet get to splitting such thin hair in that > branch ;-) Yeah, can't help it. Splitting hairs is my job. Phil
Alan Third <alan@idiocy.org> writes:
> On Sat, Jan 09, 2021 at 07:57:00PM +0000, Phillip Lord wrote:
>>
>> 3) Currently the "no-deps" version actually includes libXpm. Emacs
>> starts without it, but looks ugly. I would like to no longer special
>> case libXpm and just make the "no-deps" download really include no
>> deps. I think this is reasonable, because this download is now, really
>> special purpose and "with-deps" is the default.
>
> Do you even really need libXpm? We manage without it on NS and Cairo
> builds by using the built-in functions at src/image.c:4671. Perhaps
> that could be extended to Windows as well?
I don't think we do need it in the version with deps. AFAIU, it's just
there as the smallest dependency that can give colour icons.
Phil
Eli Zaretskii <eliz@gnu.org> writes:
>> Before I fix this, I think it is worth asking whether I still need
>> to produce a i686 binary.
>
> I cannot tell you what to do, but it would be nice to have binaries
> that can be run on older Windows versions. So if the build supports
> XP and older Windows, keeping the 32-bit build would be an advantage.
MSYS2 MinGW-w64 binaries do not support XP, they officially decided this
some years ago and now some runtime dlls use APIs not present on XP.
Eli Zaretskii <eliz@gnu.org> writes:
[...]
> I think you should build --with-nativecomp. People can always
> uninstall libgccjit or rename it if they don't want to use native
> compilation.
ATM if using --with-nativecomp libgccjit is not present (or non
functional) we stop at configure time with an error. I guess that's one
of the hairs we'll have to split then :)
Andrea
On Sat, Jan 09, 2021 at 09:33:56PM +0000, Phillip Lord wrote:
> Alan Third <alan@idiocy.org> writes:
>
> > Do you even really need libXpm? We manage without it on NS and Cairo
> > builds by using the built-in functions at src/image.c:4671. Perhaps
> > that could be extended to Windows as well?
>
> I don't think we do need it in the version with deps. AFAIU, it's just
> there as the smallest dependency that can give colour icons.
Yes, my point is we have colour icons with no deps at all by using
functions already in image.c.
--
Alan Third
> Date: Sat, 9 Jan 2021 20:47:41 +0000
> From: Alan Third <alan@idiocy.org>
> Cc: emacs-devel@gnu.org
>
> On Sat, Jan 09, 2021 at 07:57:00PM +0000, Phillip Lord wrote:
> >
> > 3) Currently the "no-deps" version actually includes libXpm. Emacs
> > starts without it, but looks ugly. I would like to no longer special
> > case libXpm and just make the "no-deps" download really include no
> > deps. I think this is reasonable, because this download is now, really
> > special purpose and "with-deps" is the default.
>
> Do you even really need libXpm? We manage without it on NS and Cairo
> builds by using the built-in functions at src/image.c:4671. Perhaps
> that could be extended to Windows as well?
Emacs supports native image display on Windows, but that is still
experimental. In time, we could indeed try to get rid of the libXpm
dependency.
> From: Andrea Corallo <akrl@sdf.org>
> Cc: Phillip Lord <phillip.lord@russet.org.uk>, emacs-devel@gnu.org
> Date: Sat, 09 Jan 2021 21:51:16 +0000
>
> Eli Zaretskii <eliz@gnu.org> writes:
>
> [...]
>
> > I think you should build --with-nativecomp. People can always
> > uninstall libgccjit or rename it if they don't want to use native
> > compilation.
>
> ATM if using --with-nativecomp libgccjit is not present (or non
> functional) we stop at configure time with an error.
The situation relevant here is that libgccjit is present at build
time, but not at ruin time (on another system).
Phillip Lord <phillip.lord@russet.org.uk> writes:
> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Did you verify that the DLLs include all of _their_ dependencies? If
>> yes, how did you do that?
>
> It's totally dependent on ntldd being correct, of course. It comes with
> a --recursive option which I used rather than reimplementing it
> myself.
cygcheck.exe has this capability as well. Try
cygcheck libjansson-4.dll
in a mingw64 console. Maybe you can use this to double check the
dependencies reported by ntldd.
Best, Arash
Andrea Corallo <akrl@sdf.org> writes:
> Eli Zaretskii <eliz@gnu.org> writes:
>
> [...]
>
>> I think you should build --with-nativecomp. People can always
>> uninstall libgccjit or rename it if they don't want to use native
>> compilation.
>
> ATM if using --with-nativecomp libgccjit is not present (or non
> functional) we stop at configure time with an error. I guess that's one
> of the hairs we'll have to split then :)
I think there are two hairs here. Having an emacs compiled with
nativecomp behave cleanly if libgccjit is not available at runtime. And
having a runtime mechanism for switching nativecomp of totally even if
it is compiled in.
Phil
My test for GMP is also currently failing. It is this: (ert-deftest feature-gmp () (should (string-match-p "GMP" system-configuration-features))) This test also fails on Emacs-28 master on my Ubuntu machine, as well as on my Windows package, although all of them report that libgmp support is available at configure time. And the test succeeds on Emacs-27 on the same machines. So it looks like the test is wrong, rather libgmp support broken. Any thoughts on why GMP is no longer in system-configuration-features for Emacs-28. Is there a better test that I can use? Can we add `gmp-available-p' or equivalent? Phil
Arash Esbati <arash@gnu.org> writes:
> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>> Did you verify that the DLLs include all of _their_ dependencies? If
>>> yes, how did you do that?
>>
>> It's totally dependent on ntldd being correct, of course. It comes with
>> a --recursive option which I used rather than reimplementing it
>> myself.
>
> cygcheck.exe has this capability as well. Try
>
> cygcheck libjansson-4.dll
>
> in a mingw64 console. Maybe you can use this to double check the
> dependencies reported by ntldd.
Thanks, I will have a poke.
Phil
I have pushed some binaries up to alpha in case anyone wants to give them a trial. Directory: https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-28/ Two download links: as zip or installer version. https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-28/emacs-28.0.50-snapshot-feature_dll-only-windows-2021-01-09-x86_64.zip https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-28/emacs-28.0.50-snapshot-feature_dll-only-windows-2021-01-09-x86_64-installer.exe Phil
> From: Phillip Lord <phillip.lord@russet.org.uk> > Cc: emacs-devel@gnu.org > Date: Sat, 09 Jan 2021 21:31:45 +0000 > > > Did you verify that the DLLs include all of _their_ dependencies? If > > yes, how did you do that? > > It's totally dependent on ntldd being correct, of course. It comes with > a --recursive option which I used rather than reimplementing it > myself. You can use objdump: objdump -p *.exe *.dll | fgrep "DLL Name:" | gawk " {print $3, $4, $5}" | sort -u Keep adding DLLs and running the above command until all the DLLs it shows are in your staging directory. > >> 4) Currently, native-comp is an "--with-nativecomp" option even on the > >> native-comp branch. It is likely to be merged this way to master? > > > > Yes, I think so. > > Are you worried about stability or newness? I guess it will become the > default at some point. It seems to have few disadvantages, other than > hammering the CPU at bit at initial start up. Usually, such radical features start as optional, but maybe we will decide otherwise. Why does it matter?
> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sat, 09 Jan 2021 22:36:36 +0100
>
> > I cannot tell you what to do, but it would be nice to have binaries
> > that can be run on older Windows versions. So if the build supports
> > XP and older Windows, keeping the 32-bit build would be an advantage.
>
> MSYS2 MinGW-w64 binaries do not support XP, they officially decided this
> some years ago and now some runtime dlls use APIs not present on XP.
That's what I knew. So in that case, I see no reason for us to
produce official 32-bit binaries, as the chances for them being used
are likely nil.
> From: Phillip Lord <phillip.lord@russet.org.uk>
> Date: Sun, 10 Jan 2021 15:14:03 +0000
>
> Any thoughts on why GMP is no longer in system-configuration-features
> for Emacs-28.
It's a bug, please report it.
Eli Zaretskii <eliz@gnu.org> writes:
>> >> 4) Currently, native-comp is an "--with-nativecomp" option even on the
>> >> native-comp branch. It is likely to be merged this way to master?
>> >
>> > Yes, I think so.
>>
>> Are you worried about stability or newness? I guess it will become the
>> default at some point. It seems to have few disadvantages, other than
>> hammering the CPU at bit at initial start up.
>
> Usually, such radical features start as optional, but maybe we will
> decide otherwise.
>
> Why does it matter?
I'd like to keep the Windows package using the default options as far as
possible, mostly because I am lazy and because it means that the windows
package updates when ever the defaults do.
Currently, we have
--without-dbus --without-compress-install
The latter is so the documentation works without gzip. The latter is
because we don't normally have dbus available. I am not sure why either
of these needed (i.e. why they are not the default on windows).
Adding --with-nativecomp isn't massively problematic, but would need
removing when and if it becomes default. Not a big problem, the less
difference between the windows package and the defaults, the happier I
would be.
Phil
Eli Zaretskii <eliz@gnu.org> writes:
>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Sat, 09 Jan 2021 22:36:36 +0100
>>
>> > I cannot tell you what to do, but it would be nice to have binaries
>> > that can be run on older Windows versions. So if the build supports
>> > XP and older Windows, keeping the 32-bit build would be an advantage.
>>
>> MSYS2 MinGW-w64 binaries do not support XP, they officially decided this
>> some years ago and now some runtime dlls use APIs not present on XP.
>
> That's what I knew. So in that case, I see no reason for us to
> produce official 32-bit binaries, as the chances for them being used
> are likely nil.
Then I shall disable the code that produces it for Emacs-28. I will
leave the code and the "-x86_64-" names in place, I think, just in case
we get any complaints from Windows 7/10 32-bit users. Then, that can go
for Emacs-29, so binaries will become simply "emacs-29.1.zip" or
"emacs-29.1-installer.exe".
Assuming that this smaller DLL only distribution tests once I start
producing snapshots, I will also remove the "-no-deps" zip from
Emacs-28. At this point, I no longer need the independent copies of the
dependencies, so they can go as well. That will reduce the total number
of files for releases from 8 binary + 1 source (for the deps), to 2
binary + 1 source.
Phil
> From: Phillip Lord <phillip.lord@russet.org.uk> > Date: Sun, 10 Jan 2021 18:23:06 +0000 > Cc: emacs-devel@gnu.org > > I'd like to keep the Windows package using the default options as far as > possible, mostly because I am lazy and because it means that the windows > package updates when ever the defaults do. It's your call. I just envision many users to expect native-comp support and asking why isn't it there. > Currently, we have > > --without-dbus --without-compress-install > > The latter is so the documentation works without gzip. The latter is > because we don't normally have dbus available. I am not sure why either > of these needed (i.e. why they are not the default on windows). The gzip thing is again your call (having gzip in the package is no big deal, IMO). Dbus is not really useful on MS-Windows, so IMO it makes no sense building with it.
Phillip Lord <phillip.lord@russet.org.uk> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> Eli Zaretskii <eliz@gnu.org> writes: >> >> [...] >> >>> I think you should build --with-nativecomp. People can always >>> uninstall libgccjit or rename it if they don't want to use native >>> compilation. >> >> ATM if using --with-nativecomp libgccjit is not present (or non >> functional) we stop at configure time with an error. I guess that's one >> of the hairs we'll have to split then :) > > > I think there are two hairs here. Having an emacs compiled with > nativecomp behave cleanly if libgccjit is not available at runtime. Right, at this stage this should be easy to implement (on Windows). > And having a runtime mechanism for switching nativecomp of totally > even if it is compiled in. We should define "switching nativecomp of" and the triggering mechanism, this might be already implmentented. Andrea
Eli Zaretskii <eliz@gnu.org> writes: >> I'd like to keep the Windows package using the default options as far as >> possible, mostly because I am lazy and because it means that the windows >> package updates when ever the defaults do. > > It's your call. I just envision many users to expect native-comp > support and asking why isn't it there. I agree. In fact, some popular packages already recommend using native-comp. >> Currently, we have >> >> --without-dbus --without-compress-install >> >> The latter is so the documentation works without gzip. The latter is >> because we don't normally have dbus available. I am not sure why either >> of these needed (i.e. why they are not the default on windows). > > The gzip thing is again your call (having gzip in the package is no > big deal, IMO). Dbus is not really useful on MS-Windows, so IMO it > makes no sense building with it. I think that Phillip's problem with that is that gzip does not exist as a MinGW package in the MSYS2 ecosystem.
> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sun, 10 Jan 2021 20:20:22 +0100
>
> I think that Phillip's problem with that is that gzip does not exist as
> a MinGW package in the MSYS2 ecosystem.
I'm surprised, but then it's very easy to build it.
> The gzip thing is again your call (having gzip in the package is no
> big deal, IMO).
BTW, we don't really need a `gzip` executable as long as we have the
`zlib-decompress-region` function, right?
Stefan
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Phillip Lord <phillip.lord@russet.org.uk>, emacs-devel@gnu.org
> Date: Sun, 10 Jan 2021 15:52:07 -0500
>
> > The gzip thing is again your call (having gzip in the package is no
> > big deal, IMO).
>
> BTW, we don't really need a `gzip` executable as long as we have the
> `zlib-decompress-region` function, right?
Not necessarily, because Info files can be read outside Emacs.
Did we completely replaced jka-compr in Emacs?
Andrea Corallo <akrl@sdf.org> writes:
> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>>
>> I think there are two hairs here. Having an emacs compiled with
>> nativecomp behave cleanly if libgccjit is not available at runtime.
>
> Right, at this stage this should be easy to implement (on Windows).
>
>> And having a runtime mechanism for switching nativecomp of totally
>> even if it is compiled in.
>
> We should define "switching nativecomp of" and the triggering mechanism,
> this might be already implmentented.
From a user perspective, I think that there are two reasons we might
want to switch off native-comp.
The first is because it's broken or providing buggy behaviour. In the
extreme, and if I understand the build process fully, I guess this is
impossible to switch of in a binary distribution, because some
native-comp happens before dumping. Other than that, we can "switch off"
native-comp by just deleting the *eln files, right, or otherwise
preventing their loading. In practice, I think this is a minor
motivation; if we discover bugs in the native-comp, they would be fixed
by making another release.
The second reason is that the initial compilation takes quite a lot of
CPU. I wouldn't like that to happen while my laptop where in battery
power, for instance. At the moment, it's possible to drop the number of
jobs that native-comp uses. I don't think that there is a way to drop
this to zero at the moment. Probably, we need that.
Phil
Eli Zaretskii <eliz@gnu.org> writes: >> From: Phillip Lord <phillip.lord@russet.org.uk> >> Date: Sun, 10 Jan 2021 18:23:06 +0000 >> Cc: emacs-devel@gnu.org >> >> I'd like to keep the Windows package using the default options as far as >> possible, mostly because I am lazy and because it means that the windows >> package updates when ever the defaults do. > > It's your call. I just envision many users to expect native-comp > support and asking why isn't it there. No, you misunderstand me. I would plan to make Emacs-28 --with-nativecomp. But if I am doing this for the windows build, it is reasonable to assume that other packagers will be doing for their builds. If all that is true, it would make sense to me for it to be the default. > >> Currently, we have >> >> --without-dbus --without-compress-install >> >> The latter is so the documentation works without gzip. The latter is >> because we don't normally have dbus available. I am not sure why either >> of these needed (i.e. why they are not the default on windows). > > The gzip thing is again your call (having gzip in the package is no > big deal, IMO). Dbus is not really useful on MS-Windows, so IMO it > makes no sense building with it. We've been down this road before. I could add gzip and I am happy to consider this. But what other tools do we add. What about aspell (or equivalent), so we can spell check? Git so we can version? If I add them, then we get more "out-of-the-box" behaviour. But where do we stop. Actually, for both of those, I'd rather see Emacs depend on their respective libraries. Magit is widely reported to be usuable on Windows; libgit integration might work around this. Anyway, getting off-topic. Phil
Eli Zaretskii <eliz@gnu.org> writes:
>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: Phillip Lord <phillip.lord@russet.org.uk>, emacs-devel@gnu.org
>> Date: Sun, 10 Jan 2021 15:52:07 -0500
>>
>> > The gzip thing is again your call (having gzip in the package is no
>> > big deal, IMO).
>>
>> BTW, we don't really need a `gzip` executable as long as we have the
>> `zlib-decompress-region` function, right?
>
> Not necessarily, because Info files can be read outside Emacs.
This is not an issue for the Windows package of Emacs, as far as I can tell.
Phil
Phillip Lord <phillip.lord@russet.org.uk> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> Phillip Lord <phillip.lord@russet.org.uk> writes: >> >>> >>> I think there are two hairs here. Having an emacs compiled with >>> nativecomp behave cleanly if libgccjit is not available at runtime. >> >> Right, at this stage this should be easy to implement (on Windows). >> >>> And having a runtime mechanism for switching nativecomp of totally >>> even if it is compiled in. >> >> We should define "switching nativecomp of" and the triggering mechanism, >> this might be already implmentented. > > >>From a user perspective, I think that there are two reasons we might > want to switch off native-comp. > > The first is because it's broken or providing buggy behaviour. In the > extreme, and if I understand the build process fully, I guess this is > impossible to switch of in a binary distribution, because some > native-comp happens before dumping. Other than that, we can "switch off" > native-comp by just deleting the *eln files, right, or otherwise > preventing their loading. In practice, I think this is a minor > motivation; if we discover bugs in the native-comp, they would be fixed > by making another release. Hi Phillip, ATM we can prevent Emacs loading .eln file in place of .elc automatically setting `load-no-native' to t. > The second reason is that the initial compilation takes quite a lot of > CPU. I wouldn't like that to happen while my laptop where in battery > power, for instance. At the moment, it's possible to drop the number of > jobs that native-comp uses. I don't think that there is a way to drop > this to zero at the moment. Probably, we need that. Setting `comp-deferred-compilation' to nil should stop any automatic attempt of compiling asycronousy. Maybe these two settings are already covering most of these needs? Regards Andrea
>> > The gzip thing is again your call (having gzip in the package is no
>> > big deal, IMO).
>> BTW, we don't really need a `gzip` executable as long as we have the
>> `zlib-decompress-region` function, right?
> Not necessarily, because Info files can be read outside Emacs.
I don't think we should worry about that: those who want to read Info
files with something else than Emacs may need to have `gzip` installed,
and I think it's perfectly acceptable.
My question was instead about whether or not Emacs does indeed make use
of `zlib-decompress-region` when it tries to read a gzipped Info file,
and whether the Windows build does always provide
`zlib-decompress-region` or whether that function also requires an
external dependency that may or may not be available (in which case
it's no better than using the `gzip` executable in this respect).
Stefan
Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>> > The gzip thing is again your call (having gzip in the package is no
>>> > big deal, IMO).
>>> BTW, we don't really need a `gzip` executable as long as we have the
>>> `zlib-decompress-region` function, right?
>> Not necessarily, because Info files can be read outside Emacs.
>
> I don't think we should worry about that: those who want to read Info
> files with something else than Emacs may need to have `gzip` installed,
> and I think it's perfectly acceptable.
>
> My question was instead about whether or not Emacs does indeed make use
> of `zlib-decompress-region` when it tries to read a gzipped Info file,
> and whether the Windows build does always provide
> `zlib-decompress-region` or whether that function also requires an
> external dependency that may or may not be available (in which case
> it's no better than using the `gzip` executable in this respect).
zlib is not given as an explicit dependency of Emacs on Windows at the
moment, although perhaps it should be. However, it is included as a
transitive dependency.
If I gzip all the info files then do M-x info and look into a node, I
get an error with message "Unpression program `gzip' not found". If I
open a gzip node with find-file-literally then do eval
(zlib-decompress-region (point-min) (point-max)), I get the contents of
the info file.
Conclusions:
1) Emacs uses gzip to decompress but doesn't need to.
2) zlib should probably be added an explicit dependency regardless,
since it's a feature of Emacs that can be disabled.
Phil
> From: Phillip Lord <phillip.lord@russet.org.uk> > Cc: emacs-devel@gnu.org > Date: Mon, 11 Jan 2021 09:59:29 +0000 > > > It's your call. I just envision many users to expect native-comp > > support and asking why isn't it there. > > No, you misunderstand me. I would plan to make Emacs-28 > --with-nativecomp. But if I am doing this for the windows build, it is > reasonable to assume that other packagers will be doing for their > builds. Not necessarily: on platforms other than MS-Windows, if Emacs was linked against some optional library, that library must be installed for Emacs to run without crashing. So on MS-Windows you could build Emacs with all the possible extensions, and let users decide which ones they want to install and use. > > The gzip thing is again your call (having gzip in the package is no > > big deal, IMO). Dbus is not really useful on MS-Windows, so IMO it > > makes no sense building with it. > > We've been down this road before. Did we reach any conclusions? > I could add gzip and I am happy to consider this. But what other > tools do we add. What about aspell (or equivalent), so we can spell > check? Git so we can version? If I add them, then we get more > "out-of-the-box" behaviour. But where do we stop. We stop where you decide to stop. (And the utility of aspell is much smaller than that of gzip, btw. And the complexity of aspell installation is OTOH significantly greater.) There's no need for any consistency here. "Consistency is the hobgoblin", and all that.
> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> Date: Mon, 11 Jan 2021 10:00:32 +0000
>
> >> BTW, we don't really need a `gzip` executable as long as we have the
> >> `zlib-decompress-region` function, right?
> >
> > Not necessarily, because Info files can be read outside Emacs.
>
> This is not an issue for the Windows package of Emacs, as far as I can tell.
Emacs installs Info files that can be used outside of Emacs. So I
think it could be an issue for Emacs.
> From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: phillip.lord@russet.org.uk, emacs-devel@gnu.org > Date: Mon, 11 Jan 2021 09:59:19 -0500 > > My question was instead about whether or not Emacs does indeed make use > of `zlib-decompress-region` when it tries to read a gzipped Info file, No, I don't think we do. > and whether the Windows build does always provide > `zlib-decompress-region` Only if Emacs was built with zlib. > or whether that function also requires an external dependency that > may or may not be available One needs both an Emacs compiled with zlib, and the zlib DLL available.
Andrea Corallo <akrl@sdf.org> writes:
>> The first is because it's broken or providing buggy behaviour. In the
>> extreme, and if I understand the build process fully, I guess this is
>> impossible to switch of in a binary distribution, because some
>> native-comp happens before dumping. Other than that, we can "switch off"
>> native-comp by just deleting the *eln files, right, or otherwise
>> preventing their loading. In practice, I think this is a minor
>> motivation; if we discover bugs in the native-comp, they would be fixed
>> by making another release.
>
> Hi Phillip,
>
> ATM we can prevent Emacs loading .eln file in place of .elc
> automatically setting `load-no-native' to t.
>
>> The second reason is that the initial compilation takes quite a lot of
>> CPU. I wouldn't like that to happen while my laptop where in battery
>> power, for instance. At the moment, it's possible to drop the number of
>> jobs that native-comp uses. I don't think that there is a way to drop
>> this to zero at the moment. Probably, we need that.
>
> Setting `comp-deferred-compilation' to nil should stop any automatic
> attempt of compiling asycronousy.
>
> Maybe these two settings are already covering most of these needs?
Yes, I think so, but it's probably an issue of thinking about the user
interface. Having this a bit more consistent (i.e. a single name space,
rather than two (native-comp and comp) and none (i.e. load-no-native))
and perhaps a single option that does both ("comp-disable").
Just thoughts.
Phil
Phillip Lord <phillip.lord@russet.org.uk> writes:
> Andrea Corallo <akrl@sdf.org> writes:
>
>>> The first is because it's broken or providing buggy behaviour. In the
>>> extreme, and if I understand the build process fully, I guess this is
>>> impossible to switch of in a binary distribution, because some
>>> native-comp happens before dumping. Other than that, we can "switch off"
>>> native-comp by just deleting the *eln files, right, or otherwise
>>> preventing their loading. In practice, I think this is a minor
>>> motivation; if we discover bugs in the native-comp, they would be fixed
>>> by making another release.
>>
>> Hi Phillip,
>>
>> ATM we can prevent Emacs loading .eln file in place of .elc
>> automatically setting `load-no-native' to t.
>>
>>> The second reason is that the initial compilation takes quite a lot of
>>> CPU. I wouldn't like that to happen while my laptop where in battery
>>> power, for instance. At the moment, it's possible to drop the number of
>>> jobs that native-comp uses. I don't think that there is a way to drop
>>> this to zero at the moment. Probably, we need that.
>>
>> Setting `comp-deferred-compilation' to nil should stop any automatic
>> attempt of compiling asycronousy.
>>
>> Maybe these two settings are already covering most of these needs?
>
>
> Yes, I think so, but it's probably an issue of thinking about the user
> interface. Having this a bit more consistent (i.e. a single name space,
> rather than two (native-comp and comp) and none (i.e. load-no-native))
> and perhaps a single option that does both ("comp-disable").
>
> Just thoughts.
Welcome thoughts from my side.
I did my best to fit current name conventions but indeed this might be
not the best arrangement, especially looking at it as single UI
interface towards the "native compiler" seen as single entity.
Looking forward to discuss and tackle this in the code review.
Andrea
Eli Zaretskii <eliz@gnu.org> writes: >> From: Phillip Lord <phillip.lord@russet.org.uk> >> Cc: emacs-devel@gnu.org >> Date: Mon, 11 Jan 2021 09:59:29 +0000 >> >> > It's your call. I just envision many users to expect native-comp >> > support and asking why isn't it there. >> >> No, you misunderstand me. I would plan to make Emacs-28 >> --with-nativecomp. But if I am doing this for the windows build, it is >> reasonable to assume that other packagers will be doing for their >> builds. > > Not necessarily: on platforms other than MS-Windows, if Emacs was > linked against some optional library, that library must be installed > for Emacs to run without crashing. > > So on MS-Windows you could build Emacs with all the possible > extensions, and let users decide which ones they want to install and > use. The installer version could support that kind of thing, but I don't want to provide too many options for the sake of confusion. The sort of people who do not want features can probably build Emacs for themselves. > >> > The gzip thing is again your call (having gzip in the package is no >> > big deal, IMO). Dbus is not really useful on MS-Windows, so IMO it >> > makes no sense building with it. >> >> We've been down this road before. > > Did we reach any conclusions? Yes, we concluded that it was hard to know where to stop! > >> I could add gzip and I am happy to consider this. But what other >> tools do we add. What about aspell (or equivalent), so we can spell >> check? Git so we can version? If I add them, then we get more >> "out-of-the-box" behaviour. But where do we stop. > > We stop where you decide to stop. (And the utility of aspell is much > smaller than that of gzip, btw. And the complexity of aspell > installation is OTOH significantly greater.) There's no need for any > consistency here. "Consistency is the hobgoblin", and all that. I think that this is wrong. Gzip is minimal use on Windows, except for info files, and the total install file size is 18Mb vs 5Mb. Compare to aspell (or hunspell) for the people installing Emacs to use Org mode? Or git for those who want to use magit. I think it would be even better (i.e. potentially quicker!) if we could access these as libraries. Still, if the number of total downloads has decreased, adding a set of command line utilities as extra options in the installer might be worth considering. I will cogitate. Phil
Thank you much for working on this!
On Sun, Jan 10, 2021 at 9:43 AM Phillip Lord <phillip.lord@russet.org.uk> wrote:
>
> I have pushed some binaries up to alpha in case anyone wants to give
> them a trial.
>
I found a problem and thought I'd been check in case it is not
situated safely between my keyboard and chair Also, the emacsclient
issue I mentioned on r/emacs was a self-inflicted wound; if I run it
with a property shortcut incantation (including -Q) it works great!
When I start emacs using the same configuration I use daily with Emacs
27.1 and Emacs 26.3, I get a backtrace. Staring at this in wonder, it
looks like it could be related to use-package, which features heavily
in my setup.
tia and please do let me know what additional help I might be.
Here's the backtrace:
Debugger entered--Lisp error: (invalid-read-syntax "Invalid byte-code object")
read(get-file-char)
require(use-package-core)
byte-code("\300\301!\210\300\302!\210\300\303!\210\300\304!\210\300\305!\210\306\307\310\311\312$\210\313\314!\207"
[require use-package-core use-package-bind-key use-package-diminish
use-package-delight use-package-ensure autoload
use-package-jump-to-package-form "use-package-jump" nil t provide
use-package] 5)
require(use-package)
(progn (require 'cl) (require 'use-package))
eval((progn (require 'cl) (require 'use-package)) t)
#f(compiled-function (&rest body) "Like `progn', but evaluates the
body at compile time if you're compiling.\nThus, the result of the
body appears to the compiler as a quoted\nconstant. In interpreted
code, this is entirely equivalent to\n`progn', except that the value
of the expression may be (but is\nnot necessarily) computed at load
time if eager macro expansion\nis enabled." #<bytecode
-0x173efc045404a52b>)((require 'cl) (require 'use-package))
(eval-when-compile (require 'cl) (require 'use-package))
eval-buffer(#<buffer *load*-573669> nil
"d:/projects/dotfiles/elisp/init-package-managment...." nil t) ;
Reading at buffer position 924
load-with-code-conversion("d:/projects/dotfiles/elisp/init-package-managment...."
"d:/projects/dotfiles/elisp/init-package-managment...." nil t)
require(init-package-managment)
(let ((pkg (car --dolist-tail--))) (require pkg) (setq
--dolist-tail-- (cdr --dolist-tail--)))
(while --dolist-tail-- (let ((pkg (car --dolist-tail--))) (require
pkg) (setq --dolist-tail-- (cdr --dolist-tail--))))
(let ((--dolist-tail-- '(init-local init-package-managment
init-editor init-org init-elisp init-projectile init-vc init-company
init-fun init-bindkey init-theme))) (while --dolist-tail-- (let ((pkg
(car --dolist-tail--))) (require pkg) (setq --dolist-tail-- (cdr
--dolist-tail--)))))
eval-buffer(#<buffer *load*-904898> nil
"d:/projects/dotfiles/elisp/init.el" nil t) ; Reading at buffer
position 3546
load-with-code-conversion("d:/projects/dotfiles/elisp/init.el"
"d:/projects/dotfiles/elisp/init.el" nil nil)
load("d:/projects/dotfiles/elisp/init")
eval-buffer(#<buffer *load*> nil "c:/Users/corwi/.emacs" nil t) ;
Reading at buffer position 414
load-with-code-conversion("c:/Users/corwi/.emacs" "c:/Users/corwi/.emacs" t t)
load("~/.emacs" noerror nomessage)
startup--load-user-init-file(#f(compiled-function () #<bytecode
0x5a5c7670ff81e72>) #f(compiled-function () #<bytecode
0xc085514c3215d0c>) t)
command-line()
normal-top-level()
Regards,
Corwin
corwin.bru.st
Can you do a compare and contrast with this snapshot version? https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-28/emacs-28.0.50-snapshot-2021-01-04-x86_64.zip This was built at about the same time as the dll-only version. I doubt that this problem is caused by the new package, but this will tell us for sure. Phil Corwin Brust <corwin@bru.st> writes: > Thank you much for working on this! > > On Sun, Jan 10, 2021 at 9:43 AM Phillip Lord > <phillip.lord@russet.org.uk> wrote: >> >> I have pushed some binaries up to alpha in case anyone wants to give >> them a trial. >> > > I found a problem and thought I'd been check in case it is not > situated safely between my keyboard and chair Also, the emacsclient > issue I mentioned on r/emacs was a self-inflicted wound; if I run it > with a property shortcut incantation (including -Q) it works great! > > When I start emacs using the same configuration I use daily with Emacs > 27.1 and Emacs 26.3, I get a backtrace. Staring at this in wonder, it > looks like it could be related to use-package, which features heavily > in my setup. > > tia and please do let me know what additional help I might be. > > Here's the backtrace: > > Debugger entered--Lisp error: (invalid-read-syntax "Invalid byte-code > object") > read(get-file-char) > require(use-package-core) > byte-code("\300\301!\210\300\302!\210\300\303!\210\300\304!\210\300\305!\210\306\307\310\311\312$\210\313\314!\207" > [require use-package-core use-package-bind-key use-package-diminish > use-package-delight use-package-ensure autoload > use-package-jump-to-package-form "use-package-jump" nil t provide > use-package] 5) > require(use-package) > (progn (require 'cl) (require 'use-package)) > eval((progn (require 'cl) (require 'use-package)) t) > #f(compiled-function (&rest body) "Like `progn', but evaluates the > body at compile time if you're compiling.\nThus, the result of the > body appears to the compiler as a quoted\nconstant. In interpreted > code, this is entirely equivalent to\n`progn', except that the value > of the expression may be (but is\nnot necessarily) computed at load > time if eager macro expansion\nis enabled." #<bytecode > -0x173efc045404a52b>)((require 'cl) (require 'use-package)) > (eval-when-compile (require 'cl) (require 'use-package)) > eval-buffer(#<buffer *load*-573669> nil > "d:/projects/dotfiles/elisp/init-package-managment...." nil t) ; > Reading at buffer position 924 > load-with-code-conversion("d:/projects/dotfiles/elisp/init-package-managment...." > "d:/projects/dotfiles/elisp/init-package-managment...." nil t) > require(init-package-managment) > (let ((pkg (car --dolist-tail--))) (require pkg) (setq > --dolist-tail-- (cdr --dolist-tail--))) > (while --dolist-tail-- (let ((pkg (car --dolist-tail--))) (require > pkg) (setq --dolist-tail-- (cdr --dolist-tail--)))) > (let ((--dolist-tail-- '(init-local init-package-managment > init-editor init-org init-elisp init-projectile init-vc init-company > init-fun init-bindkey init-theme))) (while --dolist-tail-- (let ((pkg > (car --dolist-tail--))) (require pkg) (setq --dolist-tail-- (cdr > --dolist-tail--))))) > eval-buffer(#<buffer *load*-904898> nil > "d:/projects/dotfiles/elisp/init.el" nil t) ; Reading at buffer > position 3546 > load-with-code-conversion("d:/projects/dotfiles/elisp/init.el" > "d:/projects/dotfiles/elisp/init.el" nil nil) > load("d:/projects/dotfiles/elisp/init") > eval-buffer(#<buffer *load*> nil "c:/Users/corwi/.emacs" nil t) ; > Reading at buffer position 414 > load-with-code-conversion("c:/Users/corwi/.emacs" > "c:/Users/corwi/.emacs" t t) > load("~/.emacs" noerror nomessage) > startup--load-user-init-file(#f(compiled-function () #<bytecode > 0x5a5c7670ff81e72>) #f(compiled-function () #<bytecode > 0xc085514c3215d0c>) t) > command-line() > normal-top-level() > > > Regards, > Corwin > corwin.bru.st
You are right! On Tue, Jan 12, 2021 at 3:48 AM Phillip Lord <phillip.lord@russet.org.uk> wrote: > Can you do a compare and contrast with this snapshot version? > > https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-28/emacs-28.0.50-snapshot-2021-01-04-x86_64.zip > > This was built at about the same time as the dll-only version. I doubt > that this problem is caused by the new package, but this will tell us > for sure. Same backtrace, at a glance. Open a new bug for this, then? Debugger entered--Lisp error: (invalid-read-syntax "Invalid byte-code object") read(get-file-char) require(use-package-core) byte-code("\300\301!\210\300\302!\210\300\303!\210\300\304!\210\300\305!\210\306\307\310\311\312$\210\313\314!\207" [require use-package-core use-package-bind-key use-package-diminish use-package-delight use-package-ensure autoload use-package-jump-to-package-form "use-package-jump" nil t provide use-package] 5) require(use-package) (progn (require 'cl) (require 'use-package)) eval((progn (require 'cl) (require 'use-package)) t) #f(compiled-function (&rest body) "Like `progn', but evaluates the body at compile time if you're compiling.\nThus, the result of the body appears to the compiler as a quoted\nconstant. In interpreted code, this is entirely equivalent to\n`progn', except that the value of the expression may be (but is\nnot necessarily) computed at load time if eager macro expansion\nis enabled." #<bytecode -0x173efc045404914c>)((require 'cl) (require 'use-package)) (eval-when-compile (require 'cl) (require 'use-package)) eval-buffer(#<buffer *load*-275982> nil "d:/projects/dotfiles/elisp/init-package-managment...." nil t) ; Reading at buffer position 924 load-with-code-conversion("d:/projects/dotfiles/elisp/init-package-managment...." "d:/projects/dotfiles/elisp/init-package-managment...." nil t) require(init-package-managment) (let ((pkg (car --dolist-tail--))) (require pkg) (setq --dolist-tail-- (cdr --dolist-tail--))) (while --dolist-tail-- (let ((pkg (car --dolist-tail--))) (require pkg) (setq --dolist-tail-- (cdr --dolist-tail--)))) (let ((--dolist-tail-- '(init-local init-package-managment init-editor init-org init-elisp init-projectile init-vc init-company init-fun init-bindkey init-theme))) (while --dolist-tail-- (let ((pkg (car --dolist-tail--))) (require pkg) (setq --dolist-tail-- (cdr --dolist-tail--))))) eval-buffer(#<buffer *load*-477369> nil "d:/projects/dotfiles/elisp/init.el" nil t) ; Reading at buffer position 3546 load-with-code-conversion("d:/projects/dotfiles/elisp/init.el" "d:/projects/dotfiles/elisp/init.el" nil nil) load("d:/projects/dotfiles/elisp/init") eval-buffer(#<buffer *load*> nil "c:/Users/corwi/.emacs" nil t) ; Reading at buffer position 414 load-with-code-conversion("c:/Users/corwi/.emacs" "c:/Users/corwi/.emacs" t t) load("~/.emacs" noerror nomessage) startup--load-user-init-file(#f(compiled-function () #<bytecode 0x5a5c782f8c782d2>) #f(compiled-function () #<bytecode 0xc085514c3215d0c>) t) command-line() normal-top-level() > > Regards, > > Corwin > > corwin.bru.st
Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> The gzip thing is again your call (having gzip in the package is no
>> big deal, IMO).
>
> BTW, we don't really need a `gzip` executable as long as we have the
> `zlib-decompress-region` function, right?
Following on from this discussion, I have had a thought about a better
way of producing a more function windows download and have come up with
this idea, which is just to install msys inside the Emacs install
directory.
The plan would be during the installation run the newly installed
Emacs. This would download, unpack and configure msys2, as well as
install a variety of related packages (git, find, gzip, hunspell,
aspell, what ever). For those using the zip package of Emacs, they would
achieve the same thing by running a command.
I have given this a quick bash. It could be achieved without bloating
the Emacs download; I'd have to had xz to the Emacs installer to allow
me to unpack msys, but otherwise it would have no implications. I would
not need to host either msys2 nor its source on the Gnu FTP because,
this would all come from msys2. The installation process would be Emacs
driven, which would be easier for me to write. Emacs would remain using
the DLLs we ship with it and so stable, while the underlying msys2 would
be independently updateable. And, to all intents and purposes, it would
look like a normal windows app; packaged, bundled and installed in
directory.
What I have so far:
1) w32-msys-install
which updates msys and installs git as well as adds a loader to site-start.el
2) w32-msys-site-start.el
which updates the path and so
To be fully functional 1) would need to download msys and unpack it and
I'd need to hook it into the installer.
It would solve the question of what should I package -- I'd add some
basic utilities, and thereafter, the user could add what ever they
wanted.
Thoughts? Does this seem worth following up?
Phil
Phillip Lord <phillip.lord@russet.org.uk> writes:
> Following on from this discussion, I have had a thought about a better
> way of producing a more function windows download and have come up with
> this idea, which is just to install msys inside the Emacs install
> directory.
Works for me, as long as it detects my current msys install and does not
modify it without asking my permission.
--
-- Stephe
> From: Phillip Lord <phillip.lord@russet.org.uk>
> Date: Wed, 20 Jan 2021 19:29:58 +0000
>
> The plan would be during the installation run the newly installed
> Emacs. This would download, unpack and configure msys2, as well as
> install a variety of related packages (git, find, gzip, hunspell,
> aspell, what ever). For those using the zip package of Emacs, they would
> achieve the same thing by running a command.
Could you please elaborate on how will this work for users? E.g., how
about describing the steps a user who installs Emacs should do to
install these additions? I don't think I have a clear picture of what
you intend to do, and the files you committed are just the starting
point, so they don't answer this question.
Btw, I don't think I understand why you insist on using 'concat' to
generate files names from 2 components: that's what expand-file-name
is for, and it does that better.
One other not about the code you committed is that modifying PATH from
within Emacs is not generally a good idea, it has some subtleties.
Stephen Leake <stephen_leake@stephe-leake.org> writes:
> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> Following on from this discussion, I have had a thought about a better
>> way of producing a more function windows download and have come up with
>> this idea, which is just to install msys inside the Emacs install
>> directory.
>
> Works for me, as long as it detects my current msys install and does not
> modify it without asking my permission.
I have tested this; you can have two msys installations on a machine and
they just ignore each other. In practice, someone who already has an
msys installation wouldn't want this, although there is no reason why
any technology that we put in place for managing an internal msys
shouldn't also work for a preinstalled one.
For example, there might be a command "install recommended packages"
which would just update your local msys an add those packages that work
with Emacs. Likewise, if there is an expectation that Emacs will be used
in this way, we might modify Emacs to work "properly"; for example, M-x
find tends to break on Windows because it finds the wrong "find"
command.
Phil
Eli Zaretskii <eliz@gnu.org> writes: >> From: Phillip Lord <phillip.lord@russet.org.uk> >> Date: Wed, 20 Jan 2021 19:29:58 +0000 >> >> The plan would be during the installation run the newly installed >> Emacs. This would download, unpack and configure msys2, as well as >> install a variety of related packages (git, find, gzip, hunspell, >> aspell, what ever). For those using the zip package of Emacs, they would >> achieve the same thing by running a command. > > Could you please elaborate on how will this work for users? E.g., how > about describing the steps a user who installs Emacs should do to > install these additions? I don't think I have a clear picture of what > you intend to do, and the files you committed are just the starting > point, so they don't answer this question. Considering just the .exe installer, at the end of the installation process, we would run Emacs with menu bar, scroll bar and everything switch off and run the command "w32-msys-internal-install-dialog". Emacs would say "do you want do install a handy collection of tools". User says "y", then Emacs downloads the latest msys2-date.xz, then unpacks it with xz. Emacs will run msys2.exe to initialize (hopefully without poping up windows if this will work). Then Emacs will run pacman to update and install. The user will see Emacs saying Would you like to install msys, and a handy set of tools (Y/n) Installing msys...done Updating msys...done Installing git...done When it's finished, the user starts Emacs and does vc-dir and it works, instead of seeing "git not found" errors. For those using the zip file, the user would have to launch "M-x w32-msys-install" manually. At a later date, I might add some cuter things. So, for example we might install: https://github.com/brotzeit/arch-packer or equivalent. We might create "internal-msys" package on ELPA which contains a standard set of packages and Emacs fixes to make it work, so it would be freely updatable. At which point the w32-msys-install command would install the ELPA package before doing anything else. There are also issues that when Emacs is removed msys (and any configuration) would get blitzed, so I'd have to think about how to persist msys configuration between two installs of Emacs. Currently the file organisation looks like this: c:/Program Files/Emacs/emacs-28.1/bin c:/Program Files/Emacs/emacs-28.1/share c:/Program Files/Emacs/emacs-28.1/msys (etc) but c:/Program Files/Emacs/emacs-28.1/bin c:/Program Files/Emacs/emacs-28.1/share c:/Program Files/Emacs/msys would mean that installing Emacs-28.2 would work naturally. There are also some issues about user persmissions; I only ever run Windows as administrator, so there will be some thought given as to how this would work. Neither Emacs or msys require this AFAICT. > Btw, I don't think I understand why you insist on using 'concat' to > generate files names from 2 components: that's what expand-file-name > is for, and it does that better. Because in my own code, I tend to use f.el and I have got used to having a clean, consistent and comprehensive API, so I tend to forget the underlying Emacs primitives. > One other not about the code you committed is that modifying PATH from > within Emacs is not generally a good idea, it has some subtleties. Any alternative that achieves the same thing? Phil
Stephen Leake <stephen_leake@stephe-leake.org> writes:
> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> Following on from this discussion, I have had a thought about a better
>> way of producing a more function windows download and have come up with
>> this idea, which is just to install msys inside the Emacs install
>> directory.
>
> Works for me, as long as it detects my current msys install and does not
> modify it without asking my permission.
Actually, a better way to proceed in this direction is to update the
current emacs mingw64 package; it is currently at 26.
--
-- Stephe
On 2021-01-21 18:22, Stephen Leake wrote: > Stephen Leake <stephen_leake@stephe-leake.org> writes: > >> Phillip Lord <phillip.lord@russet.org.uk> writes: >> >>> Following on from this discussion, I have had a thought about a >>> better >>> way of producing a more function windows download and have come up >>> with >>> this idea, which is just to install msys inside the Emacs install >>> directory. >> >> Works for me, as long as it detects my current msys install and does >> not >> modify it without asking my permission. > > Actually, a better way to proceed in this direction is to update the > current emacs mingw64 package; it is currently at 26. Is it? https://packages.msys2.org/package/mingw-w64-x86_64-emacs?repo=mingw64 report 27.1 Phil
Stephen Leake <stephen_leake@stephe-leake.org> writes:
> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>
>> Phillip Lord <phillip.lord@russet.org.uk> writes:
>>
>>> Following on from this discussion, I have had a thought about a better
>>> way of producing a more function windows download and have come up with
>>> this idea, which is just to install msys inside the Emacs install
>>> directory.
>>
>> Works for me, as long as it detects my current msys install and does not
>> modify it without asking my permission.
>
> Actually, a better way to proceed in this direction is to update the
> current emacs mingw64 package; it is currently at 26.
The MSYS2/MinGW64 Emacs package? It was updated to 27.1 one day after
the release.
> From: Phillip Lord <phillip.lord@russet.org.uk> > Cc: emacs-devel@gnu.org > Date: Thu, 21 Jan 2021 16:44:02 +0000 > > > Could you please elaborate on how will this work for users? E.g., how > > about describing the steps a user who installs Emacs should do to > > install these additions? I don't think I have a clear picture of what > > you intend to do, and the files you committed are just the starting > > point, so they don't answer this question. > > Considering just the .exe installer, at the end of the installation > process, we would run Emacs with menu bar, scroll bar and everything > switch off and run the command "w32-msys-internal-install-dialog". "We" being the installation process? IOW, this will start automatically at the end of the normal installation? > Emacs would say "do you want do install a handy collection of > tools". User says "y", then Emacs downloads the latest msys2-date.xz, > then unpacks it with xz. Emacs will run msys2.exe to initialize > (hopefully without poping up windows if this will work). Then Emacs will > run pacman to update and install. The user will see Emacs saying > > Would you like to install msys, and a handy set of tools (Y/n) > Installing msys...done > Updating msys...done > Installing git...done First, why Git? This is a "normal" user, not an Emacs developer, right? Git is a huge package (and comes with Vim on top of that), so maybe reconsider that part. Next, by "msys" you mean all those non-native programs that depend on msys-1.0.dll? That's again meant for MinGW developers, not "normal" users. And finally, which packages will pacman install? are you going to provide some list of packages, and if so, what will be there? (Btw, pacman can ask questions and prompt the user for confirmation, but the way you invoke it in w32-msys-run doesn't seem to be prepared for such interaction?) One other aspect that bothers me is that if the user already has some parts of MSYS/MinGW64 installed, they will now have two places with DLLs, which is a step towards "DLL hell". > We might create "internal-msys" package on ELPA which > contains a standard set of packages and Emacs fixes to make it work, so > it would be freely updatable. At which point the w32-msys-install > command would install the ELPA package before doing anything else. Is it really possible for us to distribute MSYS, given its license? > > Btw, I don't think I understand why you insist on using 'concat' to > > generate files names from 2 components: that's what expand-file-name > > is for, and it does that better. > > Because in my own code, I tend to use f.el and I have got used to having > a clean, consistent and comprehensive API, so I tend to forget the > underlying Emacs primitives. Then I think f.el has a big problem. > > One other not about the code you committed is that modifying PATH from > > within Emacs is not generally a good idea, it has some subtleties. > > Any alternative that achieves the same thing? I think the only good idea here is to tell the user to amend PATH by adding such-and-such directories to it. I don't like installers that futz with my PATH, and would hate it if Emacs did that to others. It's very easy to get that wrong, especially on Windows.
Eli Zaretskii <eliz@gnu.org> writes: >> Considering just the .exe installer, at the end of the installation >> process, we would run Emacs with menu bar, scroll bar and everything >> switch off and run the command "w32-msys-internal-install-dialog". > > "We" being the installation process? IOW, this will start > automatically at the end of the normal installation? Yep. >> Emacs would say "do you want do install a handy collection of >> tools". User says "y", then Emacs downloads the latest msys2-date.xz, >> then unpacks it with xz. Emacs will run msys2.exe to initialize >> (hopefully without poping up windows if this will work). Then Emacs will >> run pacman to update and install. The user will see Emacs saying >> >> Would you like to install msys, and a handy set of tools (Y/n) >> Installing msys...done >> Updating msys...done >> Installing git...done > > First, why Git? This is a "normal" user, not an Emacs developer, > right? Git is a huge package (and comes with Vim on top of that), so > maybe reconsider that part. That's open for question. I'd have to think about the packages I would want to use. However, for me, git is one of the first commands that Emacs complains about the absence of; this is because I use Emacs on my windows build machine, of course, which contains the Emacs git repo. But, also because most IDEs come with the ability to interact with git off the shelf. I think Windows users will expect it. Yep, it's big. I get this before installation: 319 MB (335,214,637 bytes) and this after installation. 826 MB (866,624,644 bytes) My guess is that it's the co-install of perl that is causing this issue rather than vim. But, openssh comes for free with this. And "huge" is a relative term. An installation under 1Gb seems reasonable to me. > Next, by "msys" you mean all those non-native programs that depend on > msys-1.0.dll? That's again meant for MinGW developers, not "normal" > users. Yes. Because it's got all the packages and tools and as far as I can see, they work with Emacs. > And finally, which packages will pacman install? are you going to > provide some list of packages, and if so, what will be there? Yes. Initially hard coded into the Emacs install, we might turn the list of msys packages into an ELPA package. That way, we could update the list of tools supported, without updating the Emacs distribution. > (Btw, pacman can ask questions and prompt the user for confirmation, > but the way you invoke it in w32-msys-run doesn't seem to be prepared > for such interaction?) That's why I use "--no-confirm". I'm looking for the minimum here that does something usable. > One other aspect that bothers me is that if the user already has some > parts of MSYS/MinGW64 installed, they will now have two places with > DLLs, which is a step towards "DLL hell". Yes, it is. And indeed, the DLLs distributed in the Emacs distribution might well get duplicated in the msys install. But they would be in one place. My presumption is that people who use msys already would not want this, and it would not be forced upon them. >> We might create "internal-msys" package on ELPA which >> contains a standard set of packages and Emacs fixes to make it work, so >> it would be freely updatable. At which point the w32-msys-install >> command would install the ELPA package before doing anything else. > > Is it really possible for us to distribute MSYS, given its license? You misunderstand, my poor wording. The ELPA package would not contain msys, but would contain list of msys package names and supporting code. Msys would come from the msys2 repo. >> > Btw, I don't think I understand why you insist on using 'concat' to >> > generate files names from 2 components: that's what expand-file-name >> > is for, and it does that better. >> >> Because in my own code, I tend to use f.el and I have got used to having >> a clean, consistent and comprehensive API, so I tend to forget the >> underlying Emacs primitives. > > Then I think f.el has a big problem. I was being slightly provocative; I shall divert the discussion no further. >> > One other not about the code you committed is that modifying PATH from >> > within Emacs is not generally a good idea, it has some subtleties. >> >> Any alternative that achieves the same thing? > > I think the only good idea here is to tell the user to amend PATH by > adding such-and-such directories to it. I don't like installers that > futz with my PATH, and would hate it if Emacs did that to others. > It's very easy to get that wrong, especially on Windows. There has to be a better way that this. Editing environment variables by hand is so last century. Of course, I do not want to edit the path globally. If I can do it in Emacs, then I would have to think of a different solution. Phil
> From: Phillip Lord <phillip.lord@russet.org.uk> > Cc: emacs-devel@gnu.org > Date: Thu, 21 Jan 2021 21:37:57 +0000 > > Yep, it's big. I get this before installation: > > 319 MB (335,214,637 bytes) > > and this after installation. > > 826 MB (866,624,644 bytes) > > My guess is that it's the co-install of perl that is causing this issue > rather than vim. It's Vim, and Perl, and Tcl. And at some point I'd expect them to include Python as well, to support those Git commands which rely on it. > But, openssh comes for free with this. And "huge" is a > relative term. An installation under 1Gb seems reasonable to me. I thought this started as an attempt to make the Emacs installation smaller... If the rationale is to provide a full development environment on top of MS-Windows, then indeed the size will be much larger, and you will need to include a lot of packages there. Just be sure to spell this out (including the size, perhaps) when you ask the user whether he or she wants to install that. > > Next, by "msys" you mean all those non-native programs that depend on > > msys-1.0.dll? That's again meant for MinGW developers, not "normal" > > users. > > Yes. Because it's got all the packages and tools and as far as I can > see, they work with Emacs. They mostly work, until they don't. Like with Cygwin, there are subtle incompatibilities, mainly in file names and in communications with subprocesses and response to "signals". Encoding defaults are also different. > > (Btw, pacman can ask questions and prompt the user for confirmation, > > but the way you invoke it in w32-msys-run doesn't seem to be prepared > > for such interaction?) > > That's why I use "--no-confirm". I'm looking for the minimum here that > does something usable. What happens when pacman wants to replace Bash from which it was launched, or update itself or some of its component DLLs? > > I think the only good idea here is to tell the user to amend PATH by > > adding such-and-such directories to it. I don't like installers that > > futz with my PATH, and would hate it if Emacs did that to others. > > It's very easy to get that wrong, especially on Windows. > > There has to be a better way that this. The only way I know of is to distribute a program that writes into the Registry. Don't forget that there are system-wide variables and variables specific to the current user. And some systems have the former locked down and the latter requires a UAC elevation. Good luck (you will need it) with successfully negotiating all these obstacles.
Eli Zaretskii <eliz@gnu.org> writes: >> From: Phillip Lord <phillip.lord@russet.org.uk> >> Cc: emacs-devel@gnu.org >> Date: Thu, 21 Jan 2021 21:37:57 +0000 >> >> Yep, it's big. I get this before installation: >> >> 319 MB (335,214,637 bytes) >> >> and this after installation. >> >> 826 MB (866,624,644 bytes) >> >> My guess is that it's the co-install of perl that is causing this issue >> rather than vim. > > It's Vim, and Perl, and Tcl. And at some point I'd expect them to > include Python as well, to support those Git commands which rely on > it. Yep, they don't ship a "git-core" without all this baggage unfortunately. >> But, openssh comes for free with this. And "huge" is a >> relative term. An installation under 1Gb seems reasonable to me. > > I thought this started as an attempt to make the Emacs installation > smaller... > > If the rationale is to provide a full development environment on top > of MS-Windows, then indeed the size will be much larger, and you will > need to include a lot of packages there. Just be sure to spell this > out (including the size, perhaps) when you ask the user whether he or > she wants to install that. Yes, a small install for easy downloads, but with the option for a bigger one. The small install is still important, though, because it will also mean a small upgrade from one emacs to the next. >> > Next, by "msys" you mean all those non-native programs that depend on >> > msys-1.0.dll? That's again meant for MinGW developers, not "normal" >> > users. >> >> Yes. Because it's got all the packages and tools and as far as I can >> see, they work with Emacs. > > They mostly work, until they don't. Like with Cygwin, there are > subtle incompatibilities, mainly in file names and in communications > with subprocesses and response to "signals". Encoding defaults are > also different. That's true for the msys2 commands but not the mingw64 ones? In the end, not all of the tools need for Emacs are in mingw64 part of msys2 and msys2 comes with pacman. >> > (Btw, pacman can ask questions and prompt the user for confirmation, >> > but the way you invoke it in w32-msys-run doesn't seem to be prepared >> > for such interaction?) >> >> That's why I use "--no-confirm". I'm looking for the minimum here that >> does something usable. > > What happens when pacman wants to replace Bash from which it was > launched, or update itself or some of its component DLLs? That happens straight away on first launch. I don't know yet, but am working on it. > >> > I think the only good idea here is to tell the user to amend PATH by >> > adding such-and-such directories to it. I don't like installers that >> > futz with my PATH, and would hate it if Emacs did that to others. >> > It's very easy to get that wrong, especially on Windows. >> >> There has to be a better way that this. > > The only way I know of is to distribute a program that writes into the > Registry. Don't forget that there are system-wide variables and > variables specific to the current user. And some systems have the > former locked down and the latter requires a UAC elevation. Good luck > (you will need it) with successfully negotiating all these obstacles. Would it be easier to have Emacs allow me to successfully update PATH during run? Phil
> From: Phillip Lord <phillip.lord@russet.org.uk> > Cc: emacs-devel@gnu.org > Date: Fri, 22 Jan 2021 16:14:33 +0000 > > >> > Next, by "msys" you mean all those non-native programs that depend on > >> > msys-1.0.dll? That's again meant for MinGW developers, not "normal" > >> > users. > >> > >> Yes. Because it's got all the packages and tools and as far as I can > >> see, they work with Emacs. > > > > They mostly work, until they don't. Like with Cygwin, there are > > subtle incompatibilities, mainly in file names and in communications > > with subprocesses and response to "signals". Encoding defaults are > > also different. > > That's true for the msys2 commands but not the mingw64 ones? Yes. That's why I asked about msys-1.0.dll: the programs that depend on that aren't mingw64 (native) programs. > > The only way I know of is to distribute a program that writes into the > > Registry. Don't forget that there are system-wide variables and > > variables specific to the current user. And some systems have the > > former locked down and the latter requires a UAC elevation. Good luck > > (you will need it) with successfully negotiating all these obstacles. > > Would it be easier to have Emacs allow me to successfully update PATH > during run? Update how?
phillip.lord@russet.org.uk writes:
> On 2021-01-21 18:22, Stephen Leake wrote:
>> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>>
>>> Phillip Lord <phillip.lord@russet.org.uk> writes:
>>>
>>>> Following on from this discussion, I have had a thought about a
>>>> better
>>>> way of producing a more function windows download and have come up
>>>> with
>>>> this idea, which is just to install msys inside the Emacs install
>>>> directory.
>>> Works for me, as long as it detects my current msys install and
>>> does not
>>> modify it without asking my permission.
>> Actually, a better way to proceed in this direction is to update the
>> current emacs mingw64 package; it is currently at 26.
>
> Is it?
>
> https://packages.msys2.org/package/mingw-w64-x86_64-emacs?repo=mingw64
>
> report 27.1
ah, my local mingw64 cache is out of date. Sorry.
--
-- Stephe
Eli Zaretskii <eliz@gnu.org> writes: >> > They mostly work, until they don't. Like with Cygwin, there are >> > subtle incompatibilities, mainly in file names and in communications >> > with subprocesses and response to "signals". Encoding defaults are >> > also different. >> >> That's true for the msys2 commands but not the mingw64 ones? > > Yes. That's why I asked about msys-1.0.dll: the programs that depend > on that aren't mingw64 (native) programs. But, mingw64 does not have all the packages I need. How do people use Emacs on windows? I mean, do they install find, ls, git, aspell and all the rest by hand? It's been a long time since I have done it. >> > The only way I know of is to distribute a program that writes into the >> > Registry. Don't forget that there are system-wide variables and >> > variables specific to the current user. And some systems have the >> > former locked down and the latter requires a UAC elevation. Good luck >> > (you will need it) with successfully negotiating all these obstacles. >> >> Would it be easier to have Emacs allow me to successfully update PATH >> during run? > > Update how? Using some magic that doesn't exist at the current time. I mean a way like did (by fiddling with setenv) that doesn't suffer the problems that it causes. Phil
Phillip Lord <phillip.lord@russet.org.uk> writes: > Eli Zaretskii <eliz@gnu.org> writes: > >>> > They mostly work, until they don't. Like with Cygwin, there are >>> > subtle incompatibilities, mainly in file names and in communications >>> > with subprocesses and response to "signals". Encoding defaults are >>> > also different. >>> >>> That's true for the msys2 commands but not the mingw64 ones? >> >> Yes. That's why I asked about msys-1.0.dll: the programs that depend >> on that aren't mingw64 (native) programs. > > > But, mingw64 does not have all the packages I need. Ok, here goes a data point: > How do people use Emacs on windows? I install MSYS2 and execute a shell script that installs this packages: universal-ctags-git ag putty-ssh aspell aspell-en aspell-es diffutils (the scripts prepends the names listed above with the corresponding mingw-w64-i686/x86_64 prefix). Then I install emacs with `pacman`, either from MSYS2 binary repository or from the packages I build. > I mean, do they install find, I don't use it. I don't use it on GNU/Linux either (I work 99% of the time on GNU/Linux). > ls, Is it necessary? > git, Git for Windows. > aspell See above. > and all the rest by hand? On my case, only Git for Windows and MSYS2 are installed by hand. BTW, Git for Windows is also based on MSYS2: it installs the required pieces that are required for running the parts of git that still depend on POSIX. It theory, instead of Git for Windows I could also run MSYS2's own git.exe setting magit-git-executable and vc-git-program, without adding MSYS2 /bin directory to PATH, but last time I tried it was somewhat slower than Git for Windows, which is slow enough itself. Also, the later comes with some goodies built-in, such as git-svn. MSYS2 git package requires about 30 MB, while Git for Windows 240. I execute runemacs.exe from wathever/msys64/mingw64/bin. emacs.el sets PATH and exec-path: (defun anade-a-path (path) (setenv "PATH" (concat (getenv "PATH") path-separator dir)) (setq exec-path (append exec-path (list dir)))) (anade-a-path (file-name-directory (car command-line-args))) With this I can execute programs on emacs.exe directory (that is, whatever/msys64/mingw64/bin) with no problems.
Óscar Fuentes <ofv@wanadoo.es> writes: > Phillip Lord <phillip.lord@russet.org.uk> writes: > >> But, mingw64 does not have all the packages I need. > > Ok, here goes a data point: > >> How do people use Emacs on windows? > > I install MSYS2 and execute a shell script that installs this > packages: > > universal-ctags-git ag putty-ssh aspell aspell-en aspell-es diffutils > > (the scripts prepends the names listed above with the corresponding > mingw-w64-i686/x86_64 prefix). Then I install emacs with `pacman`, > either from MSYS2 binary repository or from the packages I build. In your case, you are using msys effectively as an installer, to install mingw64 packages. Likewise, I presume, with Emacs? This seems to come in both an msys2 and mingw64 package. Any reason for putty rather than openssh? >> I mean, do they install find, > > I don't use it. I don't use it on GNU/Linux either (I work 99% of the > time on GNU/Linux). Oh, I'm a find junkie. >> ls, > > Is it necessary? Not sure. I thought dired used it, at least if it's there but maybe not. >> git, > > Git for Windows. > >> aspell > > See above. > >> and all the rest by hand? > > On my case, only Git for Windows and MSYS2 are installed by hand. > > BTW, Git for Windows is also based on MSYS2: it installs the required > pieces that are required for running the parts of git that still depend > on POSIX. > > It theory, instead of Git for Windows I could also run MSYS2's own > git.exe setting magit-git-executable and vc-git-program, without adding > MSYS2 /bin directory to PATH, but last time I tried it was somewhat > slower than Git for Windows, which is slow enough itself. Also, the > later comes with some goodies built-in, such as git-svn. MSYS2 git > package requires about 30 MB, while Git for Windows 240. Yes, I hear that magit is difficult to use on Windows. I don't know if that is fixable within Emacs, or if it is fundamental to Windows. > I execute runemacs.exe from wathever/msys64/mingw64/bin. emacs.el sets > PATH and exec-path: > > (defun anade-a-path (path) > (setenv "PATH" (concat (getenv "PATH") path-separator dir)) > (setq exec-path (append exec-path (list dir)))) > > (anade-a-path (file-name-directory (car command-line-args))) > > With this I can execute programs on emacs.exe directory (that is, > whatever/msys64/mingw64/bin) with no problems. That's a good data point! Thank you. Phil
Phillip Lord <phillip.lord@russet.org.uk> writes: > Óscar Fuentes <ofv@wanadoo.es> writes: > >> Phillip Lord <phillip.lord@russet.org.uk> writes: >> >>> But, mingw64 does not have all the packages I need. >> >> Ok, here goes a data point: >> >>> How do people use Emacs on windows? >> >> I install MSYS2 and execute a shell script that installs this >> packages: >> >> universal-ctags-git ag putty-ssh aspell aspell-en aspell-es diffutils >> >> (the scripts prepends the names listed above with the corresponding >> mingw-w64-i686/x86_64 prefix). Then I install emacs with `pacman`, >> either from MSYS2 binary repository or from the packages I build. > > In your case, you are using msys effectively as an installer, to install > mingw64 packages. Likewise, I presume, with Emacs? This seems to come in > both an msys2 and mingw64 package. > > Any reason for putty rather than openssh? There is no mingw64 package of openssh and putty is "made for Windows", which means that everything works out of the box (I'm not talking about Tramp, which I don't use.) >>> I mean, do they install find, >> >> I don't use it. I don't use it on GNU/Linux either (I work 99% of the >> time on GNU/Linux). > > Oh, I'm a find junkie. I also use find quite often while using a shell. For Emacs, ag (the silver searcher) makes rgrep & co. irrelevant and my Dired usage is quite basic. >>> ls, >> >> Is it necessary? > > Not sure. I thought dired used it, at least if it's there but maybe > not. If there is no ls available, Emacs uses its own facilities. That applies to Eshell, Dired, etc. >>> git, >> >> Git for Windows. >> >>> aspell >> >> See above. >> >>> and all the rest by hand? >> >> On my case, only Git for Windows and MSYS2 are installed by hand. >> >> BTW, Git for Windows is also based on MSYS2: it installs the required >> pieces that are required for running the parts of git that still depend >> on POSIX. >> >> It theory, instead of Git for Windows I could also run MSYS2's own >> git.exe setting magit-git-executable and vc-git-program, without adding >> MSYS2 /bin directory to PATH, but last time I tried it was somewhat >> slower than Git for Windows, which is slow enough itself. Also, the >> later comes with some goodies built-in, such as git-svn. MSYS2 git >> package requires about 30 MB, while Git for Windows 240. > > Yes, I hear that magit is difficult to use on Windows. No, not difficult, just slow. > I don't know if that is fixable within Emacs, or if it is fundamental > to Windows. Launching processes is slower in Windows than in GNU/Linux, plus Git is heavily optimized for GNU/Linux, plus (and this is not really a "plus" but a "times") Magit does a lot of git calls, about 50 runs of git whenever the status buffer is updated, last time I checked. >> I execute runemacs.exe from wathever/msys64/mingw64/bin. emacs.el sets >> PATH and exec-path: >> >> (defun anade-a-path (path) >> (setenv "PATH" (concat (getenv "PATH") path-separator dir)) >> (setq exec-path (append exec-path (list dir)))) >> >> (anade-a-path (file-name-directory (car command-line-args))) >> >> With this I can execute programs on emacs.exe directory (that is, >> whatever/msys64/mingw64/bin) with no problems. > > That's a good data point! Thank you. You're welcome.
> From: Phillip Lord <phillip.lord@russet.org.uk> > Cc: emacs-devel@gnu.org > Date: Sun, 24 Jan 2021 22:13:07 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> > They mostly work, until they don't. Like with Cygwin, there are > >> > subtle incompatibilities, mainly in file names and in communications > >> > with subprocesses and response to "signals". Encoding defaults are > >> > also different. > >> > >> That's true for the msys2 commands but not the mingw64 ones? > > > > Yes. That's why I asked about msys-1.0.dll: the programs that depend > > on that aren't mingw64 (native) programs. > > But, mingw64 does not have all the packages I need. Are we talking about you personally, or are we talking about Emacs users? If the former, then MinGW64 are not the only source of good ports of Free Software to Windows, far from that. If they don't have some package, you just go out and look for it elsewhere. If we are talking about Emacs users who will download Emacs from the GNU sites, then I'd say give them only what MinGW64 provides, so that they could use pacman to easily update that. Those who need more will have to find and install whatever they need on their own. Telling them to install MSYS ports instead risks exposing them to subtle problems, so I wouldn't recommend it. > How do people use Emacs on windows? I mean, do they install find, ls, > git, aspell and all the rest by hand? If MinGW doesn't provide those, what else can you do? Me, I ported some of the packages myself (where I found no ports that were good enough or new for me), and installed others where I found good ports. Almost all of my ports are available from the ezwinports site. > >> Would it be easier to have Emacs allow me to successfully update PATH > >> during run? > > > > Update how? > > Using some magic that doesn't exist at the current time. I mean a way > like did (by fiddling with setenv) that doesn't suffer the problems that > it causes. I don't think this magic can exist. But if someone knows, let them speak up.
> From: Phillip Lord <phillip.lord@russet.org.uk> > Date: Sun, 24 Jan 2021 23:34:29 +0000 > Cc: emacs-devel@gnu.org > > Oh, I'm a find junkie. Me too. Which is why I ported Findutils long ago (the available ports were abysmally slow). I have an updatedb job running every week until this day. > >> ls, > > > > Is it necessary? > > Not sure. I thought dired used it, at least if it's there but maybe not. I do use ls, regardless of Dired (which doesn't need it on Windows). That's about the only Coreutils program I ported myself; for the rest I use an old GnuWin32 port, which is quite adequate -- ls was the single exception.
[-- Attachment #1: Type: text/plain, Size: 313 bytes --] On Mon, Jan 25, 2021 at 7:24 AM Eli Zaretskii <eliz@gnu.org> wrote: > > Oh, I'm a find junkie. > > Me too. > Tangent, but: you might be interested in fd-find, which is an implementation of (paraphrasing here) about 80% of find that's usually several times faster. https://github.com/sharkdp/fd Hth, ~Chad [-- Attachment #2: Type: text/html, Size: 752 bytes --]
> From: chad <yandros@gmail.com>
> Date: Mon, 25 Jan 2021 11:49:24 -0800
> Cc: Phillip Lord <phillip.lord@russet.org.uk>, ofv@wanadoo.es,
> EMACS development team <emacs-devel@gnu.org>
>
> Tangent, but: you might be interested in fd-find, which is an implementation of (paraphrasing here) about
> 80% of find that's usually several times faster.
>
> https://github.com/sharkdp/fd
I know about it, but it isn't a 100% functional replacement of 'find',
and uses a confusingly different mini-language on top of that.
On Mon, 25 Jan 2021 at 15:21, Eli Zaretskii <eliz@gnu.org> wrote: > > From: Phillip Lord <phillip.lord@russet.org.uk> > > Eli Zaretskii <eliz@gnu.org> writes: > > >> Would it be easier to have Emacs allow me to successfully update PATH > > >> during run? > > > > > > Update how? > > > > Using some magic that doesn't exist at the current time. I mean a way > > like did (by fiddling with setenv) that doesn't suffer the problems that > > it causes. > > I don't think this magic can exist. But if someone knows, let them > speak up. No magic is needed. Just change exec-path and the PATH environment variable matching values, not forgetting to add Emacs' libexec dir to exec-path. I have code in my init file to select different paths, because different toolchains that I want to run in Emacs require different incompatible paths. It doesn't cause any problems, and everything behaves as you expect. This works for me because I install emacs into the /mingw64/bin directory of MSYS2 (which is what "make install" does by default, in the MSYS2/MINGW64 shell), so emacs can never fail to find its required DLLs at runtime, nor find the wrong ones. (The LoadLibrary function always searches the exe's directory and the System32 directory.) The installer should make sure all the necessary DLLs are in the same directory as emacs.exe, too, for the same reason. When running emacs from the /src directory, features that load DLLs at runtime don't work properly unless the correct DLLs are found on the path, but there's nothing mysterious, magical or unexpected about that. By the way, the MSYS2 builds of Emacs no longer apply any significant patches. There's just one, which is intended to disable ImageMagick, and which is superfluous but harmless. I hope we don't do anything that breaks the ability to build and install Emacs into an existing MSYS2 tree using "./autogen.sh && ./configure && make && make install", as their PKGBUILD script does.
> Tangent, but: you might be interested in fd-find, which is an
> implementation of (paraphrasing here) about 80% of find that's usually
> several times faster.
Any hint *why* it's faster?
Stefan
Richard Copley <rcopley@gmail.com> writes: [snip] > By the way, the MSYS2 builds of Emacs no longer apply any significant > patches. There's just one, which is intended to disable ImageMagick, > and which is superfluous but harmless. That patch is not used. I'm not sure why I didn't delete it. > I hope we don't do anything > that breaks the ability to build and install Emacs into an existing > MSYS2 tree using "./autogen.sh && ./configure && make && make > install", as their PKGBUILD script does. +1, although I don't expect that this is checked regularly by the Emacs developers. Currently my Windows machines are using 27.1 as distributed by MSYS2, but eventually I'll switch to master, building it every few months, so when 28 is released the build should be fine.
[-- Attachment #1: Type: text/plain, Size: 1903 bytes --] On Mon, Jan 25, 2021 at 12:42 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote: > > Tangent, but: you might be interested in fd-find, which is an > > implementation of (paraphrasing here) about 80% of find that's usually > > several times faster. > > Any hint *why* it's faster? > At this point I'm just passing along what I've read on the project pages, but it claims most of the speed comes from leveraging the speed efforts of ripgrep's component pieces (it's rust-based regex and ignore packages). There are two relevant chunks from ripgrep's project page; a blog post with details and benchmarks, and a summary on ripgrep's project page. Blog post: https://blog.burntsushi.net/ripgrep/ Summarizing, ripgrep is fast because: > > - It is built on top of Rust's regex engine > <https://github.com/rust-lang/regex>. Rust's regex engine uses finite > automata, SIMD and aggressive literal optimizations to make searching very > fast. (PCRE2 support can be opted into with the -P/--pcre2 flag.) > > > - Rust's regex library maintains performance with full Unicode support > by building UTF-8 decoding directly into its deterministic finite automaton > engine. > > > - It supports searching with either memory maps or by searching > incrementally with an intermediate buffer. The former is better for single > files and the latter is better for large directories. ripgrep chooses the > best searching strategy for you automatically. > > > - Applies your ignore patterns in .gitignore files using a RegexSet > <https://docs.rs/regex/1/regex/struct.RegexSet.html>. That means a > single file path can be matched against multiple glob patterns > simultaneously. > > > - It uses a lock-free parallel recursive directory iterator, courtesy > of crossbeam <https://docs.rs/crossbeam> and ignore > <https://docs.rs/ignore>. > > Hope that helps, ~Chad [-- Attachment #2: Type: text/html, Size: 5487 bytes --]
On 25.01.2021 22:42, Stefan Monnier wrote:
>> Tangent, but: you might be interested in fd-find, which is an
>> implementation of (paraphrasing here) about 80% of find that's usually
>> several times faster.
> Any hint*why* it's faster?
In my benchmarks, it isn't. But there is at least one report with
numbers that says otherwise (on your bug mailing list).
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 25 Jan 2021 15:42:13 -0500
> Cc: ofv@wanadoo.es, Eli Zaretskii <eliz@gnu.org>,
> Phillip Lord <phillip.lord@russet.org.uk>,
> EMACS development team <emacs-devel@gnu.org>
>
> > Tangent, but: you might be interested in fd-find, which is an
> > implementation of (paraphrasing here) about 80% of find that's usually
> > several times faster.
>
> Any hint *why* it's faster?
It uses several threads, and uses PCRE instead of fnmatch. The latter
is a subtle difference from what Find does, and it explains the
different results of the search.
> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 25 Jan 2021 22:17:15 +0100
>
> > I hope we don't do anything
> > that breaks the ability to build and install Emacs into an existing
> > MSYS2 tree using "./autogen.sh && ./configure && make && make
> > install", as their PKGBUILD script does.
>
> +1, although I don't expect that this is checked regularly by the Emacs
> developers.
??? How else do you think I build my Emacs?
Eli Zaretskii <eliz@gnu.org> writes:
>> > I hope we don't do anything
>> > that breaks the ability to build and install Emacs into an existing
>> > MSYS2 tree using "./autogen.sh && ./configure && make && make
>> > install", as their PKGBUILD script does.
>>
>> +1, although I don't expect that this is checked regularly by the Emacs
>> developers.
>
> ??? How else do you think I build my Emacs?
I thought you used MSYS, not MSYS2.
Do you also use the MinGW-w64 libraries provided by MSYS2?
On January 26, 2021 7:43:23 AM GMT+02:00, "Óscar Fuentes" <ofv@wanadoo.es> wrote: > Eli Zaretskii <eliz@gnu.org> writes: > > >> > I hope we don't do anything > >> > that breaks the ability to build and install Emacs into an > existing > >> > MSYS2 tree using "./autogen.sh && ./configure && make && make > >> > install", as their PKGBUILD script does. > >> > >> +1, although I don't expect that this is checked regularly by the > Emacs > >> developers. > > > > ??? How else do you think I build my Emacs? > > I thought you used MSYS, not MSYS2. That's right, I do use MSYS. But how is that relevant to breaking the build procedure? What am I missing? > Do you also use the MinGW-w64 libraries provided by MSYS2? No.
Eli Zaretskii <eliz@gnu.org> writes: > On January 26, 2021 7:43:23 AM GMT+02:00, "Óscar Fuentes" <ofv@wanadoo.es> wrote: >> Eli Zaretskii <eliz@gnu.org> writes: >> >> >> > I hope we don't do anything >> >> > that breaks the ability to build and install Emacs into an >> existing >> >> > MSYS2 tree using "./autogen.sh && ./configure && make && make >> >> > install", as their PKGBUILD script does. >> >> >> >> +1, although I don't expect that this is checked regularly by the >> Emacs >> >> developers. >> > >> > ??? How else do you think I build my Emacs? >> >> I thought you used MSYS, not MSYS2. > > That's right, I do use MSYS. That's why I wrote "although I don't expect that this [building on MSYS2] is checked regularly by the Emacs developers." > But how is that relevant to breaking the build procedure? What am I > missing? Whatever works in MSYS does not necessarily works on MSYS2.
On January 26, 2021 9:37:48 AM GMT+02:00, "Óscar Fuentes" <ofv@wanadoo.es> wrote: > Eli Zaretskii <eliz@gnu.org> writes: > > > That's right, I do use MSYS. > > That's why I wrote "although I don't expect that this [building on > MSYS2] is checked regularly by the Emacs developers." But everybody else who builds on Windows here uses MSYS2, so I still don't understand the problem you had in mind. > Whatever works in MSYS does not necessarily works on MSYS2. Do you mean MSYS2 or do you mean MinGW64? The former is supposed to work like MSYS, for the latter I always check in their sources to make sure nothing would break. And again, others build with those tools and report (rare) problems when they happen. And if people stop using our build procedure with MinGW64, there will be no reason to worry about it not working, right?
Eli Zaretskii <eliz@gnu.org> writes: >> From: Phillip Lord <phillip.lord@russet.org.uk> >> Cc: emacs-devel@gnu.org >> Date: Sun, 24 Jan 2021 22:13:07 +0000 >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> >> > They mostly work, until they don't. Like with Cygwin, there are >> >> > subtle incompatibilities, mainly in file names and in communications >> >> > with subprocesses and response to "signals". Encoding defaults are >> >> > also different. >> >> >> >> That's true for the msys2 commands but not the mingw64 ones? >> > >> > Yes. That's why I asked about msys-1.0.dll: the programs that depend >> > on that aren't mingw64 (native) programs. >> >> But, mingw64 does not have all the packages I need. > > Are we talking about you personally, or are we talking about Emacs > users? Emacs users in general. I don't use Emacs for Windows myself, at least not in any routine way. > If the former, then MinGW64 are not the only source of good ports of > Free Software to Windows, far from that. If they don't have some > package, you just go out and look for it elsewhere. I am sure that this is true. But I have to scrape together tools from several different locations, particularly if I have to host it (and thus have the responsibilty of getting all the source), it's not going to happen. I might be able to use a different package manager (like chocolaty); I'd rather use msys2 because I already use it for my build machine. > If we are talking about Emacs users who will download Emacs from the > GNU sites, then I'd say give them only what MinGW64 provides, so that > they could use pacman to easily update that. Those who need more will > have to find and install whatever they need on their own. Telling > them to install MSYS ports instead risks exposing them to subtle > problems, so I wouldn't recommend it. Okay. In which case I'll only add the mingw64 to the path. This excludes git which was one of my main aims to be honest. But at least we can add a spell checker to Emacs. > >> How do people use Emacs on windows? I mean, do they install find, ls, >> git, aspell and all the rest by hand? > > If MinGW doesn't provide those, what else can you do? Me, I ported > some of the packages myself (where I found no ports that were good > enough or new for me), and installed others where I found good ports. > Almost all of my ports are available from the ezwinports site. Give up on these tools till a better solution appears. >> >> Would it be easier to have Emacs allow me to successfully update PATH >> >> during run? >> > >> > Update how? >> >> Using some magic that doesn't exist at the current time. I mean a way >> like did (by fiddling with setenv) that doesn't suffer the problems that >> it causes. > > I don't think this magic can exist. But if someone knows, let them > speak up. I don't know what the problems are that it causes, but again, if it's not simple and we cannot provide a 90/10 solution for changing path within a running Emacs, then I will try something that doesn't need this. Phil
> But everybody else who builds on Windows here uses MSYS2, so I still > don't understand the problem you had in mind. Not everybody else. martin
Richard Copley <rcopley@gmail.com> writes:
> On Mon, 25 Jan 2021 at 15:21, Eli Zaretskii <eliz@gnu.org> wrote:
>> > From: Phillip Lord <phillip.lord@russet.org.uk>
>> > Eli Zaretskii <eliz@gnu.org> writes:
>
>> > >> Would it be easier to have Emacs allow me to successfully update PATH
>> > >> during run?
>> > >
>> > > Update how?
>> >
>> > Using some magic that doesn't exist at the current time. I mean a way
>> > like did (by fiddling with setenv) that doesn't suffer the problems that
>> > it causes.
>>
>> I don't think this magic can exist. But if someone knows, let them
>> speak up.
>
> No magic is needed. Just change exec-path and the PATH environment
> variable matching values, not forgetting to add Emacs' libexec dir to
> exec-path. I have code in my init file to select different paths,
> because different toolchains that I want to run in Emacs require
> different incompatible paths. It doesn't cause any problems, and
> everything behaves as you expect.
I do the same; I sometimes change PATH after init when I change
projects.
--
-- Stephe
Óscar Fuentes <ofv@wanadoo.es> writes:
> That's why I wrote "although I don't expect that this [building on
> MSYS2] is checked regularly by the Emacs developers."
For what it's worth, I build on MSYS2.
I only build once a month or so, and I don't install.
--
-- Stephe
> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Date: Wed, 27 Jan 2021 06:55:10 -0800
> Cc: emacs-devel@gnu.org
>
> Óscar Fuentes <ofv@wanadoo.es> writes:
>
> > That's why I wrote "although I don't expect that this [building on
> > MSYS2] is checked regularly by the Emacs developers."
>
> For what it's worth, I build on MSYS2.
I wouldn't expect the differences between MSYS and MSYS2 to be of any
importance here. It's the same Bash and the same Coreutils. And the
commands we use during the build procedure are very basic.
Sorry for late reply, but I think that while bundling msys2 is a good idea in theory, in practice it would turn into a complete nightmare. The problem, just like with bundling any third party components is that you have to maintain them. If you bundle the latest and shiniest msys2, you can never be sure if it's really properly working for our use cases. And if you bundle some pretested version, you run into "hey, please fix bug A, the upstream has already fixed it", but we can't switch to upstream due to bug B. Msys2 sort of missed the boat in that they took pacman, but didn't bother with making their own AUR.
Nikolay Kudryavtsev <nikolay.kudryavtsev@gmail.com> writes:
> Sorry for late reply, but I think that while bundling msys2 is a good
> idea in theory, in practice it would turn into a complete
> nightmare. The problem, just like with bundling any third party
> components is that you have to maintain them. If you bundle the latest
> and shiniest msys2, you can never be sure if it's really properly
> working for our use cases. And if you bundle some pretested version,
> you run into "hey, please fix bug A, the upstream has already fixed
> it", but we can't switch to upstream due to bug B.
>
> Msys2 sort of missed the boat in that they took pacman, but didn't
> bother with making their own AUR.
The work on this has stalled at the moment anyway, but my plan would be
to work out how to link an Emacs installation to an Msys2 installation,
by setting up paths correctly, as well as defining a set of packages
that are useful for Emacs.
There is always the risk that msys and Emacs work inconsistently since,
with this scenario, msys could update at any point that breaks things. I
don't think that there is any solution to this than to say that the last
release version of Emacs will work with a version of msys which is about
current at the time of release. What else can we do? In the case, that
basic Emacs functionality fails, people could always fall back to the
fully bundled Emacs with DLLs that is currently available.
The current situation where Emacs without msys2 lacks basic capabilities
such as git handling which many other editors have bundled is also
problematic!
Phil
Phillip Lord <phillip.lord@russet.org.uk> writes: > Nikolay Kudryavtsev <nikolay.kudryavtsev@gmail.com> writes: > >> Sorry for late reply, but I think that while bundling msys2 is a good >> idea in theory, in practice it would turn into a complete >> nightmare. The problem, just like with bundling any third party >> components is that you have to maintain them. If you bundle the latest >> and shiniest msys2, you can never be sure if it's really properly >> working for our use cases. And if you bundle some pretested version, >> you run into "hey, please fix bug A, the upstream has already fixed >> it", but we can't switch to upstream due to bug B. >> >> Msys2 sort of missed the boat in that they took pacman, but didn't >> bother with making their own AUR. > > The work on this has stalled at the moment anyway, but my plan would be > to work out how to link an Emacs installation to an Msys2 installation, > by setting up paths correctly, as well as defining a set of packages > that are useful for Emacs. > > There is always the risk that msys and Emacs work inconsistently since, > with this scenario, msys could update at any point that breaks things. I > don't think that there is any solution to this than to say that the last > release version of Emacs will work with a version of msys which is about > current at the time of release. What else can we do? In the case, that > basic Emacs functionality fails, people could always fall back to the > fully bundled Emacs with DLLs that is currently available. > > The current situation where Emacs without msys2 lacks basic capabilities > such as git handling which many other editors have bundled is also > problematic! (*) Introduction I'll share my opinion as a user of the GNU Emacs and Windows. I'll try to summarize my context to help you decide whether it's worth reading the entire message. Maybe the context of this thread is a bit off of mine, so perhaps the difficulties I mention don't quite apply to what Phillip Lord is considering. (*) Summary Unless this coupling Emacs-MSYS2 can be well done with a certain long-run guarantee, I'd still prefer put them together with my own hands, because this way I can guarantee the behavior of the programs I expect to get. So I think I'd need an assurance of always having a certain exact (verified with a hash sum) version of msys2. I guarantee this today myself by zipping my GNU Emacs directory and moving it to another system when something happens that I need to move, say when my computer crashes and I need to install another Windows. My guarantee is based on Windows' guarantee of backward compatibility and behavior stability, which is why I have to run Windows. (*) I manually put msys2 inside Emacs --8<---------------cut here---------------start------------->8--- %pwd c:/sys/emacs/usr/mingw %file msys2.exe msys2.exe: PE32+ executable (GUI) x86-64 (stripped to external PDB), for MS Windows --8<---------------cut here---------------end--------------->8--- Having discovered msys2 recently I immediately convinced myself to reorganize my GNU Emacs to use msys2. (Until then I was using a 32-bit GNU Emacs and installed my own selections of Eli Zaretskii's ezwinports --- they have been useful to me for various years. Thank you!) I now consider msys2 part of my GNU Emacs. (*) Tangent 1: Windows was my last resort Running Windows is not something that pleases me. I'd much rather run WindowMaker, say, to manage my windows, but I feel I'm stuck with Windows because GNU systems don't help me with keeping all I need in a single directory with an assurance that I can move that directory to a similar system and still have everything work. This is not a compliment to Windows. If GNU systems did not come with all of their variety and openess and freedom, I'm sure they could provide the same status quo provided by Windows. As I understand it, the core of issue here is what Torvalds calls the ``binary issue'' while answering these questions: https://youtu.be/5PmHRSeA2c8?t=1722 https://youtu.be/5PmHRSeA2c8?t=2473 So, I'm lucky in that the programs I need are all well-designed and can be easily ported from one system to another without requiring reinstallations and reconfigurations, otherwise Windows would also be a non-solution. For instance, if Chrome or Word were software I need, then Windows would be a non-solution. (*) Tangent 2: I need long-term assurance As an example, I've built my own OpenBSD distribution because I wanted an assurance in the behavior of the system, besides a quick installation. I install it with a single command line and it asks no questions. It comes ready to do all the things *I* usually do. The source code of everything is exactly where I expect it to be and so on. It allows me to quickly do all the experiments I do with a minimum of technological struggle. (Let me know if anyone would like a copy of such thing, by the way. It's OpenBSD 4.5, source code included. Sadly, I didn't include the GNU Emacs, though my notes say that was the very next step.)
Wayne Harris via "Emacs development discussions." <emacs-devel@gnu.org> writes: > Phillip Lord <phillip.lord@russet.org.uk> writes: > >> Nikolay Kudryavtsev <nikolay.kudryavtsev@gmail.com> writes: >> >>> Sorry for late reply, but I think that while bundling msys2 is a good >>> idea in theory, in practice it would turn into a complete >>> nightmare. The problem, just like with bundling any third party >>> components is that you have to maintain them. If you bundle the latest >>> and shiniest msys2, you can never be sure if it's really properly >>> working for our use cases. And if you bundle some pretested version, >>> you run into "hey, please fix bug A, the upstream has already fixed >>> it", but we can't switch to upstream due to bug B. >> There is always the risk that msys and Emacs work inconsistently since, >> with this scenario, msys could update at any point that breaks things. I >> don't think that there is any solution to this than to say that the last >> release version of Emacs will work with a version of msys which is about >> current at the time of release. What else can we do? In the case, that >> basic Emacs functionality fails, people could always fall back to the >> fully bundled Emacs with DLLs that is currently available. >> >> The current situation where Emacs without msys2 lacks basic capabilities >> such as git handling which many other editors have bundled is also >> problematic! > > (*) Introduction > > I'll share my opinion as a user of the GNU Emacs and Windows. I'll try > to summarize my context to help you decide whether it's worth reading > the entire message. Maybe the context of this thread is a bit off of > mine, so perhaps the difficulties I mention don't quite apply to what > Phillip Lord is considering. Always useful, to get feedback since I rarely use Emacs on windows myself. > (*) Summary > > Unless this coupling Emacs-MSYS2 can be well done with a certain > long-run guarantee, I'd still prefer put them together with my own > hands, because this way I can guarantee the behavior of the programs I > expect to get. So I think I'd need an assurance of always having a > certain exact (verified with a hash sum) version of msys2. > I can certainly appreciate that. I sit somewhere in the middle: I use Emacs build from master, but most of my emacs packages are specific versions, rather than running from everyones heads. But, like it or lump it, msys2 doesn't do that. They have a versioned, hash summed installer, but after that it just updates to the latest version, with no specific release pattern (or a rolling release if you prefer). I want to try and avoid replicating what msys2 already does which, conversely, means I can only do what msys2 does. > As an example, I've built my own OpenBSD distribution because I wanted > an assurance in the behavior of the system, besides a quick > installation. I install it with a single command line and it asks no > questions. It comes ready to do all the things *I* usually do. Something close to this, I think we could achieve. Install Emacs, have it ask "do you want to link to an msys2 installation? Do you want to install it? Do you want to update it with Emacs standard packages". So three questions, but not none. Phil
You technically can make your own msys distribution, by choosing a base package as old as you want and then installing extras from the package archives manually downloaded from msys2 binary cache. But yeah, I don't think anyone would force anyone to use the msys2-bundled installation, I just wanted to note that maintaining such distribution seems like too much work for the benefit it provides. In the ideal world we'll have something like a bundled configure-like package that would test for those POSIX utilities and then show what you're missing and maybe install them for you.
You might be interested in GNU Guix(or Nixos), since it provides exactly that ability to write one configuration script that you can later use to completely setup a fresh machine.
Nikolay Kudryavtsev <nikolay.kudryavtsev@gmail.com> writes:
> You might be interested in GNU Guix(or Nixos), since it provides
> exactly that ability to write one configuration script that you can
> later use to completely setup a fresh machine.
Since we are discussing Windows support, those tools are irrelevant,
aren't they?
Nikolay Kudryavtsev <nikolay.kudryavtsev@gmail.com> writes:
> You might be interested in GNU Guix(or Nixos), since it provides
> exactly that ability to write one configuration script that you can
> later use to completely setup a fresh machine.
Thank you for sharing that! I just read the introduction of GNU Guix.
What a marvelous work! I always wondered if things could be done this
way. The fact no tool ever was made me think it would be too difficult
to do. The fact it's written in Guile pleases me even more.
I was just mentioning them for Wayne, since his frustration with the POSIX way of distributing software is very familiar to me.
Phillip Lord <phillip.lord@russet.org.uk> writes: [...] >> (*) Summary >> >> Unless this coupling Emacs-MSYS2 can be well done with a certain >> long-run guarantee, I'd still prefer put them together with my own >> hands, because this way I can guarantee the behavior of the programs I >> expect to get. So I think I'd need an assurance of always having a >> certain exact (verified with a hash sum) version of msys2. > > I can certainly appreciate that. I sit somewhere in the middle: I use > Emacs build from master, but most of my emacs packages are specific > versions, rather than running from everyones heads. How do you do that? It seems you take the .exe but keep .el separate? That sounds crazy to me. How would I know all my new .exe still works with my very old .el files? > But, like it or lump it, msys2 doesn't do that. They have a versioned, > hash summed installer, but after that it just updates to the latest > version, with no specific release pattern (or a rolling release if you > prefer). [...] That's indeed a big problem for people who do work dependent on computer tools --- probably anyone who does any serious work on computers. I also see many non-professional users, say, who just hate the surprise of updates. (They might have all the time in the world to learn new behaviors, but they just hate it. Windows forums are full of angry customers.) That's why I loved the Guix-news posted by Nikolay Kudryavtsev. That's what I like about computer systems --- repetitive behavior with mathematical guarantee. It's incredible that people tolerate computers with human-like behavior, ``mood-dependent'' say. >> As an example, I've built my own OpenBSD distribution because I wanted >> an assurance in the behavior of the system, besides a quick >> installation. I install it with a single command line and it asks no >> questions. It comes ready to do all the things *I* usually do. > > Something close to this, I think we could achieve. Install Emacs, have > it ask "do you want to link to an msys2 installation? Do you want to > install it? Do you want to update it with Emacs standard packages". > > So three questions, but not none. That sounds nice. If I had to start my ``GNU Emacs system'' from scratch, this step would be a nice one. The GNU Emacs on Windows is handicapped without lots of other UNIX programs. I wrote this years ago: --8<---------------cut here---------------start------------->8--- ;; (*) On being in the wild, without a map ;; When you're on Windows, you're on your own. You have to provide ;; everything, that is, you need to bring the UNIX part that we can ;; replicate on Windows. (Thankfully, the EMACS shell replaces the UNIX ;; shell, even on Windows.) ;; Use this GNU debugger (when (string= system-type "windows-nt") (setq gdb-command-name "~/mingw/usr/bin/gdb.exe")) [...] ;; Use this diff program (when (string= system-type "windows-nt") (setq ediff-diff-program "~/mingw/usr/bin/diff.exe")) [...] --8<---------------cut here---------------end--------------->8---
Wayne Harris via "Emacs development discussions." <emacs-devel@gnu.org> writes: > Phillip Lord <phillip.lord@russet.org.uk> writes: > > [...] > >> I can certainly appreciate that. I sit somewhere in the middle: I use >> Emacs build from master, but most of my emacs packages are specific >> versions, rather than running from everyones heads. > > How do you do that? It seems you take the .exe but keep .el separate? > That sounds crazy to me. How would I know all my new .exe still works > with my very old .el files? Oh, sorry, I didn't describe it well. I use Emacs and all the lisp files in the main repo. My external packages mostly come from melpa-stable and elpa which are tagged versions. I sync my computers with unison and keep the packages installed using use-package; the versions formally aren't linked between machines, but tools like straight.el would do that for you. >> But, like it or lump it, msys2 doesn't do that. They have a versioned, >> hash summed installer, but after that it just updates to the latest >> version, with no specific release pattern (or a rolling release if you >> prefer). [...] > > That's indeed a big problem for people who do work dependent on computer > tools --- probably anyone who does any serious work on computers. I > also see many non-professional users, say, who just hate the surprise of > updates. (They might have all the time in the world to learn new > behaviors, but they just hate it. Windows forums are full of angry > customers.) Perhaps. Clearly msys2 did it for a reasons, and likewise a lot of Emacs users install from melpa which mostly runs on a dirty head or rolling release if you prefer. It always sounded like a nightmare to me, but it seems to work for many people. > > That's why I loved the Guix-news posted by Nikolay Kudryavtsev. That's > what I like about computer systems --- repetitive behavior with > mathematical guarantee. It's incredible that people tolerate computers > with human-like behavior, ``mood-dependent'' say. > >>> As an example, I've built my own OpenBSD distribution because I wanted >>> an assurance in the behavior of the system, besides a quick >>> installation. I install it with a single command line and it asks no >>> questions. It comes ready to do all the things *I* usually do. >> >> Something close to this, I think we could achieve. Install Emacs, have >> it ask "do you want to link to an msys2 installation? Do you want to >> install it? Do you want to update it with Emacs standard packages". >> >> So three questions, but not none. > > That sounds nice. If I had to start my ``GNU Emacs system'' from > scratch, this step would be a nice one. The GNU Emacs on Windows is > handicapped without lots of other UNIX programs. Yes, I agree. Phil