Emacs 27.1 released

Emacs 27.1 released

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 2016 bytes --]

Hi!

Version 27.1 of the Emacs text editor is now available.

For more information on Emacs, see:
  https://www.gnu.org/software/emacs

You can retrieve the source from your nearest GNU mirror by using one
of the following links:
  https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.xz
  https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.gz

You can get the PGP signatures at
  https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.xz.sig
  https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.gz.sig

The tarball is signed with the following GPG key, which can be found on
public PGP key servers:

  D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978

To retrieve the key from a PGP key server, evaluate

  gpg --keyserver hkp://keys.gnupg.net --recv-keys D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978

You can choose a mirror explicitly from the list at:
  https://www.gnu.org/prep/ftp.html

Mirrors may take some time to update; the main GNU ftp server is at:
  https://ftp.gnu.org/gnu/emacs/

Emacs 27.1 has a wide variety of new features, including:

  - Built-in support for arbitrary-size integers
  - Text shaping with HarfBuzz
  - Native support for JSON parsing
  - Better support for Cairo drawing
  - Portable dumping used instead of unexec
  - Support for XDG conventions for init files
  - Additional early-init initialization file
  - Lexical-binding is used by default
  - Built-in support for tab bar and tab-line
  - Support for resizing and rotating of images without ImageMagick

There are many more changes; for a summary see the etc/NEWS file, which
you can view from Emacs with `C-h n'.

For the complete list of changes and the people who made them, see the
various ChangeLog files in the source distribution.  For a summary of
all the people who have contributed to Emacs, see the etc/AUTHORS file.

The online manuals and website will be updated shortly.

Printed copies of the Emacs manual are available for purchase from the
Free Software Foundation's online store at:
https://shop.fsf.org/product/emacs-manual/

Regards,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Emacs 27.1 released

[Stefan Kangas]

First, a big congratulations. Second:

>   - Lexical-binding is used by default

Maybe this statement should be moderated somewhat.  I see in NEWS only that:

** Lexical binding is now used by default when evaluating interactive Elisp.

Best regards,
Stefan Kangas

Re: Emacs 27.1 released

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > ** Lexical binding is now used by default when evaluating interactive Elisp.

Does that mean that evaluating (setq foo (....)) will fail to
leave foo set?  I doubt that would ever be convenient for anyone.

However, I'm running master from June and it seems to set foo ok.



-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Emacs 27.1 released

[Noam Postavsky]

On Tue, 11 Aug 2020 at 22:24, Richard Stallman <rms@gnu.org> wrote:

>   > ** Lexical binding is now used by default when evaluating interactive Elisp.
>
> Does that mean that evaluating (setq foo (....)) will fail to
> leave foo set?  I doubt that would ever be convenient for anyone.

No, lexical binding mode only (directly) affects let-bindings. setq of
free variables will still set the dynamic/global value.

Re: Emacs 27.1 released

[Stefan Kangas]

Stefan Kangas <stefan@marxist.se> writes:

> Maybe this statement should be moderated somewhat.  I see in NEWS only that:
>
> ** Lexical binding is now used by default when evaluating interactive Elisp.

I've been seeing some confusion about this on Reddit, where people have
understood this to be the case generally.  Perhaps we should clarify the
text currently available at https://www.gnu.org/software/emacs/

Re: Emacs 27.1 released

[Lars Ingebrigtsen]

Stefan Kangas <stefan@marxist.se> writes:

> Stefan Kangas <stefan@marxist.se> writes:
>
>> Maybe this statement should be moderated somewhat.  I see in NEWS only that:
>>
>> ** Lexical binding is now used by default when evaluating interactive Elisp.
>
> I've been seeing some confusion about this on Reddit, where people have
> understood this to be the case generally.  Perhaps we should clarify the
> text currently available at https://www.gnu.org/software/emacs/

Definitely.  In fact, I think it would be better not to mention lexical
binding at all in the short-short NEWS overview there -- whether there's
lexical binding in the M-: command or not is something that almost
nobody will ever notice.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

RE: Emacs 27.1 released

[Drew Adams]

> > I've been seeing some confusion about this on Reddit, where people have
> > understood this to be the case generally.  Perhaps we should clarify the
> > text currently available at ...
> 
> Definitely.  In fact, I think it would be better not to mention lexical
> binding at all in the short-short NEWS overview there -- whether there's
> lexical binding in the M-: command or not is something that almost
> nobody will ever notice.

+1.  Maybe not "almost nobody".  But yes, this is a
relatively minor change whose description can easily
lead to (major) confusion.

Re: Emacs 27.1 released

[Mingde (Matthew) Zeng]

Wow!! Configurations!!

Nicolas Petton <nico@petton.fr> writes:

> Hi!
>
> Version 27.1 of the Emacs text editor is now available.
>
> For more information on Emacs, see:
>   https://www.gnu.org/software/emacs
>
> You can retrieve the source from your nearest GNU mirror by using one
> of the following links:
>   https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.xz
>   https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.gz
>
> You can get the PGP signatures at
>   https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.xz.sig
>   https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.gz.sig
>
> The tarball is signed with the following GPG key, which can be found on
> public PGP key servers:
>
>   D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978
>
> To retrieve the key from a PGP key server, evaluate
>
>   gpg --keyserver hkp://keys.gnupg.net --recv-keys D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978
>
> You can choose a mirror explicitly from the list at:
>   https://www.gnu.org/prep/ftp.html
>
> Mirrors may take some time to update; the main GNU ftp server is at:
>   https://ftp.gnu.org/gnu/emacs/
>
> Emacs 27.1 has a wide variety of new features, including:
>
>   - Built-in support for arbitrary-size integers
>   - Text shaping with HarfBuzz
>   - Native support for JSON parsing
>   - Better support for Cairo drawing
>   - Portable dumping used instead of unexec
>   - Support for XDG conventions for init files
>   - Additional early-init initialization file
>   - Lexical-binding is used by default
>   - Built-in support for tab bar and tab-line
>   - Support for resizing and rotating of images without ImageMagick
>
> There are many more changes; for a summary see the etc/NEWS file, which
> you can view from Emacs with `C-h n'.
>
> For the complete list of changes and the people who made them, see the
> various ChangeLog files in the source distribution.  For a summary of
> all the people who have contributed to Emacs, see the etc/AUTHORS file.
>
> The online manuals and website will be updated shortly.
>
> Printed copies of the Emacs manual are available for purchase from the
> Free Software Foundation's online store at:
> https://shop.fsf.org/product/emacs-manual/
>
> Regards,
> Nico


--
Mingde (Matthew) Zeng

Re: Emacs 27.1 released

[황병희]

Nicolas Petton <nico@petton.fr> writes:

> Hi!
>
> Version 27.1 of the Emacs text editor is now available.

Thank You Always^^^

Sincerely, Byung-Hee

-- 
^고맙습니다 _和合團結_ 감사합니다_^))//

Re: Emacs 27.1 released

[Jay Sulzberger]

On Tue, 11 Aug 2020, Nicolas Petton <nico@petton.fr> wrote:

> Hi!
>
> Version 27.1 of the Emacs text editor is now available.

Dear Emacs Workers, thank you!

Strong letter to follow.^W^W^W^WI remain, as ever your fellow student
of history and probability, and long time Emacs fan,
Jay Sulzberger

>
> For more information on Emacs, see:
>  https://www.gnu.org/software/emacs
>
> You can retrieve the source from your nearest GNU mirror by using one
> of the following links:
>  https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.xz
>  https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.gz
>
> You can get the PGP signatures at
>  https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.xz.sig
>  https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.gz.sig
>
> The tarball is signed with the following GPG key, which can be found on
> public PGP key servers:
>
>  D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978
>
> To retrieve the key from a PGP key server, evaluate
>
>  gpg --keyserver hkp://keys.gnupg.net --recv-keys D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978
>
> You can choose a mirror explicitly from the list at:
>  https://www.gnu.org/prep/ftp.html
>
> Mirrors may take some time to update; the main GNU ftp server is at:
>  https://ftp.gnu.org/gnu/emacs/
>
> Emacs 27.1 has a wide variety of new features, including:
>
>  - Built-in support for arbitrary-size integers
>  - Text shaping with HarfBuzz
>  - Native support for JSON parsing
>  - Better support for Cairo drawing
>  - Portable dumping used instead of unexec
>  - Support for XDG conventions for init files
>  - Additional early-init initialization file
>  - Lexical-binding is used by default
>  - Built-in support for tab bar and tab-line
>  - Support for resizing and rotating of images without ImageMagick
>
> There are many more changes; for a summary see the etc/NEWS file, which
> you can view from Emacs with `C-h n'.
>
> For the complete list of changes and the people who made them, see the
> various ChangeLog files in the source distribution.  For a summary of
> all the people who have contributed to Emacs, see the etc/AUTHORS file.
>
> The online manuals and website will be updated shortly.
>
> Printed copies of the Emacs manual are available for purchase from the
> Free Software Foundation's online store at:
> https://shop.fsf.org/product/emacs-manual/
>
> Regards,
> Nico
>

Re: Emacs 27.1 released

[Ulrich Mueller]

>>>>> On Tue, 11 Aug 2020, Nicolas Petton wrote:

> Version 27.1 of the Emacs text editor is now available.

Thank you very much!

Could the version number in the emacs-27 branch be updated, e.g.
to 27.1.50? Otherwise Emacs built from Git will collide with the
released version.

Re: Emacs 27.1 released

[Eli Zaretskii]

> From: Ulrich Mueller <ulm@gentoo.org>
> Date: Tue, 11 Aug 2020 10:19:02 +0200
> Cc: Emacs Devel <emacs-devel@gnu.org>
> 
> Could the version number in the emacs-27 branch be updated, e.g.
> to 27.1.50?

Done.

Re: Emacs 27.1 released

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> Could the version number in the emacs-27 branch be updated, e.g.
>> to 27.1.50?
>
> Done.

Does this mean the emacs-27 branch is reopened for patches? I would like
to merge the Tramp 2.4 changes of the last 7 months, waiting in the
Tramp repository.

Best regards, Michael.

Re: Emacs 27.1 released

[Phil Sainty]

On 12/08/20 8:09 pm, Michael Albinus wrote:
> Does this mean the emacs-27 branch is reopened for patches?

I hope so, because I just pushed a commit to it.

Apologies if I've jumped the gun there.


-Phil

Re: Emacs 27.1 released

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Ulrich Mueller <ulm@gentoo.org>,  emacs-devel@gnu.org,  nico@petton.fr
> Date: Wed, 12 Aug 2020 10:09:41 +0200
> 
> Does this mean the emacs-27 branch is reopened for patches?

Yes, but please only install bugfixes which are either safe enough for
the release branch, or very urgent, or fix regressions introduced by
Emacs 26 or 27.1.  Other changes should go to master.

Thanks.

Re: Emacs 27.1 released

[Mattias Engdegård]

12 aug. 2020 kl. 16.21 skrev Eli Zaretskii <eliz@gnu.org>:

> Yes, but please only install bugfixes which are either safe enough for
> the release branch, or very urgent, or fix regressions introduced by
> Emacs 26 or 27.1.  Other changes should go to master.

The bug fixes below (selected from ones on master made by me) should fit the criterion of being safe enough.
Any of them that you would rather not see back-ported to emacs-27?

c48bb7deb8 Preserve match data in 'kbd'
687a9d3e2f Calc: fix interval entry snag (bug#42438)
8ef84632c2 Accept lexical lambda in auto-insert-alist
6242605731 Fix spurious error in beginning-of-defun in pascal-mode (bug#41740)
73daab9991 Preserve point in pascal-mode completion (bug#41740)
2bdb2cd10d Document that {en,de}code-coding-string preserve match data
c5cf630ecd Don't clobber match data in utf-8-hfs conversion (bug#41445)
b1fe27d77d Fix calculator entry of numbers with negative exponents (bug#41347)
60cd6cce55 Calc: GCD(0,x)=GCD(x,0)=|x|, not x (bug#41279)
4af8b17149 Fix customisation of mouse-drag-and-drop-region (bug#41251)
1d559581b3 Calc: fix LU decomposition for non-numeric matrices (bug#41223)
63268253d2 Regexps cannot infloop; fix manual
20c1e7f8af Fix calculator division truncation (bug#40892)
7839390f27 Quote semanticdb-ebrowse-default-file-name in regexp
95dd8de1df chinese-hz is not ASCII compatible (bug#40407)
8d95e75eb6 utf-7 and utf-7-imap are not ASCII-compatible (bug#40407)
7195ea7532 Avoid regexp stack overflow in GDB string matching (bug#22149)

Re: Emacs 27.1 released

[Robert Pluim]

>>>>> On Wed, 12 Aug 2020 18:20:35 +0200, Mattias Engdegård <mattiase@acm.org> said:

    Mattias> 12 aug. 2020 kl. 16.21 skrev Eli Zaretskii <eliz@gnu.org>:
    >> Yes, but please only install bugfixes which are either safe enough for
    >> the release branch, or very urgent, or fix regressions introduced by
    >> Emacs 26 or 27.1.  Other changes should go to master.

    Mattias> The bug fixes below (selected from ones on master made by me) should fit the criterion of being safe enough.
    Mattias> Any of them that you would rather not see back-ported to emacs-27?

    Mattias> c48bb7deb8 Preserve match data in 'kbd'
    Mattias> 687a9d3e2f Calc: fix interval entry snag (bug#42438)
    Mattias> 8ef84632c2 Accept lexical lambda in auto-insert-alist
    Mattias> 6242605731 Fix spurious error in beginning-of-defun in pascal-mode (bug#41740)
    Mattias> 73daab9991 Preserve point in pascal-mode completion (bug#41740)
    Mattias> 2bdb2cd10d Document that {en,de}code-coding-string preserve match data
    Mattias> c5cf630ecd Don't clobber match data in utf-8-hfs conversion (bug#41445)
    Mattias> b1fe27d77d Fix calculator entry of numbers with negative exponents (bug#41347)
    Mattias> 60cd6cce55 Calc: GCD(0,x)=GCD(x,0)=|x|, not x (bug#41279)
    Mattias> 4af8b17149 Fix customisation of mouse-drag-and-drop-region (bug#41251)
    Mattias> 1d559581b3 Calc: fix LU decomposition for non-numeric matrices (bug#41223)
    Mattias> 63268253d2 Regexps cannot infloop; fix manual
    Mattias> 20c1e7f8af Fix calculator division truncation (bug#40892)
    Mattias> 7839390f27 Quote semanticdb-ebrowse-default-file-name in regexp
    Mattias> 95dd8de1df chinese-hz is not ASCII compatible (bug#40407)
    Mattias> 8d95e75eb6 utf-7 and utf-7-imap are not ASCII-compatible (bug#40407)
    Mattias> 7195ea7532 Avoid regexp stack overflow in GDB string matching (bug#22149)

If theyʼre safe enough now, why didnʼt they go into emacs-27
initially?

Certainly all the ones that just change documentation should be ok.

My list would be

* 23b04ef0e7 Use length field when dns-query is using TCP
* 00f7744c1b Check for IPv6 servers in dns.el

But I donʼt think either of those fit the criterion of urgent (and
theyʼre not regressions).

Robert

Re: Emacs 27.1 released

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Wed, 12 Aug 2020 18:20:35 +0200
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
> 12 aug. 2020 kl. 16.21 skrev Eli Zaretskii <eliz@gnu.org>:
> 
> > Yes, but please only install bugfixes which are either safe enough for
> > the release branch, or very urgent, or fix regressions introduced by
> > Emacs 26 or 27.1.  Other changes should go to master.
> 
> The bug fixes below (selected from ones on master made by me) should fit the criterion of being safe enough.
> Any of them that you would rather not see back-ported to emacs-27?
> 
> c48bb7deb8 Preserve match data in 'kbd'

Doesn't seem urgent: we've had this forever, didn't we?

> 687a9d3e2f Calc: fix interval entry snag (bug#42438)

Doesn't seem to be very important or urgent, but since it's highly
localized, I don't mind back-porting it.

> 8ef84632c2 Accept lexical lambda in auto-insert-alist

I don't think this is important enough, but you could try convincing
me by telling more details, including when this was introduced.

> 6242605731 Fix spurious error in beginning-of-defun in pascal-mode (bug#41740)

Since this is very localized, it's fine with me.

> 73daab9991 Preserve point in pascal-mode completion (bug#41740)

OK.

> 2bdb2cd10d Document that {en,de}code-coding-string preserve match data

Documentation changes are always safe on the release branch.

> c5cf630ecd Don't clobber match data in utf-8-hfs conversion (bug#41445)

OK.

> b1fe27d77d Fix calculator entry of numbers with negative exponents (bug#41347)

OK

> 60cd6cce55 Calc: GCD(0,x)=GCD(x,0)=|x|, not x (bug#41279)

I'm uneasy with this, as it would indirectly affect many functions.
Can't this wait for Emacs 28?

> 4af8b17149 Fix customisation of mouse-drag-and-drop-region (bug#41251)

OK

> 1d559581b3 Calc: fix LU decomposition for non-numeric matrices (bug#41223)

OK

> 63268253d2 Regexps cannot infloop; fix manual

Documentation again; OK.

> 20c1e7f8af Fix calculator division truncation (bug#40892)

OK

> 7839390f27 Quote semanticdb-ebrowse-default-file-name in regexp

OK

> 95dd8de1df chinese-hz is not ASCII compatible (bug#40407)

OK

> 8d95e75eb6 utf-7 and utf-7-imap are not ASCII-compatible (bug#40407)

OK

> 7195ea7532 Avoid regexp stack overflow in GDB string matching (bug#22149)

OK

Thanks.

Re: Emacs 27.1 released

[Mattias Engdegård]

12 aug. 2020 kl. 18.47 skrev Eli Zaretskii <eliz@gnu.org>:

>> c48bb7deb8 Preserve match data in 'kbd'
> 
> Doesn't seem urgent: we've had this forever, didn't we?

None of these changes are urgent, obviously. Does this mean that you consider this change somehow not safe enough, given your stated disjunction? Perhaps I misunderstood your criteria. This fix is of minor importance, anyway.

>> 8ef84632c2 Accept lexical lambda in auto-insert-alist
> 
> I don't think this is important enough, but you could try convincing
> me by telling more details, including when this was introduced.

The code is old and assumes that a list not starting with 'lambda' cannot be a function. That ceased to be true when lexical binding was introduced; it was a test failure on master that alerted us to it. Fixing this obvious flaw just seemed natural and risk-free. It's nothing I'd fight for, in any case.

>> 60cd6cce55 Calc: GCD(0,x)=GCD(x,0)=|x|, not x (bug#41279)
> 
> I'm uneasy with this, as it would indirectly affect many functions.

Which many functions did you have in mind? It was precisely the interaction with other functions that prompted the fix,  since Calc attempts to evaluate (simplify) arguments to higher-order constructs like sum() before those functions are applied.

> Can't this wait for Emacs 28?

Of course, it can wait indefinitely, like everything else on this list. However, I thought fixing it would be nice to our users, especially since 27.2 is a bug fix release, and the change is quite confined.

> Thanks.

Thank you very much for the quick review!

Re: Emacs 27.1 released

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Wed, 12 Aug 2020 20:01:24 +0200
> Cc: emacs-devel@gnu.org
> 
> 12 aug. 2020 kl. 18.47 skrev Eli Zaretskii <eliz@gnu.org>:
> 
> >> c48bb7deb8 Preserve match data in 'kbd'
> > 
> > Doesn't seem urgent: we've had this forever, didn't we?
> 
> None of these changes are urgent, obviously. Does this mean that you consider this change somehow not safe enough, given your stated disjunction?

It has a potential of affecting a lot of code and customizations,
since 'kbd' is used so widely.

> >> 8ef84632c2 Accept lexical lambda in auto-insert-alist
> > 
> > I don't think this is important enough, but you could try convincing
> > me by telling more details, including when this was introduced.
> 
> The code is old and assumes that a list not starting with 'lambda' cannot be a function. That ceased to be true when lexical binding was introduced; it was a test failure on master that alerted us to it. Fixing this obvious flaw just seemed natural and risk-free. It's nothing I'd fight for, in any case.

How easily would people bump into this in real-life use cases?  I
think the answer to that will tell how important is it to fix it in
Emacs 27.

> >> 60cd6cce55 Calc: GCD(0,x)=GCD(x,0)=|x|, not x (bug#41279)
> > 
> > I'm uneasy with this, as it would indirectly affect many functions.
> 
> Which many functions did you have in mind?

Its callers inside Calc.  Suppose there's some subtle issue with the
change, or some unintended consequence.  OTOH, how probable is it to
have a negative x here?

Re: Emacs 27.1 released

[Mattias Engdegård]

12 aug. 2020 kl. 20.27 skrev Eli Zaretskii <eliz@gnu.org>:

> It has a potential of affecting a lot of code and customizations,
> since 'kbd' is used so widely.

I don't see how, but am not going to press the matter further.

>>>> 8ef84632c2 Accept lexical lambda in auto-insert-alist
>>> 
>>> I don't think this is important enough, but you could try convincing
>>> me by telling more details, including when this was introduced.
>> 
>> The code is old and assumes that a list not starting with 'lambda' cannot be a function. That ceased to be true when lexical binding was introduced; it was a test failure on master that alerted us to it. Fixing this obvious flaw just seemed natural and risk-free. It's nothing I'd fight for, in any case.
> 
> How easily would people bump into this in real-life use cases?  I
> think the answer to that will tell how important is it to fix it in
> Emacs 27.

We have no idea. Most bugs are never reported; people blame themselves and try a different approach, or just blame Emacs in general. That's why it's important to fix bugs we find even absent any concrete evidence that it has affected anyone else.

>>>> 60cd6cce55 Calc: GCD(0,x)=GCD(x,0)=|x|, not x (bug#41279)
>>> 
>>> I'm uneasy with this, as it would indirectly affect many functions.
>> 
>> Which many functions did you have in mind?
> 
> Its callers inside Calc.  Suppose there's some subtle issue with the
> change, or some unintended consequence.

Mind being more specific? I could certainly have overlooked something, but am not helped by these vague allegations in the slightest.

>  OTOH, how probable is it to have a negative x here?

The post-hoc probability is 1 because a user was affected by and reported the bug, but that's not the point: the erroneous simplification GCD(x, 0) -> x leads to trouble later on, as the report shows.

Re: Emacs 27.1 released

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Wed, 12 Aug 2020 21:07:05 +0200
> Cc: emacs-devel@gnu.org
> 
> > How easily would people bump into this in real-life use cases?  I
> > think the answer to that will tell how important is it to fix it in
> > Emacs 27.
> 
> We have no idea. Most bugs are never reported; people blame themselves and try a different approach, or just blame Emacs in general. That's why it's important to fix bugs we find even absent any concrete evidence that it has affected anyone else.

That kind of abstract arguments don't help.  So I'm left to my own
devices, and therefore please leave this change on master.

> >>> I'm uneasy with this, as it would indirectly affect many functions.
> >> 
> >> Which many functions did you have in mind?
> > 
> > Its callers inside Calc.  Suppose there's some subtle issue with the
> > change, or some unintended consequence.
> 
> Mind being more specific?

I just counted its callers, that's all.  A change in a function that
has many callers could have unintended consequences beyond the
function itself.

> >  OTOH, how probable is it to have a negative x here?
> 
> The post-hoc probability is 1 because a user was affected by and reported the bug, but that's not the point: the erroneous simplification GCD(x, 0) -> x leads to trouble later on, as the report shows.

I thought you were going to help me make a better decision, but I
guess I was too naïve.  Please leave this one on master as well.

Re: Emacs 27.1 released

[Mattias Engdegård]

12 aug. 2020 kl. 21.21 skrev Eli Zaretskii <eliz@gnu.org>:

> I just counted its callers, that's all.  A change in a function that
> has many callers could have unintended consequences beyond the
> function itself.

I'm very happy that you are taking an interest in Calc again, and especially that you took the trouble of digging into the ramifications of this little correction! I'm most curious to hear what you found out. You see, when the change to calcFunc-gcd was made, naturally I did look at its callers carefully to assess the impact. Now since you brought it up I've done so again, and found nothing new: all users of that function are either indifferent to the change, or actively benefit from it.

It seems clear that the code is less buggy now than before the change and should therefore be preferred for the sake of correctness. However, I could be wrong, and would very much benefit from your findings so that my mistakes can be fixed on master!

Re: Emacs 27.1 released

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Thu, 13 Aug 2020 21:50:33 +0200
> Cc: emacs-devel@gnu.org
> 
> 12 aug. 2020 kl. 21.21 skrev Eli Zaretskii <eliz@gnu.org>:
> 
> > I just counted its callers, that's all.  A change in a function that
> > has many callers could have unintended consequences beyond the
> > function itself.
> 
> I'm very happy that you are taking an interest in Calc again, and especially that you took the trouble of digging into the ramifications of this little correction! I'm most curious to hear what you found out. You see, when the change to calcFunc-gcd was made, naturally I did look at its callers carefully to assess the impact. Now since you brought it up I've done so again, and found nothing new: all users of that function are either indifferent to the change, or actively benefit from it.
> 
> It seems clear that the code is less buggy now than before the change and should therefore be preferred for the sake of correctness. However, I could be wrong, and would very much benefit from your findings so that my mistakes can be fixed on master!

We are assessing this from two very different points of view: you are
thinking about fixing the bug, and I'm thinking about the dangers of
some unintended consequences of the fix destabilizing Emacs 27.2's
version of Calc.  By their very nature, unintended consequences are
unknown to you and to me, so asking for any specific details is not
useful.

(And could you please drop the thinly-veiled sarcasm, here and
elsewhere?  It makes it harder for me to discuss serious issues with
you, and I don't think I deserve the implied attitude, while doing my
job of keeping the release branch stable.)

Re: Emacs 27.1 released

[Mattias Engdegård]

14 aug. 2020 kl. 08.02 skrev Eli Zaretskii <eliz@gnu.org>:

> We are assessing this from two very different points of view: you are
> thinking about fixing the bug, and I'm thinking about the dangers of
> some unintended consequences of the fix destabilizing Emacs 27.2's
> version of Calc.  By their very nature, unintended consequences are
> unknown to you and to me, so asking for any specific details is not
> useful.

Sorry about the misunderstanding -- no malice intended. Version history indicates that you were the one bringing Calc into the Emacs tree and did quite some work on it at the time, so it would be daft to assume that you didn't have good knowledge of the code. In other words, there may very well be something you know about it that I could learn.

However, I understand that you have made up your mind. Sorry about wasting your time.

(Moving Calc to ELPA would probably be beneficial to everyone, but given the inevitable opposition to such a move, nothing I would propose seriously.)

Re: Emacs 27.1 released

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Fri, 14 Aug 2020 14:51:31 +0200
> Cc: emacs-devel@gnu.org
> 
> 14 aug. 2020 kl. 08.02 skrev Eli Zaretskii <eliz@gnu.org>:
> 
> > We are assessing this from two very different points of view: you are
> > thinking about fixing the bug, and I'm thinking about the dangers of
> > some unintended consequences of the fix destabilizing Emacs 27.2's
> > version of Calc.  By their very nature, unintended consequences are
> > unknown to you and to me, so asking for any specific details is not
> > useful.
> 
> Sorry about the misunderstanding -- no malice intended. Version history indicates that you were the one bringing Calc into the Emacs tree and did quite some work on it at the time, so it would be daft to assume that you didn't have good knowledge of the code. In other words, there may very well be something you know about it that I could learn.

I actually didn't consider the code at all in this case, I tried to
approach the issue from the POV of risk management.

> However, I understand that you have made up your mind.

If you can look at the issue from my POV, and provide arguments
relevant to the risks of back-porting these changes, I'd appreciate
that very much, as that would allow me to have a second opinion.  What
bothers me is the risk of having the fix uncover as yet unknown
problems in the callers of the function that was fixed.  Do you have
any thoughts about that?  For example, if the callers are already
broken when the fixed function misbehaved (are they?), that would be a
strong argument to cherry-pick the changes, because they cannot
possibly break what is already broken.

IOW, I'm trying to weight the advantages of fixing a problem against
the potential dangers and risks, especially the unknown ones.  Any
help in this decision-making will be appreciated.

Re: Emacs 27.1 released

[Mattias Engdegård]

14 aug. 2020 kl. 15.28 skrev Eli Zaretskii <eliz@gnu.org>:

> I actually didn't consider the code at all in this case, I tried to
> approach the issue from the POV of risk management.

I understand what you mean, but it is not very different from what a maintainer fixing a bug does; assessing consequences and risks is part of the job.

> What bothers me is the risk of having the fix uncover as yet unknown
> problems in the callers of the function that was fixed.  Do you have
> any thoughts about that?  For example, if the callers are already
> broken when the fixed function misbehaved (are they?), that would be a
> strong argument to cherry-pick the changes, because they cannot
> possibly break what is already broken.

Well, there is no indication that callers of calcFunc-gcd are of particularly low quality. Solid tests would have helped, of course: were Calc written today we would have insisted on a very different degree of test coverage, but that was not the standard coding practice at the time. Without spending the time to build a full test suite for the considerable amount of functionality of Calc, the best we can do is to fix bugs using careful analysis and good habits and add well-targeted tests for the flaw and nearby code without succumbing to exponential explosion. We then sit back and wait until we believe time has tested it.

On the other hand, Calc is fairly easy to validate in the respect that it's often just a matter of doing what is mathematically correct and things will sort themselves out. Anything going wrong after fixing one part was therefore almost by definition already broken. It was not quite what you meant, but perhaps it gives a partial answer to your question.

That said, none of the bug fixes mentioned were of high importance. The way Emacs development works, it's likely better to encourage users to build and use Emacs master rather than back-porting anything to a stable branch.

Re: Emacs 27.1 released

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> Does this mean the emacs-27 branch is reopened for patches?
>
> Yes, but please only install bugfixes which are either safe enough for
> the release branch, or very urgent, or fix regressions introduced by
> Emacs 26 or 27.1.  Other changes should go to master.

All these changes are collected for being included into Emacs 27.2. These
are almost bugs reported via debbugs, or on the Tramp ML. Changes which
are not safe for Emacs 27.2, and all new Tramp features, do not belong
to this patch set.

These changes are already included in the master branch, so we know a
little bit about their stability. And they are also propagated via GNU
ELPA; people who have reported bugs could check the solution.

> Thanks.

Best regards, Michael.

Re: Emacs 27.1 released

[phillip.lord]

Binaries for Windows have been placed on alpha.

https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-27/

Once a few people have confirmed that they are working okay, I will 
promote them to main.

Phil


On 2020-08-11 00:00, Nicolas Petton wrote:
> Hi!
> 
> Version 27.1 of the Emacs text editor is now available.
> 
> For more information on Emacs, see:
>   https://www.gnu.org/software/emacs
> 
> You can retrieve the source from your nearest GNU mirror by using one
> of the following links:
>   https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.xz
>   https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.gz
> 
> You can get the PGP signatures at
>   https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.xz.sig
>   https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.gz.sig
> 
> The tarball is signed with the following GPG key, which can be found on
> public PGP key servers:
> 
>   D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978
> 
> To retrieve the key from a PGP key server, evaluate
> 
>   gpg --keyserver hkp://keys.gnupg.net --recv-keys
> D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978
> 
> You can choose a mirror explicitly from the list at:
>   https://www.gnu.org/prep/ftp.html
> 
> Mirrors may take some time to update; the main GNU ftp server is at:
>   https://ftp.gnu.org/gnu/emacs/
> 
> Emacs 27.1 has a wide variety of new features, including:
> 
>   - Built-in support for arbitrary-size integers
>   - Text shaping with HarfBuzz
>   - Native support for JSON parsing
>   - Better support for Cairo drawing
>   - Portable dumping used instead of unexec
>   - Support for XDG conventions for init files
>   - Additional early-init initialization file
>   - Lexical-binding is used by default
>   - Built-in support for tab bar and tab-line
>   - Support for resizing and rotating of images without ImageMagick
> 
> There are many more changes; for a summary see the etc/NEWS file, which
> you can view from Emacs with `C-h n'.
> 
> For the complete list of changes and the people who made them, see the
> various ChangeLog files in the source distribution.  For a summary of
> all the people who have contributed to Emacs, see the etc/AUTHORS file.
> 
> The online manuals and website will be updated shortly.
> 
> Printed copies of the Emacs manual are available for purchase from the
> Free Software Foundation's online store at:
> https://shop.fsf.org/product/emacs-manual/
> 
> Regards,
> Nico

Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

Just to follow up on this, I have put windows binaries for Emacs 27.1 
onto alpha. I am hoping for someone to install and try them before I 
promote to the main release site. I don't use windows myself and can do 
no more than cursory "does it launch" testing.

If anyone has used them, can you respond to this message!

Phil



On 2020-08-12 13:01, phillip.lord@russet.org.uk wrote:
> Binaries for Windows have been placed on alpha.
> 
> https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-27/
> 
> Once a few people have confirmed that they are working okay, I will
> promote them to main.
> 
> Phil
> 
> 
> On 2020-08-11 00:00, Nicolas Petton wrote:
>> Hi!
>> 
>> Version 27.1 of the Emacs text editor is now available.
>> 
>> For more information on Emacs, see:
>>   https://www.gnu.org/software/emacs
>> 
>> You can retrieve the source from your nearest GNU mirror by using one
>> of the following links:
>>   https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.xz
>>   https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.gz
>> 
>> You can get the PGP signatures at
>>   https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.xz.sig
>>   https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.gz.sig
>> 
>> The tarball is signed with the following GPG key, which can be found 
>> on
>> public PGP key servers:
>> 
>>   D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978
>> 
>> To retrieve the key from a PGP key server, evaluate
>> 
>>   gpg --keyserver hkp://keys.gnupg.net --recv-keys
>> D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978
>> 
>> You can choose a mirror explicitly from the list at:
>>   https://www.gnu.org/prep/ftp.html
>> 
>> Mirrors may take some time to update; the main GNU ftp server is at:
>>   https://ftp.gnu.org/gnu/emacs/
>> 
>> Emacs 27.1 has a wide variety of new features, including:
>> 
>>   - Built-in support for arbitrary-size integers
>>   - Text shaping with HarfBuzz
>>   - Native support for JSON parsing
>>   - Better support for Cairo drawing
>>   - Portable dumping used instead of unexec
>>   - Support for XDG conventions for init files
>>   - Additional early-init initialization file
>>   - Lexical-binding is used by default
>>   - Built-in support for tab bar and tab-line
>>   - Support for resizing and rotating of images without ImageMagick
>> 
>> There are many more changes; for a summary see the etc/NEWS file, 
>> which
>> you can view from Emacs with `C-h n'.
>> 
>> For the complete list of changes and the people who made them, see the
>> various ChangeLog files in the source distribution.  For a summary of
>> all the people who have contributed to Emacs, see the etc/AUTHORS 
>> file.
>> 
>> The online manuals and website will be updated shortly.
>> 
>> Printed copies of the Emacs manual are available for purchase from the
>> Free Software Foundation's online store at:
>> https://shop.fsf.org/product/emacs-manual/
>> 
>> Regards,
>> Nico

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Sat, 15 Aug 2020 18:49:03 +0100
> From: phillip.lord@russet.org.uk
> 
> Just to follow up on this, I have put windows binaries for Emacs 27.1 
> onto alpha. I am hoping for someone to install and try them before I 
> promote to the main release site. I don't use windows myself and can do 
> no more than cursory "does it launch" testing.
> 
> If anyone has used them, can you respond to this message!

Somebody did, see bug#42844.

Thanks.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Stephen Leake]

phillip.lord@russet.org.uk writes:

> Just to follow up on this, I have put windows binaries for Emacs 27.1
> onto alpha. I am hoping for someone to install and try them before I 
> promote to the main release site. I don't use windows myself and can
> do no more than cursory "does it launch" testing.
>
> If anyone has used them, can you respond to this message!

I used the 64 bit installer; seems to work, I'm using it to reply to
this message.


-- 
-- Stephe

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Alan Third]

On Sat, Aug 15, 2020 at 06:49:03PM +0100, phillip.lord@russet.org.uk wrote:
> 
> Just to follow up on this, I have put windows binaries for Emacs 27.1 onto
> alpha. I am hoping for someone to install and try them before I promote to
> the main release site. I don't use windows myself and can do no more than
> cursory "does it launch" testing.
> 
> If anyone has used them, can you respond to this message!

I installed the 64 bit zip version on my work PC on Friday. I haven't
done much real work with it, but it was certainly fine for what I was
doing.
-- 
Alan Third

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Sivaram Neelakantan]

On Sat, Aug 15 2020,phillip.lord@russet.org.uk nil wrote:

> Just to follow up on this, I have put windows binaries for Emacs 27.1
> onto alpha. I am hoping for someone to install and try them before I 
> promote to the main release site. I don't use windows myself and can do
> no more than cursory "does it launch" testing.
>
> If anyone has used them, can you respond to this message!
>
> Phil
>
>
>

[snipped 57 lines]

Seems to work so far without any issues.  That is, if you see this
message.  And thanks for the binaries.

sivaram
-- 

Re:Emacs 27.1 Windows Binaries -- testing wanted

[范凯]

I downloaded the 64bit installer, and it works for me without any problem.<br/><br/>(version)<br/>"GNU Emacs 27.1 (build 1, x86_64-w64-mingw32)<br/> of 2020-08-12"<br/><br/>Thanks,<br/>Kai
At 2020-08-16 01:49:03, phillip.lord@russet.org.uk wrote:
>
>Just to follow up on this, I have put windows binaries for Emacs 27.1 
>onto alpha. I am hoping for someone to install and try them before I 
>promote to the main release site. I don't use windows myself and can do 
>no more than cursory "does it launch" testing.
>
>If anyone has used them, can you respond to this message!
>
>Phil
>
>
>
>On 2020-08-12 13:01, phillip.lord@russet.org.uk wrote:
>> Binaries for Windows have been placed on alpha.
>> 
>> https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-27/
>> 
>> Once a few people have confirmed that they are working okay, I will
>> promote them to main.
>> 
>> Phil
>> 
>> 
>> On 2020-08-11 00:00, Nicolas Petton wrote:
>>> Hi!
>>> 
>>> Version 27.1 of the Emacs text editor is now available.
>>> 
>>> For more information on Emacs, see:
>>>   https://www.gnu.org/software/emacs
>>> 
>>> You can retrieve the source from your nearest GNU mirror by using one
>>> of the following links:
>>>   https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.xz
>>>   https://ftpmirror.gnu.org/emacs/emacs-27.1.tar.gz
>>> 
>>> You can get the PGP signatures at
>>>   https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.xz.sig
>>>   https://ftp.gnu.org/gnu/emacs/emacs-27.1.tar.gz.sig
>>> 
>>> The tarball is signed with the following GPG key, which can be found 
>>> on
>>> public PGP key servers:
>>> 
>>>   D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978
>>> 
>>> To retrieve the key from a PGP key server, evaluate
>>> 
>>>   gpg --keyserver hkp://keys.gnupg.net --recv-keys
>>> D405AA2C862C54F17EEE6BE0E8BCD7866AFCF978
>>> 
>>> You can choose a mirror explicitly from the list at:
>>>   https://www.gnu.org/prep/ftp.html
>>> 
>>> Mirrors may take some time to update; the main GNU ftp server is at:
>>>   https://ftp.gnu.org/gnu/emacs/
>>> 
>>> Emacs 27.1 has a wide variety of new features, including:
>>> 
>>>   - Built-in support for arbitrary-size integers
>>>   - Text shaping with HarfBuzz
>>>   - Native support for JSON parsing
>>>   - Better support for Cairo drawing
>>>   - Portable dumping used instead of unexec
>>>   - Support for XDG conventions for init files
>>>   - Additional early-init initialization file
>>>   - Lexical-binding is used by default
>>>   - Built-in support for tab bar and tab-line
>>>   - Support for resizing and rotating of images without ImageMagick
>>> 
>>> There are many more changes; for a summary see the etc/NEWS file, 
>>> which
>>> you can view from Emacs with `C-h n'.
>>> 
>>> For the complete list of changes and the people who made them, see the
>>> various ChangeLog files in the source distribution.  For a summary of
>>> all the people who have contributed to Emacs, see the etc/AUTHORS 
>>> file.
>>> 
>>> The online manuals and website will be updated shortly.
>>> 
>>> Printed copies of the Emacs manual are available for purchase from the
>>> Free Software Foundation's online store at:
>>> https://shop.fsf.org/product/emacs-manual/
>>> 
>>> Regards,
>>> Nico

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Tak Kunihiro]

I downloaded emacs-27.1-x86_64.zip and it is good for a full day.
Thank you for the binary creation.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Jonathan Mitchell]

[-- Attachment #1: Type: text/plain, Size: 164 bytes --]

I tested the emacs-27.1-x86_64.zip and .exe installer and both worked fine
except that C-u C-x = on any text showed the font backend as uniscribe and
not harfbuzz.

[-- Attachment #2: Type: text/html, Size: 188 bytes --]

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> From: Jonathan Mitchell <kyle@jonathanmitchell.org>
> Date: Mon, 17 Aug 2020 21:05:24 -0500
> Cc: Tak Kunihiro <homeros.misasa@gmail.com>, tkk@misasa.okayama-u.ac.jp,
>  Emacs Devel <emacs-devel@gnu.org>
> 
> I tested the emacs-27.1-x86_64.zip and .exe installer and both worked fine except that C-u C-x = on any text
> showed the font backend as uniscribe and not harfbuzz. 

Thanks.  Can you use the dependency walker or similar tool to see if
all the dependencies of libhafbuzz-0.dll are present in the same
directory as libhafbuzz-0.dll itself?

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-18 05:55, Eli Zaretskii wrote:
>> From: Jonathan Mitchell <kyle@jonathanmitchell.org>
>> Date: Mon, 17 Aug 2020 21:05:24 -0500
>> Cc: Tak Kunihiro <homeros.misasa@gmail.com>, 
>> tkk@misasa.okayama-u.ac.jp,
>>  Emacs Devel <emacs-devel@gnu.org>
>> 
>> I tested the emacs-27.1-x86_64.zip and .exe installer and both worked 
>> fine except that C-u C-x = on any text
>> showed the font backend as uniscribe and not harfbuzz.
> 
> Thanks.  Can you use the dependency walker or similar tool to see if
> all the dependencies of libhafbuzz-0.dll are present in the same
> directory as libhafbuzz-0.dll itself?

I really need something to test the windows binaries in a more 
systematic fashion, so we can at least test all of the dependencies in a 
running Emacs.

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Jonathan Mitchell]

[-- Attachment #1.1: Type: text/plain, Size: 1224 bytes --]

I put together a quick bash script that scrapes the output of ldd in an
MSYS2 environment. Running it gave me the attached list of 56 mingw64 DLLs.

After copying those DLLs from c:\msys64\mingw64\bin\ to the bin directory
of the "no-deps" Emacs install folder, C-u C-x = is reporting Harfbuzz is
in use. That seems to have solved my problem.

On Tue, Aug 18, 2020 at 1:56 AM <phillip.lord@russet.org.uk> wrote:

> On 2020-08-18 05:55, Eli Zaretskii wrote:
> >> From: Jonathan Mitchell <kyle@jonathanmitchell.org>
> >> Date: Mon, 17 Aug 2020 21:05:24 -0500
> >> Cc: Tak Kunihiro <homeros.misasa@gmail.com>,
> >> tkk@misasa.okayama-u.ac.jp,
> >>  Emacs Devel <emacs-devel@gnu.org>
> >>
> >> I tested the emacs-27.1-x86_64.zip and .exe installer and both worked
> >> fine except that C-u C-x = on any text
> >> showed the font backend as uniscribe and not harfbuzz.
> >
> > Thanks.  Can you use the dependency walker or similar tool to see if
> > all the dependencies of libhafbuzz-0.dll are present in the same
> > directory as libhafbuzz-0.dll itself?
>
> I really need something to test the windows binaries in a more
> systematic fashion, so we can at least test all of the dependencies in a
> running Emacs.
>
> Phil
>
>

[-- Attachment #1.2: Type: text/html, Size: 1957 bytes --]

[-- Attachment #2: emacs-dll-list.txt --]
[-- Type: text/plain, Size: 963 bytes --]

libbrotlicommon.dll
libbrotlidec.dll
libbz2-1.dll
libcairo-2.dll
libcairo-gobject-2.dll
libdatrie-1.dll
libexpat-1.dll
libffi-7.dll
libfontconfig-1.dll
libfreetype-6.dll
libfribidi-0.dll
libgcc_s_seh-1.dll
libgdk_pixbuf-2.0-0.dll
libgif-7.dll
libgio-2.0-0.dll
libglib-2.0-0.dll
libgmodule-2.0-0.dll
libgmp-10.dll
libgnutls-30.dll
libgnutlsxx-28.dll
libgobject-2.0-0.dll
libgraphite2.dll
libharfbuzz-0.dll
libharfbuzz-gobject-0.dll
libharfbuzz-icu-0.dll
libharfbuzz-subset-0.dll
libhogweed-6.dll
libiconv-2.dll
libidn2-0.dll
libintl-8.dll
libjansson-4.dll
libjpeg-8.dll
liblcms2-2.dll
liblzma-5.dll
libnettle-8.dll
libp11-kit-0.dll
libpango-1.0-0.dll
libpangocairo-1.0-0.dll
libpangoft2-1.0-0.dll
libpangowin32-1.0-0.dll
libpcre-1.dll
libpixman-1-0.dll
libpng16-16.dll
librsvg-2-2.dll
libssp-0.dll
libstdc++-6.dll
libtasn1-6.dll
libthai-0.dll
libtiff-5.dll
libtiffxx-5.dll
libunistring-2.dll
libwinpthread-1.dll
libxml2-2.dll
libXpm-noX4.dll
libzstd.dll
zlib1.dll

[-- Attachment #3: emacsdeps.sh --]
[-- Type: application/x-shellscript, Size: 771 bytes --]

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Tue, 18 Aug 2020 07:56:42 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> I really need something to test the windows binaries in a more 
> systematic fashion, so we can at least test all of the dependencies in a 
> running Emacs.

For images, verify that (image-type-available-p 'TYPE) returns non-nil
for all the supported TYPEs.  For other optional features, similar
functions generally exist that can be probed; let me know if you need
more detailed advice.

Thanks.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> For images, verify that (image-type-available-p 'TYPE) returns non-nil
> for all the supported TYPEs.  For other optional features, similar
> functions generally exist that can be probed; let me know if you need
> more detailed advice.

A function that checks and shows the availability of every possible
optional feature would be handy.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 18 Aug 2020 17:57:05 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > For images, verify that (image-type-available-p 'TYPE) returns non-nil
> > for all the supported TYPEs.  For other optional features, similar
> > functions generally exist that can be probed; let me know if you need
> > more detailed advice.
> 
> A function that checks and shows the availability of every possible
> optional feature would be handy.

Many of them already have such a function.  Some need alternative
testing methods.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Tue, 18 Aug 2020 17:57:05 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > For images, verify that (image-type-available-p 'TYPE) returns non-nil
>> > for all the supported TYPEs.  For other optional features, similar
>> > functions generally exist that can be probed; let me know if you need
>> > more detailed advice.
>> 
>> A function that checks and shows the availability of every possible
>> optional feature would be handy.
>
> Many of them already have such a function.  Some need alternative
> testing methods.

Yes, but I'm talking about a unique function that displays the
availability of all optional features, so the user can see at a glance
what is available (and what is missing) on his install.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 18 Aug 2020 18:48:47 +0200
> 
> >> A function that checks and shows the availability of every possible
> >> optional feature would be handy.
> >
> > Many of them already have such a function.  Some need alternative
> > testing methods.
> 
> Yes, but I'm talking about a unique function that displays the
> availability of all optional features, so the user can see at a glance
> what is available (and what is missing) on his install.

You are talking about a w32-specific feature, yes?  Because on Posix
hosts compiled-in optional libraries aren't loaded at run time, and
are thus always available.  So I think it's unlikely we will have such
a feature.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Stefan Monnier]

> You are talking about a w32-specific feature, yes?  Because on Posix
> hosts compiled-in optional libraries aren't loaded at run time, and
> are thus always available.  So I think it's unlikely we will have such
> a feature.

FWIW, it might still be useful on posix hosts to show the set of
features that were compiled-in vs those that were not.
That info would be likely redundant with the end of the output of
`./configure`, but that doesn't necessarily make it useless.


        Stefan

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Óscar Fuentes <ofv@wanadoo.es>,
>   emacs-devel@gnu.org
> Date: Tue, 18 Aug 2020 15:32:19 -0400
> 
> FWIW, it might still be useful on posix hosts to show the set of
> features that were compiled-in vs those that were not.

We already have that, via the variables that were mentioned up-thread.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Stefan Monnier]

>> FWIW, it might still be useful on posix hosts to show the set of
>> features that were compiled-in vs those that were not.
> We already have that, via the variables that were mentioned up-thread.

I don't think those variables show directly what was *not* compiled in.


        Stefan

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 19 Aug 2020 09:10:53 -0400
> Cc: ofv@wanadoo.es, emacs-devel@gnu.org
> 
> >> FWIW, it might still be useful on posix hosts to show the set of
> >> features that were compiled-in vs those that were not.
> > We already have that, via the variables that were mentioned up-thread.
> 
> I don't think those variables show directly what was *not* compiled in.

Everything that is not in the list, obviously.  (I feel that we are
miscommunicating here.)

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Stefan Monnier]

>> >> FWIW, it might still be useful on posix hosts to show the set of
>> >> features that were compiled-in vs those that were not.
>> > We already have that, via the variables that were mentioned up-thread.
>> I don't think those variables show directly what was *not* compiled in.
> Everything that is not in the list, obviously.  (I feel that we are
> miscommunicating here.)

I think most users don't know the complete set of things that could
potentially be in the list.  Hence the usefulness of listing those
things that aren't compiled in.


        Stefan

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: ofv@wanadoo.es,  emacs-devel@gnu.org
> Date: Wed, 19 Aug 2020 11:30:58 -0400
> 
> I think most users don't know the complete set of things that could
> potentially be in the list.  Hence the usefulness of listing those
> things that aren't compiled in.

So something like GDB's --config option?

 $ gdb --config
   This GDB was configured as follows:
      configure --host=i686-pc-mingw32 --target=i686-pc-mingw32
		--with-auto-load-dir=$debugdir;$datadir/../auto-load
		--with-auto-load-safe-path=$debugdir;$datadir/../auto-load
		--with-expat
		--with-gdb-datadir=d:/usr/share/gdb/10.1 (relocatable)
		--with-jit-reader-dir=d:/usr/lib/gdb (relocatable)
		--without-libunwind-ia64
		--with-lzma
		--without-babeltrace
		--without-intel-pt
		--with-mpfr
		--with-xxhash
		--with-python=d:/usr/Python26 (relocatable)
		--with-python-libdir=d:/usr/Python26/lib (relocatable)
		--without-debuginfod
		--with-guile
		--enable-source-highlight
		--with-separate-debug-dir=d:/usr/lib/debug (relocatable)
		--with-system-gdbinit=d:/usr/etc/gdbinit (relocatable)

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-18 17:48, Óscar Fuentes wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
>>> From: Óscar Fuentes <ofv@wanadoo.es>
>>> Date: Tue, 18 Aug 2020 17:57:05 +0200
>>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> > For images, verify that (image-type-available-p 'TYPE) returns non-nil
>>> > for all the supported TYPEs.  For other optional features, similar
>>> > functions generally exist that can be probed; let me know if you need
>>> > more detailed advice.
>>> 
>>> A function that checks and shows the availability of every possible
>>> optional feature would be handy.
>> 
>> Many of them already have such a function.  Some need alternative
>> testing methods.
> 
> Yes, but I'm talking about a unique function that displays the
> availability of all optional features, so the user can see at a glance
> what is available (and what is missing) on his install.



I need something that just displays a little report. It could show the 
presence of features (image formats), and things like font rendering 
engine, whether native-comp is active (for when this hits masters). Also 
be nice to know if Emacs things it is a snapshot, or release, whether 
the binary is stripped or not, what optimization levels have been used.

All of these things have been a problem at one time or another or might 
be in future, and the build for release happens rarely enough, with 
small changes from one release to the next (for example, this time you 
updated the version number before I did the build which broke my scripts 
assumptions just a little), that I have to do small tweaks my hand. Then 
I have no ability to check I have not screwed up.

As you say, mostly useful for w32, as it's the only binary Gnu provides. 
Might be useful for other people doing binaries packages for other OS 
maybe.

I will try and write something.

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Daniel Brooks]

phillip.lord@russet.org.uk writes:

> I need something that just displays a little report. It could show the
> presence of features (image formats), and things like font rendering
> engine, whether native-comp is active (for when this hits
> masters). Also be nice to know if Emacs things it is a snapshot, or
> release, whether the binary is stripped or not, what optimization
> levels have been used.

If you'd like inspiration, Firefox does a pretty decent job at this;
just visit about:buildconfig to see. The source link is especially
useful, and it also explicitly lists the configure options so that you
can recreate the build quite exactly.

db48x

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> From: Daniel Brooks <db48x@db48x.net>
> Date: Tue, 18 Aug 2020 11:11:19 -0700
> 
> If you'd like inspiration, Firefox does a pretty decent job at this;
> just visit about:buildconfig to see.

We have system-configuration-features, which shows the equivalent
info.

But this isn't Phillip's problem: the problem here is that the Windows
build of Emacs tries to load the optional libraries when they are
first required, and if the load fails, the corresponding feature will
behave as not available, even though Emacs was built to support that
feature.

IOW, the problem here is not to know how Emacs was built, but what
optional libraries are available to it at run time on the end-user's
machine.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Daniel Brooks]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Daniel Brooks <db48x@db48x.net>
>> Date: Tue, 18 Aug 2020 11:11:19 -0700
>> 
>> If you'd like inspiration, Firefox does a pretty decent job at this;
>> just visit about:buildconfig to see.
>
> We have system-configuration-features, which shows the equivalent
> info.

And system-configuration-options, though as far as I know there's no git
commit hash.

> But this isn't Phillip's problem: the problem here is that the Windows
> build of Emacs tries to load the optional libraries when they are
> first required, and if the load fails, the corresponding feature will
> behave as not available, even though Emacs was built to support that
> feature.

That is a fun wrinkle, and I hadn't understood it. Sounds like his
report will be more like Firefox's about:support than about:buildconfig.

db48x

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Tue, 18 Aug 2020 18:20:17 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> I need something that just displays a little report. It could show the 
> presence of features (image formats), and things like font rendering 
> engine, whether native-comp is active (for when this hits masters). Also 
> be nice to know if Emacs things it is a snapshot, or release, whether 
> the binary is stripped or not, what optimization levels have been used.

Shouldn't be a problem to write such a function, perhaps as part of
admin/admin.el?

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

[-- Attachment #1: Type: text/plain, Size: 1985 bytes --]

On 2020-08-18 18:20, phillip.lord@russet.org.uk wrote:
> On 2020-08-18 17:48, Óscar Fuentes wrote:
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> Yes, but I'm talking about a unique function that displays the
>> availability of all optional features, so the user can see at a glance
>> what is available (and what is missing) on his install.
> 
> 
> 
> I need something that just displays a little report. It could show the
> presence of features (image formats), and things like font rendering
> engine, whether native-comp is active (for when this hits masters).
> Also be nice to know if Emacs things it is a snapshot, or release,
> whether the binary is stripped or not, what optimization levels have
> been used.
> 
> All of these things have been a problem at one time or another or
> might be in future, and the build for release happens rarely enough,
> with small changes from one release to the next (for example, this
> time you updated the version number before I did the build which broke
> my scripts assumptions just a little), that I have to do small tweaks
> my hand. Then I have no ability to check I have not screwed up.
> 
> As you say, mostly useful for w32, as it's the only binary Gnu
> provides. Might be useful for other people doing binaries packages for
> other OS maybe.
> 
> I will try and write something.


As a starter for 10, this is a file that tests features. The idea would 
be to include it in the main (not test) lisp hierarchy, probably with a 
single autoloaded command "feature-test" or something which would run 
ert with an appropriate selector. This would make it available for use 
in any Emacs distribution without having to include any files only found 
in the repo.

Obviously more features to go. I haven't worked out how to test the 
existence of harfbuzz yet. I guess I need to test everything with a 
"--without-" configuration option that is relevant on windows.

Thoughts? Installable to emacs-27? As EMACS/lisp/feature.el?

Phil

[-- Attachment #2: feature.el --]
[-- Type: text/plain, Size: 828 bytes --]

(require 'ert)

(ert-deftest feature-gnutls ()
  (should (gnutls-available-p)))

(ert-deftest feature-json ()
  (should
   (progn
     (require 'json)
     (fboundp json-serialize))))

(ert-deftest feature-pbm ()
  (should (image-type-available-p 'pbm)))

(ert-deftest feature-xpm ()
  (should (image-type-available-p 'xpm)))

(ert-deftest feature-bmp ()
  (should (image-type-available-p 'bmp)))

(ert-deftest feature-gif ()
  (should (image-type-available-p 'gif)))

(ert-deftest feature-png ()
  (should (image-type-available-p 'png)))

(ert-deftest feature-xpm ()
  (should (image-type-available-p 'xpm)))

(ert-deftest feature-jpeg ()
  (should (image-type-available-p 'jpeg)))

(ert-deftest feature-tiff ()
  (should (image-type-available-p 'tiff)))

(ert-deftest feature-svg ()
  (should (image-type-available-p 'svg)))

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Fri, 21 Aug 2020 20:16:41 +0100
> From: phillip.lord@russet.org.uk
> 
> As a starter for 10

Who or what is "10"?

> this is a file that tests features. The idea would 
> be to include it in the main (not test) lisp hierarchy, probably with a 
> single autoloaded command "feature-test" or something which would run 
> ert with an appropriate selector. This would make it available for use 
> in any Emacs distribution without having to include any files only found 
> in the repo.
> 
> Obviously more features to go. I haven't worked out how to test the 
> existence of harfbuzz yet.

  (car (frame-parameter nil 'font-backend))

> I guess I need to test everything with a 
> "--without-" configuration option that is relevant on windows.
> 
> Thoughts? Installable to emacs-27? As EMACS/lisp/feature.el?

I don't understand why this has to be ert tests.  Why not just a
simple function that performs a series of tests and produces a report
about each test?

Also, shouldn't it be in admin/nt?  It's w32-specific, right?  What is
the rationale for having it in lisp/ ?

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-21 20:44, Eli Zaretskii wrote:
>> Date: Fri, 21 Aug 2020 20:16:41 +0100
>> From: phillip.lord@russet.org.uk
>> 
>> As a starter for 10
> 
> Who or what is "10"?

Sorry, British idiom. It means much the same as "strawman".


>> this is a file that tests features. The idea would
>> be to include it in the main (not test) lisp hierarchy, probably with 
>> a
>> single autoloaded command "feature-test" or something which would run
>> ert with an appropriate selector. This would make it available for use
>> in any Emacs distribution without having to include any files only 
>> found
>> in the repo.
>> 
>> Obviously more features to go. I haven't worked out how to test the
>> existence of harfbuzz yet.
> 
>   (car (frame-parameter nil 'font-backend))
> 
>> I guess I need to test everything with a
>> "--without-" configuration option that is relevant on windows.
>> 
>> Thoughts? Installable to emacs-27? As EMACS/lisp/feature.el?
> 
> I don't understand why this has to be ert tests.  Why not just a
> simple function that performs a series of tests and produces a report
> about each test?

It doesn't have to be ert tests, but ert is designed to perform a series 
of tests and produce a report about each test, so it makes sense.


> Also, shouldn't it be in admin/nt?  It's w32-specific, right?  What is
> the rationale for having it in lisp/ ?

Because I want to be able to unpack a windows distribution and run the 
function. admin isn't included in the distribution (as far as I can 
tell), just the repository. I can do this by hand, but then it's easier 
to make mistakes. I'd like to be able to start Emacs, run M-x 
feature-test, and if everything passes all is good.

If this offends your sense of cleanliness, which I'd understand, it 
could equally go in `data-directory'.

Phil

RE: Emacs 27.1 Windows Binaries -- testing wanted

[Drew Adams]

> > Who or what is "10"?
> 
> Sorry, British idiom. It means much the same as "strawman".

Had to google it:

https://www.phrases.org.uk/bulletin_board/41/messages/640.html

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Fri, 21 Aug 2020 22:37:12 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> > Also, shouldn't it be in admin/nt?  It's w32-specific, right?  What is
> > the rationale for having it in lisp/ ?
> 
> Because I want to be able to unpack a windows distribution and run the 
> function.

OK, but then, as long as it's w32-specific, it should go into
w32fns.el or something.

OTOH, if we want this as a general-purpose feature (along the lines
proposed by Stefan), then yes, something like lisp/feature.el should
be appropriate.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-22 07:46, Eli Zaretskii wrote:
>> Date: Fri, 21 Aug 2020 22:37:12 +0100
>> From: phillip.lord@russet.org.uk
>> Cc: emacs-devel@gnu.org
>> 
>> > Also, shouldn't it be in admin/nt?  It's w32-specific, right?  What is
>> > the rationale for having it in lisp/ ?
>> 
>> Because I want to be able to unpack a windows distribution and run the
>> function.
> 
> OK, but then, as long as it's w32-specific, it should go into
> w32fns.el or something.

Okay. I will add lisp/w32-feature.el (I don't want to add it to 
w32-fns.el as this would force loading of ert). I won't add a command so 
that no one will run it by mistake, but will add an autoloaded function.


> OTOH, if we want this as a general-purpose feature (along the lines
> proposed by Stefan), then yes, something like lisp/feature.el should
> be appropriate.

That might be useful for others who maintain binary distributions of 
Emacs, but I don't know what they would need or want. So, I'll add 
w32-feature.el to Emacs-27. If we need something more generic, I'd be 
happy to migrate to that.

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Robert Pluim]

>>>>> On Fri, 21 Aug 2020 22:44:18 +0300, Eli Zaretskii <eliz@gnu.org> said:
    >> Obviously more features to go. I haven't worked out how to test the 
    >> existence of harfbuzz yet.

    Eli>   (car (frame-parameter nil 'font-backend))

There are vendor-provided versions of Emacs that mess with
font-backend and/or FontBackend, and the user may have changed it
themselves, so thatʼs not going to work. Perhaps we need a
`harfbuzz-available-p` defun?

Robert

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: phillip.lord@russet.org.uk,  emacs-devel@gnu.org
> Date: Mon, 24 Aug 2020 11:28:43 +0200
> 
> >>>>> On Fri, 21 Aug 2020 22:44:18 +0300, Eli Zaretskii <eliz@gnu.org> said:
>     >> Obviously more features to go. I haven't worked out how to test the 
>     >> existence of harfbuzz yet.
> 
>     Eli>   (car (frame-parameter nil 'font-backend))
> 
> There are vendor-provided versions of Emacs that mess with
> font-backend and/or FontBackend, and the user may have changed it
> themselves, so thatʼs not going to work.

I'm not sure I understand the "mess" part.  Can you show an example of
what such "messing" produces in the simple recipe I suggested above?

Also, I'm guessing those vendors don't touch the Windows builds, do
they?

> Perhaps we need a `harfbuzz-available-p` defun?

We could add that if there's a reason good enough.  The advantage of
what I proposed is that it also detects the cases where HarfBuzz is
available, but for some reason not used.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Robert Pluim]

>>>>> On Mon, 24 Aug 2020 13:40:27 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    >> Cc: phillip.lord@russet.org.uk,  emacs-devel@gnu.org
    >> Date: Mon, 24 Aug 2020 11:28:43 +0200
    >> 
    >> >>>>> On Fri, 21 Aug 2020 22:44:18 +0300, Eli Zaretskii <eliz@gnu.org> said:
    >> >> Obviously more features to go. I haven't worked out how to test the 
    >> >> existence of harfbuzz yet.
    >> 
    Eli> (car (frame-parameter nil 'font-backend))
    >> 
    >> There are vendor-provided versions of Emacs that mess with
    >> font-backend and/or FontBackend, and the user may have changed it
    >> themselves, so thatʼs not going to work.

    Eli> I'm not sure I understand the "mess" part.  Can you show an example of
    Eli> what such "messing" produces in the simple recipe I suggested above?

OpenSuse [1] puts 'FontBackend: xft,x' in the global Xresources file
used by their emacs-27 package, even though the build itself is a
Cairo + HarfBuzz build, so produces 'x' for the snippets above (since
thereʼs no xft backend in such an emacs).

    Eli> Also, I'm guessing those vendors don't touch the Windows builds, do
    Eli> they?

True. For Windows your code is much more likely to work, but is still
subject to the user changing font-backend.

    >> Perhaps we need a `harfbuzz-available-p` defun?

    Eli> We could add that if there's a reason good enough.  The advantage of
    Eli> what I proposed is that it also detects the cases where HarfBuzz is
    Eli> available, but for some reason not used.

Only from 'emacs -Q'

Robert

Footnotes:
[1]  I believe they've now fixed this

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Stefan Monnier]

> The advantage of what I proposed is that it also detects the cases
> where HarfBuzz is available, but for some reason not used.

That can be useful in some cases, but I think in the present case it's
a disadvantage: we want to know what the executable (and installed libs)
provide, rather than what the ~/.emacs chose to use.


        Stefan

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Robert Pluim <rpluim@gmail.com>,  emacs-devel@gnu.org,
>   phillip.lord@russet.org.uk
> Date: Mon, 24 Aug 2020 11:14:47 -0400
> 
> > The advantage of what I proposed is that it also detects the cases
> > where HarfBuzz is available, but for some reason not used.
> 
> That can be useful in some cases, but I think in the present case it's
> a disadvantage: we want to know what the executable (and installed libs)
> provide, rather than what the ~/.emacs chose to use.

No, we want to know what the executable and its environment, as
provided in the bundle, will yield at run time.

I agree that it might catch some irrelevant circumstances, but that's
inevitable, at least if such simple tests are being sought.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Stefan Monnier]

> No, we want to know what the executable and its environment, as
> provided in the bundle, will yield at run time.
>
> I agree that it might catch some irrelevant circumstances, but that's
> inevitable, at least if such simple tests are being sought.

Then I guess we disagree on the purpose.

For me, the purpose of such a list of features is to help the user
figure out whether a given problem comes from their config or from the
Emacs build itself.


        Stefan

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-24 17:49, Stefan Monnier wrote:
>> No, we want to know what the executable and its environment, as
>> provided in the bundle, will yield at run time.
>> 
>> I agree that it might catch some irrelevant circumstances, but that's
>> inevitable, at least if such simple tests are being sought.
> 
> Then I guess we disagree on the purpose.
> 
> For me, the purpose of such a list of features is to help the user
> figure out whether a given problem comes from their config or from the
> Emacs build itself.

Both of these purposes are useful. For my particular use case, it's the 
same thing, because I build Emacs in an environment kept specially for 
that purpose and do not have any config on that machine.

 From my perspective what I really care about, though, is getting 
something that can do some basic testing of my windows builds so I can 
(finally) get 27.1 out, and also future releases with some confidence 
that I haven't screwed up. If we can fulfil both use-cases with one 
library that's great; but I only have time to worry about one of these 
at the moment.

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rpluim@gmail.com,  emacs-devel@gnu.org,  phillip.lord@russet.org.uk
> Date: Mon, 24 Aug 2020 12:49:26 -0400
> 
> For me, the purpose of such a list of features is to help the user
> figure out whether a given problem comes from their config or from the
> Emacs build itself.

I don't see how this can be done in principle.  There are too many
factors that affect the final outcome, and telling which ones are due
to the build and which are due to the run-time environment not
directly related to the build, is night impossible, certainly by
running a few naïve tests.

I believe that you are still thinking about Posix platforms, where any
feature compiled in should be automatically available, but this thread
is not about a general-purpose facility, it's about w32-specific
facility, where things are more complex.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Stephen Leake]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> No, we want to know what the executable and its environment, as
>> provided in the bundle, will yield at run time.
>>
>> I agree that it might catch some irrelevant circumstances, but that's
>> inevitable, at least if such simple tests are being sought.
>
> Then I guess we disagree on the purpose.
>
> For me, the purpose of such a list of features is to help the user
> figure out whether a given problem comes from their config or from the
> Emacs build itself.

Run it once with -Q, once with ~/.emacs.

-- 
-- Stephe

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Stefan Monnier]

>>> No, we want to know what the executable and its environment, as
>>> provided in the bundle, will yield at run time.
>>>
>>> I agree that it might catch some irrelevant circumstances, but that's
>>> inevitable, at least if such simple tests are being sought.
>>
>> Then I guess we disagree on the purpose.
>>
>> For me, the purpose of such a list of features is to help the user
>> figure out whether a given problem comes from their config or from the
>> Emacs build itself.
>
> Run it once with -Q, once with ~/.emacs.

Yes, there are a million ways to diagnose problems.  None of them work
in all cases, and none of them are ever the only possible approach.


        Stefan

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-24 16:48, Eli Zaretskii wrote:
>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: Robert Pluim <rpluim@gmail.com>,  emacs-devel@gnu.org,
>>   phillip.lord@russet.org.uk
>> Date: Mon, 24 Aug 2020 11:14:47 -0400
>> 
>> > The advantage of what I proposed is that it also detects the cases
>> > where HarfBuzz is available, but for some reason not used.
>> 
>> That can be useful in some cases, but I think in the present case it's
>> a disadvantage: we want to know what the executable (and installed 
>> libs)
>> provide, rather than what the ~/.emacs chose to use.
> 
> No, we want to know what the executable and its environment, as
> provided in the bundle, will yield at run time.
> 
> I agree that it might catch some irrelevant circumstances, but that's
> inevitable, at least if such simple tests are being sought.


At the moment, it is enough to tell me that the distribution I have 
produced actually has the features that it has been compiled with.

I've picked simple tests because they do that and simple is better than 
complicated. I'll make it complicated when it is clear that simple is 
not complicated enough.

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Alan Third]

On Fri, Aug 21, 2020 at 08:16:41PM +0100, phillip.lord@russet.org.uk wrote:
> On 2020-08-18 18:20, phillip.lord@russet.org.uk wrote:
> As a starter for 10, this is a file that tests features. The idea would be
> to include it in the main (not test) lisp hierarchy, probably with a single
> autoloaded command "feature-test" or something which would run ert with an
> appropriate selector. This would make it available for use in any Emacs
> distribution without having to include any files only found in the repo.

Feel free to ignore me, but I feel it may be more widely useful to
provide a list of all features and mark whether they're available or
not rather than run a test that reports unavailable features as
errors.

Something like this, but better:

(defun insert-feature (description test)
  (indent-to 2)
  (insert (if test "✔" "✖"))
  (indent-to 5)
  (insert description)
  (insert "\n"))

(defun list-features ()
  (interactive)
  (switch-to-buffer (get-buffer-create "*Available Features*"))
  (erase-buffer)

  (insert-feature "JSON" (progn (require 'json) (fboundp 'json-serialize)))
  (insert-feature "GNUTLS" (gnutls-available-p))
  (insert-feature "pbm" (image-type-available-p 'pbm))
  (insert-feature "xpm" (image-type-available-p 'xpm))
  (insert-feature "bmp" (image-type-available-p 'bmp))
  (insert-feature "gif" (image-type-available-p 'gif))
  (insert-feature "png" (image-type-available-p 'png))
  (insert-feature "xpm" (image-type-available-p 'xpm))
  (insert-feature "jpeg" (image-type-available-p 'jpeg))
  (insert-feature "tiff" (image-type-available-p 'tiff))
  (insert-feature "svg" (image-type-available-p 'svg))
  (insert-feature "native images" (image-type-available-p 'native-image)))

Unless of course the idea is to automate it, I suppose, in which case
this will be of no use...
-- 
Alan Third

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-21 21:15, Alan Third wrote:
> On Fri, Aug 21, 2020 at 08:16:41PM +0100, phillip.lord@russet.org.uk 
> wrote:
>> On 2020-08-18 18:20, phillip.lord@russet.org.uk wrote:
>> As a starter for 10, this is a file that tests features. The idea 
>> would be
>> to include it in the main (not test) lisp hierarchy, probably with a 
>> single
>> autoloaded command "feature-test" or something which would run ert 
>> with an
>> appropriate selector. This would make it available for use in any 
>> Emacs
>> distribution without having to include any files only found in the 
>> repo.
> 
> Feel free to ignore me, but I feel it may be more widely useful to
> provide a list of all features and mark whether they're available or
> not rather than run a test that reports unavailable features as
> errors.
> 
> Something like this, but better:
> 
> (defun insert-feature (description test)
>   (indent-to 2)
>   (insert (if test "✔" "✖"))
>   (indent-to 5)
>   (insert description)
>   (insert "\n"))
> 
> (defun list-features ()
>   (interactive)
>   (switch-to-buffer (get-buffer-create "*Available Features*"))
>   (erase-buffer)
> 
>   (insert-feature "JSON" (progn (require 'json) (fboundp 
> 'json-serialize)))
>   (insert-feature "GNUTLS" (gnutls-available-p))
>   (insert-feature "pbm" (image-type-available-p 'pbm))
>   (insert-feature "xpm" (image-type-available-p 'xpm))
>   (insert-feature "bmp" (image-type-available-p 'bmp))
>   (insert-feature "gif" (image-type-available-p 'gif))
>   (insert-feature "png" (image-type-available-p 'png))
>   (insert-feature "xpm" (image-type-available-p 'xpm))
>   (insert-feature "jpeg" (image-type-available-p 'jpeg))
>   (insert-feature "tiff" (image-type-available-p 'tiff))
>   (insert-feature "svg" (image-type-available-p 'svg))
>   (insert-feature "native images" (image-type-available-p 
> 'native-image)))
> 
> Unless of course the idea is to automate it, I suppose, in which case
> this will be of no use...

That would be useful, but then I'd need to know what the correct answers 
are. As I build both release and snapshots for master binaries, these 
answers could well be different from different builds, so again, I have 
to remember.

With an ert test, I can even put something into the .emacs on my windows 
build machine. When I build, I should  be able to launch Emacs and it 
will run some basic "is the distribution package" working tests. I am 
thinking of features at the moment but, of course, I can run any tests 
at all -- I'd like to check binary stripping, and optimization for 
instance (the later caused problems for me before). I sanity check on 
the size of the files in the distribution would be useful (i.e. 
detecting if python has got pulled in again).

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Fri, 21 Aug 2020 22:56:42 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> That would be useful, but then I'd need to know what the correct answers 
> are.

The correct answers can be found by comparing against
system-configuration-features, I think.

And you have the same problem with your ERT-based approach, no?

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-22 07:37, Eli Zaretskii wrote:
>> Date: Fri, 21 Aug 2020 22:56:42 +0100
>> From: phillip.lord@russet.org.uk
>> Cc: emacs-devel@gnu.org
>> 
>> That would be useful, but then I'd need to know what the correct 
>> answers
>> are.
> 
> The correct answers can be found by comparing against
> system-configuration-features, I think.
> 
> And you have the same problem with your ERT-based approach, no?


No, because the correct answers are hardcoded into the file.

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Sat, 22 Aug 2020 11:21:27 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> > The correct answers can be found by comparing against
> > system-configuration-features, I think.
> > 
> > And you have the same problem with your ERT-based approach, no?
> 
> 
> No, because the correct answers are hardcoded into the file.

I'm confused: how can you know the correct answers in advance in one
case, but not in the other?

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-22 11:40, Eli Zaretskii wrote:
>> Date: Sat, 22 Aug 2020 11:21:27 +0100
>> From: phillip.lord@russet.org.uk
>> Cc: emacs-devel@gnu.org
>> 
>> > The correct answers can be found by comparing against
>> > system-configuration-features, I think.
>> >
>> > And you have the same problem with your ERT-based approach, no?
>> 
>> 
>> No, because the correct answers are hardcoded into the file.
> 
> I'm confused: how can you know the correct answers in advance in one
> case, but not in the other?

I think we are talking cross purposes. I do not want to display all the 
output, I want to know what it should be. And I don't care about the 
output from things that work as expected only things that do not. All of 
this is what ert is designed for. I might like to tweak the output a 
little, but as ert doesn't have a pluggable interface, that's harder.

Of course, I could write this functionality again. But it looks like a 
test to me. Why not use ert?

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Sat, 22 Aug 2020 11:53:54 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> On 2020-08-22 11:40, Eli Zaretskii wrote:
> >> Date: Sat, 22 Aug 2020 11:21:27 +0100
> >> From: phillip.lord@russet.org.uk
> >> Cc: emacs-devel@gnu.org
> >> 
> >> > The correct answers can be found by comparing against
> >> > system-configuration-features, I think.
> >> >
> >> > And you have the same problem with your ERT-based approach, no?
> >> 
> >> 
> >> No, because the correct answers are hardcoded into the file.
> > 
> > I'm confused: how can you know the correct answers in advance in one
> > case, but not in the other?
> 
> I think we are talking cross purposes.

Maybe so, but please bear with me, okay?

> I do not want to display all the output, I want to know what it
> should be. And I don't care about the output from things that work
> as expected only things that do not. All of this is what ert is
> designed for. I might like to tweak the output a little, but as ert
> doesn't have a pluggable interface, that's harder.

I'm not talking about using ert or not, I'm talking about determining
which of the features should check out and which shouldn't.  Are you
going to edit w32-features.el by hand each time you add or remove an
optional library from the build?  Or will the list of the features
which should check out be constructed automagically somehow?  If the
latter, I didn't see you describe how will that happen.  If the
former, it means a file in the distribution will depend on the details
of how you build the official binaries, which could be different from
what end-users do on their systems (those who build their own Emacs),
so the tests will fail for them, AFAIU.  It might also mean problems
in merging from the release branch to master, e.g. if we remove some
optional library from master that is still being used on the release
branch.

I guess what I'm saying is that I don't yet see the overall picture of
how this is supposed to work.  Apologies if I'm missing something.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-22 12:15, Eli Zaretskii wrote:
>> Date: Sat, 22 Aug 2020 11:53:54 +0100
>> From: phillip.lord@russet.org.uk
>> Cc: emacs-devel@gnu.org
>> I do not want to display all the output, I want to know what it
>> should be. And I don't care about the output from things that work
>> as expected only things that do not. All of this is what ert is
>> designed for. I might like to tweak the output a little, but as ert
>> doesn't have a pluggable interface, that's harder.
> 
> I'm not talking about using ert or not, I'm talking about determining
> which of the features should check out and which shouldn't.  Are you
> going to edit w32-features.el by hand each time you add or remove an
> optional library from the build?  Or will the list of the features
> which should check out be constructed automagically somehow?


It would be manual. The list of features changes rarely enough that it's 
hard to automate this and know that it will work. The automation is as 
likely to break as a manual system, as a release is not routine enough 
(for instance, you slightly broke my build script by updating the 
version number on Emacs-27 before I had done the windows build -- not a 
complaint, but Emacs releases are not routine).

If I think back to the bugs with Emacs binary releases, this would have 
stopped only a few: harfbuzz and svg failure in this release. It would 
not have prevented missing dependencies (I have missed all of harfbuzz, 
lcms and libjansson till late in the day), nor the failed optimization 
or stripping. But, it would stop regressions of all of those, assuming I 
can work out how to test for them all.


> If the latter, I didn't see you describe how will that happen.  If the
> former, it means a file in the distribution will depend on the details
> of how you build the official binaries, which could be different from
> what end-users do on their systems (those who build their own Emacs),
> so the tests will fail for them, AFAIU.

That is true with all of our tests, I think. json parsing is tested by 
make check and it will not work on msys2 if the dependencies are not 
installed. In this case, I am not suggesting adding ert tests to make 
check, but a manually run test specifically for understanding the 
features of a binary build. I am guessing anyone running this would be 
understand what they are doing enough to know why. And I will document 
the entry function.

> It might also mean problems in merging from the release branch to 
> master,
> e.g. if we remove some optional library from master that is still being 
> used on the release
> branch.

That problem is there anyway in build-deps.py and it is worse there 
since it will break a release rather than fail to test it. And I think 
for configure.ac also no?


> I guess what I'm saying is that I don't yet see the overall picture of
> how this is supposed to work.  Apologies if I'm missing something.

I think it is worth the discussion -- releases are far apart and it 
takes a degree of insight to get this right.

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Sat, 22 Aug 2020 13:52:22 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> > I'm not talking about using ert or not, I'm talking about determining
> > which of the features should check out and which shouldn't.  Are you
> > going to edit w32-features.el by hand each time you add or remove an
> > optional library from the build?  Or will the list of the features
> > which should check out be constructed automagically somehow?
> 
> It would be manual. The list of features changes rarely enough that it's 
> hard to automate this and know that it will work. The automation is as 
> likely to break as a manual system

I don't understand why it must break, if you base it on
system-configuration-features, or on some new file, to be generated by
the build procedure.  Did you consider one of these alternatives?  If
yes, why did you reject them?

> If I think back to the bugs with Emacs binary releases, this would have 
> stopped only a few: harfbuzz and svg failure in this release. It would 
> not have prevented missing dependencies (I have missed all of harfbuzz, 
> lcms and libjansson till late in the day), nor the failed optimization 
> or stripping.

I don't understand this, either.  If the dependencies of HarfBuzz are
not installed, then the HarfBuzz font backend will not be available,
and the way I suggested for testing its availability will tell you
HarfBuzz doesn't work, which is AFAIU what you wanted.  And the same
with any other DLLs: if their dependencies are not available, they
will fail to load when the feature test runs, and the test will fail.

> > If the latter, I didn't see you describe how will that happen.  If the
> > former, it means a file in the distribution will depend on the details
> > of how you build the official binaries, which could be different from
> > what end-users do on their systems (those who build their own Emacs),
> > so the tests will fail for them, AFAIU.
> 
> That is true with all of our tests, I think.

No, our tests use skip-unless to avoid running tests that need
features or programs which are unavailable.

> json parsing is tested by make check and it will not work on msys2
> if the dependencies are not installed.

I didn't look into this, but if json testing fails instead of skipping
all the tests, that's a bug that should be fixed.

> In this case, I am not suggesting adding ert tests to make check,
> but a manually run test specifically for understanding the features
> of a binary build. I am guessing anyone running this would be
> understand what they are doing enough to know why. And I will
> document the entry function.

That's not what worries me.  What worries me is the fact that a file
distributed as part of the Lisp libraries will have its contents
depend on the last release you personally did, with whatever quirks
that required from the tests in that file.  It doesn't sound right to
me.

> > It might also mean problems in merging from the release branch to 
> > master,
> > e.g. if we remove some optional library from master that is still being 
> > used on the release
> > branch.
> 
> That problem is there anyway in build-deps.py

But build-deps.py is under admin, whereas you are suggesting to do
this under lisp/.

> and it is worse there since it will break a release rather than fail
> to test it. And I think for configure.ac also no?

I'm not aware of such problems with configure.ac.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-22 14:08, Eli Zaretskii wrote:
>> Date: Sat, 22 Aug 2020 13:52:22 +0100
>> From: phillip.lord@russet.org.uk
>> Cc: emacs-devel@gnu.org
>> 
>> > I'm not talking about using ert or not, I'm talking about determining
>> > which of the features should check out and which shouldn't.  Are you
>> > going to edit w32-features.el by hand each time you add or remove an
>> > optional library from the build?  Or will the list of the features
>> > which should check out be constructed automagically somehow?
>> 
>> It would be manual. The list of features changes rarely enough that 
>> it's
>> hard to automate this and know that it will work. The automation is as
>> likely to break as a manual system
> 
> I don't understand why it must break, if you base it on
> system-configuration-features, or on some new file, to be generated by
> the build procedure.  Did you consider one of these alternatives?  If
> yes, why did you reject them?

I didn't consider it much. I cannot see how to go from this to something 
I can test. Say, `system-configuration-features` lists "XPM", I cannot 
turn that into something I could test.

I could modify by test something like

(ert-deftest
   (skip-unless (string-match-p "XPM" system-configuration-features))
   (should (image-type-available 'xpm))
)


>> If I think back to the bugs with Emacs binary releases, this would 
>> have
>> stopped only a few: harfbuzz and svg failure in this release. It would
>> not have prevented missing dependencies (I have missed all of 
>> harfbuzz,
>> lcms and libjansson till late in the day), nor the failed optimization
>> or stripping.
> 
> I don't understand this, either.  If the dependencies of HarfBuzz are
> not installed, then the HarfBuzz font backend will not be available,
> and the way I suggested for testing its availability will tell you
> HarfBuzz doesn't work, which is AFAIU what you wanted.  And the same
> with any other DLLs: if their dependencies are not available, they
> will fail to load when the feature test runs, and the test will fail.

It would work if, when harfbuzz was installed w32-features was updated, 
but not if it wasn't. As the harfbuzz merge didn't update build-deps.zh 
which it arguable should have, then why would a test file be updated.


>> > If the latter, I didn't see you describe how will that happen.  If the
>> > former, it means a file in the distribution will depend on the details
>> > of how you build the official binaries, which could be different from
>> > what end-users do on their systems (those who build their own Emacs),
>> > so the tests will fail for them, AFAIU.
>> 
>> That is true with all of our tests, I think.
> 
> No, our tests use skip-unless to avoid running tests that need
> features or programs which are unavailable.
> 
>> json parsing is tested by make check and it will not work on msys2
>> if the dependencies are not installed.
> 
> I didn't look into this, but if json testing fails instead of skipping
> all the tests, that's a bug that should be fixed.

I think you are correct about all of these. In either case, the tests 
will not pass!


>> In this case, I am not suggesting adding ert tests to make check,
>> but a manually run test specifically for understanding the features
>> of a binary build. I am guessing anyone running this would be
>> understand what they are doing enough to know why. And I will
>> document the entry function.
> 
> That's not what worries me.  What worries me is the fact that a file
> distributed as part of the Lisp libraries will have its contents
> depend on the last release you personally did, with whatever quirks
> that required from the tests in that file.  It doesn't sound right to
> me.


Not sure I understand this. A lisp library with "w32-" in it's name will 
have behaviour which reflects how the official binaries are supposed to 
behave. If this worries you, as I say, I can add it to `data-directory' 
instead. But, it has to be in the distribution.


>> > It might also mean problems in merging from the release branch to
>> > master,
>> > e.g. if we remove some optional library from master that is still being
>> > used on the release
>> > branch.
>> 
>> That problem is there anyway in build-deps.py
> 
> But build-deps.py is under admin, whereas you are suggesting to do
> this under lisp/.

Having a test functionality under lisp/ does, I agree, seem morally 
wrong. Likewise `data-directory'. But, the reasons for it are clear, and 
what harm would it cause in practice?


Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Sat, 22 Aug 2020 14:53:15 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> > I don't understand why it must break, if you base it on
> > system-configuration-features, or on some new file, to be generated by
> > the build procedure.  Did you consider one of these alternatives?  If
> > yes, why did you reject them?
> 
> I didn't consider it much. I cannot see how to go from this to something 
> I can test. Say, `system-configuration-features` lists "XPM", I cannot 
> turn that into something I could test.

You'd need a database of features like XPM vs tests to test each
feature.

This assumes we are talking about a general-purpose feature-testing
facility of the kind proposed by Stefan, which I'm no longer sure is
the case, see below.

> I could modify by test something like
> 
> (ert-deftest
>    (skip-unless (string-match-p "XPM" system-configuration-features))
>    (should (image-type-available 'xpm))
> )

Yes, this would also be good.

> >> If I think back to the bugs with Emacs binary releases, this would 
> >> have
> >> stopped only a few: harfbuzz and svg failure in this release. It would
> >> not have prevented missing dependencies (I have missed all of 
> >> harfbuzz,
> >> lcms and libjansson till late in the day), nor the failed optimization
> >> or stripping.
> > 
> > I don't understand this, either.  If the dependencies of HarfBuzz are
> > not installed, then the HarfBuzz font backend will not be available,
> > and the way I suggested for testing its availability will tell you
> > HarfBuzz doesn't work, which is AFAIU what you wanted.  And the same
> > with any other DLLs: if their dependencies are not available, they
> > will fail to load when the feature test runs, and the test will fail.
> 
> It would work if, when harfbuzz was installed w32-features was updated, 
> but not if it wasn't. As the harfbuzz merge didn't update build-deps.zh 
> which it arguable should have, then why would a test file be updated.

Then I don't understand what you said in the quoted part above: you
seemed to be saying that you could have detected the absence of
HarfBuzz, but not of its dependencies.  Which is why I wrote that
having HarfBuzz without the dependencies will cause loading HarfBuzz
to fail, and Emacs will think that HarfBuzz is not available.  It
doesn't matter whether the HarfBuzz DLL or its dependency DLLs are
missing: in both cases the HarfBuzz DLL will fail to load.

IOW, I thought you were describing the situation where w32-features
_was_ updated to add a test for HarfBuzz.  If not, then how could you
have discovered that the DLL itself is not in the bundle?

> >> > If the latter, I didn't see you describe how will that happen.  If the
> >> > former, it means a file in the distribution will depend on the details
> >> > of how you build the official binaries, which could be different from
> >> > what end-users do on their systems (those who build their own Emacs),
> >> > so the tests will fail for them, AFAIU.
> >> 
> >> That is true with all of our tests, I think.
> > 
> > No, our tests use skip-unless to avoid running tests that need
> > features or programs which are unavailable.
> > 
> >> json parsing is tested by make check and it will not work on msys2
> >> if the dependencies are not installed.
> > 
> > I didn't look into this, but if json testing fails instead of skipping
> > all the tests, that's a bug that should be fixed.
> 
> I think you are correct about all of these. In either case, the tests 
> will not pass!

Skipping a test and failing a test is not the same.  You are looking
for failures: tests that should have succeeded, but didn't.

> > That's not what worries me.  What worries me is the fact that a file
> > distributed as part of the Lisp libraries will have its contents
> > depend on the last release you personally did, with whatever quirks
> > that required from the tests in that file.  It doesn't sound right to
> > me.
> 
> Not sure I understand this. A lisp library with "w32-" in it's name will 
> have behaviour which reflects how the official binaries are supposed to 
> behave. If this worries you, as I say, I can add it to `data-directory' 
> instead. But, it has to be in the distribution.

I thought we were talking about a feature that could serve any end
user, including those who build their own Emacs, not just the person
who produces the official binaries.  If this is only about the
official binaries, then why does it have to be under lisp/?  Where you
build the binaries, you always have the admin/ directory, albeit not
in the distribution zip, so how can its being in admin/ be a problem
for you?  You should be able to invoke the test file from any
directory on your system, not just from the unpacked archive.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-22 15:35, Eli Zaretskii wrote:
>> Date: Sat, 22 Aug 2020 14:53:15 +0100
>> From: phillip.lord@russet.org.uk
>> Cc: emacs-devel@gnu.org
>> 
>> >> If I think back to the bugs with Emacs binary releases, this would
>> >> have
>> >> stopped only a few: harfbuzz and svg failure in this release. It would
>> >> not have prevented missing dependencies (I have missed all of
>> >> harfbuzz,
>> >> lcms and libjansson till late in the day), nor the failed optimization
>> >> or stripping.
>> >
>> > I don't understand this, either.  If the dependencies of HarfBuzz are
>> > not installed, then the HarfBuzz font backend will not be available,
>> > and the way I suggested for testing its availability will tell you
>> > HarfBuzz doesn't work, which is AFAIU what you wanted.  And the same
>> > with any other DLLs: if their dependencies are not available, they
>> > will fail to load when the feature test runs, and the test will fail.
>> 
>> It would work if, when harfbuzz was installed w32-features was 
>> updated,
>> but not if it wasn't. As the harfbuzz merge didn't update 
>> build-deps.zh
>> which it arguable should have, then why would a test file be updated.
> 
> Then I don't understand what you said in the quoted part above: you
> seemed to be saying that you could have detected the absence of
> HarfBuzz, but not of its dependencies. Which is why I wrote that
> having HarfBuzz without the dependencies will cause loading HarfBuzz
> to fail, and Emacs will think that HarfBuzz is not available.  It
> doesn't matter whether the HarfBuzz DLL or its dependency DLLs are
> missing: in both cases the HarfBuzz DLL will fail to load.
> 
> IOW, I thought you were describing the situation where w32-features
> _was_ updated to add a test for HarfBuzz.  If not, then how could you
> have discovered that the DLL itself is not in the bundle?


So, there have been two problems with harfbuzz. First, when harfbuzz was 
installed, build-deps.py was not updated. A test system would not have 
picked this up. But someone complained about it in pre-release or 
snapshot builds. After that I added harfbuzz to build-deps.py, but it 
wasn't actually running. Iff I had added a test to w32-feature.el for 
harfbuzz, which this would have been discovered earlier.


>> > No, our tests use skip-unless to avoid running tests that need
>> > features or programs which are unavailable.
>> >
>> >> json parsing is tested by make check and it will not work on msys2
>> >> if the dependencies are not installed.
>> >
>> > I didn't look into this, but if json testing fails instead of skipping
>> > all the tests, that's a bug that should be fixed.
>> 
>> I think you are correct about all of these. In either case, the tests
>> will not pass!
> 
> Skipping a test and failing a test is not the same.  You are looking
> for failures: tests that should have succeeded, but didn't.

Indeed not. But, skipping a test because something is not compiled in is 
wrong iff the feature is supposed to be compiled in. Running "make 
check" on my windows build machine will only pick up errors with json 
parsing if libjansson was installed. It would nice to have a "make 
check" which assumes that all standard features (appropriate for the 
platform) are installed and fails (not skips).


>> > That's not what worries me.  What worries me is the fact that a file
>> > distributed as part of the Lisp libraries will have its contents
>> > depend on the last release you personally did, with whatever quirks
>> > that required from the tests in that file.  It doesn't sound right to
>> > me.
>> 
>> Not sure I understand this. A lisp library with "w32-" in it's name 
>> will
>> have behaviour which reflects how the official binaries are supposed 
>> to
>> behave. If this worries you, as I say, I can add it to 
>> `data-directory'
>> instead. But, it has to be in the distribution.
> 
> I thought we were talking about a feature that could serve any end
> user, including those who build their own Emacs, not just the person
> who produces the official binaries. If this is only about the
> official binaries, then why does it have to be under lisp/?  Where you
> build the binaries, you always have the admin/ directory, albeit not
> in the distribution zip, so how can its being in admin/ be a problem
> for you?  You should be able to invoke the test file from any
> directory on your system, not just from the unpacked archive.

Because I have multiple admin directories, as I build both snapshot and 
release binaries. I do this by having a git worktree for emacs-27 and 
another for master. By putting the lisp test file in the distribution, I 
can guarantee that it is the right one.

I am guessing that I would be the main user of this, and possibly the 
only user. It might be useful for anyone building Emacs binaries, 
however, so that they could check that their binary distribution has all 
of the standard features compiled in (or dynamically loadable). But this 
would require more logic as the standard features are not quite the same 
on all platforms, I think. It might also be useful for if someone 
reports that a feature is missing from a windows distribution that they 
have downloaded as it would provide a standard way of checking.

Regardless, these are secondary issues. Right now, I want something that 
means I can check whether I have created a regression of the svg or 
harfbuzz issues. Without this, I am going to be very wary of trying 
alternative mechanisms for identifying dependencies; I still think that 
the windows binary distribution is unnecessarily complicated, based on 
the msys2 dependencies as given.

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Sat, 22 Aug 2020 16:13:52 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> > Skipping a test and failing a test is not the same.  You are looking
> > for failures: tests that should have succeeded, but didn't.
> 
> Indeed not. But, skipping a test because something is not compiled in is 
> wrong iff the feature is supposed to be compiled in. Running "make 
> check" on my windows build machine will only pick up errors with json 
> parsing if libjansson was installed.

That's a separate problem.  If you want to be able to detect it as
well, you'd probably need a separate test for features that appear in
system-configuration-features.

> I still think that the windows binary distribution is unnecessarily
> complicated, based on the msys2 dependencies as given.

I agree, but if you want to depend on MSYS2 dependency tracking,
that's part of the price, no?

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-22 16:19, Eli Zaretskii wrote:
>> Date: Sat, 22 Aug 2020 16:13:52 +0100
>> From: phillip.lord@russet.org.uk
>> Cc: emacs-devel@gnu.org
>> 
>> > Skipping a test and failing a test is not the same.  You are looking
>> > for failures: tests that should have succeeded, but didn't.
>> 
>> Indeed not. But, skipping a test because something is not compiled in 
>> is
>> wrong iff the feature is supposed to be compiled in. Running "make
>> check" on my windows build machine will only pick up errors with json
>> parsing if libjansson was installed.
> 
> That's a separate problem.  If you want to be able to detect it as
> well, you'd probably need a separate test for features that appear in
> system-configuration-features.
> 
>> I still think that the windows binary distribution is unnecessarily
>> complicated, based on the msys2 dependencies as given.
> 
> I agree, but if you want to depend on MSYS2 dependency tracking,
> that's part of the price, no?

Yes. And I am happy to experiment with alternatives. But I want 
something to be able to test the binary distributions that the 
alternatives would produce.

So where are we up to? w32-features.el? In lisp/ or etc/

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Sat, 22 Aug 2020 18:28:25 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> So where are we up to? w32-features.el? In lisp/ or etc/

I'd like to see the code first, in a version that satisfies you
functionality-wise, because otherwise I fear that my opinion on the
location might be based on incorrect or inaccurate assumptions.

Thanks.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-22 18:32, Eli Zaretskii wrote:
>> Date: Sat, 22 Aug 2020 18:28:25 +0100
>> From: phillip.lord@russet.org.uk
>> Cc: emacs-devel@gnu.org
>> 
>> So where are we up to? w32-features.el? In lisp/ or etc/
> 
> I'd like to see the code first, in a version that satisfies you
> functionality-wise, because otherwise I fear that my opinion on the
> location might be based on incorrect or inaccurate assumptions.
> 
> Thanks.

A full version? I sent my starting point through before attached a few 
back. Very simple.

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Stefan Monnier]

> A full version? I sent my starting point through before attached a few
>   back. Very simple.

The sample codes I've seen don't seem specific to w32 (which is good),
so I think the file name shouldn't include "w32".


        Stefan

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-22 21:32, Stefan Monnier wrote:
>> A full version? I sent my starting point through before attached a few
>>   back. Very simple.
> 
> The sample codes I've seen don't seem specific to w32 (which is good),
> so I think the file name shouldn't include "w32".
> 

I've added

(ert-deftest feature-optimization ()
   (should
    (string-match-p "CFLAGS=-O2" system-configuration-options)))

which is not strictly a feature and is windows specific. I would like to 
be able to add something that checks the size of the installation as 
well. I don't want to get too bound up in making things OS independent.

Phil

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Stefan Monnier]

> (ert-deftest feature-optimization ()
>   (should
>    (string-match-p "CFLAGS=-O2" system-configuration-options)))
>
> which is not strictly a feature and is windows specific.

Doesn't look Windows-specific to me.

> I would like to be
> able to add something that checks the size of the installation as
>  well. I don't want to get too bound up in making things OS independent.

It's easy to add some stuff under `(when (eq system-type 'windows-nt) ...)`
or something like that so it's still useful on other systems.


        Stefan

Re: Emacs 27.1 Windows Binaries -- testing wanted

[Eli Zaretskii]

> Date: Sat, 22 Aug 2020 21:18:26 +0100
> From: phillip.lord@russet.org.uk
> Cc: emacs-devel@gnu.org
> 
> > I'd like to see the code first, in a version that satisfies you
> > functionality-wise, because otherwise I fear that my opinion on the
> > location might be based on incorrect or inaccurate assumptions.
> > 
> > Thanks.
> 
> A full version? I sent my starting point through before attached a few 
> back. Very simple.

How far away is the full version?  I don't need to see all the
decorations, like the commentary, license, etc., just the code of a
version that is close to the final one.

What you sent looked like a general idea.  Is that how the final
version will look like? a separate ert test for each feature, and the
tests added and removed manually?  Or will there be some glue you
didn't yet show?

Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip.lord]

On 2020-08-23 06:36, Eli Zaretskii wrote:
>> Date: Sat, 22 Aug 2020 21:18:26 +0100
>> From: phillip.lord@russet.org.uk
>> Cc: emacs-devel@gnu.org
>> 
>> > I'd like to see the code first, in a version that satisfies you
>> > functionality-wise, because otherwise I fear that my opinion on the
>> > location might be based on incorrect or inaccurate assumptions.
>> >
>> > Thanks.
>> 
>> A full version? I sent my starting point through before attached a few
>> back. Very simple.
> 
> How far away is the full version?  I don't need to see all the
> decorations, like the commentary, license, etc., just the code of a
> version that is close to the final one.
> 
> What you sent looked like a general idea.  Is that how the final
> version will look like? a separate ert test for each feature, and the
> tests added and removed manually?  Or will there be some glue you
> didn't yet show?


No that was the idea. I'd also add tests for other packaging errors that 
have come up. If I can do it, testing for optimization, binary stripping 
would be good. And, a test for total size of the installed package would 
be nice.

Phil