Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Eli Zaretskii]

> -(defcustom python-interpreter "python"
> +(defcustom python-interpreter
> +  (cond ((executable-find "python3") "python3")
> +        ((executable-find "python") "python")
> +        (t "python3"))

Stefan, are you sure this change is for the better?  python3 is not
guaranteed to be available (e.g., it isn't on my system), so the above
is a net loss for such systems.  Bug#70722 and bug#70815 have some
information about this mess, as it happens on different systems.  Are
you sure this change is not specific to what you see on macOS, for
example?  We could make the default value different on different
systems, if we really need that, but as a single value, I think
"python" is better, as it covers many more use cases (including
Python 2.x, which you seem to exclude entirely).

This kind of change should have been discussed, I think.

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Xiyue Deng]

Eli Zaretskii <eliz@gnu.org> writes:

>> -(defcustom python-interpreter "python"
>> +(defcustom python-interpreter
>> +  (cond ((executable-find "python3") "python3")
>> +        ((executable-find "python") "python")
>> +        (t "python3"))
>
> Stefan, are you sure this change is for the better?  python3 is not
> guaranteed to be available (e.g., it isn't on my system), so the above
> is a net loss for such systems.  Bug#70722 and bug#70815 have some
> information about this mess, as it happens on different systems.  Are
> you sure this change is not specific to what you see on macOS, for
> example?  We could make the default value different on different
> systems, if we really need that, but as a single value, I think
> "python" is better, as it covers many more use cases (including
> Python 2.x, which you seem to exclude entirely).
>

If I may provide a data point: on Debian Python 2 was removed completely
since Debian 11 Bullseye[1], and the default interpreter for Python 3 is
named "python3".  During the transition period of retiring Python 2
(which the PSF stopped supporting since 2020), it is decided that
"python" should always refer to Python 2 and "python3" is used for
version 3 obviously so that the 2 Python versions can coexist and apps
and libs can test against both versions at the same time.  Now that the
transition has done long ago, Python 3 is the default on Debian systems
and now only "python3" command is available.  There does exist a package
"python-is-python3"[2] that provides a symlink from "python" to
"python3", but it is not installed by default so should not be relied on
directly.

On some other distribution (like Fedora) both "python" and "python3" are
available[3].  Though AIUI, in Python 3 only era, "python3" is
guaranteed to be available in most cases while "python" is not.  (I am
curious about which system you are using?)

So TL;DR, on some distribution (e.g. Debian) only "python3" command is
guaranteed to be available by default.

> This kind of change should have been discussed, I think.
>

[1] https://wiki.debian.org/Python
[2] https://packages.debian.org/stable/python-is-python3
[3] https://fedoraproject.org/wiki/Changes/Python_means_Python3

-- 
Xiyue Deng

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Augusto Stoffel]

On Fri, 27 Sep 2024 at 00:59, Xiyue Deng wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> -(defcustom python-interpreter "python"
>>> +(defcustom python-interpreter
>>> +  (cond ((executable-find "python3") "python3")
>>> +        ((executable-find "python") "python")
>>> +        (t "python3"))
>>
>> Stefan, are you sure this change is for the better?  python3 is not
>> guaranteed to be available (e.g., it isn't on my system), so the above
>> is a net loss for such systems.  Bug#70722 and bug#70815 have some
>> information about this mess, as it happens on different systems.  Are
>> you sure this change is not specific to what you see on macOS, for
>> example?  We could make the default value different on different
>> systems, if we really need that, but as a single value, I think
>> "python" is better, as it covers many more use cases (including
>> Python 2.x, which you seem to exclude entirely).

I disagree with that default, for the reason I will explain after a
short rant.

> If I may provide a data point: on Debian Python 2 was removed completely
> since Debian 11 Bullseye[1], and the default interpreter for Python 3 is
> named "python3".  During the transition period of retiring Python 2
> (which the PSF stopped supporting since 2020), it is decided that
> "python" should always refer to Python 2 and "python3" is used for
> version 3 obviously so that the 2 Python versions can coexist and apps
> and libs can test against both versions at the same time.  Now that the
> transition has done long ago, Python 3 is the default on Debian systems
> and now only "python3" command is available.  There does exist a package
> "python-is-python3"[2] that provides a symlink from "python" to
> "python3", but it is not installed by default so should not be relied on
> directly.

What a horrible design.  Anyway, I now understand why Emacs has that
roundabout default value.

> On some other distribution (like Fedora) both "python" and "python3" are
> available[3].  Though AIUI, in Python 3 only era, "python3" is
> guaranteed to be available in most cases while "python" is not.  (I am
> curious about which system you are using?)

This is the sensible way to proceed: python is linked to the default
Python version (/usr/bin/python3.12 in a recent release), as is python3.
Other Python versions can be easily installed in parallel as python3.X
or python2.X.  Finally, if you create a "virtual environment" with
pythonX.Y, then in that venv the python command refers to pythonX.Y.

And this last bit is the important part: if the user (via venvs or
whatever other means) symlinked python to some program, whatever it is,
then Emacs should respect that decision.

Therefore (assuming we keep the workaround for Debian) the default value
should be:

  (defcustom python-interpreter
    (cond ((executable-find "python") "python")
          ((executable-find "python3") "python3")
          (t "python"))

> So TL;DR, on some distribution (e.g. Debian) only "python3" command is
> guaranteed to be available by default.

(My distro has symlinked commands node -> node-20, perl -> perl5.38.2
and many similar ones.  So Debian doesn't have any unversioned commands
like that?  Does any shebang from third parties work at all?)

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Xiyue Deng]

Augusto Stoffel <arstoffel@gmail.com> writes:

> On Fri, 27 Sep 2024 at 00:59, Xiyue Deng wrote:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> -(defcustom python-interpreter "python"
>>>> +(defcustom python-interpreter
>>>> +  (cond ((executable-find "python3") "python3")
>>>> +        ((executable-find "python") "python")
>>>> +        (t "python3"))
>>>
>>> Stefan, are you sure this change is for the better?  python3 is not
>>> guaranteed to be available (e.g., it isn't on my system), so the above
>>> is a net loss for such systems.  Bug#70722 and bug#70815 have some
>>> information about this mess, as it happens on different systems.  Are
>>> you sure this change is not specific to what you see on macOS, for
>>> example?  We could make the default value different on different
>>> systems, if we really need that, but as a single value, I think
>>> "python" is better, as it covers many more use cases (including
>>> Python 2.x, which you seem to exclude entirely).
>
> I disagree with that default, for the reason I will explain after a
> short rant.
>
>> If I may provide a data point: on Debian Python 2 was removed completely
>> since Debian 11 Bullseye[1], and the default interpreter for Python 3 is
>> named "python3".  During the transition period of retiring Python 2
>> (which the PSF stopped supporting since 2020), it is decided that
>> "python" should always refer to Python 2 and "python3" is used for
>> version 3 obviously so that the 2 Python versions can coexist and apps
>> and libs can test against both versions at the same time.  Now that the
>> transition has done long ago, Python 3 is the default on Debian systems
>> and now only "python3" command is available.  There does exist a package
>> "python-is-python3"[2] that provides a symlink from "python" to
>> "python3", but it is not installed by default so should not be relied on
>> directly.
>
> What a horrible design.

What's more horrible is the incompatibility between Python V2 and V3
making the transition really hard.  And PSF provided a tool for
migrating your code (2to3), creating different code base for the same
app/lib which made it even worse.  It got better since we had "six" and
"future" that let you use the same code to behave the same between V2
and V3.  I believe that using "python" and "python3" referring to
different Python versions was necessary to test the code during the
transition (and still is because some people may still be using Python 2
somewhere.)

> Anyway, I now understand why Emacs has that roundabout default value.
>
>> On some other distribution (like Fedora) both "python" and "python3" are
>> available[3].  Though AIUI, in Python 3 only era, "python3" is
>> guaranteed to be available in most cases while "python" is not.  (I am
>> curious about which system you are using?)
>
> This is the sensible way to proceed: python is linked to the default
> Python version (/usr/bin/python3.12 in a recent release), as is python3.
> Other Python versions can be easily installed in parallel as python3.X
> or python2.X.  Finally, if you create a "virtual environment" with
> pythonX.Y, then in that venv the python command refers to pythonX.Y.
>
> And this last bit is the important part: if the user (via venvs or
> whatever other means) symlinked python to some program, whatever it is,
> then Emacs should respect that decision.
>
> Therefore (assuming we keep the workaround for Debian) the default value
> should be:
>
>   (defcustom python-interpreter
>     (cond ((executable-find "python") "python")
>           ((executable-find "python3") "python3")
>           (t "python"))
>
>> So TL;DR, on some distribution (e.g. Debian) only "python3" command is
>> guaranteed to be available by default.
>
> (My distro has symlinked commands node -> node-20, perl -> perl5.38.2
> and many similar ones.

If you language is backward-compatible, sure.

> So Debian doesn't have any unversioned commands like that?

As I mentioned, you need to install "python-is-python3" to get the
unversioned "python" command to point to "python3", which is not
installed by default.

> Does any shebang from third parties work at all?)

I think nowadays most of the tutorial use "#!/usr/bin/env python3",
which is guaranteed to be available.  Of course you also have the
workaround above.

-- 
Xiyue Deng

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Stefan Kangas]

Augusto Stoffel <arstoffel@gmail.com> writes:

> This is the sensible way to proceed: python is linked to the default
> Python version (/usr/bin/python3.12 in a recent release), as is python3.

That point may or may not be valid, but it should have been raised
around 5 years ago with the distro developers that decided not to do
that.

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sun, 29 Sep 2024 08:37:13 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> 
> Augusto Stoffel <arstoffel@gmail.com> writes:
> 
> > This is the sensible way to proceed: python is linked to the default
> > Python version (/usr/bin/python3.12 in a recent release), as is python3.
> 
> That point may or may not be valid, but it should have been raised
> around 5 years ago with the distro developers that decided not to do
> that.

That's not what I see on a few systems to which I have access.

What's the harm of testing "python" first, anyway?  Several systems
use that, and doing so will let our users an extra degree of freedom:
they can change Python version without any adjustments needed in
Emacs.  Sounds like a win-win situation to me.

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Augusto Stoffel]

On Sun, 29 Sep 2024 at 18:49, Eli Zaretskii wrote:

> What's the harm of testing "python" first, anyway?  Several systems
> use that, and doing so will let our users an extra degree of freedom:
> they can change Python version without any adjustments needed in
> Emacs.  Sounds like a win-win situation to me.

Exactly.  That was the important part of my previous email, anyway.

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> -(defcustom python-interpreter "python"
>> +(defcustom python-interpreter
>> +  (cond ((executable-find "python3") "python3")
>> +        ((executable-find "python") "python")
>> +        (t "python3"))
>
> Stefan, are you sure this change is for the better?  python3 is not
> guaranteed to be available (e.g., it isn't on my system), so the above
> is a net loss for such systems.  Bug#70722 and bug#70815 have some
> information about this mess, as it happens on different systems.  Are
> you sure this change is not specific to what you see on macOS, for
> example?  We could make the default value different on different
> systems, if we really need that, but as a single value, I think
> "python" is better, as it covers many more use cases

There is no "python" binary or symlink by default on Debian, Ubuntu,
Fedora, etc.  This means that for the vast majority of GNU/Linux users,
the commands `python-sort-imports', `python-add-import',
`python-remove-import', and `python-fix-imports' have been broken by
default.  That is now fixed.

On macOS, "python" is a symlink to "python3", so there is no problem there.

Note that this is the same value as for `python-shell-interpreter', and
that we have had no issues with.

>  (including Python 2.x, which you seem to exclude entirely).

If there is no "python3", it will fall back to "python".  If that
happens to be Python 2 on your machine, that is what will be used.

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sun, 29 Sep 2024 08:23:04 -0700
> Cc: emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> -(defcustom python-interpreter "python"
> >> +(defcustom python-interpreter
> >> +  (cond ((executable-find "python3") "python3")
> >> +        ((executable-find "python") "python")
> >> +        (t "python3"))
> >
> > Stefan, are you sure this change is for the better?  python3 is not
> > guaranteed to be available (e.g., it isn't on my system), so the above
> > is a net loss for such systems.  Bug#70722 and bug#70815 have some
> > information about this mess, as it happens on different systems.  Are
> > you sure this change is not specific to what you see on macOS, for
> > example?  We could make the default value different on different
> > systems, if we really need that, but as a single value, I think
> > "python" is better, as it covers many more use cases
> 
> There is no "python" binary or symlink by default on Debian, Ubuntu,
> Fedora, etc.

There is a "python" (a symlink) on Trisquel, and I think someone else
said here the same about another variety (Arch?).

> This means that for the vast majority of GNU/Linux users,

I don't think I agree with the "vast majority" part.

> the commands `python-sort-imports', `python-add-import',
> `python-remove-import', and `python-fix-imports' have been broken by
> default.  That is now fixed.

We had the previous value for a long time, and I don't recollect any
complaints.  So calling this "broken" is too far-reaching.

> On macOS, "python" is a symlink to "python3", so there is no problem there.

On MS-Windows, there's no "python3", just "python".

> Note that this is the same value as for `python-shell-interpreter', and
> that we have had no issues with.

Not everyone invokes the interpreter.

> >  (including Python 2.x, which you seem to exclude entirely).
> 
> If there is no "python3", it will fall back to "python".  If that
> happens to be Python 2 on your machine, that is what will be used.

Can we be a bit more friendly to systems where the previous default
was better?  (Like I said: this should have been discussed in
advance.)

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Eli Zaretskii]

> Date: Sun, 29 Sep 2024 18:31:11 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > There is no "python" binary or symlink by default on Debian, Ubuntu,
> > Fedora, etc.
> 
> There is a "python" (a symlink) on Trisquel, and I think someone else
> said here the same about another variety (Arch?).

And I see at least one Fedora system, where "python" _is_ available
(and is a symlink).

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> There is no "python" binary or symlink by default on Debian, Ubuntu,
>> Fedora, etc.
>
> There is a "python" (a symlink) on Trisquel, and I think someone else
> said here the same about another variety (Arch?).

> And I see at least one Fedora system, where "python" _is_ available
> (and is a symlink).

Maybe I confused Fedora with CentOS.  In any case, systems where
"python" is a symlink should have no problem with this change.

(Arch removed Python 2 from their archive in 2022, FWIW.  It's possible
that they symlink "python" to "python3"; I don't know.)

>> the commands `python-sort-imports', `python-add-import',
>> `python-remove-import', and `python-fix-imports' have been broken by
>> default.  That is now fixed.
>
> We had the previous value for a long time, and I don't recollect any
> complaints.  So calling this "broken" is too far-reaching.

Well, I'm telling you that it doesn't work.  For example, `M-x
python-sort-imports' on Ubuntu 22 gives:

    Debugger entered--Lisp error: (file-missing "Searching for program"
    "No such file or directory" "python")

BTW, the fact that the previous default has been there for a long time
is not surprising.  Most distributions migrated to Python 3 by default
only in recent years (starting in earnest around 2018-2019).

>> Note that this is the same value as for `python-shell-interpreter', and
>> that we have had no issues with.
>
> Not everyone invokes the interpreter.

Not everyone invokes `M-x python-sort-imports' either, so I don't think
I see your point.

>> If there is no "python3", it will fall back to "python".  If that
>> happens to be Python 2 on your machine, that is what will be used.
>
> Can we be a bit more friendly to systems where the previous default
> was better?

Which systems are those, and what is the problem?

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sun, 29 Sep 2024 09:05:29 -0700
> Cc: emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> There is no "python" binary or symlink by default on Debian, Ubuntu,
> >> Fedora, etc.
> >
> > There is a "python" (a symlink) on Trisquel, and I think someone else
> > said here the same about another variety (Arch?).
> 
> > And I see at least one Fedora system, where "python" _is_ available
> > (and is a symlink).
> 
> Maybe I confused Fedora with CentOS.  In any case, systems where
> "python" is a symlink should have no problem with this change.

Yes, they will: we now force them to use Python 3.x, even if they
point the 'python' symlink to Python 2.x.

> (Arch removed Python 2 from their archive in 2022, FWIW.  It's possible
> that they symlink "python" to "python3"; I don't know.)

What happens when there's Python 4.x? will we change the value again?

> >> the commands `python-sort-imports', `python-add-import',
> >> `python-remove-import', and `python-fix-imports' have been broken by
> >> default.  That is now fixed.
> >
> > We had the previous value for a long time, and I don't recollect any
> > complaints.  So calling this "broken" is too far-reaching.
> 
> Well, I'm telling you that it doesn't work.  For example, `M-x
> python-sort-imports' on Ubuntu 22 gives:
> 
>     Debugger entered--Lisp error: (file-missing "Searching for program"
>     "No such file or directory" "python")

I'm not saying we should necessarily go back to a single fixed value.
But we could, for example, test "python" before "python3".  That
should solve the above case.

> > Can we be a bit more friendly to systems where the previous default
> > was better?
> 
> Which systems are those, and what is the problem?

I think I already answered that.

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> Yes, they will: we now force them to use Python 3.x, even if they
> point the 'python' symlink to Python 2.x.

To the extent that we "force" anyone with user option defaults, we have
already done that with `python-shell-interpreter'.

> What happens when there's Python 4.x? will we change the value again?

Guido has stated publicly that the Python core team intends to basically
just continue incrementing the minor version until the end of time, and
that there will never be a Python 4.  So, luckily for us, it's not
likely to be an issue for a very long time.

> I'm not saying we should necessarily go back to a single fixed value.
> But we could, for example, test "python" before "python3".  That
> should solve the above case.

OK, then let's do that.  It also seems to be Augusto's preferred option.
It should work on all my machines: let's hope that it works without any
problems elsewhere too.

Assuming that we care about consistency between `python-interpreter' and
`python-shell-interpreter' (I think we should), this means reversing the
decision to prefer Python 3 that was made in Emacs 28.1 (commit
fa686f099800).

So I've now made this change in dd4c67907eb.

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sun, 29 Sep 2024 14:41:15 -0700
> Cc: emacs-devel@gnu.org, Augusto Stoffel <arstoffel@gmail.com>, kobarity <kobarity@gmail.com>
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I'm not saying we should necessarily go back to a single fixed value.
> > But we could, for example, test "python" before "python3".  That
> > should solve the above case.
> 
> OK, then let's do that.  It also seems to be Augusto's preferred option.
> It should work on all my machines: let's hope that it works without any
> problems elsewhere too.
> 
> Assuming that we care about consistency between `python-interpreter' and
> `python-shell-interpreter' (I think we should), this means reversing the
> decision to prefer Python 3 that was made in Emacs 28.1 (commit
> fa686f099800).
> 
> So I've now made this change in dd4c67907eb.

Thanks.

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Rudolf Adamkovič]

Stefan Kangas <stefankangas@gmail.com> writes:

> On macOS, "python" is a symlink to "python3", so there is no problem
> there.

FYI:

On the latest macOS 15.0 (24A335):

$ python
bash: python: command not found

(Only `python3' works.)

Rudy
-- 
"The power of mathematics rests on its evasion of all unnecessary
thought."  --- Ernst Mach, 1838-1916

Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
http://adamkovic.org

Re: master 7f0a252f216 1/2: Prefer python3 for python-interpreter

[Robert Pluim]

>>>>> On Sun, 06 Oct 2024 00:50:21 +0200, Rudolf Adamkovič <rudolf@adamkovic.org> said:

    Rudolf> Stefan Kangas <stefankangas@gmail.com> writes:
    >> On macOS, "python" is a symlink to "python3", so there is no problem
    >> there.

    Rudolf> FYI:

    Rudolf> On the latest macOS 15.0 (24A335):

    Rudolf> $ python
    Rudolf> bash: python: command not found

    Rudolf> (Only `python3' works.)

This is true on at least 14.7 as well.

Robert
--