ELPA submission: python-import

ELPA submission: python-import

[Augusto Stoffel]

Hi,

I would like to submit the following package to ELPA:

    https://github.com/astoff/python-import

Its purpose is to edit the imports at the top of Python files more
conveniently.  There are already several packages for this on MELPA.
This one relies on the isort program for the editing and code analysis,
which is hopefully a reasonably robust solution.

(No need to clarify the formalities re package headers etc.)

Thanks!

Re: ELPA submission: python-import

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Date: Fri, 29 Jul 2022 16:15:29 +0200
> 
> I would like to submit the following package to ELPA:
> 
>     https://github.com/astoff/python-import
> 
> Its purpose is to edit the imports at the top of Python files more
> conveniently.  There are already several packages for this on MELPA.
> This one relies on the isort program for the editing and code analysis,
> which is hopefully a reasonably robust solution.

Thanks, but would it perhaps make sense to add this to python-mode
that we have in core?

Re: ELPA submission: python-import

[Augusto Stoffel]

On Fri, 29 Jul 2022 at 18:18, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Augusto Stoffel <arstoffel@gmail.com>
>> Date: Fri, 29 Jul 2022 16:15:29 +0200
>> 
>> I would like to submit the following package to ELPA:
>> 
>>     https://github.com/astoff/python-import
>> 
>> Its purpose is to edit the imports at the top of Python files more
>> conveniently.  There are already several packages for this on MELPA.
>> This one relies on the isort program for the editing and code analysis,
>> which is hopefully a reasonably robust solution.
>
> Thanks, but would it perhaps make sense to add this to python-mode
> that we have in core?

I suppose two things speaking against it:

- The package depends on two external tools (isort and pyflakes) that
  are popular but neither built-in nor de facto standards.

- Sometimes the heuristics used there can be wrong (e.g. if relative
  imports (IMHO usually a bad idea anyway) are used).  I haven't used
  the package enough to judge how much of a problem this is.

That said, if someone else gets the chance to give the code a good test
and is convinced by it, I wouldn't mind at all to have it added directly
to python-mode.

Re: ELPA submission: python-import

[Philip Kaludercic]

Augusto Stoffel <arstoffel@gmail.com> writes:

> Hi,
>
> I would like to submit the following package to ELPA:
>
>     https://github.com/astoff/python-import

Why do you depend on Emacs 28.1?  Setting aside convenience functions
like length=, all you really need is project-files from Emacs 27.1,
right?  And if you add a dependency on the right version of project.el
that implemented that function, you could even reduce the (transitive)
dependency to Emacs 26.1.

> Its purpose is to edit the imports at the top of Python files more
> conveniently.  There are already several packages for this on MELPA.
> This one relies on the isort program for the editing and code analysis,
> which is hopefully a reasonably robust solution.

I am not familiar with isort, and I suppose there are other lightweight
Emacs users like me who haven't heard of it either.  It might therefore
be good to add a link to some documentation, especially on how to
install it since that can sometimes be tricky when using Python.

> (No need to clarify the formalities re package headers etc.)
>
> Thanks!

Re: ELPA submission: python-import

[Stefan Monnier]

> I suppose two things speaking against it:
>
> - The package depends on two external tools (isort and pyflakes) that
>   are popular but neither built-in nor de facto standards.
>
> - Sometimes the heuristics used there can be wrong (e.g. if relative
>   imports (IMHO usually a bad idea anyway) are used).  I haven't used
>   the package enough to judge how much of a problem this is.

FWIW, I don't find those to be good reasons to keep the code separate.
It's quite common for some parts of the functionality of a major modes
to be less thorough (e.g. depend on the presence of some unusual tool,
or depend on specific conventions, ...): it's not ideal, but it's better
than nothing, and also makes it "obvious" where to improve the code.
IOW it can encourage further improvements.


        Stefan

Re: ELPA submission: python-import

[Augusto Stoffel]

On Fri, 29 Jul 2022 at 12:25, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>> I suppose two things speaking against it:
>>
>> - The package depends on two external tools (isort and pyflakes) that
>>   are popular but neither built-in nor de facto standards.
>>
>> - Sometimes the heuristics used there can be wrong (e.g. if relative
>>   imports (IMHO usually a bad idea anyway) are used).  I haven't used
>>   the package enough to judge how much of a problem this is.
>
> FWIW, I don't find those to be good reasons to keep the code separate.
> It's quite common for some parts of the functionality of a major modes
> to be less thorough (e.g. depend on the presence of some unusual tool,
> or depend on specific conventions, ...): it's not ideal, but it's better
> than nothing, and also makes it "obvious" where to improve the code.
> IOW it can encourage further improvements.

Sounds good then.  I'll develop and test the code a bit more locally for
a few weeks and then submit a patch.

In the meanwhile I'll keep updating the git repo linked in my previous
message.  Feedback from anyone interested in checking it out is welcome.

Re: ELPA submission: python-import

[Augusto Stoffel]

Hi Philip,

See my response to your comments below.

But more importantly, since now the plan is to add those features to
python.el, which is built-in but also extracted to be distributed on
ELPA for Emacs ≥ 24.4 --- have you thought of a scheme whereby one could
use new Emacs features on those kinds of packages, but then
automatically stick a (require 'compat) at the top of their ELPA
version?

On Fri, 29 Jul 2022 at 17:53, Philip Kaludercic <philip.kaludercic@fau.de> wrote:

> Why do you depend on Emacs 28.1?  Setting aside convenience functions
> like length=, all you really need is project-files from Emacs 27.1,
> right?  And if you add a dependency on the right version of project.el
> that implemented that function, you could even reduce the (transitive)
> dependency to Emacs 26.1.

I'm also using the MODES argument of the `interactive' form.  But yes, I
have no good reason to require 28 other than being unfamiliar/not
remembering the history of the new features, which makes it hard to
"promise" compatibility with older Emacsen.

> I am not familiar with isort, and I suppose there are other lightweight
> Emacs users like me who haven't heard of it either.  It might therefore
> be good to add a link to some documentation, especially on how to
> install it since that can sometimes be tricky when using Python.

Sure, I'll will do that.

Re: ELPA submission: python-import

[Visuwesh]

[சனி ஜூலை 30, 2022] Augusto Stoffel wrote:

> I'm also using the MODES argument of the `interactive' form.

Can you not use the modes spec in the declare form?

Re: ELPA submission: python-import

[Augusto Stoffel]

On Sat, 30 Jul 2022 at 14:42, Visuwesh <visuweshm@gmail.com> wrote:

> [சனி ஜூலை 30, 2022] Augusto Stoffel wrote:
>
>> I'm also using the MODES argument of the `interactive' form.
>
> Can you not use the modes spec in the declare form?

Yes, see e.g. how devdocs.el (from ELPA) sets the `completion-predicate'
symbol property of some commands.

Re: ELPA submission: python-import

[Philip Kaludercic]

Augusto Stoffel <arstoffel@gmail.com> writes:

> Hi Philip,
>
> See my response to your comments below.
>
> But more importantly, since now the plan is to add those features to
> python.el, which is built-in but also extracted to be distributed on
> ELPA for Emacs ≥ 24.4 --- have you thought of a scheme whereby one could
> use new Emacs features on those kinds of packages, but then
> automatically stick a (require 'compat) at the top of their ELPA
> version?

We have recently added compat as a dependency for ERC, but that only
required support back until 27.1.  Sadly it isn't as simple as just
adding (require 'compat nil 'noerror) at the top of the file, as compat
is neither complete not transparent.

I am not saying it isn't impossible, but if there is a dependency on
project.el, it might make sense to raise the minimum version of ELPA's
python.el?  Alternatively, one could try and reduce the minimum version
of project.el, but that might be tricky.

> On Fri, 29 Jul 2022 at 17:53, Philip Kaludercic <philip.kaludercic@fau.de> wrote:
>
>> Why do you depend on Emacs 28.1?  Setting aside convenience functions
>> like length=, all you really need is project-files from Emacs 27.1,
>> right?  And if you add a dependency on the right version of project.el
>> that implemented that function, you could even reduce the (transitive)
>> dependency to Emacs 26.1.
>
> I'm also using the MODES argument of the `interactive' form.  But yes, I
> have no good reason to require 28 other than being unfamiliar/not
> remembering the history of the new features, which makes it hard to
> "promise" compatibility with older Emacsen.

Good point, I had missed that.  I still believe that was an unfortunate
design decision, but as Visuwesh using the declare form might help?

>> I am not familiar with isort, and I suppose there are other lightweight
>> Emacs users like me who haven't heard of it either.  It might therefore
>> be good to add a link to some documentation, especially on how to
>> install it since that can sometimes be tricky when using Python.
>
> Sure, I'll will do that.

Re: ELPA submission: python-import

[Augusto Stoffel]

On Sat, 30 Jul 2022 at 13:58, Philip Kaludercic <philip.kaludercic@fau.de> wrote:

> I am not saying it isn't impossible, but if there is a dependency on
> project.el, it might make sense to raise the minimum version of ELPA's
> python.el?  Alternatively, one could try and reduce the minimum version
> of project.el, but that might be tricky.

That would be very desirable (respectively sad if impossible).  One of
the improvements I would like to propose next is a project-dedicated
version of the Python shell (analogous to project-shell).

So that's already 2 features that would depend on project.el.

In principle it should be possible to declare that users of Emacs ≤ 26
need to stick to python.el ≤ 0.28, but I guess the package manager
doesn't allow this kind of thing, right?

> We have recently added compat as a dependency for ERC, but that only
> required support back until 27.1.  Sadly it isn't as simple as just
> adding (require 'compat nil 'noerror) at the top of the file, as compat
> is neither complete not transparent.

Okay, for sure compat doesn't free you from knowing about the history of
the features and limitations of the compat replacements.  But is there
any deeper reason why adding (require 'compat nil 'noerror) can be
problematic?

Re: ELPA submission: python-import

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Its purpose is to edit the imports at the top of Python files more
  > > conveniently.  There are already several packages for this on MELPA.
  > > This one relies on the isort program for the editing and code analysis,
  > > which is hopefully a reasonably robust solution.

Multiplying packages is added complexity.
Would it be better to add this to the existing major mode for Python?
That would avoid the extra complexity
and would ensure people who use Emacs to edit Python programs
find out about this feature.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: ELPA submission: python-import

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > FWIW, I don't find those to be good reasons to keep the code separate.
  > It's quite common for some parts of the functionality of a major modes
  > to be less thorough (e.g. depend on the presence of some unusual tool,
  > or depend on specific conventions, ...): it's not ideal, but it's better
  > than nothing, and also makes it "obvious" where to improve the code.
  > IOW it can encourage further improvements.

I don't use Python, and I don't know what isort or pyflakes do,
but if they are not extremely complicated we could write Lisp code
to do the job this facility needs.

Putting this feature in a separate package would have two disadvantages:
* It would make Emacs more complex.
* It would discourage users from finding out about the feature.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: ELPA submission: python-import

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I am not saying it isn't impossible, but if there is a dependency on
  > project.el, it might make sense to raise the minimum version of ELPA's
  > python.el?

Is it necessary for any of these python files depend on project.el?
If these features such as python-import  are added to python.el,
doesn't it eliminate the need for that?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: ELPA submission: python-import

[Augusto Stoffel]

On Sat, 30 Jul 2022 at 23:28, Richard Stallman <rms@gnu.org> wrote:

> I don't use Python, and I don't know what isort or pyflakes do,
> but if they are not extremely complicated we could write Lisp code
> to do the job this facility needs.

I've been using a similar package from MELPA (pyimport) that does
exactly that, i.e., parsing the Python code from Lisp.  Unfortunately,
the parsing is a bit naive and fails for certain code formatting styles.

Of course the said package could be improved (the task couldn't be
regarded as “extremely complicated”), but I don't see the motivation of
doing so when the effort can be offloaded to an external tool that
already exists and is free and pretty reliable.

> Putting this feature in a separate package would have two disadvantages:
> * It would make Emacs more complex.
> * It would discourage users from finding out about the feature.

Yes, I agree now.

Re: ELPA submission: python-import

[Augusto Stoffel]

On Sat, 30 Jul 2022 at 23:32, Richard Stallman <rms@gnu.org> wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > I am not saying it isn't impossible, but if there is a dependency on
>   > project.el, it might make sense to raise the minimum version of ELPA's
>   > python.el?
>
> Is it necessary for any of these python files depend on project.el?
> If these features such as python-import  are added to python.el,
> doesn't it eliminate the need for that?

AFAICT, there's nothing in python.el that helps figuring out which
project a buffer belongs to, or, say, list all files belonging that
project (which exactly is what project.el does).

python-mode mainly provides functionality to edit individual Python
files, but it's natural to add things that need to know about the
project as well.  Import management is one of them; another interesting
feature would be a Python shell that is not "global" but rather
dedicated to a project (in the same vein as `project-shell').

So I think it makes sense for python.el (and prog-modes in general) to
rely on project.el to implement some more advanced features.

Re: ELPA submission: python-import

[Philip Kaludercic]

Augusto Stoffel <arstoffel@gmail.com> writes:

> On Sat, 30 Jul 2022 at 13:58, Philip Kaludercic <philip.kaludercic@fau.de> wrote:
>
>> I am not saying it isn't impossible, but if there is a dependency on
>> project.el, it might make sense to raise the minimum version of ELPA's
>> python.el?  Alternatively, one could try and reduce the minimum version
>> of project.el, but that might be tricky.
>
> That would be very desirable (respectively sad if impossible).  One of
> the improvements I would like to propose next is a project-dedicated
> version of the Python shell (analogous to project-shell).
>
> So that's already 2 features that would depend on project.el.

I'd say we should ask the maintainer of python.el, but there appears to
be nobody who is explicitly responsible for that file.  I'd say that
Emacs 26.1 is old enough to legitimise updating the minimum version.

> In principle it should be possible to declare that users of Emacs ≤ 26
> need to stick to python.el ≤ 0.28, but I guess the package manager
> doesn't allow this kind of thing, right?

No, the "archive-contents" doesn't appear to only contain the
information for a single version of a package.

>> We have recently added compat as a dependency for ERC, but that only
>> required support back until 27.1.  Sadly it isn't as simple as just
>> adding (require 'compat nil 'noerror) at the top of the file, as compat
>> is neither complete not transparent.
>
> Okay, for sure compat doesn't free you from knowing about the history of
> the features and limitations of the compat replacements.  But is there
> any deeper reason why adding (require 'compat nil 'noerror) can be
> problematic?

Currently, not to my knowledge.  But there hasn't been any feedback on
ERC's recent change, so this might change.

Re: ELPA submission: python-import

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > I am not saying it isn't impossible, but if there is a dependency on
>   > project.el, it might make sense to raise the minimum version of ELPA's
>   > python.el?
>
> Is it necessary for any of these python files depend on project.el?
> If these features such as python-import  are added to python.el,
> doesn't it eliminate the need for that?

If the Augusto's propose features were to be added, then python.el would
depend on project.el, at the very least because of this function:

--8<---------------cut here---------------start------------->8---
(defun python-import--project-files ()
  "Return a list of all Python files in the current project.
If not in a project, just return the `default-directory'."
  (if-let (proj (project-current))
      (seq-filter (lambda (s) (string-suffix-p ".py" s))
                  (project-files proj))
    default-directory))
--8<---------------cut here---------------end--------------->8---

(where btw. `string-suffix-p' could be replaced by
`file-name-extension').

The alternative would be to reimplement `project-files' in python.el,
but I would consider that an undesirable duplication of effort that
would most likely not do what the user would want.

Re: ELPA submission: python-import

[Stefan Monnier]

> If the Augusto's propose features were to be added, then python.el would
> depend on project.el, at the very least because of this function:
>
> --8<---------------cut here---------------start------------->8---
> (defun python-import--project-files ()
>   "Return a list of all Python files in the current project.
> If not in a project, just return the `default-directory'."
>   (if-let (proj (project-current))
>       (seq-filter (lambda (s) (string-suffix-p ".py" s))
>                   (project-files proj))
>     default-directory))
> --8<---------------cut here---------------end--------------->8---
>
> (where btw. `string-suffix-p' could be replaced by
> `file-name-extension').
>
> The alternative would be to reimplement `project-files' in python.el,
> but I would consider that an undesirable duplication of effort that
> would most likely not do what the user would want.

The package doesn't need to depend on `project`: the code can simply
signal an error if project is not available.


        Stefan