Better way to require with shorthands/renamed symbols

Better way to require with shorthands/renamed symbols

[Stefan Kangas]

joaotavora@gmail.com (João Távora) writes:

> +(defun t-reverse-lines (s) (string-join (reverse (sns-lines s)) "\n")
> +
> +;; Local Variables:
> +;; elisp-shorthands: (("t-" . "my-tricks-")
> +;;                    ("sns-" . "some-nice-string-utils-"))
> +;; End:

This is really nice and straight-forward, thanks!

I see one problem here.  It basically splits the require up in two:
I need to look both at the start and at the end of the file to
understand what is going on.

In the previous discussion, Adam Porter suggested this:[1]

    (shorthand-require 'magnar-string :with "^s-" :as "magnar-string-")

I like that proposal a lot, but it's a bit too verbose for my taste.
It also seems to flip the :as and :with from what I would expect (in
Python, "import x as y" means that x is available "as" y in this file).

In my ideal world, we should be able to just say simply:

    (require 'magnar-string :as "s-")

There could be a variable inside magnar-string.el that gives us the
rest, e.g.:

    elisp-default-import-prefix: "^magnar-string-"

WDYT?

Footnotes:
[1]  https://lists.gnu.org/r/emacs-devel/2020-05/msg01754.html

Re: Better way to require with shorthands/renamed symbols

[Dmitry Gutov]

On 27.09.2021 14:10, Stefan Kangas wrote:
> In my ideal world, we should be able to just say simply:
> 
>      (require 'magnar-string :as "s-")

This would be much nicer, yes.

Re: Better way to require with shorthands/renamed symbols

[João Távora]

Stefan Kangas <stefankangas@gmail.com> writes:

> joaotavora@gmail.com (João Távora) writes:
>
>> +(defun t-reverse-lines (s) (string-join (reverse (sns-lines s)) "\n")
>> +
>> +;; Local Variables:
>> +;; elisp-shorthands: (("t-" . "my-tricks-")
>> +;;                    ("sns-" . "some-nice-string-utils-"))
>> +;; End:
>
> This is really nice and straight-forward, thanks!
>
> I see one problem here.  It basically splits the require up in two:
> I need to look both at the start and at the end of the file to
> understand what is going on.
>
> In the previous discussion, Adam Porter suggested this:[1]
>
>     (shorthand-require 'magnar-string :with "^s-" :as "magnar-string-")
>
> I like that proposal a lot, but it's a bit too verbose for my taste.
> It also seems to flip the :as and :with from what I would expect (in
> Python, "import x as y" means that x is available "as" y in this file).
>
> In my ideal world, we should be able to just say simply:
>
>     (require 'magnar-string :as "s-")
>

I wrote before that something like is reasonable and sounds useful.  But
I was rash in that assessment.  Or rather, it must be very clearly
thought out.

One should understand (or perhaps, as a programmer, one _must_
understand) that the Shorthands feature works at the Lisp reader level.

It's tricky to think about the execution of that manifestation of the
'require' special and what it does at the Shorthands level.  You're
probably thinking of putting it on the top of your file, which is the
most common case, but far from the only use.  What if it happens at
non-top level, in an IF form?  What if it happen sat top level but in
the middle of the file?  It would amount to setting variables that
influence the reader during the read process.

Of course, that is exactly what Common Lisp's IN-PACKAGE does.  But that
comes with other challenges, prominent among which teaching tools like
xref, ElDoc, describe-function and completions what the "current"
shorthands are.  You see, those tools are always finding symbols in the
current buffer and they also need to know the value of
'elisp-shorthands' (soon to be renamed 'read-shorthands', hopefully).

We'd need to update 'elisp-shorthands' whenever point moves perhaps? Or
just-in-time before an 'intern-soft' is called.  That is how SLY and
SLIME do it.  Maybe.  At any rate, this is more work.  Perhaps, when we
arrive at that that point, we decide to put the effort and implement
IN-PACKAGE, since it's so well understood (...but is it? Not by some, I
fear, maybe that'll change).

For now, I'd just like people to experiment with Shorthands.

As Stefan Monnier once wrote (here?), implementing namespace systems
isn't hard in itself (it wasn't here).  It's making the tooling around
it consistent.  The latter was only possible with the Shorthands feature
because of the relative dumbness of its approach.

João

Shorthands have landed on master (Was: Better way to require with shorthands/renamed symbols)

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 213 bytes --]

Hello all.

Anyway, as you may have noticed, "Shorthands" have landed on master.
Please check out the NEWS entry and the manual section on "Shorthands".

For the impatient, I leave with with a tiny animated gif.


[-- Attachment #2: shorthand-demo.gif --]
[-- Type: image/gif, Size: 846811 bytes --]

[-- Attachment #3: Type: text/plain, Size: 560 bytes --]


Dmitry Gutov <dgutov@yandex.ru> writes:
> On 27.09.2021 14:10, Stefan Kangas wrote:
>> In my ideal world, we sho  uld be able to just say simply:
>>      (require 'magnar-string :as "s-")
> This would be much nicer, yes.

Unfortunately, that comes with many other implications that you may or
may not be seeing.  I've replied to Stefan in detail.  Anyway, I hope
people can briefly try out the feature as it is now, and report any
bugs, slightly delaying their "much nicer" suggestion (which are more
than expected and very welcome).

João

Re: Better way to require with shorthands/renamed symbols

[Dmitry Gutov]

On 27.09.2021 15:24, João Távora wrote:

> One should understand (or perhaps, as a programmer, one _must_
> understand) that the Shorthands feature works at the Lisp reader level.
> 
> It's tricky to think about the execution of that manifestation of the
> 'require' special and what it does at the Shorthands level.  You're
> probably thinking of putting it on the top of your file, which is the
> most common case, but far from the only use.  What if it happens at
> non-top level, in an IF form?  What if it happen sat top level but in
> the middle of the file?  It would amount to setting variables that
> influence the reader during the read process.

I think the key part is for this info to reside somewhere near the top 
of the file (if not in the require form, then above it, maybe).

> As Stefan Monnier once wrote (here?), implementing namespace systems
> isn't hard in itself (it wasn't here).  It's making the tooling around
> it consistent.  The latter was only possible with the Shorthands feature
> because of the relative dumbness of its approach.

BTW, it seems the new feature will need dedicated support in 
xref-backend-references as well.

Re: Shorthands have landed on master (Was: Better way to require with shorthands/renamed symbols)

[Dmitry Gutov]

On 27.09.2021 15:54, João Távora wrote:
> Anyway, I hope
> people can briefly try out the feature as it is now, and report any
> bugs, slightly delaying their "much nicer" suggestion (which are more
> than expected and very welcome).

I will, thanks.

Re: Better way to require with shorthands/renamed symbols

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 27.09.2021 15:24, João Távora wrote:
>
>> One should understand (or perhaps, as a programmer, one _must_
>> understand) that the Shorthands feature works at the Lisp reader level.
>> It's tricky to think about the execution of that manifestation of
>> the
>> 'require' special and what it does at the Shorthands level.  You're
>> probably thinking of putting it on the top of your file, which is the
>> most common case, but far from the only use.  What if it happens at
>> non-top level, in an IF form?  What if it happen sat top level but in
>> the middle of the file?  It would amount to setting variables that
>> influence the reader during the read process.
>
> I think the key part is for this info to reside somewhere near the top
> of the file (if not in the require form, then above it, maybe).

OK.  I agree that that is much better from a usability point of view.
And the key part of my position is that currently, we shouldn't try to
make it a part of Lisp code.  Maybe we can somehoe have file-local
variables in other points of the file?

Would that suffice for you and Stefan now?

João

Re: Better way to require with shorthands/renamed symbols

[Stefan Kangas]

João Távora <joaotavora@gmail.com> writes:

> [...] Maybe we can somehoe have file-local
> variables in other points of the file?
>
> Would that suffice for you and Stefan now?

Sure, if we can figure out how to do that in a clean way.  Otherwise, we
are probably better off just leaving things as they are, rather than
complicating things just for the sake of it.  Maybe it's already fine as
it is, given the scope of the feature.

I guess most files will not use this, so it should okay if those that
do just add a note "see below" to their requires section.

Re: Better way to require with shorthands/renamed symbols

[João Távora]

On Mon, Sep 27, 2021 at 4:05 PM Stefan Kangas <stefankangas@gmail.com> wrote:

> > [...] Maybe we can somehoe have file-local
> > variables in other points of the file?
> > Would that suffice for you and Stefan now?
> Sure, if we can figure out how to do that in a clean way.

I don't think it's particularly hard to do cleanly.  Just think of how
file-local variables are already supported in the first line
of a file.

>  Otherwise, we are probably better off just leaving things as they are,
> rather than complicating things just for the sake of it.

Yes, "for the sake of it" is obviously bad.  But solving particular
problems that are well described is fine with me.  For example, I'd like
to be able to activate shorthands in certain parts of the buffer, just like
one switches to a package in Common Lisp.  But I'm a CL guy and like
packages.

On this topic, it has come to my attention that some people are sore
or frustrated about how this feature has somehow "defeated" their
preferred idea.  I think that doesn't make sense.  If those ideas aren't
happening it is not because of my work.

I hope people come forward and state what it is they would like to do
namespacing-wise in Elisp.  And then help everyone else think if it is
possible to actually implement it without impacting the existing tooling
(too much). Ideally, a prototype of this hypothetical feature be made.

> Maybe it's already fine as  it is, given the scope of the feature.

The Shorthands feature doesn't have any particular "scope", IMO. It
depends on the use you want to give it.  I believe it will be used for
importing `s.el` and `s.el`-dependent libraries into Emacs or *-ELPA
somehow. That's one application.

Personally, I plan on using it in newer Emacs28-only versions of my
packages: i hate typing/reading long prefixes. Maybe a clean Elisp
version can be developed for older Emacsen, I dunno (the first
version I did was Elisp only, but I don't remember if it worked fully
like this one).

João

Re: Better way to require with shorthands/renamed symbols

[Stefan Kangas]

João Távora <joaotavora@gmail.com> writes:

>>  Otherwise, we are probably better off just leaving things as they are,
>> rather than complicating things just for the sake of it.
>
> Yes, "for the sake of it" is obviously bad.  But solving particular
> problems that are well described is fine with me.

I guess I'm trying to say that it would also be fine to wait until we
have more experience with this.

In any case, I fully trust you will do the right thing with this
feedback.

> On this topic, it has come to my attention that some people are sore
> or frustrated about how this feature has somehow "defeated" their
> preferred idea.  I think that doesn't make sense.  If those ideas aren't
> happening it is not because of my work.

Fully agreed, that makes no sense.

> The Shorthands feature doesn't have any particular "scope", IMO. It
> depends on the use you want to give it.  I believe it will be used for
> importing `s.el` and `s.el`-dependent libraries into Emacs or *-ELPA
> somehow. That's one application.
>
> Personally, I plan on using it in newer Emacs28-only versions of my
> packages: i hate typing/reading long prefixes. Maybe a clean Elisp
> version can be developed for older Emacsen, I dunno (the first
> version I did was Elisp only, but I don't remember if it worked fully
> like this one).

Absolutely, I intend to experiment with it as well.  Currently, I use
the nameless package (and have done for many years), and I'm quite happy
with it.  It basically detects in "my-long-library-name.el" symbols
looking like `my-long-library-name-foo-bar' and hides the package part
so you only see `:foo-bar'.  The text is still there, but it is out of
sight.  Extremely handy, and it works in any package.

It obviously doesn't solve the problem of very short prefix packages
like "s.el", etc.  It's mostly just a very useful hack.

The main drawback though is that you still need to type out the package
name.  So I intend to start experimenting with very short shorthand
prefixes like ":".  It will be interesting to see how useful/usable it
turns out to be.

Re: Better way to require with shorthands/renamed symbols

[João Távora]

On Mon, Sep 27, 2021 at 9:12 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> >>  Otherwise, we are probably better off just leaving things as they are,
> >> rather than complicating things just for the sake of it.
> >
> > Yes, "for the sake of it" is obviously bad.  But solving particular
> > problems that are well described is fine with me.
>
> I guess I'm trying to say that it would also be fine to wait until we
> have more experience with this.

Yeah, that is exactly what I wish will happen.  For people to try
it and to detect where it solves their problems and where it doesn't.
Not to mention say where it has _created_ problems (i.e. where
it might have introduced bugs).

João

Re: Shorthands have landed on master

[Philip Kaludercic]

João Távora <joaotavora@gmail.com> writes:

> Hello all.
>
> Anyway, as you may have noticed, "Shorthands" have landed on master.
> Please check out the NEWS entry and the manual section on "Shorthands".
>
> For the impatient, I leave with with a tiny animated gif.

From what I see, elisp-shorthands only allows a shorter prefix to be
mapped to a longer prefix, right? So this doesn't allow me to use cl-lib
without the cl- prefix, or remove the compat-- prefix from compatibility
functions?

-- 
	Philip Kaludercic

Re: Shorthands have landed on master

[João Távora]

On Mon, Sep 27, 2021 at 11:40 PM Philip Kaludercic <philipk@posteo.net> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> > Hello all.
> >
> > Anyway, as you may have noticed, "Shorthands" have landed on master.
> > Please check out the NEWS entry and the manual section on "Shorthands".
> >
> > For the impatient, I leave with with a tiny animated gif.
>
> From what I see, elisp-shorthands only allows a shorter prefix to be
> mapped to a longer prefix, right? So this doesn't allow me to use cl-lib
> without the cl- prefix,

You need to have _some_ prefix.  It cannot be of 0 length.  Best
you can do is '-loop' and '-destructuring-bind', for example.

In fact, I lie.  You _can_ map 'loop' to 'cl-loop' explicitly.
'loop' is the shorthand.  'cl-loop' is still the symbol.
But you have to do it symbol by symbol.

That's because I've purposely turned off "freer" renamings in
favor of prefixes, simplicity and read speed.  But they are
not impossible.

> or remove the compat-- prefix from compatibility
> functions?

It's too late in the day for me to understand what you mean by
"remove" here :-)  You must first understand what shorthands are:
they are merely file-local aliases to symbols which keep their
full names.

Does this help in answering your question?

If it doesn't, provide a toy example with files and code in it,
and illustrate clearly what you would like to be able to write,
but can't (right now) Then maybe I can say if shorthands can
help you.

João

Re: Better way to require with shorthands/renamed symbols

[T.V Raman]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=gb18030, Size: 496 bytes --]

This is a suggestion for thinking about *after* we have had some
experience using this:

Consider a prefix separator char like ':' so that the shorthand is
easily distinguishable as a shorthand prefix? Might help in many ways:

1. Indicates to the reader that new: is a shorthand prefix.
   2. Enable refactoring/renaming code to easily change the prefix or
      undo it if desired 
-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
7©4 Id: kg:/m/0285kf1  •0Ü8

Re: Better way to require with shorthands/renamed symbols

[Ihor Radchenko]

Stefan Kangas <stefankangas@gmail.com> writes:

> The main drawback though is that you still need to type out the package
> name.

You can simply
(bind-key "_" #'nameless-insert-name-or-self-insert nameless-mode-map)

Best,
Ihor

Re: Shorthands have landed on master

[Philip Kaludercic]

João Távora <joaotavora@gmail.com> writes:

> On Mon, Sep 27, 2021 at 11:40 PM Philip Kaludercic <philipk@posteo.net> wrote:
>>
>> João Távora <joaotavora@gmail.com> writes:
>>
>> > Hello all.
>> >
>> > Anyway, as you may have noticed, "Shorthands" have landed on master.
>> > Please check out the NEWS entry and the manual section on "Shorthands".
>> >
>> > For the impatient, I leave with with a tiny animated gif.
>>
>> From what I see, elisp-shorthands only allows a shorter prefix to be
>> mapped to a longer prefix, right? So this doesn't allow me to use cl-lib
>> without the cl- prefix,
>
> You need to have _some_ prefix.  It cannot be of 0 length.  Best
> you can do is '-loop' and '-destructuring-bind', for example.
>
> In fact, I lie.  You _can_ map 'loop' to 'cl-loop' explicitly.
> 'loop' is the shorthand.  'cl-loop' is still the symbol.
> But you have to do it symbol by symbol.
>
> That's because I've purposely turned off "freer" renamings in
> favor of prefixes, simplicity and read speed.  But they are
> not impossible.

I see. Do you think it would make sense to use predefined shorthand
groups, in case someone *would* want to use cl-lib without the prefix?

>> or remove the compat-- prefix from compatibility
>> functions?
>
> It's too late in the day for me to understand what you mean by
> "remove" here :-)  You must first understand what shorthands are:
> they are merely file-local aliases to symbols which keep their
> full names.

This was in reference to the compat.el discussion from a few days
ago. It was mentioned that shorthands might provide an alternative to
aliasing function definitions. But this doesn't make much sense, if a
prefix has to be used, short or long: I want to call
(macroexp-file-name), not (co-macroexp-file-name), (~macroexp-file-name)
or (compat--macroexp-file-name) to provide transparent compatibility.

> Does this help in answering your question?

After trying out

     ;; elisp-shorthands: (("" . "cl-"))

I saw what went wrong and I understand why. On that topic, I think the
behaviour was inadequate, because a lot of unrelated commands also
broke that might become annoying.

> If it doesn't, provide a toy example with files and code in it,
> and illustrate clearly what you would like to be able to write,
> but can't (right now) Then maybe I can say if shorthands can
> help you.
>
> João
>
>

-- 
	Philip Kaludercic

Re: Shorthands have landed on master

[João Távora]

On Tue, Sep 28, 2021 at 8:15 AM Philip Kaludercic <philipk@posteo.net> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> > On Mon, Sep 27, 2021 at 11:40 PM Philip Kaludercic <philipk@posteo.net> wrote:

> I see. Do you think it would make sense to use predefined shorthand
> groups, in case someone *would* want to use cl-lib without the prefix?

Yes, I do think that makes sense.

...meaning that drawbacks aren't immediately visible to me right now :-)

Since elisp-shorthands is a file-local variable, I think it allows (eval)
specs.  Perhaps you can try with that, for now? I don't think it'd be
hard to add better support for such predefined groups of shorthands.

> This was in reference to the compat.el discussion from a few days
> ago. It was mentioned that shorthands might provide an alternative to
> aliasing function definitions. But this doesn't make much sense, if a
> prefix has to be used, short or long: I want to call
> (macroexp-file-name), not (co-macroexp-file-name), (~macroexp-file-name)
> or (compat--macroexp-file-name) to provide transparent compatibility.

If I understand this right, then I think you may solve it similarly to the
cl- situation.

But you understand that, for the moment, shorthands are only supported in
Emacs 28, right?  Not sure where (in which versions) the library you're
intending to write would be loaded.  If that library is loaded in Emacs 27,
elisp-shorthands is simply ignored and the file is loaded normally.

João

Re: Shorthands have landed on master

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Tue, 28 Sep 2021 10:03:01 +0100
> Cc: Adam Porter <adam@alphapapa.net>, Dmitry Gutov <dgutov@yandex.ru>,
>  Stefan Kangas <stefankangas@gmail.com>, emacs-devel <emacs-devel@gnu.org>
> 
> Since elisp-shorthands is a file-local variable, I think it allows (eval)
> specs.

But that will be unsafe, no?

Re: Shorthands have landed on master

[João Távora]

On Tue, Sep 28, 2021 at 10:14 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Tue, 28 Sep 2021 10:03:01 +0100
> > Cc: Adam Porter <adam@alphapapa.net>, Dmitry Gutov <dgutov@yandex.ru>,
> >  Stefan Kangas <stefankangas@gmail.com>, emacs-devel <emacs-devel@gnu.org>
> >
> > Since elisp-shorthands is a file-local variable, I think it allows (eval)
> > specs.
>
> But that will be unsafe, no?

Yes, this is why I wrote "for now".  It does seem plausible that
Emacs provides pre-canned alists of substitutions such as
the one used for reading 'loop' as 'cl-loop', 'second' as
'cl-second', and so on. Those would be "safe" AFAIK and we
could have a mechanism to mark them so.

João

Re: Shorthands have landed on master

[Philip Kaludercic]

João Távora <joaotavora@gmail.com> writes:

>> This was in reference to the compat.el discussion from a few days
>> ago. It was mentioned that shorthands might provide an alternative to
>> aliasing function definitions. But this doesn't make much sense, if a
>> prefix has to be used, short or long: I want to call
>> (macroexp-file-name), not (co-macroexp-file-name), (~macroexp-file-name)
>> or (compat--macroexp-file-name) to provide transparent compatibility.
>
> If I understand this right, then I think you may solve it similarly to the
> cl- situation.
>
> But you understand that, for the moment, shorthands are only supported in
> Emacs 28, right?  Not sure where (in which versions) the library you're
> intending to write would be loaded.  If that library is loaded in Emacs 27,
> elisp-shorthands is simply ignored and the file is loaded normally.

Yes, my idea was to use shorthands for versions greater or equal to
28.1. 

> João

-- 
	Philip Kaludercic

Re: Shorthands have landed on master (Was: Better way to require with shorthands/renamed symbols)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Anyway, as you may have noticed, "Shorthands" have landed on master.

Congratulations and thanks.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Better way to require with shorthands/renamed symbols

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Consider a prefix separator char like ':' so that the shorthand is
  > easily distinguishable as a shorthand prefix? Might help in many ways:

We plan to use this for existing prefixes that do not have a colon.
For instance, 's-'.

 
-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)