Re: Help sought understanding shorthands wrt modules/packages

["Gerd Möllmann" <gerd.moellmann@gmail.com>]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > What I meant was using shorthands as the basis for a general mechanism
>   > that somehow solves a "problem" that other languages use modules for.
>
> Modules in other languages work correctly because they operate on
> variable _meanings_.  For instance, on function definitions, on
> variables, on classes, on class methods.  If one name is the name of a
> function and the name of a variable, the module system handles each of
> those meanings separately.

I must admit that I don't understand what you are referring to.  Could
you please give an example of a module system doing that?  At first
glance It sounds like visibility rules, or something, but I don't
remember a language with different visibility rules for functions and
variables, ATM.

> It's not possible to do that job correctly in Lisp at read time,
> because that precludss handling each meaning of the symbol separately.
> Inevitably, CL packages are flawed compared with other languages'
> module systems.  The basic design decision to make them work at the
> level of reading made that inevitable.  We can call this the
> "polyvalency problem."

Well.  Let me start by saying that I don't think of CL packages as
modules, but, if anything but packages, as namespaces.  The term module
is somewhat open to definition, different languages use the term
differently.  Some even have both with completely different menaings,
like C++.

>
> Module systems in other languages fit together with lexical scoping.
> Lexically scoped local variables, functions, or classes don't interact
> with the module system.  There is no way for the Lisp reader to tell
> which uses of a symbol are local.  Inevitably, 
>
>   (defun my-member (elt list) ...)
>
> is going to inherit `elt' and `list' as symbols from the package
> of standard Lisp functions, rather than the current package.

I think Michael Albinus already showed that this is not inevitable in CL
(shadow, shadowing-import).  If one wants, you can have your own elt and
car.

>
> The grave problem of :USE in CL packages could be fixed by replacing
> it with a new construct that specifies a list of symbols to be
> inherited from each other package, perhaos with renaming.

I fail to understand the "grave" problem, sorry.  Could you please give
a more concrete example?

BTW, in CL, if you don't want to use-package you can also import just
the list of symbols you want.  Without renaming, but I guess that could
be added, if somone wants that.  (I mention the idea of renaming in
cl-packages.org, BTW, on the branch, but I'm still not sure it's worth
it.)

> Instead of
> (:USE "bar") meaning to inherit all the external symbols of bar, you
> would write (:INHERIT "bar" "entrypoint" ("barhack" . "hack")).
> Instead of the kludgy "get an error if you :USE two packages that
> clash", with :INHERIT you could avoid or fix conflicts one by one.

I think what you mean can be done with import, but I'm not sure.  Like I
said, import doesn't have the renaming part.

>
> We could call this system "corrected CL packages."
> It would not be compatible but it would be better.

It could be compatible, at least I don't see a reason right now why it
couldn't.

>
> This sort of inheritance could be implemented by means of shorthands,
> but it could also be implemented directly in lread.c the same way
> shorthands are implemented.
>
> The idea of CL packages is to use multiple obarrays, but that
> implementation adds unnecessary complexity and unnecessary change.  We
> can treat names with package prefixes, such as `bar:entrypoint', as
> ordinary names and put them all in one obarray.  We could implement
> all of the corrected CL packages system with just one obarray.
>
> With this approach, reading a file in package bar would transform
> every name by concatenating `bar:' at the front.

Let me just remark that this is a read-time solution like CL packages,
so all you said above applies to shorthands as well as packages.

So, that's 1:1.

>
> Normally name concatenation is messy and desirable to avoid, but
> perhaos if done in a controlled way through a package system it would
> not be messy.  At least, not any more messy than is inevitable in a
> package system that operates at read time.

Perhaps, but I find CL packages not messy at all, TBH.  Instead, I quite
strongly feel that shorthand-packages are messy.  And I still fail to
see how shorthands address the problem a name conflicts in the first
place.  One obarray = name conflicts.  But we've been there, I think.

>
> We could hide this concatenation most of the time by making print
> cut off the package prefixes that are redundant in their context.

See, that's what I meant when I wrote "there is no design".  We could,
perhaps, maybe, maybe not, hm.  Why not use what's there and works?

And it's not even difficult to implement or understand, at least to the
degree I did, before I lost impetus.  It's not even 0.01% of redisplay,
for example.

> I wish I could have written all this in a coherent way at the start of
> this discussion.  But this is an issue I dealt with 40 years ago,
> before GNU, and it had become buried in my memory.  It has come back
> to me only gradually.

No problem.

>
> Eli wrote:
>
>   > Moreover, from my POV, the jury is still out on the question of
>   > whether we at all need packages in Emacs.  "Programming in the large"
>   > doesn't sound very relevant to how Emacs Lisp is used.
>
> I share this feeling, except that I take it a bit further.  In the
> 1980s I thought about whether we should have packages in Emacs, and
> concluded that using a naming conventions is simpler, easier and more
> flexible.  So I chose that.
>
> If CL packages were unproblematical, I'd say, "Sure, why not?  We
> aren't desperate to keep Emacs small nowadays."  But they are
> problematical, and that is a reason to choose to do without them.

Please spend some effort to explain the problematic cases. 

> The only drawback of our current use of naming conventions is the need
> to write long names in your code when referring to functions and
> variables defined in other parts of Emacs.  If your code refers to
> many such entry points of a certain part of Emacs, maybe you want
> to abbreviate those.
>
> If that is what you want perhaps shorthands are enough.  You can
> define abbreviations for the entry points that your package uses
> often.
>
> That gives you total control over what abbreviations to use in your
> package, does not impinge on any other code including the code that
> defines the entry points you want to abbreviate, and it causes no
> confusions once you understand what shorthands are.
>
> The only problem we can't avoid is the inconvenience in grep, but that
> will happen with any sort of package system that permits abbreviation
> of names.  It is not a reason to prefer CL packages.
>
> You can even avoid the polyvalency problem this way.  For instance,
> you can define `foo-fun' as a shorthand for `find-outer-otter' the
> function, and `foo-var' as a shorthand for `find-outer-otter' the
> variable.
>
> In fact, either abbreviation _could_ be used for either the function
> or the variable; there is nothing to enforce the distinction between
> those two abbreviations.  But if you wish to maintain that distinction
> in your code, you can.

I think to all of this I've already said something, so I won't bore
anyone with more :-).