Re: Help sought understanding shorthands wrt modules/packages

[Richard Stallman <rms@gnu.org>]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What I meant was using shorthands as the basis for a general mechanism
  > that somehow solves a "problem" that other languages use modules for.

Modules in other languages work correctly because they operate on
variable _meanings_.  For instance, on function definitions, on
variables, on classes, on class methods.  If one name is the name of a
function and the name of a variable, the module system handles each of
those meanings separately.  

It's not possible to do that job correctly in Lisp at read time,
because that precludss handling each meaning of the symbol separately.
Inevitably, CL packages are flawed compared with other languages'
module systems.  The basic design decision to make them work at the
level of reading made that inevitable.  We can call this the
"polyvalency problem."

Module systems in other languages fit together with lexical scoping.
Lexically scoped local variables, functions, or classes don't interact
with the module system.  There is no way for the Lisp reader to tell
which uses of a symbol are local.  Inevitably, 

  (defun my-member (elt list) ...)

is going to inherit `elt' and `list' as symbols from the package
of standard Lisp functions, rather than the current package.

The grave problem of :USE in CL packages could be fixed by replacing
it with a new construct that specifies a list of symbols to be
inherited from each other package, perhaos with renaming.  Instead of
(:USE "bar") meaning to inherit all the external symbols of bar, you
would write (:INHERIT "bar" "entrypoint" ("barhack" . "hack")).
Instead of the kludgy "get an error if you :USE two packages that
clash", with :INHERIT you could avoid or fix conflicts one by one.

We could call this system "corrected CL packages."
It would not be compatible but it would be better.

This sort of inheritance could be implemented by means of shorthands,
but it could also be implemented directly in lread.c the same way
shorthands are implemented.

The idea of CL packages is to use multiple obarrays, but that
implementation adds unnecessary complexity and unnecessary change.  We
can treat names with package prefixes, such as `bar:entrypoint', as
ordinary names and put them all in one obarray.  We could implement
all of the corrected CL packages system with just one obarray.

With this approach, reading a file in package bar would transform
every name by concatenating `bar:' at the front.

Normally name concatenation is messy and desirable to avoid, but
perhaos if done in a controlled way through a package system it would
not be messy.  At least, not any more messy than is inevitable in a
package system that operates at read time.

We could hide this concatenation most of the time by making print
cut off the package prefixes that are redundant in their context.

I wish I could have written all this in a coherent way at the start of
this discussion.  But this is an issue I dealt with 40 years ago,
before GNU, and it had become buried in my memory.  It has come back
to me only gradually.

Eli wrote:

  > Moreover, from my POV, the jury is still out on the question of
  > whether we at all need packages in Emacs.  "Programming in the large"
  > doesn't sound very relevant to how Emacs Lisp is used.

I share this feeling, except that I take it a bit further.  In the
1980s I thought about whether we should have packages in Emacs, and
concluded that using a naming conventions is simpler, easier and more
flexible.  So I chose that.

If CL packages were unproblematical, I'd say, "Sure, why not?  We
aren't desperate to keep Emacs small nowadays."  But they are
problematical, and that is a reason to choose to do without them.

The only drawback of our current use of naming conventions is the need
to write long names in your code when referring to functions and
variables defined in other parts of Emacs.  If your code refers to
many such entry points of a certain part of Emacs, maybe you want
to abbreviate those.

If that is what you want perhaps shorthands are enough.  You can
define abbreviations for the entry points that your package uses
often.

That gives you total control over what abbreviations to use in your
package, does not impinge on any other code including the code that
defines the entry points you want to abbreviate, and it causes no
confusions once you understand what shorthands are.

The only problem we can't avoid is the inconvenience in grep, but that
will happen with any sort of package system that permits abbreviation
of names.  It is not a reason to prefer CL packages.

You can even avoid the polyvalency problem this way.  For instance,
you can define `foo-fun' as a shorthand for `find-outer-otter' the
function, and `foo-var' as a shorthand for `find-outer-otter' the
variable.

In fact, either abbreviation _could_ be used for either the function
or the variable; there is nothing to enforce the distinction between
those two abbreviations.  But if you wish to maintain that distinction
in your code, you can.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)