A short defense of shorthands.el (but CL packages are still better) (Was: Help sought understanding shorthands wrt modules/packages)

["João Távora" <joaotavora@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4443 bytes --]

On Thu, Nov 3, 2022 at 5:12 AM Gerd Möllmann <gerd.moellmann@gmail.com>
wrote:

    João Távora <joaotavora@gmail.com> writes:

    > Hi Gerd, I'm there one who implemented shorthands in Emacs, and i sure
    > don't think they are a substitute for CL packages.

    Thanks for letting me know, Joao.

No problem.  Allow me to mount a short defense of shorthands.  In doing
so, I don't mean to imply -- by any measure -- that CL packages are a
much welcomed addition to Emacs. They are, absolutely.

* shorthands are designed to perform namespacing operations with minimal
  or even no changes to the Lisp forms of an Elisp file.  The
  hypothetical file x.el:

  (defvar x--bar 42)
  (defun x-foo () x--bar)
  (provide 'x)
  ;; x.el ends here
  ;; Local Variables:
  ;; read-symbol-shorthands: (("x-" . "xenomorph-"))
  ;; End:

  which will pollute the global namespace when loaded into Emacs 27,
  will, when loaded into Emacs >28, intern xenomorph--bar and
  xenomorph-foo instead of x--bar and x-foo.

  Its user file yummy.el

  (require 'x)
  (defun yummy () (x-foo))
  ;; yummy.el ends here
  ;; Local Variables:
  ;; read-symbol-shorthands: (("x-" . "xenomorph-"))
  ;; End:

  can also be loaded into Emacs 27 and Emacs 28.  The interaction
  between the two packages works in both cases, but in Emacs 28 the
  global namespace won't be polluted.

* The above use case was motivated by the s.el, dash.el and f.el
  libraries which incur in this namespace pollution.  To be clear, all
  packages pollute the namespace but these short prefixed ones were
  especially heavy polluters, since short names naturally appear more
  frequently in completion lists.

* If CL packages had been used instead, this "double duty" wouldn't have
  been possible, because x.el and y.el would have to be changed
  considerably (admittedly in a rather straightforward fashion)

* When shorthands were presented, much criticism was leveled at it, some
  stemming from a misunderstanding of the specific problem this attempts
  to solve.

* But one of the criticisms is pretty reasonable: this "breaks grep"
  because the same symbol can now be referenced by two different
  character strings from two different contexts.  Also two different
  symbols are designated by the same character string, again in two
  different contexts.

  However, the same is true for every namespacing facility by
  definition.  This is what namespacing systems do.

 * Anyway, the problem is that

    grep xenomorph-foo x.el yummy.el

   doesn't return anything, even though xenomorph-foo is really
   the name of the symbol in obarray.  Obviously, grep doesn't
   understand ELisp.

* In my opinion, the part that is missing from shorthands is a tool that
  replaces grep (for Lisp symbolic uses, of course) and understands and
  can be used as a backend for xref-find-references.  One can think
  of different approaches for realizing this tool

* The most promising approach, IMO, to fix this is to create a new
  binary program, call it 'sexgrep' (for "Symbolic Expression Grep")
  which can be run separately from Emacs but which uses Emacs's reader
  syntax.  It could reuse lread.c maybe, or reimplement relevant parts
  of it.  The program's input is a full symbol name and a number of
  Elisp files.  By using a source-tracking reader and understanding the
  relatively simple syntax of shorthand definitions, it'll make

    sexpgrep xenomorph-foo yummy.el x.el

  find matches for the search pattern in those two files (on line 2)
  respectively.

* Taking the approach with CL packages would be more difficult, because
  the program would need to have a Lisp evaluator that understands
  CL:DEFPACKAGE and CL:IN-PACKAGE, not just a reader.  This is why, in
  the Common Lisp implementations that I've used, a global in-memory
  database of symbols is used instead.  The database is kept up-to-date
  whenever the code is read and loaded (which may occur in two different
  moments in time, if compiled files are used).

  This also works nicely (SLIME and SLY use it to great effect).  But it
  fails to search code that isn't loaded and is subject to some annoying
  but resolvable problems (like when loading CL fasls that were compiled
  on a different machine, for example.  Maybe Helmut Eller, SLIME
  author, has some good input in this regard)

[-- Attachment #2: Type: text/html, Size: 5007 bytes --]