bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[Ruijie Yu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

--8<---------------cut here---------------start------------->8---
In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.37, cairo version 1.17.8) of 2023-05-06 built on ruijie
Repository revision: f261226d9be4630572df322b2c4f48713c9c2fce
Repository branch: makepkg
System Description: Arch Linux
--8<---------------cut here---------------end--------------->8---

Hello,

While trying out João's breadcrumb package, I noticed a few issues
regarding `read-symbol-shorthands'.  Searching this variable on debbugs
yields zero results, so I figured maybe I should at least let the
bugtracker know about them.

First, docstrings do not resolve shorthands at the moment.  [Not the
main point of this thread.  If we are interested in changing that
behavior, we can discuss it in a new thread.]

Second, autoload generation, at least the portion responsible for
`package-vc-install', does not seem to translate the shorthands.

What I did:

    ;; ~15-days-old master
    $ emacs -Q -nw

    M-x load-lib package-vc RET

    M-: (package-vc-install '(breadcrumb
        :url "https://github.com/joaotavora/breadcrumb"
        :rev "e508856a59d18a0d006e215497b5190b3b517791")) RET

    M-x find-lib breadcrumb-autoloads.el RET

Note these portions in breadcrumb.el:

--8<---------------cut here---------------start------------->8---
;;;###autoload
(defun bc-imenu-crumbs ()
  "Describe point inside the Imenu tree of current file."
  ...)

...

;; Local Variables:
;; read-symbol-shorthands: (("bc-" . "breadcrumb-"))
;; End:
--8<---------------cut here---------------end--------------->8---

What I noticed from the generated autoloads file,
breadcrumb-autoloads.el:

--8<---------------cut here---------------start------------->8---
;;; Generated autoloads from breadcrumb.el

(autoload 'bc-imenu-crumbs "breadcrumb" "\
Describe point inside the Imenu tree of current file.")
...
--8<---------------cut here---------------end--------------->8---

The first argument to `autoload' here should probably be translated to
the full name `breadcrumb-imenu-crumbs', and likewise elsewhere.

-- 
Best,


RY

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[Ruijie Yu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Ruijie Yu via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> writes:

> While trying out João's breadcrumb package, I noticed a few issues
> regarding `read-symbol-shorthands'.  Searching this variable on debbugs
> yields zero results, so I figured maybe I should at least let the
> bugtracker know about them.

Sorry, correction: I guess one cannot just search for keywords on
debbugs.  I searched on bug archive just now and found out that there
are 29 threads on this variable.

-- 
Best,


RY

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[Stefan Kangas]

João, could you take a look at the below bug?

It seems to be related to the shorthands feature, and your package
breadcrumb.

Ruijie Yu <ruijie@netyu.xyz> writes:

> In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
>  3.24.37, cairo version 1.17.8) of 2023-05-06 built on ruijie
> Repository revision: f261226d9be4630572df322b2c4f48713c9c2fce
> Repository branch: makepkg
> System Description: Arch Linux
>
>
> Hello,
>
> While trying out João's breadcrumb package, I noticed a few issues
> regarding `read-symbol-shorthands'.  Searching this variable on debbugs
> yields zero results, so I figured maybe I should at least let the
> bugtracker know about them.
>
> First, docstrings do not resolve shorthands at the moment.  [Not the
> main point of this thread.  If we are interested in changing that
> behavior, we can discuss it in a new thread.]
>
> Second, autoload generation, at least the portion responsible for
> `package-vc-install', does not seem to translate the shorthands.
>
> What I did:
>
>     ;; ~15-days-old master
>     $ emacs -Q -nw
>
>     M-x load-lib package-vc RET
>
>     M-: (package-vc-install '(breadcrumb
>         :url "https://github.com/joaotavora/breadcrumb"
>         :rev "e508856a59d18a0d006e215497b5190b3b517791")) RET
>
>     M-x find-lib breadcrumb-autoloads.el RET
>
> Note these portions in breadcrumb.el:
>
> ;;;###autoload
> (defun bc-imenu-crumbs ()
>   "Describe point inside the Imenu tree of current file."
>   ...)
>
> ...
>
> ;; Local Variables:
> ;; read-symbol-shorthands: (("bc-" . "breadcrumb-"))
> ;; End:
>
>
> What I noticed from the generated autoloads file,
> breadcrumb-autoloads.el:
>
> ;;; Generated autoloads from breadcrumb.el
>
> (autoload 'bc-imenu-crumbs "breadcrumb" "\
> Describe point inside the Imenu tree of current file.")
> ...
>
> The first argument to `autoload' here should probably be translated to
> the full name `breadcrumb-imenu-crumbs', and likewise elsewhere.

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[João Távora]

[-- Attachment #1.1: Type: text/plain, Size: 993 bytes --]

On Wed, Sep 6, 2023, 12:03 Stefan Kangas <stefankangas@gmail.com> wrote:

> João, could you take a look at the below bug?
>
> It seems to be related to the shorthands feature, and your package
> breadcrumb.
>

The bug report for breadcrumb in itself is no longer relevant since that
code isn't there anymore.  This is a duplicate of

   https://github.com/joaotavora/breadcrumb/issues/3

which has already been solved.  I updated the autoloads to use
longhand versions already a couple of months ago.

I'll leave it to you if this issue can be closed or not.

Anyway, the problem with shorthands and autoloads remains.
The GitHub user 'dickmao' proposed a patch there, which I
didn't test, but I attach here.

My idea for an alternative patch would be for the "autoload
extractor" (which I'm not familiar with) to see the shorthand
autoload but place it in longhand form in the generated
autoloads file.  I don't have a patch for that idea,
unfortunately.

João

[-- Attachment #1.2: Type: text/html, Size: 2889 bytes --]

[-- Attachment #2: patch (1).txt --]
[-- Type: text/plain, Size: 4406 bytes --]

From 8a5275afa57160f7b7a9d71af2342c06e0c0d652 Mon Sep 17 00:00:00 2001
From: dickmao <none>
Date: Fri, 12 May 2023 21:44:47 -0400
Subject: [PATCH] propagate shorthands to autoloads

---
 lisp/emacs-lisp/generate-lisp-file.el |  7 +++++--
 lisp/emacs-lisp/loaddefs-gen.el       | 15 ++++++++++-----
 2 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/lisp/emacs-lisp/generate-lisp-file.el b/lisp/emacs-lisp/generate-lisp-file.el
index c6148003a5e..6d1e5292871 100644
--- a/lisp/emacs-lisp/generate-lisp-file.el
+++ b/lisp/emacs-lisp/generate-lisp-file.el
@@ -63,7 +63,8 @@ generate-lisp-file-heading
 
 (cl-defun generate-lisp-file-trailer (file &key version inhibit-provide
                                       (coding 'utf-8-emacs-unix) autoloads
-                                      compile provide inhibit-native-compile)
+                                      compile provide inhibit-native-compile
+                                      shorthands)
   "Insert a standard trailer for FILE.
 By default, this trailer inhibits version control, byte
 compilation, updating autoloads, and uses a `utf-8-emacs-unix'
@@ -81,7 +82,7 @@ generate-lisp-file-trailer
 If `standard-output' is bound to a buffer, insert in that buffer.
 If no, insert at point in the current buffer.
 
-If INHITBIT-NATIVE-COMPILE is non-nil, add a cookie to inhibit
+If INHIBIT-NATIVE-COMPILE is non-nil, add a cookie to inhibit
 native compilation.  (By default, a file will be native-compiled
 if it's also byte-compiled)."
   (with-current-buffer (if (bufferp standard-output)
@@ -110,6 +111,8 @@ generate-lisp-file-trailer
                       (if (eq coding t)
                           'utf-8-emacs-unix
                         coding))))
+    (when shorthands
+      (insert (format ";; read-symbol-shorthands: %S\n" shorthands)))
     (insert
      ";; End:\n\n"
      ";;; " (file-name-nondirectory file) " ends here\n")))
diff --git a/lisp/emacs-lisp/loaddefs-gen.el b/lisp/emacs-lisp/loaddefs-gen.el
index 8fafb458438..f437a952606 100644
--- a/lisp/emacs-lisp/loaddefs-gen.el
+++ b/lisp/emacs-lisp/loaddefs-gen.el
@@ -494,7 +494,7 @@ loaddefs-generate--compute-prefixes
             (push name prefs)))))
     (loaddefs-generate--make-prefixes prefs load-name)))
 
-(defun loaddefs-generate--rubric (file &optional type feature compile)
+(defun loaddefs-generate--rubric (file &optional type feature compile shorthands)
   "Return a string giving the appropriate autoload rubric for FILE.
 TYPE (default \"autoloads\") is a string stating the type of
 information contained in FILE.  TYPE \"package\" acts like the default,
@@ -520,7 +520,8 @@ loaddefs-generate--rubric
        file :provide (and (stringp feature) feature)
        :compile compile
        :inhibit-native-compile t
-       :inhibit-provide (not feature))
+       :inhibit-provide (not feature)
+       :shorthands shorthands)
       (buffer-string))))
 
 ;;;###autoload
@@ -565,7 +566,7 @@ loaddefs-generate
                                                   t files-re))
 			       (if (consp dir) dir (list dir)))))
          (updating (and (file-exists-p output-file) (not generate-full)))
-         (defs nil))
+         defs shorthands)
 
     ;; Allow the excluded files to be relative.
     (setq excluded-files
@@ -594,7 +595,11 @@ loaddefs-generate
             (when (or package-data (not excluded))
               (setq defs (nconc (loaddefs-generate--parse-file
                                  file output-file package-data)
-                                defs))))))
+                                defs)
+                    shorthands (nconc (with-temp-buffer
+                                        (save-excursion (insert-file-contents file))
+                                        (hack-read-symbol-shorthands))
+                                      shorthands))))))
       (progress-reporter-done progress))
 
     ;; First group per output file.
@@ -606,7 +611,7 @@ loaddefs-generate
           (if (and updating (file-exists-p loaddefs-file))
               (insert-file-contents loaddefs-file)
             (insert (loaddefs-generate--rubric
-                     loaddefs-file nil t include-package-version))
+                     loaddefs-file nil t include-package-version shorthands))
             (search-backward "\f")
             (when extra-data
               (insert extra-data)
-- 
2.38.1

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[Stefan Kangas]

João Távora <joaotavora@gmail.com> writes:

> This is a duplicate of
>
>    https://github.com/joaotavora/breadcrumb/issues/3
>
> which has already been solved.  I updated the autoloads to use
> longhand versions already a couple of months ago.

OK, thanks.

> I'll leave it to you if this issue can be closed or not.

I guess the issue here is what you describe below?  If so, I guess it
should remain open until it is fixed.

> Anyway, the problem with shorthands and autoloads remains.
> The GitHub user 'dickmao' proposed a patch there, which I
> didn't test, but I attach here.
>
> My idea for an alternative patch would be for the "autoload
> extractor" (which I'm not familiar with) to see the shorthand
> autoload but place it in longhand form in the generated
> autoloads file.  I don't have a patch for that idea,
> unfortunately.

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[Joseph Turner via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Also see https://lists.gnu.org/archive/html/bug-gnu-emacs/2023-11/msg01151.html

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[Joseph Turner via Bug reports for GNU Emacs, the Swiss army knife of text editors]

From: Stefan Kangas <stefankangas@gmail.com>

> João Távora <joaotavora@gmail.com> writes:

> > This is a duplicate of
> >
> >    https://github.com/joaotavora/breadcrumb/issues/3
> >
> > which has already been solved.  I updated the autoloads to use
> > longhand versions already a couple of months ago.

> OK, thanks.

> > I'll leave it to you if this issue can be closed or not.

> I guess the issue here is what you describe below?  If so, I guess it
> should remain open until it is fixed.

> > Anyway, the problem with shorthands and autoloads remains.
> > The GitHub user 'dickmao' proposed a patch there, which I
> > didn't test, but I attach here.
> >
> > My idea for an alternative patch would be for the "autoload
> > extractor" (which I'm not familiar with) to see the shorthand
> > autoload but place it in longhand form in the generated
> > autoloads file.  I don't have a patch for that idea,
> > unfortunately.

This bug affects not only autoloaded function definitions, but any
autoloaded form.  For example, the following will not generate the
correct autoloads:

;;;###autoload
(with-eval-after-load 'optional-dependency
  (cl-pushnew 'sns-var optional-dependency-var))

Instead, you must expand the shorthand symbol into its full form:

;;;###autoload
(with-eval-after-load 'optional-dependency
  (cl-pushnew 'some-nice-string-utils-var optional-dependency-var))

For a real-world example, see

https://git.sr.ht/~ushin/hyperdrive.el/commit/48afc51ac22713f2c54f75a7ea3b5c6b0e12b53d

Thank you!

Joseph

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 712 bytes --]

On Wed, Nov 29, 2023, 07:45 Joseph Turner <joseph@breatheoutbreathe.in>
wrote:

> :
>
> ;;;###autoload
> (with-eval-after-load 'optional-dependency
>   (cl-pushnew 'sns-var optional-dependency-var))
>
> Instead, you must expand the shorthand symbol into its full form:
>
> ;;;###autoload
> (with-eval-after-load 'optional-dependency
>   (cl-pushnew 'some-nice-string-utils-var optional-dependency-var))
>
> For a real-world example, see
>
>
> https://git.sr.ht/~ushin/hyperdrive.el/commit/48afc51ac22713f2c54f75a7ea3b5c6b0e12b53d


Thanks for the extra case. I think this merits a fix in the autoload
extractor, watch should become autoload aware. I'll try to look at it.

João

>
>

[-- Attachment #2: Type: text/html, Size: 1435 bytes --]

feature that was needed for read-symbol-shorthands

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

When the shorthands feasture was installed, I found that another
feature was needed: a way to explicitly load a specified file, and
specify an added shorthands list for the contents of that file, to be
used in addition to what the file itself specifies.

Did that ever get implemented?  Is there a way to do it now?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: feature that was needed for read-symbol-shorthands

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1039 bytes --]

On Sat, Dec 2, 2023, 03:19 Richard Stallman <rms@gnu.org> wrote:

When the shorthands feasture was installed, I found that another
> feature was needed: a way to explicitly load a specified file, and
> specify an added shorthands list for the contents of that file, to be
> used in addition to what the file itself specifies.
>
> Did that ever get implemented?


I don't think so, no.

  Is there a way to do it now?
>

Maybe. Have you thought about practical consequences? The file would no
longer be the single source of truth as to what symbols are interned when
it is read.

For example, visiting such a file after having it loaded like that would
lead to surprising behavior for many features (help buffers, eldoc, etc).
And it would become impossible to analyze the file without loading it or
guessing how it is meant to be loaded.

What about compiled files? Would you do shorthand substitution at
elc-loading time? It's not read-time anymore.

For these reasons I recommend against doing this.

João

[-- Attachment #2: Type: text/html, Size: 1843 bytes --]

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[Joseph Turner via Bug reports for GNU Emacs, the Swiss army knife of text editors]

João Távora <joaotavora@gmail.com> writes:

> On Wed, Nov 29, 2023, 07:45 Joseph Turner <joseph@breatheoutbreathe.in> wrote:
>
>  :
>
>  ;;;###autoload
>  (with-eval-after-load 'optional-dependency
>    (cl-pushnew 'sns-var optional-dependency-var))
>
>  Instead, you must expand the shorthand symbol into its full form:
>
>  ;;;###autoload
>  (with-eval-after-load 'optional-dependency
>    (cl-pushnew 'some-nice-string-utils-var optional-dependency-var))
>
>  For a real-world example, see
>
>  https://git.sr.ht/~ushin/hyperdrive.el/commit/48afc51ac22713f2c54f75a7ea3b5c6b0e12b53d
>
> Thanks for the extra case. I think this merits a fix in the autoload extractor, watch should become autoload aware. I'll try to look at it.

I believe this bug was fixed in c52d17d91ad

Can we close this bug?

Thanks,

Joseph

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[João Távora]

On Thu, Apr 18, 2024 at 9:01 PM Joseph Turner
<joseph@breatheoutbreathe.in> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> > Thanks for the extra case. I think this merits a fix in the autoload extractor, watch should become autoload aware. I'll try to look at it.
>
> I believe this bug was fixed in c52d17d91ad
>
> Can we close this bug?

Fine by me, you're the one originally requesting all these bug fixes,
so if you confirm that everything's working correctly with your
extensive use of shorthands, we should close.

But I've been away from Emacs development for the last months (and
hope to be disconnecting myself more progressively) so I might not be
the best person to ask.

João

bug#63480: 30.0.50; [BUG] unimplemented logic regarding read-symbol-shorthands

[Joseph Turner via Bug reports for GNU Emacs, the Swiss army knife of text editors]

João Távora <joaotavora@gmail.com> writes:

> On Thu, Apr 18, 2024 at 9:01 PM Joseph Turner
> <joseph@breatheoutbreathe.in> wrote:
>>
>> João Távora <joaotavora@gmail.com> writes:
>>
>> > Thanks for the extra case. I think this merits a fix in the
>> > autoload extractor, watch should become autoload aware. I'll try
>> > to look at it.
>>
>> I believe this bug was fixed in c52d17d91ad
>>
>> Can we close this bug?
>
> Fine by me, you're the one originally requesting all these bug fixes,
> so if you confirm that everything's working correctly with your
> extensive use of shorthands, we should close.
>
> But I've been away from Emacs development for the last months (and
> hope to be disconnecting myself more progressively) so I might not be
> the best person to ask.

Thank you, João!

Warmly,

Joseph