bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Dave Abrahams]

[I hope this is the right venue for feature requests.  If not, sorry!]

When a file can't be found via the path given in a diagnostic, I'd like to be able to override the prompt that emacs gives to locate the file.  compilation-parse-errors-filename-function happens too early, while the buffer is being initially parsed.  A similar compilation-locate-file-function would be great.  If there's interest I could try to develop a patch.

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Eli Zaretskii]

> From: Dave Abrahams <dave@boostpro.com>
> Date: Tue, 20 Jun 2023 15:30:58 -0700
> 
> [I hope this is the right venue for feature requests.  If not, sorry!]

This is the right place.

> When a file can't be found via the path given in a diagnostic, I'd like to be able to override the prompt that emacs gives to locate the file.  compilation-parse-errors-filename-function happens too early, while the buffer is being initially parsed.  A similar compilation-locate-file-function would be great.  If there's interest I could try to develop a patch.

Before discussing patches, could you please describe your situation?
Specifically, how does it happen that neither the compilation
directory plus the file name shown by the diagnostics nor the search
of compilation-search-path succeed to locate the file?  And what would
the hook function you suggest adding do to resolve this, i.e. how
would it know where to look for the file?

Thanks.

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Dave Abrahams]

[-- Attachment #1: Type: text/plain, Size: 1314 bytes --]



> On Jun 21, 2023, at 4:24 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Dave Abrahams <dave@boostpro.com>
>> Date: Tue, 20 Jun 2023 15:30:58 -0700
>> 
>> [I hope this is the right venue for feature requests.  If not, sorry!]
> 
> This is the right place.
> 
>> When a file can't be found via the path given in a diagnostic, I'd like to be able to override the prompt that emacs gives to locate the file.  compilation-parse-errors-filename-function happens too early, while the buffer is being initially parsed.  A similar compilation-locate-file-function would be great.  If there's interest I could try to develop a patch.
> 
> Before discussing patches, could you please describe your situation?
> Specifically, how does it happen that neither the compilation
> directory plus the file name shown by the diagnostics nor the search
> of compilation-search-path succeed to locate the file?  And what would
> the hook function you suggest adding do to resolve this, i.e. how
> would it know where to look for the file?

When Swift asserts, it doesn't write the path to the file into the output, because… reasons I grudgingly accept.  So one needs to guess at the file.  https://github.com/apple/swift/issues/66777 contains all the details, but if you want additional info, please ask.

[-- Attachment #2: Type: text/html, Size: 1819 bytes --]

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> When a file can't be found via the path given in a diagnostic, I'd
>> like to be able to override the prompt that emacs gives to locate
>> the file.  compilation-parse-errors-filename-function happens too
>> early, while the buffer is being initially parsed.  A similar
>> compilation-locate-file-function would be great.  If there's
>> interest I could try to develop a patch.
>
> Before discussing patches, could you please describe your situation?
> Specifically, how does it happen that neither the compilation
> directory plus the file name shown by the diagnostics nor the search
> of compilation-search-path succeed to locate the file?  And what would
> the hook function you suggest adding do to resolve this, i.e. how
> would it know where to look for the file?

I'm not the original requester but for cross compilation this could be
useful. E.g I compile Sailfish OS RPM packages with mb2 which runs
rpmbuild in scratchbox2.

The patches in the output come from the command line output that is the
same as if it was ran outside of the cross compilation target however it
should resolve through the target rather than the host unless it is
sourced from the very same package that I compile right now.

E.g. /usr/include/stdio.h should be resolved to ->
/srv/mer/sailfishos-devel-aarch64/usr/include.

When building with open build service the situation is very similar.

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Eli Zaretskii]

> From: Björn Bidar <bjorn.bidar@thaodan.de>
> Cc: Dave Abrahams <dave@boostpro.com>,  64198@debbugs.gnu.org
> Date: Thu, 22 Jun 2023 00:52:36 +0300
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> When a file can't be found via the path given in a diagnostic, I'd
> >> like to be able to override the prompt that emacs gives to locate
> >> the file.  compilation-parse-errors-filename-function happens too
> >> early, while the buffer is being initially parsed.  A similar
> >> compilation-locate-file-function would be great.  If there's
> >> interest I could try to develop a patch.
> >
> > Before discussing patches, could you please describe your situation?
> > Specifically, how does it happen that neither the compilation
> > directory plus the file name shown by the diagnostics nor the search
> > of compilation-search-path succeed to locate the file?  And what would
> > the hook function you suggest adding do to resolve this, i.e. how
> > would it know where to look for the file?
> 
> I'm not the original requester but for cross compilation this could be
> useful. E.g I compile Sailfish OS RPM packages with mb2 which runs
> rpmbuild in scratchbox2.
> 
> The patches in the output come from the command line output that is the
> same as if it was ran outside of the cross compilation target however it
> should resolve through the target rather than the host unless it is
> sourced from the very same package that I compile right now.
> 
> E.g. /usr/include/stdio.h should be resolved to ->
> /srv/mer/sailfishos-devel-aarch64/usr/include.

And you cannot use compilation-search-path for achieving that result?

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Eli Zaretskii]

> From: Dave Abrahams <dave@boostpro.com>
> Date: Wed, 21 Jun 2023 09:09:08 -0700
> Cc: 64198@debbugs.gnu.org
> 
>  Before discussing patches, could you please describe your situation?
>  Specifically, how does it happen that neither the compilation
>  directory plus the file name shown by the diagnostics nor the search
>  of compilation-search-path succeed to locate the file?  And what would
>  the hook function you suggest adding do to resolve this, i.e. how
>  would it know where to look for the file?
> 
> When Swift asserts, it doesn't write the path to the file into the output, because… reasons I grudgingly
> accept.  So one needs to guess at the file.  https://github.com/apple/swift/issues/66777 contains all the
> details, but if you want additional info, please ask.

I don't know what Swift is, and that issue talks about something
called #fileID, which I again have no idea about.  So yes, please do
elaborate, and TIA.

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Dave Abrahams]

> On Jun 22, 2023, at 10:02 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Dave Abrahams <dave@boostpro.com>
>> Date: Wed, 21 Jun 2023 09:09:08 -0700
>> Cc: 64198@debbugs.gnu.org
>> 
>> Before discussing patches, could you please describe your situation?
>> Specifically, how does it happen that neither the compilation
>> directory plus the file name shown by the diagnostics nor the search
>> of compilation-search-path succeed to locate the file?  And what would
>> the hook function you suggest adding do to resolve this, i.e. how
>> would it know where to look for the file?
>> 
>> When Swift asserts, it doesn't write the path to the file into the output, because… reasons I grudgingly
>> accept.  So one needs to guess at the file.  https://github.com/apple/swift/issues/66777 contains all the
>> details, but if you want additional info, please ask.
> 
> I don't know what Swift is, and that issue talks about something
> called #fileID, which I again have no idea about.  So yes, please do
> elaborate, and TIA.

Swift is a programming language.  When its assertions (like C's assert() macro) fail, a diagnostic is output with a file ID and line. The file ID looks like a path, but it's really not. If it's a path in the filesystem, that's coincidental but it's likely to point to the right file.  I don't want to manually hunt down the file every time an assertion fails; I'd rather emacs made an educated guess based on the current project.  It's obviously not the right answer for everyone, but I'd like to have the hooks needed to implement that without too much hackery.

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Eli Zaretskii]

> From: Dave Abrahams <dave@boostpro.com>
> Date: Thu, 22 Jun 2023 10:11:29 -0700
> Cc: 64198@debbugs.gnu.org
> 
> > On Jun 22, 2023, at 10:02 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> From: Dave Abrahams <dave@boostpro.com>
> >> Date: Wed, 21 Jun 2023 09:09:08 -0700
> >> Cc: 64198@debbugs.gnu.org
> >> 
> >> Before discussing patches, could you please describe your situation?
> >> Specifically, how does it happen that neither the compilation
> >> directory plus the file name shown by the diagnostics nor the search
> >> of compilation-search-path succeed to locate the file?  And what would
> >> the hook function you suggest adding do to resolve this, i.e. how
> >> would it know where to look for the file?
> >> 
> >> When Swift asserts, it doesn't write the path to the file into the output, because… reasons I grudgingly
> >> accept.  So one needs to guess at the file.  https://github.com/apple/swift/issues/66777 contains all the
> >> details, but if you want additional info, please ask.
> > 
> > I don't know what Swift is, and that issue talks about something
> > called #fileID, which I again have no idea about.  So yes, please do
> > elaborate, and TIA.
> 
> Swift is a programming language.  When its assertions (like C's assert() macro) fail, a diagnostic is output with a file ID and line. The file ID looks like a path, but it's really not. If it's a path in the filesystem, that's coincidental but it's likely to point to the right file.  I don't want to manually hunt down the file every time an assertion fails; I'd rather emacs made an educated guess based on the current project.  It's obviously not the right answer for everyone, but I'd like to have the hooks needed to implement that without too much hackery.

I'm okay with adding a hook function which will be called from
compilation-find-file if compilation-find-file-1 fails to find the
file.  But I must say that you are trying to use compilation-mode in a
situation for which it was never been designed to be used.

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Daniel Martín via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dave Abrahams <dave@boostpro.com> writes:
>
> Swift is a programming language.  When its assertions (like C's
> assert() macro) fail, a diagnostic is output with a file ID and
> line. The file ID looks like a path, but it's really not. If it's a
> path in the filesystem, that's coincidental but it's likely to point
> to the right file.  I don't want to manually hunt down the file every
> time an assertion fails; I'd rather emacs made an educated guess based
> on the current project.  It's obviously not the right answer for
> everyone, but I'd like to have the hooks needed to implement that
> without too much hackery.

This is the Elisp code you posted on GitHub, based on customizing
compilation-parse-errors-filename-function:

(defun dwa/path-tails-matching (path paths)
  "Returns the elements of PATHS with PATH as their suffix, matching by path component."
  (let ((tail (file-name-concat "/" path)))
    (seq-filter (lambda (full-path) (string-suffix-p tail full-path)) paths)))

(defun dwa/compilation-parse-errors-filename-in-project (path-in-error)
  "If PATH-IN-ERROR is found in the filesystem, returns it
unchanged; otherwise tries to return a unique file in the current
project likely to be identified by PATH-IN-ERROR.  Returns
PATH-IN-ERROR unchanged if no such file can be found."
  (if (file-exists-p path-in-error) path-in-error
    ;; First look for relative path suffixes.
    (let* ((candidates (project-files (project-current t)))
	   (full-matches (dwa/path-tails-matching path-in-error candidates)))

      ;; If not found, try just the filename component of
      ;; path-in-error; Swift assertions are weird and include a
      ;; module name that doesn't necessarily correspond to anything in the
      ;; filesystem.
      (cond ((null full-matches)
	     (let ((filename-matches
		   (dwa/path-tails-matching (file-name-nondirectory path-in-error) candidates)))
	       (cond ((null filename-matches) path-in-error)
		     ;; unique match; return it
		     ((null (cadr filename-matches)) (car filename-matches))
		     ;; multiple matches; consider prompting for the user to select one.
		     (t path-in-error))))
	    ;; unique match; return it
	    ((null (cadr matches)) (car matches))
	     ; multiple matches; consider prompting for the user to select one.
	    (t path-in-error)))))

(setq compilation-parse-errors-filename-function 'dwa/compilation-parse-errors-filename-in-project)

I've tried it and it seems to work well, although I agree that the code
is a bit hacky and prone to error.  How would the new hooks you propose
simplify this customization code?

Similar use cases have been discussed in the mailing list in the past.
Someone proposed to make compilation-find-file customizable, which may
not be a bad idea.  See
https://lists.gnu.org/archive/html/emacs-devel/2021-04/msg00910.html

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Dave Abrahams]

[-- Attachment #1: Type: text/plain, Size: 3841 bytes --]



> On Jun 22, 2023, at 10:58 AM, Daniel Martín <mardani29@yahoo.es> wrote:
> 
> Dave Abrahams <dave@boostpro.com <mailto:dave@boostpro.com>> writes:
>> 
>> Swift is a programming language.  When its assertions (like C's
>> assert() macro) fail, a diagnostic is output with a file ID and
>> line. The file ID looks like a path, but it's really not. If it's a
>> path in the filesystem, that's coincidental but it's likely to point
>> to the right file.  I don't want to manually hunt down the file every
>> time an assertion fails; I'd rather emacs made an educated guess based
>> on the current project.  It's obviously not the right answer for
>> everyone, but I'd like to have the hooks needed to implement that
>> without too much hackery.
> 
> This is the Elisp code you posted on GitHub, based on customizing
> compilation-parse-errors-filename-function:
> 
> (defun dwa/path-tails-matching (path paths)
>  "Returns the elements of PATHS with PATH as their suffix, matching by path component."
>  (let ((tail (file-name-concat "/" path)))
>    (seq-filter (lambda (full-path) (string-suffix-p tail full-path)) paths)))
> 
> (defun dwa/compilation-parse-errors-filename-in-project (path-in-error)
>  "If PATH-IN-ERROR is found in the filesystem, returns it
> unchanged; otherwise tries to return a unique file in the current
> project likely to be identified by PATH-IN-ERROR.  Returns
> PATH-IN-ERROR unchanged if no such file can be found."
>  (if (file-exists-p path-in-error) path-in-error
>    ;; First look for relative path suffixes.
>    (let* ((candidates (project-files (project-current t)))
> 	   (full-matches (dwa/path-tails-matching path-in-error candidates)))
> 
>      ;; If not found, try just the filename component of
>      ;; path-in-error; Swift assertions are weird and include a
>      ;; module name that doesn't necessarily correspond to anything in the
>      ;; filesystem.
>      (cond ((null full-matches)
> 	     (let ((filename-matches
> 		   (dwa/path-tails-matching (file-name-nondirectory path-in-error) candidates)))
> 	       (cond ((null filename-matches) path-in-error)
> 		     ;; unique match; return it
> 		     ((null (cadr filename-matches)) (car filename-matches))
> 		     ;; multiple matches; consider prompting for the user to select one.
> 		     (t path-in-error))))
> 	    ;; unique match; return it
> 	    ((null (cadr matches)) (car matches))
> 	     ; multiple matches; consider prompting for the user to select one.
> 	    (t path-in-error)))))
> 
> (setq compilation-parse-errors-filename-function 'dwa/compilation-parse-errors-filename-in-project)
> 
> I've tried it and it seems to work well, although I agree that the code
> is a bit hacky and prone to error.  How would the new hooks you propose
> simplify this customization code?

It would not.  The problem with this code is that it runs too early.  That means two things:

1. if you get 1000 errors in different files, this code will try to resolve each one, accessing the filesystem during parsing, before the compilation buffer even font locks, which is really inefficient especially if you only care about the first of these errors.
2. There's no good way to prompt the user for a match when there are multiple matches.
 
I want a hook that will be run at the time compile.el currently prompts me to locate the file in the filesystem: the first time I visit an error at a given path and no file is found there by emacs.

> Similar use cases have been discussed in the mailing list in the past.
> Someone proposed to make compilation-find-file customizable, which may
> not be a bad idea.  See
> https://lists.gnu.org/archive/html/emacs-devel/2021-04/msg00910.html

Yes, I think I could probably get where I need to by advising compilation-find-file.

[-- Attachment #2: Type: text/html, Size: 42366 bytes --]

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Dave Abrahams]

> On Jun 22, 2023, at 10:23 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> I must say that you are trying to use compilation-mode in a
> situation for which it was never been designed to be used.

Of course I am, but I don't know why you're saying so.  Compilation-mode was obviously designed to be a general facility, thus the great flexiblity in the regexp alists and compilation-search-path, etc., so the original designer knew their ability to envision the details of future use cases was limited. I'm suggesting a feature that makes it more adaptable to un-envisioned use cases.

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Dave Abrahams]

[-- Attachment #1: Type: text/plain, Size: 262 bytes --]



> On Jun 22, 2023, at 2:49 PM, Dave Abrahams <dave@boostpro.com> wrote:
> 
> Yes, I think I could probably get where I need to by advising compilation-find-file.

I take it back; no I couldn't. I need too much of the logic it implements before prompting.

[-- Attachment #2: Type: text/html, Size: 964 bytes --]

bug#64198: Feature request: compile.el's "Find this in" prompt should be hookable.

[Eli Zaretskii]

> From: Dave Abrahams <dave@boostpro.com>
> Date: Thu, 22 Jun 2023 14:56:04 -0700
> Cc: 64198@debbugs.gnu.org
> 
> > On Jun 22, 2023, at 10:23 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > I must say that you are trying to use compilation-mode in a
> > situation for which it was never been designed to be used.
> 
> Of course I am, but I don't know why you're saying so.  Compilation-mode was obviously designed to be a general facility

No, it was designed to work with programs that emit notifications in
certain supported formats, which are assumed to include the file name
and the line/column numbers of the locus of interest.  See the file
etc/compilation.txt in the Emacs source tree.

Anyway, I already said I'm okay with a hook there, so you are
responding to a marginal comment.  Would you like to submit a patch
that adds such a hook in compilation-find-file?  Then we could declare
victory, even if we disagree about that minor issue.

Thanks.