bug#47799: 28.0.50; Default `project-files' implementation doesn't work with quoted filenames

[Philipp <p.stephani2@gmail.com>]

> Am 17.05.2021 um 01:22 schrieb Dmitry Gutov <dgutov@yandex.ru>:
> 
> On 16.05.2021 16:37, Philipp wrote:
> 
>> One thing that came to my mind is: in general, in Elisp (not just XRef), we spend lots of time parsing filenames to support remote and quoted filenames.  Other languages probably solve this by introducing proper types for filenames (e.g. the Java Path class), which can then hold preprocessed information about the underlying filesystem (or special file name handler, in the case of Elisp).  How about doing similar for Elisp?  For example, introduce a `parsed-file-name' class or structure holding the remote/quoting state, or attach it to string properties?  I haven't tried out that idea, but I think it could significantly speed up the parsing (since we'd only have to do it once and don't have to search for filename handlers all the time), as well as remain backward-compatible to "plain" unparsed filenames by allowing both strings and this new object type.  WDYT?
> 
> That sounds like an interesting idea to explore.
> 
> We create/concatenate those file names inside project-files, and then "parse" them again to convert to local names inside xref-matches-in-files. Creating such structures might indeed save us on some parsing and garbage generation.
> 
> Experiments and patches welcome.
> 
> What I was also thinking of previously, is some "fileset" data structure which could contain a list of local file names and their connection in a separate slot. Maybe even separating the parent/root directory into a separate slot when feasible, to minimize GC further, though that might complicate applications.
> 
> A more structured "file" value format might make this stuff easier to use indeed, and perhaps the performance difference will be negligible.

I think those are very good ideas.  The "fileset" structure sounds like a pretty good abstraction.

> 
> The difficulty is having a method like project-files return one format for some users, and another for users who want to take advantage of this performance improvement. Or we break the compatibility and/or introduce a new method with this new behavior.

A general design approach in OOP is to not treat abstract virtual functions (generic functions in ELisp terminology) as part of the public interface of a type; i.e., abstract functions can be implemented, but shouldn't be called outside of the module that defines them (project.el in this case).  That allows for changes like this: implementers could freely return the new fileset structure because only project.el would call project-files.  Not sure how much ELisp code adheres to this principle, though.  If there's too much code (outside of project.el) that relies on project-files returning a list, we need to indeed fall back to some of the other options.