Proposal for search-input-file

Proposal for search-input-file

[(]

Hello Guix,

While the nicest way to search for files or directories in Guix build
phases is SEARCH-INPUT-FILE or SEARCH-INPUT-DIRECTORY:

  (search-input-file inputs "bin/foobarbaz")
  (search-input-directory inputs "share/foobarbaz-state")

there are many situations where you need to either wrap it up in an ugly
way or just use #$(this-package-input) instead:

  (dirname (search-input-file inputs "lib/libfoobarbaz.so"))
  (string-append #$(this-package-input "foobarbaz") "lib")

To make this common case less annoying, while further improving other
aspects of the usability of SEARCH-INPUT-* in general, we could add
keyword arguments to the SEARCH-INPUT procedures, and possibly merge
them into one:

  ;; old
  (search-input-file INPUTS-ALIST PATH)
  (search-input-directory INPUTS-ALIST PATH)

  ;; new
  (find-in-inputs INPUTS-ALIST PATH-OR-PATHS [PREDICATE?]
                  [#:contents CHILD-PATH-OR-PATHS]
                  [#:regexp? REGEXP?]
                  [#:collect? COLLECT?])

  ;; auxillary (for use with PREDICATE?)
  (regular-file? PATH/PORT/FD)
  (directory? PATH/PORT/FD)
  (symlink? PATH/PORT/FD)

We could simply add the new functionality to SEARCH-INPUT-* if a new
procedure is considered undesirable, though that would make the
PREDICATE? key somewhat less useful.

Of course, not all of the proposals are crucial, so it doesn't matter if
we get rid of, say, REGEXP? if it's decided that there wouldn't be that
many use cases.  Really, the crux of the proposal is the #:CONTENTS
argument; the other bits are just extras I thought might be nice to have
since we're already adding keyword arguments.

Anyway, here's the full proposal:

INPUTS-ALIST would work the same way as before; the difference between
the second parameters of the two versions is that FIND-IN-INPUTS could
accept a list of paths to find, in which case the first file *or*
directory matching *any* of the paths in the list would be returned.

If REGEXP? is #T, then the given string or list of strings would be
treated as regexps, returning the first path matching any of them.

If COLLECT? is #T, however, the procedure will return *all* the paths
files across *every* input that matches the path/regexp/list of paths/
list of regexps.

PREDICATE? is an optional argument that specifies a predicate that
accepts a possible path.  If PREDICATE? returns #F, the path will not
be returned, even if it matches PATH-OR-PATHS.  The auxillary procedures
proposed above (DIRECTORY?, SYMLINK?, etc) would be what you'd typically
put in this field.

CHILD-PATH-OR-PATHS may be a list much like PATH-OR-PATHS which
specifies files or directories that the PATH-OR-PATHS *must* contain if
they are to be returned.  This should use regular expressions if REGEXP?
is enabled.  Obviously, #:CONTENTS implies that the PATH-OR-PATHS must
be directories, so DIRECTORY? is not necessary if using it.

For example, to get a list of all the 'lib/' and 'share/' directories
that contain 'pkgconfig/' subdirectories, you'd write this:

  (find-in-inputs inputs '("lib" "share")
                  #:contents "pkgconfig"
                  #:collect? #t)

or to get all the library files in INPUTS:

  (find-in-inputs inputs "lib/lib.+\\.(so|a)(\\..*)?" regular-file?
                  #:regexp? #t)

or to get the 'bin/' directory of 'guix' (the LIST is used for
demonstration purposes; I'd imagine you'd just specify "guix" in
#:CONTENTS if you were actually doing this):

  (find-in-inputs inputs "bin" #:contents (list "guix" "guix-daemon"))

This last example is basically the main thing I'm proposing; as I said,
the rest is all fluff, though possibly useful fluff.  And I suppose our
existing FIND-FILES procedure could be amended to support the new
FIND-IN-INPUTS options, too.

  -- (

Re: Proposal for search-input-file

[Liliana Marie Prikler]

Hi (,

Am Donnerstag, dem 10.08.2023 um 14:28 +0100 schrieb (:
> 
> Hello Guix,
> 
> While the nicest way to search for files or directories in Guix build
> phases is SEARCH-INPUT-FILE or SEARCH-INPUT-DIRECTORY:
> 
>   (search-input-file inputs "bin/foobarbaz")
>   (search-input-directory inputs "share/foobarbaz-state")
> 
> there are many situations where you need to either wrap it up in an
> ugly way or just use #$(this-package-input) instead:
> 
>   (dirname (search-input-file inputs "lib/libfoobarbaz.so"))
>   (string-append #$(this-package-input "foobarbaz") "lib")
> 
> To make this common case less annoying, while further improving other
> aspects of the usability of SEARCH-INPUT-* in general, we could add
> keyword arguments to the SEARCH-INPUT procedures, and possibly merge
> them into one:
> 
>   ;; old
>   (search-input-file INPUTS-ALIST PATH)
>   (search-input-directory INPUTS-ALIST PATH)
> 
>   ;; new
>   (find-in-inputs INPUTS-ALIST PATH-OR-PATHS [PREDICATE?]
>                   [#:contents CHILD-PATH-OR-PATHS]
>                   [#:regexp? REGEXP?]
>                   [#:collect? COLLECT?])
> 
>   ;; auxillary (for use with PREDICATE?)
>   (regular-file? PATH/PORT/FD)
>   (directory? PATH/PORT/FD)
>   (symlink? PATH/PORT/FD)
> 
> We could simply add the new functionality to SEARCH-INPUT-* if a new
> procedure is considered undesirable, though that would make the
> PREDICATE? key somewhat less useful.
I think adding new procedures is fine, but you have to keep them
readable.  Your procedure is trying to do too many things at once and
thus losing the simplicity that search-input-file/search-input-
directory has.

Part of what you're trying to achieve is already implemented via search
paths, so I think we can focus on the case where we want to find a
particular file, but strip parts of its name.

I think the most general definition that's still useful is this

(define* (search-input-file* inputs predicate
                             #:key (hint "<unknown>"))
  "Find the first file in INPUTS matching PREDICATE.  The return value
of PREDICATE is returned as-is, so patterns like 
  (search-input-file* inputs
    (lambda (parent)
      (and (file-exists? (string-append parent \"/lib/foobar.so\"))
           (string-append parent \"/lib\"))))
work as expected.

When PREDICATE returns no match, raise a search error using HINT for a
file name."
  (match inputs
    (((_ . directories) ...)
     (or (any predicate directories)
         (raise (condition
                 (&search-error (path directories) 
                                (file hint))))))))

Of course you can then go on to define special cases, e.g. matching
regular expressions or the like.

Cheers

Re: Proposal for search-input-file

[(]

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
> I think the most general definition that's still useful is this
>
> (define* (search-input-file* inputs predicate
>                              #:key (hint "<unknown>"))
>   "Find the first file in INPUTS matching PREDICATE.  The return value
> of PREDICATE is returned as-is, so patterns like 
>   (search-input-file* inputs
>     (lambda (parent)
>       (and (file-exists? (string-append parent \"/lib/foobar.so\"))
>            (string-append parent \"/lib\"))))
> work as expected.
>
> When PREDICATE returns no match, raise a search error using HINT for a
> file name."
>   (match inputs
>     (((_ . directories) ...)
>      (or (any predicate directories)
>          (raise (condition
>                  (&search-error (path directories) 
>                                 (file hint))))))))

You're right; this is better.

 -- (

Re: Proposal for search-input-file

[Ludovic Courtès]

Hi!

Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:

> (define* (search-input-file* inputs predicate
>                              #:key (hint "<unknown>"))
>   "Find the first file in INPUTS matching PREDICATE.  The return value
> of PREDICATE is returned as-is, so patterns like 
>   (search-input-file* inputs
>     (lambda (parent)
>       (and (file-exists? (string-append parent \"/lib/foobar.so\"))
>            (string-append parent \"/lib\"))))
> work as expected.
>
> When PREDICATE returns no match, raise a search error using HINT for a
> file name."

This would be equivalent to:

  (dirname (search-input-file inputs "/lib/libfoo.so"))

right?

Another option would be a procedure that does precisely that: return the
parent.

Thoughts?

Ludo’.

Re: Proposal for search-input-file

[Liliana Marie Prikler]

Am Mittwoch, dem 16.08.2023 um 16:49 +0200 schrieb Ludovic Courtès:
> Hi!
> 
> Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:
> 
> > (define* (search-input-file* inputs predicate
> >                              #:key (hint "<unknown>"))
> >   "Find the first file in INPUTS matching PREDICATE.  The return
> > value
> > of PREDICATE is returned as-is, so patterns like 
> >   (search-input-file* inputs
> >     (lambda (parent)
> >       (and (file-exists? (string-append parent \"/lib/foobar.so\"))
> >            (string-append parent \"/lib\"))))
> > work as expected.
> > 
> > When PREDICATE returns no match, raise a search error using HINT
> > for a
> > file name."
> 
> This would be equivalent to:
> 
>   (dirname (search-input-file inputs "/lib/libfoo.so"))
> 
> right?
> 
> Another option would be a procedure that does precisely that: return
> the parent.
> 
> Thoughts?
In the generic case, your needle could be very long, e.g.
/lib/python/site-packages/…/something.so → /lib.
Granted, it isn't very likely to occur, but build system writers don't
always make the smartest decisions.

Cheers