[ANN] guile-file-names 0.2

[ANN] guile-file-names 0.2

[Brandon Invergo]

Hello,

I'm happy to announce the release of guile-file-names 0.2.

The (file-names) module provides methods for manipulating file names.
Its design distinguishes between the human-friendly string format of
filenames ("/usr/bin/guile") and a more Scheme-friendly representation
to take out all the little nuisances of working with file names.

This release sees bugs fixed, under-the-hood improvements, and new
features added.  See below for an excerpt from the NEWS file.

Download:
http://brandon.invergo.net/software/download/guile-file-names/guile-file-names-0.2.tar.gz
http://brandon.invergo.net/software/download/guile-file-names/guile-file-names-0.2.tar.gz.sig

guile-file-names-0.2.tar.gz sha256sum:
28d63bec3e484ae5c98431b77f0a1bcc50e2f351cf432b383009106a152c9dcf

If you use the library, please consider reporting any bugs you encounter
or feature requests you might have on the issue tracker on Gitlab:

https://gitlab.com/brandoninvergo/guile-file-names

Direct email to me is also fine, if you don't have a Gitlab account.

NEWS:

** Features

*** Native file-globbing sub-module
A new sub-module (file-names glob) has been added.  This module
provides the <file-name> method glob, which is used to perform
"file-globbing."   File-globbing is the method of finding multiple
files, e.g. at the shell, by matching wildcards
(e.g. "/usr/lib/*.so").

glob supports shell-style wildcards (*, ?, [] and **) as
well as full regular expressions.

*** string->file-name now takes optional keyword arguments
This allows you to override system- or parsed- defaults for the
volume, separator and case-sensitivity.

*** New <file-name> methods remove-prefix and remove-prefix!
These new methods convert an absolute file name to a relative one by
removing a leading directory prefix.

*** Specialized write and display methods for <file-name> objects

Running write or display on a <file-name> object now prints a
recognizable string representation of the file name rather than the
default cryptic GOOPS-object format:

#+BEGIN_EXAMPLE
scheme@(guile-user)> (use-modules (file-names))
scheme@(guile-user)> (define f (string->file-name "/usr/bin/guile"))
scheme@(guile-user)> (simple-format #t "~a\n" f)
#<<file-name> /usr/bin/guile>
#+END_EXAMPLE

** Bugs

*** Resolving an absolute file name
It is now an explicit error if there are double-dots at the root of
the file name (e.g. "/../foo/bar").  Also, a bug was fixed where
sequential double-dots ("../../../blah.txt") were not handled correctly.

*** file-name=? now behaves correctly for relative file names.

--
-brandon

Re: [ANN] guile-file-names 0.2

[Mark H Weaver]

Hi Brandon,

Brandon Invergo <brandon@invergo.net> writes:

> I'm happy to announce the release of guile-file-names 0.2.
>
> The (file-names) module provides methods for manipulating file names.
> Its design distinguishes between the human-friendly string format of
> filenames ("/usr/bin/guile") and a more Scheme-friendly representation
> to take out all the little nuisances of working with file names.
>
> This release sees bugs fixed, under-the-hood improvements, and new
> features added.  See below for an excerpt from the NEWS file.

I see that you are using 'set!' to mutate several core bindings in the
(guile) module to much slower GOOPS generic functions.  For example, you
'set!' the core 'append' procedure to a GOOPS generic function that adds
support for appending together your <file-name> objects.

If you must override core procedures, then please use #:export and
#:replace in the 'define-module' form, and simply 'define' the new
binding in your module instead of using 'set!'.  That way, the bindings
in (guile) will be left unchanged, and your new bindings will only be
used in modules that import your module.

I can understand your decision to overload Guile's primitives that
accept file names.  However, I disagree with your decision to overload
'append' and 'append!'.  Those are not intended to be generic functions
in Scheme.  If they were, we wouldn't have 'string-append' and
'symbol-append'.

     Regards,
       Mark

Re: [ANN] guile-file-names 0.2

[Mark H Weaver]

Hi again,

I wrote:
> If you must override core procedures, then please use #:export and
> #:replace in the 'define-module' form, and simply 'define' the new
> binding in your module instead of using 'set!'.  That way, the bindings
> in (guile) will be left unchanged, and your new bindings will only be
> used in modules that import your module.

Sorry, I meant to give an example of this.  Please see srfi/srfi-45.scm
in the Guile source distribution for an example of the preferred way to
override core bindings, such that only modules that import your module
are affected.

      Thanks,
        Mark

Re: [ANN] guile-file-names 0.2

[Brandon Invergo]

Hi Mark,

Mark H Weaver writes:

> Sorry, I meant to give an example of this.  Please see srfi/srfi-45.scm
> in the Guile source distribution for an example of the preferred way to
> override core bindings, such that only modules that import your module
> are affected.

Thanks for the example.  I will make that change for the next release.
Actually, it's good that you mentioned it now because I want to go back
and make file-name-specialized variants of the rest of the core
procedures that normally take file-name strings (e.g. file ports).

I'll admit this wasn't a carefully thought-out strategy.  I was going by
the example in the GOOPS "Generic Function and Method Examples" section
of the manual (where + is specialized to handle <my-complex> objects).
I was having trouble figuring out how to do this such that the code
could be distributed as a module, and that was the only section that I
found that gave any clue.  I missed, however, the "#:replace"
functionality of define-module.

I would suggest that this section of the manual be updated to
demonstrate the preferred way to do it.

I also have no problem with replacing my specialized append and append!
methods with file-name-append and file-name-append! (in fact, that's
what they were originally, before I de-namespaced the method symbols).
It's still early days, so few people, if anybody, will be affected by
such a change.

Thanks for the tip!

--
-brandon

Re: exporting GOOPS generic functions, was: [ANN] guile-file-names 0.2

[Brandon Invergo]

Hi Mark or anyone else able to help,

Thanks again for raising this issue with my guile-file-names module:

Mark H Weaver writes:

> I see that you are using 'set!' to mutate several core bindings in the
> (guile) module to much slower GOOPS generic functions.  For example, you
> 'set!' the core 'append' procedure to a GOOPS generic function that adds
> support for appending together your <file-name> objects.

I've renamed the <file-name> append methods to something else that fits
well with the names of other methods.

However, I would still like to provide versions of all of the various
file-system procedures specialized on <file-name> objects.

Herein lies the rub: naturally, none of the examples that I see in the
SRFIsn use GOOPS.  I can't figure out how to export from my module
methods that are derived from GOOPS generic functions of built-in
procedures.

Take, for example, providing absolute-file-name? for <file-name> objects
without clobbering the ability to call it on strings.

From what I understand, this should simply entail calling define-method
and adding the symbol to the define-module #:replace list.
define-method will first call define-generic an absolute-file-name? and
it will use the original procedure as the default of the generic
function.

So, as simple as:

    (define-module (file-names)
      ...
      #:replace (absolute-file-name?))

    (define-method (absolute-file-name? (f <file-name>)) ...)

When using the module, absolute-file-name? now works for <file-name>
objects, however it no longer works for strings:

    scheme@(guile-user)> (absolute-file-name? "/home/brandon/.guile")
    ERROR: In procedure scm-error:
    No applicable method for #<<generic> absolute-file-name? (1)> in call (absolute-file-name? "/home/brandon/.guile")

So, the original procedure isn't being used as the default for the
generic function?  How should I handle this?

Obviously I can't just do

    (define-method (absolute-file-name? (f <file-name>)) ...)
    (define-method (absolute-file-name? (f <string>))
      (absolute-file-name? f))

Because of infinite recursion.

I can try something like this:

    (let ((old-absolute-file-name? absolute-file-name?))
      (define-generic absolute-file-name?)
      (define-method (absolute-file-name? (f <file-name>))
        (proper-list? (route f)))
      (define-method (absolute-file-name? (f <string>))
        (old-absolute-file-name? f)))

But that strangely gives me this upon compiling the module:

    While compiling expression:
    Unbound variable: absolute-file-name?

I'm not sure what to make of that.  A compile-time error, but why?

That last attempt, of course, looks a lot like what's recommended in the
GOOPS manual ("8.6.5 Generic Function and Method Examples"), which would
be like this:

    (define-generic new-absolute-file-name?)
    (let ((old-absolute-file-name? absolute-file-name?))
      (define-method (new-absolute-file-name? (f <file-name>))
        (proper-list? (route f)))
      (define-method (new-absolute-file-name? (f <string>))
        (old-absolute-file-name? f)))
    (set! absolute-file-name? new-absolute-file-name?)

That works, but as you rightly point out, that rebinds the symbol
outright and thus doesn't even require it to be exported from the
module.  So, that won't play nicely with others.

I can try to take what you wrote very literally:

> If you must override core procedures, then please use #:export and
> #:replace in the 'define-module' form, and simply 'define' the new
> binding in your module instead of using 'set!'.  That way, the bindings
> in (guile) will be left unchanged, and your new bindings will only be
> used in modules that import your module.

    (define-module (file-names)
      ...
      #:export (absolute-file-name? ...)
      #:replace (absolute-file-name?))

    (define-generic new-absolute-file-name?)
    (let ((old-absolute-file-name? absolute-file-name?))
      (define-method (new-absolute-file-name? (f <file-name>))
        (proper-list? (route f)))
      (define-method (new-absolute-file-name? (f <string>))
        (old-absolute-file-name? f)))
    (define absolute-file-name? new-absolute-file-name?)

...i.e. just changing set! to define.  However, that gives me the same
"Unbound variable" compile-time error as above.

I'm clearly missing something obvious and I have no idea how to proceed.
GOOPS isn't behaving the way I would expect based on the manual:

 -- syntax: define-generic symbol
     Create a generic function with name SYMBOL and bind it to the
     variable SYMBOL.  If SYMBOL was previously bound to a Scheme
     procedure (or procedure-with-setter), the old procedure (and
     setter) is incorporated into the new generic function as its
     default procedure (and setter).  Any other previous value,
     including an existing generic function, is discarded and replaced
     by a new, empty generic function.

The procedure is written in Scheme, not C (in ice-9/boot-9.scm), so I
don't expect to have to treat it specially in any way like primitives
might require.

So, I feel like I'm blindly trying to get things to work without
understanding what's happening under the hood.  That's not the right way
to go about things.

So, how does one provide a module that replaces symbols for built-in
procedures with generic functions, such that these generics provide both
specialized methods and a default function using the original
procedure?

Thanks for your help,

-brandon

ps - You say that I should use #:export and #:replace, but according to
the manual, #:replace "[exports] all identifiers in LIST ... and mark[s]
them as “replacing bindings”."  So, shouldn't be unnecessary to put the
symbol in both lists?

Re: exporting GOOPS generic functions, was: [ANN] guile-file-names 0.2

[Mark H Weaver]

Hi Brandon,

Brandon Invergo <brandon@invergo.net> writes:

> I can try something like this:
>
>     (let ((old-absolute-file-name? absolute-file-name?))
>       (define-generic absolute-file-name?)
>       (define-method (absolute-file-name? (f <file-name>))
>         (proper-list? (route f)))
>       (define-method (absolute-file-name? (f <string>))
>         (old-absolute-file-name? f)))
>
> But that strangely gives me this upon compiling the module:
>
>     While compiling expression:
>     Unbound variable: absolute-file-name?
>
> I'm not sure what to make of that.  A compile-time error, but why?

It's because you tried to export a binding that doesn't exist at the
top-level of your module.  That's because the definitions you gave above
are not in a top-level context.  They are local variables.

One easy solution is to use Guile's (@ module var) syntax to explicitly
reference a variable from a particular module.  For example:

    (define-method (absolute-file-name? (f <string>))
      ((@ (guile) absolute-file-name?) f)))

Here, the reference (@ (guile) absolute-file-name?) explicitly asks for
the binding from the (guile) module.

Another option is to *rename* bindings while importing them, using
either #:prefix or #:select.  For example, see
module/system/repl/command.scm in Guile's source, which includes the
following lines in its 'define-module' form:

  #:use-module ((ice-9 pretty-print) #:select ((pretty-print . pp)))
  #:use-module ((system vm inspect) #:select ((inspect . %inspect)))

Those modules include 'pretty-print' and 'inspect' among their exported
bindings, but the bindings will be visible as 'pp' and '%inspect' within
command.scm.

When using #:select, the imports are limited to the bindings you
specifically asked for.  Sometimes it's more convenient to add a prefix
to every binding imported from a given module, like the following
example from Guile's module/ice-9/streams.scm:

(define-module (ice-9 streams)
  #:use-module ((srfi srfi-41) #:prefix srfi-41:)

Bindings can also be renamed when they are exported, and (ice-9 streams)
does this.

> ps - You say that I should use #:export and #:replace, but according to
> the manual, #:replace "[exports] all identifiers in LIST ... and mark[s]
> them as “replacing bindings”."  So, shouldn't be unnecessary to put the
> symbol in both lists?

Yes, you're right.  If a binding is listed in the #:replace list,
there's no need to add it to the #:export list.

* * *

Finally, I wanted to mention that, in my opinion, the degree to which
this library modifies existing Guile bindings is regrettable.  I'm glad
that Guile is hackable in this way, but I generally prefer to avoid
libraries that do these kinds of hacks, unless there is a compelling
reason to justify it.

In my opinion, it would be better to use plain strings as the
representation for file names, and instead provide procedures to
conveniently manipulate those strings.

Consider:

(1) Your <file-name> objects are much larger than plain strings in terms
of their memory use.

(2) Every time you pass a <file-name> object to a file system primitive,
it needs to be converted back into a string, which makes them slower to
use as well.

(3) If Guile ever extends our file system primitives in the future,
e.g. by adding additional keyword or optional arguments, your library
will likely cause problems with that.  For example, a few years ago we
added the #:guess-encoding and #:encoding keyword arguments to
'open-file' and several other similar procedures.

Also, I had a question: a big part of your library seems to be aimed at
supporting Windows drive letters and separators.  Have tested your
library on Guile running on Windows?

One thing I know is that there's no need for us to use backslash as the
file separator on Windows.  My understanding is that the POSIX
compatibility layer, used in Guile's Windows port, automatically handles
interpreting forward slashes as a directory separator.  Guile already
includes code to try to ensure that from the perspective of Guile code,
we will see and use forward slashes uniformly, regardless of platform.
For example, see 'scm_i_mirror_backslashes' in libguile/load.c and its
associated comment.

      Regards,
        Mark

Re: exporting GOOPS generic functions, was: [ANN] guile-file-names 0.2

[Mark H Weaver]

Hello again,

> Brandon Invergo <brandon@invergo.net> writes:
>
>> I can try something like this:
>>
>>     (let ((old-absolute-file-name? absolute-file-name?))
>>       (define-generic absolute-file-name?)
>>       (define-method (absolute-file-name? (f <file-name>))
>>         (proper-list? (route f)))
>>       (define-method (absolute-file-name? (f <string>))
>>         (old-absolute-file-name? f)))
>>
>> But that strangely gives me this upon compiling the module:
>>
>>     While compiling expression:
>>     Unbound variable: absolute-file-name?
>>
>> I'm not sure what to make of that.  A compile-time error, but why?
>
> It's because you tried to export a binding that doesn't exist at the
> top-level of your module.

I should explain more clearly what happened here.  The short answer is:
When you export a variable, it immediately shadows any imported bindings
with the same name.

The longer answer is that before a variable can be exported, we first
need a variable object to add to the public interface.  If you export a
variable before it has been defined (the usual case), Guile allocates a
fresh variable object and immediately adds it to the local module table,
although it marks the variable as "unbound", which essentially means
that the variable pretends not to exist.  However, the existence of this
"unbound" variable *does* have the effect of hiding any imports with the
same name.

       Mark