Re: Optimizing union.scm

[ludo@gnu.org (Ludovic Courtès)]

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Mark H Weaver <mhw@netris.org> skribis:

[...]

>>> * I've not yet updated tests/union.scm, which tests internal procedures
>>>   in union.scm that no longer exist.
>>
>> Right, the ‘tree-union’ and ‘delete-duplicate-leaves’ tests will have to
>> be removed.
>>
>> However, could you add similar tests?  They will have to use the real
>> ‘union-build’, though.  Maybe the ‘with-file-tree’ can be useful to
>> write the tests.
>
> Could you help with this?  I'm a bit overloaded right now.

OK, I’ll add a few tests.

>> Some stylistic comments:
>>
>>> (define (names-in-directory dirname)
>>>   (let ((dir (opendir dirname)))
>>>     (let loop ((filenames '()))
>>>       (match (readdir dir)
>>>         ((or "." "..")
>>>          (loop filenames))
>>>         ((? eof-object?)
>>>          (closedir dir)
>>>          filenames)
>>>         (name
>>>          (loop (cons name filenames)))))))
>>
>> Rather use something like:
>>
>>   (scandir directory (lambda (file)
>>                        (not (member file '("." "..")))))
>>
>> ‘scandir’ also has the advantage of being deterministic (it sorts
>> entries.)
>
> I looked at 'scandir', but it's very wasteful.  For one thing, it
> unconditionally calls 'lstat' on every directory entry, which entails
> over ten thousand unnecessary 'lstat' system calls, a lot of unnecessary
> I/O to read the inodes, the use of a vhash to keep track of where it has
> been to detect cycles (since it's based on 'file-system-fold'), etc.

Oh right, makes sense.

Then just s/filenames/files/ and (sort files string<?) at the end.

>>>     (format log-port "`~a' ~~> `~a'~%" input output)
>>>     (symlink input output))
>>>
>>>   (define (union output inputs)
>>>     (match inputs
>>>       ((input)
>>>        ;; There's only one input, so just make a link.
>>>        (make-link input output))
>>>       (_
>>>        (receive (dirs files) (partition directory? inputs)
>>
>> Rather SRFI-11 let-values.
>
> Hmm.  In simple cases like this, I find 'receive' much more attractive
> than 'let-values', since the latter would involve triple-nested parens,
> which is a bit much even for me.
>
> Can you explain why you prefer 'let-values' to 'receive' in this case?

Because I’m used to it, but here it’s mostly for consistency.

> Also, how do you feel about 'call-with-values'?  In an earlier draft of
> this code, I used 'call-with-values' with 'match-lambda*' as the
> consumer, which eliminated the need for the 'cond'.  Would you prefer
> that?

Yeah I don’t feel too strongly, so whatever you find appropriate.  :-)

Thanks!

Ludo’.