unofficial mirror of guile-devel@gnu.org 
 help / color / mirror / Atom feed
* file names embedded in .go
@ 2010-04-19 14:52 Andy Wingo
  2010-04-19 21:46 ` Thien-Thi Nguyen
                   ` (2 more replies)
  0 siblings, 3 replies; 19+ messages in thread
From: Andy Wingo @ 2010-04-19 14:52 UTC (permalink / raw)
  To: guile-devel

Hey all,

I recently added a global fluid, %file-port-name-canonicalization, which
defaults to #f. But if it's 'absolute, the port name of a file port will
be canonicalized to the absolute path; or, if it's 'relative, the port
name is the canonical name of the file, relative to the %load-path, or
the file name as given otherwise.

The intention was to allow the user to control (port-filename P), so
that the user could find e.g. the absolute path corresponding to that
port at the time that it was made.

The 'relative mode is particularly useful, because it's the
`port-filename' that gets embedded in a .go file. Oftentimes you would
want to be able to map a .go file to a corresponding .scm, and currently
it's difficult. For example if you compile in a +build subdirectory of
Guile via `../configure', then the `port-filename' is e.g.
"../../module/ice-9/boot-9.scm", which doesn't make any sense.

One could instead embed the absolute path, but then you have the problem
that you might remove your build or source directory; it doesn't make
sense to have installed files pointing to build paths. DWARF does this,
but only because with C you don't install the source, but with Guile you
should.

So for me the right thing to do, by default, is to embed a path relative
to the `%load-path', so that one can find the source file via something
like:

    (use-modules (system vm program))
    (define (program-source-file p)
      (search-path %load-path (source:file (program-source p 0))))

I added a keyword argument to compile-file and compile-and-load for this
purpose, to parameterize %file-port-name-canonicalization during the
extent of the given file's compilation. So now files compiled with
guile-tools compile will have their relative path embedded in them, so
that we can map .go files to .scm files, whether installed or
uninstalled.

Thoughts?

Andy
-- 
http://wingolog.org/




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: file names embedded in .go
  2010-04-19 14:52 file names embedded in .go Andy Wingo
@ 2010-04-19 21:46 ` Thien-Thi Nguyen
  2010-04-20  0:08   ` Jose A. Ortega Ruiz
  2010-04-20  9:45   ` Andy Wingo
  2010-04-19 23:12 ` port-filename and path canonicalization Ludovic Courtès
  2010-04-19 23:23 ` file names embedded in .go Ludovic Courtès
  2 siblings, 2 replies; 19+ messages in thread
From: Thien-Thi Nguyen @ 2010-04-19 21:46 UTC (permalink / raw)
  To: guile-devel

() Andy Wingo <wingo@pobox.com>
() Mon, 19 Apr 2010 16:52:09 +0200

   Thoughts?

It sounds complicated.

Does the .go format support the usual ELF-y sections, like
.rodata, .data, etc?  Maybe we could just include the source
verbatim in such a section (and avoid mapping it at runtime).

thi




^ permalink raw reply	[flat|nested] 19+ messages in thread

* port-filename and path canonicalization
  2010-04-19 14:52 file names embedded in .go Andy Wingo
  2010-04-19 21:46 ` Thien-Thi Nguyen
@ 2010-04-19 23:12 ` Ludovic Courtès
  2010-04-20  9:42   ` Andy Wingo
  2010-04-19 23:23 ` file names embedded in .go Ludovic Courtès
  2 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2010-04-19 23:12 UTC (permalink / raw)
  To: guile-devel

Hi!

Andy Wingo <wingo@pobox.com> writes:

> I recently added a global fluid, %file-port-name-canonicalization, which
> defaults to #f. But if it's 'absolute, the port name of a file port will
> be canonicalized to the absolute path; or, if it's 'relative, the port
> name is the canonical name of the file, relative to the %load-path, or
> the file name as given otherwise.
>
> The intention was to allow the user to control (port-filename P), so
> that the user could find e.g. the absolute path corresponding to that
> port at the time that it was made.

My feeling is that ports shouldn’t have to deal with paths because
that’s a separate concern.  The %file-port-name-canonicalization fluid
seems like an inelegant hack to me.

When applications have special requirements about paths, then it should
be up to the application logic to deal with that.

Thanks,
Ludo’.





^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: file names embedded in .go
  2010-04-19 14:52 file names embedded in .go Andy Wingo
  2010-04-19 21:46 ` Thien-Thi Nguyen
  2010-04-19 23:12 ` port-filename and path canonicalization Ludovic Courtès
@ 2010-04-19 23:23 ` Ludovic Courtès
  2 siblings, 0 replies; 19+ messages in thread
From: Ludovic Courtès @ 2010-04-19 23:23 UTC (permalink / raw)
  To: guile-devel

Hey,

Andy Wingo <wingo@pobox.com> writes:

> Oftentimes you would
> want to be able to map a .go file to a corresponding .scm, and currently
> it's difficult. For example if you compile in a +build subdirectory of
> Guile via `../configure', then the `port-filename' is e.g.
> "../../module/ice-9/boot-9.scm", which doesn't make any sense.

Likewise, this caused problems for coverage testing, where relative
paths weren’t correctly interpreted by LCOV, hence this change to embed
absolute paths:
<http://git.sv.gnu.org/cgit/guile.git/commit/?h=wip-coverage&id=873e4d69a08a9ceb2a7a0296130b6a884738324f>.

> One could instead embed the absolute path, but then you have the problem
> that you might remove your build or source directory; it doesn't make
> sense to have installed files pointing to build paths. DWARF does this,
> but only because with C you don't install the source, but with Guile you
> should.

Both good points!

(This obviously invalidates the absolute path change above.)

> So for me the right thing to do, by default, is to embed a path relative
> to the `%load-path', so that one can find the source file via something
> like:
>
>     (use-modules (system vm program))
>     (define (program-source-file p)
>       (search-path %load-path (source:file (program-source p 0))))

OK, though that’s unavoidably ambiguous: ‘%load-path’ could be different
than its compile-time value, etc.

> I added a keyword argument to compile-file and compile-and-load for this
> purpose,

OK.

> to parameterize %file-port-name-canonicalization during the extent of
> the given file's compilation.

As noted in the other thread, I’d rather get rid of this fluid and have
‘compile’ & co. call ‘canonicalize-path’.

Thanks,
Ludo’.





^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: file names embedded in .go
  2010-04-19 21:46 ` Thien-Thi Nguyen
@ 2010-04-20  0:08   ` Jose A. Ortega Ruiz
  2010-04-20 11:35     ` Thien-Thi Nguyen
  2010-04-20  9:45   ` Andy Wingo
  1 sibling, 1 reply; 19+ messages in thread
From: Jose A. Ortega Ruiz @ 2010-04-20  0:08 UTC (permalink / raw)
  To: guile-devel

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> () Andy Wingo <wingo@pobox.com>
> () Mon, 19 Apr 2010 16:52:09 +0200
>
>    Thoughts?
>
> It sounds complicated.
>
> Does the .go format support the usual ELF-y sections, like
> .rodata, .data, etc?  Maybe we could just include the source
> verbatim in such a section (and avoid mapping it at runtime).

I'm not sure if i'm understanding you correctly, but if, by source, you
mean the scheme source, that'd be terrible news for tools such as
Geiser. With that in mind, i like Andy's proposal, and would even add a
way to control how the paths are printed in the debugger backtraces
(where one cannot call scheme functions as the one shown by Andy),
although perhaps that's already implicit in the proposal (and, anyway,
is not strictly necessary: i can implement the
relative->absolute-in-load-path mapping in elisp).

jao
-- 
If at first you don't succeed, try and try and try again. And then give up.
There's no point making a damn fool out of yourself.
 -Dilbert's Rules of Work, slightly paraphrased





^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: port-filename and path canonicalization
  2010-04-19 23:12 ` port-filename and path canonicalization Ludovic Courtès
@ 2010-04-20  9:42   ` Andy Wingo
  2010-04-20 11:15     ` Thien-Thi Nguyen
  2010-04-20 16:57     ` Ludovic Courtès
  0 siblings, 2 replies; 19+ messages in thread
From: Andy Wingo @ 2010-04-20  9:42 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guile-devel

Hi :)

On Tue 20 Apr 2010 01:12, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> writes:
>
>> I recently added a global fluid, %file-port-name-canonicalization, which
>> defaults to #f. But if it's 'absolute, the port name of a file port will
>> be canonicalized to the absolute path; or, if it's 'relative, the port
>> name is the canonical name of the file, relative to the %load-path, or
>> the file name as given otherwise.
>>
>> The intention was to allow the user to control (port-filename P), so
>> that the user could find e.g. the absolute path corresponding to that
>> port at the time that it was made.
>
> My feeling is that ports shouldn’t have to deal with paths because
> that’s a separate concern.  The %file-port-name-canonicalization fluid
> seems like an inelegant hack to me.
>
> When applications have special requirements about paths, then it should
> be up to the application logic to deal with that.

I am inclined to agree. A few complications cloud my view, though.

1. While ports do not have anything to do with file names / paths,
*file* ports certainly do -- because not only do they use the given path
to open the file, they set that path as the port's filename, providing
the only means for reverse-mapping ports to filenames (which is the end
goal here, reverse-mapping objects to filenames).

2. I think a fluid is still necessary, because a file being
compiled can do an `include' or `include-from-path', or even
`open-input-file' in a macro, and all these cases you would want the
same %file-port-name-canonicalization to take effect.

3. The only correct time to do a path canonicalization is when the file
is opened, because at another time, you might not be in the same current
directory, so relative paths would resolve incorrectly.

4. The application-level code is nastier if it has to canonicalize,
because a relative canonicalization cannot in general be passed to
open-input-file. For example

  (open-input-file "../../module/ice-9/boot-9.scm")

is not the same as

  (open-input-file "ice-9/boot-9.scm")

So you'd have to do a set-port-filename! on the port, mucking up your
code -- and how would you decide what to set? In N places you'd have to
duplicate fport_canonicalize_filename, and you'd probably have to make
scm_i_relativize_path public.


When I realized all of that I decided to go with the minimal correct
solution, though it is a bit hacky. Applications are still free to do
their own thing, as %file-port-name-canonicalization defaults to #f, but
the useful 'absolute and 'relative behaviors are more convenient and
robust.

I'd be happy to have some cleaner solution, though. Do you have any
ideas?

Cheers,

Andy
-- 
http://wingolog.org/




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: file names embedded in .go
  2010-04-19 21:46 ` Thien-Thi Nguyen
  2010-04-20  0:08   ` Jose A. Ortega Ruiz
@ 2010-04-20  9:45   ` Andy Wingo
  2010-04-20 10:34     ` Thien-Thi Nguyen
  1 sibling, 1 reply; 19+ messages in thread
From: Andy Wingo @ 2010-04-20  9:45 UTC (permalink / raw)
  To: Thien-Thi Nguyen; +Cc: guile-devel

On Mon 19 Apr 2010 23:46, Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> () Andy Wingo <wingo@pobox.com>
> () Mon, 19 Apr 2010 16:52:09 +0200
>
>    Thoughts?
>
> It sounds complicated.
>
> Does the .go format support the usual ELF-y sections, like
> .rodata, .data, etc?

Nope, though we should just switch to use ELF at some point. Not this
month though :)

>  Maybe we could just include the source
> verbatim in such a section (and avoid mapping it at runtime).

As Jao notes it is useful to be able to map at runtime, and currently we
do this well, mapping to line and column within parts of procedures.

But it would be good to be able to map to expressions too, and having
the original source available without mapping to the filesystem should
be an option. It's tough without having the whole environment, though.
Something to think about :)

Andy
-- 
http://wingolog.org/




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: file names embedded in .go
  2010-04-20  9:45   ` Andy Wingo
@ 2010-04-20 10:34     ` Thien-Thi Nguyen
  0 siblings, 0 replies; 19+ messages in thread
From: Thien-Thi Nguyen @ 2010-04-20 10:34 UTC (permalink / raw)
  To: guile-devel

() Andy Wingo <wingo@pobox.com>
() Tue, 20 Apr 2010 11:45:33 +0200

   we should just switch to use ELF at some point.

OK, i'll keep my ear to the ground for that.

   >  Maybe we could just include the source
   > verbatim in such a section (and avoid mapping it at runtime).

   As Jao notes it is useful to be able to map at runtime, and currently we
   do this well, mapping to line and column within parts of procedures.

   But it would be good to be able to map to expressions too, and having
   the original source available without mapping to the filesystem should
   be an option. It's tough without having the whole environment, though.
   Something to think about :)

Agreed.  Sorry, i wasn't precise.  I should have said:

> (and map it only if needed at runtime)

Anyone interested in this kind of stuff should probably take a look
at Guile 1.4.x script ‘scan-md-module’, which does things the hard way.
That is, if you find yourself doing the same, beware!

thi




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: port-filename and path canonicalization
  2010-04-20  9:42   ` Andy Wingo
@ 2010-04-20 11:15     ` Thien-Thi Nguyen
  2010-04-21  8:49       ` Ludovic Courtès
  2010-04-20 16:57     ` Ludovic Courtès
  1 sibling, 1 reply; 19+ messages in thread
From: Thien-Thi Nguyen @ 2010-04-20 11:15 UTC (permalink / raw)
  To: guile-devel

() Andy Wingo <wingo@pobox.com>
() Tue, 20 Apr 2010 11:42:58 +0200

   I'd be happy to have some cleaner solution, though.
   Do you have any ideas?

Another idea is to conceptually separate "filename" to be "directory"
and "basename" and have file ports maintain the directory portion as a
file descriptor (or equivalent) internally, exposed through some kind of
opaque handle.

With this separation, one can start to play with other virtualized
location schemes (FUSE, tarball, network byte stream, VMS-style logical
names, etc), while also addressing the actual need directly: finding
"neighbors" of the already opened file.

One (important for users' software freedom) neighbor is the source code;
others might be debugging info, runtime stats (thinking of future JIT),
previous versions (for ABI negotiation/fallback), and so on.

If the separation is applied earlier, that is, if directory objects can
be made without opening a file, you get "browsing".

FWIW, in Guile 1.4.x, i was headed towards this with module catalogs
(inspired by SLIB's catalog system), but it is not yet fully realized,
and might never be.  Perhaps i can move some of that to Guile 1.9.

thi




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: file names embedded in .go
  2010-04-20  0:08   ` Jose A. Ortega Ruiz
@ 2010-04-20 11:35     ` Thien-Thi Nguyen
  2010-04-20 19:15       ` Jose A. Ortega Ruiz
  0 siblings, 1 reply; 19+ messages in thread
From: Thien-Thi Nguyen @ 2010-04-20 11:35 UTC (permalink / raw)
  To: guile-devel

() "Jose A. Ortega Ruiz" <jao@gnu.org>
() Tue, 20 Apr 2010 02:08:32 +0200

   I'm not sure if i'm understanding you correctly, but if, by
   source, you mean the scheme source, that'd be terrible news for
   tools such as Geiser.

I'm not familiar w/ Geiser.  What does it do?  What are its restrictions?

thi




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: port-filename and path canonicalization
  2010-04-20  9:42   ` Andy Wingo
  2010-04-20 11:15     ` Thien-Thi Nguyen
@ 2010-04-20 16:57     ` Ludovic Courtès
  2010-04-22 11:10       ` Andy Wingo
  1 sibling, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2010-04-20 16:57 UTC (permalink / raw)
  To: Andy Wingo; +Cc: guile-devel

Hi,

Andy Wingo <wingo@pobox.com> writes:

> On Tue 20 Apr 2010 01:12, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Andy Wingo <wingo@pobox.com> writes:
>>
>>> I recently added a global fluid, %file-port-name-canonicalization, which
>>> defaults to #f. But if it's 'absolute, the port name of a file port will
>>> be canonicalized to the absolute path; or, if it's 'relative, the port
>>> name is the canonical name of the file, relative to the %load-path, or
>>> the file name as given otherwise.
>>>
>>> The intention was to allow the user to control (port-filename P), so
>>> that the user could find e.g. the absolute path corresponding to that
>>> port at the time that it was made.
>>
>> My feeling is that ports shouldn’t have to deal with paths because
>> that’s a separate concern.  The %file-port-name-canonicalization fluid
>> seems like an inelegant hack to me.
>>
>> When applications have special requirements about paths, then it should
>> be up to the application logic to deal with that.

IOW, applications are free to do:

  (open-input-file (canonicalize-path filename))

instead of:

  (open-input-file filename)

And that’s all it takes at the application level.

> 2. I think a fluid is still necessary, because a file being
> compiled can do an `include' or `include-from-path', or even
> `open-input-file' in a macro, and all these cases you would want the
> same %file-port-name-canonicalization to take effect.

Indeed, this one is tricky.

I still think it’s application-specific, though.  How about calling the
fluid, say, %compiler-file-name-canonicalization instead?  :-)

> 3. The only correct time to do a path canonicalization is when the file
> is opened, because at another time, you might not be in the same current
> directory, so relative paths would resolve incorrectly.

Yes.

> 4. The application-level code is nastier if it has to canonicalize,
> because a relative canonicalization

What do you mean by “relative canonicalization”?

(I have Glibc’s ‘canonicalize_file_name ()’ in mind, which returns an
absolute path, so I’m confused.)

> cannot in general be passed to open-input-file. For example
>
>   (open-input-file "../../module/ice-9/boot-9.scm")
>
> is not the same as
>
>   (open-input-file "ice-9/boot-9.scm")

Agreed.  :-)

> So you'd have to do a set-port-filename! on the port, mucking up your
> code -- and how would you decide what to set? In N places you'd have to
> duplicate fport_canonicalize_filename, and you'd probably have to make
> scm_i_relativize_path public.

I failed to get the transition at “So”.  :-)

What does scm_i_relativize_path do?  (It lacks a leading comment, hint
hint.  ;-))

Thanks for taking the time to explain!

Ludo’.




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: file names embedded in .go
  2010-04-20 11:35     ` Thien-Thi Nguyen
@ 2010-04-20 19:15       ` Jose A. Ortega Ruiz
  2010-04-21  7:45         ` Thien-Thi Nguyen
  0 siblings, 1 reply; 19+ messages in thread
From: Jose A. Ortega Ruiz @ 2010-04-20 19:15 UTC (permalink / raw)
  To: guile-devel

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> () "Jose A. Ortega Ruiz" <jao@gnu.org>
> () Tue, 20 Apr 2010 02:08:32 +0200
>
>    I'm not sure if i'm understanding you correctly, but if, by
>    source, you mean the scheme source, that'd be terrible news for
>    tools such as Geiser.
>
> I'm not familiar w/ Geiser. What does it do? What are its
> restrictions?

It's an Emacs environment for Scheme similar to Slime: a repl and a
bunch of utilities to edit, navigate and obtain information about the
code. Among them, jumping to an identifier's definition: for that, i
need guile to tell me where in the file system the identifier is bound.
Geiser does not usually collect metadata on the elisp side, relying on
the running guile to provide it. The richer that dynamic metadata is,
the more functionality Geiser can provide.

A bit more information is available at http://gitorious.org/geiser and
links thereof (specially the README; no real documentation yet, i'm
afraid).

jao
-- 
Dealing with failure is easy: Work hard to improve. Success is also
easy to handle: You've solved the wrong problem. Work hard to improve.
  - Alan Perlis, Epigrams in Programing





^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: file names embedded in .go
  2010-04-20 19:15       ` Jose A. Ortega Ruiz
@ 2010-04-21  7:45         ` Thien-Thi Nguyen
  0 siblings, 0 replies; 19+ messages in thread
From: Thien-Thi Nguyen @ 2010-04-21  7:45 UTC (permalink / raw)
  To: guile-devel

() "Jose A. Ortega Ruiz" <jao@gnu.org>
() Tue, 20 Apr 2010 21:15:25 +0200

   It's an Emacs environment for Scheme similar to Slime: a repl and a
   bunch of utilities to edit, navigate and obtain information about the
   code. Among them, jumping to an identifier's definition: for that, i
   need guile to tell me where in the file system the identifier is bound.
   Geiser does not usually collect metadata on the elisp side, relying on
   the running guile to provide it. The richer that dynamic metadata is,
   the more functionality Geiser can provide.

   A bit more information is available at http://gitorious.org/geiser and
   links thereof (specially the README; no real documentation yet, i'm
   afraid).

Thanks for the pointer.  I'll take a look next time on the net.

I think (designing out loud) programs like Geiser will have no problem
if they use Emacs' features that support "virtually located" files.
See ‘file-name-handler-alist’, for the most fundamental mechanism.

Of course the inferior Guile needs to provide a clean syntax that doesn't
conflict with other subsystems.  There are basically two approaches: work on
top of something else (best candidate: TRAMP), or sidestep all others.

Programs that don't build on Emacs are another matter, but who in their
right mind goes outside Emacs?  ;-)

thi




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: port-filename and path canonicalization
  2010-04-20 11:15     ` Thien-Thi Nguyen
@ 2010-04-21  8:49       ` Ludovic Courtès
  2010-04-21 19:16         ` Thien-Thi Nguyen
  0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2010-04-21  8:49 UTC (permalink / raw)
  To: guile-devel

Hi,

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> Another idea is to conceptually separate "filename" to be "directory"
> and "basename" and have file ports maintain the directory portion as a
> file descriptor (or equivalent) internally, exposed through some kind of
> opaque handle.

I think open file ports shouldn’t grant any authority beyond access to
the open file.  Just like an open file descriptor doesn’t convey any
authority beyond access to the underlying file (if we omit ‘..’ lookups
on a directory file descriptor with openat(3)).

Thanks,
Ludo’.





^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: port-filename and path canonicalization
  2010-04-21  8:49       ` Ludovic Courtès
@ 2010-04-21 19:16         ` Thien-Thi Nguyen
  2010-04-21 22:26           ` Ludovic Courtès
  0 siblings, 1 reply; 19+ messages in thread
From: Thien-Thi Nguyen @ 2010-04-21 19:16 UTC (permalink / raw)
  To: guile-devel

() ludo@gnu.org (Ludovic Courtès)
() Wed, 21 Apr 2010 10:49:05 +0200

   I think open file ports shouldn’t grant any authority beyond
   access to the open file.  Just like an open file descriptor
   doesn’t convey any authority beyond access to the underlying
   file (if we omit ‘..’ lookups on a directory file descriptor
   with openat(3)).

I agree (and was about to cite openat(3) et al -- glad you
beat me to it!), but that's neither here nor there:

Whether or not the authority associated with the containing
directory is user-visible is a design detail of the directory
object.  (More information need not imply more access.)

That is, if a file port supports ‘file-port-directory’, then how
to use/restrict the resulting object is left up to higher layers,
where it belongs.

Reifying directories is good for both security and efficiency.
Why chase symlinks and {l}stat(2) more than necessary?

thi




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: port-filename and path canonicalization
  2010-04-21 19:16         ` Thien-Thi Nguyen
@ 2010-04-21 22:26           ` Ludovic Courtès
  2010-04-22  7:42             ` Thien-Thi Nguyen
  0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2010-04-21 22:26 UTC (permalink / raw)
  To: guile-devel

Hi,

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> That is, if a file port supports ‘file-port-directory’, then how
> to use/restrict the resulting object is left up to higher layers,
> where it belongs.

I would put it the other way round: if an application wants to implement
‘file-port-directory’, then it’s its responsibility to associate the
necessary information (and authority) with open file ports (those under
its control, that is.)

It’s an application of the principle of least authority.

Thanks,
Ludo’.





^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: port-filename and path canonicalization
  2010-04-21 22:26           ` Ludovic Courtès
@ 2010-04-22  7:42             ` Thien-Thi Nguyen
  0 siblings, 0 replies; 19+ messages in thread
From: Thien-Thi Nguyen @ 2010-04-22  7:42 UTC (permalink / raw)
  To: guile-devel

() ludo@gnu.org (Ludovic Courtès)
() Thu, 22 Apr 2010 00:26:43 +0200

   > That is, if a file port supports ‘file-port-directory’, then how
   > to use/restrict the resulting object is left up to higher layers,
   > where it belongs.

   I would put it the other way round: if an application wants to
   implement ‘file-port-directory’, then it’s its responsibility
   to associate the necessary information (and authority) with
   open file ports (those under its control, that is.)

   It’s an application of the principle of least authority.

Sure, given the model that access and authority are equivalent.
My point is that it is possible to use a model where they aren't.
A system that supports the latter model can build the former on top.
But a system that supports only the former model can only fake the
latter model, later.

It's always easier to combine separate things later than to
separate conflated things later.  The canonicalization wranglings
highlight this.

Canonicalization is a high-level concept needed because the
low-level access was not granular enough.  A filename with
directory components of an already validated (opened, accessed,
processed in the Right Way) file implies that all the parent
directories have also *already* been validated (in their own way).
But that, and related, information is lost, so we need to
canonicalize in order to re-validate those portions.

If that directory information is saved (simple internal caching),
these particular recomputations are avoidable.

If that directory information is available to the application
(reified and appropriately sanitized), the application can layer
authority measures as it sees fit (as you state in the first
paragraph, IIUC), in addition to enjoying the efficiency gain,
and the need for canonicalization disappears.

(Actually, the need for canonicalization disappears even with
simple internal caching, but since the context of this discussion
is process B finding neighbors to a file created by process A,
it is no longer an internal matter.  Or perhaps that's just me
jumping ahead (i've got bugs, debuggers and debugging file formats
on the mind lately)?)

OK, back to the grindstone.  I'm digging my way out RCS duties
(one might view this as preparation for History Objects ;-) right
now, but hopefully will be able to transform word spew to code
spew (for Guile) in the next several weeks.

A last parting word/pointer on this thread re "path":
 (info "(standards) GNU Manuals")

thi




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: port-filename and path canonicalization
  2010-04-20 16:57     ` Ludovic Courtès
@ 2010-04-22 11:10       ` Andy Wingo
  2010-04-22 12:50         ` Ludovic Courtès
  0 siblings, 1 reply; 19+ messages in thread
From: Andy Wingo @ 2010-04-22 11:10 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guile-devel

Hi Ludovic,

On Tue 20 Apr 2010 18:57, ludo@gnu.org (Ludovic Courtès) writes:

>> 2. I think a fluid is still necessary, because a file being
>> compiled can do an `include' or `include-from-path', or even
>> `open-input-file' in a macro, and all these cases you would want the
>> same %file-port-name-canonicalization to take effect.
>
> Indeed, this one is tricky.
>
> I still think it’s application-specific, though.  How about calling the
> fluid, say, %compiler-file-name-canonicalization instead?  :-)

Are you proposing that all file-opening functions check the
%compiler-file-name-canonicalization fluid? Seems better to me for the
name of the fluid to be "close" to the function whose behavior it
changes.

>> 4. The application-level code is nastier if it has to canonicalize,
>> because a relative canonicalization
>
> What do you mean by “relative canonicalization”?
>
> (I have Glibc’s ‘canonicalize_file_name ()’ in mind, which returns an
> absolute path, so I’m confused.)
>
>> cannot in general be passed to open-input-file. For example
>>
>>   (open-input-file "../../module/ice-9/boot-9.scm")
>>
>> is not the same as
>>
>>   (open-input-file "ice-9/boot-9.scm")
>
> Agreed.  :-)

If you build out-of-source, you might end up with

  guile-tools compile -o ice-9/boot-9.go ../../modules/ice-9/boot-9.scm

So the file you open is "../../modules/ice-9/boot-9.scm", but you want
to give it a "relative canonicalization" -- in this case
"ice-9/boot-9.scm" is the canonicalization of
../../modules/ice-9/boot-9.scm, *relative* to "../../modules" (or indeed
to "/home/wingo/src/guile/modules").

>> So you'd have to do a set-port-filename! on the port, mucking up your
>> code -- and how would you decide what to set? In N places you'd have to
>> duplicate fport_canonicalize_filename, and you'd probably have to make
>> scm_i_relativize_path public.
>
> I failed to get the transition at “So”.  :-)
>
> What does scm_i_relativize_path do?  (It lacks a leading comment, hint
> hint.  ;-))

You could try a little harder ;-) Perhaps the paragraph above explains;
you would have to do:

  (define (open-input-file* path)
    (let ((p (open-input-file path)))
      (case (fluid-ref %file-port-name-canonicalization)
        ((absolute)
         (set-port-filename! p (canonicalize-path path)))
        ((relative)
         (set-port-filename! p (relativize-path (canonicalize-path path) %load-path))))
      p))

which is a bit ugly; e.g. include-from-path would need to do this, as
would any third-party equivalent of include-from-path, and even
down to the open-file level. Yuk.

Andy
-- 
http://wingolog.org/




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: port-filename and path canonicalization
  2010-04-22 11:10       ` Andy Wingo
@ 2010-04-22 12:50         ` Ludovic Courtès
  0 siblings, 0 replies; 19+ messages in thread
From: Ludovic Courtès @ 2010-04-22 12:50 UTC (permalink / raw)
  To: Andy Wingo; +Cc: guile-devel

Hi,

Andy Wingo <wingo@pobox.com> writes:

> On Tue 20 Apr 2010 18:57, ludo@gnu.org (Ludovic Courtès) writes:
>
>>> 2. I think a fluid is still necessary, because a file being
>>> compiled can do an `include' or `include-from-path', or even
>>> `open-input-file' in a macro, and all these cases you would want the
>>> same %file-port-name-canonicalization to take effect.
>>
>> Indeed, this one is tricky.
>>
>> I still think it’s application-specific, though.  How about calling the
>> fluid, say, %compiler-file-name-canonicalization instead?  :-)
>
> Are you proposing that all file-opening functions check the
> %compiler-file-name-canonicalization fluid?

No, I’m talking about file-opening functions of the compiler, which
include ‘compile-file’, ‘include’, etc.

> If you build out-of-source, you might end up with
>
>   guile-tools compile -o ice-9/boot-9.go ../../modules/ice-9/boot-9.scm
>
> So the file you open is "../../modules/ice-9/boot-9.scm", but you want
> to give it a "relative canonicalization" -- in this case
> "ice-9/boot-9.scm" is the canonicalization of
> ../../modules/ice-9/boot-9.scm, *relative* to "../../modules" (or indeed
> to "/home/wingo/src/guile/modules").

OK, understood.

>>> So you'd have to do a set-port-filename! on the port, mucking up your
>>> code -- and how would you decide what to set? In N places you'd have to
>>> duplicate fport_canonicalize_filename, and you'd probably have to make
>>> scm_i_relativize_path public.
>>
>> I failed to get the transition at “So”.  :-)
>>
>> What does scm_i_relativize_path do?  (It lacks a leading comment, hint
>> hint.  ;-))

BTW, my understanding is that scm_i_relativize_path does lexical
“relativization”, which doesn’t work on POSIX and GNU systems where ‘..’
resolution is /not/ lexical.  (See
<http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames> for details.)

The following should work:

--8<---------------cut here---------------start------------->8---
(define (relativize-path path dir)
  ;; Return PATH relative to DIR or #f on failure.
  (let ((path (canonicalize-path path))
        (dir  (canonicalize-path dir)))
    (and=> (string-prefix-length dir path)
           (lambda (start)
             (substring path (+ 1 start) (string-length path))))))

scheme@(guile-user)> (relativize-path "../guile/module/ice-9/boot-9.scm" "module")
"ice-9/boot-9.scm"
scheme@(guile-user)> (relativize-path "module/ice-9/boot-9.scm" "../guile/module")
"ice-9/boot-9.scm"
--8<---------------cut here---------------end--------------->8---

It still does lexical relativization but does so after canonicalization
of both arguments.

> you would have to do:
>
>   (define (open-input-file* path)
>     (let ((p (open-input-file path)))
>       (case (fluid-ref %file-port-name-canonicalization)
>         ((absolute)
>          (set-port-filename! p (canonicalize-path path)))
>         ((relative)
>          (set-port-filename! p (relativize-path (canonicalize-path path) %load-path))))
>       p))
>
> which is a bit ugly; e.g. include-from-path would need to do this, as
> would any third-party equivalent of include-from-path, and even
> down to the open-file level. Yuk.

Wouldn’t it suffice if only ‘compile-file’, ‘include’, and
‘include-from-path’ relativize paths?

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2010-04-22 12:50 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-04-19 14:52 file names embedded in .go Andy Wingo
2010-04-19 21:46 ` Thien-Thi Nguyen
2010-04-20  0:08   ` Jose A. Ortega Ruiz
2010-04-20 11:35     ` Thien-Thi Nguyen
2010-04-20 19:15       ` Jose A. Ortega Ruiz
2010-04-21  7:45         ` Thien-Thi Nguyen
2010-04-20  9:45   ` Andy Wingo
2010-04-20 10:34     ` Thien-Thi Nguyen
2010-04-19 23:12 ` port-filename and path canonicalization Ludovic Courtès
2010-04-20  9:42   ` Andy Wingo
2010-04-20 11:15     ` Thien-Thi Nguyen
2010-04-21  8:49       ` Ludovic Courtès
2010-04-21 19:16         ` Thien-Thi Nguyen
2010-04-21 22:26           ` Ludovic Courtès
2010-04-22  7:42             ` Thien-Thi Nguyen
2010-04-20 16:57     ` Ludovic Courtès
2010-04-22 11:10       ` Andy Wingo
2010-04-22 12:50         ` Ludovic Courtès
2010-04-19 23:23 ` file names embedded in .go Ludovic Courtès

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).