subbytevectors

subbytevectors

[Andy Wingo]

Hi,

It would be very convenient to offer bytevectors that give a view on
some other data structure, possibly another bytevector.  We already have
that, to an extent, with pointer->bytevector.  We can consecrate that
with some subbytevector facility.

I think it's a good idea but it has some costs.  One is that currently,
in the R6RS bytevector specification, one bytevector cannot alias
another.  This knowledge can allow the optimizer to do more things.
Another point is that since Guile can control the allocation of
bytevectors, it can ensure their alignments, and so compile e.g. a
(bytevector-u32-ref bv 12 (native-endianness)) to an efficient access,
knowing that it is aligned.  If we offer subbytevectors, this won't be
possible in general.

Again, the gain in expressiveness is probably worth it, but I wanted to
put the question out there to see if anyone has an opinion.

Regards,

Andy
-- 
http://wingolog.org/

Re: subbytevectors

[Thien-Thi Nguyen]

() Andy Wingo <wingo@pobox.com>
() Sat, 09 Jun 2012 13:07:15 +0200

   Again, the gain in expressiveness is probably worth it

Overall, i am concerned about quick fixes and slow suffering
in the Guile design.  To break it down from different angles:

Thinking positively:

If you want to make a case for such a facility, why not
show some code, both without (status quo) and with (proposed)?
It should be clear what expressiveness is gained, and how.

Thinking negatively: 

Guile 1.4.x has ‘make-shared-substring’, which is similar in
spirit (since strings of that era are basically byte vectors),
but i believe later Guile versions dropped that.  It might be
instructive to (reconstruct if necessary and) follow that chain
of reasoning to avoid repeating a similar flip-flop.

Thinking abstractly:

IIRC, SRFI 13 suggests that its support for substrings would
not be necessary if programmers wrote code using "range" style.
Could the client code you have in mind be rephrased like that?

Re: subbytevectors

[Andy Wingo]

Hello,

On Sat 09 Jun 2012 17:16, Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> If you want to make a case for such a facility, why not
> show some code, both without (status quo) and with (proposed)?
> It should be clear what expressiveness is gained, and how.

For example, let's say I mmap a big file.

  (define x (mmap-file "/usr/lib/debug/lib/x86_64-linux-gnu/libc-2.13.so"))

I did some computation and have figured out that there is a region of
interest between bytes 121241 and 121263 that interests me.  I would
like to be able to pass off that region to some other piece of code,
without giving it access to the entire bytevector.  I would also like to
be able to pass around  

> Guile 1.4.x has ‘make-shared-substring’

Guile 1.8, released in 2005, has substring/shared.  So does Guile 2.0.
IIRC -- and this was a looooooooong time ago -- Marius wanted to remove
it, for ease of implementation, but users were using it, so he had to
put it back in.

Strings and bytevectors are fundamentally different, though.

> IIRC, SRFI 13 suggests that its support for substrings would
> not be necessary if programmers wrote code using "range" style.
> Could the client code you have in mind be rephrased like that?

These are complementary strategies.  Using ranges is usually more
efficient, but more at times it is too cumbersome.  Sometimes also you
really want to restrict authority to only a certain window of the
bytevector.

For example, I am currently working on a problem that involves lots of
work on a shared bytevector.  I have to be careful to avoid printing out
the bytevector because it takes a few seconds and trashes my terminal
history.  If I had subbytevectors, this wouldn't be as acute a problem.

Andy
-- 
http://wingolog.org/

Re: subbytevectors

[Ludovic Courtès]

Hi,

Andy Wingo <wingo@pobox.com> skribis:

> On Sat 09 Jun 2012 17:16, Thien-Thi Nguyen <ttn@gnuvola.org> writes:
>
>> If you want to make a case for such a facility, why not
>> show some code, both without (status quo) and with (proposed)?
>> It should be clear what expressiveness is gained, and how.
>
> For example, let's say I mmap a big file.
>
>   (define x (mmap-file "/usr/lib/debug/lib/x86_64-linux-gnu/libc-2.13.so"))
>
> I did some computation and have figured out that there is a region of
> interest between bytes 121241 and 121263 that interests me.  I would
> like to be able to pass off that region to some other piece of code,
> without giving it access to the entire bytevector.  I would also like to
> be able to pass around  

What about using copying (or rather, copy-on-write) sub-bytevectors to
start with?  That would avoid the aliasing issue; OTOH COW would make
the implementation more complex.

Thanks,
Ludo’.

Re: subbytevectors

[Andy Wingo]

On Mon 11 Jun 2012 13:55, ludo@gnu.org (Ludovic Courtès) writes:

> What about using copying (or rather, copy-on-write) sub-bytevectors to
> start with?  That would avoid the aliasing issue; OTOH COW would make
> the implementation more complex.

Not a bad idea.  The FFI can still introduce aliasing, though I don't
know if that matters.  Alignment is still a problem though.  Plus it's a
bit more complexity, though an optimizer should be able to do something
about that.

I'll see how much work an implementation would be.

Andy
-- 
http://wingolog.org/

Re: subbytevectors

[Ludovic Courtès]

Hi,

Andy Wingo <wingo@pobox.com> skribis:

> On Mon 11 Jun 2012 13:55, ludo@gnu.org (Ludovic Courtès) writes:
>
>> What about using copying (or rather, copy-on-write) sub-bytevectors to
>> start with?  That would avoid the aliasing issue; OTOH COW would make
>> the implementation more complex.
>
> Not a bad idea.  The FFI can still introduce aliasing, though I don't
> know if that matters.  Alignment is still a problem though.  Plus it's a
> bit more complexity, though an optimizer should be able to do something
> about that.

Currently alignment is more of a theoretical problem because we are not
yet at a point where such an optimization matters (and I think it
wouldn’t matter on x86 anyway), no?  :-)

Though in principle, I agree we should avoid preventing future
optimizations.

Ludo’.