What is the point of bytevectors?

What is the point of bytevectors?

[divoplade]

Hello guile users,

I am writing a library mixing some scheme code and C code, and I have
two options for interfacing C strings:
1. Use bytevectors;
2. Use strings with byte access semantics (so-called latin-1, which is
really a misleading name since it will most certainly contain utf-8-
encoded unicode text).

From the C side, they have nearly identical APIs, and the conversion
functions do not transcode anything.

From the scheme side, however:
1. The bytevector library needs to be imported;
2. The function names have way more characters to type;
3. The bytevector library is missing a lot of text functions (like
join, split, trim, pad, searching...).

If the user wants to always manipulate unicode (decoded) strings, using
either bytevectors or latin-1 strings require transcoding to enter the
library and to exit the library, so either option is valid.

But if the user wants to always manipulate utf-8-encoded strings [1],
using bytevectors is impossible or much more difficult (see points
above).

So, why should I ever use bytevectors?

divoplade

[1] https://utf8everywhere.org/

Re: What is the point of bytevectors?

[Linus Björnstam]

The point is to work with binary data, of which the most common type of C strings are one kind.

If you are using libguile in your C code you can use guile strings in your C code and pass them around to avoid the encoding/decoding overhead. Or, the other way around, expose the procedures that work with whatever C string representation your are using to guile.

If your latin1 strings contain unicode data they are not latin1.

-- 
  Linus Björnstam

On Sat, 12 Sep 2020, at 09:49, divoplade wrote:
> Hello guile users,
> 
> I am writing a library mixing some scheme code and C code, and I have
> two options for interfacing C strings:
> 1. Use bytevectors;
> 2. Use strings with byte access semantics (so-called latin-1, which is
> really a misleading name since it will most certainly contain utf-8-
> encoded unicode text).
> 
> From the C side, they have nearly identical APIs, and the conversion
> functions do not transcode anything.
> 
> From the scheme side, however:
> 1. The bytevector library needs to be imported;
> 2. The function names have way more characters to type;
> 3. The bytevector library is missing a lot of text functions (like
> join, split, trim, pad, searching...).
> 
> If the user wants to always manipulate unicode (decoded) strings, using
> either bytevectors or latin-1 strings require transcoding to enter the
> library and to exit the library, so either option is valid.
> 
> But if the user wants to always manipulate utf-8-encoded strings [1],
> using bytevectors is impossible or much more difficult (see points
> above).
> 
> So, why should I ever use bytevectors?
> 
> divoplade
> 
> [1] https://utf8everywhere.org/
> 
> 
>

Re: What is the point of bytevectors?

[John Cowan]

On Sat, Sep 12, 2020 at 3:51 AM divoplade <d@divoplade.fr> wrote:


> From the scheme side, however:
> 1. The bytevector library needs to be imported;
>

If a single line is a serious overhead, then your program is tiny and you
can do whatever you want: it will be obvious to whoever maintains the
program next (including you).


> 2. The function names have way more characters to type;
>

Use an autocompleting editor such as Emacs.


> 3. The bytevector library is missing a lot of text functions (like
> join, split, trim, pad, searching...).
>

That is indeed an issue.  Fortunately SRFI 207, though not yet final, comes
to your rescue. You can't get the spiffy syntax described at <
https://srfi.schemers.org/srfi-207/srfi-207.html> unless you modify the
Guile reader, but the procedures documented there can be found at <
https://github.com/Zipheir/bytestrings>.



John Cowan          http://vrici.lojban.org/~cowan        cowan@ccil.org
It was impossible to inveigle
Georg Wilhelm Friedrich Hegel
Into offering the slightest apology
For his Phenomenology.                      --W. H. Auden, from "People"
(1953)

Re: What is the point of bytevectors?

[divoplade]

Dear John Cowan,

Le samedi 12 septembre 2020 à 12:48 -0400, John Cowan a écrit :
> > 3. The bytevector library is missing a lot of text functions (like
> > join, split, trim, pad, searching...).
> 
> That is indeed an issue.  Fortunately SRFI 207, though not yet final,
> comes to your rescue. You can't get the spiffy syntax described at <
> https://srfi.schemers.org/srfi-207/srfi-207.html> unless you modify
> the Guile reader, but the procedures documented there can be found at
> <https://github.com/Zipheir/bytestrings>;.

Thank you for your answer, it is exactly what I was looking for.

Best regards,

divoplade

Re: What is the point of bytevectors?

[John Cowan]

You're welcome!

On Sat, Sep 12, 2020 at 1:14 PM divoplade <d@divoplade.fr> wrote:

> Dear John Cowan,
>
> Le samedi 12 septembre 2020 à 12:48 -0400, John Cowan a écrit :
> > > 3. The bytevector library is missing a lot of text functions (like
> > > join, split, trim, pad, searching...).
> >
> > That is indeed an issue.  Fortunately SRFI 207, though not yet final,
> > comes to your rescue. You can't get the spiffy syntax described at <
> > https://srfi.schemers.org/srfi-207/srfi-207.html> unless you modify
> > the Guile reader, but the procedures documented there can be found at
> > <https://github.com/Zipheir/bytestrings>;.
>
> Thank you for your answer, it is exactly what I was looking for.
>
> Best regards,
>
> divoplade
>
>