Re: more advanced bytevector => supervectors

[Matt Wette <matt.wette@gmail.com>]

maybe guile could consider regexp's in scheme
see  https://ds26gte.github.io/pregexp/index.html

On 9/2/21 8:45 AM, Stefan Israelsson Tampe wrote:
> Hi guilers!
>
> My next project is to explore a more advanced bytevector structure 
> than today's bytevectors.
> I think that having this infrastructure in guile and core code taken 
> advantage of it like having strings otop of it and allowing our string 
> library to use those (the difficult case is probably to get regexps 
> working properly)
>
> Anyhow this is my initial comment in the code:
>
> #|
> The idea of this data structure is to note that when we employ full
> large datastructure scans we can allow for a much more rich and 
> featurefull
> datastructure then a simple bytevector. The reason is that we can divide
> the data in byte chunks and spend most of the time scanning copying maping
> those areas with usual methis, even optimized C - code taken advantage 
> of advanced cpu opcodes are possible here. ANd by dividing it in 
> chunks we get a lot of new features with essentiually no cose with 
> more than complexity which we manage mostly in scheme. We gain many 
> things,
>
> 1. Many new features that can vary on the pages
>
> 2. less memory hogs as
>    a. it can use copy ion write semantics
>    b. it does not need to fins 1GB continuous blocks
>
> 3. potentially faster operations as we can fast foorward the zeroe on 
> write
>    pages compared to pure bytevectors
>
> 4. we can allow for swaping and refferential datastructures to speed 
> up copying
>    even further
>
> 5. can get better fiber features of C programs that spends seconds or 
> minutes on
>    performing an operation because it will just spend a microsecond or 
> such
>    in C-land and then swap back to Scheme. CTRL-C will also work nicely.
>
> 6. We could potentially build a string library untop of these 
> datastructures and
>    also we could add features to pages that lead to a much richer 
> interface.
>
> 7. resizing is much faster end efficient
>
> 8. reversing is super quick
>
> 9. queues and stacks of byte data can have a very efficient 
> implementations
>
> Drawback:
> 1. More complex as we need to consider boudaries
> 2. Slower one off operations like bytevector-u8-get as guile compiles the
>    core operatoins to quite effective JIT CPU encodings. But maybe we can
>    disign some caching to make those operations much faster and even have
>    suporting JIT operations.
>
> |#
>
> WDYT ?
>