is there a way to serialize hash tables?

is there a way to serialize hash tables?

[Tim Meehan]

I have a big-ish blob of key-value pairs that I would like to not have to
store as text and then convert to a hashtable when I am filtering my data.

Right now I am saving it as an alist and then converting back to a
hashtable using "alist->hashtable" from SRFI 69. I was just hoping that
there was something more clever ... perhaps it is plenty clever enough.

Re: is there a way to serialize hash tables?

[John Cowan]

I think that's the best bet, although JSON is also a possibility.  I'm
working on a more general text-serialization solution, but it will be quite
some time before I have a chance to work on it.

On Fri, Dec 25, 2020 at 10:28 PM Tim Meehan <btmeehan@gmail.com> wrote:

> I have a big-ish blob of key-value pairs that I would like to not have to
> store as text and then convert to a hashtable when I am filtering my data.
>
> Right now I am saving it as an alist and then converting back to a
> hashtable using "alist->hashtable" from SRFI 69. I was just hoping that
> there was something more clever ... perhaps it is plenty clever enough.
>

Re: is there a way to serialize hash tables?

[Linus Björnstam]

Alex and I got the speed of guile-json up to about 19megs a second under guile 3 on a 4 year old intel i5, which may or may not be adequate for your needs, Tim.

I have some code I wrote during that spurt that preforms slightly better than guile-json in general (about 3-4% on my machine. Alex could not repeat those numbers) an on files with many large strings in particular (10% in extreme cases).  If that is interesting I could put it online somewhere. The functioning code is a derivate work of guile-json, so the GPL applies.

-- 
  Linus Björnstam

On Sat, 26 Dec 2020, at 05:28, John Cowan wrote:
> I think that's the best bet, although JSON is also a possibility.  I'm
> working on a more general text-serialization solution, but it will be quite
> some time before I have a chance to work on it.
> 
> On Fri, Dec 25, 2020 at 10:28 PM Tim Meehan <btmeehan@gmail.com> wrote:
> 
> > I have a big-ish blob of key-value pairs that I would like to not have to
> > store as text and then convert to a hashtable when I am filtering my data.
> >
> > Right now I am saving it as an alist and then converting back to a
> > hashtable using "alist->hashtable" from SRFI 69. I was just hoping that
> > there was something more clever ... perhaps it is plenty clever enough.
> >
>

Re: is there a way to serialize hash tables?

[Tim Meehan]

Thanks a lot guys - I did a test with the stored alist and it was plenty
fast enough for what I needed. I had looked at guile-json for reading in
some settings. It is good to know that it is very fast also.

On Sat, Dec 26, 2020 at 6:10 AM Linus Björnstam <linus.internet@fastmail.se>
wrote:

> Alex and I got the speed of guile-json up to about 19megs a second under
> guile 3 on a 4 year old intel i5, which may or may not be adequate for your
> needs, Tim.
>
> I have some code I wrote during that spurt that preforms slightly better
> than guile-json in general (about 3-4% on my machine. Alex could not repeat
> those numbers) an on files with many large strings in particular (10% in
> extreme cases).  If that is interesting I could put it online somewhere.
> The functioning code is a derivate work of guile-json, so the GPL applies.
>
> --
>   Linus Björnstam
>
> On Sat, 26 Dec 2020, at 05:28, John Cowan wrote:
> > I think that's the best bet, although JSON is also a possibility.  I'm
> > working on a more general text-serialization solution, but it will be
> quite
> > some time before I have a chance to work on it.
> >
> > On Fri, Dec 25, 2020 at 10:28 PM Tim Meehan <btmeehan@gmail.com> wrote:
> >
> > > I have a big-ish blob of key-value pairs that I would like to not have
> to
> > > store as text and then convert to a hashtable when I am filtering my
> data.
> > >
> > > Right now I am saving it as an alist and then converting back to a
> > > hashtable using "alist->hashtable" from SRFI 69. I was just hoping that
> > > there was something more clever ... perhaps it is plenty clever enough.
> > >
> >
>

Re: is there a way to serialize hash tables?

[Aleix Conchillo Flaqué]

You almost got it right Linus, it's Aleix. :-)

For the curious, it sounds something like "Alaish" or here in the US most
people can pronounce it as "Alesh". But more accurately [aleʃ] (
https://en.m.wikipedia.org/wiki/Help:IPA/Catalan)

Anyways, happy hacking and happy New Year to everyone!

Aleix

On Sat, Dec 26, 2020, 4:10 AM Linus Björnstam <linus.internet@fastmail.se>
wrote:

> Alex and I got the speed of guile-json up to about 19megs a second under
> guile 3 on a 4 year old intel i5, which may or may not be adequate for your
> needs, Tim.
>
> I have some code I wrote during that spurt that preforms slightly better
> than guile-json in general (about 3-4% on my machine. Alex could not repeat
> those numbers) an on files with many large strings in particular (10% in
> extreme cases).  If that is interesting I could put it online somewhere.
> The functioning code is a derivate work of guile-json, so the GPL applies.
>
> --
>   Linus Björnstam
>
> On Sat, 26 Dec 2020, at 05:28, John Cowan wrote:
> > I think that's the best bet, although JSON is also a possibility.  I'm
> > working on a more general text-serialization solution, but it will be
> quite
> > some time before I have a chance to work on it.
> >
> > On Fri, Dec 25, 2020 at 10:28 PM Tim Meehan <btmeehan@gmail.com> wrote:
> >
> > > I have a big-ish blob of key-value pairs that I would like to not have
> to
> > > store as text and then convert to a hashtable when I am filtering my
> data.
> > >
> > > Right now I am saving it as an alist and then converting back to a
> > > hashtable using "alist->hashtable" from SRFI 69. I was just hoping that
> > > there was something more clever ... perhaps it is plenty clever enough.
> > >
> >
>
>

Re: is there a way to serialize hash tables?

[Linus Björnstam]

I have added it to my phone's dictionary, but for some reason it still autocorrects to Alex. Sorry about that. I know very well that it is Aleix.

I remember I had the same problem last time.

-- 
  Linus Björnstam

On Wed, 30 Dec 2020, at 00:18, Aleix Conchillo Flaqué wrote:
> You almost got it right Linus, it's Aleix. :-)
> 
> For the curious, it sounds something like "Alaish" or here in the US 
> most people can pronounce it as "Alesh". But more accurately [aleʃ] 
> (https://en.m.wikipedia.org/wiki/Help:IPA/Catalan)
> 
> Anyways, happy hacking and happy New Year to everyone!
> 
> Aleix
> 
> On Sat, Dec 26, 2020, 4:10 AM Linus Björnstam 
> <linus.internet@fastmail.se> wrote:
> > Alex and I got the speed of guile-json up to about 19megs a second under guile 3 on a 4 year old intel i5, which may or may not be adequate for your needs, Tim.
> > 
> > I have some code I wrote during that spurt that preforms slightly better than guile-json in general (about 3-4% on my machine. Alex could not repeat those numbers) an on files with many large strings in particular (10% in extreme cases).  If that is interesting I could put it online somewhere. The functioning code is a derivate work of guile-json, so the GPL applies.
> > 
> > -- 
> >   Linus Björnstam
> > 
> > On Sat, 26 Dec 2020, at 05:28, John Cowan wrote:
> > > I think that's the best bet, although JSON is also a possibility.  I'm
> > > working on a more general text-serialization solution, but it will be quite
> > > some time before I have a chance to work on it.
> > > 
> > > On Fri, Dec 25, 2020 at 10:28 PM Tim Meehan <btmeehan@gmail.com> wrote:
> > > 
> > > > I have a big-ish blob of key-value pairs that I would like to not have to
> > > > store as text and then convert to a hashtable when I am filtering my data.
> > > >
> > > > Right now I am saving it as an alist and then converting back to a
> > > > hashtable using "alist->hashtable" from SRFI 69. I was just hoping that
> > > > there was something more clever ... perhaps it is plenty clever enough.
> > > >
> > >
> >

Re: is there a way to serialize hash tables?

[Aleix Conchillo Flaqué]

No worries at all :-). I know you know. I just took the opportunity to
explain how it is pronounced, I'm aware most people don't know.


Best,

Aleix

On Wed, Dec 30, 2020, 12:48 AM Linus Björnstam <linus.internet@fastmail.se>
wrote:

> I have added it to my phone's dictionary, but for some reason it still
> autocorrects to Alex. Sorry about that. I know very well that it is Aleix.
>
> I remember I had the same problem last time.
>
> --
>   Linus Björnstam
>
> On Wed, 30 Dec 2020, at 00:18, Aleix Conchillo Flaqué wrote:
> > You almost got it right Linus, it's Aleix. :-)
> >
> > For the curious, it sounds something like "Alaish" or here in the US
> > most people can pronounce it as "Alesh". But more accurately [aleʃ]
> > (https://en.m.wikipedia.org/wiki/Help:IPA/Catalan)
> >
> > Anyways, happy hacking and happy New Year to everyone!
> >
> > Aleix
> >
> > On Sat, Dec 26, 2020, 4:10 AM Linus Björnstam
> > <linus.internet@fastmail.se> wrote:
> > > Alex and I got the speed of guile-json up to about 19megs a second
> under guile 3 on a 4 year old intel i5, which may or may not be adequate
> for your needs, Tim.
> > >
> > > I have some code I wrote during that spurt that preforms slightly
> better than guile-json in general (about 3-4% on my machine. Alex could not
> repeat those numbers) an on files with many large strings in particular
> (10% in extreme cases).  If that is interesting I could put it online
> somewhere. The functioning code is a derivate work of guile-json, so the
> GPL applies.
> > >
> > > --
> > >   Linus Björnstam
> > >
> > > On Sat, 26 Dec 2020, at 05:28, John Cowan wrote:
> > > > I think that's the best bet, although JSON is also a possibility.
> I'm
> > > > working on a more general text-serialization solution, but it will
> be quite
> > > > some time before I have a chance to work on it.
> > > >
> > > > On Fri, Dec 25, 2020 at 10:28 PM Tim Meehan <btmeehan@gmail.com>
> wrote:
> > > >
> > > > > I have a big-ish blob of key-value pairs that I would like to not
> have to
> > > > > store as text and then convert to a hashtable when I am filtering
> my data.
> > > > >
> > > > > Right now I am saving it as an alist and then converting back to a
> > > > > hashtable using "alist->hashtable" from SRFI 69. I was just hoping
> that
> > > > > there was something more clever ... perhaps it is plenty clever
> enough.
> > > > >
> > > >
> > >
>

Re: is there a way to serialize hash tables?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 757 bytes --]

On Fri, Dec 25, 2020 at 09:27:49PM -0600, Tim Meehan wrote:
> I have a big-ish blob of key-value pairs that I would like to not have to
> store as text and then convert to a hashtable when I am filtering my data.
> 
> Right now I am saving it as an alist and then converting back to a
> hashtable using "alist->hashtable" from SRFI 69. I was just hoping that
> there was something more clever ... perhaps it is plenty clever enough.

If that "dumping/restoring" seems too expensive (have you measured it?),
then another alternative might be to use "external" dictionaries, like
guile-dbm [1] or guile-wiredtiger [2] 

Cheers

[1] https://www.gnuvola.org/software/guile-gdbm/
[2] https://guix.gnu.org/packages/guile-wiredtiger-0.7.0/

 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]