guile-json 0.2.0 released

guile-json 0.2.0 released

[Aleix Conchillo Flaqué]

Hi,

I am pleased to announce guile-json 0.2.0. No big changes but it comes
with better parser error information. Row and column number can now be
obtained if a "json-invalid" exception is thrown, which is pretty
convenient.

http://savannah.nongnu.org/projects/guile-json

Happy hacking!

Aleix

Re: guile-json 0.2.0 released

[Panicz Maciej Godek]

[-- Attachment #1: Type: text/plain, Size: 2524 bytes --]

Hi!
I'm glad that you've made a great piece of work to integrate guile with the
rest of the world. I see you implemed it using guile's hash tables and
lists, which seems sound.
The problem lies within the way guile deals with hash tables. Actually,
there are several problems that need to be addressed:

- firstly, guile's hash tables are maps, not dictionaries, so they are
insensitive to order. This behaviour is desired if we want to use them to
represent sets or maps; however, PHP arrays, and -- as I presume --
JavaScript objects -- store the information about the order of their
elements. Lisp-style sollution would be to store them as assoc lists, which
in turn have linear access time.

- secondly, there is no default notation to create hash tables nor sets;
using them forces
a programmer to drop homoiconicity, as their default print representation
is #<hash-table 1c8a940 1/31> or something even uglier. I think that this
is done properly in Clojure.

- thirdly, refering to hashes (as well as assoc lists, goops' object slots,
vector/array elements) is truly cumbersome. I once wrote a
hash-read-extension that allowed to refer to hashes/arrays/slots... using
uniform notation #[object key], and to allow for nested references like #[
... #[#[object key1] key2 ] ... ] using simpified notation #[object : key1
: key2 : ... ]. The implementation is rather inefficient when it comes to
performance, but makes coding much more efficient, and it can be found
here, if anyone's interested:
https://bitbucket.org/panicz/slayer/src/33cf0116da95dea6928ab1011994569b5a5181ad/extra/ref.scm?at=goose-3d
One could ask: why can't vectors, arrays, objects and hashes simply be
applicable? (Of course, it is possible to implement applicable collections
even now, but at a cost of loosing their print representation)

I think the issue with applicable goops objects emerged before, when
someone wanted to implement python in guile, and wanted to provide a
__call__ metod.

- lastly, guile's support for hash tables is limited -- there ain't even a
built-in function that would return the size of a hash-table. My
implementation is inefficient (linear time), and it looks like this:
(define (hash-size hash-map)
  (length (hash-values hash-map)))

Some time ago I saw here someone who was using the print representation of
a hash-table to read the number of its elements, but it seems like a nasty
hack, and it stopped working once the print representation changed.

Sorry if I was a little off topic :)
Be the avant-garde!
M.

[-- Attachment #2: Type: text/html, Size: 3147 bytes --]

Re: guile-json 0.2.0 released

[Taylan Ulrich B.]

Panicz Maciej Godek <godek.maciek@gmail.com> writes:

> - firstly, guile's hash tables are maps, not dictionaries, so they are
> insensitive to order. This behaviour is desired if we want to use them
> to represent sets or maps; however, PHP arrays, and -- as I presume --
> JavaScript objects -- store the information about the order of their
> elements. Lisp-style sollution would be to store them as assoc lists,
> which in turn have linear access time.

From json.org (which is official):
"An object is an unordered set of name/value pairs."

(Object means dictionary/map in JSON terminology.)

I think this is consistent with common expectations from a dictionary
data structure.  At least in my experience.  Should the ordering be
important, a supplementary key-vector (or list) could be used easily
enough in my opinion; bundling that functionality into hash-tables is
probably not worth it unless it is trivial to implement.

> - secondly, there is no default notation to create hash tables nor
> sets; using them forces
> a programmer to drop homoiconicity, as their default print
> representation is #<hash-table 1c8a940 1/31> or something even uglier.
> I think that this is done properly in Clojure.

That is not what homoiconicity means.  There are more data types that
lack a proper external representation; most notably procedures.  For
transmission of data, converting to an alist and back is probably good
enough; this can also be used as a "hack" for having "literal"
dictionaries in code: (alist->dictionary '(...))

So again, this is probably nothing that needs be implemented urgently.

> - thirdly, refering to hashes (as well as assoc lists, goops' object
> slots, vector/array elements) is truly cumbersome. I once wrote a
> hash-read-extension that allowed to refer to hashes/arrays/slots...
> using uniform notation #[object key], and to allow for nested
> references like #[ ... #[#[object key1] key2 ] ... ] using simpified
> notation #[object : key1 : key2 : ... ]. The implementation is rather
> inefficient when it comes to performance, but makes coding much more
> efficient, and it can be found here, if anyone's interested:
> https://bitbucket.org/panicz/slayer/src/33cf0116da95dea6928ab1011994569b5a5181ad/extra/ref.
> scm?at=goose-3d
> One could ask: why can't vectors, arrays, objects and hashes simply be
> applicable? (Of course, it is possible to implement applicable
> collections even now, but at a cost of loosing their print
> representation)

SRFI-105 is probably the preferable solution to this problem, since it's
an SRFI.  Guile already supports it, but I don't know how many accessors
are built-in; it should be trivial to implement any you want though.

> - lastly, guile's support for hash tables is limited -- there ain't
> even a built-in function that would return the size of a hash-table.
> My implementation is inefficient (linear time), and it looks like
> this:
> (define (hash-size hash-map)
> (length (hash-values hash-map)))

I don't know how exactly hash-tables are implemented in Guile, but one
probably needs to walk through the whole structure to count the size;
then the most efficient simple implementation of `hash-size' is one
which walks through it only once:

(hash-fold (lambda (key value size) (1+ size)) 0 hash-table)

Other than that, the size could be kept in the internal representation
of the hash-table, but I'm not sure of the pay-offs.

Kind regards,
Taylan :)

Re: guile-json 0.2.0 released

[Panicz Maciej Godek]

[-- Attachment #1: Type: text/plain, Size: 6279 bytes --]

2013/4/4 Taylan Ulrich B. <taylanbayirli@gmail.com>

> Panicz Maciej Godek <godek.maciek@gmail.com> writes:
>
> > - firstly, guile's hash tables are maps, not dictionaries, so they are
> > insensitive to order. This behaviour is desired if we want to use them
> > to represent sets or maps; however, PHP arrays, and -- as I presume --
> > JavaScript objects -- store the information about the order of their
> > elements. Lisp-style sollution would be to store them as assoc lists,
> > which in turn have linear access time.
>
> From json.org (which is official):
> "An object is an unordered set of name/value pairs."
>
> (Object means dictionary/map in JSON terminology.)
>
> I think this is consistent with common expectations from a dictionary
> data structure.  At least in my experience.  Should the ordering be
> important, a supplementary key-vector (or list) could be used easily
> enough in my opinion; bundling that functionality into hash-tables is
> probably not worth it unless it is trivial to implement.


Well, I see why the representation based on hash tables is adequate for
JSON. There are, however, situations, when one wants to have an ordered
set, and it's good to have choice. Clojure, for instance, offers such
choice, and from the perspective of a programmer it's better to have a
choice.


> > - secondly, there is no default notation to create hash tables nor
> > sets; using them forces
> > a programmer to drop homoiconicity, as their default print
> > representation is #<hash-table 1c8a940 1/31> or something even uglier.
> > I think that this is done properly in Clojure.
>
> That is not what homoiconicity means.  There are more data types that
> lack a proper external representation; most notably procedures.  For
> transmission of data, converting to an alist and back is probably good
> enough; this can also be used as a "hack" for having "literal"
> dictionaries in code: (alist->dictionary '(...))
>
> Of course it can. However it's not convenient. I use emacs+geiser and when
I want to see the content of a variable -- if it's a list or some other
basic type -- I just point a cursor on it and I get the value in a
minibuffer. When I want to see the content of hash-table, I need to
explicitly evaluate (hash-map->list cons my-hash-table), which seems
unnecessary When a hash-table is nested, it turns into a nightmare. If
there was a more reader-friendly representation of hashes, it would be far
more convenient. I could come up with some representation myself, and use
it in my programs, but I guess that this problem is more serious and
requires more attention.

All in all, you don't write vector->list and list->vector to get a nice
printable representation of vectors -- there was an issue, and it has been
solved. Racket has its printable representation of hashes, which is much
nicer than the one of guile (although still not perfect).

So again, this is probably nothing that needs be implemented urgently.


Judging by the speed at which subsequent Reports on algorithmic language
Scheme are released, schemers don't seem know the word "urgent" :) Which is
good.
However, I think it is an important matter. I also think that it is no good
for scheme implementations to diverge from one another, if there is no good
reason for that.

Note that easy-to-use hash tables are what win the market for PHP, despite
many drawbacks of that language.

> - thirdly, refering to hashes (as well as assoc lists, goops' object
> > slots, vector/array elements) is truly cumbersome. I once wrote a
> > hash-read-extension that allowed to refer to hashes/arrays/slots...
> > using uniform notation #[object key], and to allow for nested
> > references like #[ ... #[#[object key1] key2 ] ... ] using simpified
> > notation #[object : key1 : key2 : ... ]. The implementation is rather
> > inefficient when it comes to performance, but makes coding much more
> > efficient, and it can be found here, if anyone's interested:
> >
> https://bitbucket.org/panicz/slayer/src/33cf0116da95dea6928ab1011994569b5a5181ad/extra/ref
> .
> > scm?at=goose-3d
> > One could ask: why can't vectors, arrays, objects and hashes simply be
> > applicable? (Of course, it is possible to implement applicable
> > collections even now, but at a cost of loosing their print
> > representation)
>
> SRFI-105 is probably the preferable solution to this problem, since it's
> an SRFI.  Guile already supports it, but I don't know how many accessors
> are built-in; it should be trivial to implement any you want though.
>
> I know of no other implementation of Scheme than Guile which supports
SRFI-105. And I guess that the implementators will remain reluctant with
regard to it, as it introduces more randomness to the way the code can be
written.
On the other hand, what are the argumets against making hash-tables,
vectors et al. applicable, assuming that "programming languages should be
designed not by piling feature on top of feature, but by removing the
weaknesses and restrictions that make additional features appear necessary"?

> - lastly, guile's support for hash tables is limited -- there ain't
> > even a built-in function that would return the size of a hash-table.
> > My implementation is inefficient (linear time), and it looks like
> > this:
> > (define (hash-size hash-map)
> > (length (hash-values hash-map)))
>
> I don't know how exactly hash-tables are implemented in Guile, but one
> probably needs to walk through the whole structure to count the size;
> then the most efficient simple implementation of `hash-size' is one
> which walks through it only once:
>
> (hash-fold (lambda (key value size) (1+ size)) 0 hash-table)
>
> Other than that, the size could be kept in the internal representation
> of the hash-table, but I'm not sure of the pay-offs.
>
> Plainly, the size is kept in the internal representation of the hash table:

typedef struct scm_t_hashtable {
unsigned long n_items;        /* number of items in table */
...

cf.
http://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=libguile/hashtab.h;h=82ed22e66eb1f5045793cfc55cca0be040d4aab1;hb=HEAD#l66

It would be really cheap&easy to get it from there. I just wanted to show
that hash-tables are neglected in Scheme in general, and in Guile in
particular.

Best regards!
M.

[-- Attachment #2: Type: text/html, Size: 9215 bytes --]

Re: guile-json 0.2.0 released

[Nala Ginrut]

If not so, I've sent a patch to get hash-size and hash-items directly.
Maybe it's still useful.

Re: guile-json 0.2.0 released

[Taylan Ulrich B.]

Panicz Maciej Godek <godek.maciek@gmail.com> writes:

> Well, I see why the representation based on hash tables is adequate
> for JSON. There are, however, situations, when one wants to have an
> ordered set, and it's good to have choice. Clojure, for instance,
> offers such choice, and from the perspective of a programmer it's
> better to have a choice.
>
> [...snip...]
>
> Of course it can. However it's not convenient. I use emacs+geiser and
> when I want to see the content of a variable -- if it's a list or some
> other basic type -- I just point a cursor on it and I get the value in
> a minibuffer. When I want to see the content of hash-table, I need to
> explicitly evaluate (hash-map->list cons my-hash-table), which seems
> unnecessary When a hash-table is nested, it turns into a nightmare. If
> there was a more reader-friendly representation of hashes, it would be
> far more convenient. I could come up with some representation myself,
> and use it in my programs, but I guess that this problem is more
> serious and requires more attention.
>
> All in all, you don't write vector->list and list->vector to get a
> nice printable representation of vectors -- there was an issue, and it
> has been solved. Racket has its printable representation of hashes,
> which is much nicer than the one of guile (although still not
> perfect).

How important these matters are is very subjective I guess.  To be
honest, I don't have a strong opinion at all, but was writing what I
believe would be the stance of most Guile developers.  If you believe
these features to be important for the larger Guile user-base, you might
want to bring that to the attention of developers (perhaps already
achieved with these mails), or send patches. :)

> Judging by the speed at which subsequent Reports on algorithmic
> language Scheme are released, schemers don't seem know the word
> "urgent" :) Which is good.
> However, I think it is an important matter. I also think that it is no
> good for scheme implementations to diverge from one another, if there
> is no good reason for that.

Yet you seem to be recommending highly non-standard features.  Or did I
misinterpret something?

> Note that easy-to-use hash tables are what win the market for PHP,
> despite many drawbacks of that language.

What I know about PHP associative-arrays is that they are created by
using the pseudo-function array(), with an alternative syntax, and are
in many ways indistinguishable from normal arrays, which I seem to
remember had bit me already one of the few times I had to use PHP.  As
someone with a generic sense for consistency and orthogonality, I found
all that very distasteful.  If PHP wins "the market" with such an
implementation, then there's probably something wrong with the market.
(There is, indeed, I believe.)

> I know of no other implementation of Scheme than Guile which supports
> SRFI-105. And I guess that the implementators will remain reluctant
> with regard to it, as it introduces more randomness to the way the
> code can be written.

It's a relatively young SRFI.  It's very well-thought-out though in my
opinion, and offers a *generic* solution to the kind of problem you
described.  It doesn't introduce "randomness," it introduces
well-defined extensions to the reader-syntax.  I'd urge you to read it
if you haven't already.

> On the other hand, what are the argumets against making hash-tables,
> vectors et al. applicable, assuming that "programming languages should
> be designed not by piling feature on top of feature, but by removing
> the weaknesses and restrictions that make additional features appear
> necessary"?

I don't have any arguments against that; in fact I just learned what it
means for an object to be applicable.  Seems like a nice idea on the
surface, and I can't think of any immediate draw-backs.  I'm not sure if
it might be confusing to have more than two types of applicable objects
(syntax-keywords and procedures), and I could probably only answer that
question for myself after having written some code using that feature.
Would be interested in hearing other people's opinion on this...

> Plainly, the size is kept in the internal representation of the hash
> table:
>
> typedef struct scm_t_hashtable {
> unsigned long n_items; /* number of items in table */
> ...
>
> cf.
> http://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=libguile/hashtab.
> h;h=82ed22e66eb1f5045793cfc55cca0be040d4aab1;hb=HEAD#l66
>
> It would be really cheap&easy to get it from there. I just wanted to
> show that hash-tables are neglected in Scheme in general, and in Guile
> in particular.

That sounds nasty indeed.  It's late for me today; if no one's quicker
than me, tomorrow I might see that I implement hash-size in C as my
first Guile-contribution. :P

Regards,
Taylan

Re: guile-json 0.2.0 released

[Aleix Conchillo Flaqué]

Thank you both for the comments.

I must admit that I got kind of lost and I ended up not knowing if you
had any specific suggestions for guile-json. It is my very first guile
(scheme) package and I am pretty new to scheme.

From what I understood, the main concerns were:

1. Hash tables might not be a proper way to represent JSON objects.

2. Syntax for accessing JSON objects.

For 1, as Taylan mentioned, json.org clearly says that JSON objects
are unordered. So I thought a hash table was the right data structure
to use. I initially thought about using association lists, but that's
ordered and performance might be worst for large objects.

May be it would be better to have a json-object type and procedures to
access it (json-object-ref ...) and internally it could be a hash
table, or an alist. Then, maybe, the user could specify if he wants to
get ordered json-objects or not, and internally use hash tables or
alists.

For 2, yes, a better syntax would be ideal. I don't know about
SRFI-105, but I'll take a look into it.

Best,

Aleix

Re: guile-json 0.2.0 released

[Daniel Hartwig]

On 5 April 2013 06:21, Taylan Ulrich B. <taylanbayirli@gmail.com> wrote:
> Panicz Maciej Godek <godek.maciek@gmail.com> writes:
>
>> Plainly, the size is kept in the internal representation of the hash
>> table:
>>
>> typedef struct scm_t_hashtable {
>> unsigned long n_items; /* number of items in table */
>> ...
>>
>> cf.
>> http://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=libguile/hashtab.
>> h;h=82ed22e66eb1f5045793cfc55cca0be040d4aab1;hb=HEAD#l66
>>
>> It would be really cheap&easy to get it from there. I just wanted to
>> show that hash-tables are neglected in Scheme in general, and in Guile
>> in particular.
>
> That sounds nasty indeed.  It's late for me today; if no one's quicker
> than me, tomorrow I might see that I implement hash-size in C as my
> first Guile-contribution. :P
>

Note that exposing a constant-time “number of items” operation has been
proposed several times recently and remains not included.  Please read
the previous threads before proposing this yet again.

Regards

Re: guile-json 0.2.0 released

[Daniel Hartwig]

On 4 April 2013 20:06, Panicz Maciej Godek <godek.maciek@gmail.com> wrote:
> There are, however, situations, when one wants to have an ordered set,
> and it's good to have choice. Clojure, for instance, offers such choice, and
> from the perspective of a programmer it's better to have a choice.
>

Note that Scheme provides the essential building blocks to create such
an ordered map data type.  It is rather simple, at most a few dozen
lines.  Further, you can define the ordering semantics to precisely
suit your use case.  No doubt for every user of an ordered map they
have something else in mind, i.e. insertion-order, key-order, when to
perform the ordering (bulk insert followed by sort, or sort as
inserted).

One of the main distinguishing points of Scheme is that it does not
attempt to provide absolutely everything you could ever need ready
to use, but instead gives a powerful set of well-defined tools so
that you can construct solutions to any problem.

From the perspective of a Schemer, its better to have _tools_.  If you
want a language where absolutely every feature possible is living in
the box ready to go, then you have plenty of choices for those.


Rather than just adding feature atop feature, the objective is to
isolate the few core functions that truely add utility to the
programming environment, then provide those.


>>
>> > - secondly, there is no default notation to create hash tables nor
>> > sets; using them forces
>> > a programmer to drop homoiconicity, as their default print
>> > representation is #<hash-table 1c8a940 1/31> or something even uglier.
>> > I think that this is done properly in Clojure.
>>
>> That is not what homoiconicity means.  There are more data types that
>> lack a proper external representation; most notably procedures.  For
>> transmission of data, converting to an alist and back is probably good
>> enough; this can also be used as a "hack" for having "literal"
>> dictionaries in code: (alist->dictionary '(...))
>>

Hash tables are not just a set of (key, value) pairs, they also
include the particular hash and equality procedures that are used with
them.  These could be arbitrary procedures, and procedures can not
generally be converted to a string and back again, so, by extension,
neither can hash tables even if you could do that for their content.

It would be misleading to provide a write format that appears to be
read syntax.

> Of course it can. However it's not convenient. I use emacs+geiser and when I
> want to see the content of a variable -- if it's a list or some other basic
> type -- I just point a cursor on it and I get the value in a minibuffer.
> When I want to see the content of hash-table, I need to explicitly evaluate
> (hash-map->list cons my-hash-table), which seems unnecessary When a
> hash-table is nested, it turns into a nightmare.

So hook your tools to do that automatically when the value at point is
a hash table.  You can take on the arbitrary performance penalty.
Please, no patches to change geisers current behaviour.

> On the other hand, what are the argumets against making hash-tables, vectors
> et al. applicable, assuming that "programming languages should be designed
> not by piling feature on top of feature, but by removing the weaknesses and
> restrictions that make additional features appear necessary"?
>

Erm, your quote seems to argue against your own position?

Any applicable (“message passing”) interface is going to wrap
procedures that perform the lookup, update, etc..  Since those
procedures must be present anyway, it is those procedures provided by
the Guile API.  This is the more natural interface.

Again, you have the tools to build a message passing interface from
the tools provided.  It is a trivial task, but does not add value to
the set of tools provided.

>> > - lastly, guile's support for hash tables is limited -- there ain't
>> > even a built-in function that would return the size of a hash-table.
>> > My implementation is inefficient (linear time), and it looks like
>> > this:
>> > (define (hash-size hash-map)
>> > (length (hash-values hash-map)))
>>

Question: what _value_ does that information add to your algorithm?


Regards

Re: guile-json 0.2.0 released

[Noah Lavine]

[-- Attachment #1: Type: text/plain, Size: 1911 bytes --]

Hello,

I haven't thought about this whole email, but I disagree with this part of
your response:

On Thu, Apr 4, 2013 at 8:17 PM, Daniel Hartwig <mandyke@gmail.com> wrote:

> Hash tables are not just a set of (key, value) pairs, they also
> include the particular hash and equality procedures that are used with
> them.  These could be arbitrary procedures, and procedures can not
> generally be converted to a string and back again, so, by extension,
> neither can hash tables even if you could do that for their content.
>

Although hash tables in general do include arbitrary procedures, in Guile's
implementation there are only three to choose from, so it should be
possible to represent them in syntax.

For exactly this reason, I believe that actually "hash table" is a bad name
for the data structure. I think of Guile's hash tables as a generic
dictionary structure with average O(1)-time lookup, insertion and deletion.
It makes sense to me to give this structure an external representation,
because it has a lot of common uses. (It seems especially useful to me for
configuration files.)

That does privilege one sort of generic dictionary structure over other
implementations of the same interface - such as hash tables with different
hash procedures, as you mentioned. This is fine with me, because I think
that use cases fall into two categories. In the common case, the dictionary
will not be in a time- or performance-critical part of a program, and
programmers should use Guile's built-in dictionary types and not waste time
implementing their own. In the rare case, when dictionary lookups are time-
or space-critical and must be optimized, *then* it's worth it to design
custom hash functions and implement hash tables from vectors and similar
things. However, I think that case is sufficiently uncommon that it should
not hold back the common case from having convenient syntax.

Best,
Noah

[-- Attachment #2: Type: text/html, Size: 2371 bytes --]

Re: guile-json 0.2.0 released

[Panicz Maciej Godek]

[-- Attachment #1: Type: text/plain, Size: 1837 bytes --]

2013/4/5 Aleix Conchillo Flaqué <aconchillo@gmail.com>

> Thank you both for the comments.
>
> I must admit that I got kind of lost and I ended up not knowing if you
> had any specific suggestions for guile-json. It is my very first guile
> (scheme) package and I am pretty new to scheme.
>
> From what I understood, the main concerns were:
>
> 1. Hash tables might not be a proper way to represent JSON objects.
>
> 2. Syntax for accessing JSON objects.
>
> For 1, as Taylan mentioned, json.org clearly says that JSON objects
> are unordered. So I thought a hash table was the right data structure
> to use. I initially thought about using association lists, but that's
> ordered and performance might be worst for large objects.
>

Yes, so as it turns out, it was my ignorance that caused me to raise this
topic :)
Hash tables are fine, but I think that guile's support for them isn't
convinient, because their print representation isn't very informative
(which occured to me when I was trying to test how exactly guile-json works)


> May be it would be better to have a json-object type and procedures to
> access it (json-object-ref ...) and internally it could be a hash
> table, or an alist. Then, maybe, the user could specify if he wants to
> get ordered json-objects or not, and internally use hash tables or
> alists.
>

I think that, if the specification says that it's unordered, there's no
actual need to complicate things overly :)


> For 2, yes, a better syntax would be ideal. I don't know about
> SRFI-105, but I'll take a look into it.
>
> SRFI-105 offers only infix/m-exp notation within curly braces and a more
decent way to index arrays (using postfix brackets, as in most popular
programming languages)

Sorry if I caused unnecessary confusion ;o

Best regards
M.

[-- Attachment #2: Type: text/html, Size: 2761 bytes --]

Re: guile-json 0.2.0 released

[Daniel Hartwig]

[-- Attachment #1: Type: text/plain, Size: 1495 bytes --]

On 05/04/2013 10:47 AM, "Noah Lavine" <noah.b.lavine@gmail.com> wrote:
>
> Hello,
>
> I haven't thought about this whole email, but I disagree with this part
of your response:
>
> On Thu, Apr 4, 2013 at 8:17 PM, Daniel Hartwig <mandyke@gmail.com> wrote:
>>
>> Hash tables are not just a set of (key, value) pairs, they also
>> include the particular hash and equality procedures that are used with
>> them.  These could be arbitrary procedures, and procedures can not
>> generally be converted to a string and back again, so, by extension,
>> neither can hash tables even if you could do that for their content.
>
>
> Although hash tables in general do include arbitrary procedures, in
Guile's implementation there are only three to choose from, so it should be
possible to represent them in syntax.
>

I think you miss the hashx procedures.

> For exactly this reason, I believe that actually "hash table" is a bad
name for the data structure. I think of Guile's hash tables as a generic
dictionary structure with average O(1)-time lookup, insertion and deletion.

Where do you get your definition of 'hash table' that the guile type does
not apply?

> In the rare case, when dictionary lookups are time- or space-critical and
must be optimized, *then* it's worth it to design custom hash functions and
implement hash tables from vectors and similar things.

That's what the current data type is anyway, and can be used with custom
hash?  I'm not sure I follow the distinctions you are making.

[-- Attachment #2: Type: text/html, Size: 1850 bytes --]

Re: guile-json 0.2.0 released

[Panicz Maciej Godek]

[-- Attachment #1: Type: text/plain, Size: 8282 bytes --]

2013/4/5 Daniel Hartwig <mandyke@gmail.com>

> On 4 April 2013 20:06, Panicz Maciej Godek <godek.maciek@gmail.com> wrote:
> > There are, however, situations, when one wants to have an ordered set,
> > and it's good to have choice. Clojure, for instance, offers such choice,
> and
> > from the perspective of a programmer it's better to have a choice.
> >
>
> Note that Scheme provides the essential building blocks to create such
> an ordered map data type.  It is rather simple, at most a few dozen
> lines.  Further, you can define the ordering semantics to precisely
> suit your use case.  No doubt for every user of an ordered map they
> have something else in mind, i.e. insertion-order, key-order, when to
> perform the ordering (bulk insert followed by sort, or sort as
> inserted).
>
> One of the main distinguishing points of Scheme is that it does not
> attempt to provide absolutely everything you could ever need ready
> to use, but instead gives a powerful set of well-defined tools so
> that you can construct solutions to any problem.
>
>
I certainly wouldn't want Guile to provide *absolutely* everything; I
rather think that it is a matter of a proper weight shift. Guile's already
went away from the minimalistic spirit of Scheme, e.g. by introducing
keyword arguments in the core -- despite the fact that everyone could
implement their own keyword argument mechanism if they wanted to. However,
although this feature is unnecessary, it's good that it's done in an
efficient and uniform way, because it's very handy.


> From the perspective of a Schemer, its better to have _tools_.  If you
> want a language where absolutely every feature possible is living in
> the box ready to go, then you have plenty of choices for those.
>
>
No language provides absolutely every feature possible, and there'll never
be such a language, because features stem from applications, which are a
subject of human invention and are not known a priori.


> Rather than just adding feature atop feature, the objective is to
> isolate the few core functions that truely add utility to the
> programming environment, then provide those.


I certainly agree with you on this one. The question is about the nature of
that utility. Programming languages are not only about programming, but
also about communication, and that's why I think that the common things
should be done in a common way, instead of pushing programmers towards
idiosyncracies. This regards especially a language that aspires to be
ubiquitous.


> >> > - secondly, there is no default notation to create hash tables nor
> >> > sets; using them forces
> >> > a programmer to drop homoiconicity, as their default print
> >> > representation is #<hash-table 1c8a940 1/31> or something even uglier.
> >> > I think that this is done properly in Clojure.
> >>
> >> That is not what homoiconicity means.  There are more data types that
> >> lack a proper external representation; most notably procedures.  For
> >> transmission of data, converting to an alist and back is probably good
> >> enough; this can also be used as a "hack" for having "literal"
> >> dictionaries in code: (alist->dictionary '(...))
> >>
>
> Hash tables are not just a set of (key, value) pairs, they also
> include the particular hash and equality procedures that are used with
> them.  These could be arbitrary procedures, and procedures can not
> generally be converted to a string and back again, so, by extension,
> neither can hash tables even if you could do that for their content.
>
> It would be misleading to provide a write format that appears to be
> read syntax.


Again, it is a discussion between generality and commonness. For a
practical language, common tasks should be easy.

> Of course it can. However it's not convenient. I use emacs+geiser and
> when I
> > want to see the content of a variable -- if it's a list or some other
> basic
> > type -- I just point a cursor on it and I get the value in a minibuffer.
> > When I want to see the content of hash-table, I need to explicitly
> evaluate
> > (hash-map->list cons my-hash-table), which seems unnecessary When a
> > hash-table is nested, it turns into a nightmare.
>
> So hook your tools to do that automatically when the value at point is
> a hash table.  You can take on the arbitrary performance penalty.
> Please, no patches to change geisers current behaviour.


Well, geiser does a quite simple job -- it displays the print
representation of an evaluated expression in the minibuffer. And it would
be enough to change that representation in guile to make it work. It works
easily with geiser+racket, where one can write, for instance
(hash-ref #hash((a . 1)(b . 2)) 'a) ===> 1
(I'm not saying that I like that syntax/representation, but it's definitely
more convinient than in guile)

> On the other hand, what are the argumets against making hash-tables,
> vectors
> > et al. applicable, assuming that "programming languages should be
> designed
> > not by piling feature on top of feature, but by removing the weaknesses
> and
> > restrictions that make additional features appear necessary"?
> >
>
> Erm, your quote seems to argue against your own position?
>
>
Maybe it does to you, but to me it's not so obvious.
The fact that various objects can't be applicable can be perceived as a
weakness and restriction, because it decreases the number of acceptable
forms.
Especially when we confront it with GOOPS. Why, for instance, can't we write
(define-generic (call (v <vector>) (i <integer>)) (vector-ref v i))
and then use it like that: (#(0 1 2) 1) ===> 1
That would be a general mechanism, which would increase the number of ways
in which the language can be used.

Any applicable (“message passing”) interface is going to wrap
> procedures that perform the lookup, update, etc..  Since those
> procedures must be present anyway, it is those procedures provided by
> the Guile API.  This is the more natural interface.
>
> Again, you have the tools to build a message passing interface from
> the tools provided.  It is a trivial task, but does not add value to
> the set of tools provided.


I don't think that's entirely true. I could wrap, for instance, a vector in
a closure, but then I loose the advantage of its print representation and
read syntax. Unless you know how to achieve it without loosing those
advantages.

>> > - lastly, guile's support for hash tables is limited -- there ain't
> >> > even a built-in function that would return the size of a hash-table.
> >> > My implementation is inefficient (linear time), and it looks like
> >> > this:
> >> > (define (hash-size hash-map)
> >> > (length (hash-values hash-map)))
> >>
>
> Question: what _value_ does that information add to your algorithm?
>
>
I'm not sure if I get your question right. Are you asking why did I need to
implement the hash-size procedure?
I was implementing a client/server protocol. The packages were s-exps sent
over UDP sockets, so I could parse them with plain read. However, the size
of UDP packages was limited, so I needed a mechanism that would allow me to
split one set of data into several packages.
First, an initial package was sent that looked like this:
`(begin-transaction ,id ,count)
where id is specific for a set of packages. Then, subsequent packages were
sent that looked like this
`(transaction ,id ,n ,data),
where n was the ordinal number of the package.
I made no assumptions about the order in which the packages arrive, and it
could happen, that the initial package arrived after some of the data
packages -- therefore the number of packages in a transaction could be
known after first data package was sent. That's why I couldn't use a
vector. Of course, I could synchronize with the initial package, and then
use vector, but then I'd need an additional variable to know how many
packages were received. I decidede to use a hash also because it knows how
many elements it contains, and it would work easily even for duplicating
packages. It turned out, however, that getting that number is non-trivial.
I don't know if that answers your question.

Best regards,
M.

[-- Attachment #2: Type: text/html, Size: 10793 bytes --]

Re: guile-json 0.2.0 released

[Noah Lavine]

[-- Attachment #1: Type: text/plain, Size: 2475 bytes --]

Hello,

On Fri, Apr 5, 2013 at 5:35 AM, Daniel Hartwig <mandyke@gmail.com> wrote:

>
> On 05/04/2013 10:47 AM, "Noah Lavine" <noah.b.lavine@gmail.com> wrote:
> > Although hash tables in general do include arbitrary procedures, in
> Guile's implementation there are only three to choose from, so it should be
> possible to represent them in syntax.
> >
> I think you miss the hashx procedures.
>

Yes, you're right. My mistake.

>  > For exactly this reason, I believe that actually "hash table" is a bad
> name for the data structure. I think of Guile's hash tables as a generic
> dictionary structure with average O(1)-time lookup, insertion and deletion.
>
> Where do you get your definition of 'hash table' that the guile type does
> not apply?
>
That was a bad way to put it. I think of the Guile type as defined by an
interface, not a particular implementation, so I think "dictionary" is
better than "hash table". But it certainly is a hash table on the inside.

> > In the rare case, when dictionary lookups are time- or space-critical
> and must be optimized, *then* it's worth it to design custom hash functions
> and implement hash tables from vectors and similar things.
>
> That's what the current data type is anyway, and can be used with custom
> hash?  I'm not sure I follow the distinctions you are making.
>
I may have explained it poorly last time. I agree that in general, there
are many different variants on a hash table, and as you say, they cannot
all be written down easily - you can have different hash procedures,
different ways to handle collisions, and different policies on when to grow
and shrink the table (and maybe more things). However, I think that it's
worth having a printed representation for one particular sort of hash
table, even though that privileges one sort of hash table over another,
because it makes things very convenient in the common case where you don't
care about performance enough to customize your hash table. In the
performance-critical case, where you are customizing the features of your
hash, you can write a custom serializer too.

Basically, I think that while it is impossible to serialize all of the
things meant by "hash table", it is worthwhile to have a syntax form that
means "I want to store these objects in a hash-like dictionary structure,
and I don't need to worry too much about performance." I think the gain in
convenience is large, and the case where you can't use this is pretty rare.

Best,
Noah

[-- Attachment #2: Type: text/html, Size: 3418 bytes --]

Re: guile-json 0.2.0 released

[Panicz Maciej Godek]

[-- Attachment #1: Type: text/plain, Size: 7449 bytes --]

2013/4/5 Taylan Ulrich B. <taylanbayirli@gmail.com>


> > All in all, you don't write vector->list and list->vector to get a
>  > nice printable representation of vectors -- there was an issue, and it
> > has been solved. Racket has its printable representation of hashes,
> > which is much nicer than the one of guile (although still not
> > perfect).
>
> How important these matters are is very subjective I guess.  To be
> honest, I don't have a strong opinion at all, but was writing what I
> believe would be the stance of most Guile developers.  If you believe
> these features to be important for the larger Guile user-base, you might
> want to bring that to the attention of developers (perhaps already
> achieved with these mails), or send patches. :)


Well, actually I was hoping to bring attention of developers :)
After the discussion I agree that ordered hashes aren't so important, but
the printable representation and read syntax for hashes (or at least some
sort of hashes) and unordered sets would be comfortable and I believe it
wouldn't hurt (although Daniel Hartwig in one of this posts' follow-ups
claims that it could be misleading, but I'm not sure if we understood each
other entirely)

> Judging by the speed at which subsequent Reports on algorithmic
> > language Scheme are released, schemers don't seem know the word
> > "urgent" :) Which is good.
> > However, I think it is an important matter. I also think that it is no
> > good for scheme implementations to diverge from one another, if there
> > is no good reason for that.
>
> Yet you seem to be recommending highly non-standard features.  Or did I
> misinterpret something?


Well, I don't know how you interpreted that :) If it comes to applicable
objects, then I think that it should remain guile-specific, but it would
harmonize with the spirit of GOOPS. If it comes to syntax/representation of
hash tables, then it is non-standard, but mainly because it's not
standardized -- SRFI-69, which defines hash tables, says nothing about it's
display format, so the implementations are free to choose whichever format
they like. And if it is so, why shouldn't it be a useful and informative
format? And why couldn't it be made in such a way that it could become a
read-syntax at once?

> Note that easy-to-use hash tables are what win the market for PHP,
> > despite many drawbacks of that language.
>
> What I know about PHP associative-arrays is that they are created by
> using the pseudo-function array(), with an alternative syntax, and are
> in many ways indistinguishable from normal arrays, which I seem to
> remember had bit me already one of the few times I had to use PHP.  As
> someone with a generic sense for consistency and orthogonality, I found
> all that very distasteful.  If PHP wins "the market" with such an
> implementation, then there's probably something wrong with the market.
> (There is, indeed, I believe.)


I agree. But Adam Smith would say that it's the market who says what's
right and what's wrong ;)
And when it comes to PHP, it's true that it's an extremely incoherent and
clumsy language, but fortunately it's semantics diverges towards that of
lexically scoped lisps, so php 5.3 allows to create closures. And if it
comes to collections, normal arrays are mixed with associative arrays,
which is sound in a way, because they share a common interface. And they
are way simpler than, for instance, in perl, although (or because) perl
forces a programmer to know what he's doing.

> I know of no other implementation of Scheme than Guile which supports
> > SRFI-105. And I guess that the implementators will remain reluctant
> > with regard to it, as it introduces more randomness to the way the
> > code can be written.
>
> It's a relatively young SRFI.  It's very well-thought-out though in my
> opinion, and offers a *generic* solution to the kind of problem you
> described.  It doesn't introduce "randomness," it introduces
> well-defined extensions to the reader-syntax.  I'd urge you to read it
> if you haven't already.
>
>
I've read about it even before it became an SRFI draft, at David Wheeler's
website, so I know the idea quite well. When I say about randomness, I'm
not saying that it's not well-defined, but that it's conceptually or
cognitively incoherent. First, there's the idea of having an infix
notation. In a way I can understand it, because we're all educated to write
"two plus two" instead of "add two to two". But once the infix notation is
available, am I obliged to write {2 + 2}, or can I mix it with (+ 2 2)
freely? That's the unnecessary choice that I need to make. But I can
swallow that. However, since the infix notation is available, how will a
common infix expression like {2 * 3 + 4 / 5} behave? As expected? Nope. And
inside curly braces, do round parentheses behave as precedence operators,
or as they behave outside curly braces? And what about curly braces? Let's
look at some examples:
30. {(map - ns)} => (map - ns)
31. {map(- ns)} => (map - ns) as well
Why on earth would anyone want to write {(map - ns)} instead of (map - ns)?
And why is it cooler to write {f{x}} or {f(x)} instead of (f x)? Because it
somehow resembles the notation used by mathematicians? So it's because of
an idol! And why {f{- x}} maps to (f (- x)), while {f (- x)} maps to (f -
x), and not the other way around? And why do I have to remember that?!
I could go on like that. As you can see, a lot of arbitrary choices were
made in this SRFI, which were justified in this way or the other. And I
think that ony a few persons in the world will ever know all of those
justifications. For the remaining ones, reading code that uses "curly
infix" extensively will be a nightmare. Besides, it's so un-scheme if there
are a few people who know better than you how you are going to use the
language -- it would be sufficient if the reader transformed {...} into
(curly ...) or something like that, because then the programmer could
utilize the curly braces any way he wanted to using plain scheme macros,
whether it be an infix notation (either with or without precedence) or list
comprehensions, or lazy evaluation, or function composition, or
currying/binding, or indexing, or whatever.

> On the other hand, what are the argumets against making hash-tables,
> > vectors et al. applicable, assuming that "programming languages should
> > be designed not by piling feature on top of feature, but by removing
> > the weaknesses and restrictions that make additional features appear
> > necessary"?
>
> I don't have any arguments against that; in fact I just learned what it
> means for an object to be applicable.  Seems like a nice idea on the
> surface, and I can't think of any immediate draw-backs.  I'm not sure if
> it might be confusing to have more than two types of applicable objects
> (syntax-keywords and procedures), and I could probably only answer that
> question for myself after having written some code using that feature.
> Would be interested in hearing other people's opinion on this...


Me too, and that's why I posted this suggestion ;)
From David Wheeler's perspective, the advantage would be that, if he wished
to, he could make numbers also applicable, and have an infix notation in
round parentheses, if that's so important to him. From my perspective, it
would inrease the number of possible usages of langugage, and -- in
particular -- could help with array/hash indexing.

Best regards!
M.

[-- Attachment #2: Type: text/html, Size: 10235 bytes --]

Re: guile-json 0.2.0 released

[Taylan Ulrich B.]

Panicz Maciej Godek <godek.maciek@gmail.com> writes:
> I agree. But Adam Smith would say that it's the market who says what's
> right and what's wrong ;)

I don't know Adam Smith, but I know that I disagree with this particular
quote, at least at face value.  I will continue to disregard adoption
rates while considering a programming language as a role-model.  The
reasons for that are way beyond the scope of this discussion I'm afraid.

> I've read about it even before it became an SRFI draft, at David
> Wheeler's website, so I know the idea quite well. When I say about
> randomness, I'm not saying that it's not well-defined, but that it's
> conceptually or cognitively incoherent. First, there's the idea of
> having an infix notation. In a way I can understand it, because we're
> all educated to write "two plus two" instead of "add two to two". But
> once the infix notation is available, am I obliged to write {2 + 2},
> or can I mix it with (+ 2 2) freely? That's the unnecessary choice
> that I need to make. But I can swallow that. However, since the infix
> notation is available, how will a common infix expression like {2 * 3
> + 4 / 5} behave? As expected? Nope. And inside curly braces, do round
> parentheses behave as precedence operators, or as they behave outside
> curly braces? And what about curly braces? Let's look at some
> examples:
> 30. {(map - ns)} ⇒ (map - ns)
> 31. {map(- ns)} ⇒ (map - ns) as well
> Why on earth would anyone want to write {(map - ns)} instead of (map -
> ns)? And why is it cooler to write {f{x}} or {f(x)} instead of (f x)?
> Because it somehow resembles the notation used by mathematicians? So
> it's because of an idol! And why {f{- x}} maps to (f (- x)), while {f
> (- x)} maps to (f - x), and not the other way around? And why do I
> have to remember that?!
> I could go on like that. As you can see, a lot of arbitrary choices
> were made in this SRFI, which were justified in this way or the other.
> And I think that ony a few persons in the world will ever know all of
> those justifications. For the remaining ones, reading code that uses
> "curly infix" extensively will be a nightmare. Besides, it's so
> un-scheme if there are a few people who know better than you how you
> are going to use the language -- it would be sufficient if the reader
> transformed {...} into (curly ...) or something like that, because
> then the programmer could utilize the curly braces any way he wanted
> to using plain scheme macros, whether it be an infix notation (either
> with or without precedence) or list comprehensions, or lazy
> evaluation, or function composition, or currying/binding, or indexing,
> or whatever.

I don't intend to discuss the usefulness of SRFI-105 in general, but it
offers a relatively clean solution to the problem you mentioned, so I
see no reason not to use it to solve that problem.  {object[key]} does
not look much worse than e.g. #[object key] (and {object[key][key]}
looks better than #[[object key] key] IMO), is already implemented in
Guile, and actually has a chance to become standard.

I would use SRFI-105 for now, and/or depending on how important it is to
me, would work on a good general interface for making applicable objects
in Guile.

Re: guile-json 0.2.0 released

[Daniel Hartwig]

On 8 April 2013 04:38, Taylan Ulrich B. <taylanbayirli@gmail.com> wrote:
> Panicz Maciej Godek <godek.maciek@gmail.com> writes:
>> I agree. But Adam Smith would say that it's the market who says what's
>> right and what's wrong ;)
>
> I don't know Adam Smith, but I know that I disagree with this particular
> quote, at least at face value.  I will continue to disregard adoption
> rates while considering a programming language as a role-model.

A very tidy statement.  Perhaps more applicable to Scheme than some
other languages.

> The
> reasons for that are way beyond the scope of this discussion I'm afraid.
>

Re: guile-json 0.2.0 released

[Daniel Hartwig]

On 7 April 2013 19:40, Panicz Maciej Godek <godek.maciek@gmail.com> wrote:
>
> 2013/4/5 Taylan Ulrich B. <taylanbayirli@gmail.com>
>
>
>>
>> > All in all, you don't write vector->list and list->vector to get a
>> > nice printable representation of vectors -- there was an issue, and it
>> > has been solved.

There was never an issue.  Vectors have a read syntax that any vector
can be unambiguously displayed as.  Hash tables do not share this
property.

>> > Racket has its printable representation of hashes,
>> > which is much nicer than the one of guile (although still not
>> > perfect).
>>
>> How important these matters are is very subjective I guess.  To be
>> honest, I don't have a strong opinion at all, but was writing what I
>> believe would be the stance of most Guile developers.  If you believe
>> these features to be important for the larger Guile user-base, you might
>> want to bring that to the attention of developers (perhaps already
>> achieved with these mails), or send patches. :)
>
>
> Well, actually I was hoping to bring attention of developers :)
> After the discussion I agree that ordered hashes aren't so important, but
> the printable representation and read syntax for hashes (or at least some
> sort of hashes) and unordered sets would be comfortable and I believe it
> wouldn't hurt (although Daniel Hartwig in one of this posts' follow-ups
> claims that it could be misleading, but I'm not sure if we understood each
> other entirely)

To clarify: it is misleading to have a format that will display the
contents of some hash tables, and not others.  This is, e.g. defining
a read syntax for a blessed type of hash-table.

Just which pair of equality and hash procedures constitutes the
“common case” for Scheme code?  In other languages, there is a well
established, primary equality test which the builtin map type is
defined for.  This is not the case for Scheme where any particular
application is equally likely to use ‘eq?’, ‘eqv?’, ‘equal?’, or
other.  It is highly application specific to the point of varying
within a single program, and guile rightly does not bless one
particular choice over the others.

Nothing prevents an application from making its choice and using e.g.
srfi-105 to provide a suitable read syntax.


As a final comment.  You mentioned writing configuring files using
hash tables.  This is quite common in some languages, though for lispy
programs I note they are more often configured using straight $lisp
source file or module.  See, e.g. emacs and tekuti.


Regards