guildhall status

guildhall status

[Andy Wingo]

Hi,

I wanted to bootstrap the guildhall stuff, so last week I started poking
at Andreas' excellent dorodango (http://www.nongnu.org/dorodango/).  I
forked his repo on gitorious, removed a bunch of compat stuff for other
implementations, imported dependencies into the archive, replaced the
http stuff with Guile's (web) foo, and wired up a standard build system.

The result is here:

  https://gitorious.org/~wingo/dorodango/guildhall

You can:

  git clone git://gitorious.org/~wingo/dorodango/guildhall.git

Then ./configure && make && make check.

You will only be able to succeed there if you have (web client), which I
have pushed to Guile's stable-2.0 branch.

Wherever you see "doro" in the docs, replace it with "guild hall".
Perhaps in the future we should drop the "hall", so "guild hall update"
-> "guild update" or something.  Anyway, a thought for another time.

So, the status:

  1) Builds.
  2) Passes make check.
  3) Can update the available list.
  4) Everything else is untested :-)

Next up:

  1) Check status of dorodango functionality.
  2) Fix things that don't work.
  3) Profit?
  4) Start thinking about hosting and accounts and UX and stuff.

I will see if I can get work to sponsor a server that we can use, and
see if we can get it aliased to guildhall.gnu.org -- unless someone else
would like to provide the server.  It would be nice to have root on that
server, FWIW.  It could be a VM.

As far as relation with dorodango goes, we should do our best to keep
the guildhall compatible with dorodango archives on the net.  We should
also try hard to share code, but that is secondary.  Farther along I
would like to rename (dorodango ...) in our source to (guildhall ...) or
something so that we don't conflict with upstream.  I would also like to
reduce the number of bundled dependencies, and for the ones that are
left, include them under the (guildhall ...) namespace, making them
effectively private.  That way you can also install dorodango on your
machine, if you wish, and also install the wak- packages, industria,
ocelotl, etc.

I would really love for someone to take up this project.  I can help
getting it to the minimally functional state.  Please let me know if you
are interested.

Regards,

Andy
-- 
http://wingolog.org/

Re: guildhall status

[dsmich]

---- Andy Wingo <wingo@pobox.com> wrote: 
> Hi,
> 
> I wanted to bootstrap the guildhall stuff, so last week I started poking
> at Andreas' excellent dorodango (http://www.nongnu.org/dorodango/).  I
> forked his repo on gitorious, removed a bunch of compat stuff for other
> implementations, imported dependencies into the archive, replaced the
> http stuff with Guile's (web) foo, and wired up a standard build system.
> 
> The result is here:
> 
>   https://gitorious.org/~wingo/dorodango/guildhall

Cool!

> You can:
> 
>   git clone git://gitorious.org/~wingo/dorodango/guildhall.git
> 
> Then ./configure && make && make check.
> 
> You will only be able to succeed there if you have (web client), which I
> have pushed to Guile's stable-2.0 branch.

A (web client) test is failing for me:

$ git describe
v2.0.2-13-g037a680

$ ./check-guile web-request.test
Testing /home/dsmith/src/guile/meta/guile ... web-request.test
with GUILE_LOAD_PATH=/home/dsmith/src/guile/test-suite
Running web-request.test
FAIL: web-request.test: example-1: (equal? (request-host (build-request (string->uri http://www.gnu.org/))) www.gnu.org)

Totals for this test run:
passes:                 7
failures:               1
unexpected passes:      0
expected failures:      0
unresolved test cases:  0
untested test cases:    0
unsupported test cases: 0
errors:                 0


The test passes with the included git diff.  Seems request-host returns a pair instead of a string, so just use the car.

-Dale



-Dale

Re: guildhall status

[dsmich]

[-- Attachment #1: Type: text/plain, Size: 225 bytes --]

---- dsmich@roadrunner.com wrote: 
 
> The test passes with the included git diff.  Seems request-host returns a pair instead of a string, so just use the car.
> 

Bah.  Looks like I hozed attaching it.  Another try.

-Dale


[-- Attachment #2: web-client.diff --]
[-- Type: text/x-patch, Size: 553 bytes --]

diff --git a/test-suite/tests/web-request.test b/test-suite/tests/web-request.test
index b1182d2..157075a 100644
--- a/test-suite/tests/web-request.test
+++ b/test-suite/tests/web-request.test
@@ -48,7 +48,7 @@ Accept-Language: en-gb, en;q=0.9\r
         (request? r)))
     
     (pass-if (equal?
-              (request-host (build-request (string->uri "http://www.gnu.org/")))
+              (car (request-host (build-request (string->uri "http://www.gnu.org/"))))
               "www.gnu.org"))
     
     (pass-if (equal? (request-method r) 'GET))

Re: guildhall status

[Andy Wingo]

On Fri 15 Jul 2011 21:10, <dsmich@roadrunner.com> writes:

> A (web client) test is failing for me:

Sorry about that!  A thinko.  I pushed a patch similar to yours.

Cheers,

Andy
-- 
http://wingolog.org/

Re: guildhall status

[Ludovic Courtès]

Hi!

Andy Wingo <wingo@pobox.com> skribis:

> So, the status:
>
>   1) Builds.
>   2) Passes make check.
>   3) Can update the available list.
>   4) Everything else is untested :-)

Sounds like great news!

> I will see if I can get work to sponsor a server that we can use, and
> see if we can get it aliased to guildhall.gnu.org -- unless someone else
> would like to provide the server.  It would be nice to have root on that
> server, FWIW.  It could be a VM.

I believe FSF could lend us a server or VM, which would thus be
“independent”, but we’d need to check.

> As far as relation with dorodango goes, we should do our best to keep
> the guildhall compatible with dorodango archives on the net.  We should
> also try hard to share code, but that is secondary.  Farther along I
> would like to rename (dorodango ...) in our source to (guildhall ...) or
> something so that we don't conflict with upstream.  I would also like to
> reduce the number of bundled dependencies, and for the ones that are
> left, include them under the (guildhall ...) namespace, making them
> effectively private.  That way you can also install dorodango on your
> machine, if you wish, and also install the wak- packages, industria,
> ocelotl, etc.

Sounds cool.

Since the project may become quite central, it would be nice if it could
be FSF-copyrighted.  That’s obviously something to discuss with Andreas,
but the sooner the better.

Thanks for the fast & efficient work!

Ludo’.

Re: guildhall status

[Andy Wingo]

Hi!

Thanks for the feedback!

On Mon 18 Jul 2011 15:06, ludo@gnu.org (Ludovic Courtès) writes:

>> I will see if I can get work to sponsor a server that we can use, and
>> see if we can get it aliased to guildhall.gnu.org -- unless someone else
>> would like to provide the server.  It would be nice to have root on that
>> server, FWIW.  It could be a VM.
>
> I believe FSF could lend us a server or VM, which would thus be
> “independent”, but we’d need to check.

Yeah that would be nice.  I have not had much luck with FSF folk in the
past though.  I was just thinking about gcc.gnu.org which I believe is
hosted by others.

> Since the project may become quite central, it would be nice if it could
> be FSF-copyrighted.  That’s obviously something to discuss with Andreas,
> but the sooner the better.

If it would become part of Guile I would agree with you.  If it stays
separate then it is less clear.  Copyright assignment is actually not a
precondition to be GNU, AFAIK, and all other things being equal, I would
prefer it if we all kept our copyrights and kept the code as GPLv3+.

Bradley Kuhn writes here:

  Regarding ©AAs, I'd like to note finally that FSF does not require
  ©AAs for all GNU packages. This confusion is so common that I'd like
  to draw attention to it, even thought it's only a tangential point in
  this context. FSF's ©AA is only mandatory, to my knowledge, on those
  GNU packages where either (a) FSF employees developed the first
  versions or (b) the original developers themselves asked to assign
  copyright to FSF, upon their project joining GNU. In all other cases,
  FSF assignment is optional.

  http://www.ebb.org/bkuhn/blog/2011/07/07/harmony-harmful.html

I'll ping FSF folk.

Andy
-- 
http://wingolog.org/

Re: guildhall status

[Ludovic Courtès]

Hi!

Andy Wingo <wingo@pobox.com> skribis:

> On Mon 18 Jul 2011 15:06, ludo@gnu.org (Ludovic Courtès) writes:
>
>>> I will see if I can get work to sponsor a server that we can use, and
>>> see if we can get it aliased to guildhall.gnu.org -- unless someone else
>>> would like to provide the server.  It would be nice to have root on that
>>> server, FWIW.  It could be a VM.
>>
>> I believe FSF could lend us a server or VM, which would thus be
>> “independent”, but we’d need to check.
>
> Yeah that would be nice.  I have not had much luck with FSF folk in the
> past though.

How about giving them a chance, and switching to Igalia if we don’t get
positive feedback in a timely fashion?

I believe sysadmins@gnu.org is the place to ask for such things; copying
gnu-advisory@ may help.  Would you like to try?  :-)

>> Since the project may become quite central, it would be nice if it could
>> be FSF-copyrighted.  That’s obviously something to discuss with Andreas,
>> but the sooner the better.
>
> If it would become part of Guile I would agree with you.  If it stays
> separate then it is less clear.  Copyright assignment is actually not a
> precondition to be GNU, AFAIK, and all other things being equal, I would
> prefer it if we all kept our copyrights and kept the code as GPLv3+.

Yes, assignment is optional, but it’s a plus for “strategical” projects.

In this case, both approaches would be OK I suppose.

Thanks,
Ludo’.

Re: guildhall status

[Ludovic Courtès]

Andy Wingo <wingo@pobox.com> skribis:

> Yeah that would be nice.  I have not had much luck with FSF folk in the
> past though.  I was just thinking about gcc.gnu.org which I believe is
> hosted by others.

I see you’ve emailed them in the meantime, thanks!

Ludo’.

Re: guildhall status

[Andreas Rottmann]

Andy Wingo <wingo@pobox.com> writes:

> Hi,
>
> I wanted to bootstrap the guildhall stuff, so last week I started poking
> at Andreas' excellent dorodango (http://www.nongnu.org/dorodango/).  I
> forked his repo on gitorious, removed a bunch of compat stuff for other
> implementations, imported dependencies into the archive, replaced the
> http stuff with Guile's (web) foo, and wired up a standard build system.
>
> The result is here:
>
>   https://gitorious.org/~wingo/dorodango/guildhall
>
> You can:
>
>   git clone git://gitorious.org/~wingo/dorodango/guildhall.git
>
> Then ./configure && make && make check.
>
> You will only be able to succeed there if you have (web client), which I
> have pushed to Guile's stable-2.0 branch.
>
> Wherever you see "doro" in the docs, replace it with "guild hall".
> Perhaps in the future we should drop the "hall", so "guild hall update"
> -> "guild update" or something.  Anyway, a thought for another time.
>
> So, the status:
>
>   1) Builds.
>   2) Passes make check.
>   3) Can update the available list.
>   4) Everything else is untested :-)
>
Cool!

> Next up:
>
>   1) Check status of dorodango functionality.
>   2) Fix things that don't work.
>   3) Profit?
>   4) Start thinking about hosting and accounts and UX and stuff.
>
> I will see if I can get work to sponsor a server that we can use, and
> see if we can get it aliased to guildhall.gnu.org -- unless someone else
> would like to provide the server.  It would be nice to have root on that
> server, FWIW.  It could be a VM.
>
> As far as relation with dorodango goes, we should do our best to keep
> the guildhall compatible with dorodango archives on the net.  We should
> also try hard to share code, but that is secondary.  Farther along I
> would like to rename (dorodango ...) in our source to (guildhall ...) or
> something so that we don't conflict with upstream.  I would also like to
> reduce the number of bundled dependencies, and for the ones that are
> left, include them under the (guildhall ...) namespace, making them
> effectively private.
>
+1; dorodango's dependencies are quite stable, so duplicating them in
the guidhall version should not be much of an issue.  As for reducing
their number, I've posted a mail with some ideas about how this could be
started off a while ago [0].  The thing that would be provide the most
benefit IMHO would be an interface to libzip, using Guile's dynamic FFI;
that would allow guildhall to get rid of industria as a dependency, and
speed up package installation quite a bit.  Also, this code could
eventually be folded back into dorodango, once spells' FFI is working on
Guile.  I'm in principle interested in working on a libzip interface,
but I'd rather get spells' FFI finished on Guile first.

>  That way you can also install dorodango on your
> machine, if you wish, and also install the wak- packages, industria,
> ocelotl, etc.
>
> I would really love for someone to take up this project.  I can help
> getting it to the minimally functional state.  Please let me know if you
> are interested.
>
Yeah, it would be cool if someone took care of this.  I can help with
any dorodango-related issues.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: guildhall status

[Andy Wingo]

Hi Andreas!

On Sat 23 Jul 2011 12:37, Andreas Rottmann <a.rottmann@gmx.at> writes:

> dorodango's dependencies are quite stable, so duplicating them in the
> guidhall version should not be much of an issue.  As for reducing
> their number, I've posted a mail with some ideas about how this could
> be started off a while ago [0].

I'll post another mail about the status, but TBH I'm not terribly
concerned about dependencies at this point.  After trimming things a bit
and importing the remainder under the (guild ...) namespace, I don't
notice slowness; `guild hall' runs in about 150 ms.  I would obviously
like it to be less (30ms would be ideal, 50ms good), but we'd need to
profile things, and it doesn't much matter.

> The thing that would be provide the most benefit IMHO would be an
> interface to libzip, using Guile's dynamic FFI; that would allow
> guildhall to get rid of industria as a dependency, and speed up
> package installation quite a bit.  Also, this code could eventually be
> folded back into dorodango, once spells' FFI is working on Guile.  I'm
> in principle interested in working on a libzip interface, but I'd
> rather get spells' FFI finished on Guile first.

Cool, this would be great work.  FWIW I'm not going to focus on it,
though I am very happy to facilitate any needed changes in Guile
itself.  I wrote a little test program:

    #!/usr/bin/env guile
    !#

    (use-modules (guild weinholt compression zip))

    (define (extract-zip-to-current-dir zip)
      (let* ((p (open-file zip "rb")))
        (for-each
         (lambda (rec)
           (if (central-directory? rec)
               (extract-file p (central-directory->file-record p rec) rec)))
         (get-central-directory p))))

    (define (main arg0 zip)
      (extract-zip-to-current-dir zip))

    (apply main (command-line))

You can chmod +x it and run it after having installed the guild-hall or
uninstalled within ./env.  It extracts what I presume was your test
case, http://rotty.yi.org/doro/experimental/xitomatl-20110103.zip, in
8.75 seconds here, which is not good but it can be endured.

Also, libzip?  That's not present on all machines.  Given that you
already have the zip-parsing stuff from industria, why not just inflate
the files with zlib?

>> I would really love for someone to take up this project.  I can help
>> getting it to the minimally functional state.  Please let me know if you
>> are interested.
>>
> Yeah, it would be cool if someone took care of this.  I can help with
> any dorodango-related issues.

Great.  I think we are close to reaching the point where people can use
this thing.

Regards,

Andy
-- 
http://wingolog.org/

Re: guildhall status

[Andy Wingo]

Hi!

An update on the guild-hall.

On Fri 15 Jul 2011 13:55, Andy Wingo <wingo@pobox.com> writes:

>   git clone git://gitorious.org/~wingo/dorodango/guildhall.git

It is still here, with lots of activity, mostly administrative.  It can
now be installed, has minimal docs (from dorodango), and passes
distcheck.  I trimmed dependencies and otherwise folded things under the
(guild ...) namespace.

Also it is the (guild ...) namespace right now -- I am having trouble
with metaphors.  It seems like the guild-hall package doesn't actually
install the guild hall, more like it gives access to guild halls that
might be on the internet.  Anyway, something to figure out.  I think
that the right answer will come once I move the commands down from under
"guild hall" -- so "guild update" instead of "guild hall update".

> Next up:
>
>   1) Check status of dorodango functionality.
>   2) Fix things that don't work.
>   3) Profit?
>   4) Start thinking about hosting and accounts and UX and stuff.

I still haven't made much progress on the first three, but as regards
the last:

> I will see if I can get work to sponsor a server that we can use, and
> see if we can get it aliased to guildhall.gnu.org -- unless someone else
> would like to provide the server.  It would be nice to have root on that
> server, FWIW.  It could be a VM.

Igalia did kindly offer, but Ludovic made the good point that we should
try the FSF first, and they do seem to be able to host a VM for us, so
we will probably use the FSF.

But beyond hosting the bundles and available.scm, I am still not clear
on what guildhall.gnu.org should offer.  It should probably have a
facility for submitting packages, along with a GPG keyring.  Perhaps it
should run tests on the whole guildhall archive every so often, for
consistency and also for style perhaps.  Perhaps it should host
documentation.  Who knows!

> I would really love for someone to take up this project.  I can help
> getting it to the minimally functional state.  Please let me know if you
> are interested.

I'm still interested, though maybe it will be easier when things
stabilize a bit.  We should probably figure out where to host the
guild-hall stuff as well.  In the meantime patches and such are most
welcome!  Clone the archive above and get to hacking :-)

Andy
-- 
http://wingolog.org/

Re: guildhall status

[Andreas Rottmann]

Andy Wingo <wingo@pobox.com> writes:

> Hi!
>
> An update on the guild-hall.
>
> On Fri 15 Jul 2011 13:55, Andy Wingo <wingo@pobox.com> writes:
>
>>   git clone git://gitorious.org/~wingo/dorodango/guildhall.git
>
> It is still here, with lots of activity, mostly administrative.  It can
> now be installed, has minimal docs (from dorodango), and passes
> distcheck.  I trimmed dependencies and otherwise folded things under the
> (guild ...) namespace.
>
> Also it is the (guild ...) namespace right now -- I am having trouble
> with metaphors.  It seems like the guild-hall package doesn't actually
> install the guild hall, more like it gives access to guild halls that
> might be on the internet.  Anyway, something to figure out.  I think
> that the right answer will come once I move the commands down from under
> "guild hall" -- so "guild update" instead of "guild hall update".
>
Yeah, maybe a "hall" is what dorodango calls (in its somewhat
old-fashioned way ;-)) a "repository".

>> Next up:
>>
>>   1) Check status of dorodango functionality.
>>   2) Fix things that don't work.
>>   3) Profit?
>>   4) Start thinking about hosting and accounts and UX and stuff.
>
> I still haven't made much progress on the first three, but as regards
> the last:
>
>> I will see if I can get work to sponsor a server that we can use, and
>> see if we can get it aliased to guildhall.gnu.org -- unless someone else
>> would like to provide the server.  It would be nice to have root on that
>> server, FWIW.  It could be a VM.
>
> Igalia did kindly offer, but Ludovic made the good point that we should
> try the FSF first, and they do seem to be able to host a VM for us, so
> we will probably use the FSF.
>
> But beyond hosting the bundles and available.scm, I am still not clear
> on what guildhall.gnu.org should offer.  It should probably have a
> facility for submitting packages, along with a GPG keyring.  Perhaps it
> should run tests on the whole guildhall archive every so often, for
> consistency and also for style perhaps.  Perhaps it should host
> documentation.  Who knows!
>
I've been hacking on two things in this general area:

- A Scheme script to allow updates to a repository, verifying uploaded
  packages (from a local directory) against a GPG keyring using detached
  signatures (using gpgv(1) for signature verification).

- A web interface somewhat similiar to <http://packages.debian.org>,
  based on SSAX and the ocelotl libsoup-based HTTPd.  Perhaps I will
  implement the ocelotl HTTPd API on top of Guile's webserver -- this
  would allow code to be shared quite seamlessly.

I can push both, if there's interest, but the web interface is in its
very early stages ATM.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>