packaging Bootstrap for Cuirass' web interface

packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Hello!

We are going to embed Bootstrap files to our Cuirass service.  Wouldn't
it be better to package Bootstrap and get the service to use the package
directly?  I think it would make maintenance easier, and I'm unsure it's
good to have huge files on which we don't have control in our Cuirass
git repository.  WDYT?

Clément

Re: packaging Bootstrap for Cuirass' web interface

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]

Clément Lassieur <clement@lassieur.org> ezt írta (időpont: 2018. júl. 23.,
H, 9:48):

> Hello!
>
> We are going to embed Bootstrap files to our Cuirass service.  Wouldn't
> it be better to package Bootstrap and get the service to use the package
> directly?  I think it would make maintenance easier, and I'm unsure it's
> good to have huge files on which we don't have control in our Cuirass
> git repository.  WDYT?
>
>
I think this is a good idea. However, I think we can go on as is for now,
and refactor later. I don't know how much effort
would this involve. Having a minimal subset of Bootstrap that is currently
needed for Cuirass can be done easily I guess, but that is not usually we
package things.
Packaging Bootstrap with all the bells and whistles seems like a big
effort. WDYT? We could also make this minimal package not publicly visible,
if we go this way.


> Clément
>

[-- Attachment #2: Type: text/html, Size: 1354 bytes --]

Re: packaging Bootstrap for Cuirass' web interface

[Ludovic Courtès]

Hi there!

Gábor Boskovits <boskovits@gmail.com> skribis:

> Clément Lassieur <clement@lassieur.org> ezt írta (időpont: 2018. júl. 23.,
> H, 9:48):
>
>> Hello!
>>
>> We are going to embed Bootstrap files to our Cuirass service.  Wouldn't
>> it be better to package Bootstrap and get the service to use the package
>> directly?  I think it would make maintenance easier, and I'm unsure it's
>> good to have huge files on which we don't have control in our Cuirass
>> git repository.  WDYT?
>>
>>
> I think this is a good idea. However, I think we can go on as is for now,
> and refactor later.

I agree.

Actually I think bundling (a subset of) Bootstrap in Cuirass would be
acceptable because Bootstrap is not the kind of thing usually provided
as a package.

However, bundling should be done in a way that makes it easy to track
where those files come from and to update them.  We could either use a
Git submodule, or have clear commit messages that specify the repo and
commit that we used.

Thanks,
Ludo’.

Re: packaging Bootstrap for Cuirass' web interface

[Björn Höfling]

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]

On Mon, 23 Jul 2018 09:48:15 +0200
Clément Lassieur <clement@lassieur.org> wrote:

> Hello!
> 
> We are going to embed Bootstrap files to our Cuirass service.
> Wouldn't it be better to package Bootstrap and get the service to use
> the package directly?  I think it would make maintenance easier, and
> I'm unsure it's good to have huge files on which we don't have
> control in our Cuirass git repository.  WDYT?
> 
> Clément

Building from "source" would be cool, but looks like a lot of "fun":

https://getbootstrap.com/docs/4.0/getting-started/build-tools/#tooling-setup

Here is the npm build script. Look at the dev-dependencies:

https://github.com/twbs/bootstrap/blob/v4.0.0-beta.2/package.json

NPM-Yogurt.

I think the only change to go from sources is write your own simple
build scripts and call your own SASS-compiler and JS minifier.


Björn

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Björn Höfling <bjoern.hoefling@bjoernhoefling.de> writes:

> On Mon, 23 Jul 2018 09:48:15 +0200
> Clément Lassieur <clement@lassieur.org> wrote:
>
>> Hello!
>> 
>> We are going to embed Bootstrap files to our Cuirass service.
>> Wouldn't it be better to package Bootstrap and get the service to use
>> the package directly?  I think it would make maintenance easier, and
>> I'm unsure it's good to have huge files on which we don't have
>> control in our Cuirass git repository.  WDYT?
>> 
>> Clément
>
> Building from "source" would be cool, but looks like a lot of "fun":
>
> https://getbootstrap.com/docs/4.0/getting-started/build-tools/#tooling-setup
>
> Here is the npm build script. Look at the dev-dependencies:
>
> https://github.com/twbs/bootstrap/blob/v4.0.0-beta.2/package.json
>
> NPM-Yogurt.
>
> I think the only change to go from sources is write your own simple
> build scripts and call your own SASS-compiler and JS minifier.

I had no idea it was that complicated!  Thanks for letting me know.

Clément

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi there!
>
> Gábor Boskovits <boskovits@gmail.com> skribis:
>
>> Clément Lassieur <clement@lassieur.org> ezt írta (időpont: 2018. júl. 23.,
>> H, 9:48):
>>
>>> Hello!
>>>
>>> We are going to embed Bootstrap files to our Cuirass service.  Wouldn't
>>> it be better to package Bootstrap and get the service to use the package
>>> directly?  I think it would make maintenance easier, and I'm unsure it's
>>> good to have huge files on which we don't have control in our Cuirass
>>> git repository.  WDYT?
>>>
>>>
>> I think this is a good idea. However, I think we can go on as is for now,
>> and refactor later.
>
> I agree.
>
> Actually I think bundling (a subset of) Bootstrap in Cuirass would be
> acceptable because Bootstrap is not the kind of thing usually provided
> as a package.
>
> However, bundling should be done in a way that makes it easy to track
> where those files come from and to update them.  We could either use a
> Git submodule, or have clear commit messages that specify the repo and
> commit that we used.

Understood.  I prefer clear commit messages to submodules :-)

Tatiana, could you please add the version and the repo link to the
commit message that adds static files?  The idea is that people should
be able to update it easily.

If there is anything special, any kind of complicated procedure, that we
need to follow in order to upgrade Bootstrap, could you write it in a
README?  But I imagine it should be fairly simple.

Thank you,
Clément

RE: packaging Bootstrap for Cuirass' web interface

[Tatiana Sholokhova]

[-- Attachment #1: Type: text/plain, Size: 2137 bytes --]

Hi, Clement!

Done! =) I’ve changed commit message with static files and rebased the latest commits, so my branch has only two commits now as we discuss before.

--
Best regards,
Tatiana

От: Clément Lassieur
Отправлено: 23 июля 2018 г. в 13:29
Кому: Ludovic Courtès; Tatiana Sholokhova
Копия: Gábor Boskovits; Guix-devel
Тема: Re: packaging Bootstrap for Cuirass' web interface

Ludovic Courtès <ludo@gnu.org> writes:

> Hi there!
>
> Gábor Boskovits <boskovits@gmail.com> skribis:
>
>> Clément Lassieur <clement@lassieur.org> ezt írta (időpont: 2018. júl. 23.,
>> H, 9:48):
>>
>>> Hello!
>>>
>>> We are going to embed Bootstrap files to our Cuirass service.  Wouldn't
>>> it be better to package Bootstrap and get the service to use the package
>>> directly?  I think it would make maintenance easier, and I'm unsure it's
>>> good to have huge files on which we don't have control in our Cuirass
>>> git repository.  WDYT?
>>>
>>>
>> I think this is a good idea. However, I think we can go on as is for now,
>> and refactor later.
>
> I agree.
>
> Actually I think bundling (a subset of) Bootstrap in Cuirass would be
> acceptable because Bootstrap is not the kind of thing usually provided
> as a package.
>
> However, bundling should be done in a way that makes it easy to track
> where those files come from and to update them.  We could either use a
> Git submodule, or have clear commit messages that specify the repo and
> commit that we used.

Understood.  I prefer clear commit messages to submodules :-)

Tatiana, could you please add the version and the repo link to the
commit message that adds static files?  The idea is that people should
be able to update it easily.

If there is anything special, any kind of complicated procedure, that we
need to follow in order to upgrade Bootstrap, could you write it in a
README?  But I imagine it should be fairly simple.

Thank you,
Clément



---
Это сообщение проверено на вирусы антивирусом Avast.
https://www.avast.com/antivirus

[-- Attachment #2: Type: text/html, Size: 6365 bytes --]

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Hi Tatiana,

Tatiana Sholokhova <tanja201396@gmail.com> writes:

> Hi, Clement!
>
> Done! =) I’ve changed commit message with static files and rebased the
> latest commits, so my branch has only two commits now as we discuss
> before.

Great!  I rebased your commits onto my "multiple inputs" changes.  I
couldn't force-push your branch so I created another branch:
"web-interface-rebased".

Could you please check that everything works as expected with my rebase?
I'm using it on my instance: https://cuirass.lassieur.org:8081/.

The configuration format changed.  Here are some examples of new format:

https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/modules/sysadmin/services.scm?#n60
https://git.lassieur.org/cgit/emacs.git/tree/guix/config.scm#n775

And of course the "examples" are updated and still work
(https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/tree/examples).

Thank you!
Clément

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Clément Lassieur <clement@lassieur.org> writes:

> Hi Tatiana,
>
> Tatiana Sholokhova <tanja201396@gmail.com> writes:
>
>> Hi, Clement!
>>
>> Done! =) I’ve changed commit message with static files and rebased the
>> latest commits, so my branch has only two commits now as we discuss
>> before.
>
> Great!  I rebased your commits onto my "multiple inputs" changes.  I
> couldn't force-push your branch so I created another branch:
> "web-interface-rebased".
>
> Could you please check that everything works as expected with my rebase?
> I'm using it on my instance: https://cuirass.lassieur.org:8081/.

Hmm there seems to be a problem:

2018-07-25T10:08:48 GET /
In cuirass/http.scm:
   292:21  3 (url-handler _ _ #<<channel> getq: #<atomic-box 406b490?>)
In cuirass/templates.scm:
    73:16  2 (specifications-table #<procedure 7fe14c0d1200 at cuira?>)
In srfi/srfi-1.scm:
    589:5  1 (map #<procedure 7fe14c02b358 at cuirass/templates.scm?> ?)
In unknown file:
           0 (scm-error wrong-type-arg "map" "Wrong type argument: ?" ?)
In procedure map: Wrong type argument: #<procedure 7fe14c0d1200 at cuirass/http.scm:293:22 (db)>

It can be due to my rebase though.  I'll investigate.

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Clément Lassieur <clement@lassieur.org> writes:

> Clément Lassieur <clement@lassieur.org> writes:
>
>> Hi Tatiana,
>>
>> Tatiana Sholokhova <tanja201396@gmail.com> writes:
>>
>>> Hi, Clement!
>>>
>>> Done! =) I’ve changed commit message with static files and rebased the
>>> latest commits, so my branch has only two commits now as we discuss
>>> before.
>>
>> Great!  I rebased your commits onto my "multiple inputs" changes.  I
>> couldn't force-push your branch so I created another branch:
>> "web-interface-rebased".
>>
>> Could you please check that everything works as expected with my rebase?
>> I'm using it on my instance: https://cuirass.lassieur.org:8081/.
>
> Hmm there seems to be a problem:
>
> 2018-07-25T10:08:48 GET /
> In cuirass/http.scm:
>    292:21  3 (url-handler _ _ #<<channel> getq: #<atomic-box 406b490?>)
> In cuirass/templates.scm:
>     73:16  2 (specifications-table #<procedure 7fe14c0d1200 at cuira?>)
> In srfi/srfi-1.scm:
>     589:5  1 (map #<procedure 7fe14c02b358 at cuirass/templates.scm?> ?)
> In unknown file:
>            0 (scm-error wrong-type-arg "map" "Wrong type argument: ?" ?)
> In procedure map: Wrong type argument: #<procedure 7fe14c0d1200 at cuirass/http.scm:293:22 (db)>
>
> It can be due to my rebase though.  I'll investigate.

I also get this if I send many requests at the same time.

--8<---------------cut here---------------start------------->8---
Uncaught exception in fiber ##f:
In ice-9/boot-9.scm:
    841:4  4 (with-throw-handler _ _ _)
In web/server/fiberized.scm:
2018-07-25T10:33:56 HTTP connection from 127.0.0.1
   124:18  3 (_)
In ice-9/suspendable-ports.scm:
     83:4  2 (write-bytes #<closed: file 3ee3230> #vu8(137 80 78 # …) …)
In unknown file:
           1 (port-write #<closed: file 3ee3230> #vu8(137 80 78 71 …) …)
In ice-9/boot-9.scm:
   752:25  0 (dispatch-exception _ _ _)
ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
In procedure fport_write: Broken pipe
--8<---------------cut here---------------end--------------->8---

Weird :-)

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Hello Tatiana!

An interesting command to test that everything works correctly is:

parallel 'echo {} && curl http://localhost:8080/' ::: {1..100}

You can use it on an example server:

./pre-inst-env cuirass --specifications=examples/random.scm --database=test.db
or
./pre-inst-env cuirass --specifications=examples/hello-singleton.scm --database=test.db

It run 100 times 'curl http://localhost:8080/' in parallel, so it's good
at showing issues related to multi-threading.

(you need 'guix package -i parallel' :-))

Those issues are hard to debug, I'll investigate too.

Clément

Re: packaging Bootstrap for Cuirass' web interface

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 2404 bytes --]

Clément Lassieur <clement@lassieur.org> ezt írta (időpont: 2018. júl. 25.,
Sze, 10:46):

> Clément Lassieur <clement@lassieur.org> writes:
>
> > Clément Lassieur <clement@lassieur.org> writes:
> >
> >> Hi Tatiana,
> >>
> >> Tatiana Sholokhova <tanja201396@gmail.com> writes:
> >>
> >>> Hi, Clement!
> >>>
> >>> Done! =) I’ve changed commit message with static files and rebased the
> >>> latest commits, so my branch has only two commits now as we discuss
> >>> before.
> >>
> >> Great!  I rebased your commits onto my "multiple inputs" changes.  I
> >> couldn't force-push your branch so I created another branch:
> >> "web-interface-rebased".
> >>
> >> Could you please check that everything works as expected with my rebase?
> >> I'm using it on my instance: https://cuirass.lassieur.org:8081/.
> >
> > Hmm there seems to be a problem:
> >
> > 2018-07-25T10:08:48 GET /
> > In cuirass/http.scm:
> >    292:21  3 (url-handler _ _ #<<channel> getq: #<atomic-box 406b490?>)
> > In cuirass/templates.scm:
> >     73:16  2 (specifications-table #<procedure 7fe14c0d1200 at cuira?>)
> > In srfi/srfi-1.scm:
> >     589:5  1 (map #<procedure 7fe14c02b358 at cuirass/templates.scm?> ?)
> > In unknown file:
> >            0 (scm-error wrong-type-arg "map" "Wrong type argument: ?" ?)
> > In procedure map: Wrong type argument: #<procedure 7fe14c0d1200 at
> cuirass/http.scm:293:22 (db)>
> >
> > It can be due to my rebase though.  I'll investigate.
>
> I also get this if I send many requests at the same time.
>
> --8<---------------cut here---------------start------------->8---
> Uncaught exception in fiber ##f:
> In ice-9/boot-9.scm:
>     841:4  4 (with-throw-handler _ _ _)
> In web/server/fiberized.scm:
> 2018-07-25T10:33:56 HTTP connection from 127.0.0.1
>    124:18  3 (_)
> In ice-9/suspendable-ports.scm:
>      83:4  2 (write-bytes #<closed: file 3ee3230> #vu8(137 80 78 # …) …)
> In unknown file:
>            1 (port-write #<closed: file 3ee3230> #vu8(137 80 78 71 …) …)
> In ice-9/boot-9.scm:
>    752:25  0 (dispatch-exception _ _ _)
> ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
> In procedure fport_write: Broken pipe
> --8<---------------cut here---------------end--------------->8---
>
>
Hello Clément,

Do you have any idea what has gone wrong here?
Do you need help?


> Weird :-)
>
>

[-- Attachment #2: Type: text/html, Size: 3515 bytes --]

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Gábor Boskovits <boskovits@gmail.com> writes:

>> > Hmm there seems to be a problem:
>> >
>> > 2018-07-25T10:08:48 GET /
>> > In cuirass/http.scm:
>> >    292:21  3 (url-handler _ _ #<<channel> getq: #<atomic-box 406b490?>)
>> > In cuirass/templates.scm:
>> >     73:16  2 (specifications-table #<procedure 7fe14c0d1200 at cuira?>)
>> > In srfi/srfi-1.scm:
>> >     589:5  1 (map #<procedure 7fe14c02b358 at cuirass/templates.scm?> ?)
>> > In unknown file:
>> >            0 (scm-error wrong-type-arg "map" "Wrong type argument: ?" ?)
>> > In procedure map: Wrong type argument: #<procedure 7fe14c0d1200 at
>> cuirass/http.scm:293:22 (db)>
>> >
>> > It can be due to my rebase though.  I'll investigate.
>>
>> I also get this if I send many requests at the same time.
>>
>> --8<---------------cut here---------------start------------->8---
>> Uncaught exception in fiber ##f:
>> In ice-9/boot-9.scm:
>>     841:4  4 (with-throw-handler _ _ _)
>> In web/server/fiberized.scm:
>> 2018-07-25T10:33:56 HTTP connection from 127.0.0.1
>>    124:18  3 (_)
>> In ice-9/suspendable-ports.scm:
>>      83:4  2 (write-bytes #<closed: file 3ee3230> #vu8(137 80 78 # …) …)
>> In unknown file:
>>            1 (port-write #<closed: file 3ee3230> #vu8(137 80 78 71 …) …)
>> In ice-9/boot-9.scm:
>>    752:25  0 (dispatch-exception _ _ _)
>> ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
>> In procedure fport_write: Broken pipe
>> --8<---------------cut here---------------end--------------->8---
>>
>>
> Hello Clément,
>
> Do you have any idea what has gone wrong here?
> Do you need help?

Hello Gábor,

I'll investigate before I ask for help, but any help is always welcome :-)

Thanks,
Clément

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Gábor Boskovits <boskovits@gmail.com> writes:

> Hello Clément,
>
> Do you have any idea what has gone wrong here?

Yes, I think Fibers channels aren't thread-safe.  The bug happens when
CALL-WITH-CRITICAL-SECTION is called at the same time from two different
threads.  So, nothing to do with Tatiana's code. :-)

I'll investigate further, report the bug, and try to find a workaround.

Clément

Re: packaging Bootstrap for Cuirass' web interface

[Ludovic Courtès]

Hi!

Clément Lassieur <clement@lassieur.org> skribis:

> I also get this if I send many requests at the same time.
>
> Uncaught exception in fiber ##f:
> In ice-9/boot-9.scm:
>     841:4  4 (with-throw-handler _ _ _)
> In web/server/fiberized.scm:
> 2018-07-25T10:33:56 HTTP connection from 127.0.0.1
>    124:18  3 (_)
> In ice-9/suspendable-ports.scm:
>      83:4  2 (write-bytes #<closed: file 3ee3230> #vu8(137 80 78 # …) …)
> In unknown file:
>            1 (port-write #<closed: file 3ee3230> #vu8(137 80 78 71 …) …)
> In ice-9/boot-9.scm:
>    752:25  0 (dispatch-exception _ _ _)
> ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
> In procedure fport_write: Broken pipe

That happens if the client sends a request and closes the connection,
and after that the server tries to reply to the client; since the
connection is now closed, the server gets EPIPE.

In practice this happens in particular when the server takes too much
time to reply and its client has left in the meantime.

Ludo’.

Re: packaging Bootstrap for Cuirass' web interface

[Ludovic Courtès]

Clément Lassieur <clement@lassieur.org> skribis:

> Gábor Boskovits <boskovits@gmail.com> writes:
>
>> Hello Clément,
>>
>> Do you have any idea what has gone wrong here?
>
> Yes, I think Fibers channels aren't thread-safe.

It’s always a possibility but that seems unlikely.

Ludo’.

Re: packaging Bootstrap for Cuirass' web interface

[Ludovic Courtès]

Hello Tatiana & Clément,

Clément Lassieur <clement@lassieur.org> skribis:

> Tatiana Sholokhova <tanja201396@gmail.com> writes:
>
>> Hi, Clement!
>>
>> Done! =) I’ve changed commit message with static files and rebased the
>> latest commits, so my branch has only two commits now as we discuss
>> before.
>
> Great!  I rebased your commits onto my "multiple inputs" changes.  I
> couldn't force-push your branch so I created another branch:
> "web-interface-rebased".
>
> Could you please check that everything works as expected with my rebase?
> I'm using it on my instance: https://cuirass.lassieur.org:8081/.

Awesome.  I’d love to have it deployed on berlin.guixsd.org before I go
on vacation!

Ludo’.

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Ludovic Courtès <ludo@gnu.org> writes:

> Clément Lassieur <clement@lassieur.org> skribis:
>
>> Gábor Boskovits <boskovits@gmail.com> writes:
>>
>>> Hello Clément,
>>>
>>> Do you have any idea what has gone wrong here?
>>
>> Yes, I think Fibers channels aren't thread-safe.
>
> It’s always a possibility but that seems unlikely.

Indeed, I spoke too soon :-)

The bug is here:

--8<---------------cut here---------------start------------->8---
(define (make-critical-section . args)
  "Return a channel used to implement a critical section.  That channel can
then be passed to 'join-critical-section', which will ensure sequential
ordering.  ARGS are the arguments of the critical section.

Critical sections are implemented by passing the procedure to execute to a
dedicated fiber."
  (let ((channel (make-channel)))
    (spawn-fiber
     (lambda ()
       (let loop ()
         (match (get-message channel)
           ((? procedure? proc)
            (put-message channel (apply proc args))))
         (loop))))
    channel))

(define (call-with-critical-section channel proc)
  "Call PROC in the critical section corresponding to CHANNEL.  Return the
result of PROC."
  (put-message channel proc)
  (get-message channel))
--8<---------------cut here---------------end--------------->8---

Say I have 2 concurrent fibers (F1 and F2) wanting to serialize
messages through the critical section.

- F1-PUT-MESSAGE puts F1-MESSAGE to CRITICAL-SECTION
- CRITICAL-SECTION gets F1-MESSAGE, which unblocks F1-PUT-MESSAGE
- F1-GET-MESSAGE is called and blocks

- F2-PUT-MESSAGE puts F2-MESSAGE, and the only receiver available
  is... F1-GET-MESSAGE, because CRITICAL-SECTION is busy
- F2-GET-MESSAGE is called and block

- CRITICAL-SECTION is done handling F1-MESSAGE, and put
  F1-MESSAGE-MODIFIED to...  F2-GET-MESSAGE.

F2-MESSAGE is a procedure, whereas F1-MESSAGE-MODIFIED is a list, which
causes the (map f F2-MESSAGE) error.

A solution could be to use two channels, in and out:

--8<---------------cut here---------------start------------->8---
(define (make-critical-section . args)
  "Return a pair of channels used to implement a critical section.  It can
then be passed to 'with-critical-section', which will ensure sequential
ordering.  ARGS are the arguments of the critical section.

Critical sections are implemented by passing the procedure to execute to a
dedicated fiber."
  (let ((channel-in (make-channel))
        (channel-out (make-channel)))
    (spawn-fiber
     (lambda ()
       (let loop ()
         (match (get-message channel-in)
           ((? procedure? proc)
            (put-message channel-out (apply proc args))))
         (loop))))
    (cons channel-in channel-out)))

(define (call-with-critical-section channel proc)
  "Call PROC in the critical section corresponding to CHANNEL.  Return the
result of PROC."
  (match channel
    ((channel-out . channel-in)
     (begin
       (put-message channel-out proc)
       (get-message channel-in)))))
--8<---------------cut here---------------end--------------->8---

WDYT?

Clément

Re: packaging Bootstrap for Cuirass' web interface

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1321 bytes --]

Björn Höfling <bjoern.hoefling@bjoernhoefling.de> writes:

> On Mon, 23 Jul 2018 09:48:15 +0200
> Clément Lassieur <clement@lassieur.org> wrote:
>
>> Hello!
>> 
>> We are going to embed Bootstrap files to our Cuirass service.
>> Wouldn't it be better to package Bootstrap and get the service to use
>> the package directly?  I think it would make maintenance easier, and
>> I'm unsure it's good to have huge files on which we don't have
>> control in our Cuirass git repository.  WDYT?
>> 
>> Clément
>
> Building from "source" would be cool, but looks like a lot of "fun":
>
> https://getbootstrap.com/docs/4.0/getting-started/build-tools/#tooling-setup
>
> Here is the npm build script. Look at the dev-dependencies:
>
> https://github.com/twbs/bootstrap/blob/v4.0.0-beta.2/package.json
>
> NPM-Yogurt.
>
> I think the only change to go from sources is write your own simple
> build scripts and call your own SASS-compiler and JS minifier.

See also:

http://dustycloud.org/blog/javascript-packaging-dystopia/
"Let's Package jQuery: A Javascript Packaging Dystopian Novella"

by Christopher Webber.  jQuery also appears to be a dependency of
Bootstrap (listed under Bootstrap's "peerDependencies").

It's quite a large Gordian knot.  I hope we can cut it someday!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: packaging Bootstrap for Cuirass' web interface

[Ludovic Courtès]

Hi Clément!

Clément Lassieur <clement@lassieur.org> skribis:

> The bug is here:
>
> (define (make-critical-section . args)
>   "Return a channel used to implement a critical section.  That channel can
> then be passed to 'join-critical-section', which will ensure sequential
> ordering.  ARGS are the arguments of the critical section.
>
> Critical sections are implemented by passing the procedure to execute to a
> dedicated fiber."
>   (let ((channel (make-channel)))
>     (spawn-fiber
>      (lambda ()
>        (let loop ()
>          (match (get-message channel)
>            ((? procedure? proc)
>             (put-message channel (apply proc args))))
>          (loop))))
>     channel))
>
> (define (call-with-critical-section channel proc)
>   "Call PROC in the critical section corresponding to CHANNEL.  Return the
> result of PROC."
>   (put-message channel proc)
>   (get-message channel))
>
>
> Say I have 2 concurrent fibers (F1 and F2) wanting to serialize
> messages through the critical section.
>
> - F1-PUT-MESSAGE puts F1-MESSAGE to CRITICAL-SECTION
> - CRITICAL-SECTION gets F1-MESSAGE, which unblocks F1-PUT-MESSAGE
> - F1-GET-MESSAGE is called and blocks
>
> - F2-PUT-MESSAGE puts F2-MESSAGE, and the only receiver available
>   is... F1-GET-MESSAGE, because CRITICAL-SECTION is busy
> - F2-GET-MESSAGE is called and block
>
> - CRITICAL-SECTION is done handling F1-MESSAGE, and put
>   F1-MESSAGE-MODIFIED to...  F2-GET-MESSAGE.
>
> F2-MESSAGE is a procedure, whereas F1-MESSAGE-MODIFIED is a list, which
> causes the (map f F2-MESSAGE) error.

D’oh!  Good catch!

> A solution could be to use two channels, in and out:
>
> (define (make-critical-section . args)
>   "Return a pair of channels used to implement a critical section.  It can
> then be passed to 'with-critical-section', which will ensure sequential
> ordering.  ARGS are the arguments of the critical section.
>
> Critical sections are implemented by passing the procedure to execute to a
> dedicated fiber."
>   (let ((channel-in (make-channel))
>         (channel-out (make-channel)))
>     (spawn-fiber
>      (lambda ()
>        (let loop ()
>          (match (get-message channel-in)
>            ((? procedure? proc)
>             (put-message channel-out (apply proc args))))
>          (loop))))
>     (cons channel-in channel-out)))
>
> (define (call-with-critical-section channel proc)
>   "Call PROC in the critical section corresponding to CHANNEL.  Return the
> result of PROC."
>   (match channel
>     ((channel-out . channel-in)
>      (begin
>        (put-message channel-out proc)
>        (get-message channel-in)))))
>
> WDYT?

Or maybe like this:

(define (make-critical-section . args)
  "Return a channel used to implement a critical section.  That channel can
then be passed to 'join-critical-section', which will ensure sequential
ordering.  ARGS are the arguments of the critical section.

Critical sections are implemented by passing the procedure to execute to a
dedicated fiber."
  (let ((channel (make-channel)))
    (spawn-fiber
     (lambda ()
       (let loop ()
         (match (get-message channel)
           (((? channel? reply) . (? procedure? proc))
            (put-message reply (apply proc args))))
         (loop))))
    channel))

(define (call-with-critical-section channel proc)
  "Call PROC in the critical section corresponding to CHANNEL.  Return the
result of PROC."
  (let ((reply (make-channel)))
    (put-message channel (cons reply proc))
    (get-message reply)))

That makes it clear that the reply channel is used only by the calling
fiber.

WDYT?

Thanks for debugging this!

Ludo’.

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Ludovic Courtès <ludo@gnu.org> writes:

> Or maybe like this:
>
> (define (make-critical-section . args)
>   "Return a channel used to implement a critical section.  That channel can
> then be passed to 'join-critical-section', which will ensure sequential
> ordering.  ARGS are the arguments of the critical section.
>
> Critical sections are implemented by passing the procedure to execute to a
> dedicated fiber."
>   (let ((channel (make-channel)))
>     (spawn-fiber
>      (lambda ()
>        (let loop ()
>          (match (get-message channel)
>            (((? channel? reply) . (? procedure? proc))
>             (put-message reply (apply proc args))))
>          (loop))))
>     channel))
>
> (define (call-with-critical-section channel proc)
>   "Call PROC in the critical section corresponding to CHANNEL.  Return the
> result of PROC."
>   (let ((reply (make-channel)))
>     (put-message channel (cons reply proc))
>     (get-message reply)))
>
> That makes it clear that the reply channel is used only by the calling
> fiber.

I like it, but in that case MAKE-CHANNEL is called every time
CALL-WITH-CRITICAL-SECTION is called.  Do you think it is a significant
overhead?

Clément

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Clément Lassieur <clement@lassieur.org> writes:

>> That makes it clear that the reply channel is used only by the calling
>> fiber.
>
> I like it, but in that case MAKE-CHANNEL is called every time
> CALL-WITH-CRITICAL-SECTION is called.  Do you think it is a significant
> overhead?

I guess it doesn't.  I'll push a patch with this then.

Thank you,
Clément

Re: packaging Bootstrap for Cuirass' web interface

[Ludovic Courtès]

Clément Lassieur <clement@lassieur.org> skribis:

> Clément Lassieur <clement@lassieur.org> writes:
>
>>> That makes it clear that the reply channel is used only by the calling
>>> fiber.
>>
>> I like it, but in that case MAKE-CHANNEL is called every time
>> CALL-WITH-CRITICAL-SECTION is called.  Do you think it is a significant
>> overhead?
>
> I guess it doesn't.  I'll push a patch with this then.

Yeah.  I guess the overhead doesn’t matter much because we’re already on
the slow path of a critical section, presumably to perform an expensive
operation (database access.)

Thanks!

Ludo.

Re: packaging Bootstrap for Cuirass' web interface

[Clément Lassieur]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Clément Lassieur <clement@lassieur.org> skribis:
>
>> I also get this if I send many requests at the same time.
>>
>> Uncaught exception in fiber ##f:
>> In ice-9/boot-9.scm:
>>     841:4  4 (with-throw-handler _ _ _)
>> In web/server/fiberized.scm:
>> 2018-07-25T10:33:56 HTTP connection from 127.0.0.1
>>    124:18  3 (_)
>> In ice-9/suspendable-ports.scm:
>>      83:4  2 (write-bytes #<closed: file 3ee3230> #vu8(137 80 78 # …) …)
>> In unknown file:
>>            1 (port-write #<closed: file 3ee3230> #vu8(137 80 78 71 …) …)
>> In ice-9/boot-9.scm:
>>    752:25  0 (dispatch-exception _ _ _)
>> ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
>> In procedure fport_write: Broken pipe
>
> That happens if the client sends a request and closes the connection,
> and after that the server tries to reply to the client; since the
> connection is now closed, the server gets EPIPE.
>
> In practice this happens in particular when the server takes too much
> time to reply and its client has left in the meantime.

Oh, got it.  Thank you for explaining!

Clément