HTTPS for Hydra

HTTPS for Hydra

[Roel Janssen]

Dear list,

I would like to propose adding HTTPS support for hydra.gnu.org.  The
direct need to have this set up, is to allow the build status icons to
load on the packages page of the Guix website.

Fortunately, this should be possible without causing a lot of trouble
because Hydra uses nginx as web server.  Here's the nginx manual on
adding support for SSL/TLS:

  http://nginx.org/en/docs/http/configuring_https_servers.html

I'm not sure what the policy for SSL/TLS certificates is, but
personally, I think a LetsEncrypt certificate would be fine:

  https://www.letsencrypt.org

A short guide to get it up and running is here:

  https://adambard.com/blog/using-letsencrypt-with-nginx/

What do you think about adding SSL/TLS to Hydra?  And is anyone with
access to hydra.gnu.org willing to take the time to configure nginx and
get a certificate?

Kind regards,
Roel

Re: HTTPS for Hydra

[Leo Famulari]

On Thu, Feb 04, 2016 at 11:56:52PM +0100, Roel Janssen wrote:
> Dear list,
> 
> I would like to propose adding HTTPS support for hydra.gnu.org.  The
> direct need to have this set up, is to allow the build status icons to
> load on the packages page of the Guix website.
> 
> Fortunately, this should be possible without causing a lot of trouble
> because Hydra uses nginx as web server.  Here's the nginx manual on
> adding support for SSL/TLS:
> 
>   http://nginx.org/en/docs/http/configuring_https_servers.html
> 
> I'm not sure what the policy for SSL/TLS certificates is, but
> personally, I think a LetsEncrypt certificate would be fine:
> 
>   https://www.letsencrypt.org
> 
> A short guide to get it up and running is here:
> 
>   https://adambard.com/blog/using-letsencrypt-with-nginx/

If we decide to use Let's Encrypt, I recommend using the "webroot" [0]
method instead of the method described in that link. The webroot method
does not require server downtime, while the method used in that link
does require you to stop the nginx server every couple months when you
renew the certificates.

> 
> What do you think about adding SSL/TLS to Hydra?  And is anyone with
> access to hydra.gnu.org willing to take the time to configure nginx and
> get a certificate?
> 
> Kind regards,
> Roel
> 

[0]
http://letsencrypt.readthedocs.org/en/latest/using.html#webroot

Re: HTTPS for Hydra

[Ludovic Courtès]

Hello!

It took a while, but finally hydra.gnu.org has its own Let’s Encrypt
certificate and is available over https, woohoo!

Back to the initial problem, we (i.e., you ;-)) can now modify
packages.js so that it uses either http or https to use Hydra’s API,
depending on the current URL.

Thoughts?

Ludo’.

Re: HTTPS for Hydra

[Roel Janssen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: 0001-website-packages-Support-both-http-and-https-request.patch --]
[-- Type: text/x-patch, Size: 1103 bytes --]

From 28cfe3d56d2139fc4a50ac9b20b2a73fe12f5a6c Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel@gnu.org>
Date: Tue, 15 Mar 2016 22:45:32 +0100
Subject: [PATCH] website: packages: Support both http and https requests to
 hydra.gnu.org.

* website/static/base/js/packages.js (set_build_status): Use the protocol of the current URL.
---
 website/static/base/js/packages.js | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/website/static/base/js/packages.js b/website/static/base/js/packages.js
index 246d828..12b827f 100644
--- a/website/static/base/js/packages.js
+++ b/website/static/base/js/packages.js
@@ -17,7 +17,7 @@ function set_build_status (pkg_string)
           pkgIcon.src = "../static/base/img/status-icons/"+ pkgInfo[0]["buildstatus"] + ".png";
       }
     }
-    xhttp.open("GET", "http://hydra.gnu.org/api/latestbuilds?nr=1&project=gnu&jobset=master&job="+ pkg_string, true);
+      xhttp.open("GET", window.location.href.split(":")[0] + "://hydra.gnu.org/api/latestbuilds?nr=1&project=gnu&jobset=master&job="+ pkg_string, true);
     xhttp.send();
   }
 }
-- 
2.6.3


[-- Attachment #2: Type: text/plain, Size: 421 bytes --]

Hello Ludo,

Here's the patch.

Kind regards,
Roel Janssen

Ludovic Courtès writes:

> Hello!
>
> It took a while, but finally hydra.gnu.org has its own Let’s Encrypt
> certificate and is available over https, woohoo!
>
> Back to the initial problem, we (i.e., you ;-)) can now modify
> packages.js so that it uses either http or https to use Hydra’s API,
> depending on the current URL.
>
> Thoughts?
>
> Ludo’.

Re: HTTPS for Hydra

[Ludovic Courtès]

Roel Janssen <roel@gnu.org> skribis:

> From 28cfe3d56d2139fc4a50ac9b20b2a73fe12f5a6c Mon Sep 17 00:00:00 2001
> From: Roel Janssen <roel@gnu.org>
> Date: Tue, 15 Mar 2016 22:45:32 +0100
> Subject: [PATCH] website: packages: Support both http and https requests to
>  hydra.gnu.org.
>
> * website/static/base/js/packages.js (set_build_status): Use the protocol of the current URL.

Pushed with minor formatting tweaks.

And now…  https://www.gnu.org/software/guix/packages/
It works!  :-)

Thanks for being patient!

Ludo’.

Re: HTTPS for Hydra

[Roel Janssen]

Ludovic Courtès writes:

> Roel Janssen <roel@gnu.org> skribis:
>
>> From 28cfe3d56d2139fc4a50ac9b20b2a73fe12f5a6c Mon Sep 17 00:00:00 2001
>> From: Roel Janssen <roel@gnu.org>
>> Date: Tue, 15 Mar 2016 22:45:32 +0100
>> Subject: [PATCH] website: packages: Support both http and https requests to
>>  hydra.gnu.org.
>>
>> * website/static/base/js/packages.js (set_build_status): Use the protocol of the current URL.
>
> Pushed with minor formatting tweaks.
>
> And now…  https://www.gnu.org/software/guix/packages/
> It works!  :-)
>
> Thanks for being patient!

Awesome work getting hydra.gnu.org to serve using HTTPS!  At last, we've
got it working completely.

Thanks!

Re: HTTPS for Hydra

[Pjotr Prins]

On Tue, Mar 15, 2016 at 11:01:27PM +0100, Ludovic Courtès wrote:
> Roel Janssen <roel@gnu.org> skribis:
> 
> > From 28cfe3d56d2139fc4a50ac9b20b2a73fe12f5a6c Mon Sep 17 00:00:00 2001
> > From: Roel Janssen <roel@gnu.org>
> > Date: Tue, 15 Mar 2016 22:45:32 +0100
> > Subject: [PATCH] website: packages: Support both http and https requests to
> >  hydra.gnu.org.
> >
> > * website/static/base/js/packages.js (set_build_status): Use the protocol of the current URL.
> 
> Pushed with minor formatting tweaks.
> 
> And now…  https://www.gnu.org/software/guix/packages/
> It works!  :-)

Sure does! For those who miss it, if you click on expand package you
can see the build status of every target :) Great work!

Pj.

Re: HTTPS for Hydra

[Roel Janssen]

It looks like we've got that CORS problem again.
Did you change the web server configuration at hydra.gnu.org?

Kind regards,
Roel

Re: HTTPS for Hydra

[Ludovic Courtès]

Roel Janssen <roel@gnu.org> skribis:

> It looks like we've got that CORS problem again.

What makes you say so?

> Did you change the web server configuration at hydra.gnu.org?

Starting from a few hours ago, nginx at hydra.gnu.org times out after
~10s instead of 60s.  So when hydra.gnu.org is loaded, the /api requests
time out and we don’t get build status icons.

Ludo’.

Re: HTTPS for Hydra

[Roel Janssen]

Ludovic Courtès writes:

> Roel Janssen <roel@gnu.org> skribis:
>
>> It looks like we've got that CORS problem again.
>
> What makes you say so?

The "web console" in Firefox reported so.  However, I cannot reproduce
that anymore. :)

>> Did you change the web server configuration at hydra.gnu.org?
>
> Starting from a few hours ago, nginx at hydra.gnu.org times out after
> ~10s instead of 60s.  So when hydra.gnu.org is loaded, the /api requests
> time out and we don’t get build status icons.

Well, 10 seconds is a lot of time.  Time for a faster API response from
Hydra ;)

Thanks for looking into this.

Re: HTTPS for Hydra

[Alex Kost]

Roel Janssen (2016-03-18 00:53 +0300) wrote:

> Ludovic Courtès writes:
[...]
>> Starting from a few hours ago, nginx at hydra.gnu.org times out after
>> ~10s instead of 60s.  So when hydra.gnu.org is loaded, the /api requests
>> time out and we don’t get build status icons.
>
> Well, 10 seconds is a lot of time.  Time for a faster API response from
> Hydra ;)

Heh, I also suffer from the slowness, as it makes impossible to look at
some particular Hydra info using emacs interface (for example, by
pressing "B" in a "Guix Package List" buffer), as things like this:

  http://hydra.gnu.org/api/latestbuilds?nr=3&job=wget-1.17.1.x86_64-linux

always time out :-(

-- 
Alex

Re: HTTPS for Hydra

[Ludovic Courtès]

Roel Janssen <roel@gnu.org> skribis:

> Ludovic Courtès writes:

>> Starting from a few hours ago, nginx at hydra.gnu.org times out after
>> ~10s instead of 60s.  So when hydra.gnu.org is loaded, the /api requests
>> time out and we don’t get build status icons.
>
> Well, 10 seconds is a lot of time.

I used a shorter timeout for some of the requests, in particular
.narinfo requests (aka. “updating the list of substitutes”.)

> Time for a faster API response from Hydra ;)

You can’t imagine the load peaks this machine reaches.  :-)

Ludo’.

Re: HTTPS for Hydra

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Roel Janssen (2016-03-18 00:53 +0300) wrote:
>
>> Ludovic Courtès writes:
> [...]
>>> Starting from a few hours ago, nginx at hydra.gnu.org times out after
>>> ~10s instead of 60s.  So when hydra.gnu.org is loaded, the /api requests
>>> time out and we don’t get build status icons.
>>
>> Well, 10 seconds is a lot of time.  Time for a faster API response from
>> Hydra ;)
>
> Heh, I also suffer from the slowness, as it makes impossible to look at
> some particular Hydra info using emacs interface (for example, by
> pressing "B" in a "Guix Package List" buffer), as things like this:
>
>   http://hydra.gnu.org/api/latestbuilds?nr=3&job=wget-1.17.1.x86_64-linux
>
> always time out :-(

I’ve noticed that too, but what to do?

10 seconds is already way more than what is acceptable from a UI
viewpoint.  Increasing the timeout may contribute to increasing the load
on the machine, too.

Ludo’.

Re: HTTPS for Hydra

[Alex Kost]

Ludovic Courtès (2016-03-19 00:08 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Roel Janssen (2016-03-18 00:53 +0300) wrote:
>>
>>> Ludovic Courtès writes:
>> [...]
>>>> Starting from a few hours ago, nginx at hydra.gnu.org times out after
>>>> ~10s instead of 60s.  So when hydra.gnu.org is loaded, the /api requests
>>>> time out and we don’t get build status icons.
>>>
>>> Well, 10 seconds is a lot of time.  Time for a faster API response from
>>> Hydra ;)
>>
>> Heh, I also suffer from the slowness, as it makes impossible to look at
>> some particular Hydra info using emacs interface (for example, by
>> pressing "B" in a "Guix Package List" buffer), as things like this:
>>
>>   http://hydra.gnu.org/api/latestbuilds?nr=3&job=wget-1.17.1.x86_64-linux
>>
>> always time out :-(
>
> I’ve noticed that too, but what to do?

I was just complaining :-) I realize that Hydra is horribly overloaded,
so apparently nothing can be done with this.

> 10 seconds is already way more than what is acceptable from a UI
> viewpoint.  Increasing the timeout may contribute to increasing the load
> on the machine, too.

Sure, I didn't mean to increase the timeout.  I agree with Roel that 10
seconds is a lot of time.  As I said I was just complaining, and
actually I didn't expect any reply on that message.  Sorry for
bothering :-)

-- 
Alex