Continuous integration

Continuous integration

[Andreas Politz]

(Sorry for throwing around buzz words here.)

Michael and I are trying to fix some errors with file-notification.
None of us has all the participating libraries and OS at hand, which
means we are unable to test any patches thoroughly.

So, I wondered whether it be possible to run the test-suite
automatically for every commit/branch/OS.  The results could be
published via mailing-list.

What do you think ?

-ap

Re: Continuous integration

[Phillip Lord]

On Tue, March 21, 2017 3:45 pm, Andreas Politz wrote:
>

> (Sorry for throwing around buzz words here.)
>
>
> Michael and I are trying to fix some errors with file-notification.
> None of us has all the participating libraries and OS at hand, which
> means we are unable to test any patches thoroughly.
>
> So, I wondered whether it be possible to run the test-suite
> automatically for every commit/branch/OS.  The results could be published
> via mailing-list.
>
> What do you think ?


To some extent it's already there.

https://hydra.nixos.org/jobset/gnu/emacs-trunk/all

It doesnt work for every branch though, which is unfortunate, and I think
this is something that Emacs devel would really benefit from.

Re: Continuous integration

[Michael Albinus]

"Phillip Lord" <phillip.lord@russet.org.uk> writes:

>> Michael and I are trying to fix some errors with file-notification.
>> None of us has all the participating libraries and OS at hand, which
>> means we are unable to test any patches thoroughly.
>>
>> So, I wondered whether it be possible to run the test-suite
>> automatically for every commit/branch/OS.  The results could be published
>> via mailing-list.
>>
>> What do you think ?
>
>
> To some extent it's already there.
>
> https://hydra.nixos.org/jobset/gnu/emacs-trunk/all
>
> It doesnt work for every branch though, which is unfortunate, and I think
> this is something that Emacs devel would really benefit from.

hydra is helpful, although I've stopped reading the mailing list due to
too many false reports. Maybe the situation has been improved last
weeks?

The other point is that it focuses on GNU/Linux. This is OK, but it
doesn't help us with our recent problem, running tests for kqueue on
*BSD and/or OS X.

Best regards, Michael.

Re: Continuous integration

[Phillip Lord]

On Tue, March 21, 2017 7:46 pm, Michael Albinus wrote:
> "Phillip Lord" <phillip.lord@russet.org.uk> writes:
>
>> To some extent it's already there.
>>
>>
>> https://hydra.nixos.org/jobset/gnu/emacs-trunk/all
>>
>>
>> It doesnt work for every branch though, which is unfortunate, and I
>> think this is something that Emacs devel would really benefit from.
>
> hydra is helpful, although I've stopped reading the mailing list due to
> too many false reports. Maybe the situation has been improved last weeks?

I don't think so. It's a vicious circle, unfortunately. If not enough
people check it, then its more likely to be broken.

> The other point is that it focuses on GNU/Linux. This is OK, but it
> doesn't help us with our recent problem, running tests for kqueue on *BSD
> and/or OS X.

I agree with this. I had a look at buildbot and got it working cleanly (on
gnu/linux). It supports BSD, MacOS and windows workers also, as well as
build-any-branch semantics.

From my own perspective, being able to start multi-OS CI builds on any
branch would be a very good addition to Emacs.

Phil

Re: Continuous integration

[Michael Albinus]

"Phillip Lord" <phillip.lord@russet.org.uk> writes:

>> hydra is helpful, although I've stopped reading the mailing list due to
>> too many false reports. Maybe the situation has been improved last weeks?
>
> I don't think so. It's a vicious circle, unfortunately. If not enough
> people check it, then its more likely to be broken.

It's even worse now. For failed test runs, there is no log file anymore.
This makes it useless.

> Phil

Best regards, Michael.

Re: Continuous integration

[Toon Claes]

Andreas Politz <politza@hochschule-trier.de> writes:

> So, I wondered whether it be possible to run the test-suite
> automatically for every commit/branch/OS.  The results could be
> published via mailing-list.

Some time ago Ted Zlatanov proposed to use GitLab to improve the
development process:

http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00937.html

GitLab could take care of running CI, because it runs CI when commits
gets pushed to it.

The GitLab Runner can be installed on a large number of platforms:

https://docs.gitlab.com/runner/install/

And the builds logs will be available on the GitLab installation.

I know several people on this list are not familiar with
GitLab/GitHub/BitBucket, that's why Ted asked
savannah-hackers-public@gnu.org if it was possible to run a GitLab
installation on FSF/GNU hardware, but I've never heard anything else
from it. 

http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg01133.html

I think it could be really interesting to give GitLab a try in the Emacs
development workflow. And I am also willing to help to set this up.


-- Toon

Re: Continuous integration

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 934 bytes --]


() Toon Claes <toon@iotcl.com>
() Wed, 22 Mar 2017 09:46:40 +0100 (CET)

   Some time ago Ted Zlatanov proposed to use GitLab to improve
   the development process

(tangent) I tried to create GitLab account several times but it
gave me a 422 error (w/o further explanation) each time.  What's
the probem, i wonder?  My creds ain't good enough, i suppose...

(on topic) I wanted the GitLab account to be able to push (and
thus work on, publicly, using the nice GitLab facilities) ZOW:

 http://www.gnuvola.org/software/zow/

which can be used to whip up CI/CD for Emacs (or anything,
really) in Emacs Lisp.  (Admit it, you saw it coming! :-D)

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Continuous integration

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > (tangent) I tried to create GitLab account several times but it
  > gave me a 422 error (w/o further explanation) each time.  What's
  > the probem, i wonder?  My creds ain't good enough, i suppose...

That could be a serious flaw in GitLab, if it is a policy
rather than a bug.

What sort of "creds" are they checking?  What do they _say_ a person
needs in order to make an account?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Continuous integration

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 1693 bytes --]


() Richard Stallman <rms@gnu.org>
() Wed, 22 Mar 2017 12:42:02 -0400

   What sort of "creds" are they checking?

I don't know what they're checking.  My understanding is that
the site is not actually running the "community edition" variant
(from ‘git clone https://gitlab.com/gitlab-org/gitlab-ce.git’),
so even if i could understand the source in short time (i'm just
starting to study Ruby -- hints from those w/ more experience
very welcome!), the answer might not be evident there anyway.

This happens in a web-browser HTTPS session, so implicit creds
are whatever such a session entails (plus whatever the spooks
manage to sidecar into/over/around the session, i suppose).

  What do they _say_ a person needs in order to make an account?

AFAIR, there were no instructions on what constitutes an
acceptable login name, password, or email address.  Just some
input text fields.

You need to fill these in w/ a login name that's not already
assigned, a password for that login (entered twice to validate),
and an email address.  Both the "not already assigned" and the
password validation checks require Javascript, i think.

Almost forgot: there's also the usual spate of big-SaaSS icons.
Presumably you don't *need* them, but can use them (or let them
(ab)use you, more likely) as substitute forms of authentication.
Those creds i definitely lack.

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Continuous integration

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >    What sort of "creds" are they checking?

  > I don't know what they're checking.

Before relying on Gitlab for anything, we need to find out what
in practice a person needs in order to make a Gitlab account.
Until we know that, we can't reach any conclusion about whether it is ok
to use for Emacs development.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Continuous integration

[Toon Claes]

Richard Stallman <rms@gnu.org> writes:

> Before relying on Gitlab for anything, we need to find out what
> in practice a person needs in order to make a Gitlab account.
> Until we know that, we can't reach any conclusion about whether it is ok
> to use for Emacs development.

The issue Thien-Thi Nguyen was having with registering at GitLab.com was
caused by the fact their cookies were turned off completely.

At the moment it is just not possible to register, or sign in, at
GitLab.com with cookies disabled.

I've noticed many other sites, including savannah.gnu.org, rely on
cookies to log in, so I'm hoping this is not an issue. Or am I mistaken?

That being said, all the results of the emacs tests ran by GitLab CI are
public and available without logging in.

The investigation of the issue of not being able to register can be
found here: https://gitlab.com/gitlab-org/gitlab-ce/issues/30401


-- Toon



DISCLAIMER: I'm working for GitLab.com, but I'm an avid emacs user
and available to help finding the suited CI solution, whether that
is GitLab CI, Hydra, Buildbot, or something else.

Re: Continuous integration

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 1693 bytes --]


() Richard Stallman <rms@gnu.org>
() Wed, 22 Mar 2017 12:42:02 -0400

   What sort of "creds" are they checking?

I don't know what they're checking.  My understanding is that
the site is not actually running the "community edition" variant
(from ‘git clone https://gitlab.com/gitlab-org/gitlab-ce.git’),
so even if i could understand the source in short time (i'm just
starting to study Ruby -- hints from those w/ more experience
very welcome!), the answer might not be evident there anyway.

This happens in a web-browser HTTPS session, so implicit creds
are whatever such a session entails (plus whatever the spooks
manage to sidecar into/over/around the session, i suppose).

  What do they _say_ a person needs in order to make an account?

AFAIR, there were no instructions on what constitutes an
acceptable login name, password, or email address.  Just some
input text fields.

You need to fill these in w/ a login name that's not already
assigned, a password for that login (entered twice to validate),
and an email address.  Both the "not already assigned" and the
password validation checks require Javascript, i think.

Almost forgot: there's also the usual spate of big-SaaSS icons.
Presumably you don't *need* them, but can use them (or let them
(ab)use you, more likely) as substitute forms of authentication.
Those creds i definitely lack.

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Continuous integration

[Ted Zlatanov]

On Wed, 22 Mar 2017 09:46:40 +0100 (CET) Toon Claes <toon@iotcl.com> wrote: 

TC> Some time ago Ted Zlatanov proposed to use GitLab to improve the
TC> development process:

TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00937.html

TC> GitLab could take care of running CI, because it runs CI when commits
TC> gets pushed to it.

Absolutely. I think the benefits reach beyond that--especially if a pull
request workflow could be set up. Right now it's "push into branch; ask
for comments" which is delightfully retro. Together with per-branch CI
(so the changes on the branch can be tested before they are merged, as
opposed to post-merge) this could result in a greatly improved developer
experience.

(Hydra is a good service, but it doesn't offer that level of integration
currently, and I think it would be a bit harder to set that up.)

TC> I know several people on this list are not familiar with
TC> GitLab/GitHub/BitBucket, that's why Ted asked
TC> savannah-hackers-public@gnu.org if it was possible to run a GitLab
TC> installation on FSF/GNU hardware, but I've never heard anything else
TC> from it.

TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg01133.html

Also note the recent discussion about why the Docker Hub web site's
Javascript usage made the Docker Hub service unacceptable. I hope we
don't waste time on discussing a GitLab installation if it doesn't fit
that specific requirement (since it runs a web server).

TC> I think it could be really interesting to give GitLab a try in the Emacs
TC> development workflow. And I am also willing to help to set this up.

Same here.

On Wed, 22 Mar 2017 13:16:39 +0100 Thien-Thi Nguyen <ttn@gnu.org> wrote: 

TN> (tangent) I tried to create GitLab account several times but it
TN> gave me a 422 error (w/o further explanation) each time.  What's
TN> the probem, i wonder?  My creds ain't good enough, i suppose...

Oh, you mean the GitLab hosted CI/CD accounts on gitlab.com. Toon and I
are proposing something different: a FSF/GNU hosted installation of the
GPL-ed GitLab software on local hardware.

Ted

Re: Continuous integration

[Alex]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Also note the recent discussion about why the Docker Hub web site's
> Javascript usage made the Docker Hub service unacceptable. I hope we
> don't waste time on discussing a GitLab installation if it doesn't fit
> that specific requirement (since it runs a web server).

Other self-hosted options are https://gitea.io/en-US/ and
https://gogs.io; both also use a lot of JS but as far as I can tell it’s
all free.

They don’t have built-in CI, though, so that would probably require a
separate dedicated CI server like Jenkins.

Re: Continuous integration

[Toon Claes]

Alex <dunn.alex@gmail.com> writes:

> Other self-hosted options are https://gitea.io/en-US/ and
> https://gogs.io; both also use a lot of JS but as far as I can tell it’s
> all free.

Even if the javascript is free, we should verify if it works properly
with LibreJS.

Re: Continuous integration

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 797 bytes --]


() Ted Zlatanov <tzz@lifelogs.com>
() Wed, 22 Mar 2017 09:14:58 -0400

   TN> (tangent) [...] create GitLab account

   Oh, you mean the GitLab hosted CI/CD accounts on gitlab.com.

Yes.

   Toon and I are proposing something different: a FSF/GNU
   hosted installation of the GPL-ed GitLab software on local
   hardware.

I see.  Well, i hope ZOW can fill the gap until that time comes.
Certainly, it runs on my (local) machine, which has nowhere near
the requisite amount of RAM for a GitLab instance...

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Continuous integration

[John Wiegley]

>>>>> "TN" == Thien-Thi Nguyen <ttn@gnu.org> writes:

TN>    Toon and I are proposing something different: a FSF/GNU
TN>    hosted installation of the GPL-ed GitLab software on local
TN>    hardware.

TN> I see. Well, i hope ZOW can fill the gap until that time comes. Certainly,
TN> it runs on my (local) machine, which has nowhere near the requisite amount
TN> of RAM for a GitLab instance...

Have we verified yet that every aspect of GitLab is free (libre) software?

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Continuous integration

[Stefan Monnier]

> Have we verified yet that every aspect of GitLab is free (libre) software?

AFAIK, yes, the gitlab "Community Edition" software is fully free (the
FSF looked at that when assessing its recommendation for hosting
providers).  And even the JS software you implicitly need to download
and execute when accessing the "Enterprise Edition" (which is what is
used on gitlab.com) is also fully free (tho maybe not fully recognized
as such by librejs).


        Stefan

Re: Continuous integration

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]

On Fri, Mar 31, 2017 at 14:24:44 -0400, Stefan Monnier wrote:
> AFAIK, yes, the gitlab "Community Edition" software is fully free (the
> FSF looked at that when assessing its recommendation for hosting
> providers).

I feel that this needs clarification, just in case.

The FSF itself did not look at anything---there were a number of
volunteers that Zak Rogoff organized.  I evaluated GitLab.com against rms'
criteria, but that is GitLab.com as a _hosting provider, not
self-hosting_.  I did not perform a software evaluation, aside from
ensuring that the JS served to the client is free (I had already worked
with them in the past to change the EE license to liberate all JS and
use Piwik instead of Google Analytics).

If that's a concern, someone else will have to do so.  If there is a
licensing issue, it'd certainly be a bug, and from my experience with
them, they'd get it resolved in short order.

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: Continuous integration

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > TN> I see. Well, i hope ZOW can fill the gap until that time comes. Certainly,
  > TN> it runs on my (local) machine, which has nowhere near the requisite amount
  > TN> of RAM for a GitLab instance...

  > Have we verified yet that every aspect of GitLab is free (libre) software?

Actually, that is not what we need.  What we want to know is that this
CI procedure works without the _users_ having to run any nonfree
software.  (In this case _users_ will be Emacs developers.)

What runs in their server is no direct concern for us.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Continuous integration

[Toon Claes]

Ted Zlatanov <tzz@lifelogs.com> writes:
> Absolutely. I think the benefits reach beyond that--especially if a pull
> request workflow could be set up.

Let's not try to change too many thing at once. But I've been working
with a Pull Request/Merge Request workflow for quite some time now, and
I like it!

> Also note the recent discussion about why the Docker Hub web site's
> Javascript usage made the Docker Hub service unacceptable. I hope we
> don't waste time on discussing a GitLab installation if it doesn't fit
> that specific requirement (since it runs a web server).

The javascript on GitLab is free, but at the moment it is not compatible
with LibreJS. There is an issue about this, but there isn't much
progress on it:

https://gitlab.com/gitlab-org/gitlab-ce/issues/15621

If that is a blocking issue, we should trigger the GitLab team so maybe
it will get a higher priority.

> Oh, you mean the GitLab hosted CI/CD accounts on gitlab.com. Toon and I
> are proposing something different: a FSF/GNU hosted installation of the
> GPL-ed GitLab software on local hardware.

Yes, as far as I understand, the FSF/GNU community does not like relying
on third party hosting, so that why we are suggesting to install a
self-hosted GitLab instance on FSF/GNU systems.

Maybe as temporary solution, we might put a git mirror on GitLab.com and
set up GitLab CI to see if we can get it to run the tests.


-- Toon

Re: Continuous integration

[Phillip Lord]

Toon Claes <toon@iotcl.com> writes:

> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Absolutely. I think the benefits reach beyond that--especially if a pull
>> request workflow could be set up.
>
> Yes, as far as I understand, the FSF/GNU community does not like relying
> on third party hosting, so that why we are suggesting to install a
> self-hosted GitLab instance on FSF/GNU systems.
>
> Maybe as temporary solution, we might put a git mirror on GitLab.com and
> set up GitLab CI to see if we can get it to run the tests.

That should work, but is a slight pain to mirror since the CI
configuration is committed.

Phil

Re: Continuous integration

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Wed, 22 Mar 2017 09:14:58 -0400
> 
> Absolutely. I think the benefits reach beyond that--especially if a pull
> request workflow could be set up. Right now it's "push into branch; ask
> for comments" which is delightfully retro.

Actually, it's more like "obtain write access, then push your changes
directly".

Re: Continuous integration

[Toon Claes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Wed, 22 Mar 2017 09:14:58 -0400
>> 
>> Absolutely. I think the benefits reach beyond that--especially if a pull
>> request workflow could be set up. Right now it's "push into branch; ask
>> for comments" which is delightfully retro.
>
> Actually, it's more like "obtain write access, then push your changes
> directly".

A platform like GitLab relies heavily on pushing code to feature
branches, and merge to master when someone approves. I know this is a
big change compared to the current workflow, that's why I am proposing
to start of by setting up CI (which will work with either workflow).

-- Toon

Re: Continuous integration

[Phillip Lord]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Wed, 22 Mar 2017 09:46:40 +0100 (CET) Toon Claes <toon@iotcl.com> wrote: 
>
> TC> Some time ago Ted Zlatanov proposed to use GitLab to improve the
> TC> development process:
>
> TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00937.html
>
> TC> GitLab could take care of running CI, because it runs CI when commits
> TC> gets pushed to it.
>
> Absolutely. I think the benefits reach beyond that--especially if a pull
> request workflow could be set up. Right now it's "push into branch; ask
> for comments" which is delightfully retro. Together with per-branch CI
> (so the changes on the branch can be tested before they are merged, as
> opposed to post-merge) this could result in a greatly improved developer
> experience.

I'd certainly agree with this. Inline code review would be nice also,
especially given that Emacs git is set up for non-squashing.


> TC> I know several people on this list are not familiar with
> TC> GitLab/GitHub/BitBucket, that's why Ted asked
> TC> savannah-hackers-public@gnu.org if it was possible to run a GitLab
> TC> installation on FSF/GNU hardware, but I've never heard anything else
> TC> from it.
>
> TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg01133.html
>
> Also note the recent discussion about why the Docker Hub web site's
> Javascript usage made the Docker Hub service unacceptable. I hope we
> don't waste time on discussing a GitLab installation if it doesn't fit
> that specific requirement (since it runs a web server).

If we are going to test builds across different platforms then, it is
worth noting, that we technically have to use non-free software. We
cannot test the windows/macos builds without it. I don't know whether
this is a problem or not; it is not dissimilar from building and
distributing Emacs windows binaries.

I mentioned earlier that I've also tried buildbot. It works, the
building is nice, and it's just CI. It would plugin to the existing
savannah infrastructure, so might be less distruptive. The web front end
does not, however, work with librejs.


Phil

Re: Continuous integration

[raman]

Would be nice to target a tool that is usable from within Emacs  --
rather than dumbing things down to a What You See Is All You Have
browser environment. 

Is it safe to assume that people developing Emacs are using Emacs as an
editor?

Browser-based tools get inflicted on one in a heterogeneous  environment
where different individuals use a variety of editors, platforms and
machine types --- in this instance --- emacs-Development --- it would be
nice to leverage the strenghts of Emacs if it is indeed a common
component of our varied environments.
-- 

Re: Continuous integration

[Phillip Lord]

raman <raman@google.com> writes:

> Would be nice to target a tool that is usable from within Emacs  --
> rather than dumbing things down to a What You See Is All You Have
> browser environment. 

To an extent, although, probably not to the extent that it is usuable
everywhere else. Nor to the extent that we have to develop something
specific rather than using software that already exists.

> Is it safe to assume that people developing Emacs are using Emacs as an
> editor?

As an editor, but not necessarily for everything else.


> Browser-based tools get inflicted on one in a heterogeneous  environment
> where different individuals use a variety of editors, platforms and
> machine types --- in this instance --- emacs-Development --- it would be
> nice to leverage the strenghts of Emacs if it is indeed a common
> component of our varied environments.

Both of the tools that have been mentioned (gitlab and buildbot) have
IRC interfaces, so that could be contacted this way. Another solution
would be to extend them to serve up an org file, with hyperlinks through
to logs, and so forth.

AFAICT, the current solution (hydra) doesn't offer an emacs front end.

Phil

Re: Continuous integration

[T.V Raman]

As long as we dont end up doing things like diff/merge  of source code
inside a browser window, I'll stay mostly happy.

Phillip Lord writes:
 > raman <raman@google.com> writes:
 > 
 > > Would be nice to target a tool that is usable from within Emacs  --
 > > rather than dumbing things down to a What You See Is All You Have
 > > browser environment. 
 > 
 > To an extent, although, probably not to the extent that it is usuable
 > everywhere else. Nor to the extent that we have to develop something
 > specific rather than using software that already exists.
 > 
 > > Is it safe to assume that people developing Emacs are using Emacs as an
 > > editor?
 > 
 > As an editor, but not necessarily for everything else.
 > 
 > 
 > > Browser-based tools get inflicted on one in a heterogeneous  environment
 > > where different individuals use a variety of editors, platforms and
 > > machine types --- in this instance --- emacs-Development --- it would be
 > > nice to leverage the strenghts of Emacs if it is indeed a common
 > > component of our varied environments.
 > 
 > Both of the tools that have been mentioned (gitlab and buildbot) have
 > IRC interfaces, so that could be contacted this way. Another solution
 > would be to extend them to serve up an org file, with hyperlinks through
 > to logs, and so forth.
 > 
 > AFAICT, the current solution (hydra) doesn't offer an emacs front end.
 > 
 > Phil

-- 

-- 

Re: Continuous integration

[Phillip Lord]

Oh, I was talking about CI, nothing else.

gitlab and that sort of tool doesn't force you to do diffing or merging
of source code in the web interface. The code review tools, I don't know
about. Some systems have pretty good email interfaces for discussing
diffs; I haven't used gitlabs one, so I don't know.

Phil

raman@google.com (T.V Raman) writes:

> As long as we dont end up doing things like diff/merge  of source code
> inside a browser window, I'll stay mostly happy.
>
> Phillip Lord writes:
>  > raman <raman@google.com> writes:
>  > 
>  > > Would be nice to target a tool that is usable from within Emacs  --
>  > > rather than dumbing things down to a What You See Is All You Have
>  > > browser environment. 
>  > 
>  > To an extent, although, probably not to the extent that it is usuable
>  > everywhere else. Nor to the extent that we have to develop something
>  > specific rather than using software that already exists.
>  > 
>  > > Is it safe to assume that people developing Emacs are using Emacs as an
>  > > editor?
>  > 
>  > As an editor, but not necessarily for everything else.
>  > 
>  > 
>  > > Browser-based tools get inflicted on one in a heterogeneous  environment
>  > > where different individuals use a variety of editors, platforms and
>  > > machine types --- in this instance --- emacs-Development --- it would be
>  > > nice to leverage the strenghts of Emacs if it is indeed a common
>  > > component of our varied environments.
>  > 
>  > Both of the tools that have been mentioned (gitlab and buildbot) have
>  > IRC interfaces, so that could be contacted this way. Another solution
>  > would be to extend them to serve up an org file, with hyperlinks through
>  > to logs, and so forth.
>  > 
>  > AFAICT, the current solution (hydra) doesn't offer an emacs front end.
>  > 
>  > Phil
>
> -- 

Re: Continuous integration

[Toon Claes]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> Some systems have pretty good email interfaces for discussing
> diffs; I haven't used gitlabs one, so I don't know.

The initial review would happen online through the web interface. You
can see the diff online and add a comment to a specific line (of the
diff).

Such comments would produce mail notifications, and GitLab supports
replying to that email to add a comment.

If you aware of tools that won't require the initial review through the
web interface, I really am interested to see how they work.


-- Toon

Re: Continuous integration

[Chad Brown]

> On 23Mar, 2017, at 14:29, Toon Claes <toon@iotcl.com> wrote:
> 
> If you aware of tools that won't require the initial review through the
> web interface, I really am interested to see how they work.

Would someone with a functional setup for such a system (GitLab,
etc.) be willing to check and see how functional it is in eww, please?
(The only one I have easy access to is github, which is problematic
for other reasons).

Thanks in advance,
~Chad

Re: Continuous integration

[Yuri Khan]

On Fri, Mar 24, 2017 at 5:05 AM, Chad Brown <yandros@gmail.com> wrote:

> Would someone with a functional setup for such a system (GitLab,
> etc.) be willing to check and see how functional it is in eww, please?

Nohow, as far as I can tell. I do not have a functional eww setup, but
I disabled Javascript in Firefox and GitLab review functionality
pretty much broke down completely. It can’t even display a diff.

Also, even with Javascript, I find the implementation of code review
in GitLab horrible. I have had better experience with
Bugzilla+Splinter (which still requires Javascript but is more
pleasant UX-wise).

Re: Continuous integration

[Phillip Lord]

Chad Brown <yandros@gmail.com> writes:

>> On 23Mar, 2017, at 14:29, Toon Claes <toon@iotcl.com> wrote:
>> 
>> If you aware of tools that won't require the initial review through the
>> web interface, I really am interested to see how they work.
>
> Would someone with a functional setup for such a system (GitLab,
> etc.) be willing to check and see how functional it is in eww, please?
> (The only one I have easy access to is github, which is problematic
> for other reasons).


Both of them have online versions either as service or a demo. As far as
I can tell, neither of them are functional within eww.

Ultimately, if our main criteria for a CI and code review system is
"does it work with EWW" or "can we have an emacs interface", then I
think that it's likely we are going to end up with something that is
poorly functional. Picking a good system and then making Emacs work with
it seems a better way forward.

Phil

Re: Continuous integration

[raman]

phillip.lord@russet.org.uk (Phillip Lord) writes:
I'd phrase the  requirement slightly differently from what Phillip says
below.

"Functional" is really in the eye of the beholder -- most of the
JS-based browser  environments   lack good APIs that keep the UI
front-end separate from the app-logic back-end.


So asking "does it work in EWW" itself may be a red-herring -- a
well-designed system would let you implement a clean UI from the emacs
end, eeither with or without EWW.

For a good example of such API separation, see Ipython notebooks and how
package ein in Emacs leverages that separation. 
> Chad Brown <yandros@gmail.com> writes:
>
>>> On 23Mar, 2017, at 14:29, Toon Claes <toon@iotcl.com> wrote:
>>> 
>>> If you aware of tools that won't require the initial review through the
>>> web interface, I really am interested to see how they work.
>>
>> Would someone with a functional setup for such a system (GitLab,
>> etc.) be willing to check and see how functional it is in eww, please?
>> (The only one I have easy access to is github, which is problematic
>> for other reasons).
>
>
> Both of them have online versions either as service or a demo. As far as
> I can tell, neither of them are functional within eww.
>
> Ultimately, if our main criteria for a CI and code review system is
> "does it work with EWW" or "can we have an emacs interface", then I
> think that it's likely we are going to end up with something that is
> poorly functional. Picking a good system and then making Emacs work with
> it seems a better way forward.
>
> Phil
>
>

-- 

Re: Continuous integration

[Ted Zlatanov]

On Fri, 24 Mar 2017 10:37:42 +0000 phillip.lord@russet.org.uk (Phillip Lord) wrote: 

PL> Ultimately, if our main criteria for a CI and code review system is
PL> "does it work with EWW" or "can we have an emacs interface", then I
PL> think that it's likely we are going to end up with something that is
PL> poorly functional. Picking a good system and then making Emacs work with
PL> it seems a better way forward.

Agreed; it's probably going to be possible to use the GitLab API. It's
pretty thorough https://docs.gitlab.com/ee/api/README.html

Ted

Re: Continuous integration

[Phillip Lord]

On Fri, March 24, 2017 4:31 pm, Ted Zlatanov wrote:
> On Fri, 24 Mar 2017 10:37:42 +0000 phillip.lord@russet.org.uk (Phillip
> Lord) wrote:
>
>
> PL> Ultimately, if our main criteria for a CI and code review system is
> PL> "does it work with EWW" or "can we have an emacs interface", then I
> PL> think that it's likely we are going to end up with something that is
> PL> poorly functional. Picking a good system and then making Emacs work
> with PL> it seems a better way forward.
>
>
> Agreed; it's probably going to be possible to use the GitLab API. It's
> pretty thorough https://docs.gitlab.com/ee/api/README.html


Just so.

I think we are jagging here. We have two CI systems that people have
looked at (gitlab, and I have buildbot working). gitlab also does code
review, and git front end hosting.

Which leaves us with the three technical/social questions:

1) Are John, Eli and the other devels interested and willing to sanction
adding this form of infrastructure (even as a trial)

2) Does the FSF have resource to provide the bare metal hosting assuming
we want to self-host, even as a trial?

3) Who is prepared to do an installation?

And, I would also pose a fourth moral question:

4) In addition to gnu/linux, can we provide a windows and mac builder,
given that this would by necessity involve non-free software (i.e. the
OSes).


In answer to question 3), I'm happy to do installation for a buildbot
based solution, since I already tried it just for "fun".

Phil

Re: Continuous integration

[Stefan Monnier]

> 4) In addition to gnu/linux, can we provide a windows and mac builder,
> given that this would by necessity involve non-free software (i.e. the
> OSes).

FWIW, I have heard rumors that you can build several Windows programs
under GNU/Linux (and I guess that's why Debian includes a mingw32
package).  Not sure if Emacs falls into this category.  At least I know
it's possible to *run* the w32 version of Emacs under GNU/Linux using
Wine (last time I used it, it suffered from some quirks but was good
enough for me to reproduce a bug locally).


        Stefan

Re: Continuous integration

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Fri, 24 Mar 2017 14:37:53 -0400
> 
> > 4) In addition to gnu/linux, can we provide a windows and mac builder,
> > given that this would by necessity involve non-free software (i.e. the
> > OSes).
> 
> FWIW, I have heard rumors that you can build several Windows programs
> under GNU/Linux (and I guess that's why Debian includes a mingw32
> package).  Not sure if Emacs falls into this category.

It doesn't, for the same reason Emacs cannot be cross-compiled
targeting any other system: building Emacs requires running the built
binary.  As long as dumping Emacs is being used, it's also impractical
with Emacs.

> At least I know it's possible to *run* the w32 version of Emacs
> under GNU/Linux using Wine

AFAIK, running Emacs under Wine requires a specially-formatted
command, like "wine emacs.exe" or some such, so the build system needs
changes to support that, and so does CI.

Re: Continuous integration

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> 4) In addition to gnu/linux, can we provide a windows and mac builder,
>> given that this would by necessity involve non-free software (i.e. the
>> OSes).
>
> FWIW, I have heard rumors that you can build several Windows programs
> under GNU/Linux (and I guess that's why Debian includes a mingw32
> package).  Not sure if Emacs falls into this category.  At least I know
> it's possible to *run* the w32 version of Emacs under GNU/Linux using
> Wine (last time I used it, it suffered from some quirks but was good
> enough for me to reproduce a bug locally).

That may be true, but even if it works, it's not testing the windows
build, it's testing a windows like build.

Phil

Re: Continuous integration

[Eli Zaretskii]

> Date: Fri, 24 Mar 2017 18:07:39 -0000
> From: "Phillip Lord" <phillip.lord@russet.org.uk>
> 
> 1) Are John, Eli and the other devels interested and willing to sanction
> adding this form of infrastructure (even as a trial)

Why do you need a sanction?  AFAIU, this doesn't require anything from
the project, once there are volunteers willing to set this up.  Am I
missing something?

Re: Continuous integration

[Phillip Lord]

On Fri, March 24, 2017 6:59 pm, Eli Zaretskii wrote:
>> Date: Fri, 24 Mar 2017 18:07:39 -0000
>> From: "Phillip Lord" <phillip.lord@russet.org.uk>
>>
>>
>> 1) Are John, Eli and the other devels interested and willing to
>> sanction adding this form of infrastructure (even as a trial)
>
> Why do you need a sanction?  AFAIU, this doesn't require anything from
> the project, once there are volunteers willing to set this up.  Am I
> missing something?


Because of question 2 and 4. Do we have a machine to put it on, and do we
also want a non-free build.

Phil

Re: Continuous integration

[Eli Zaretskii]

> Date: Fri, 24 Mar 2017 21:35:44 -0000
> From: "Phillip Lord" <phillip.lord@russet.org.uk>
> Cc: "Phillip Lord" <phillip.lord@russet.org.uk>,
>  emacs-devel@gnu.org
> 
> On Fri, March 24, 2017 6:59 pm, Eli Zaretskii wrote:
> >> Date: Fri, 24 Mar 2017 18:07:39 -0000
> >> From: "Phillip Lord" <phillip.lord@russet.org.uk>
> >>
> >>
> >> 1) Are John, Eli and the other devels interested and willing to
> >> sanction adding this form of infrastructure (even as a trial)
> >
> > Why do you need a sanction?  AFAIU, this doesn't require anything from
> > the project, once there are volunteers willing to set this up.  Am I
> > missing something?
> 
> Because of question 2 and 4. Do we have a machine to put it on, and do we
> also want a non-free build.

I think Ted answered that, and I agree with him.

Re: Continuous integration

[Ted Zlatanov]

On Fri, 24 Mar 2017 21:59:42 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> Date: Fri, 24 Mar 2017 18:07:39 -0000
>> From: "Phillip Lord" <phillip.lord@russet.org.uk>
>> 
>> 1) Are John, Eli and the other devels interested and willing to sanction
>> adding this form of infrastructure (even as a trial)

EZ> Why do you need a sanction?  AFAIU, this doesn't require anything from
EZ> the project, once there are volunteers willing to set this up.  Am I
EZ> missing something?

There's no need for a sanction; we need the hardware, and Toon
referenced that I asked the savannah admins:
https://lists.gnu.org/archive/html/savannah-hackers-public/2016-07/msg00021.html

The answer was basically "we have no resources, it's up to you."

So I guess it's up to us.

Ted

Re: Continuous integration

[Phillip Lord]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Fri, 24 Mar 2017 21:59:42 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 
>
>>> Date: Fri, 24 Mar 2017 18:07:39 -0000
>>> From: "Phillip Lord" <phillip.lord@russet.org.uk>
>>> 
>>> 1) Are John, Eli and the other devels interested and willing to sanction
>>> adding this form of infrastructure (even as a trial)
>
> EZ> Why do you need a sanction?  AFAIU, this doesn't require anything from
> EZ> the project, once there are volunteers willing to set this up.  Am I
> EZ> missing something?
>
> There's no need for a sanction; we need the hardware, and Toon
> referenced that I asked the savannah admins:
> https://lists.gnu.org/archive/html/savannah-hackers-public/2016-07/msg00021.html
>
> The answer was basically "we have no resources, it's up to you."

The answer that I read was "we are waiting for more resource from FSF too".

> So I guess it's up to us.


That does sound likely though.

Phil

Re: Continuous integration

[Toon Claes]

"Phillip Lord" <phillip.lord@russet.org.uk> writes:
> 3) Who is prepared to do an installation?
>
> In answer to question 3), I'm happy to do installation for a buildbot
> based solution, since I already tried it just for "fun".

As a trial I hope to have time somewhere this week to set up CI using
GitLab.com. It probably won't be the final hosting solution, but this is
at the moment the easiest path to set up a trial installation.

When we have both, we can do a objective comparison.


-- Toon

Re: Continuous integration

[Ted Zlatanov]

On Mon, 27 Mar 2017 11:54:41 +0200 (CEST) Toon Claes <toon@iotcl.com> wrote: 

TC> "Phillip Lord" <phillip.lord@russet.org.uk> writes:
>> 3) Who is prepared to do an installation?
>> 
>> In answer to question 3), I'm happy to do installation for a buildbot
>> based solution, since I already tried it just for "fun".

TC> As a trial I hope to have time somewhere this week to set up CI using
TC> GitLab.com. It probably won't be the final hosting solution, but this is
TC> at the moment the easiest path to set up a trial installation.

TC> When we have both, we can do a objective comparison.

I'd strongly advise against using gitlab.com. Please review the
conversation here about the Docker Hub site: if *any* part of the
gitlab.com usage requires non-free software, even just account creation,
it won't be acceptable.

If you are willing to run a standalone instance (maybe a EC2 micro
instance), that's the best way to get it going right now.

Ted

Re: Continuous integration

[Phillip Lord]

Toon Claes <toon@iotcl.com> writes:

> "Phillip Lord" <phillip.lord@russet.org.uk> writes:
>> 3) Who is prepared to do an installation?
>>
>> In answer to question 3), I'm happy to do installation for a buildbot
>> based solution, since I already tried it just for "fun".
>
> As a trial I hope to have time somewhere this week to set up CI using
> GitLab.com. It probably won't be the final hosting solution, but this is
> at the moment the easiest path to set up a trial installation.
>
> When we have both, we can do a objective comparison.

I have the buildbot installation up now. Slightly harder work than I
hoped, but not too bad.

http://emacs.bioswarm.net:8010

Currently, it's running a single build (full build from clean, through
to tests). It will build any branch (following a change). The build
takes about 60 mins (or 30 mins with parallel builds). In practice, I'd
probably add a "incremental recompile and test" job which would be much
quicker. The builds are running on the master which is probably not
ideal.

Anyway, comments welcome.

Phil

Re: Continuous integration

[Lars Brinkhoff]

phillip.lord@russet.org.uk (Phillip Lord) writes:
> I have the buildbot installation up now. Slightly harder work than I
> hoped, but not too bad.
>
> http://emacs.bioswarm.net:8010

I see two tests are failing: one in ediff-ptch-tests and one
inpython-tests.  I don't see those when I build and test myself.

Is there a way to see build results split into separate branches?

Re: Continuous integration

[Phillip Lord]

On Thu, March 30, 2017 2:47 pm, Lars Brinkhoff wrote:
> phillip.lord@russet.org.uk (Phillip Lord) writes:
>> I have the buildbot installation up now. Slightly harder work than I
>> hoped, but not too bad.
>>
>> http://emacs.bioswarm.net:8010
>>
>
> I see two tests are failing: one in ediff-ptch-tests and one
> inpython-tests.  I don't see those when I build and test myself.

Yep, we need a special target for CI which dumps the output of these files
to screen. AFAICT, there is no way to see the log files. The python-test
is breaking, I think, because it assumes that Emacs is not being built in
a virtual environment (which it is with build bot).

>
>
> Is there a way to see build results split into separate branches?
>

The UI doesn't seem to uncover this, although it may be possible (I am not
a build bot expert). But I can seperate things into queues. In use I'm
make one for emacs-25, one for master, one for everything else.

I've fiddled a bit with the config -- there are now two builds: one
incremental, and one bootstrap. The quick one should be faster.

Re: Continuous integration

[Toon Claes]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> I have the buildbot installation up now. Slightly harder work than I
> hoped, but not too bad.
>
> http://emacs.bioswarm.net:8010

Cool!

Meanwhile, I also have set up CI using GitLab.com.

The project is over here, and is being mirrored from Savannah:

https://gitlab.com/emacs-ci/emacs

To set this up, I had to add a .gitlab-ci.yml file to the root of the
project, with the following content:

#+BEGIN_SRC yaml
image: debian:unstable

before_script:
  - apt update -qq
  - apt install -y -qq build-essential autoconf automake pkg-config libtool m4 autoconf-archive gtk-doc-tools libxml2-utils gobject-introspection libgirepository1.0-dev libglib2.0-dev libjson-glib-dev libncurses-dev

stages:
  - test

test:
  stage: test
  script:
    - ./autogen.sh
    - ./configure --without-makeinfo --with-gnutls=no
    - make check
#+END_SRC

This file probably can be improved in many ways, but I got a successful
build. You can visit the build log here:

https://gitlab.com/emacs-ci/emacs/builds/13595493

If someone with write access would agree to commit the .gitlab-ci.yml
file, the daily mirroring will pick up commits, and GitLab will build
them automatically.

If anybody would like direct write access to the GitLab.com repository,
please click the "Request Access" button on the project page (see link
above).

> Currently, it's running a single build (full build from clean, through
> to tests). It will build any branch (following a change). The build
> takes about 60 mins (or 30 mins with parallel builds). In practice, I'd
> probably add a "incremental recompile and test" job which would be much
> quicker. The builds are running on the master which is probably not
> ideal.

That's great work Phil! I still have to figure out everything it does,
but it seems to be very comprehensive.

The set up at GitLab.com is doing quite the same at the moment. Doing
incremental recompilation would be quite hard on GitLab, because each
build is done in a clean Docker container, so you'll have to export
artifacts and reuse them each time.

GitLab has a feature called pipelines, which allows you to chain builds
together in stages. So this could be an example pipeline:

test --> build some GNU/Linux distro
     \
      -> build macOS
     \
      -> build Windows

The build stages won't be executed if the test stage failed.
If I understand it correctly, buildbot does something similar?
At the moment I only have configured 1 stage on GitLab, because only 1
was needed at the moment. The build stages shown in the flowchart above
can be added in a later stage to make regular builds for different
platforms automatically.

At first sight, also the concept of Workers on buildbot (called Runners
on GitLab) are quite similar.

What I do not yet understand is what the Builders are, and what the
difference is between full and quick?


-- Toon

Re: Continuous integration

[Ted Zlatanov]

On Tue, 4 Apr 2017 22:19:37 +0200 (CEST) Toon Claes <toon@iotcl.com> wrote: 

TC> Meanwhile, I also have set up CI using GitLab.com.

TC> The project is over here, and is being mirrored from Savannah:

TC> https://gitlab.com/emacs-ci/emacs

Toon, that's great. I've requested access to collaborate.

For RMS and anyone else interested in evaluating the suitability of
GitLab, please note that the GnuTLS project is also hosted there, so
they may have already evaluated the free software suitability of it:
https://gitlab.com/gnutls/gnutls

Thanks
Ted

Re: Continuous integration

[Toon Claes]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Tue, 4 Apr 2017 22:19:37 +0200 (CEST) Toon Claes <toon@iotcl.com> wrote: 
>
> Toon, that's great. I've requested access to collaborate.

Thanks! I've given you access!

> For RMS and anyone else interested in evaluating the suitability of
> GitLab, please note that the GnuTLS project is also hosted there, so
> they may have already evaluated the free software suitability of it:
> https://gitlab.com/gnutls/gnutls

As Thien-Thi Nguyen mentioned, some people might have trouble to
register at GitLab.com at the moment, depending on the browser and the
addons you have enabled. There is an issue about this (which you should
be able to see without registering):

https://gitlab.com/gitlab-org/gitlab-ce/issues/30401



-- Toon

Re: Continuous integration

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It is ok to host a project on gitlab.  What I don't know is about the
CI functionalities.  If it works to use them with JS turned off, or
with LibreJS active, then they are ok.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Continuous integration

[Lars Brinkhoff]

Richard Stallman wrote:
> It is ok to host a project on gitlab.  What I don't know is about the
> CI functionalities.  If it works to use them with JS turned off, or
> with LibreJS active, then they are ok.

If they are ok, I'd like to suggest Toon Claes' .gitlab-ci.yml to be
added to the repository:

https://gitlab.com/emacs-ci/emacs/merge_requests/1/diffs

Then there would be automatic building and testing for all commits with
results visible on the GitLab web stite.

Re: Continuous integration

[Glenn Morris]

Lars Brinkhoff wrote:

> Then there would be automatic building and testing for all commits with
> results visible on the GitLab web stite.

Why are you ignoring the existing continuous integration system that
Emacs and other GNU projects have been using for years?

http://hydra.nixos.org/project/gnu

Re: Continuous integration

[Lars Brinkhoff]

Glenn Morris writes:
> Lars Brinkhoff wrote:
>> Then there would be automatic building and testing for all commits with
>> results visible on the GitLab web stite.
>
> Why are you ignoring the existing continuous integration system that
> Emacs and other GNU projects have been using for years?
>
> http://hydra.nixos.org/project/gnu

I'm not ignoring it, I posted a comment about it a week ago or so.

Re: Continuous integration

[Ted Zlatanov]

I'd like to start collecting evaluation criteria for a CI solution for
Emacs developers. We'll evaluate BuildBot, Hydra, and GitLab and any
others people feel should be considered. Everyone will have a chance to
comment, vote, and contribute. But first we have to agree on what we're
evaluating.

What features of a CI solution do you consider important that we can
rate objectively?

Thanks
Ted

Re: Continuous integration

[John Wiegley]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

TZ> I'd like to start collecting evaluation criteria for a CI solution for
TZ> Emacs developers. We'll evaluate BuildBot, Hydra, and GitLab and any
TZ> others people feel should be considered. Everyone will have a chance to
TZ> comment, vote, and contribute. But first we have to agree on what we're
TZ> evaluating.

There has been some exploration done on GitLab already, I wonder if they have
some data to share with you?

TZ> What features of a CI solution do you consider important that we can rate
TZ> objectively?

Letting us know when the build fails, which commit caused it to fail, and the
ability to do this on all the platforms that we care about.

Some of the other features that have been discussed, such as code review and
pull requests, are nice, but I think not necessary just to have a helpful CI.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Continuous integration

[Phillip Lord]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:
>
> TZ> I'd like to start collecting evaluation criteria for a CI solution for
> TZ> Emacs developers. We'll evaluate BuildBot, Hydra, and GitLab and any
> TZ> others people feel should be considered. Everyone will have a chance to
> TZ> comment, vote, and contribute. But first we have to agree on what we're
> TZ> evaluating.
>
> There has been some exploration done on GitLab already, I wonder if they have
> some data to share with you?

And buildbot.


> TZ> What features of a CI solution do you consider important that we can rate
> TZ> objectively?
>
> Letting us know when the build fails, which commit caused it to fail, and the
> ability to do this on all the platforms that we care about.
>
> Some of the other features that have been discussed, such as code review and
> pull requests, are nice, but I think not necessary just to have a helpful CI.

Building all branches though, so that pull requests can be build in a
clean environment is important.

Multiplatform (i.e. including windows and mac) builds would be nice.

Local replicability of the environment (via docker or vm) would be good
as well.

Nice GUI. Important, and not to be underestimated.

Phil

Re: Continuous integration

[Ted Zlatanov]

On Wed, 31 May 2017 12:25:28 -0700 John Wiegley <jwiegley@gmail.com> wrote: 

JW> There has been some exploration done on GitLab already, I wonder if they have
JW> some data to share with you?

I think everyone is waiting. Besides me, no one else seems to have used
the test GitLab instance. I strongly encourage everyone to look around
the Hydra instance we use today, GitLab, BuildBot, and other CI systems
they may know.

From the responses, here are the criteria for "a helpful CI" as John put it:

Builds:

* build logs and good notifications
* good platform coverage
* clean builds of all branches+commits and reporting on each one's build
* local replicability of build environment via Docker or VM
* store build artifacts (packages, tarballs, etc.)

UI:

* good UI/UX and multiple requests for a Web GUI too
* support special build requests: specific branch, target, test (via web or email)

Software and maintainer/company:

* Free software
* probable long-term support; ie they have a solid business plan
* personal logins to comment on builds or specific code

Nice to have:

* pull request awareness (not necessarily PRs in the CI system itself)
* code review capability

In order to keep the evaluation objective, I'll keep out of the voting.
If you just want to vote, please send your votes to me directly by
e-mail. But please feel free to vote and comment here; just make sure
to make it clear that you're voting so I can keep track.

Thanks
Ted

Re: Continuous integration

[Dmitry Gutov]

On 7/14/17 11:08 PM, Ted Zlatanov wrote:

> I think everyone is waiting. Besides me, no one else seems to have used
> the test GitLab instance.

All the latest builds there are broken now.

And no email notifications means no actual usage.

> I strongly encourage everyone to look around
> the Hydra instance we use today, GitLab, BuildBot, and other CI systems
> they may know.

I suspect what we actually need to do is to

   - (optionally) Build a table of check marks with how the CI options 
correspond to the criteria we've enumerated.
   - Nag the Emacs maintainers and the FSF personnel about giving us a 
hardware, or even setting up a GitLab instance themselves, for the Emacs 
developers to use.
   - (important) Somehow deal with the perpetually-broken build and _set 
up email notifications_. There is no other practical way to encourage 
everyone to use the CI. And that's more essential than a (reasonable) 
choice of the CI server.

I think the voting may be considered optional since Richard more or less 
okay'd a FSF GitLab installation. If somebody actively disagrees, nobody 
said we absolutely have to have just one CI server.

On the subject of checkmarks, though:

- GitLab is reasonably feature-rich, and it has a Web UI that's familiar 
to many younger developers. Which is a huge plus. Code Review capability 
included.
- BuildBot seemingly has no code review capability, and seems to be only 
a lego-like system for a ultimately flexible build pipelines. Not our 
primary pain point, I think.
- Hydra is fairly limited in terms of features (not code review, for one 
thing) and has a minimal UI. It's also broken at the moment 
(http://hydra.nixos.org/jobset/gnu/emacs-trunk isn't functional, and 
there are JS errors if you open the Error Console; CORS seems to be the 
problem).

So GitLab seems to come ahead as an obvious winner to me. If you really 
want us to vote, probably better to put that call into a new thread, 
instead of deep in the innards of this one.

Re: Continuous integration

[Ted Zlatanov]

On Mon, 17 Jul 2017 00:36:13 +0300 Dmitry Gutov <dgutov@yandex.ru> wrote: 

DG> On 7/14/17 11:08 PM, Ted Zlatanov wrote:
>> I think everyone is waiting. Besides me, no one else seems to have used
>> the test GitLab instance.

DG> And no email notifications means no actual usage.

We turned off e-mail notifications so we don't annoy people.

DG> I suspect what we actually need to do is to

DG>   - (optionally) Build a table of check marks with how the CI options correspond
DG> to the criteria we've enumerated.

I don't think that's necessary.

DG>   - Nag the Emacs maintainers and the FSF personnel about giving us a hardware,
DG> or even setting up a GitLab instance themselves, for the Emacs developers to
DG> use.

We already did that. We're at the stage of determining what to set up as
"the Emacs CI system."

DG>   - (important) Somehow deal with the perpetually-broken build and _set up email
DG> notifications_. There is no other practical way to encourage everyone to use the
DG> CI. And that's more essential than a (reasonable) choice of the CI server.

If you're talking about the temporary evaluation setup on gitlab.com, I
can work on that. But remember we're evaluating the CI system, not
actually switching to gitlab.com.

DG> So GitLab seems to come ahead as an obvious winner to me. If you really want us
DG> to vote, probably better to put that call into a new thread, instead of deep in
DG> the innards of this one.

I'll record your vote and have posted a new article calling for votes.

Thanks
Ted

request for votes for continuous integration system

[Ted Zlatanov]

Reposting as suggested by Dmitry Gutov.

I strongly encourage everyone to look around the Hydra instance we
use today, GitLab, BuildBot, and other CI systems they may know.

From the responses, here are the criteria for "a helpful CI" as John put it:

Builds:

* build logs and good notifications
* good platform coverage
* clean builds of all branches+commits and reporting on each one's build
* local replicability of build environment via Docker or VM
* store build artifacts (packages, tarballs, etc.)

UI:

* good UI/UX and multiple requests for a Web GUI too
* support special build requests: specific branch, target, test (via web or email)

Software and maintainer/company:

* Free software
* probable long-term support; ie they have a solid business plan
* personal logins to comment on builds or specific code

Nice to have:

* pull request awareness (not necessarily PRs in the CI system itself)
* code review capability

In order to keep the evaluation objective, I'll keep out of the voting.
If you just want to vote, please send your votes to me directly by
e-mail. But please feel free to vote and comment here; just make sure
to make it clear that you're voting so I can keep track.

You can vote for multiple CI systems if you think that's a good thing.

Thanks
Ted

Re: request for votes for continuous integration system

[John Wiegley]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

TZ> I strongly encourage everyone to look around the Hydra instance we
TZ> use today, GitLab, BuildBot, and other CI systems they may know.

What was the final outcome of the voting, in case I missed it?

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: request for votes for continuous integration system

[Ted Zlatanov]

On Fri, 11 Aug 2017 10:36:37 -0700 John Wiegley <jwiegley@gmail.com> wrote: 

>>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:
TZ> I strongly encourage everyone to look around the Hydra instance we
TZ> use today, GitLab, BuildBot, and other CI systems they may know.

JW> What was the final outcome of the voting, in case I missed it?

GitLab has carried the vote by a large majority (8 votes). I got about
half of those in private e-mail. The great majority of Emacs developers
haven't had a strong opinion in emacs-devel or elsewhere.

Meanwhile, gitlab.com has issues mirroring the Emacs repository, but
they said that's a system-wide issue, and they host many thousands of
repositories. So I don't think that will be an issue with a dedicated
private installation, but I'm bringing it up in the interest of full
disclosure: https://gitlab.com/emacs-ci/emacs/issues/3

Unless there are objections, I ask (again) for a machine to host a
GitLab installation. I can help set it up, but it has to be allocated
like the GNU ELPA server, internally. Once it's up and running, we can
set it up and start doing CI against all emacs.git commits.

Ted

Re: request for votes for continuous integration system

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 502 bytes --]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Unless there are objections, I ask (again) for a machine to host a
> GitLab installation. I can help set it up, but it has to be allocated
> like the GNU ELPA server, internally. Once it's up and running, we can
> set it up and start doing CI against all emacs.git commits.

That would be awesome!  I guess that the GitLab instance would only be
used for its CI though, or are we planning to use Merge Requests and/or
its issue tracker as well?

Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: request for votes for continuous integration system

[Ted Zlatanov]

On Fri, 11 Aug 2017 23:41:08 +0200 Nicolas Petton <nicolas@petton.fr> wrote: 

NP> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Unless there are objections, I ask (again) for a machine to host a
>> GitLab installation. I can help set it up, but it has to be allocated
>> like the GNU ELPA server, internally. Once it's up and running, we can
>> set it up and start doing CI against all emacs.git commits.

NP> That would be awesome!  I guess that the GitLab instance would only be
NP> used for its CI though, or are we planning to use Merge Requests and/or
NP> its issue tracker as well?

For now, we're only planning to do CI. After the CI piece is set up and
working, we can discuss other features. The next step is a machine that
can host the GitLab instance. Who can get that set up on the FSF/GNU
side? It could live on the GNU ELPA server or on a new machine.

The GitLab version of the emacs.git repository (and any others needed)
will be a pure mirror, so merge/pull requests won't have an effect.

We will disable the web UI as far as possible so users are not misled
into trying to use those features.

Ted

Re: request for votes for continuous integration system

[John Wiegley]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

TZ> Unless there are objections, I ask (again) for a machine to host a GitLab
TZ> installation. I can help set it up, but it has to be allocated like the
TZ> GNU ELPA server, internally. Once it's up and running, we can set it up
TZ> and start doing CI against all emacs.git commits.

Richard, is this something the sysadmins at the FSF can help with?

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: request for votes for continuous integration system

[Ted Zlatanov]

On Fri, 11 Aug 2017 16:49:25 -0700 John Wiegley <jwiegley@gmail.com> wrote: 

>>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:
TZ> Unless there are objections, I ask (again) for a machine to host a GitLab
TZ> installation. I can help set it up, but it has to be allocated like the
TZ> GNU ELPA server, internally. Once it's up and running, we can set it up
TZ> and start doing CI against all emacs.git commits.

JW> Richard, is this something the sysadmins at the FSF can help with?

Ping again. It's been a while and any help would be welcome in setting
up a GitLab CI system for Emacs.

Thanks
Ted

Re: request for votes for continuous integration system

[Ian Kelling]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Fri, 11 Aug 2017 16:49:25 -0700 John Wiegley <jwiegley@gmail.com> wrote: 
>
>>>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:
> TZ> Unless there are objections, I ask (again) for a machine to host a GitLab
> TZ> installation. I can help set it up, but it has to be allocated like the
> TZ> GNU ELPA server, internally. Once it's up and running, we can set it up
> TZ> and start doing CI against all emacs.git commits.
>
> JW> Richard, is this something the sysadmins at the FSF can help with?
>
> Ping again. It's been a while and any help would be welcome in setting
> up a GitLab CI system for Emacs.
>
> Thanks
> Ted

We recently put a new cluster in operation and can create the machine
pretty easily now. Some of the details like ssh keys and
responsibilities are better discussed off list. Anyone else who wants to
volunteer to admin this system, please reply here or to me and I will
add you to the thread. I'm starting the thread with all the users I see
on the elpa machine.

-- 
Ian Kelling | Senior Systems Administrator, Free Software Foundation
GPG Key: B125 F60B 7B28 7FF6 A2B7  DF8F 170A F0E2 9542 95DF
https://fsf.org | https://gnu.org

Re: request for votes for continuous integration system

[Toon Claes]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Meanwhile, gitlab.com has issues mirroring the Emacs repository, but
> they said that's a system-wide issue, and they host many thousands of
> repositories. So I don't think that will be an issue with a dedicated
> private installation, but I'm bringing it up in the interest of full
> disclosure: https://gitlab.com/emacs-ci/emacs/issues/3
>
> Unless there are objections, I ask (again) for a machine to host a
> GitLab installation. I can help set it up, but it has to be allocated
> like the GNU ELPA server, internally. Once it's up and running, we can
> set it up and start doing CI against all emacs.git commits.

Before you start setting up emacs's private GitLab instance for CI, I
should explain something.

GitLab is built in 2 flavours, CE and EE:
- GitLab CE is Free Software, and licensed under MIT Expat License.
- GitLab EE is built on top of GitLab CE, it includes extra features
  that are destined for enterprise users. GitLab EE has a proprietary
  license. https://gitlab.com/gitlab-org/gitlab-ee/blob/master/LICENSE

The feature to mirror a repo, from Savannah to GitLab, is an EE
feature. So you cannot use that feature without a subscription.
(see https://about.gitlab.com/products/).

So to use GitLab CI, there are a few options:

- Set up a private GitLab EE instance with a paid subscription
  => I doubt GNU/FSF would agree with this

- Set up a private GitLab CE instance and build some custom scripts to
  /manually/ sync the repo from Savannah to GitLab.

- Keep using GitLab.com (the SaaS solution GitLab provides, which is
  running GitLab EE).

- Instead of the savannah repo, use the GitLab repo as the main one.
  => I don't see this happen in the near future.


-- Toon

Re: request for votes for continuous integration system

[Paul Eggert]

Toon Claes wrote:
> - Set up a private GitLab CE instance and build some custom scripts to
>    /manually/  sync the repo from Savannah to GitLab.

As this is the only option that uses free software, it's the only one that a GNU 
project like Emacs could reasonably use.

Re: request for votes for continuous integration system

[Ted Zlatanov]

On Sun, 13 Aug 2017 00:18:01 -0700 Paul Eggert <eggert@cs.ucla.edu> wrote: 

PE> Toon Claes wrote:
>> - Set up a private GitLab CE instance and build some custom scripts to
>> /manually/  sync the repo from Savannah to GitLab.

PE> As this is the only option that uses free software, it's the only one that a GNU
PE> project like Emacs could reasonably use.

Yes. It's not hard to automate: `git fetch --all --tags --prune' to a
local clone (bare repo), followed by `git push -f --prune --mirror' to
the GitLab remote. We just need a machine where the script can live.

Ted

Re: request for votes for continuous integration system

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Unless there are objections, I ask (again) for a machine to host a
  > GitLab installation. I can help set it up, but it has to be allocated
  > like the GNU ELPA server, internally. Once it's up and running, we can
  > set it up and start doing CI against all emacs.git commits.

I asked the sysadmins about this.  They are setting up new server
infrastructure, which is a big job.  It looks like it will take 2 or 3 months.

Once that is done, they will be able to make a new virtual machine.

I'm sorry for the delay.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: request for votes for continuous integration system

[Ted Zlatanov]

On Fri, 18 Aug 2017 11:06:50 -0400 Richard Stallman <rms@gnu.org> wrote: 

RS> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
RS> [[[ whether defending the US Constitution against all enemies,     ]]]
RS> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

>> Unless there are objections, I ask (again) for a machine to host a
>> GitLab installation. I can help set it up, but it has to be allocated
>> like the GNU ELPA server, internally. Once it's up and running, we can
>> set it up and start doing CI against all emacs.git commits.

RS> I asked the sysadmins about this.  They are setting up new server
RS> infrastructure, which is a big job.  It looks like it will take 2 or 3 months.

RS> Once that is done, they will be able to make a new virtual machine.

RS> I'm sorry for the delay.

Thank you for moving things forward. Let us know if we can help.

Ted

Re: Continuous integration

[Dmitry Gutov]

On 5/31/17 9:26 PM, Ted Zlatanov wrote:

> What features of a CI solution do you consider important that we can
> rate objectively?

I think:

- Readable, functional web UI listing builds and the branches, and 
commits, that they correspond to.
- Email notifications to the mailing list, as well as the authors of 
commits (author and committer if they are different) that broke the build.

Nothing else really comes to mind.

Re: Continuous integration

[Stephen Leake]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I'd like to start collecting evaluation criteria for a CI solution for
> Emacs developers. We'll evaluate BuildBot, Hydra, and GitLab and any
> others people feel should be considered. Everyone will have a chance to
> comment, vote, and contribute. But first we have to agree on what we're
> evaluating.
>
> What features of a CI solution do you consider important that we can
> rate objectively?

Free software

probable long-term support; ie they have a solid business plan

support special build requests
    specific branch, target, test
    via web or email

-- 
-- Stephe

Re: Continuous integration

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 737 bytes --]

Ted Zlatanov <tzz@lifelogs.com> schrieb am Mi., 31. Mai 2017 um 20:27 Uhr:

>
> What features of a CI solution do you consider important that we can
> rate objectively?
>

- Good web-based user interface
- Web interface for code reviews for pull requests
- System should automatically test pull requests and notify the author if
they break any unit test
- System should prevent pushing code to non-scratch branches that would
break unit tests
- Running unit tests on a broad selection of supported systems
- Running unit tests on every commit in non-scratch branches, if there is
breakage notifying the author which commit is the culprit
- Integration of test running and code review (feedback in the code review
tool about test status)

[-- Attachment #2: Type: text/html, Size: 1074 bytes --]

Re: Continuous integration

[Ted Zlatanov]

On Sun, 09 Apr 2017 12:35:28 -0400 Glenn Morris <rgm@gnu.org> wrote: 

GM> Lars Brinkhoff wrote:
>> Then there would be automatic building and testing for all commits with
>> results visible on the GitLab web stite.

GM> Why are you ignoring the existing continuous integration system that
GM> Emacs and other GNU projects have been using for years?

GM> http://hydra.nixos.org/project/gnu

I think it's fair to evaluate software that has the potential to improve
Emacs developers' experience.

IMHO Hydra (compared to GitLab) has a poor UI, is not very configurable
(especially on a per-developer level), and does not integrate deeply
with Git. It also has no support for code review or pull/merge requests.

I think notifications and webhooks are better in GitLab but that's
debatable.

The Nix build system is pretty arcane as well.

Finally, I don't think using GitLab precludes Hydra or any other CI
system.

Ted

Re: Continuous integration

[Stefan Monnier]

> with Git. It also has no support for code review or pull/merge requests.

Gitlab-CI doesn't have code review or pull/merge requests either, does it?

> Finally, I don't think using GitLab precludes Hydra or any other CI
> system.

AFAIU, the current discussion is not about Gitlab but about CI systems
(including Gitlab-CI).


        Stefan

Re: Continuous integration

[Lars Brinkhoff]

Stefan Monnier wrote:
> Gitlab-CI doesn't have code review or pull/merge requests either, does it?

No, not in Gitlab-CI.  For that you'd have to use the full GitLab.

Re: Continuous integration

[Ted Zlatanov]

On Tue, 11 Apr 2017 09:37:09 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> with Git. It also has no support for code review or pull/merge requests.
SM> Gitlab-CI doesn't have code review or pull/merge requests either, does it?

You're right. I meant it can easily integrate with the rest of the
GitLab toolchain that does have those.

Ted

Re: Continuous integration

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> with Git. It also has no support for code review or pull/merge requests.
>
> Gitlab-CI doesn't have code review or pull/merge requests either, does it?
>
>> Finally, I don't think using GitLab precludes Hydra or any other CI
>> system.
>
> AFAIU, the current discussion is not about Gitlab but about CI systems
> (including Gitlab-CI).

Even as it stands, gitlab-ci would support pull/merge requests better
than hydra because it should build the branches for us. At the moment,
hydra is limited to emacs-25 and master. It's not possible for a
developer to schedule a clean build at the moment. Achieving this would
be a good thing I think.

Phil

Re: Continuous integration

[Phillip Lord]

Toon Claes <toon@iotcl.com> writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> I have the buildbot installation up now. Slightly harder work than I
>> hoped, but not too bad.
>>
>> http://emacs.bioswarm.net:8010
>
> This file probably can be improved in many ways, but I got a successful
> build. You can visit the build log here:
>
> https://gitlab.com/emacs-ci/emacs/builds/13595493

It's very nice, I think nicer that buildbot, especially as it seems to
have got branches sorted out properly.


>> Currently, it's running a single build (full build from clean, through
>> to tests). It will build any branch (following a change). The build
>> takes about 60 mins (or 30 mins with parallel builds). In practice, I'd
>> probably add a "incremental recompile and test" job which would be much
>> quicker. The builds are running on the master which is probably not
>> ideal.
>
> That's great work Phil! I still have to figure out everything it does,
> but it seems to be very comprehensive.
>
> The set up at GitLab.com is doing quite the same at the moment. Doing
> incremental recompilation would be quite hard on GitLab, because each
> build is done in a clean Docker container, so you'll have to export
> artifacts and reuse them each time.
>
> GitLab has a feature called pipelines, which allows you to chain builds
> together in stages. So this could be an example pipeline:
>
> test --> build some GNU/Linux distro
>      \
>       -> build macOS
>      \
>       -> build Windows
>
> The build stages won't be executed if the test stage failed.
> If I understand it correctly, buildbot does something similar?

I can do anything I like with buildbot. It's programmatic. The flip side
is, of course, you have to program it; it's not very declarative.


> At the moment I only have configured 1 stage on GitLab, because only 1
> was needed at the moment. The build stages shown in the flowchart above
> can be added in a later stage to make regular builds for different
> platforms automatically.
>
> At first sight, also the concept of Workers on buildbot (called Runners
> on GitLab) are quite similar.
>
> What I do not yet understand is what the Builders are, and what the
> difference is between full and quick?


"Builders" are different ways of building things. So, you might have one
specific to windows, or one which runs tests and one which does not.

I've tried to set it up to do an incremental build (that's what "quick"
is). But, I haven't done it right. It's building all branches, but
doesn't understand them. So it's doing subsequent incremental builds on
different branches in the same working directory which is causing the
sorts of breakages that you would expect. Having incremental builds
working is kind of important, I think because the bootstrap is so
slow. Working out how to clean them next time after a failure is
important though.

Phil

Re: Continuous integration

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 581 bytes --]

>>>>> "PL" == Phillip Lord <phillip.lord@russet.org.uk> writes:

PL> 1) Are John, Eli and the other devels interested and willing to sanction
PL> adding this form of infrastructure (even as a trial)

Not only does it have my sanction, but I actually setup a trial gitlab on my
own server at the beginning of last year -- although it didn't go far. So if
someone has renewed energy to make this happen, I'm all for it.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

Re: Continuous integration

[Andreas Politz]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> I mentioned earlier that I've also tried buildbot. It works, the
> building is nice, and it's just CI.

How do you run the different OSs ?

-ap

Re: Continuous integration

[Phillip Lord]

Andreas Politz <politza@hochschule-trier.de> writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> I mentioned earlier that I've also tried buildbot. It works, the
>> building is nice, and it's just CI.
>
> How do you run the different OSs ?


Same way they all do -- a master which runs the website, database and
schedules the build jobs on workers. The workers can have different
OSs.

Phil

Re: Continuous integration

[Lars Brinkhoff]

Another option might be Drone.io:
https://github.com/drone/drone

It's a free continuous integration server implemented in Go.  Builds are
run in Docker containers.