all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Multi-OS Emacs buildbot?
@ 2020-12-19 15:58 Lars Ingebrigtsen
  2020-12-19 16:27 ` dick.r.chiang
                   ` (6 more replies)
  0 siblings, 7 replies; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 15:58 UTC (permalink / raw)
  To: emacs-devel

Over the past few months, I've installed a whole bunch of VMs with
different operating systems to help debug various problems.  Now that
I've got them, I'm wondering whether it would be helpful to do a
buildbot kind of thing across them all?

This can either be a lot of work if done right, or it can be done very
simply.  Doing it right would mean automatically building on all
commits, and notifying the committer about breakages etc...  but that's,
like, real work, and I'm not volunteering to do that at all.

A much simpler solution would be to just write a script that ssh-es to
all the VMs once a day, "git pull; make bootstrap; make check" and then
make a summary report on all the tests that fail to some new Emacs
mailing list.  I can do that; it's trivial.

It'd be...  somewhat useful?  But not really that useful?  Opinions?

(My current VM collection is FreeBSD, NetBSD, OpenBSD, Windows 10
w/Cygwin, Macos-es catalina, mojave and high sierra (with HomeBrew),
Macos catalina (with Macports), and Fedora.  Big Sur and Windows 10
w/MinGW soon to follow.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 15:58 Multi-OS Emacs buildbot? Lars Ingebrigtsen
@ 2020-12-19 16:27 ` dick.r.chiang
  2020-12-19 17:49   ` Lars Ingebrigtsen
  2020-12-19 18:43 ` Pankaj Jangid
                   ` (5 subsequent siblings)
  6 siblings, 1 reply; 69+ messages in thread
From: dick.r.chiang @ 2020-12-19 16:27 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

I suspect you're aware commercial entities (called CI tools) build across
multiple platforms for no financial costs (only political ones).  But I can
never tell with the graybeards at emacs-devel.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 16:27 ` dick.r.chiang
@ 2020-12-19 17:49   ` Lars Ingebrigtsen
  0 siblings, 0 replies; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 17:49 UTC (permalink / raw)
  To: dick.r.chiang; +Cc: emacs-devel

dick.r.chiang@gmail.com writes:

> I suspect you're aware commercial entities (called CI tools) build across
> multiple platforms for no financial costs (only political ones).  But I can
> never tell with the graybeards at emacs-devel.

That was a convoluted way of writing "I hereby offer to set up CI for
Emacs", I think.  Or did I misread you?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 15:58 Multi-OS Emacs buildbot? Lars Ingebrigtsen
  2020-12-19 16:27 ` dick.r.chiang
@ 2020-12-19 18:43 ` Pankaj Jangid
  2020-12-19 18:48   ` Lars Ingebrigtsen
  2020-12-19 19:10 ` Lars Ingebrigtsen
                   ` (4 subsequent siblings)
  6 siblings, 1 reply; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-19 18:43 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

> This can either be a lot of work if done right, or it can be done very
> simply.  Doing it right would mean automatically building on all
> commits, and notifying the committer about breakages etc...  but that's,
> like, real work, and I'm not volunteering to do that at all.

So it goes like this:

=> git pull on driver system once in a day
=> make a list of commits
=> on each vm (concurrent)
=>   pick commit = last
=>   make bootstrap; make check
=>   if fail
=>     for commit in [(last-1)..0]
=>       make bootstrap; make check
=>       if pass
=>         make report for commit+1
=>         email-to author of commit+1
=>         break
=>   else
=>     OK

Or something more ambitious?

> A much simpler solution would be to just write a script that ssh-es to
> all the VMs once a day, "git pull; make bootstrap; make check" and then
> make a summary report on all the tests that fail to some new Emacs
> mailing list.  I can do that; it's trivial.

I guess this is sufficient for us.

The realtime stuff, like build on each commit, will be useful only to
the authors who are directly committing to the repository. They will get
to know within 15mins if there is a problem in commit. But those who are
sending patches there will be some delay.

> It'd be...  somewhat useful?  But not really that useful?  Opinions?

I guess such a system will be more useful during the period when we are
about to release. Otherwise we’ll be spending too much energy. People
are anyway build on their systems.

Or, we can use this system to find out the faulty commit, once someone
reports an issue.

> (My current VM collection is FreeBSD, NetBSD, OpenBSD, Windows 10
> w/Cygwin, Macos-es catalina, mojave and high sierra (with HomeBrew),
> Macos catalina (with Macports), and Fedora.  Big Sur and Windows 10
> w/MinGW soon to follow.)

I heard aws is also offering macos now. Does it actually run on VM? I
thought Apple hardware is must for macos. Or is it something that runs
on real Mac[(book)] only?



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 18:43 ` Pankaj Jangid
@ 2020-12-19 18:48   ` Lars Ingebrigtsen
  2020-12-19 18:57     ` Yuri Khan
  2020-12-19 19:01     ` Pankaj Jangid
  0 siblings, 2 replies; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 18:48 UTC (permalink / raw)
  To: emacs-devel

Pankaj Jangid <pankaj@codeisgreat.org> writes:

> Or something more ambitious?

Way less ambitious.  Almost literally "for i in machines; ssh $i; git
pull; make; make check | tail -n 20; done | mail".

With some timeouts and additional filtering.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 18:48   ` Lars Ingebrigtsen
@ 2020-12-19 18:57     ` Yuri Khan
  2020-12-19 19:01     ` Pankaj Jangid
  1 sibling, 0 replies; 69+ messages in thread
From: Yuri Khan @ 2020-12-19 18:57 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: Emacs developers

On Sun, 20 Dec 2020 at 01:49, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Way less ambitious.  Almost literally "for i in machines; ssh $i; git
> pull; make; make check | tail -n 20; done | mail".

GNU Parallel[1] might be of some use.

[1]: https://www.gnu.org/software/parallel/



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 18:48   ` Lars Ingebrigtsen
  2020-12-19 18:57     ` Yuri Khan
@ 2020-12-19 19:01     ` Pankaj Jangid
  2020-12-19 19:02       ` Lars Ingebrigtsen
  1 sibling, 1 reply; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-19 19:01 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Pankaj Jangid <pankaj@codeisgreat.org> writes:
>
>> Or something more ambitious?
>
> Way less ambitious.  Almost literally "for i in machines; ssh $i; git
> pull; make; make check | tail -n 20; done | mail".

We are not check if it pass or fail.


> With some timeouts and additional filtering.  :-)
            ^^^^^^^^                ^^^^^^^^^

Here is the real work I guess. Which filters are you thinking? and
timeouts I didn’t get. Is this about build time? That could be machine
specific.




^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 19:01     ` Pankaj Jangid
@ 2020-12-19 19:02       ` Lars Ingebrigtsen
  2020-12-19 19:04         ` Lars Ingebrigtsen
  0 siblings, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 19:02 UTC (permalink / raw)
  To: emacs-devel

Pankaj Jangid <pankaj@codeisgreat.org> writes:

> Here is the real work I guess. Which filters are you thinking?

The email should be pretty, but since I'm not sure this has any value at
all, I'm not thinking much of anything.

> and timeouts I didn’t get. Is this about build time? That could be
> machine specific.

Some of the checks will invariably hang sometimes.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 19:02       ` Lars Ingebrigtsen
@ 2020-12-19 19:04         ` Lars Ingebrigtsen
  2020-12-19 19:15           ` Pankaj Jangid
  2020-12-19 19:27           ` Mising posts in thread from gwene Was: " Pankaj Jangid
  0 siblings, 2 replies; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 19:04 UTC (permalink / raw)
  To: emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Some of the checks will invariably hang sometimes.

Or variably hang all the time.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 15:58 Multi-OS Emacs buildbot? Lars Ingebrigtsen
  2020-12-19 16:27 ` dick.r.chiang
  2020-12-19 18:43 ` Pankaj Jangid
@ 2020-12-19 19:10 ` Lars Ingebrigtsen
  2020-12-19 21:45   ` 'make check' errors on Debian (was: Multi-OS Emacs buildbot?) Basil L. Contovounesios
  2020-12-20  6:39   ` Multi-OS Emacs buildbot? Richard Stallman
  2020-12-19 20:16 ` Alan Third
                   ` (3 subsequent siblings)
  6 siblings, 2 replies; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 19:10 UTC (permalink / raw)
  To: emacs-devel

I did a run-through of a bunch of the VMs, and we apparently have "make
check" failures on absolutely all of them currently...  except Debian.
Of course, the question is: Do we care sufficiently that something is
failing on NetBSD?  I mean, to put in some work on that, given our
limited developer resources.  Perhaps not?

On the other hand, it would be nice to know that Emacs builds at all on
Fedora, say -- it might give us greater confidence when making changes
that we're not breaking something fundamental on a bunch of systems.  Go
fast and don't break things.

If we want to do this, but don't want to prioritise fixing test failures
on systems we don't actually use ourselves, we could just disable test
more aggressively (and "FIXME" them). 

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 19:04         ` Lars Ingebrigtsen
@ 2020-12-19 19:15           ` Pankaj Jangid
  2020-12-19 19:18             ` Lars Ingebrigtsen
  2020-12-19 19:27           ` Mising posts in thread from gwene Was: " Pankaj Jangid
  1 sibling, 1 reply; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-19 19:15 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> Some of the checks will invariably hang sometimes.
>
> Or variably hang all the time.

Okay. Let me write and come up with something. I take this up.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 19:15           ` Pankaj Jangid
@ 2020-12-19 19:18             ` Lars Ingebrigtsen
  2020-12-20  2:35               ` Pankaj Jangid
  2020-12-20 13:58               ` Dmitry Gutov
  0 siblings, 2 replies; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 19:18 UTC (permalink / raw)
  To: emacs-devel

Pankaj Jangid <pankaj@codeisgreat.org> writes:

> Okay. Let me write and come up with something. I take this up.

Hey, hang on a minute.  :-)  I appreciate the enthusiasm, but we should
first decide whether we want this or not.  And remember, Dick promised
to set up a real multi-platform CI thing at some buildbot service or
other, and we won't need this super-simple one then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Mising posts in thread from gwene Was: Multi-OS Emacs buildbot?
  2020-12-19 19:04         ` Lars Ingebrigtsen
  2020-12-19 19:15           ` Pankaj Jangid
@ 2020-12-19 19:27           ` Pankaj Jangid
  2020-12-19 19:42             ` Lars Ingebrigtsen
  1 sibling, 1 reply; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-19 19:27 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

Recently I have started to read the mailing list via gwene. I see that
some posts are coming very late. Context is missing in a thread in such
a case.

Is this known or the issue has appeared recently?



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Mising posts in thread from gwene Was: Multi-OS Emacs buildbot?
  2020-12-19 19:27           ` Mising posts in thread from gwene Was: " Pankaj Jangid
@ 2020-12-19 19:42             ` Lars Ingebrigtsen
  2020-12-20  2:40               ` Pankaj Jangid
  0 siblings, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 19:42 UTC (permalink / raw)
  To: emacs-devel

Pankaj Jangid <pankaj@codeisgreat.org> writes:

> Recently I have started to read the mailing list via gwene. I see that
> some posts are coming very late. Context is missing in a thread in such
> a case.
>
> Is this known or the issue has appeared recently?

The FSF mailing lists are sometimes slow (because of the thorough spam
handling).  So if somebody Cc's you directly, you'll often get the
responses much sooner than if you get the responses via the mailing
list.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 15:58 Multi-OS Emacs buildbot? Lars Ingebrigtsen
                   ` (2 preceding siblings ...)
  2020-12-19 19:10 ` Lars Ingebrigtsen
@ 2020-12-19 20:16 ` Alan Third
  2020-12-19 21:11   ` Lars Ingebrigtsen
  2020-12-19 20:20 ` Michael Albinus
                   ` (2 subsequent siblings)
  6 siblings, 1 reply; 69+ messages in thread
From: Alan Third @ 2020-12-19 20:16 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

On Sat, Dec 19, 2020 at 04:58:38PM +0100, Lars Ingebrigtsen wrote:
> Over the past few months, I've installed a whole bunch of VMs with
> different operating systems to help debug various problems.  Now that
> I've got them, I'm wondering whether it would be helpful to do a
> buildbot kind of thing across them all?
> 
> This can either be a lot of work if done right, or it can be done very
> simply.  Doing it right would mean automatically building on all
> commits, and notifying the committer about breakages etc...  but that's,
> like, real work, and I'm not volunteering to do that at all.
> 
> A much simpler solution would be to just write a script that ssh-es to
> all the VMs once a day, "git pull; make bootstrap; make check" and then
> make a summary report on all the tests that fail to some new Emacs
> mailing list.  I can do that; it's trivial.
> 
> It'd be...  somewhat useful?  But not really that useful?  Opinions?
> 
> (My current VM collection is FreeBSD, NetBSD, OpenBSD, Windows 10
> w/Cygwin, Macos-es catalina, mojave and high sierra (with HomeBrew),
> Macos catalina (with Macports), and Fedora.  Big Sur and Windows 10
> w/MinGW soon to follow.)

You should perhaps have a look at the gitlab CI/CD stuff that's
running on emba.gnu.org.

In theory you could install a gitlab runner on each of your VMs and
emba could push the compile jobs out to them.
-- 
Alan Third



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 15:58 Multi-OS Emacs buildbot? Lars Ingebrigtsen
                   ` (3 preceding siblings ...)
  2020-12-19 20:16 ` Alan Third
@ 2020-12-19 20:20 ` Michael Albinus
  2020-12-19 20:46 ` Daniel Martín
  2020-12-22 15:37 ` Nikolay Kudryavtsev
  6 siblings, 0 replies; 69+ messages in thread
From: Michael Albinus @ 2020-12-19 20:20 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

Hi Lars,

> Over the past few months, I've installed a whole bunch of VMs with
> different operating systems to help debug various problems.  Now that
> I've got them, I'm wondering whether it would be helpful to do a
> buildbot kind of thing across them all?
>
> This can either be a lot of work if done right, or it can be done very
> simply.  Doing it right would mean automatically building on all
> commits, and notifying the committer about breakages etc...  but that's,
> like, real work, and I'm not volunteering to do that at all.
>
> A much simpler solution would be to just write a script that ssh-es to
> all the VMs once a day, "git pull; make bootstrap; make check" and then
> make a summary report on all the tests that fail to some new Emacs
> mailing list.  I can do that; it's trivial.
>
> It'd be...  somewhat useful?  But not really that useful?  Opinions?
>
> (My current VM collection is FreeBSD, NetBSD, OpenBSD, Windows 10
> w/Cygwin, Macos-es catalina, mojave and high sierra (with HomeBrew),
> Macos catalina (with Macports), and Fedora.  Big Sur and Windows 10
> w/MinGW soon to follow.)

We have the CI infrastructure on emba.gnu.org. Looks like your VMs are
perfect candidates for GitLab runners from that.

<https://emba.gnu.org/emacs/emacs/pipelines>

<https://docs.gitlab.com/runner/>

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 15:58 Multi-OS Emacs buildbot? Lars Ingebrigtsen
                   ` (4 preceding siblings ...)
  2020-12-19 20:20 ` Michael Albinus
@ 2020-12-19 20:46 ` Daniel Martín
  2020-12-19 21:12   ` Lars Ingebrigtsen
  2020-12-22 15:37 ` Nikolay Kudryavtsev
  6 siblings, 1 reply; 69+ messages in thread
From: Daniel Martín @ 2020-12-19 20:46 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:
>
> It'd be...  somewhat useful?  But not really that useful?  Opinions?

I think it's a good idea.  As other have said, I think we could somehow
add your machines to https://emba.gnu.org/emacs/emacs as GitLab Runners.

I'd be careful with the macOS VMs, though: Apple does not allow
virtualizing macOS in non-Apple hardware.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 20:16 ` Alan Third
@ 2020-12-19 21:11   ` Lars Ingebrigtsen
  2020-12-20  8:50     ` Michael Albinus
  0 siblings, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 21:11 UTC (permalink / raw)
  To: Alan Third; +Cc: emacs-devel

Alan Third <alan@idiocy.org> writes:

> In theory you could install a gitlab runner on each of your VMs and
> emba could push the compile jobs out to them.

Yeah, that's more work than I'm willing to do.

This reminds me, though -- why isn't EMBA sending out "you broke the
build" messages to people?  That'd be quite useful, wouldn't it?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 20:46 ` Daniel Martín
@ 2020-12-19 21:12   ` Lars Ingebrigtsen
  0 siblings, 0 replies; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 21:12 UTC (permalink / raw)
  To: Daniel Martín; +Cc: emacs-devel

Daniel Martín <mardani29@yahoo.es> writes:

> I'd be careful with the macOS VMs, though: Apple does not allow
> virtualizing macOS in non-Apple hardware.

I've signed no contract with Apple, and I'm fortunately not in the US,
so I'm free to run whatever I want on my private machine.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* 'make check' errors on Debian (was: Multi-OS Emacs buildbot?)
  2020-12-19 19:10 ` Lars Ingebrigtsen
@ 2020-12-19 21:45   ` Basil L. Contovounesios
  2020-12-19 21:48     ` 'make check' errors on Debian Lars Ingebrigtsen
  2020-12-20  6:39   ` Multi-OS Emacs buildbot? Richard Stallman
  1 sibling, 1 reply; 69+ messages in thread
From: Basil L. Contovounesios @ 2020-12-19 21:45 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I did a run-through of a bunch of the VMs, and we apparently have "make
> check" failures on absolutely all of them currently...  except Debian.

Really?  I've been getting the errors that follow my signature for quite
a while now on Debian.

-- 
Basil

(current-time-zone) ;; => (0 "GMT")

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.16.0, Xaw3d scroll bars)
 of 2020-12-19 built on thunk
Repository revision: b9edbaed01a91d5fc6235fc679d8e0cd827f6fa9
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12010000
System Description: Debian GNU/Linux bullseye/sid

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O2 -march=native' --config-cache
 --prefix=/home/blc/.local --with-x-toolkit=lucid
 --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB
NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT
LIBOTF ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS
LIBSYSTEMD JSON PDUMPER LCMS2

Important settings:
  value of $LANG: en_IE.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Test solar-sunrise-sunset backtrace:
  signal(ert-test-failed (((should (< (abs (- sunrise 7.27)) epsilon))
  ert-fail(((should (< (abs (- sunrise 7.27)) epsilon)) :form (< 1.003
Test solar-sunrise-sunset condition:
    (ert-test-failed
     ((should
       (<
	(abs ...)
	epsilon))
      :form
      (< 1.0033333324640985 0.016666666666666666)
      :value nil))
   FAILED  1/1  solar-sunrise-sunset (0.003660 sec)

Test nsm-check-local-subnet-ipv4 condition:
    (ert-test-failed
     ((should
       (eq t
	   (nsm-should-check "google.com")))
      :form
      (eq t nil)
      :value nil))
   FAILED  1/2  nsm-check-local-subnet-ipv4 (0.085710 sec)

Test time-stamp-format-am-pm condition:
    (ert-test-failed
     ((should
       (equal
	(time-stamp-string "%#p" ref-time1)
	"pm"))
      :form
      (equal "" "pm")
      :value nil :explanation
      (arrays-of-different-length 0 2 "" "pm" first-mismatch-at 0)))
   FAILED   7/27  time-stamp-format-am-pm (0.000140 sec)

Test time-stamp-format-ignored-modifiers condition:
    (ert-test-failed
     ((should
       (equal
	(time-stamp-string "%.,@-+_ ^(stuff)P" ref-time3)
	"AM"))
      :form
      (equal "  " "AM")
      :value nil :explanation
      (array-elt 0
		 (different-atoms
		  (32 "#x20" "? ")
		  (65 "#x41" "?A")))))
   FAILED  13/27  time-stamp-format-ignored-modifiers (0.000121 sec)

Test time-stamp-format-string-width condition:
    (ert-test-failed
     ((should
       (equal
	(time-stamp-string "%1P" ref-time3)
	"A"))
      :form
      (equal " " "A")
      :value nil :explanation
      (array-elt 0
		 (different-atoms
		  (32 "#x20" "? ")
		  (65 "#x41" "?A")))))
   FAILED  20/27  time-stamp-format-string-width (0.000126 sec)



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: 'make check' errors on Debian
  2020-12-19 21:45   ` 'make check' errors on Debian (was: Multi-OS Emacs buildbot?) Basil L. Contovounesios
@ 2020-12-19 21:48     ` Lars Ingebrigtsen
  0 siblings, 0 replies; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-19 21:48 UTC (permalink / raw)
  To: Basil L. Contovounesios; +Cc: emacs-devel

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> I did a run-through of a bunch of the VMs, and we apparently have "make
>> check" failures on absolutely all of them currently...  except Debian.
>
> Really?  I've been getting the errors that follow my signature for quite
> a while now on Debian.

I don't get any of those, and neither does hydra nor emba (which are
both Debian, I guess?)  You should file a bug report.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 19:18             ` Lars Ingebrigtsen
@ 2020-12-20  2:35               ` Pankaj Jangid
  2020-12-20  4:17                 ` Thien-Thi Nguyen
  2020-12-20  6:54                 ` Tim Landscheidt
  2020-12-20 13:58               ` Dmitry Gutov
  1 sibling, 2 replies; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-20  2:35 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> Okay. Let me write and come up with something. I take this up.
>
> Hey, hang on a minute.  :-)  I appreciate the enthusiasm, but we should
> first decide whether we want this or not.  And remember, Dick promised
> to set up a real multi-platform CI thing at some buildbot service or
> other, and we won't need this super-simple one then.

Definitely I was going to hold till today. I was very late. :-) Read the
whole thread now. Looks like we already have most parts.

Even if that infrastruction is ready, I would certainly want a script to
find out the commit that broke the build on my hardware. That reverse
choronological loop.

If someone has that ready then please share. Or may be that could be
part of the repository.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Mising posts in thread from gwene Was: Multi-OS Emacs buildbot?
  2020-12-19 19:42             ` Lars Ingebrigtsen
@ 2020-12-20  2:40               ` Pankaj Jangid
  0 siblings, 0 replies; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-20  2:40 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> Recently I have started to read the mailing list via gwene. I see that
>> some posts are coming very late. Context is missing in a thread in such
>> a case.
>>
>> Is this known or the issue has appeared recently?
>
> The FSF mailing lists are sometimes slow (because of the thorough spam
> handling).  So if somebody Cc's you directly, you'll often get the
> responses much sooner than if you get the responses via the mailing
> list.

Okay. So switching off MFT for important groups is a good idea. Or may
be use C-c C-f C-a for specifc threads.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20  2:35               ` Pankaj Jangid
@ 2020-12-20  4:17                 ` Thien-Thi Nguyen
  2020-12-20  5:16                   ` Pankaj Jangid
  2020-12-20  6:54                 ` Tim Landscheidt
  1 sibling, 1 reply; 69+ messages in thread
From: Thien-Thi Nguyen @ 2020-12-20  4:17 UTC (permalink / raw)
  To: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 769 bytes --]


() Pankaj Jangid <pankaj@codeisgreat.org>
() Sun, 20 Dec 2020 08:05:03 +0530

   Even if that infrastruction is ready, I would certainly want
   a script to find out the commit that broke the build on my
   hardware. That reverse choronological loop.

How would that loop be helpful?  If the late{st,r} commit works,
i don't see the point looking for trouble.  What am i missing?

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)               ; (2020) Software Libero
   (pcase (context query)               ;       = Dissenso Etico
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 219 bytes --]

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20  4:17                 ` Thien-Thi Nguyen
@ 2020-12-20  5:16                   ` Pankaj Jangid
  0 siblings, 0 replies; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-20  5:16 UTC (permalink / raw)
  To: Thien-Thi Nguyen; +Cc: emacs-devel

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> () Pankaj Jangid <pankaj@codeisgreat.org>
> () Sun, 20 Dec 2020 08:05:03 +0530
>
>    Even if that infrastruction is ready, I would certainly want
>    a script to find out the commit that broke the build on my
>    hardware. That reverse choronological loop.
>
> How would that loop be helpful?  If the late{st,r} commit works,
> i don't see the point looking for trouble.  What am i missing?

That script will not be useful only if the last commit works. It is useful
to file a bug if the last build fails. Usually, I git-pull once in a day
and that comes with many commits. If HEAD works then no worries. Else
try HEAD~1 and so on...



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 19:10 ` Lars Ingebrigtsen
  2020-12-19 21:45   ` 'make check' errors on Debian (was: Multi-OS Emacs buildbot?) Basil L. Contovounesios
@ 2020-12-20  6:39   ` Richard Stallman
  2020-12-20 17:30     ` Lars Ingebrigtsen
  1 sibling, 1 reply; 69+ messages in thread
From: Richard Stallman @ 2020-12-20  6:39 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Of course, the question is: Do we care sufficiently that something is
  > failing on NetBSD?  I mean, to put in some work on that, given our
  > limited developer resources.  Perhaps not?

That is presented as one question but really it is a range of
questions.

Do we care about NetBSD enough to debug Emacs failures on NetBSD
on our own initiative?  The Emacs maintainers should not.

Do we care about NetBSD enough to arrange to inform people
who might want to work on that debugging?  I think we should,
unless informing them is harder than I'd expect.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)





^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20  2:35               ` Pankaj Jangid
  2020-12-20  4:17                 ` Thien-Thi Nguyen
@ 2020-12-20  6:54                 ` Tim Landscheidt
  2020-12-20  8:17                   ` Pankaj Jangid
  1 sibling, 1 reply; 69+ messages in thread
From: Tim Landscheidt @ 2020-12-20  6:54 UTC (permalink / raw)
  To: emacs-devel

Pankaj Jangid <pankaj@codeisgreat.org> wrote:

>>> Okay. Let me write and come up with something. I take this up.

>> Hey, hang on a minute.  :-)  I appreciate the enthusiasm, but we should
>> first decide whether we want this or not.  And remember, Dick promised
>> to set up a real multi-platform CI thing at some buildbot service or
>> other, and we won't need this super-simple one then.

> Definitely I was going to hold till today. I was very late. :-) Read the
> whole thread now. Looks like we already have most parts.

> Even if that infrastruction is ready, I would certainly want a script to
> find out the commit that broke the build on my hardware. That reverse
> choronological loop.

> If someone has that ready then please share. Or may be that could be
> part of the repository.

I'm not sure what you mean exactly.  If you want to find out
which commit broke a build (in any Git repository), you can
use "git bisect" to tell Git which commit worked and which
commit failed, and Git will then run an arbitrary command
(e. g. "make && make check") to see which commit broke the
build.  (NB: Strictly speaking, this only works when the
failure is, eh, monotonous (?), i. e. the build worked for
all commits from last-good to the-one-before-first-failure
and failed for all commits from first-failure to now.)

In the context of build farms, you can make the command more
complex, e. g. set up a container/copy a VM template, ssh
into the instance and run "git checkout $commit && make &&
make check" there.

Tim




^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20  6:54                 ` Tim Landscheidt
@ 2020-12-20  8:17                   ` Pankaj Jangid
  2020-12-20  8:56                     ` Thien-Thi Nguyen
  2020-12-20 20:44                     ` Tim Landscheidt
  0 siblings, 2 replies; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-20  8:17 UTC (permalink / raw)
  To: Tim Landscheidt; +Cc: emacs-devel

Tim Landscheidt <tim@tim-landscheidt.de> writes:

>> Even if that infrastruction is ready, I would certainly want a script to
>> find out the commit that broke the build on my hardware. That reverse
>> choronological loop.
>
>> If someone has that ready then please share. Or may be that could be
>> part of the repository.
>
> I'm not sure what you mean exactly.  If you want to find out
> which commit broke a build (in any Git repository), you can
> use "git bisect" to tell Git which commit worked and which
> commit failed, and Git will then run an arbitrary command
> (e. g. "make && make check") to see which commit broke the
> build.

That might be useful if Git already has the information about good and
bad commits. It helps in narrowing down problems.

> (NB: Strictly speaking, this only works when the
> failure is, eh, monotonous (?), i. e. the build worked for
> all commits from last-good to the-one-before-first-failure
> and failed for all commits from first-failure to now.)

My use-case is very simple. I pull and make bootstrap daily. And on some
odd day if the build breaks, I want to file a bug. If I have the
information which commit broke it then it helps the maintainers. Even if
the failure is monotonous (not sure what to call linear may be), that
may help in narrowing down the issue. It could be inaccurate but still.

> In the context of build farms, you can make the command more
> complex, e. g. set up a container/copy a VM template, ssh
> into the instance and run "git checkout $commit && make &&
> make check" there.

As Lars asked/said, not sure if that is of any help to us.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 21:11   ` Lars Ingebrigtsen
@ 2020-12-20  8:50     ` Michael Albinus
  2020-12-20 14:04       ` Dmitry Gutov
  2020-12-20 17:35       ` Lars Ingebrigtsen
  0 siblings, 2 replies; 69+ messages in thread
From: Michael Albinus @ 2020-12-20  8:50 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: Alan Third, emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> In theory you could install a gitlab runner on each of your VMs and
>> emba could push the compile jobs out to them.
>
> Yeah, that's more work than I'm willing to do.

Maybe somebody will help you? I, for example, am already thinking
about GitLab runners. But I didn't start to implement.

> This reminds me, though -- why isn't EMBA sending out "you broke the
> build" messages to people?  That'd be quite useful, wouldn't it?

Because nobody has implemented it? And it isn't trivial. Since every
commit triggers the whole machinery on Emba, only the *first* commit
which has broken something should send the message.

Imagine, I have broken Tramp (not unlikely, you know). So I will get a
message, and everybody who commits afterwards, would also get such
message, until I have fixed this. Not so nice.

AFAIK, GitLab has no memory. It doesn't remember, whether the previous
job has failed.

And think about several breakages, for example you have broken Gnus in
parallel (this *is* unlikely). GitLab sees only that the job returns
with error, it doesn't know the reason.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20  8:17                   ` Pankaj Jangid
@ 2020-12-20  8:56                     ` Thien-Thi Nguyen
  2020-12-20 10:10                       ` Pankaj Jangid
  2020-12-20 20:44                     ` Tim Landscheidt
  1 sibling, 1 reply; 69+ messages in thread
From: Thien-Thi Nguyen @ 2020-12-20  8:56 UTC (permalink / raw)
  To: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]


() Pankaj Jangid <pankaj@codeisgreat.org>
() Sun, 20 Dec 2020 13:47:46 +0530

   My use-case is very simple. I pull and make bootstrap
   daily. And on some odd day if the build breaks, I want to
   file a bug. If I have the information which commit broke it
   then it helps the maintainers. Even if the failure is
   monotonous (not sure what to call linear may be), that may
   help in narrowing down the issue. It could be inaccurate but
   still.

It sounds like you're talking about "continuous" vs "transient".

What you seek is the first in a series of failures, no matter
the subsequent outcomes, in an attempt to reduce the variables.

No matter the subsequent outcomes, because if they are:

(a) success (transient state) we still might be legitimately
    worried about that failure mode re-appearing;

(b) failure (continuous state) we want to debug w/ as few
    changes as possibile (reducing the variability).

Bottom line: first failure is easier to debug (usually :-D).

Did i miss anything?

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)               ; (2020) Software Libero
   (pcase (context query)               ;       = Dissenso Etico
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 219 bytes --]

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20  8:56                     ` Thien-Thi Nguyen
@ 2020-12-20 10:10                       ` Pankaj Jangid
  0 siblings, 0 replies; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-20 10:10 UTC (permalink / raw)
  To: Thien-Thi Nguyen; +Cc: emacs-devel

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> It sounds like you're talking about "continuous" vs "transient".
>
> What you seek is the first in a series of failures, no matter
> the subsequent outcomes, in an attempt to reduce the variables.
>
> No matter the subsequent outcomes, because if they are:
>
> (a) success (transient state) we still might be legitimately
>     worried about that failure mode re-appearing;
>
> (b) failure (continuous state) we want to debug w/ as few
>     changes as possibile (reducing the variability).
>
> Bottom line: first failure is easier to debug (usually :-D).
>
> Did i miss anything?

You have covered more. :-) For my simple case, I was interested in (b)
only. (a) may be too complex. If a subsequent build is ‘success’ then it
could mean two things - (1) the issue is fixed (2) another commit has
caused it to behave. If we have a good test case coverage then nothing
to worry in case of (a).




^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 19:18             ` Lars Ingebrigtsen
  2020-12-20  2:35               ` Pankaj Jangid
@ 2020-12-20 13:58               ` Dmitry Gutov
  2020-12-20 17:40                 ` Lars Ingebrigtsen
  1 sibling, 1 reply; 69+ messages in thread
From: Dmitry Gutov @ 2020-12-20 13:58 UTC (permalink / raw)
  To: Lars Ingebrigtsen, emacs-devel

On 19.12.2020 21:18, Lars Ingebrigtsen wrote:
> And remember, Dick promised
> to set up a real multi-platform CI thing at some buildbot service or
> other, and we won't need this super-simple one then.

In case you're not being snarky here:

There are free-ish CI solutions which we could use, one example is 
CircleCI (https://circleci.com/open-source/).

But they usually support only a limited number of code hosting 
platforms. E.g. CircleCI only supports GitHub and BitBucket.

So to use it we would have to rely on a GitHub mirror. Can/should we do 
that?

There is one at https://github.com/emacs-mirror/emacs/, but we don't 
know who controls it, so we can't ask them to set up the integration 
from that side.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20  8:50     ` Michael Albinus
@ 2020-12-20 14:04       ` Dmitry Gutov
  2020-12-20 14:11         ` Michael Albinus
  2020-12-20 17:35       ` Lars Ingebrigtsen
  1 sibling, 1 reply; 69+ messages in thread
From: Dmitry Gutov @ 2020-12-20 14:04 UTC (permalink / raw)
  To: Michael Albinus, Lars Ingebrigtsen; +Cc: Alan Third, emacs-devel

On 20.12.2020 10:50, Michael Albinus wrote:
> AFAIK, GitLab has no memory. It doesn't remember, whether the previous
> job has failed.

I think it does. At least, at my place of employment I do receive emails 
with titles like "Pipeline #xx has failed" or "Pipeline #yy has been 
fixed", with, naturally, no emails for pipelines that simply succeed.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 14:04       ` Dmitry Gutov
@ 2020-12-20 14:11         ` Michael Albinus
  2020-12-20 17:32           ` Dmitry Gutov
  0 siblings, 1 reply; 69+ messages in thread
From: Michael Albinus @ 2020-12-20 14:11 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Lars Ingebrigtsen, Alan Third, emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

Hi Dmitry,

> I think it does. At least, at my place of employment I do receive
> emails with titles like "Pipeline #xx has failed" or "Pipeline #yy has
> been fixed", with, naturally, no emails for pipelines that simply
> succeed.

This feature shall work also in our GitLab instance; messages about
failed pipelines shall be sent to <emacs-buildstatus@gnu.org>. This
worked sometimes, but it fails again after a while. No idea what's
wrong.

My statement about missing GitLab memory was, that a failed pipeline
doesn't know, whether the previous pipeline has succeed or failed. With
that knowledge, one could decrease the numbers of messages to be sent.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20  6:39   ` Multi-OS Emacs buildbot? Richard Stallman
@ 2020-12-20 17:30     ` Lars Ingebrigtsen
  0 siblings, 0 replies; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-20 17:30 UTC (permalink / raw)
  To: Richard Stallman; +Cc: emacs-devel

Richard Stallman <rms@gnu.org> writes:

> Do we care about NetBSD enough to arrange to inform people
> who might want to work on that debugging?  I think we should,
> unless informing them is harder than I'd expect.

Yeah, I guess that has some value...  It might possibly entice some
people (who uses Emacs on these systems) to have a whack at fixing some
of the problems?  Perhaps worth a try.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 14:11         ` Michael Albinus
@ 2020-12-20 17:32           ` Dmitry Gutov
  2020-12-22  6:23             ` Toon Claes
  0 siblings, 1 reply; 69+ messages in thread
From: Dmitry Gutov @ 2020-12-20 17:32 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Lars Ingebrigtsen, Alan Third, emacs-devel

On 20.12.2020 16:11, Michael Albinus wrote:

> This feature shall work also in our GitLab instance; messages about
> failed pipelines shall be sent to <emacs-buildstatus@gnu.org>. This
> worked sometimes, but it fails again after a while. No idea what's
> wrong.

Yeah, we should look into it.

> My statement about missing GitLab memory was, that a failed pipeline
> doesn't know, whether the previous pipeline has succeed or failed. With
> that knowledge, one could decrease the numbers of messages to be sent.

But it does. Or else it couldn't provide the email subjects I gave 
example of. "has failed" and "has been fixed" are for the firs failure 
and the first fixed build, correspondingly.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20  8:50     ` Michael Albinus
  2020-12-20 14:04       ` Dmitry Gutov
@ 2020-12-20 17:35       ` Lars Ingebrigtsen
  2020-12-20 18:11         ` Michael Albinus
  1 sibling, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-20 17:35 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Alan Third, emacs-devel

Michael Albinus <michael.albinus@gmx.de> writes:

> Maybe somebody will help you? I, for example, am already thinking
> about GitLab runners. But I didn't start to implement.

The VMs are running on my machine on the desk over there *points*, so
I'm a bit leery of hooking them up to some external system that can
issue commands.

>> This reminds me, though -- why isn't EMBA sending out "you broke the
>> build" messages to people?  That'd be quite useful, wouldn't it?
>
> Because nobody has implemented it? And it isn't trivial. Since every
> commit triggers the whole machinery on Emba, only the *first* commit
> which has broken something should send the message.
>
> Imagine, I have broken Tramp (not unlikely, you know). So I will get a
> message, and everybody who commits afterwards, would also get such
> message, until I have fixed this. Not so nice.
>
> AFAIK, GitLab has no memory. It doesn't remember, whether the previous
> job has failed.

When I've worked with CI systems, they usually are pretty good at
assigning blame for a breakage?  I'd be surprised if GitLab didn't have
such a basic feature.  But I know nothing about GitLab, and the
dashboard at emba isn't very encouraging -- it seems to say that most of
the builds failed because of git locking issues:

----
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
----

So it looks like it needs some work?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 13:58               ` Dmitry Gutov
@ 2020-12-20 17:40                 ` Lars Ingebrigtsen
  2020-12-20 18:03                   ` Dmitry Gutov
  0 siblings, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-20 17:40 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

>> And remember, Dick promised
>> to set up a real multi-platform CI thing at some buildbot service or
>> other, and we won't need this super-simple one then.
>
> In case you're not being snarky here:
>
> There are free-ish CI solutions which we could use, one example is
> CircleCI (https://circleci.com/open-source/).

I am totally being snarky, because (in my experience) setting up CI for
a project takes a bit of work to get to a useful state, especially when
doing CI over a wide range of systems.  If somebody is willing to do the
work, that'd be wonderful, but apparently nobody is?

So instead we have the current state, where hydra builds on Debian, and
that's it.  (Plus Emba, and I don't know what systems it's building on,
really.)

> So to use it we would have to rely on a GitHub mirror. Can/should we
> do that?
>
> There is one at https://github.com/emacs-mirror/emacs/, but we don't
> know who controls it, so we can't ask them to set up the integration
> from that side.

I think anybody's that interested can set this up in whatever way
they want to.  The CI doesn't have to be a GNU/FSF-approved thing.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 17:40                 ` Lars Ingebrigtsen
@ 2020-12-20 18:03                   ` Dmitry Gutov
  2020-12-20 18:08                     ` Lars Ingebrigtsen
  0 siblings, 1 reply; 69+ messages in thread
From: Dmitry Gutov @ 2020-12-20 18:03 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

On 20.12.2020 19:40, Lars Ingebrigtsen wrote:
>> So to use it we would have to rely on a GitHub mirror. Can/should we
>> do that?
>>
>> There is one athttps://github.com/emacs-mirror/emacs/, but we don't
>> know who controls it, so we can't ask them to set up the integration
>> from that side.
> I think anybody's that interested can set this up in whatever way
> they want to.  The CI doesn't have to be a GNU/FSF-approved thing.

True, but such a CI shouldn't (?) be allowed to send emails to Emacs 
committers, I think.

And if CircleCI was, say, integrated with a GitHub clone of the Emacs 
repository, I'm guessing it would only send emails to the GH account 
that does the "cloning" and pushes to that repository. So some 
forwarding would be necessary, and at that junction the information 
about individual pushes (to Savannah's Git) is lost.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 18:03                   ` Dmitry Gutov
@ 2020-12-20 18:08                     ` Lars Ingebrigtsen
  2020-12-20 18:30                       ` Dmitry Gutov
  0 siblings, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-20 18:08 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> True, but such a CI shouldn't (?) be allowed to send emails to Emacs
> committers, I think.

It could send mails to a mailing list, though...  and perhaps people
could opt in to having mails sent to themselves?

> And if CircleCI was, say, integrated with a GitHub clone of the Emacs
> repository, I'm guessing it would only send emails to the GH account
> that does the "cloning" and pushes to that repository. So some
> forwarding would be necessary, and at that junction the information
> about individual pushes (to Savannah's Git) is lost.

Hm.  I don't quite see how that pipeline would lose anything.  I mean, a
commit is a commit.  (I'm assuming the CI works on individual commits,
and not on pushes.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 17:35       ` Lars Ingebrigtsen
@ 2020-12-20 18:11         ` Michael Albinus
  2020-12-22  6:28           ` Lars Ingebrigtsen
  0 siblings, 1 reply; 69+ messages in thread
From: Michael Albinus @ 2020-12-20 18:11 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: Alan Third, emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

Hi Lars,

>> Maybe somebody will help you? I, for example, am already thinking
>> about GitLab runners. But I didn't start to implement.
>
> The VMs are running on my machine on the desk over there *points*, so
> I'm a bit leery of hooking them up to some external system that can
> issue commands.

Yes, it is also my concern. Well, I have a local OpenBSD VM, I also plan
to do first tests installing a GitLab runner there. Just to get a feeling.

Where to put such VMs afterwards is another story. Or you install a
local GitLab instance in your environment, and let the CI jobs run
there?

>> Imagine, I have broken Tramp (not unlikely, you know). So I will get a
>> message, and everybody who commits afterwards, would also get such
>> message, until I have fixed this. Not so nice.
>>
>> AFAIK, GitLab has no memory. It doesn't remember, whether the previous
>> job has failed.
>
> When I've worked with CI systems, they usually are pretty good at
> assigning blame for a breakage?  I'd be surprised if GitLab didn't have
> such a basic feature.

Maybe it has, and I don't know. I know just the trigger "pipeline is
broken" and "pipeline is working"; GitLab sends the messages exactly
when the trigger changes.

For blaming^W informing *people*, we need a more fine-grained trigger.

> But I know nothing about GitLab, and the
> dashboard at emba isn't very encouraging -- it seems to say that most of
> the builds failed because of git locking issues:
>
> ----
> Another git process seems to be running in this repository, e.g.
> an editor opened by 'git commit'. Please make sure all processes
> are terminated then try again. If it still fails, a git process
> may have crashed in this repository earlier:
> ----
>
> So it looks like it needs some work?

Yes, it needs some love. Likely this happens when there are too many
commits in fast sequence; the resulting pipelines block each other. It
is on my TODO to check what's up; perhaps I should give it more
priority.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 18:08                     ` Lars Ingebrigtsen
@ 2020-12-20 18:30                       ` Dmitry Gutov
  2020-12-22  6:27                         ` Lars Ingebrigtsen
  0 siblings, 1 reply; 69+ messages in thread
From: Dmitry Gutov @ 2020-12-20 18:30 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

On 20.12.2020 20:08, Lars Ingebrigtsen wrote:
> Dmitry Gutov<dgutov@yandex.ru>  writes:
> 
>> True, but such a CI shouldn't (?) be allowed to send emails to Emacs
>> committers, I think.
> It could send mails to a mailing list, though...  and perhaps people
> could opt in to having mails sent to themselves?

Either would imply some organizational approval, I think.

Or else I would be re-sending build results emails with URLs pointing to 
CircleCI from my personal email to some official Emacs mailing list, 
and... well, either we as a project approve of that kind of integration, 
or we don't. I certainly don't need anybody coming around and blaming me 
as supporting proprietary platforms this way.

> I'm assuming the CI works on individual commits,
> and not on pushes.

On pushes. It might be not apparent from the web interface, but the 
Gitlab CI skips intermediate commits for, say, efficiency reasons.

It's common practice, IME.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20  8:17                   ` Pankaj Jangid
  2020-12-20  8:56                     ` Thien-Thi Nguyen
@ 2020-12-20 20:44                     ` Tim Landscheidt
  2020-12-21  3:36                       ` Pankaj Jangid
  1 sibling, 1 reply; 69+ messages in thread
From: Tim Landscheidt @ 2020-12-20 20:44 UTC (permalink / raw)
  To: emacs-devel

Pankaj Jangid <pankaj@codeisgreat.org> wrote:

> […]

>> (NB: Strictly speaking, this only works when the
>> failure is, eh, monotonous (?), i. e. the build worked for
>> all commits from last-good to the-one-before-first-failure
>> and failed for all commits from first-failure to now.)

> My use-case is very simple. I pull and make bootstrap daily. And on some
> odd day if the build breaks, I want to file a bug. If I have the
> information which commit broke it then it helps the maintainers. Even if
> the failure is monotonous (not sure what to call linear may be), that
> may help in narrowing down the issue. It could be inaccurate but still.

> […]

Git records the changes to the local repository in a refer-
ence log.  One can access that with "git reflog" and/or "git
rev-parse".  (For example:

| [tim@passepartout ~/src/emacs]$ git rev-parse 'HEAD'
| 2d790c6c57b244447390c023679752243e0049c9
| [tim@passepartout ~/src/emacs]$ git rev-parse 'HEAD@{one.week.ago}'
| 2d790c6c57b244447390c023679752243e0049c9
| [tim@passepartout ~/src/emacs]$ git rev-parse 'HEAD@{two.week.ago}'
| 67a8bdb90c9b5865b7f17290c7135b1a5458c36d
| [tim@passepartout ~/src/emacs]$

suggests that I last fetched the Emacs repository some time
between one and two weeks ago.)

So, untested, if you encounter a bug in HEAD, you could
start the bisection, mark that commit as "bad", then iterate
over HEAD@{1}, HEAD@{2}, etc. marking failures as "bad" com-
mits as you go along, until you find a working commit, mark
that as "good", and then start "git bisect run".

(I hope it is clear that I'm not advocating against using CI
systems; I just want to point out how one can narrow down a
bug if either a repository does not have a CI system attach-
ed or the CI system was not configured to track a particular
bug.)

Tim



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 20:44                     ` Tim Landscheidt
@ 2020-12-21  3:36                       ` Pankaj Jangid
  2020-12-21  3:40                         ` Lars Ingebrigtsen
  0 siblings, 1 reply; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-21  3:36 UTC (permalink / raw)
  To: Tim Landscheidt; +Cc: emacs-devel

Tim Landscheidt <tim@tim-landscheidt.de> writes:

> Git records the changes to the local repository in a refer-
> ence log.  One can access that with "git reflog" and/or "git
> rev-parse".  (For example:
>
> | [tim@passepartout ~/src/emacs]$ git rev-parse 'HEAD'
> | 2d790c6c57b244447390c023679752243e0049c9
> | [tim@passepartout ~/src/emacs]$ git rev-parse 'HEAD@{one.week.ago}'
> | 2d790c6c57b244447390c023679752243e0049c9
> | [tim@passepartout ~/src/emacs]$ git rev-parse 'HEAD@{two.week.ago}'
> | 67a8bdb90c9b5865b7f17290c7135b1a5458c36d
> | [tim@passepartout ~/src/emacs]$
>
> suggests that I last fetched the Emacs repository some time
> between one and two weeks ago.)
>
> So, untested, if you encounter a bug in HEAD, you could
> start the bisection, mark that commit as "bad", then iterate
> over HEAD@{1}, HEAD@{2}, etc. marking failures as "bad" com-
> mits as you go along, until you find a working commit, mark
> that as "good", and then start "git bisect run".

I didn’t know git has a builtin system like this. We were probably on
the same page. I was doing it outside git. Building previous commits in
reverse order and recording somewhere. Thanks for sharing this.

> (I hope it is clear that I'm not advocating against using CI
> systems; I just want to point out how one can narrow down a
> bug if either a repository does not have a CI system attach-
> ed or the CI system was not configured to track a particular
> bug.)

The two approaches have different context. CI is a centralized approach
where commits are comming and the system points out a faulty one and
informs the maintainers. The other approach is distributed. So many
people are already pulling code and building on their systems. Probably
a script can ask them, “*this* particular commit has issues, do you want
to report a bug?”. Of course, the later one covers only the platforms
which are most used and leave out the others. Lars has discovered that
so many *BSD builds are failing. Only a centralized system will be
effective in such a case.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-21  3:36                       ` Pankaj Jangid
@ 2020-12-21  3:40                         ` Lars Ingebrigtsen
  2020-12-21  5:58                           ` Pankaj Jangid
  0 siblings, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-21  3:40 UTC (permalink / raw)
  To: emacs-devel

Pankaj Jangid <pankaj@codeisgreat.org> writes:

> Lars has discovered that so many *BSD builds are failing.

Well, not failing, exactly -- Emacs builds and works on the BSDs, but
there's a varying list of test failures.  Here's the FreeBSD one, for
instance:

SUMMARY OF TEST RESULTS
-----------------------
Files examined: 312
Ran 4271 tests, 4192 results as expected, 14 unexpected, 65 skipped
8 files contained unexpected results:
  src/emacs-module-tests.log
  lisp/wdired-tests.log
  lisp/progmodes/flymake-tests.log
  lisp/net/dbus-tests.log
  lisp/epg-tests.log
  lisp/dired-aux-tests.log
  lisp/cedet/semantic-utest-ia.log
  lisp/net/tramp-tests.log


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-21  3:40                         ` Lars Ingebrigtsen
@ 2020-12-21  5:58                           ` Pankaj Jangid
  0 siblings, 0 replies; 69+ messages in thread
From: Pankaj Jangid @ 2020-12-21  5:58 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> Lars has discovered that so many *BSD builds are failing.
>
> Well, not failing, exactly -- Emacs builds and works on the BSDs, but
> there's a varying list of test failures.  Here's the FreeBSD one, for
> instance:

Oh! sorry about that. I meant that in more generic sense - all kinds of
failures.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 17:32           ` Dmitry Gutov
@ 2020-12-22  6:23             ` Toon Claes
  2020-12-22 17:32               ` Michael Albinus
  0 siblings, 1 reply; 69+ messages in thread
From: Toon Claes @ 2020-12-22  6:23 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Lars Ingebrigtsen, Michael Albinus, Alan Third, emacs-devel

On Sun, Dec 20 2020, Dmitry Gutov wrote:

> On 20.12.2020 16:11, Michael Albinus wrote:
>
>> This feature shall work also in our GitLab instance; messages about
>> failed pipelines shall be sent to <emacs-buildstatus@gnu.org>. This
>> worked sometimes, but it fails again after a while. No idea what's
>> wrong.
>
> Yeah, we should look into it.

If I'm not mistaken we disabled it on purpose. EMBA was set up as a
try-out, it not the official CI, as far as I understood. I've been asked
to disable mails from GitLab CI to emacs-buildstatus@gnu.org because not
everybody following that address would like it.

If people here agree to enable it again, I don't mind doing that.


--
Toon



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 18:30                       ` Dmitry Gutov
@ 2020-12-22  6:27                         ` Lars Ingebrigtsen
  2020-12-22 12:01                           ` Dmitry Gutov
  0 siblings, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-22  6:27 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

>> I'm assuming the CI works on individual commits,
>> and not on pushes.
>
> On pushes. It might be not apparent from the web interface, but the
> Gitlab CI skips intermediate commits for, say, efficiency reasons.
>
> It's common practice, IME.

Right.  But when there's a failure, I thought they bisected back to the
first failing commit in that batch?

Anyway, I see that I was wrong about hydra -- it does builds on Macos,
too?  At least it talks about "darwin"...  but the output from that
build doesn't seem to contain compilation output from the .m files that
I'm used to.  And it doesn't run "make check" there, for some reason?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-20 18:11         ` Michael Albinus
@ 2020-12-22  6:28           ` Lars Ingebrigtsen
  0 siblings, 0 replies; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-22  6:28 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Alan Third, emacs-devel

Michael Albinus <michael.albinus@gmx.de> writes:

> Where to put such VMs afterwards is another story. Or you install a
> local GitLab instance in your environment, and let the CI jobs run
> there?

Sounds like work to me.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-22  6:27                         ` Lars Ingebrigtsen
@ 2020-12-22 12:01                           ` Dmitry Gutov
  0 siblings, 0 replies; 69+ messages in thread
From: Dmitry Gutov @ 2020-12-22 12:01 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel

On 22.12.2020 08:27, Lars Ingebrigtsen wrote:
>> On pushes. It might be not apparent from the web interface, but the
>> Gitlab CI skips intermediate commits for, say, efficiency reasons.
>>
>> It's common practice, IME.
> Right.  But when there's a failure, I thought they bisected back to the
> first failing commit in that batch?

I don't remember any of them doing that.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-19 15:58 Multi-OS Emacs buildbot? Lars Ingebrigtsen
                   ` (5 preceding siblings ...)
  2020-12-19 20:46 ` Daniel Martín
@ 2020-12-22 15:37 ` Nikolay Kudryavtsev
  2020-12-23  7:10   ` Lars Ingebrigtsen
  6 siblings, 1 reply; 69+ messages in thread
From: Nikolay Kudryavtsev @ 2020-12-22 15:37 UTC (permalink / raw)
  To: Lars Ingebrigtsen, emacs-devel

Hi Lars.

A few years ago I wrote a builder script for Windows that installs all 
of the required dependencies for building Emacs:

https://github.com/sg2002/ms-windows-builder.el

This may be of some use to you in your project, if you decide to pursue it.

I had thoughts of refactoring this into a more generic cross-platform 
tool, with the end goal of abstracting away the OS and the OS's package 
manager, but it's a lot of work... Then again, the benefit is pretty clear.

-- 
Best Regards,
Nikolay Kudryavtsev




^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-22  6:23             ` Toon Claes
@ 2020-12-22 17:32               ` Michael Albinus
  2020-12-23  7:11                 ` Lars Ingebrigtsen
  2020-12-25 17:19                 ` Toon Claes
  0 siblings, 2 replies; 69+ messages in thread
From: Michael Albinus @ 2020-12-22 17:32 UTC (permalink / raw)
  To: Toon Claes; +Cc: Lars Ingebrigtsen, emacs-devel, Alan Third, Dmitry Gutov

Toon Claes <toon@iotcl.com> writes:

Hi Toon,

>>> This feature shall work also in our GitLab instance; messages about
>>> failed pipelines shall be sent to <emacs-buildstatus@gnu.org>. This
>>> worked sometimes, but it fails again after a while. No idea what's
>>> wrong.
>>
>> Yeah, we should look into it.
>
> If I'm not mistaken we disabled it on purpose. EMBA was set up as a
> try-out, it not the official CI, as far as I understood. I've been asked
> to disable mails from GitLab CI to emacs-buildstatus@gnu.org because not
> everybody following that address would like it.
>
> If people here agree to enable it again, I don't mind doing that.

I've just checked my mail archive, I couldn't see such a deactivation
message. But this doesn't mean anything but there's no such a message in
my archive ...

However, the mailing list is empty. It shouldn't hurt anybody if
buildstatus messages arrive there.

> Toon

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-22 15:37 ` Nikolay Kudryavtsev
@ 2020-12-23  7:10   ` Lars Ingebrigtsen
  2020-12-23 15:37     ` Eli Zaretskii
  0 siblings, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-23  7:10 UTC (permalink / raw)
  To: Nikolay Kudryavtsev; +Cc: emacs-devel

Nikolay Kudryavtsev <nikolay.kudryavtsev@gmail.com> writes:

> A few years ago I wrote a builder script for Windows that installs all
> of the required dependencies for building Emacs:
>
> https://github.com/sg2002/ms-windows-builder.el
>
> This may be of some use to you in your project, if you decide to pursue it.

Thanks -- it seems like a lot of code to build Emacs, though.  But I
haven't yet attempted doing that under MinGW, so perhaps it's really
that complicated...

> I had thoughts of refactoring this into a more generic cross-platform
> tool, with the end goal of abstracting away the OS and the OS's
> package manager, but it's a lot of work... Then again, the benefit is
> pretty clear.

I've written up instructions on building Emacs on just about all the
operating systems, and it turns out to be pretty easy on most of
them...  if you know which magical incantations to use.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-22 17:32               ` Michael Albinus
@ 2020-12-23  7:11                 ` Lars Ingebrigtsen
  2020-12-23  8:13                   ` Michael Albinus
  2020-12-25 17:19                 ` Toon Claes
  1 sibling, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-23  7:11 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Toon Claes, emacs-devel, Alan Third, Dmitry Gutov

Michael Albinus <michael.albinus@gmx.de> writes:

> However, the mailing list is empty. It shouldn't hurt anybody if
> buildstatus messages arrive there.

Yeah, the hydra build messages, at least, should go there, I think?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-23  7:11                 ` Lars Ingebrigtsen
@ 2020-12-23  8:13                   ` Michael Albinus
  0 siblings, 0 replies; 69+ messages in thread
From: Michael Albinus @ 2020-12-23  8:13 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: Toon Claes, emacs-devel, Alan Third, Dmitry Gutov

Lars Ingebrigtsen <larsi@gnus.org> writes:

Hi Lars,

>> However, the mailing list is empty. It shouldn't hurt anybody if
>> buildstatus messages arrive there.
>
> Yeah, the hydra build messages, at least, should go there, I think?

They did. Then it was broken somehow, and as far as I understand Glenn,
there are problems to reanimate.

Yes, of course, it would be very helpful.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-23  7:10   ` Lars Ingebrigtsen
@ 2020-12-23 15:37     ` Eli Zaretskii
  0 siblings, 0 replies; 69+ messages in thread
From: Eli Zaretskii @ 2020-12-23 15:37 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: nikolay.kudryavtsev, emacs-devel

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 23 Dec 2020 08:10:32 +0100
> Cc: emacs-devel@gnu.org
> 
> > https://github.com/sg2002/ms-windows-builder.el
> >
> > This may be of some use to you in your project, if you decide to pursue it.
> 
> Thanks -- it seems like a lot of code to build Emacs, though.  But I
> haven't yet attempted doing that under MinGW, so perhaps it's really
> that complicated...

You should only install the necessary tools, which is a one-time job.
GCC and Binutils for compiling C programs; Bash, Autoconf, Make, and
Coreutils for running the configury and the build, Texinfo for
producing the Info manuals, Gawk, Grep, gzip...  Thereafter, Emacs
configures and builds on Windows as on any other system, using the
same scripts and Makefiles.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-22 17:32               ` Michael Albinus
  2020-12-23  7:11                 ` Lars Ingebrigtsen
@ 2020-12-25 17:19                 ` Toon Claes
  2020-12-25 17:34                   ` Michael Albinus
  2020-12-25 19:40                   ` Lars Ingebrigtsen
  1 sibling, 2 replies; 69+ messages in thread
From: Toon Claes @ 2020-12-25 17:19 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Lars Ingebrigtsen, Dmitry Gutov, Alan Third, emacs-devel

On Tue, Dec 22 2020, Michael Albinus wrote:

> However, the mailing list is empty. It shouldn't hurt anybody if
> buildstatus messages arrive there.

I've had some issues, but they should be working again.
https://lists.gnu.org/archive/html/emacs-buildstatus/2020-12/msg00001.html


-- 
Toon



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-25 17:19                 ` Toon Claes
@ 2020-12-25 17:34                   ` Michael Albinus
  2020-12-25 19:40                   ` Lars Ingebrigtsen
  1 sibling, 0 replies; 69+ messages in thread
From: Michael Albinus @ 2020-12-25 17:34 UTC (permalink / raw)
  To: Toon Claes; +Cc: Lars Ingebrigtsen, Dmitry Gutov, Alan Third, emacs-devel

Toon Claes <toon@iotcl.com> writes:

Hi Toon,

> I've had some issues, but they should be working again.
> https://lists.gnu.org/archive/html/emacs-buildstatus/2020-12/msg00001.html

Thanks! I've got also the info from Fermin, that the authentication
email has found its way.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-25 17:19                 ` Toon Claes
  2020-12-25 17:34                   ` Michael Albinus
@ 2020-12-25 19:40                   ` Lars Ingebrigtsen
  2020-12-25 20:17                     ` Michael Albinus
  1 sibling, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-25 19:40 UTC (permalink / raw)
  To: Toon Claes; +Cc: Alan Third, Dmitry Gutov, Michael Albinus, emacs-devel

Toon Claes <toon@iotcl.com> writes:

> On Tue, Dec 22 2020, Michael Albinus wrote:
>
>> However, the mailing list is empty. It shouldn't hurt anybody if
>> buildstatus messages arrive there.
>
> I've had some issues, but they should be working again.
> https://lists.gnu.org/archive/html/emacs-buildstatus/2020-12/msg00001.html

Yup.  The messages aren't really that informative, though?  They just
say that something has failed and then there's a link you have to click
to see what.  Would it be possible to include the "make check" summary,
at least?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-25 19:40                   ` Lars Ingebrigtsen
@ 2020-12-25 20:17                     ` Michael Albinus
  2020-12-25 20:22                       ` Lars Ingebrigtsen
  0 siblings, 1 reply; 69+ messages in thread
From: Michael Albinus @ 2020-12-25 20:17 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: Toon Claes, Dmitry Gutov, Alan Third, emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

Hi Lars,

> Toon Claes <toon@iotcl.com> writes:
>
>> On Tue, Dec 22 2020, Michael Albinus wrote:
>>
>>> However, the mailing list is empty. It shouldn't hurt anybody if
>>> buildstatus messages arrive there.
>>
>> I've had some issues, but they should be working again.
>> https://lists.gnu.org/archive/html/emacs-buildstatus/2020-12/msg00001.html
>
> Yup.  The messages aren't really that informative, though?  They just
> say that something has failed and then there's a link you have to click
> to see what.  Would it be possible to include the "make check" summary,
> at least?

You might follow the job link, like
<https://emba.gnu.org/emacs/emacs/-/jobs/9532/raw>. At the bottom line
you'll see the summary.

Btw, I have a private package emba.el, which shows you all (or the last
ones) jobs, and their status. Plus following a link to the log page, in
Emacs.

Needs some polishing, 'tho.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-25 20:17                     ` Michael Albinus
@ 2020-12-25 20:22                       ` Lars Ingebrigtsen
  2020-12-25 20:38                         ` Michael Albinus
  0 siblings, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-25 20:22 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Toon Claes, Dmitry Gutov, Alan Third, emacs-devel

Michael Albinus <michael.albinus@gmx.de> writes:

>> Yup.  The messages aren't really that informative, though?  They just
>> say that something has failed and then there's a link you have to click
>> to see what.

[...]

> You might follow the job link, like
> <https://emba.gnu.org/emacs/emacs/-/jobs/9532/raw>. At the bottom line
> you'll see the summary.

I think that's what I said.  :-)

But the request was:

>>  Would it be possible to include the "make check" summary,
>> at least?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-25 20:22                       ` Lars Ingebrigtsen
@ 2020-12-25 20:38                         ` Michael Albinus
  2020-12-25 20:43                           ` Lars Ingebrigtsen
  0 siblings, 1 reply; 69+ messages in thread
From: Michael Albinus @ 2020-12-25 20:38 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: Toon Claes, Dmitry Gutov, Alan Third, emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

> But the request was:
>
>>>  Would it be possible to include the "make check" summary,
>>> at least?

That's not so simple. Jobs fail for different reasons, and there isn't
always a make check summary.

So you would need to implement kind of log file parser.

And if you start to implement different kind of jobs (for example jobs
for build, jobs for test), it will be even more complicate.

The best approximation I can think of is to add the last N lines of the
log file to the notification message.

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-25 20:38                         ` Michael Albinus
@ 2020-12-25 20:43                           ` Lars Ingebrigtsen
  2020-12-26  8:56                             ` Michael Albinus
  0 siblings, 1 reply; 69+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-25 20:43 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Toon Claes, Dmitry Gutov, Alan Third, emacs-devel

Michael Albinus <michael.albinus@gmx.de> writes:

> That's not so simple. Jobs fail for different reasons, and there isn't
> always a make check summary.
>
> So you would need to implement kind of log file parser.
>
> And if you start to implement different kind of jobs (for example jobs
> for build, jobs for test), it will be even more complicate.

Well, the "make check" output is pretty regular, so it shouldn't be very
difficult to recognise.

> The best approximation I can think of is to add the last N lines of the
> log file to the notification message.

That's better than nothing.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-25 20:43                           ` Lars Ingebrigtsen
@ 2020-12-26  8:56                             ` Michael Albinus
  2020-12-27  9:03                               ` Toon Claes
  0 siblings, 1 reply; 69+ messages in thread
From: Michael Albinus @ 2020-12-26  8:56 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: Toon Claes, Dmitry Gutov, Alan Third, emacs-devel

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> And if you start to implement different kind of jobs (for example jobs
>> for build, jobs for test), it will be even more complicate.
>
> Well, the "make check" output is pretty regular, so it shouldn't be very
> difficult to recognise.

Again, we have already jobs which don't call "make check". And this type
of jobs will increase. See for example
<https://emba.gnu.org/emacs/emacs/-/jobs/9526/raw>.

>> The best approximation I can think of is to add the last N lines of the
>> log file to the notification message.
>
> That's better than nothing.

Yes. Toon, I have no idea whether this would be possible in the mail
template?

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-26  8:56                             ` Michael Albinus
@ 2020-12-27  9:03                               ` Toon Claes
  2020-12-27 10:01                                 ` Michael Albinus
  0 siblings, 1 reply; 69+ messages in thread
From: Toon Claes @ 2020-12-27  9:03 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Lars Ingebrigtsen, Dmitry Gutov, Alan Third, emacs-devel

On Sat, Dec 26 2020, Michael Albinus wrote:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Well, the "make check" output is pretty regular, so it shouldn't be very
>> difficult to recognise.

GitLab is able to parse JUnit [1] formatted test output. It's an XML
format specified by IBM [2].

But as far as I know, it's currently not possible to include these
results in the pipeline emails. So that won't help us much at this
point.

>>> The best approximation I can think of is to add the last N lines of the
>>> log file to the notification message.
>>
>> That's better than nothing.
>
> Yes. Toon, I have no idea whether this would be possible in the mail
> template?

Until a few months ago the last 30 lines of the log were included in the
emails. But unfortunately they were removed [3] for security and
performance reasons. There is a follow-up issue [4] to discuss an
alternative.

We can apply a patch to our installation to revert the above, we did
something like this before. But there's an automatic update set up on
the machine, so we need a mechanism to reapply the patch after updating.


--
Toon

[1] https://docs.gitlab.com/ee/ci/unit_test_reports.html
[2] https://www.ibm.com/support/knowledgecenter/en/SSQ2R2_14.1.0/com.ibm.rsar.analysis.codereview.cobol.doc/topics/cac_useresults_junit.html
[3] https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42395.patch
[4] https://gitlab.com/gitlab-org/gitlab/-/issues/290306



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-27  9:03                               ` Toon Claes
@ 2020-12-27 10:01                                 ` Michael Albinus
  2020-12-30 13:43                                   ` Toon Claes
  0 siblings, 1 reply; 69+ messages in thread
From: Michael Albinus @ 2020-12-27 10:01 UTC (permalink / raw)
  To: Toon Claes; +Cc: Lars Ingebrigtsen, Dmitry Gutov, Alan Third, emacs-devel

Toon Claes <toon@iotcl.com> writes:

>>>> The best approximation I can think of is to add the last N lines of the
>>>> log file to the notification message.
>>>
>>> That's better than nothing.
>>
>> Yes. Toon, I have no idea whether this would be possible in the mail
>> template?
>
> Until a few months ago the last 30 lines of the log were included in the
> emails. But unfortunately they were removed [3] for security and
> performance reasons. There is a follow-up issue [4] to discuss an
> alternative.
>
> We can apply a patch to our installation to revert the above, we did
> something like this before. But there's an automatic update set up on
> the machine, so we need a mechanism to reapply the patch after updating.

It appended something like this

--8<---------------cut here---------------start------------->8---
Stage: test
Name: test-all
Trace: Makefile:319: recipe for target 'check-doit' failed
make[2]: Leaving directory '/builds/emacs/emacs/test'
make[2]: *** [check-doit] Error 1
Makefile:294: recipe for target 'check-expensive' failed
make[1]: *** [check-expensive] Error 2
make[1]: Leaving directory '/builds/emacs/emacs/test'
make: *** [check-expensive] Error 2
Makefile:959: recipe for target 'check-expensive' failed
section_end:1567624331:build_script
section_start:1567624331:after_script
section_end:1567624333:after_script
section_start:1567624333:upload_artifacts_on_failure
section_end:1567624335:upload_artifacts_on_failure
ERROR: Job failed: exit code 1
--8<---------------cut here---------------end--------------->8---

So it is some information that stage test has failed, but it shows only
the Makefile output, and not the more informative test summary. That
doesn't help.

> Toon

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-27 10:01                                 ` Michael Albinus
@ 2020-12-30 13:43                                   ` Toon Claes
  2020-12-30 15:28                                     ` Alan Third
  2020-12-31 14:14                                     ` Michael Albinus
  0 siblings, 2 replies; 69+ messages in thread
From: Toon Claes @ 2020-12-30 13:43 UTC (permalink / raw)
  To: Michael Albinus; +Cc: Lars Ingebrigtsen, Dmitry Gutov, Alan Third, emacs-devel

On Sun, Dec 27 2020, Michael Albinus wrote:

> It appended something like this
>
> ...
>
> So it is some information that stage test has failed, but it shows only
> the Makefile output, and not the more informative test summary. That
> doesn't help.

I'm not familiar with the Emacs test suite, but where else can we get
info about the test results? Is there another log file?


--
Toon



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-30 13:43                                   ` Toon Claes
@ 2020-12-30 15:28                                     ` Alan Third
  2020-12-31 14:14                                     ` Michael Albinus
  1 sibling, 0 replies; 69+ messages in thread
From: Alan Third @ 2020-12-30 15:28 UTC (permalink / raw)
  To: Toon Claes; +Cc: Lars Ingebrigtsen, emacs-devel, Michael Albinus, Dmitry Gutov

On Wed, Dec 30, 2020 at 02:43:32PM +0100, Toon Claes wrote:
> On Sun, Dec 27 2020, Michael Albinus wrote:
> 
> > It appended something like this
> >
> > ...
> >
> > So it is some information that stage test has failed, but it shows only
> > the Makefile output, and not the more informative test summary. That
> > doesn't help.
> 
> I'm not familiar with the Emacs test suite, but where else can we get
> info about the test results? Is there another log file?

As I recall there's one log file per test .el file produced in src,
lib-src and lisp under test.

I wouldn't imagine it would be much work to save them as artifacts?
-- 
Alan Third



^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: Multi-OS Emacs buildbot?
  2020-12-30 13:43                                   ` Toon Claes
  2020-12-30 15:28                                     ` Alan Third
@ 2020-12-31 14:14                                     ` Michael Albinus
  1 sibling, 0 replies; 69+ messages in thread
From: Michael Albinus @ 2020-12-31 14:14 UTC (permalink / raw)
  To: Toon Claes; +Cc: Lars Ingebrigtsen, Dmitry Gutov, Alan Third, emacs-devel

Toon Claes <toon@iotcl.com> writes:

>> So it is some information that stage test has failed, but it shows only
>> the Makefile output, and not the more informative test summary. That
>> doesn't help.
>
> I'm not familiar with the Emacs test suite, but where else can we get
> info about the test results? Is there another log file?

As Alan wrote already, there is one log file per test file. If a test run
(per test file) fails, the log file is shown. See definition of Makefile
variable WRITE_LOG:

--8<---------------cut here---------------start------------->8---
## Save logs, and show logs for failed tests.
WRITE_LOG = > $@ 2>&1 || { STAT=$$?; cat $@; exit $$STAT; }
--8<---------------cut here---------------end--------------->8---

However, since we ran into a job timeout, the job has stopped before we
could see the log file. This is fixed now for process-tests.el, because
every single test is wrapped now by a timeout of 60 seconds, which let
the tests fail in time.

The Makefile knows also special handling when running on hydra, showing
the test progress unconditionally in the Makefile output. Currently, it
has the value:

--8<---------------cut here---------------start------------->8---
ifdef EMACS_HYDRA_CI
## On Hydra, always show logs for certain problematic tests.
lisp/net/tramp-tests.log \
: WRITE_LOG = 2>&1 | tee $@
endif
--8<---------------cut here---------------end--------------->8---

That is, running Tramp tests will always show the progress in the
Makefile output. We could use the same feature for emba if we want.

> Toon

Best regards, Michael.



^ permalink raw reply	[flat|nested] 69+ messages in thread

end of thread, other threads:[~2020-12-31 14:14 UTC | newest]

Thread overview: 69+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-12-19 15:58 Multi-OS Emacs buildbot? Lars Ingebrigtsen
2020-12-19 16:27 ` dick.r.chiang
2020-12-19 17:49   ` Lars Ingebrigtsen
2020-12-19 18:43 ` Pankaj Jangid
2020-12-19 18:48   ` Lars Ingebrigtsen
2020-12-19 18:57     ` Yuri Khan
2020-12-19 19:01     ` Pankaj Jangid
2020-12-19 19:02       ` Lars Ingebrigtsen
2020-12-19 19:04         ` Lars Ingebrigtsen
2020-12-19 19:15           ` Pankaj Jangid
2020-12-19 19:18             ` Lars Ingebrigtsen
2020-12-20  2:35               ` Pankaj Jangid
2020-12-20  4:17                 ` Thien-Thi Nguyen
2020-12-20  5:16                   ` Pankaj Jangid
2020-12-20  6:54                 ` Tim Landscheidt
2020-12-20  8:17                   ` Pankaj Jangid
2020-12-20  8:56                     ` Thien-Thi Nguyen
2020-12-20 10:10                       ` Pankaj Jangid
2020-12-20 20:44                     ` Tim Landscheidt
2020-12-21  3:36                       ` Pankaj Jangid
2020-12-21  3:40                         ` Lars Ingebrigtsen
2020-12-21  5:58                           ` Pankaj Jangid
2020-12-20 13:58               ` Dmitry Gutov
2020-12-20 17:40                 ` Lars Ingebrigtsen
2020-12-20 18:03                   ` Dmitry Gutov
2020-12-20 18:08                     ` Lars Ingebrigtsen
2020-12-20 18:30                       ` Dmitry Gutov
2020-12-22  6:27                         ` Lars Ingebrigtsen
2020-12-22 12:01                           ` Dmitry Gutov
2020-12-19 19:27           ` Mising posts in thread from gwene Was: " Pankaj Jangid
2020-12-19 19:42             ` Lars Ingebrigtsen
2020-12-20  2:40               ` Pankaj Jangid
2020-12-19 19:10 ` Lars Ingebrigtsen
2020-12-19 21:45   ` 'make check' errors on Debian (was: Multi-OS Emacs buildbot?) Basil L. Contovounesios
2020-12-19 21:48     ` 'make check' errors on Debian Lars Ingebrigtsen
2020-12-20  6:39   ` Multi-OS Emacs buildbot? Richard Stallman
2020-12-20 17:30     ` Lars Ingebrigtsen
2020-12-19 20:16 ` Alan Third
2020-12-19 21:11   ` Lars Ingebrigtsen
2020-12-20  8:50     ` Michael Albinus
2020-12-20 14:04       ` Dmitry Gutov
2020-12-20 14:11         ` Michael Albinus
2020-12-20 17:32           ` Dmitry Gutov
2020-12-22  6:23             ` Toon Claes
2020-12-22 17:32               ` Michael Albinus
2020-12-23  7:11                 ` Lars Ingebrigtsen
2020-12-23  8:13                   ` Michael Albinus
2020-12-25 17:19                 ` Toon Claes
2020-12-25 17:34                   ` Michael Albinus
2020-12-25 19:40                   ` Lars Ingebrigtsen
2020-12-25 20:17                     ` Michael Albinus
2020-12-25 20:22                       ` Lars Ingebrigtsen
2020-12-25 20:38                         ` Michael Albinus
2020-12-25 20:43                           ` Lars Ingebrigtsen
2020-12-26  8:56                             ` Michael Albinus
2020-12-27  9:03                               ` Toon Claes
2020-12-27 10:01                                 ` Michael Albinus
2020-12-30 13:43                                   ` Toon Claes
2020-12-30 15:28                                     ` Alan Third
2020-12-31 14:14                                     ` Michael Albinus
2020-12-20 17:35       ` Lars Ingebrigtsen
2020-12-20 18:11         ` Michael Albinus
2020-12-22  6:28           ` Lars Ingebrigtsen
2020-12-19 20:20 ` Michael Albinus
2020-12-19 20:46 ` Daniel Martín
2020-12-19 21:12   ` Lars Ingebrigtsen
2020-12-22 15:37 ` Nikolay Kudryavtsev
2020-12-23  7:10   ` Lars Ingebrigtsen
2020-12-23 15:37     ` Eli Zaretskii

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.