wip-refactor-emacs-ui and Hydra interface

wip-refactor-emacs-ui and Hydra interface

[Alex Kost]

Hello, I have pushed 'wip-refactor-emacs-ui' branch which contains emacs
interface for displaying Hydra builds and jobsets.  This 'list'/'info'
interface is absolutely the same as the one for displaying packages and
generations.

The most part of the branch is a step-by-step generalizing of the
list/info interface code, so that any type of data could be displayed
using it.  The changes are huge but mostly internal, though some user
variables were renamed/replaced (I'll mention the changes later).

So if someone wish to try this branch, the following new commands are
available:

M-x guix-hydra-latest-builds  (with C-u you'll be prompted for many things)
M-x guix-hydra-queued-builds
M-x guix-hydra-jobsets

Also you can press "B" key in a package list buffer to display latest
builds for the current package.

The branch is almost ready, only documentation and a couple of internal
things left.

A screenshot made several days ago: http://i.imgur.com/BvhHlkS.png

P.S.  Remember that Hydra is slow and Emacs is single-threaded, so a
synchronous process of receiving data from Hydra may "freeze" your Emacs
(can be break with "C-g" of course) for some time (especially for
complicated queries).

-- 
Alex

Re: wip-refactor-emacs-ui and Hydra interface

[Ludovic Courtès]

Hi Alex!

Sorry for the late reply.  I wanted to take the time to sit back, relax,
and enjoy the new stuff…  and I haven’t been disappointed!  :-)

This is awesome.  Some queries seem to take longer on Hydra than others
(for instance, getting the list of builds of a given jobset is slow,
whereas getting the list of latest builds, queued builds, etc. is pretty
fast.)  At any rate, navigating all this from the familiar interface is
great.

Perhaps one small improvement would be to add links from Package Info
buffers to the corresponding Guix Hydra Build Info?

I haven’t looked at the code yet, but it’s your area of expertise and
I’m confident.  :-)  I skim over it when you think it’s ready; are there
parts you’d like to be reviewed more carefully?

Anyway, let me know what you think and how you’d like to proceed.

Thank you!

Ludo’.

Re: wip-refactor-emacs-ui and Hydra interface

[Alex Kost]

Ludovic Courtès (2015-12-18 01:23 +0300) wrote:

> Hi Alex!
>
> Sorry for the late reply.  I wanted to take the time to sit back, relax,
> and enjoy the new stuff…  and I haven’t been disappointed!  :-)

Great, thanks for giving your time to this!

> This is awesome.  Some queries seem to take longer on Hydra than others
> (for instance, getting the list of builds of a given jobset is slow,
> whereas getting the list of latest builds, queued builds, etc. is pretty
> fast.)

Yes, I also noticed that processing complex queries takes longer time.
Perhaps SQL queries that stand behind this Hydra API are not efficient,
dunno :-)

> Perhaps one small improvement would be to add links from Package Info
> buffers to the corresponding Guix Hydra Build Info?

Do you mean a button that will do the same as "B" key in "Package List"?

Anyway, I think eventually I'll add more various buttons to "Package
Info" buffers (like "Hydra Builds", "Build Log", "Size", "Graph"), and
more keys to "Package List" buffers for the same things.

> I haven’t looked at the code yet, but it’s your area of expertise and
> I’m confident.  :-)  I skim over it when you think it’s ready; are there
> parts you’d like to be reviewed more carefully?

Well, I always feel uncomfortable with commit messages.  But I think the
main part that should be reviewed is documentation (in commit 731f5e4).

As for the changes in code, they are huge (and I would say
"unskimmable"), and I think no one will want to look at them :-)

> Anyway, let me know what you think and how you’d like to proceed.

OK, I have done some final changes and rebased the branch on master, so
(from my point of view) it is ready for merging.  Should I spam the list
with these (37) patches?

-- 
Alex

Re: wip-refactor-emacs-ui and Hydra interface

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 2497 bytes --]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-12-18 01:23 +0300) wrote:
>
>> Hi Alex!
>>
>> Sorry for the late reply.  I wanted to take the time to sit back, relax,
>> and enjoy the new stuff…  and I haven’t been disappointed!  :-)
>
> Great, thanks for giving your time to this!
>
>> This is awesome.  Some queries seem to take longer on Hydra than others
>> (for instance, getting the list of builds of a given jobset is slow,
>> whereas getting the list of latest builds, queued builds, etc. is pretty
>> fast.)
>
> Yes, I also noticed that processing complex queries takes longer time.
> Perhaps SQL queries that stand behind this Hydra API are not efficient,
> dunno :-)
>
>> Perhaps one small improvement would be to add links from Package Info
>> buffers to the corresponding Guix Hydra Build Info?
>
> Do you mean a button that will do the same as "B" key in "Package List"?
>
> Anyway, I think eventually I'll add more various buttons to "Package
> Info" buffers (like "Hydra Builds", "Build Log", "Size", "Graph"), and
> more keys to "Package List" buffers for the same things.
>
>> I haven’t looked at the code yet, but it’s your area of expertise and
>> I’m confident.  :-)  I skim over it when you think it’s ready; are there
>> parts you’d like to be reviewed more carefully?
>
> Well, I always feel uncomfortable with commit messages.  But I think the
> main part that should be reviewed is documentation (in commit 731f5e4).

So, regarding the documentation:

  +@node Emacs Hydra
  +@section Hydra
  +
  +Guix also provides an interface for @code{hydra.gnu.org} (Guix build
  +farm) that allows to display Hydra builds and jobsets.  This interface
  +is mostly the same as the ``list''/``info'' interface for displaying
  +packages and generations (@pxref{Emacs Package Management}).

I would suggest adding a bit of context, like this:

  The continuous integration server at @code{hydra.gnu.org} builds all
  the distribution packages on the supported architectures and serves
  them as substitutes (@pxref{Substitutes}).  Continuous integration is
  currently orchestrated by @uref{https://nixos.org/hydra/, Hydra}.

  This section describes an Emacs interface to query Hydra to know the
  build status of specific packages, discover recent and ongoing builds,
  view build logs, and so on.  This interface is mostly the same as …

WDYT?

I would also add a cross-reference like:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 779 bytes --]

diff --git a/doc/guix.texi b/doc/guix.texi
index 06d70ba..bf523aa 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -1568,7 +1568,9 @@ also result from derivation builds, can be available as substitutes.
 
 The @code{hydra.gnu.org} server is a front-end to a build farm that
 builds packages from the GNU distribution continuously for some
-architectures, and makes them available as substitutes.  This is the
+architectures, and makes them available as substitutes (@pxref{Emacs
+Hydra}, for information on how to query the continuous integration
+server).  This is the
 default source of substitutes; it can be overridden by passing the
 @option{--substitute-urls} option either to @command{guix-daemon}
 (@pxref{daemon-substitute-urls,, @code{guix-daemon --substitute-urls}})

[-- Attachment #3: Type: text/plain, Size: 677 bytes --]


> As for the changes in code, they are huge (and I would say
> "unskimmable"), and I think no one will want to look at them :-)

Well, yeah.  :-)

>> Anyway, let me know what you think and how you’d like to proceed.
>
> OK, I have done some final changes and rebased the branch on master, so
> (from my point of view) it is ready for merging.  Should I spam the list
> with these (37) patches?

I don’t think so.  :-)

AFAICS only elisp code is touched, and the rest of the Emacs code is
supposed to be functionally equivalent, right?

I’m tempted to punt on this one and let you push the whole series.
What do people think?

Thank you!

Ludo’.

Re: wip-refactor-emacs-ui and Hydra interface

[Alex Kost]

Ludovic Courtès (2015-12-22 00:16 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2015-12-18 01:23 +0300) wrote:
[...]
>>> I haven’t looked at the code yet, but it’s your area of expertise and
>>> I’m confident.  :-)  I skim over it when you think it’s ready; are there
>>> parts you’d like to be reviewed more carefully?
>>
>> Well, I always feel uncomfortable with commit messages.  But I think the
>> main part that should be reviewed is documentation (in commit 731f5e4).
>
> So, regarding the documentation:
>
>   +@node Emacs Hydra
>   +@section Hydra
>   +
>   +Guix also provides an interface for @code{hydra.gnu.org} (Guix build
>   +farm) that allows to display Hydra builds and jobsets.  This interface
>   +is mostly the same as the ``list''/``info'' interface for displaying
>   +packages and generations (@pxref{Emacs Package Management}).
>
> I would suggest adding a bit of context, like this:
>
>   The continuous integration server at @code{hydra.gnu.org} builds all
>   the distribution packages on the supported architectures and serves
>   them as substitutes (@pxref{Substitutes}).  Continuous integration is
>   currently orchestrated by @uref{https://nixos.org/hydra/, Hydra}.
>
>   This section describes an Emacs interface to query Hydra to know the
>   build status of specific packages, discover recent and ongoing builds,
>   view build logs, and so on.  This interface is mostly the same as …
>
> WDYT?

Sure!

> I would also add a cross-reference like:
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 06d70ba..bf523aa 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -1568,7 +1568,9 @@ also result from derivation builds, can be available as substitutes.
>  
>  The @code{hydra.gnu.org} server is a front-end to a build farm that
>  builds packages from the GNU distribution continuously for some
> -architectures, and makes them available as substitutes.  This is the
> +architectures, and makes them available as substitutes (@pxref{Emacs
> +Hydra}, for information on how to query the continuous integration
> +server).  This is the
>  default source of substitutes; it can be overridden by passing the
>  @option{--substitute-urls} option either to @command{guix-daemon}
>  (@pxref{daemon-substitute-urls,, @code{guix-daemon --substitute-urls}})

Great, I have fixed the commit locally for these documentation changes.

>>> Anyway, let me know what you think and how you’d like to proceed.
>>
>> OK, I have done some final changes and rebased the branch on master, so
>> (from my point of view) it is ready for merging.  Should I spam the list
>> with these (37) patches?
>
> I don’t think so.  :-)
>
> AFAICS only elisp code is touched, and the rest of the Emacs code is
> supposed to be functionally equivalent, right?

Yes, only elisp code for 'list'/'info' UI was touched.  I'm not sure
what "functionally equivalent" means.  For a user, there shouldn't be
any visible changes.  But the internals (I mean the code itself) was
changed completely.  There are also some changes in user variables (I'm
mentioning them in another message).

> I’m tempted to punt on this one and let you push the whole series.
> What do people think?

I agree with this proposal :-)

-- 
Alex

Re: wip-refactor-emacs-ui and Hydra interface

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-12-22 00:16 +0300) wrote:

[...]

>> I’m tempted to punt on this one and let you push the whole series.
>> What do people think?
>
> I agree with this proposal :-)

Let’s do this!

Thank you,
Ludo’.

Re: wip-refactor-emacs-ui and Hydra interface

[Alex Kost]

Alex Kost (2015-12-11 14:45 +0300) wrote:

> Hello, I have pushed 'wip-refactor-emacs-ui' branch which contains emacs
> interface for displaying Hydra builds and jobsets.  This 'list'/'info'
> interface is absolutely the same as the one for displaying packages and
> generations.
>
> The most part of the branch is a step-by-step generalizing of the
> list/info interface code, so that any type of data could be displayed
> using it.  The changes are huge but mostly internal, though some user
> variables were renamed/replaced (I'll mention the changes later).

Ahem… not "some" but "quite many".  So the following variables (that can
be potentially set by a user) will be changed after merging this branch:

- "guix-list-column-format" and "guix-list-column-value-methods"
  replaced by "guix-ENTRY-TYPE-list-format" variables.

- "guix-info-displayed-params", "guix-info-insert-methods",
  "guix-info-insert-ENTRY-TYPE-function" and
  "guix-package-info-heading-params" replaced by
  "guix-ENTRY-TYPE-info-format" variables (every aspect of displaying
  data in "info" buffers is controlled by these variables now).

- "guix-param-titles" and "guix-list-column-titles" replaced by
  "guix-ENTRY-TYPE-BUFFER-TYPE-titles" variables.

- "guix-list-describe-warning-count" replaced by
  "guix-ENTRY-TYPE-list-describe-warning-count".

- "guix-package-info-fill-heading" replaced by "guix-info-fill".

- "guix-info-fill-column" removed ("fill-column" is used instead).

- "guix-info-ignore-empty-vals" renamed to
  "guix-info-ignore-empty-values".

- "guix-output-name-width" renamed to
  "guix-generation-output-name-width".

- "guix-buffer-name-function" renamed to
  "guix-ui-buffer-name-function".

- "guix-update-after-operation" renamed to
  "guix-ui-update-after-operation".

ENTRY-TYPE means "package", "output" or "generation".
BUFFER-TYPE means "info" or "list".

-- 
Alex

Re: wip-refactor-emacs-ui and Hydra interface

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Alex Kost (2015-12-11 14:45 +0300) wrote:
>
>> Hello, I have pushed 'wip-refactor-emacs-ui' branch which contains emacs
>> interface for displaying Hydra builds and jobsets.  This 'list'/'info'
>> interface is absolutely the same as the one for displaying packages and
>> generations.
>>
>> The most part of the branch is a step-by-step generalizing of the
>> list/info interface code, so that any type of data could be displayed
>> using it.  The changes are huge but mostly internal, though some user
>> variables were renamed/replaced (I'll mention the changes later).
>
> Ahem… not "some" but "quite many".  So the following variables (that can
> be potentially set by a user) will be changed after merging this branch:
>
> - "guix-list-column-format" and "guix-list-column-value-methods"
>   replaced by "guix-ENTRY-TYPE-list-format" variables.
>
> - "guix-info-displayed-params", "guix-info-insert-methods",
>   "guix-info-insert-ENTRY-TYPE-function" and
>   "guix-package-info-heading-params" replaced by
>   "guix-ENTRY-TYPE-info-format" variables (every aspect of displaying
>   data in "info" buffers is controlled by these variables now).
>
> - "guix-param-titles" and "guix-list-column-titles" replaced by
>   "guix-ENTRY-TYPE-BUFFER-TYPE-titles" variables.
>
> - "guix-list-describe-warning-count" replaced by
>   "guix-ENTRY-TYPE-list-describe-warning-count".
>
> - "guix-package-info-fill-heading" replaced by "guix-info-fill".
>
> - "guix-info-fill-column" removed ("fill-column" is used instead).
>
> - "guix-info-ignore-empty-vals" renamed to
>   "guix-info-ignore-empty-values".
>
> - "guix-output-name-width" renamed to
>   "guix-generation-output-name-width".
>
> - "guix-buffer-name-function" renamed to
>   "guix-ui-buffer-name-function".
>
> - "guix-update-after-operation" renamed to
>   "guix-ui-update-after-operation".
>
> ENTRY-TYPE means "package", "output" or "generation".
> BUFFER-TYPE means "info" or "list".

OK.  I think the documentation is updated in the branch, right?

Could you in a separate commit prepare a NEWS entry for 0.9.1 that
succinctly lists these differences?

Thanks,
Ludo’.

Re: wip-refactor-emacs-ui and Hydra interface

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 404 bytes --]

Ludovic Courtès (2015-12-30 18:40 +0300) wrote:

[...]
> OK.  I think the documentation is updated in the branch, right?

Yes (however not all of these variables are/were documented)

> Could you in a separate commit prepare a NEWS entry for 0.9.1 that
> succinctly lists these differences?

Sure.  Is the attached patch sufficient?  If it is, I suppose this
branch can be merged now, OK?


[-- Attachment #2: 0001-Update-NEWS.patch --]
[-- Type: text/x-patch, Size: 1784 bytes --]

From 67069e2beb02a28a138266b9fd97255a6f6f101b Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Thu, 31 Dec 2015 22:42:03 +0300
Subject: [PATCH] Update NEWS

* NEWS: Mention Emacs interface for Hydra and changes in variables.
---
 NEWS | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/NEWS b/NEWS
index 922265c..c35c7d6 100644
--- a/NEWS
+++ b/NEWS
@@ -10,6 +10,37 @@ Copyright © 2013, 2014, 2015 Ludovic Courtès <ludo@gnu.org>
 
 Please send Guix bug reports to bug-guix@gnu.org.
 
+* Changes in 0.9.1 (since 0.9.0)
+
+** Package management
+
+*** Emacs interface for hydra.gnu.org
+*** Changes in Emacs interface variables
+In the following names, BUFFER-TYPE means "info" or "list";
+ENTRY-TYPE means "package", "output" or "generation".
+
+**** Removed
+- guix-info-fill-column
+- guix-info-insert-ENTRY-TYPE-function
+
+**** Renamed
+- guix-info-ignore-empty-vals -> guix-info-ignore-empty-values
+- guix-output-name-width -> guix-generation-output-name-width
+- guix-buffer-name-function -> guix-ui-buffer-name-function
+- guix-update-after-operation -> guix-ui-update-after-operation
+- guix-search-params -> guix-package-search-params
+
+**** Replaced
+- guix-list-column-format, guix-list-column-value-methods ->
+  guix-ENTRY-TYPE-list-format
+- guix-info-displayed-params, guix-info-insert-methods,
+  guix-package-info-heading-params -> guix-ENTRY-TYPE-info-format
+- guix-param-titles, guix-list-column-titles ->
+  guix-ENTRY-TYPE-BUFFER-TYPE-titles
+- guix-list-describe-warning-count ->
+  guix-ENTRY-TYPE-list-describe-warning-count
+- guix-package-info-fill-heading -> guix-info-fill
+
 * Changes in 0.9.0 (since 0.8.3)
 
 ** Package management
-- 
2.6.3

Re: wip-refactor-emacs-ui and Hydra interface

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-12-30 18:40 +0300) wrote:

[...]

>> Could you in a separate commit prepare a NEWS entry for 0.9.1 that
>> succinctly lists these differences?
>
> Sure.  Is the attached patch sufficient?  If it is, I suppose this
> branch can be merged now, OK?

Sure!

> From 67069e2beb02a28a138266b9fd97255a6f6f101b Mon Sep 17 00:00:00 2001
> From: Alex Kost <alezost@gmail.com>
> Date: Thu, 31 Dec 2015 22:42:03 +0300
> Subject: [PATCH] Update NEWS
>
> * NEWS: Mention Emacs interface for Hydra and changes in variables.

Perfect, thanks!

Ludo’.

Re: wip-refactor-emacs-ui and Hydra interface

[Alex Kost]

Ludovic Courtès (2016-01-01 18:53 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2015-12-30 18:40 +0300) wrote:
>
> [...]
>
>>> Could you in a separate commit prepare a NEWS entry for 0.9.1 that
>>> succinctly lists these differences?
>>
>> Sure.  Is the attached patch sufficient?  If it is, I suppose this
>> branch can be merged now, OK?
>
> Sure!

OK, merged, thanks!  Let's see how many bugs I introduced :-)

-- 
Alex