Proposal: A new build-system API

Proposal: A new build-system API

[Mája Tomášek]

Hello guix,

I have been lately trying to write a build system or two. And I have
found myself in a weird situation. The current API documentation is
lackluster and the API itself isn't really ergonomic.

First, I need to clarify that I understand that there is a lack of
documentation, it is completely acceptable, but as I didn't really get
the build system API, it was hard for me to grasp the intrices.

Second, I appologize if I am touching on a conversation that already
happened before, but I couldn't find it in the guix devel mailing list,
as there were too many hits when I searched.

Third, this API change would cause breakage of the old, so it isn't
without a compromise, I think it could be done with a deprecation, but
it would be, in my opinion, difficult.

Currently the package and build system APIs don't mesh together well.

(package
        ...
        (inputs list-of-inputs)
        (build-system some-build-system)
        (arguments (list-of-quoted #:key value))
        ...
        )

What would I propose? To design the build system API similiarly as the
package API. With records.

For a simple package, almost nothing would change:

(package
        ...
        (inputs list-of-inputs)
        (build-system (some-build-system))
        ...
        )

But for a more complex package:

(package
        ...
        (inputs list-of-inputs)
        (build-system
                (some-build-system
                        (phases new-list-of-phases)
                        (strip-binaries? #f)))
        ...
        )

This would be the user-facing API, as for the build system developer
API for a simple build system:

(define-build-system some-build-system
                     (inherit gnu-build-system)
                     (name 'some)
                     (description "Some build system")
                     (phases %standard-phases)
                     (strip-binaries? #t)
                     ...
                     (builder (thunked)
                              (build-system->builder this-build-system))
)

The build-system->builder method would generate the build, with
arguments properly adjusted, records translated into keyword arguments,
for a standard build system, the alternative,
would be to provide a (lambda (build-system)
some-code-that-returns-derivation).

The current system is good, but when you need to write your own build
system, you needlessly need to write thins like ungexping arguments,
running gexp->derivation, which is really the system by which the guix
daemon operates, but which could really be abstracted away from the
build system developer. The code writing for a complete build system is repetetive
and complicated.

I don't know, if I have missed some things, this could be a really bad
proposal, and if it is, feel free to ignore it. I have tried to come up
with something new, and I am not the most profficient with either guix
or guile. This was my shot at trying to be helpful in something bigger
than updating packages or fixing small service bugs.

With wishes of the best of luck,
Maya

PS. I appologize for spelling errors.

Re: Proposal: A new build-system API

[zimoun]

Hi,

Thanks for your comments.

On lun., 26 sept. 2022 at 10:39, Mája Tomášek <maya.tomasek@disroot.org> wrote:

> (package
>         ...
>         (inputs list-of-inputs)
>         (build-system some-build-system)
>         (arguments (list-of-quoted #:key value))
>         ...
>         )

[...]

> (package
>         ...
>         (inputs list-of-inputs)
>         (build-system
>                 (some-build-system
>                         (phases new-list-of-phases)
>                         (strip-binaries? #f)))
>         ...
>         )

Hum, from the surface, your proposal is to just move the arguments.
Currently, ’arguments’ are related to ’build-system’ – therefore, I do
not the difference between your proposal and the current situation.


However, this…

> (define-build-system some-build-system
>                      (inherit gnu-build-system)
>                      (name 'some)
>                      (description "Some build system")
>                      (phases %standard-phases)
>                      (strip-binaries? #t)
>                      ...
>                      (builder (thunked)
>                               (build-system->builder this-build-system))
> )
>
> The build-system->builder method would generate the build, with
> arguments properly adjusted, records translated into keyword arguments,
> for a standard build system, the alternative,
> would be to provide a (lambda (build-system)
> some-code-that-returns-derivation).
>
> The current system is good, but when you need to write your own build
> system, you needlessly need to write thins like ungexping arguments,
> running gexp->derivation, which is really the system by which the guix
> daemon operates, but which could really be abstracted away from the
> build system developer. The code writing for a complete build system is repetetive
> and complicated.

…is something different.  Yeah, maybe some glue helpers could ease the
creation of new build-system.  Nevertheless, please note that a
build-system somehow needs some plumbing and I am not convinced that
it would be doable to define a new build-system without diving in some
G-exp here or there.

I agree that the composition of existing bricks is not always
straightforward and it could be improved, eventually. :-)


Cheers,
simon

Re: Proposal: A new build-system API

[Mája Tomášek]

zimoun <zimon.toutoune@gmail.com> writes:

Hi,

Thanks for your reply, I appreciate you taking time to respond to my
silly message <3.

> Hi,
>
> Thanks for your comments.
>
> On lun., 26 sept. 2022 at 10:39, Mája Tomášek <maya.tomasek@disroot.org> wrote:
>
>> (package
>>         ...
>>         (inputs list-of-inputs)
>>         (build-system some-build-system)
>>         (arguments (list-of-quoted #:key value))
>>         ...
>>         )
>
> [...]
>
>> (package
>>         ...
>>         (inputs list-of-inputs)
>>         (build-system
>>                 (some-build-system
>>                         (phases new-list-of-phases)
>>                         (strip-binaries? #f)))
>>         ...
>>         )
>
> Hum, from the surface, your proposal is to just move the arguments.
> Currently, ’arguments’ are related to ’build-system’ – therefore, I do
> not the difference between your proposal and the current situation.

The main point was to define a more ergonomic and sanitized way to
configure the build system. For example, there's currently no guarantee
that #:phases will contain list of functions and the error from that
will be cryptic. Contrast that with the service-configuration API. There
are field sanitizers that ensure that the configuration is propper and
easily one can introspect what the build system accepts and what are the
defaults.

>
> However, this…
>
>> (define-build-system some-build-system
>>                      (inherit gnu-build-system)
>>                      (name 'some)
>>                      (description "Some build system")
>>                      (phases %standard-phases)
>>                      (strip-binaries? #t)
>>                      ...
>>                      (builder (thunked)
>>                               (build-system->builder this-build-system))
>> )
>>
>> The build-system->builder method would generate the build, with
>> arguments properly adjusted, records translated into keyword arguments,
>> for a standard build system, the alternative,
>> would be to provide a (lambda (build-system)
>> some-code-that-returns-derivation).
>>
>> The current system is good, but when you need to write your own build
>> system, you needlessly need to write thins like ungexping arguments,
>> running gexp->derivation, which is really the system by which the guix
>> daemon operates, but which could really be abstracted away from the
>> build system developer. The code writing for a complete build system is repetetive
>> and complicated.
>
> …is something different.  Yeah, maybe some glue helpers could ease the
> creation of new build-system.  Nevertheless, please note that a
> build-system somehow needs some plumbing and I am not convinced that
> it would be doable to define a new build-system without diving in some
> G-exp here or there.
>
> I agree that the composition of existing bricks is not always
> straightforward and it could be improved, eventually. :-)

I understand that some gexps are nescessary, and I really do love gexps
:D. Yet my point was mostly on that as a build system developer, I need
to concern myself with store-monad and derivations, even though I am
more concerned with the package build steps.

Best regards,
Maya

Re: Proposal: A new build-system API

[Philip McGrath]

Hi,

On 9/26/22 14:07, Mája Tomášek wrote:
> For example, there's currently no guarantee
> that #:phases will contain list of functions and the error from that
> will be cryptic. Contrast that with the service-configuration API. There
> are field sanitizers that ensure that the configuration is propper and
> easily one can introspect what the build system accepts and what are the
> defaults.
> 

This in particular is absolutely a valid point. It reminds me that there 
was some discussion a while ago about error checking/reporting generally 
in Guix in which I advocated for adopting contracts à la Racket. I'd 
started putting together a minimal implementation at the time, but I got 
diverted by other things: I'll try to get back to it.

-Philip