Building from git

Building from git

[Nicolas Débonnaire]

[-- Attachment #1: Type: text/plain, Size: 742 bytes --]

Hello,
I would like to add a package to guix and send a patch but I'm facing
problems following the chapter "22.1 Building from git"

What I've done so far:

git clone https://git.savannah.gnu.org/git/guix.git


git fetch origin keyring:keyring

guix git authenticate 9edb3f66fd807b096b48283debdcddccfea34bad \

  "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"


guix shell -D guix --pure


./bootstrap


./configure --localstatedir=/var --syscondir=/etc


make


After the make I have the following error.


Error: fontconfig:Didn't find expected font family. Perhaps URW Type 1
fonts need installing?


Then if I run make authenticate as stated in the documentation it
fails with the error: guix: command not found.


Have a nice day

[-- Attachment #2: Type: text/html, Size: 1235 bytes --]

Re: Building from git

[Wojtek Kosior via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 2067 bytes --]

> guix shell -D guix --pure
> 
> [...]
> 
> Then if I run make authenticate as stated in the documentation it
> fails with the error: guix: command not found.

It appears you were still within the guix shell spawned with the first
command when you tried to run `make authenticate`. Guix was not
available inside that shell.

You should `exit` from the running bash session and then run `make
authenticate`. Afterwards you can enter the development shell once
again with `guix shell -D guix --pure`.

Or, you can instead try adding guix to the shell you're spawning, with

    guix shell -D guix guix --pure

I think this will work but haven't checked myself

> Have a nice day

You too :) Happy Hacking!
Wojtek

-- (sig_start)
website: https://koszko.org/koszko.html
fingerprint: E972 7060 E3C5 637C 8A4F  4B42 4BC5 221C 5A79 FD1A
follow me on Fediverse: https://friendica.me/profile/koszko/profile

♥ R29kIGlzIHRoZXJlIGFuZCBsb3ZlcyBtZQ== | ÷ c2luIHNlcGFyYXRlZCBtZSBmcm9tIEhpbQ==
✝ YnV0IEplc3VzIGRpZWQgdG8gc2F2ZSBtZQ== | ? U2hhbGwgSSBiZWNvbWUgSGlzIGZyaWVuZD8=
-- (sig_end)


On Sat, 2 Sep 2023 11:03:14 +0200 Nicolas Débonnaire <n.debonnaire@gmail.com> wrote:

> Hello,
> I would like to add a package to guix and send a patch but I'm facing
> problems following the chapter "22.1 Building from git"
> 
> What I've done so far:
> 
> git clone https://git.savannah.gnu.org/git/guix.git
> 
> 
> git fetch origin keyring:keyring
> 
> guix git authenticate 9edb3f66fd807b096b48283debdcddccfea34bad \
> 
>   "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"
> 
> 
> guix shell -D guix --pure
> 
> 
> ./bootstrap
> 
> 
> ./configure --localstatedir=/var --syscondir=/etc
> 
> 
> make
> 
> 
> After the make I have the following error.
> 
> 
> Error: fontconfig:Didn't find expected font family. Perhaps URW Type 1
> fonts need installing?
> 
> 
> Then if I run make authenticate as stated in the documentation it
> fails with the error: guix: command not found.
> 
> 
> Have a nice day

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: Building from git

[Simon Tournier]

Hi,

On Sat, 02 Sep 2023 at 11:03, Nicolas Débonnaire <n.debonnaire@gmail.com> wrote:

> guix shell -D guix --pure
> ./bootstrap
> ./configure --localstatedir=/var --syscondir=/etc
> make

[...]

> Error: fontconfig:Didn't find expected font family. Perhaps URW Type 1
> fonts need installing?

Hum, weird.  That’s because the documentation seems failing, I guess.

Could you share which Git commit you are building?  And using which Guix
revision, before guix shell, what is the output of “guix describe“?




> Then if I run make authenticate as stated in the documentation it
> fails with the error: guix: command not found.

Yeah, I think that’s expected because ’make’ failed.  Quoting:

        If anything fails, take a look at installation instructions (*note
        Installation::) or send a message to the mailing list
        <guix-devel@gnu.org>.

           From there on, you can authenticate all the commits included in your
        checkout by running:

             make authenticate

However, hum maybe there is bug with that command on pure environment.
The manual is maybe inaccurate.

The Makefile does not run ‘guix git authenticate’ using ./pre-inst-env.
And that’s probably to ensure the source of trust.  If one corrupt the
commit that is built, then ’make authenticate’ would authenticate the
corruption because it would run the corrupted newly built guix command.
Currently, ’make authenticate’ run one guix command that had already
been authenticated.  Well, that’s my understanding.


Cheers,
simon

Re: Building from git

[Bruno Victal]

Hi,

On 2023-09-07 13:06, Simon Tournier wrote:
> Hi,
> 
> On Sat, 02 Sep 2023 at 11:03, Nicolas Débonnaire <n.debonnaire@gmail.com> wrote:
> 
>> guix shell -D guix --pure
>> ./bootstrap
>> ./configure --localstatedir=/var --syscondir=/etc
>> make
> 
> [...]
> 
>> Error: fontconfig:Didn't find expected font family. Perhaps URW Type 1
>> fonts need installing?
> 
> Hum, weird.  That’s because the documentation seems failing, I guess.
> 
> Could you share which Git commit you are building?  And using which Guix
> revision, before guix shell, what is the output of “guix describe“?

I've recently documented this fontconfig issue with URW fonts, see
<https://issues.guix.gnu.org/65741>.


-- 
Furthermore, I consider that nonfree software must be eradicated.

Cheers,
Bruno.

Re: Building from git

[wolf]

[-- Attachment #1: Type: text/plain, Size: 2101 bytes --]

On 2023-09-07 14:06:05 +0200, Simon Tournier wrote:
> Hi,
> 
> On Sat, 02 Sep 2023 at 11:03, Nicolas Débonnaire <n.debonnaire@gmail.com> wrote:
> 
> > guix shell -D guix --pure
> > ./bootstrap
> > ./configure --localstatedir=/var --syscondir=/etc
> > make
> 
> [...]
> 
> > Error: fontconfig:Didn't find expected font family. Perhaps URW Type 1
> > fonts need installing?
> 
> Hum, weird.  That’s because the documentation seems failing, I guess.
> 
> Could you share which Git commit you are building?  And using which Guix
> revision, before guix shell, what is the output of “guix describe“?
> 
> 
> 
> 
> > Then if I run make authenticate as stated in the documentation it
> > fails with the error: guix: command not found.
> 
> Yeah, I think that’s expected because ’make’ failed.  Quoting:
> 
>         If anything fails, take a look at installation instructions (*note
>         Installation::) or send a message to the mailing list
>         <guix-devel@gnu.org>.
> 
>            From there on, you can authenticate all the commits included in your
>         checkout by running:
> 
>              make authenticate
> 
> However, hum maybe there is bug with that command on pure environment.
> The manual is maybe inaccurate.
> 
> The Makefile does not run ‘guix git authenticate’ using ./pre-inst-env.
> And that’s probably to ensure the source of trust.  If one corrupt the
> commit that is built, then ’make authenticate’ would authenticate the
> corruption because it would run the corrupted newly built guix command.
> Currently, ’make authenticate’ run one guix command that had already
> been authenticated.  Well, that’s my understanding.

Hmm, but the recipe for the authenticate rule comes from the (possibly)
compromised source, no?  So the attacker can just modify the recipe instead of
the command going the authentication.  Am I missing something?

> 
> 
> Cheers,
> simon
> 

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Building from git

[Simon Tournier]

Hi,

On Thu, 07 Sep 2023 at 19:45, wolf <wolf@wolfsden.cz> wrote:

>> The Makefile does not run ‘guix git authenticate’ using ./pre-inst-env.
>> And that’s probably to ensure the source of trust.  If one corrupt the
>> commit that is built, then ’make authenticate’ would authenticate the
>> corruption because it would run the corrupted newly built guix command.
>> Currently, ’make authenticate’ run one guix command that had already
>> been authenticated.  Well, that’s my understanding.
>
> Hmm, but the recipe for the authenticate rule comes from the (possibly)
> compromised source, no?  So the attacker can just modify the recipe instead of
> the command going the authentication.  Am I missing something?

Yes, the corruption of Makefile.am can be the corruption I was talking about.

Well, for more explanations one can maybe read:

        [bug#57909] bug#57910: [PATCH] Add link to 'pre-inst-env' from 'installing from git' docs
        Ludovic Courtès <ludo@gnu.org>
        Sat, 24 Sep 2022 17:58:29 +0200
        id:87k05s7oii.fsf_-_@gnu.org
        https://issues.guix.gnu.org//57910
        https://issues.guix.gnu.org/msgid/87k05s7oii.fsf_-_@gnu.org
        https://yhetil.org/guix/87k05s7oii.fsf_-_@gnu.org

        [bug#57909] bug#57910: [PATCH] Add link to 'pre-inst-env' from 'installing from git' docs
        Maxime Devos <maximedevos@telenet.be>
        Sat, 24 Sep 2022 18:23:10 +0200
        id:ec49e6c2-a542-7d95-0d73-10b2816c59d2@telenet.be
        https://issues.guix.gnu.org//57910
        https://issues.guix.gnu.org/msgid/ec49e6c2-a542-7d95-0d73-10b2816c59d2@telenet.be
        https://yhetil.org/guix/ec49e6c2-a542-7d95-0d73-10b2816c59d2@telenet.be

Cheers,
simon

Re: Building from git

[Nicolas Débonnaire]

[-- Attachment #1: Type: text/plain, Size: 2065 bytes --]

Hi,
Looks like it's working.
I was able to complete the "building from git" section of the documentation
after an update of guix.
Thanks everyone.

Le sam. 9 sept. 2023 à 11:01, Simon Tournier <zimon.toutoune@gmail.com> a
écrit :

> Hi,
>
> On Thu, 07 Sep 2023 at 19:45, wolf <wolf@wolfsden.cz> wrote:
>
> >> The Makefile does not run ‘guix git authenticate’ using ./pre-inst-env.
> >> And that’s probably to ensure the source of trust.  If one corrupt the
> >> commit that is built, then ’make authenticate’ would authenticate the
> >> corruption because it would run the corrupted newly built guix command.
> >> Currently, ’make authenticate’ run one guix command that had already
> >> been authenticated.  Well, that’s my understanding.
> >
> > Hmm, but the recipe for the authenticate rule comes from the (possibly)
> > compromised source, no?  So the attacker can just modify the recipe
> instead of
> > the command going the authentication.  Am I missing something?
>
> Yes, the corruption of Makefile.am can be the corruption I was talking
> about.
>
> Well, for more explanations one can maybe read:
>
>         [bug#57909] bug#57910: [PATCH] Add link to 'pre-inst-env' from
> 'installing from git' docs
>         Ludovic Courtès <ludo@gnu.org>
>         Sat, 24 Sep 2022 17:58:29 +0200
>         id:87k05s7oii.fsf_-_@gnu.org
>         https://issues.guix.gnu.org//57910
>         https://issues.guix.gnu.org/msgid/87k05s7oii.fsf_-_@gnu.org
>         https://yhetil.org/guix/87k05s7oii.fsf_-_@gnu.org
>
>         [bug#57909] bug#57910: [PATCH] Add link to 'pre-inst-env' from
> 'installing from git' docs
>         Maxime Devos <maximedevos@telenet.be>
>         Sat, 24 Sep 2022 18:23:10 +0200
>         id:ec49e6c2-a542-7d95-0d73-10b2816c59d2@telenet.be
>         https://issues.guix.gnu.org//57910
>
> https://issues.guix.gnu.org/msgid/ec49e6c2-a542-7d95-0d73-10b2816c59d2@telenet.be
>
> https://yhetil.org/guix/ec49e6c2-a542-7d95-0d73-10b2816c59d2@telenet.be
>
> Cheers,
> simon
>

[-- Attachment #2: Type: text/html, Size: 3582 bytes --]

Re: Building from git

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 508 bytes --]

Hi,

wolf <wolf@wolfsden.cz> writes:

> Hmm, but the recipe for the authenticate rule comes from the (possibly)
> compromised source, no?  So the attacker can just modify the recipe instead of
> the command going the authentication.  Am I missing something?

You can use a previously trusted guix to do the authentication.  `make
authenticate` is here for committers to check that their commits are all
properly signed before pushing (it's used as a pre-push hook).

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Building from git

[Wojtek Kosior via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 2850 bytes --]

Hello Josselin

> wolf <wolf@wolfsden.cz> writes:
> 
> > Hmm, but the recipe for the authenticate rule comes from the (possibly)
> > compromised source, no?  So the attacker can just modify the recipe instead of
> > the command going the authentication.  Am I missing something?  
> 
> You can use a previously trusted guix to do the authentication.  `make
> authenticate` is here for committers to check that their commits are all
> properly signed before pushing (it's used as a pre-push hook).

From my understanding of the documentation, `make authenticate` is not
just for committers but for all people who do a `git pull` in Guix tree
and want to verify that the newly pulled commits do come from the
committers. It it is not the case, then the documentation should
probably be modified to make it clear.

The recipe is not from an untrusted source mecause the Makefile is not
tracked by git. Rather, it gets generated when first building Guix. And
— as the documentation instructs — the initial checkout gets
authenticated with `guix git authenticate` rather than with `make
authenticate` so it can't get compromised that easily.

Had someone managed to serve us a commit that adds another Makefile
with a backdoor, git would report a conflict upon pulling. I believe
this is what the implementors had in mind. Please clarify if this is
wrong.

I do see 1 loophole here, though. One could serve a compromised
makefile under the name "GNUmakefile" and `make authenticate` would
happily choose it over the non-compromised "Makefile". I was planning
to start a new thread about it for some time... but this one seems like
a just as appropriate place to mention the issue.

It shouldn't be hard to fix. It boils down to having ./configure create
a GNUmakefile as well. Perhaps as a symlink to the original Makefile?

Best,
Wojtek

-- (sig_start)
website: https://koszko.org/koszko.html
fingerprint: E972 7060 E3C5 637C 8A4F  4B42 4BC5 221C 5A79 FD1A
follow me on Fediverse: https://friendica.me/profile/koszko/profile

♥ R29kIGlzIHRoZXJlIGFuZCBsb3ZlcyBtZQ== | ÷ c2luIHNlcGFyYXRlZCBtZSBmcm9tIEhpbQ==
✝ YnV0IEplc3VzIGRpZWQgdG8gc2F2ZSBtZQ== | ? U2hhbGwgSSBiZWNvbWUgSGlzIGZyaWVuZD8=
-- (sig_end)


On Fri, 08 Sep 2023 11:10:37 +0200 Josselin Poiret <dev@jpoiret.xyz> wrote:

> Hi,
> 
> wolf <wolf@wolfsden.cz> writes:
> 
> > Hmm, but the recipe for the authenticate rule comes from the (possibly)
> > compromised source, no?  So the attacker can just modify the recipe instead of
> > the command going the authentication.  Am I missing something?  
> 
> You can use a previously trusted guix to do the authentication.  `make
> authenticate` is here for committers to check that their commits are all
> properly signed before pushing (it's used as a pre-push hook).
> 
> Best,

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: Building from git

[wolf]

[-- Attachment #1: Type: text/plain, Size: 5437 bytes --]

On 2023-09-08 11:47:56 +0200, Wojtek Kosior wrote:
> Hello Josselin
> 
> > wolf <wolf@wolfsden.cz> writes:
> > 
> > > Hmm, but the recipe for the authenticate rule comes from the (possibly)
> > > compromised source, no?  So the attacker can just modify the recipe instead of
> > > the command going the authentication.  Am I missing something?  
> > 
> > You can use a previously trusted guix to do the authentication.  `make
> > authenticate` is here for committers to check that their commits are all
> > properly signed before pushing (it's used as a pre-push hook).
> 
> From my understanding of the documentation, `make authenticate` is not
> just for committers but for all people who do a `git pull` in Guix tree
> and want to verify that the newly pulled commits do come from the
> committers. It it is not the case, then the documentation should
> probably be modified to make it clear.
> 
> The recipe is not from an untrusted source mecause the Makefile is not
> tracked by git. Rather, it gets generated when first building Guix. And
> — as the documentation instructs — the initial checkout gets
> authenticated with `guix git authenticate` rather than with `make
> authenticate` so it can't get compromised that easily.
> 
> Had someone managed to serve us a commit that adds another Makefile
> with a backdoor, git would report a conflict upon pulling. I believe
> this is what the implementors had in mind. Please clarify if this is
> wrong.

Yes, I believe this reasoning is wrong.  Even ignoring the fact that people
might run git clean or use worktrees, you can just attack the Makefile.am.  I
created a new commit in my checkout:

    commit b3b378ad8f725f16be0602113e7f2d2afd89a920 (HEAD -> master)
    Author: x <y@z>
    Date:   Fri Sep 8 11:04:44 2023 +0000
    
        this commit is so not signed and valid
    
    diff --git a/Makefile.am b/Makefile.am
    index 922913355c..e5f7c37491 100644
    --- a/Makefile.am
    +++ b/Makefile.am
    @@ -883,10 +883,7 @@ channel_intro_signer = BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA
     GUIX_GIT_KEYRING = origin/keyring
     authenticate:
            $(AM_V_at)echo "Authenticating Git checkout..." ;       \
    -       guix git authenticate                                   \
    -           --keyring=$(GUIX_GIT_KEYRING)                       \
    -           --cache-key=channels/guix --stats                   \
    -           "$(channel_intro_commit)" "$(channel_intro_signer)"
    +       echo "Don't worry, your checkout is just fine... :)"
     
     # Assuming Guix is already installed and the daemon is up and running, this
     # rule builds from $(srcdir), creating and building derivations.

guix git authenticate fails, as expected:

    Authenticating commits 9edb3f6 to b3b378a (1 new commits)...
    [##############################################################################]guix git: error: commit b3b378ad8f725f16be0602113e7f2d2afd89a920 lacks a signature

The missing new line after ] is somewhat meh, but it correctly fails.  However
make authenticate does pass:

    $ guix shell -D guix guix --pure -- make authenticate
     cd . && /bin/sh /home/wolf/src/guix/build-aux/missing automake-1.16 --gnu Makefile
    Makefile.am:896: warning: AM_GNU_GETTEXT used but 'po' not in SUBDIRS
     cd . && /bin/sh ./config.status Makefile depfiles
    config.status: creating Makefile
    config.status: executing depfiles commands
    Authenticating Git checkout...
    Don't worry, your checkout is just fine... :)

I mean, if make authenticate is just for the convenience of the committers, then
this is completely fine.  But the documentation does not currently read that
way.

> 
> I do see 1 loophole here, though. One could serve a compromised
> makefile under the name "GNUmakefile" and `make authenticate` would
> happily choose it over the non-compromised "Makefile". I was planning
> to start a new thread about it for some time... but this one seems like
> a just as appropriate place to mention the issue.
> 
> It shouldn't be hard to fix. It boils down to having ./configure create
> a GNUmakefile as well. Perhaps as a symlink to the original Makefile?
> 
> Best,
> Wojtek
> 
> -- (sig_start)
> website: https://koszko.org/koszko.html
> fingerprint: E972 7060 E3C5 637C 8A4F  4B42 4BC5 221C 5A79 FD1A
> follow me on Fediverse: https://friendica.me/profile/koszko/profile
> 
> ♥ R29kIGlzIHRoZXJlIGFuZCBsb3ZlcyBtZQ== | ÷ c2luIHNlcGFyYXRlZCBtZSBmcm9tIEhpbQ==
> ✝ YnV0IEplc3VzIGRpZWQgdG8gc2F2ZSBtZQ== | ? U2hhbGwgSSBiZWNvbWUgSGlzIGZyaWVuZD8=
> -- (sig_end)
> 
> 
> On Fri, 08 Sep 2023 11:10:37 +0200 Josselin Poiret <dev@jpoiret.xyz> wrote:
> 
> > Hi,
> > 
> > wolf <wolf@wolfsden.cz> writes:
> > 
> > > Hmm, but the recipe for the authenticate rule comes from the (possibly)
> > > compromised source, no?  So the attacker can just modify the recipe instead of
> > > the command going the authentication.  Am I missing something?  
> > 
> > You can use a previously trusted guix to do the authentication.  `make
> > authenticate` is here for committers to check that their commits are all
> > properly signed before pushing (it's used as a pre-push hook).
> > 
> > Best,

W.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Building from git

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 1901 bytes --]

Hi both of you (I'm replying to both at the same time),

wolf <wolf@wolfsden.cz> writes:

On 2023-09-08 11:47:56 +0200, Wojtek Kosior wrote:
> Hello Josselin
> 
> > wolf <wolf@wolfsden.cz> writes:
> > 
> > > Hmm, but the recipe for the authenticate rule comes from the (possibly)
> > > compromised source, no?  So the attacker can just modify the recipe instead of
> > > the command going the authentication.  Am I missing something?  
> > 
> > You can use a previously trusted guix to do the authentication.  `make
> > authenticate` is here for committers to check that their commits are all
> > properly signed before pushing (it's used as a pre-push hook).
> 
> From my understanding of the documentation, `make authenticate` is not
> just for committers but for all people who do a `git pull` in Guix tree
> and want to verify that the newly pulled commits do come from the
> committers. It it is not the case, then the documentation should
> probably be modified to make it clear.
> 
> The recipe is not from an untrusted source mecause the Makefile is not
> tracked by git. Rather, it gets generated when first building Guix. And
> — as the documentation instructs — the initial checkout gets
> authenticated with `guix git authenticate` rather than with `make
> authenticate` so it can't get compromised that easily.

If you've already authenticated the initial check-out, what is the point
of `make authenticate` then?  Maybe the manual isn't that clear, but as
wolf points out `make authenticate` itself cannot be a guarantee as it
requires trust in the Makefiles, creating a chicken-and-egg problem
.
> I mean, if make authenticate is just for the convenience of the committers, then
> this is completely fine.  But the documentation does not currently read that
> way.

Yes, I believe this should then be clarified.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]