Re: store reference detection (was Re: JARs and reference scanning)

[Mark H Weaver <mhw@netris.org>]

Chris Marusich <cmmarusich@gmail.com> writes:

> Mark H Weaver <mhw@netris.org> writes:
>
>> Hartmut Goebel <h.goebel@crazy-compilers.com> writes:
>>
>>> Am 02.05.2017 um 14:43 schrieb Ludovic Courtès:
>>>> Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:
>>>>
>>>>> Am 27.04.2017 um 15:46 schrieb Ludovic Courtès:
>>>>>> ‘propagated-inputs’ is one way to manually specify run-time references.
>>>>>> It works at the package level and not at the store level—that is, the
>>>>>> store item’s references are unaffected by what ‘propagated-inputs’
>>>>>> contains.  It’s usually enough for our purposes though.
>>>>> I'm not sure if 'propagated-inputs' are enough. For example
>>>>> "python-passlib" as propagated-input python-py-bcrypt, but the later
>>>>> does not show up as reference, requisite nor referrer:
>>>> Right, that’s what I meant by “not at the store level” above.
>>>>
>>>> Ludo’.
>>>  So I propose to add a small text file ".guix-dependencies' to all
>>> language's packages which do not add some kind of references themselves:
>>> Python, Perl, Java, etc.
>>
>> I have thought of doing this in the past, but there's another more
>> difficult problem that would also need to be solved: how to make
>> grafting work for these non-plaintext references.  If grafting doesn't
>> work, there's a good chance that software with known security flaws will
>> continue to be executed.
>
> That's a good thing to keep in mind.  I think that the references we're
> talking about putting into a ".guix-dependencies" file or into an
> uncompressed JAR file are in fact "plaintext" in the sense that these
> files are not using compression, encryption, esoteric encodings, or
> other obfuscations which might defeat the reference scanning or grafting
> mechanisms.  I'm not convinced we need these things (a list of
> dependencies in ".guix-dependencies" or embedded classpaths in JAR
> files), but if we used them, I don't think it would interfere with
> reference scanning or grafting.  Would it?

It would not interfere, but it could have the effect of *hiding*
security problems due to a failure to graft properly.

If there are embedded references that Guix cannot see and therefore are
not transformed by grafting, we are most likely to become aware of this
problem when the referenced store item is reclaimed by the garbage
collector, thus breaking things and prompting us to investigate.  This
is in fact exactly how I discovered another problem of this kind:

  https://bugs.gnu.org/24703

If we create a redundant set of references in another file, then
problems like this could go undetected for a long time.

Adding more 'propagating-inputs' would have the same problem, and cause
other problems as well.

One thing we could do is add a phase to certain build systems, which
would try to use additional methods to detect these hidden references,
and deliberately cause the build to fail in that case, raising the alarm
that grafting would silently fail.

What do you think?

      Mark