Re: missing patch for texlive-bin (e77412362f)

[Timothy Sample <samplet@ngyro.com>]

Hi,

zimoun <zimon.toutoune@gmail.com> writes:

> On Thu, 03 Feb 2022 at 10:46, Timothy Sample <samplet@ngyro.com> wrote:
>
>> The bad news is that 0.75 is not there.  At first I was going to
>> apologize for the shortcomings of the sampling approach... until I
>> realized you are trying to trick me!  ;)  Unless I’m misreading the Git
>> history, that patch appeared and disappeared on core-updates and was
>> never part of master.
>
> Because of the good news, the same could be applied for these patches,
> no? [...]  [I]t “only” misses to dissamble this data and add an entry
> to the database, no?

Yes.  I could add that commit to the database, evaluate it, and load all
the sources.  I’m inclined not to, but I’m open to being convinced.  (I
really like how simple the current system is conceptually.)

> I miss what you mean by «was never part of master».  After the merge,
> what was core-updates and what was master is somehow indistinguishable,
> no?  Or are you walking only to first-parent after merge commit?  Well,
> Git history and sorting leads to headache; as git-log doc shows. :-)
>
> I think it is fine to simplify “complex” history with a sampling
> considering only first-parent walk.

That’s about it.  To my mind, “The History of the Guix Package Database”
*is* the first parent walk that you describe.  Of course, that’s just my
feeling.  There’s lots of room for disagreement there.  Basically, if
you can’t reach a commit by starting at 1.0.0 and running ‘guix pull’
without arguments, it doesn’t exist!

>> That being said, coverage is not perfect.  The most obvious problem (to
>> me) is the sampling approach.  Surely there are sources that are missed
>> by only examining one commit per week.  This can be checked and fixed by
>> using data from the Guix Data Service, which has data from essentially
>> every Guix commit.
>
> No, the Data Service and even Cuirass are using a sampling approach too;
> they do not process all the commits.
>
> Cuirass uses a «every 5 minutes» approach; please CI savvy people
> correct me if I mistake.  The Data Service uses a «batch guix-commits»
> approach; more details in this thread [1].

Thanks for letting me know about this.  Maybe I’m too optimistic!
Either way, the Data Service data is likely much more accurate than PoG,
and could still help build confidence.

> Well, the coverage is twofold, IMHO.
>
>  1. preserve what is currently entering in Guix
>  2. archive what was available in Guix
>
> About #1, the main mechanism are sources.json, “guix lint”, and update
> disarchive-db (now done by CI).  What is missed should be fixed by #2.
>
> About #2, it is hard to fix all the issues at once.  One commit per week
> already provides a good view to spot some problems.  Somehow, process
> all the commits just means burn more CPU; it seems “easy” once the
> infrastructure is in-place, no?

More or less.  Burning CPU is definitely the main thing holding back
processing all the commits, but it would likely take a bit of effort to
get code that works for around one hundred commits to work for
thousands.  The second thing is diminishing returns.  Burning *way* more
CPU to track down a couple sources feels a little wasteful to me.

For me, the scope of PoG is perfect the way it is.  It’s big enough to
be useful, but not so big to be overwhelming.  There are lots of serious
problems to be addressed, too.

That being said, I’m willing to change things.  A lot of this is just my
gut feeling.  :)  If everyone else is clamouring to have more commits
or to track core-updates or whatever, I’m all ears!


-- Tim