Re: Outreachy: Timeline tasks

[Christopher Baines <mail@cbaines.net>]

[-- Attachment #1: Type: text/plain, Size: 3555 bytes --]


Luciana Lima Brito <lubrito@posteo.net> writes:

> On Wed, 28 Apr 2021 19:17:51 +0100
> Christopher Baines <mail@cbaines.net> wrote:
>
>> So, there's already some code for timing different parts of the data
>> loading process, if you look in the job output and search for ", took
>> " you should see timings printed out.
>>
>> These timings being printed out does help, but having the information
>> in the log doesn't make it easy to figure out which part is the
>> slowest for example.
>>
>> I'd also not consider this a "one off" thing, the data loading code
>> will continue to change as Guix changes and it's performance will
>> probably change too.
>>
>> I've been wondering about visualisations, I remember systemd had a
>> feature to plot the systems boot as a image which made seeing which
>> parts are slow much easier (here's an example [1]).
>>
>> 1: https://lizards.opensuse.org/wp-content/uploads/2012/07/plot001.gif
>
> This is interesting! In fact, one of the things that attracted me was
> the possibility to work with visualizations, as I saw that on the
> roadmap there is one task related to provide statistics over time).
> My master degree is on Information Visualization, so I would appreciate
> very much if I could help with that.
>
> In this matter, we should determine what else, other than time, would
> be interesting to see. The visualization should be clear enough about
> timing but should also provide information about what could be related
> to the delays, such as size of the queries, complexity, the return it
> gives... So, first I think we should determine what information we want
> to see, then depending on the variables, we choose a suitable way to
> present the visualization.

I think time is the main thing, since that alone will help to identify
which are the slow parts.

> About implementing, I'm kind of new to guile and I never built a
> visualization in guile, so I don't know which libraries it would take to
> build a visual work like that. Depending on what we have,
> interactions could be compromised, and instead we would have to work
> with charts (static visualizations). Can you tell me more about that?

Given the Guix Data Service outputs HTML as well as JSON, it might be
possible to build something with HTML. The package history pages sort of
visualise the data by adding some grey bars to the table [1].

1: http://data.guix.gnu.org/repository/1/branch/master/package/guix

> And one last thing, a visualization can be simple or can be very
> complex.The time for that should be carefully taken into account in
> order to not impair the main goal which is the improvements of the slow
> parts.

Indeed, simplicity is a good thing to keep in mind.

>> > About the improvements on the performance of slow parts, it is a
>> > little bit abstract for me to see now how to break it in smaller
>> > tasks. I do believe that it would require to reformulate some parts
>> > of the queries, and as their result may change a bit, tweaks could
>> > be required on the code too. My point is, how would I propose an
>> > improvement approach if I don't even know what exactly is to be
>> > improved? But I imagine that work on this second task is more
>> > demanding than the first and will take most of the time of the
>> > internship.
>>
>> As I said before, this part is dependent on deciding where the areas
>> for improvement are. Maybe have a look through one of the job logs on
>> data.guix.gnu.org and see if you can spot some slow parts?
>
> I'll look into that and get back to you.

Great.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]