From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 4IbBDhK/iWAfFQAAgWs5BA (envelope-from ) for ; Wed, 28 Apr 2021 22:01:22 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id eHJ+ChK/iWAaVAAAB5/wlQ (envelope-from ) for ; Wed, 28 Apr 2021 20:01:22 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 85F19E0B2 for ; Wed, 28 Apr 2021 22:01:21 +0200 (CEST) Received: from localhost ([::1]:58248 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lbqN6-0008Nb-L2 for larch@yhetil.org; Wed, 28 Apr 2021 16:01:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33762) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lbqMj-0008MD-Rk for guix-devel@gnu.org; Wed, 28 Apr 2021 16:00:57 -0400 Received: from mira.cbaines.net ([212.71.252.8]:36950) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lbqMc-0001qg-UJ for guix-devel@gnu.org; Wed, 28 Apr 2021 16:00:57 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 2E11927BC7C; Wed, 28 Apr 2021 21:00:48 +0100 (BST) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 9bf6de40; Wed, 28 Apr 2021 20:00:47 +0000 (UTC) References: <20210428145941.4bd0dd6f@lubrito> <87y2d2e0j4.fsf@cbaines.net> <20210428162030.2cab4106@lubrito> User-agent: mu4e 1.4.15; emacs 27.1 From: Christopher Baines To: Luciana Lima Brito Subject: Re: Outreachy: Timeline tasks In-reply-to: <20210428162030.2cab4106@lubrito> Date: Wed, 28 Apr 2021 21:00:44 +0100 Message-ID: <87v986dvrn.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1619640081; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=JQfH8LKsktRxf0goNpAYnboH6NDqeOdlvaYZiZKQOHQ=; b=cc64m6b+UgGruj+DzwR5p+KM06kM2IFiD6IHAmdsJfMrgzdxTbpCLvDB3huB24opJb9Uk4 9GAp/nZq2Z6r9nu9zPHCVt8gDjZXh09xgXgrHNhpKDoxGmq4kA+erHw5jfZ00SI3alffsR L2rrks7LT6RztiWkRzKuqFZNgVU6y5We5V0nEWFXUebsjyUfCM21UpsiOFwmASNhbGbr9e DGscwKh6nUWl7bXcmmkNveATyCM5ImxN3O3Y7Qhu6hFo503pbEqZFmmk58tsjde65+5GR6 Z7nw4SqOStCBuiwVNpaV7NRiP+pPhxS2zCDHIWWu0x3zPB9lqAMaCW1UQZ1syA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1619640081; a=rsa-sha256; cv=none; b=Il0I9GCY0AfQxGnjymyU8zETPCLOAFOT4cr9TuiPJxhfNpYtTKOmcnRAMYMIuXryNoI8AF FWxIYkJF6GXPM+LIna09x0yOFILbDMkp8TW5tFWqtUOagS00tyW+68g0zT9qOkmRzc9GvS SbFINMqKDcwSUyUp/ghVnCHfsXO1P70zI/oMG3fJiZIMLHqmYb4jUnyxL3+z8LL8jpWagB xMr6wp+Q7aZ0jxOz+gki/6073mjYxnwn03rlO2sZUry8+cbHPFkOCbSqY1fYnYj1owLvZ1 0bsX2lU/rX08eOPj/CNuRe9Y542ephpd589xEUHEmyYMKJyyaIc6CxjxpCuqVg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -4.55 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 85F19E0B2 X-Spam-Score: -4.55 X-Migadu-Scanner: scn0.migadu.com X-TUID: 2lQoVInrXTvW --=-=-= Content-Type: text/plain Luciana Lima Brito writes: > On Wed, 28 Apr 2021 19:17:51 +0100 > Christopher Baines wrote: > >> So, there's already some code for timing different parts of the data >> loading process, if you look in the job output and search for ", took >> " you should see timings printed out. >> >> These timings being printed out does help, but having the information >> in the log doesn't make it easy to figure out which part is the >> slowest for example. >> >> I'd also not consider this a "one off" thing, the data loading code >> will continue to change as Guix changes and it's performance will >> probably change too. >> >> I've been wondering about visualisations, I remember systemd had a >> feature to plot the systems boot as a image which made seeing which >> parts are slow much easier (here's an example [1]). >> >> 1: https://lizards.opensuse.org/wp-content/uploads/2012/07/plot001.gif > > This is interesting! In fact, one of the things that attracted me was > the possibility to work with visualizations, as I saw that on the > roadmap there is one task related to provide statistics over time). > My master degree is on Information Visualization, so I would appreciate > very much if I could help with that. > > In this matter, we should determine what else, other than time, would > be interesting to see. The visualization should be clear enough about > timing but should also provide information about what could be related > to the delays, such as size of the queries, complexity, the return it > gives... So, first I think we should determine what information we want > to see, then depending on the variables, we choose a suitable way to > present the visualization. I think time is the main thing, since that alone will help to identify which are the slow parts. > About implementing, I'm kind of new to guile and I never built a > visualization in guile, so I don't know which libraries it would take to > build a visual work like that. Depending on what we have, > interactions could be compromised, and instead we would have to work > with charts (static visualizations). Can you tell me more about that? Given the Guix Data Service outputs HTML as well as JSON, it might be possible to build something with HTML. The package history pages sort of visualise the data by adding some grey bars to the table [1]. 1: http://data.guix.gnu.org/repository/1/branch/master/package/guix > And one last thing, a visualization can be simple or can be very > complex.The time for that should be carefully taken into account in > order to not impair the main goal which is the improvements of the slow > parts. Indeed, simplicity is a good thing to keep in mind. >> > About the improvements on the performance of slow parts, it is a >> > little bit abstract for me to see now how to break it in smaller >> > tasks. I do believe that it would require to reformulate some parts >> > of the queries, and as their result may change a bit, tweaks could >> > be required on the code too. My point is, how would I propose an >> > improvement approach if I don't even know what exactly is to be >> > improved? But I imagine that work on this second task is more >> > demanding than the first and will take most of the time of the >> > internship. >> >> As I said before, this part is dependent on deciding where the areas >> for improvement are. Maybe have a look through one of the job logs on >> data.guix.gnu.org and see if you can spot some slow parts? > > I'll look into that and get back to you. Great. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmCJvuxfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XcxWg/+MAFm/S+hrTfvr4RP79UHQNGlBAg2IQLB b+DzjnIW3pFAmeQ/lA7XHORqaI0pqG3P0imvUV3K3QV7M9m6p8NMz3zHRxHkEn7J jMRYWTkH0qHxH1ybeKEqgkCd2o8gbSsRwBlskkbYgi48NzuvFo8Ty2GELoicsnPZ q6SozsOlT1Biix/7gj7+ZH7yZG8eBYPJU/ktEE2Cigl4FQLG/LuxqGrBrX7s6oo3 pmSyaodfz62uF1IXWQ9gI6YDCKv+8Hsi/fAdZ6IY9Tm7J1/7dFO98NRITm81Jqxn HGAN18VpflsQIa9pFSEPZMH/QBE0wsZzIN4vJFBpBZ6tAtYpuXTgccj2baONijUI uW+GNwsi5C4YFadwatRiNz91mIJCcrGtj87YZifktzOLhdQM+lxszqNn654VosEa 0QjF3Ta4HOyiJ6GIrBwvFC7/lYKtpaHenPbnAgNXly4fFZgQFL1o5u3gS8KeTFw6 qIapZIgGxJVUctswHfSdNXZvlSOhAS/UJ4u0z5dTn+fOEpTKzl9oMchd3Z/yBF9W hf/hI8NKKGho5/tKssyM85NiRHygV1aaVmgc9goIA4DPDz6L824PToJrPFPw6Zs2 yVUS7c5THsAs3elucmBsV6+GkF6l+QE7CPp0JNFK9C4YG344sF7mePv1qGQ7O9j+ 2QJ7SgetXVw= =Aw49 -----END PGP SIGNATURE----- --=-=-=--