From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:8:6d80::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id uGv6J661iWBlOAEAgWs5BA (envelope-from ) for ; Wed, 28 Apr 2021 21:21:18 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id EMKgI661iWBvTQAAbx9fmQ (envelope-from ) for ; Wed, 28 Apr 2021 19:21:18 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 52BCD143D4 for ; Wed, 28 Apr 2021 21:21:18 +0200 (CEST) Received: from localhost ([::1]:33414 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lbpkL-0007rP-B1 for larch@yhetil.org; Wed, 28 Apr 2021 15:21:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50738) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lbpjr-0007nB-2g for guix-devel@gnu.org; Wed, 28 Apr 2021 15:20:47 -0400 Received: from mout02.posteo.de ([185.67.36.66]:56437) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lbpjn-0005KJ-Lb for guix-devel@gnu.org; Wed, 28 Apr 2021 15:20:46 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 6F3342400FC for ; Wed, 28 Apr 2021 21:20:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1619637640; bh=0fcDCmhfVTJwBRN5UVgvclTZYodyPLsNIu6b+HTqBeo=; h=Date:From:To:Cc:Subject:From; b=VD48T3kSJG8iMIEcEDEGZ94KmO4XtZmWCa6A7JN2bg25sfd87lDO9nxWxQ/PIa0gI javIsKtADSs9RF6FDsH9c6qEpDcckKjtM8+0tsgPDZ6/PWEZlSZgh/YuxIFHvpCTtU o0JAO26ROucgzSaCRZ+jMWjpGC+xeRRh7CNOMNIsYdayGAUwPEgGRnvnptJ3+JLCVG isFr5ihTMDaWb6rzp+cLRocentSQVqSr7JUq+mL9ycHIDuhd4iWhitp3eYqxsQZlLw iSu6YUalgPCHUAjBkZbQ0N3rvxwGNe0zz0N/Wii1yw37LMFVGLwhHup2hqOhEBkz0b 15VmHUnimz9CA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4FVpPf2XrGz6tmQ; Wed, 28 Apr 2021 21:20:37 +0200 (CEST) Date: Wed, 28 Apr 2021 19:20:30 +0000 From: Luciana Lima Brito To: Christopher Baines Subject: Re: Outreachy: Timeline tasks Message-ID: <20210428162030.2cab4106@lubrito> In-Reply-To: <87y2d2e0j4.fsf@cbaines.net> References: <20210428145941.4bd0dd6f@lubrito> <87y2d2e0j4.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Received-SPF: pass client-ip=185.67.36.66; envelope-from=lubrito@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1619637678; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=oCquXwB20hZKcj+0ICYP2H+ijkJRA7RFFGxyERI59i4=; b=cONaFi1/w8A3k6UJRZfLnGrEzBEphzPSFMIOjm7ndyIRkVFULGUovPoPsEdT909n/6PG62 InJjVeSZbpyA7nuElfUcgZAY6m2+cVqwrWfKNPa8GEF/Kb7TLZMeH0R2tWWqoVUdfCitQa 4zv9oSHG/zo6lJbEYoKZxnUqnoAFzswU18hS6+yMFASzl/k2NFSkawCxZL2APeVYshpE6Z 3gATuCastyID72ef4Y2UvSrfqGQ3DCpJ6ADtkqva2BeTtnPv1si5Z2fNzjrZf9ieIwQXJX bb/BI6i+26KV4d5k2+dBmf94FsCGvvZTeioHLccLPSfAudNbrHKxm54GU8KT3w== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1619637678; a=rsa-sha256; cv=none; b=aTCPRGnBmCik0IEaYPX7EVPmUEkoKYbzSIh6s80gJdaNEQC6erx7ntusQ3BU9ls6ANPiqX KUxE20kgss5zPFyjen+M58pzseMD5aVXVzXDP1/xWolj21aupxCzp2FCuRdcHFaT8BhpFc oJR/oH+nDbOYzz5SC0iohxGpmcjGMYDsAHPCCZpuD40JXGEJ6GfPS06f3UmXdS+/gP+BH5 hPlXfWRgiRRtU6Q2vjtD/N0+//S8xdpFIs0BKs2wgevkNaR5sHEkBWbEA92wNyZyIG9OY2 eTtxkLcRR4WoV4zMEEHXXuCowjeG3gREG20ZRj+mcVf9Aqokt0b1P48ccP7CVg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=VD48T3kS; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -2.65 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=VD48T3kS; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 52BCD143D4 X-Spam-Score: -2.65 X-Migadu-Scanner: scn0.migadu.com X-TUID: SNpT5nTUWOTD On Wed, 28 Apr 2021 19:17:51 +0100 Christopher Baines wrote: > So, there's already some code for timing different parts of the data > loading process, if you look in the job output and search for ", took > " you should see timings printed out. > > These timings being printed out does help, but having the information > in the log doesn't make it easy to figure out which part is the > slowest for example. > > I'd also not consider this a "one off" thing, the data loading code > will continue to change as Guix changes and it's performance will > probably change too. > > I've been wondering about visualisations, I remember systemd had a > feature to plot the systems boot as a image which made seeing which > parts are slow much easier (here's an example [1]). > > 1: https://lizards.opensuse.org/wp-content/uploads/2012/07/plot001.gif This is interesting! In fact, one of the things that attracted me was the possibility to work with visualizations, as I saw that on the roadmap there is one task related to provide statistics over time). My master degree is on Information Visualization, so I would appreciate very much if I could help with that. In this matter, we should determine what else, other than time, would be interesting to see. The visualization should be clear enough about timing but should also provide information about what could be related to the delays, such as size of the queries, complexity, the return it gives... So, first I think we should determine what information we want to see, then depending on the variables, we choose a suitable way to present the visualization. About implementing, I'm kind of new to guile and I never built a visualization in guile, so I don't know which libraries it would take to build a visual work like that. Depending on what we have, interactions could be compromised, and instead we would have to work with charts (static visualizations). Can you tell me more about that? And one last thing, a visualization can be simple or can be very complex.The time for that should be carefully taken into account in order to not impair the main goal which is the improvements of the slow parts. > > > About the improvements on the performance of slow parts, it is a > > little bit abstract for me to see now how to break it in smaller > > tasks. I do believe that it would require to reformulate some parts > > of the queries, and as their result may change a bit, tweaks could > > be required on the code too. My point is, how would I propose an > > improvement approach if I don't even know what exactly is to be > > improved? But I imagine that work on this second task is more > > demanding than the first and will take most of the time of the > > internship. > > As I said before, this part is dependent on deciding where the areas > for improvement are. Maybe have a look through one of the job logs on > data.guix.gnu.org and see if you can spot some slow parts? I'll look into that and get back to you. -- Best Regards, Luciana Lima Brito MSc. in Computer Science