From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id OILuOLPNe2OYMQAAbAwnHQ (envelope-from ) for ; Mon, 21 Nov 2022 20:12:52 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id YIwNOLPNe2P6EgEAG6o9tA (envelope-from ) for ; Mon, 21 Nov 2022 20:12:51 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7C9C939CC6 for ; Mon, 21 Nov 2022 20:12:51 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oxCDb-0007Ou-AV; Mon, 21 Nov 2022 14:12:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oxCDX-0007NO-Fi for guix-devel@gnu.org; Mon, 21 Nov 2022 14:12:31 -0500 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oxCDV-0007My-Gh; Mon, 21 Nov 2022 14:12:31 -0500 Received: by mail-wr1-x42c.google.com with SMTP id n3so4743543wrp.5; Mon, 21 Nov 2022 11:12:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=paY75jnQH2A77kkaSkDZuzSPt22Z+U8sPRB25+MAZS8=; b=SR2inpipLi2tgpEuYvrfLdYCBN9MlxL2dXVYancHgP11wQDNkB+url1y6izNAanOGd 5XBX7C6pBXGoEca7K9qUmebiT65blyRHNAzkitHCBEDSjbsOewIuEXhkx85dZ8btyC5d vfdRPdBNPYjeKilucK6Oy6Vk29bQ+zaLdcZCfR2pHYGfjpgai+Z0UYZW2vxw47Fyn0Ji Mkvy+n77p4UUNUaIc/NZDIxjPBI932i8HWeijTR9j4lEcGXCYU1Skr+JBqRktAbpKQU0 lg7sV3ym5x9/vNlJKwYA1FtFo9S4Q5l/AxAdz16qXnr299SHlEro9GjnYMfLVXQjDJzr JQGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=paY75jnQH2A77kkaSkDZuzSPt22Z+U8sPRB25+MAZS8=; b=Qv3ZCsYq2mNtqCbTHZCU0ppHqMdeEHMW1wn6VC0D8MDMdCbRSfQgNupOF7ccsqH0Rn Ol7RMIoGSP5X7W6zA0NqGWsj0TiknvWUZD8WgS/wi0WQQA6xCKbO5EoIUIIOscIfmqlS o1p4cQ0HXBsfyGkuJT+l5SdwJ315ffDiBJ6uLXXM5gt2SQZylPwXmPsUjR4xQHjTgWPE tMGsg6Qt2WrVCLuzuxaQ44BD1Tiq+sYnV/eDF1jJ4ffySyaj8fYM1UpVL977zY+jIpkO qMCs/pf6tIsOhGP/H1ND1JMDtl0oRlrsR2rwvRNzifW55bigiFE0nb/hW15TuSyktIqf 0oHg== X-Gm-Message-State: ANoB5pkMzWi3e1k1hiOlWMt8KAV9T4FsbwzoPCbZWpicV4Is4CSHBZgF kyMuAd7/bjV5NMDeYlLQcNlvRRL4Jxs= X-Google-Smtp-Source: AA0mqf4W28MXxPeCm2SlOw5stsFhJw9ODyh6eJcB6w5J4kLNd/ZxSCRh8D9gvZdBQiXbBQtwLPG7wA== X-Received: by 2002:adf:cd86:0:b0:236:8a6d:72a1 with SMTP id q6-20020adfcd86000000b002368a6d72a1mr11691647wrj.682.1669057947592; Mon, 21 Nov 2022 11:12:27 -0800 (PST) Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id v11-20020a05600c444b00b003b95ed78275sm15653931wmn.20.2022.11.21.11.12.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Nov 2022 11:12:26 -0800 (PST) From: zimoun To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: jgart , Guix Devel Subject: Re: (M)ELPA package metadata accuracy In-Reply-To: <87mt8kmxtx.fsf_-_@gnu.org> References: <20221116070959.GB4217@dismail.de> <87edu1ipem.fsf@gnu.org> <20221117190626.GB12202@dismail.de> <86cz9kk71y.fsf@gmail.com> <87mt8kmxtx.fsf_-_@gnu.org> Date: Mon, 21 Nov 2022 20:07:26 +0100 Message-ID: <87cz9gunwx.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=zimon.toutoune@gmail.com; helo=mail-wr1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1669057971; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=paY75jnQH2A77kkaSkDZuzSPt22Z+U8sPRB25+MAZS8=; b=DzhH3wHJuOF/D3yr0eHjIdBQwpSKUo3JFxV/7Y6hfbMw0yMnmYWBZYTjBpaeR3yVvNNZMF 0nTqVcVRKh+c0QmB3Orcv8bfrS4o1QN3D2po6VyOrub2tQVj5v5KRl0a3aw9eEH7lZ7olz DgFhWrdaE7jKxHdenJLoE+GTeV+ukkl0nursB2MPhFzYjI6vJWASuUA8++UhFtg9C+JSTW /EbdrE2sGzFnsKlFfh3xKiqUp1IL0gwhLGXTgU2gMLDbEbbtNYfxCLKwF0opWxIu6cIR/u SOWylVLaZGh0r74rwhyRuL5VcGyRXj/Df4VeySPxWObaqvm57B3O7jxccpIw9w== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1669057971; a=rsa-sha256; cv=none; b=nS3t3to4nWJ9mqDg1Mc2n5YMOy1pITmlRMVJR5eKc1GMKZnpFv5ZTLe9Q5Pczt+r6waCLC Y4bywYQ1Y4MxxnEWcT4bYsFJ9IK9JstChKfh9OBy/VKtkiH6B+8dd7cScnGT71TvTTx8aU eTrf2Ic4ciMe9PkXd9tuCJ6gbq4a/ITAgURWDVOLxS4/Sy2t93ltRj9of6a1gZsfTgH+8z EspVNZZQ/QEA0vtPF9+p3E/LlToZDzcW0h/qL+46Pt7AjeqKPGkym9hx+ISBqYDmiY6fMI mPNb8jnfvLa7X/eweKG2GHDEQEkXQuseHQFJYF2afPiifHVjUqm9bniazwYTBg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=SR2inpip; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -5.01 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=SR2inpip; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 7C9C939CC6 X-Spam-Score: -5.01 X-Migadu-Scanner: scn0.migadu.com X-TUID: khNdEscYUk3h --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo, On Mon, 21 Nov 2022 at 11:00, Ludovic Court=C3=A8s wrote: > So that=E2=80=99d be 72% accurate package metadata for (M)ELPA, not too b= ad! IMHO, one could extrapolate that if someone applies a similar approach as Lars for CRAN, then 70-75% of (M)ELPA should build out-of-the box. BTW, unrelated to the topic but note that I engaged a discussion [1] with Org folk about using Guix as a brick for quality assurance of Emacs packages =E2=80=93 starting with Org. ;-) Well, the GNU project should reu= se its own components instead of relying on other distros. :-) 1: > I tried to estimate repository package data accuracy for my PackagingCon > talk last year in a sophisticated way (perhaps too sophisticated): > > https://git.savannah.gnu.org/cgit/guix/maintenance.git/plain/talks/pack= aging-con-2021/grail/talk.20211110.pdf > (slide 53) > > https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00297.html IIRC, you mainly investigated the accuracy of the importers=E2=80=A6 > I think we should identify common sources of inaccuracy in package > metadata =E2=80=A6and indeed maybe a systematic analysis of the modifications of the default emacs-build-system could help in 2 directions: 1. maybe draw some heuristics to improve the importer accuracy, > talk with repo maintainers to improve on that. and 2. report upstream if some patterns appear. Well, for packages using =E2=80=99emacs-build-system=E2=80=99, only compari= ng =E2=80=99keyword=E2=80=99 arguments, it reads: emacs : 1234 =3D 1234 =3D 878 + 356 ("phases" . 213) ("tests?" . 144) ("test-command" . 127) ("include" . 87) ("emacs" . 25) ("exclude" . 20) ("modules" . 7) ("imported-modules" . 4) ("parallel-tests?" . 1)=20 Considering this 356 packages, 144 modifies the keyword #:tests?. Note that =E2=80=99#:tests? #t=E2=80=99 is counted in these 144 and it reads, $ ag 'tests\? #t' gnu/packages/emacs-xyz.scm | wc -l 117 Ah! It requires some investigations. :-) Indeed, it could be worth to identify common sources of the extra modifications we are doing compared to the default emacs-build-system. Cheers, simon --=-=-= Content-Type: text/plain Content-Disposition: inline; filename=arguments-vs-import-bis.scm Content-Description: script (use-modules (guix) (gnu) (srfi srfi-1) (ice-9 match)) (define %table (make-hash-table)) (define (keyword-as-string arg) (map (compose symbol->string keyword->symbol) (filter keyword? arg))) (define (count-items lst) (if (null? lst) '() (let ((sorted (sort lst string<=?))) (sort (fold (lambda (keyword result) (match result ((head tail ...) (match head ((k . v) (if (string= k keyword) (cons `(,k . ,(+ 1 v)) tail) (cons `(,keyword . 1) result))) (_ (format #t "Error:")))) (_ (format #t "Error:")))) (list `(,(car sorted) . 1)) (cdr sorted)) (lambda (x1 x2) (match x1 ((k1 . v1) (match x2 ((k2 . v2) (> v1 v2)))))))))) (fold-packages (lambda (package result) (let ((bs (build-system-name (package-build-system package))) (arg (package-arguments package))) (match (hash-ref result bs) ((tot wo wi args) (if (null? arg) (hash-set! result bs (list (1+ tot) (1+ wo) wi args)) (hash-set! result bs (list (1+ tot) wo (1+ wi) (append (keyword-as-string arg) args))))) (#f (if (null? arg) (hash-set! result bs (list 1 1 0 '())) (hash-set! result bs (list 1 0 1 (keyword-as-string arg))))) (_ (format #t "Error: ~s~%" (package-name package)))) result)) %table) (define fmt "~13s: ~4s = ~4s = ~4s + ~4s ~{ ~s ~}~%") (format #t fmt 'key 'tot 'tot 'no-arguments 'arguments (list 'pattern?)) (hash-for-each-handle (lambda (kv) (match kv ((key . value) (match value ((tot wo wi args) (format #t fmt key (+ wo wi) tot wo wi (count-items args))))))) %table) --=-=-=--