unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* guix melpa mirror!
@ 2022-11-16 13:09 jgart
  2022-11-16 15:52 ` Joshua Branson
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: jgart @ 2022-11-16 13:09 UTC (permalink / raw)
  To: Guix Devel

Have people seen this yet?

https://github.com/babariviere/guix-emacs

pretty epic I think

https://github.com/babariviere/guix-emacs/blob/master/docs/how_it_works.org


https://github.com/babariviere/guix-emacs/blob/master/scripts/update

https://github.com/babariviere/guix-emacs/blob/master/emacs/packages/melpa.scm


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: guix melpa mirror!
  2022-11-16 13:09 guix melpa mirror! jgart
@ 2022-11-16 15:52 ` Joshua Branson
  2022-11-16 16:48   ` jgart
  2022-11-17 15:14 ` Ludovic Courtès
  2022-11-18  9:46 ` guix melpa mirror! Mekeor Melire
  2 siblings, 1 reply; 9+ messages in thread
From: Joshua Branson @ 2022-11-16 15:52 UTC (permalink / raw)
  To: jgart; +Cc: Guix Devel

jgart <jgart@dismail.de> writes:

> Have people seen this yet?
>
> https://github.com/babariviere/guix-emacs
>
> pretty epic I think
>
> https://github.com/babariviere/guix-emacs/blob/master/docs/how_it_works.org
>
>
> https://github.com/babariviere/guix-emacs/blob/master/scripts/update
>
> https://github.com/babariviere/guix-emacs/blob/master/emacs/packages/melpa.scm
>

That is pretty rad!  I wonder if guix has packaged all of the elpa
packages?  If not, then this source code might be useful.  :)


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: guix melpa mirror!
  2022-11-16 15:52 ` Joshua Branson
@ 2022-11-16 16:48   ` jgart
  0 siblings, 0 replies; 9+ messages in thread
From: jgart @ 2022-11-16 16:48 UTC (permalink / raw)
  To: Joshua Branson; +Cc: Guix Devel

On Wed, 16 Nov 2022 10:52:24 -0500 Joshua Branson <jbranso@dismail.de> wrote:
> That is pretty rad!  I wonder if guix has packaged all of the elpa
> packages?  If not, then this source code might be useful.  :)

There's a similar completist automated project for R/CRAN repos that's
on GitHub by rekado and lars:

https://github.com/guix-science/guix-cran


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: guix melpa mirror!
  2022-11-16 13:09 guix melpa mirror! jgart
  2022-11-16 15:52 ` Joshua Branson
@ 2022-11-17 15:14 ` Ludovic Courtès
  2022-11-18  1:06   ` jgart
  2022-11-18  9:46 ` guix melpa mirror! Mekeor Melire
  2 siblings, 1 reply; 9+ messages in thread
From: Ludovic Courtès @ 2022-11-17 15:14 UTC (permalink / raw)
  To: jgart; +Cc: Guix Devel

jgart <jgart@dismail.de> skribis:

> Have people seen this yet?
>
> https://github.com/babariviere/guix-emacs
>
> pretty epic I think
>
> https://github.com/babariviere/guix-emacs/blob/master/docs/how_it_works.org

Fun!

Would be nice to share more code with (guix import elpa).

Ludo’.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: guix melpa mirror!
  2022-11-17 15:14 ` Ludovic Courtès
@ 2022-11-18  1:06   ` jgart
  2022-11-18  8:20     ` zimoun
  0 siblings, 1 reply; 9+ messages in thread
From: jgart @ 2022-11-18  1:06 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Guix Devel

On Thu, 17 Nov 2022 16:14:41 +0100 Ludovic Courtès <ludo@gnu.org> wrote:
> Fun!
> 
> Would be nice to share more code with (guix import elpa).

Ah, yes that's a great idea!! I hadn't made the association yet between those two.

I'll start reading through them with that in mind.

I would love for guix to have very powerful automated package creation for all language ecosystems!

Long term big picture goal thinking here ;()

elpa's a start


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: guix melpa mirror!
  2022-11-18  1:06   ` jgart
@ 2022-11-18  8:20     ` zimoun
  2022-11-21 10:00       ` (M)ELPA package metadata accuracy Ludovic Courtès
  0 siblings, 1 reply; 9+ messages in thread
From: zimoun @ 2022-11-18  8:20 UTC (permalink / raw)
  To: jgart, Ludovic Courtès; +Cc: Guix Devel

[-- Attachment #1: Type: text/plain, Size: 3346 bytes --]

Hi,

On Thu, 17 Nov 2022 at 19:06, jgart <jgart@dismail.de> wrote:

> I would love for guix to have very powerful automated package creation
> for all language ecosystems! 

Well, it strongly depends on the quality of the targeted language
ecosystem.  For some, they provide enough metadata to rely on for good
automatizing; for instance, R with CRAN or Bioconductor.

Sadly, for many others ecosystem, they (upstream) do not provide enough
metadata to automatically fill all the package fields.  And some manual
tweaks are required.

For example, let count the number of packages that are tweaking their
’arguments’ fields (from ’#:tests? #f’ to complex phases modifications).
This is far from being a perfect metrics but it is a rough indication
about upstream quality: if they provide clean package respecting their
build system or if the package requires Guix adjustments.

Well, I get:

      r            : 2093 = 2093 = 1991 + 102 

which is good (only ~5% require ’arguments’ tweaks), but

      python       : 2630 = 2630 = 803  + 1827

is bad (only ~31% do not require an ’arguments’ tweak).

About Emacs, it reads,

      emacs        : 1222 = 1222 = 874  + 348 

From my point of view, it seems that it would be hard to have full
automated Emacs packaging for Guix.  Well, it requires some work to find
some heuristics at importing (converting) time and that’s not
straightforward, IMHO.

Cheers,
simon

--8<---------------cut here---------------start------------->8---
$ guix repl -- arguments-vs-import.scm
key          : tot  = tot  = no-arguments + arguments
ocaml        : 57   = 57   = 0    + 57  
haskell      : 723  = 723  = 505  + 218 
clojure      : 11   = 11   = 0    + 11  
qt           : 226  = 226  = 98   + 128 
copy         : 105  = 105  = 1    + 104 
maven        : 1    = 1    = 0    + 1   
node         : 48   = 48   = 2    + 46  
minetest-mod : 18   = 18   = 18   + 0   
chicken      : 9    = 9    = 0    + 9   
emacs        : 1222 = 1222 = 874  + 348 
linux-module : 14   = 14   = 0    + 14  
raw          : 1    = 1    = 1    + 0   
glib-or-gtk  : 128  = 128  = 31   + 97  
asdf/source  : 659  = 659  = 553  + 106 
dune         : 226  = 226  = 39   + 187 
go           : 483  = 483  = 0    + 483 
cmake        : 1115 = 1115 = 92   + 1023
minify       : 11   = 11   = 1    + 10  
perl         : 823  = 823  = 710  + 113 
android-ndk  : 11   = 11   = 0    + 11  
waf          : 25   = 25   = 0    + 25  
trivial      : 223  = 223  = 0    + 223 
julia        : 273  = 273  = 129  + 144 
r            : 2093 = 2093 = 1991 + 102 
guile        : 41   = 41   = 14   + 27  
elm          : 29   = 29   = 24   + 5   
cargo        : 3342 = 3342 = 195  + 3147
ruby         : 475  = 475  = 80   + 395 
rebar        : 19   = 19   = 11   + 8   
scons        : 14   = 14   = 0    + 14  
font         : 85   = 85   = 61   + 24  
rakudo       : 21   = 21   = 10   + 11  
gnu          : 4241 = 4241 = 812  + 3429
asdf/ecl     : 637  = 637  = 434  + 203 
asdf/sbcl    : 678  = 678  = 453  + 225 
ant          : 462  = 462  = 1    + 461 
meson        : 448  = 448  = 88   + 360 
texlive      : 143  = 143  = 0    + 143 
python       : 2630 = 2630 = 803  + 1827
--8<---------------cut here---------------end--------------->8---


[-- Attachment #2: script.scm --]
[-- Type: text/plain, Size: 1688 bytes --]

(use-modules (guix)
             (gnu)
             (ice-9 match))


(define table (make-hash-table))

(fold-packages (lambda (package result)
                 (let ((bs  (build-system-name
                             (package-build-system package)))
                       (arg (package-arguments    package)))
                   (match (hash-ref result bs)
                     ((tot wo wi)
                      (if (null? arg)
                          (hash-set! result bs (list
                                                (1+ tot)
                                                (1+ wo)
                                                wi))
                          (hash-set! result bs (list
                                                (1+ tot)
                                                wo
                                                (1+ wi)))))
                     (#f (if (null? arg)
                             (hash-set! result bs (list 1 1 0))
                             (hash-set! result bs (list 1 0 1))))
                     (_ (format #t "Error: ~s~%" (package-name package))))
                   result))
              table)


(define fmt "~13s: ~4s = ~4s = ~4s + ~4s~%")
(format #t fmt
        'key 'tot 'tot 'no-arguments 'arguments)
(hash-for-each-handle (lambda (kv)
                        (match kv
                          ((key . value)
                           (match value
                             ((tot wo wi)
                              (format #t fmt
                                      key
                                      (+ wo wi)
                                      tot wo wi))))))
                      table)

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: guix melpa mirror!
  2022-11-16 13:09 guix melpa mirror! jgart
  2022-11-16 15:52 ` Joshua Branson
  2022-11-17 15:14 ` Ludovic Courtès
@ 2022-11-18  9:46 ` Mekeor Melire
  2 siblings, 0 replies; 9+ messages in thread
From: Mekeor Melire @ 2022-11-18  9:46 UTC (permalink / raw)
  To: guix-devel

2022-11-16 / 07:09 / jgart@dismail.de:

> Have people seen this yet?
> https://github.com/babariviere/guix-emacs

yes, i used this for a while (as you might have seen in the list of
closed issues). but i regularly ran into errors and failures. on the
other hand, the author reacted quite quickly and fixed the mistakes.
still, personally, i didn't find it stable enough to use.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* (M)ELPA package metadata accuracy
  2022-11-18  8:20     ` zimoun
@ 2022-11-21 10:00       ` Ludovic Courtès
  2022-11-21 19:07         ` zimoun
  0 siblings, 1 reply; 9+ messages in thread
From: Ludovic Courtès @ 2022-11-21 10:00 UTC (permalink / raw)
  To: zimoun; +Cc: jgart, Guix Devel

Hi!

zimoun <zimon.toutoune@gmail.com> skribis:

> For example, let count the number of packages that are tweaking their
> ’arguments’ fields (from ’#:tests? #f’ to complex phases modifications).
> This is far from being a perfect metrics but it is a rough indication
> about upstream quality: if they provide clean package respecting their
> build system or if the package requires Guix adjustments.
>
> Well, I get:
>
>       r            : 2093 = 2093 = 1991 + 102 
>
> which is good (only ~5% require ’arguments’ tweaks), but
>
>       python       : 2630 = 2630 = 803  + 1827
>
> is bad (only ~31% do not require an ’arguments’ tweak).
>
> About Emacs, it reads,
>
>       emacs        : 1222 = 1222 = 874  + 348 


So that’d be 72% accurate package metadata for (M)ELPA, not too bad!

I tried to estimate repository package data accuracy for my PackagingCon
talk last year in a sophisticated way (perhaps too sophisticated):

  https://git.savannah.gnu.org/cgit/guix/maintenance.git/plain/talks/packaging-con-2021/grail/talk.20211110.pdf
  (slide 53)

  https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00297.html

I think we should identify common sources of inaccuracy in package
metadata and talk with repo maintainers to improve on that.  For MELPA,
it shouldn’t be hard to get in touch so there’s a real opportunity here.

Thanks,
Ludo’.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: (M)ELPA package metadata accuracy
  2022-11-21 10:00       ` (M)ELPA package metadata accuracy Ludovic Courtès
@ 2022-11-21 19:07         ` zimoun
  0 siblings, 0 replies; 9+ messages in thread
From: zimoun @ 2022-11-21 19:07 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: jgart, Guix Devel

[-- Attachment #1: Type: text/plain, Size: 2334 bytes --]

Hi Ludo,

On Mon, 21 Nov 2022 at 11:00, Ludovic Courtès <ludo@gnu.org> wrote:

> So that’d be 72% accurate package metadata for (M)ELPA, not too bad!

IMHO, one could extrapolate that if someone applies a similar approach
as Lars for CRAN, then 70-75% of (M)ELPA should build out-of-the box.

BTW, unrelated to the topic but note that I engaged a discussion [1]
with Org folk about using Guix as a brick for quality assurance of Emacs
packages – starting with Org. ;-)  Well, the GNU project should reuse
its own components instead of relying on other distros. :-)

1: <https://list.orgmode.org/orgmode/87sfiiycat.fsf@gmail.com/>


> I tried to estimate repository package data accuracy for my PackagingCon
> talk last year in a sophisticated way (perhaps too sophisticated):
>
>   https://git.savannah.gnu.org/cgit/guix/maintenance.git/plain/talks/packaging-con-2021/grail/talk.20211110.pdf
>   (slide 53)
>
>   https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00297.html

IIRC, you mainly investigated the accuracy of the importers…

> I think we should identify common sources of inaccuracy in package
> metadata

…and indeed maybe a systematic analysis of the modifications of the
default emacs-build-system could help in 2 directions:

 1. maybe draw some heuristics to improve the importer accuracy,

>              talk with repo maintainers to improve on that.

and 2. report upstream if some patterns appear.


Well, for packages using ’emacs-build-system’, only comparing ’keyword’
arguments, it reads:

        emacs        : 1234 = 1234 = 878  + 356
            ("phases" . 213)
            ("tests?" . 144)
            ("test-command" . 127)
            ("include" . 87)
            ("emacs" . 25)
            ("exclude" . 20)
            ("modules" . 7)
            ("imported-modules" . 4)
            ("parallel-tests?" . 1) 

Considering this 356 packages, 144 modifies the keyword #:tests?.  Note
that ’#:tests? #t’ is counted in these 144 and it reads,

    $ ag 'tests\? #t' gnu/packages/emacs-xyz.scm | wc -l
    117

Ah!  It requires some investigations. :-)

Indeed, it could be worth to identify common sources of the extra
modifications we are doing compared to the default emacs-build-system.


Cheers,
simon


[-- Attachment #2: script --]
[-- Type: text/plain, Size: 2831 bytes --]

(use-modules (guix)
             (gnu)
             (srfi srfi-1)
             (ice-9 match))


(define %table (make-hash-table))

(define (keyword-as-string arg)
  (map (compose symbol->string keyword->symbol)
       (filter keyword? arg)))

(define (count-items lst)
  (if (null? lst)
      '()
      (let ((sorted (sort lst string<=?)))
        (sort
         (fold (lambda (keyword result)
                 (match result
                   ((head tail ...)
                    (match head
                      ((k . v)
                       (if (string= k keyword)
                           (cons `(,k . ,(+ 1 v)) tail)
                           (cons `(,keyword . 1) result)))
                      (_ (format #t "Error:"))))
                   (_ (format #t "Error:"))))
               (list `(,(car sorted) . 1))
               (cdr sorted))
         (lambda (x1 x2)
           (match x1
             ((k1 . v1)
              (match x2
                ((k2 . v2)
                 (> v1 v2))))))))))


(fold-packages (lambda (package result)
                 (let ((bs  (build-system-name
                             (package-build-system package)))
                       (arg (package-arguments     package)))
                   (match (hash-ref result bs)
                     ((tot wo wi args)
                      (if (null? arg)
                          (hash-set! result bs (list
                                                (1+ tot)
                                                (1+ wo)
                                                wi args))
                          (hash-set! result bs (list
                                                (1+ tot)
                                                wo
                                                (1+ wi)
                                                (append (keyword-as-string arg)
                                                        args)))))
                     (#f (if (null? arg)
                             (hash-set! result bs (list 1 1 0 '()))
                             (hash-set! result bs (list 1 0 1 (keyword-as-string arg)))))
                     (_ (format #t "Error: ~s~%" (package-name package))))
                   result))
              %table)


(define fmt "~13s: ~4s = ~4s = ~4s + ~4s  ~{ ~s ~}~%")
(format #t fmt
        'key 'tot 'tot 'no-arguments 'arguments (list 'pattern?))
(hash-for-each-handle (lambda (kv)
                        (match kv
                          ((key . value)
                           (match value
                             ((tot wo wi args)
                              (format #t fmt
                                      key
                                      (+ wo wi)
                                      tot wo wi (count-items args)))))))
                      %table)

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2022-11-21 19:12 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-16 13:09 guix melpa mirror! jgart
2022-11-16 15:52 ` Joshua Branson
2022-11-16 16:48   ` jgart
2022-11-17 15:14 ` Ludovic Courtès
2022-11-18  1:06   ` jgart
2022-11-18  8:20     ` zimoun
2022-11-21 10:00       ` (M)ELPA package metadata accuracy Ludovic Courtès
2022-11-21 19:07         ` zimoun
2022-11-18  9:46 ` guix melpa mirror! Mekeor Melire

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).