unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Compress VCS checkouts?
@ 2018-02-02 20:50 Eric Bavier
  2018-02-03 20:10 ` Efraim Flashner
  2018-02-05 10:27 ` Ludovic Courtès
  0 siblings, 2 replies; 5+ messages in thread
From: Eric Bavier @ 2018-02-02 20:50 UTC (permalink / raw)
  To: guix-devel


[-- Attachment #1.1: Type: text/plain, Size: 966 bytes --]

Hello Guix,

A while back I toyed with the idea of a "Sources DVD" for Guix, much
like Trisquel has. While looking into the feasability of this, I
noticed that the git/svn/cvs/hg checkouts our packages use are not
compressed in the store.  So I wrote a simple script to compare the raw
size of all source checkouts with their corresponding xz-compressed
tarballs.  

At the time, I couldn't convince myself the trouble was worth the
effort; we had only 385MiB of uncompressed source checkouts, which
would compress to 135MiB.

Since that initial investigation, the number of packages using VCS
checkouts has risen from 50 to 389, and the on-disk size of that source
has risen to 3784MiB uncompressed.  We could compress that down to
538MiB.

Compression in not free, but I think it would be nice for our build
servers who have to store and serve (which, iirc, requires them to
compress it anyhow) this source.

What do people think?

`~Eric

[-- Attachment #1.2: vcs-sizes.txt --]
[-- Type: text/plain, Size: 50208 bytes --]

name                                  path                                                           size (MiB) tar.xz    rel%
footswitch                            cwrqb...-footswitch-0.1-1.7cb0a93-checkout                          0.04    0.01   17.5%
interrobang                           lab0p...-interrobang-0.0.0-1.8965437                                0.08    0.02   27.2%
mkbootimg                             imjbx...-android-platform-system-core-7.1.2_r6-checkout             8.68    1.11   12.8%
git-repo                              rk0vy...-git-repo-1.12.37-checkout                                  0.47    0.09   19.2%
android-udev-rules                    i5qbm...-android-udev-rules-20171113-checkout                       0.06    0.02   25.0%
fluidsynth                            ghszj...-fluidsynth-1.1.9-checkout                                  2.79    0.47   16.8%
faust                                 g8qzg...-faust-0.9.90-checkout                                    148.59   16.36   11.0%
qjackrcd                              chq81...-git-checkout                                               0.25    0.10   38.3%
noise-repellent                       srfp9...-noise-repellent-0.1.4-checkout                             0.15    0.03   18.5%
faust                                 yfjxh...-git-checkout                                             203.48   22.98   11.3%
filteraudio                           80bgn...-filteraudio-0.0.0-1.2fc6695-checkout                       1.02    0.15   14.9%
raul                                  2x87s...-raul-0.8.9-1.4db870b2b-checkout                            0.39    0.16   40.6%
bluez-alsa                            gbva4...-bluez-alsa-1.2.0-checkout                                  0.31    0.05   16.1%
lv2-mda-epiano                        9q4bs...-git-checkout                                               2.07    1.66   79.9%
lv2-devel                             986wf...-git-checkout                                               1.55    0.30   19.2%
newick-utils                          qkis8...-newick-utils-1.6-1.da121155-checkout                     133.61   24.13   18.1%
libdivsufsort                         9ahyi...-git-checkout                                               0.19    0.03   13.4%
r-chipkernels                         9kms2...-r-chipkernels-1.1-1.c9cfcacb6                              0.07    0.01   16.2%
bits                                  zadz5...-bits-2.13.0-1.3cc456789-checkout                           2.77    0.52   18.9%
piranha                               bvrk4...-git-checkout                                              32.86    2.31    7.0%
smithlab-cpp                          mgvpc...-smithlab-cpp-0.1.728a097-checkout                          0.15    0.02   14.2%
bpp-popgen                            7ika1...-bpp-popgen-2.2.0-1.e472bac-checkout                        0.82    0.09   10.8%
bpp-core                              njqf5...-bpp-core-2.2.0-1.7d8bced-checkout                          1.92    0.22   11.5%
seek                                  gl4rn...-hg-checkout                                                8.43    0.74    8.8%
bppsuite                              yq220...-bppsuite-2.2.0-1.c516147-checkout                          1.23    0.13   10.3%
mosaik                                43sqg...-mosaik-2.2.30                                             13.57    2.78   20.5%
bpp-phyl                              kg339...-bpp-phyl-2.2.0-1.0c07167-checkout                          3.64    0.37   10.1%
bpp-seq                               z586w...-bpp-seq-2.2.0-1.6cfa079-checkout                           2.32    0.20    8.8%
tremc                                 0654m...-tremc-0.9.0-2.e06d08d-checkout                             0.44    0.24   53.6%
syslinux                              clcb3...-syslinux-6.04-pre-1.bb41e93-checkout                      19.39    9.32   48.1%
vboot-utils                           6a6lh...-vboot-utils-R63-10032.B-checkout                          63.68   21.70   34.1%
catch                                 067v7...-catch-1.3.5                                                2.12    0.18    8.6%
go-gopkg.in-check.v1                  6wjyv...-go-gopkg.in-check.v1-0.0.0-0.20d25e2-checkout              0.14    0.03   18.3%
chez-sockets                          3il8x...-chez-sockets-0.0-1.bce9688-checkout                        0.10    0.02   21.6%
chez-web                              g2j99...-chez-web-2.0-1.5fd177f-checkout                            0.26    0.07   25.2%
cuirass                               n0ydb...-cuirass-0.0.1-10.9cfea9f                                   0.27    0.05   18.9%
sfarkxtc                              v6hfw...-git-checkout                                               0.05    0.01   27.5%
eschalot                              yv9sb...-eschalot-1.2.0-1.0bf31d8                                   0.67    0.19   28.9%
libb2                                 fn6hl...-libb2-0.0.0-1.60ea749-checkout                             1.11    0.20   18.0%
kurly                                 q7p2v...-git-checkout                                               0.05    0.01   23.0%
mongo-tools                           l2fv7...-mongo-tools-3.4.0-checkout                                17.41    2.96   17.0%
go-gopkg.in-mgo.v2                    arawv...-go-gopkg.in-mgo.v2-2016.08.01-checkout                     1.99    0.31   15.5%
guile-wiredtiger                      pjcs0...-guile-wiredtiger-0.6.3-checkout                            1.24    1.06   85.6%
stress-make                           pnwyd...-stress-make-1.0-0.506e6cf-checkout                         0.06    0.02   27.6%
emacs-god-mode                        4jwih...-emacs-god-mode-20151005.925.1-6cf0807b6-checkout           0.07    0.01   18.3%
emacs-ob-ipython                      x3pzh...-git-checkout                                               0.63    0.50   78.1%
emacs-string-inflection               3jay5...-emacs-string-inflection-1.0.6-checkout                     0.03    0.00   10.2%
emacs-yasnippet-snippets              88gc0...-emacs-yasnippet-snippets-1-1.885050d3-checkout             0.70    0.06    7.9%
emacs-strace-mode                     8dvr6...-emacs-strace-mode-0.0.2-1.6a69b4b-checkout                 0.04    0.01   26.8%
emacs-grep-a-lot                      560jq...-emacs-grep-a-lot-1.0.7-checkout                            0.05    0.01   26.2%
emacs-dream-theme                     xrcq6...-emacs-dream-theme-0.0.0-1.107a11d-checkout                 0.02    0.00   15.0%
emacs-ws-butler                       n6fzb...-emacs-ws-butler-0.6-checkout                               0.03    0.01   18.3%
emacs-julia-mode                      y8rmq...-emacs-julia-mode-0.3-1.115d4dc8-checkout                   0.17    0.02   13.9%
emacs-undo-tree                       zy83l...-emacs-undo-tree-0.6.6-checkout                             0.17    0.03   17.6%
emacs-restclient                      6gjw4...-emacs-restclient-1.07a3888-checkout                        0.05    0.01   19.1%
emacs-el-search                       qsr69...-emacs-el-search-0.0-1.f26277b.tar.gz                       0.09    0.02   22.1%
emacs-ivy                             r6j1m...-emacs-ivy-0.10.0-checkout                                  0.62    0.11   17.6%
emacs-helm-make                       1crr4...-emacs-helm-make-0.1.0-1.21c1bfa-checkout                   0.02    0.01   20.6%
emacs-helm-c-yasnippet                57mnf...-emacs-helm-c-yasnippet-0.6.7-1.65ca732-checkout            0.06    0.03   48.3%
emacs-writegood-mode                  xp9b3...-emacs-writegood-mode-checkout                              0.02    0.00   20.5%
emacs-butler                          45ilp...-emacs-butler-0.2.4                                         0.03    0.01   18.7%
emacs-slack                           26cds...-emacs-slack-0-1.58b1309-checkout                           0.40    0.05   12.8%
emacs-danneskjold-theme               90hbr...-emacs-danneskjold-theme-0.0.0-1.8733d2f-checkout           0.54    0.44   81.6%
emacs-circe                           bsyci...-emacs-circe-2.6-checkout                                   0.60    0.20   33.9%
emacs-typo                            2ixcw...-emacs-typo-1.1                                             0.03    0.01   20.4%
emacs-memoize                         1h1zy...-emacs-memoize-20130421.b55eab0.tar.gz                      0.02    0.00   20.9%
emacs-browse-at-remote                7g6c5...-emacs-browse-at-remote-0.9.0-1.b5cff79-checkout            0.04    0.01   16.5%
emacs-request                         mq5ll...-emacs-request-0.3.0-checkout                               0.19    0.04   19.6%
emacs-deferred                        pzbya...-emacs-deferred-0.3.2                                       0.20    0.03   16.3%
emacs-dired-hacks                     qdsww...-emacs-dired-hacks-0.0.1-1.eda6800-checkout                 0.22    0.04   18.2%
emacs-graphviz-dot-mode               rind2...-emacs-graphviz-dot-mode-0.3.10-1.fdaabbc-checkout          0.57    0.30   51.3%
emacs-lua-mode                        07v2y...-emacs-lua-mode-20151025.1-652e299cb.checkout               0.14    0.03   21.6%
emacs-sr-speedbar                     bljyp...-emacs-sr-speedbar-20161025-0.77a83fb-checkout              0.03    0.01   19.3%
propeller-load                        5gwzk...-propeller-load-4c46ecbe79ffbecd2ce918497ace5b956736b5a3-checkout  620.10   67.80   10.9%
proplib                               p1dkd...-proplib-4c46ecbe79ffbecd2ce918497ace5b956736b5a3-checkout  620.10   67.81   10.9%
binutils-vc4                          gfhc6...-binutils-vc4-2.23.51-0.708acc8-checkout                  141.97   14.40   10.1%
gcc-vc4                               kh6bc...-gcc-vc4-5.4.0-checkout                                   564.45   71.02   12.6%
spinsim                               0dwyg...-spinsim-66915a7ad1a3a2cf990a725bb341fab8d11eb620-checkout    0.75    0.10   13.4%
libjaylink                            j7zmg...-libjaylink-0.1.0-2.699b700-checkout                        0.32    0.05   16.0%
kicad                                 58h1j...-kicad-4.0-2.5f4599f-checkout                              98.86   11.69   11.8%
disorderfs                            akxf1...-git-checkout                                               0.08    0.02   21.5%
b43-tools                             ckiq9...-b43-tools-20140625-1.8dce532-checkout                      0.78    0.10   13.0%
ovmf                                  vm3bw...-edk2-20170116-1.13a50a6-checkout                         201.26   20.34   10.1%
rkflashtool                           j8y6m...-rkflashtool-0.0.0-1.094bd64-checkout                       0.08    0.01   17.3%
ntk                                   k82sa...-ntk-1.3.0-checkout                                         8.64    1.93   22.3%
font-go                               gb8iv...-go-image-20170330-1.f03a046-checkout                      16.19    3.07   19.0%
icestorm                              59gfn...-icestorm-0.0-1-12b2295c9-checkout                          2.82    0.18    6.2%
arachne-pnr                           nix78...-arachne-pnr-0.0-1-52e69ed20-checkout                       0.56    0.06   11.5%
ois                                   kjwgf...-ois-1.3                                                    1.01    0.10   10.2%
open-adventure                        iiiwy...-open-adventure-2.5-2.d43854f-checkout                      1.67    0.12    7.0%
emulation-station                     h1rjr...-emulation-station-2.0.1-checkout                           4.81    0.73   15.1%
badass                                3216w...-badass-0.0-0.3c3cd66-checkout                              0.02    0.00   20.1%
roguebox-adventures                   l63ak...-roguebox-adventures-2.1.2-1.19a2c34-checkout              18.18   16.98   93.4%
dosbox                                w5ik0...-dosbox-0.74.svn3947-checkout                               5.05    0.79   15.6%
dia                                   i79dl...-dia-0.97.2-fbc3061-checkout                               41.73    7.07   16.9%
byzanz                                avvhn...-byzanz-0.2-1.f7af3a5-checkout                              0.67    0.10   15.4%
gnu-standards                         kbwrs...-cvs-checkout                                               0.38    0.10   25.5%
guile-gnunet                          lrmhb...-guile-gnunet-0.0.383eac2-checkout                          0.24    0.04   14.9%
go-github-com-alsm-ioprogress         0jqqf...-git-checkout                                               0.02    0.00   21.9%
go-github-com-aki237-nscjar           7njw3...-git-checkout                                               0.02    0.00   14.3%
go-github-com-davidjpeacock-cli       qqk5g...-git-checkout                                               0.30    0.05   15.2%
go-gopkg.in-tomb.v2                   0rc3i...-go-gopkg.in-tomb.v2-0.0.0-0.d5d1b58.tar.gz                 0.03    0.00   15.5%
go-github.com-jessevdk-go-flags       a7cln...-go-github.com-jessevdk-go-flags-1.3.0-checkout             0.24    0.04   18.6%
gpscorrelate                          r62p1...-git-checkout                                               0.46    0.13   29.2%
brdf-explorer                         34908...-brdf-explorer-1.0.0-1.5b2cd46f3-checkout                  18.81    3.68   19.6%
roffit                                ir6rd...-roffit-e5228388e3faf2b7f1ae5bd048ad46ed565304c6-checkout    0.04    0.01   22.5%
ganv                                  l7rfj...-git-checkout                                               0.54    0.17   31.9%
guile-wm                              l0lr7...-guile-wm-1.0-1.f3c7b3b-checkout                            0.89    0.18   20.4%
guile-xcb                             4idd3...-guile-xcb-1.3-1.db7d5a3-checkout                           1.50    0.25   16.4%
guile-dsv                             9hgd3...-guile-dsv-0.2.1-checkout                                   0.30    0.05   17.9%
guile2.0-git                          plx98...-guile-git-0.0-4.951a32c-checkout                           0.36    0.08   21.0%
guile-gdbm-ffi                        vllal...-guile-gdbm-ffi-20120209.fa1d5b6-checkout                   0.06    0.01   23.6%
guile-ics                             ppb04...-guile-ics-0.1.1-checkout                                   0.30    0.05   17.2%
guile-miniadapton                     5hq5z...-guile-miniadapton-0-1.1b5749422-checkout                   0.02    0.00   16.1%
guile-for-guile-emacs                 afxy2...-guile-for-guile-emacs-20150510.d8d9a8d-checkout           23.97    3.44   14.4%
guile-syntax-highlight                b7dsl...-guile-syntax-highlight-0.0.0.a047675-checkout              0.05    0.01   17.7%
guildhall                             f330d...-guildhall-0-1.2fe2cc539-checkout                           1.53    0.20   12.9%
guile-minikanren                      jcx53...-guile-minikanren-20150424.e844d85-checkout                 0.10    0.01   13.0%
mcron2                                pyj2k...-mcron2-1.0.8-0.31baff1-checkout                            0.27    0.06   22.6%
guile-bash                            7p7mk...-guile-bash-0.1.6-0.1eabc56-checkout                        0.20    0.03   17.0%
corrode                               nb842...-corrode-0.0.1-b6699fb-checkout                             0.29    0.07   22.7%
cl-yale-haskell                       vn1sy...-yale-haskell-85f94c72a16c5f70301dd8db04cde9de2d7dd270-checkout    2.64    0.50   18.9%
idris-bifunctors                      2b3y9...-idris-bifunctors-0.1-1.53d06a6-checkout                    0.14    0.01    5.7%
idris-lens                            hfs5h...-idris-lens-0.1-1.26f0120-checkout                          0.08    0.01    7.9%
idris-lightyear                       nz2ch...-idris-lightyear-0.1-1.6d65ad1-checkout                     0.07    0.01   14.9%
idris-wl-pprint                       0422w...-idris-wl-pprint-0.1-1.1d365fc-checkout                     0.06    0.01   14.6%
stb-image-for-extempore               ca99d...-stb-image-for-extempore-0-1.152a250a7-checkout             3.38    0.69   20.6%
java-aopalliance                      ybdw8...-java-aopalliance-1.0                                       0.24    0.02    6.9%
java-cofoja                           kh0sm...-java-cofoja-1.3-checkout                                   0.79    0.08   10.2%
java-geronimo-xbean-reflect           cyk69...-java-geronimo-xbean-reflect-4.5                            4.73    0.22    4.7%
java-jmh                              iyq7l...-java-jmh-1.17.5-checkout                                   6.08    0.24    3.9%
classpath                             0y51r...-classpath-0.99-1.e7c13ee0c-checkout                       65.42    7.16   10.9%
javacc                                0x6b3...-javacc-4.1-checkout                                        2.83    0.57   20.1%
java-jnacl                            klgqr...-git-checkout                                               0.15    0.01    5.0%
js-json2                              51862...-js-json2-2016-10-28.1-031b1d9-checkout                     0.05    0.01   19.5%
mujs                                  x6ss1...-mujs-1.0.2-checkout                                        0.42    0.09   20.5%
kodi-cli                              cyi7s...-kodi-cli-1.1-1.104dc23-checkout                            0.04    0.01   25.6%
crossguid                             jwhwq...-crossguid-0.0-1.8f399e8-checkout                           0.20    0.04   21.1%
libusb4java                           267pc...-git-checkout                                               0.17    0.02   10.7%
python-lirc                           hhz9b...-python-lirc-1.2.1-1.4091fe9                                0.07    0.02   21.3%
sbcl-cl-unicode                       npz10...-cl-unicode-0.1.5-1.9fcd06f-checkout                        2.65    0.25    9.3%
cl-trivial-gray-streams               2hs2f...-trivial-gray-streams-0.0.0-1.0483ade-checkout              0.05    0.01   15.6%
sbcl-alexandria                       x91h6...-alexandria-0.0.0-1.926a066-checkout                        0.22    0.04   19.3%
gcl                                   bsi2d...-gcl-2.6.12-1.5956140-checkout                             28.57    4.44   15.5%
sbcl-cl-uglify-js                     rwnr0...-git-checkout                                               0.10    0.02   15.9%
femtolisp                             iviwg...-femtolisp-0.0.0-1.68c5b12-checkout                         2.59    0.25    9.8%
sbcl-parse-js                         7hph5...-sbcl-parse-js-fbadc6029bec7039602abfc06c73bb52970998f6-checkout    0.05    0.01   18.6%
selene                                8m547...-Selene                                                     0.17    0.02   14.0%
ghmm                                  g8by8...-ghmm-0.9-rc3-0.2341                                        2.59    0.46   17.7%
esmtp                                 n5lsc...-esmtp-1.2-checkout                                         0.13    0.04   27.8%
dovecot-libsodium-plugin              bn6q7...-dovecot-libsodium-plugin-0.0.0-1.044de73-checkout          0.07    0.01   21.2%
notmuch-addrlookup-c                  ziksg...-notmuch-addrlookup-c-8-1.88f156d-checkout                  0.03    0.01   23.6%
notifymuch                            7k1wg...-notifymuch-0.1-1.9d4aaf5-checkout                          0.03    0.00   11.0%
blis-haswell                          4p35s...-blis-0.2.2-checkout                                       12.80    0.78    6.1%
maxflow                               b4s25...-maxflow-3.04-checkout                                      0.11    0.02   22.0%
reducelcs                             cg88w...-reducelcs-1.0-1.474f88d-checkout                           0.08    0.02   25.6%
psyclpc                               ln6m2...-psyclpc-20160821-2.61cf9aa-checkout                        6.47    1.09   16.8%
dino                                  ay0qv...-dino-0.0-3.f25fadde2-checkout                              1.82    0.18    9.8%
libtoxcore                            q58hf...-libtoxcore-0.0.0-1.755f084-checkout                        1.96    0.28   14.2%
microcom                              rar0g...-microcom-2016.01.09-checkout                               0.10    0.02   22.9%
jack-capture                          n37sl...-jack-capture-0.9.73-checkout                               0.23    0.05   21.8%
gx-vintage-fuzz-master-lv2            xfzfs...-gx-vintage-fuzz-master-lv2-0.1-checkout                    0.08    0.03   35.0%
non-mixer                             bg1pp...-non-sequencer-1.9.5-3.10c31e5-checkout                     8.04    3.08   38.3%
gx-tone-mender-lv2                    rcygs...-gx-tone-mender-lv2-0-1.b6780b4a3-checkout                  0.11    0.02   20.3%
gx-overdriver-lv2                     r5mf1...-gx-overdriver-lv2-0-1.ed7180198-checkout                   0.07    0.02   26.0%
mod-utilities                         4m1b4...-mod-utilities-0-2.80ea3ea9f-checkout                       6.03    2.27   37.7%
gx-saturator-lv2                      4b0nw...-gx-saturator-lv2-0-3.605330f43-checkout                    1.84    0.76   41.4%
gx-guvnor-lv2                         8fbs3...-gx-guvnor-lv2-0.1-checkout                                 0.16    0.03   19.2%
gx-switchless-wah-lv2                 9ha8q...-gx-switchless-wah-lv2-0-2.7b0869120-checkout               0.06    0.02   25.8%
gx-voodoo-fuzz-lv2                    mpyz2...-gx-voodoo-fuzz-lv2-0.1-checkout                            0.08    0.02   24.9%
gx-suppa-tone-bender-lv2              dpgpd...-gx-suppa-tone-bender-lv2-0.1-checkout                      0.07    0.02   25.7%
libechonest                           p5rkc...-libechonest-2.3.1-checkout                                12.23   11.51   94.1%
gx-push-pull-lv2                      0q7as...-gx-push-pull-lv2-0-1.7f76ae206-checkout                    0.07    0.02   26.2%
gx-slow-gear-lv2                      w7nd3...-gx-slow-gear-lv2-0-3.5d37e775b-checkout                    1.66    0.86   51.8%
instantmusic                          y36yp...-instantmusic-1.0-1.300891d-checkout                        0.06    0.01   13.5%
mod-host                              vbllj...-mod-host-0.10.6-2.299a39774-checkout                       0.43    0.06   13.7%
sonivox-eas                           kw163...-sonivox-eas-1.1.0-checkout                                 3.17    0.72   22.6%
ingen                                 8hlqc...-ingen-0.0.0-2.cc4a4db33-checkout                           2.22    0.40   17.9%
gx-vbass-preamp-lv2                   xgb1m...-gx-vbass-preamp-lv2-0-2.eb999b0ca-checkout                 0.73    0.35   48.0%
gx-super-fuzz-lv2                     00mjm...-gx-super-fuzz-lv2-0.1-checkout                             0.08    0.02   24.6%
gx-hyperion-lv2                       s21j4...-gx-hyperion-lv2-0.1-checkout                               0.06    0.02   26.2%
muse-sequencer                        f3lw8...-muse-sequencer-3.0.0-checkout                             37.11    6.03   16.3%
tlsdate                               zq6dn...-tlsdate-0.0.13-checkout                                    0.84    0.21   25.0%
guix                                  5krqj...-guix-0.14.0-7.33988f9-checkout                            41.56    7.36   17.7%
patches                               lpvs8...-patches-0.0-1.ef1b8a7-checkout                             0.11    0.02   21.8%
pumpa                                 wyps7...-pumpa-0.9.2-checkout                                       1.76    0.26   14.7%
python-axolotl-curve25519             yla44...-python-axolotl-curve25519-0.1-checkout                     0.31    0.07   21.6%
python2-elib.intl                     dzrar...-git-checkout                                               0.13    0.03   18.9%
python2-kivy-next                     skyhx...-python-kivy-next-1.9.1-1.a988c5e-checkout                 27.99   17.70   63.2%
freerdp                               r4xxl...-freerdp-1.1-1.03ab683-checkout                            16.32    3.60   22.1%
enki                                  sd00n...-enki-2.0pre-0.afd2d8e-checkout                             4.66    3.51   75.2%
ruby-minitar                          h1wrk...-ruby-minitar-0.5.4-1.e25205ec-checkout                     0.15    0.02   15.7%
ruby-nokogumbo                        nrqzd...-ruby-nokogumbo-1.4.7-1.fb51ff29-checkout                   0.07    0.01   15.8%
ruby-minitest-pretty-diff             xlbza...-ruby-minitest-pretty-diff-0.1-1.11f32e93-checkout          0.04    0.00    6.3%
ruby-tdiff                            7fj7m...-ruby-tdiff-0.3.3-1.b662a604-checkout                       0.06    0.01    8.5%
ruby-nokogiri-diff                    43pbf...-ruby-nokogiri-diff-0.2.0-1.a38491e4-checkout               0.07    0.00    7.6%
sicp                                  srxd5...-sicp-20170703-1.225c172-checkout                           9.61    2.85   29.7%
scheme48-rx                           pcx0a...-scheme48-rx-0.0.0-2.dd9037f-checkout                       0.20    0.04   19.7%
libsemanage                           5rjhg...-selinux-20170804-checkout                                 28.99    3.04   10.5%
python-setools                        amj5l...-python-setools-4.1.1-checkout                              3.44    0.27    7.9%
s-shell                               rp40m...-s-shell-0.0.0-2.da2e5c2-checkout                           0.10    0.02   15.9%
rc                                    7wvw5...-rc-1.7.4-checkout                                          0.42    0.11   26.8%
linenoise                             5zgrz...-linenoise-1.0-1.2105ce4-checkout                           0.06    0.01   23.9%
scsh                                  1pjgq...-scsh-0.0.0-1.1144324-checkout                              0.91    0.18   19.5%
usbredir                              gsh9j...-git-checkout                                               0.37    0.06   15.0%
r-catterplots                         rmccq...-r-catterplots-0-2.40063ec57-checkout                       0.81    0.65   80.1%
r-tgstat                              brklf...-hg-checkout                                                0.23    0.06   28.1%
r-tgconfig                            jb4pl...-hg-checkout                                                0.11    0.02   13.9%
human                                 zr1zd...-human-0.2-1.50c80e6-checkout                               0.02    0.00   20.8%
libutf                                ipc6a...-libutf-0.0.0-1.ff4c606-checkout                            1.71    0.16    9.3%
lchat                                 n182g...-lchat-0.0.0-2.25d90f4-checkout                             0.05    0.02   39.9%
go-github-com-sasha-s-go-deadlock     42k4i...-go-github-com-sasha-s-go-deadlock-0.1.0-1.03d40e5-checkout    0.04    0.01   24.7%
go-github-com-templexxx-xor           3kv6g...-go-github-com-templexxx-xor-0.1.2-checkout                 0.04    0.01   14.5%
go-github-com-golang-snappy           lkk0z...-go-github-com-golang-snappy-0.0.0-0.553a641-checkout       0.20    0.05   24.1%
go-golang-org-x-net-internal-iana     5jahp...-go.googlesource.com-net-0.0.0-1.d866cfc-checkout           4.27    0.66   15.5%
go-github-com-klauspost-cpuid         8wnn5...-go-github-com-klauspost-cpuid-1.1-0.eae9b3e-checkout       0.42    0.25   58.5%
go-github-com-bkaradzic-go-lz4        9975v...-go-github-com-bkaradzic-go-lz4-0.0.0-0.7224d8d-checkout    0.62    0.18   29.8%
go-github-com-xtaci-smux              vnzhz...-go-github-com-xtaci-smux-1.0.6-0.c3e1824-checkout          0.16    0.12   78.0%
go-golang-org-x-crypto-cast5          bn0nd...-go.googlesource.com-crypto-0.0.0-1.95a4943-checkout        3.46    1.21   34.9%
go-github-com-kardianos-osext         rnhvi...-go-github-com-kardianos-osext-0.0.0-1.ae77be6-checkout     0.02    0.00   22.5%
go-github-com-chmduquesne-rollinghash-adler32  9qwvc...-go-github-com-chmduquesne-rollinghash-adler32-0.0.0-1.3dc7875-checkout    0.13    0.02   12.8%
go-github-com-edsrzf-mmap-go          qvi0x...-go-github-com-edsrzf-mmap-go-0.0.0-0.0bce6a6-checkout      0.02    0.01   22.9%
go-golang-org-x-net-ipv4              a78xp...-go-golang-org-x-net-ipv4-0.0.0-1.d866cfc-checkout          4.27    0.66   15.5%
go-github-com-pkg-errors              d989q...-go-github-com-pkg-errors-0.0.0-1.e881fd5-checkout          0.05    0.01   19.8%
go-github-com-calmh-du                vk1xv...-go-github-com-calmh-du-1.0.1-checkout                      0.03    0.00    7.9%
go-github-com-minio-sha256-simd       38mjh...-go-github-com-minio-sha256-simd-0.0.0-1.ad98a36-checkout    0.48    0.04    8.1%
go-github-com-oschwald-maxminddb-golang  w2qkr...-go-github-com-oschwald-maxminddb-golang-1.2.0-0.26fe5ac-checkout    0.08    0.02   19.2%
go-github-com-lib-pq                  kl3iv...-go-github-com-lib-pq-0.0.0-1.83612a5-checkout              0.37    0.07   19.8%
go-github-com-tjfoc-gmsm-sm4          j9sdg...-go-github-com-tjfoc-gmsm-sm4-0.0.0-1.98aa888-checkout      0.23    0.05   19.7%
go-github-com-audriusbutkevicius-go-nat-pmp  3sag6...-go-github-com-audriusbutkevicius-go-nat-pmp-0.0.0-0.452c976-checkout    0.01    0.00   18.2%
go-github-com-d4l3k-messagediff       ky0nc...-go-github-com-d4l3k-messagediff-1.1.0-0.29f32d8-checkout    0.05    0.01   16.4%
go-golang-org-x-text-unicode-norm     ac0n4...-go.googlesource.com-text-0.0.0-1.e19ae14-checkout         25.84    3.23   12.5%
go-github-com-audriusbutkevicius-pfilter  nzfpq...-go-github-com-audriusbutkevicius-pfilter-0.0.0-2.9dca34a-checkout    0.01    0.00   20.7%
go-github-com-gogo-protobuf-protoc-gen-gogo  7zacv...-go-github-com-gogo-protobuf-protoc-gen-gogo-0.2-0.efccd33-checkout   23.43    0.80    3.4%
go-github-com-oschwald-geoip2-golang  jf28a...-go-github-com-oschwald-geoip2-golang-1.1.0-checkout        0.04    0.01   14.9%
go-github-com-gogo-protobuf           if2x5...-go-github-com-gogo-protobuf-0.5-1.35b81a0-checkout        15.17    0.68    4.5%
go-github-com-jackpal-gateway         vbzn3...-go-github-com-jackpal-gateway-0.0.0-0.5795ac8-checkout     0.02    0.00   16.5%
go-github-com-petermattis-goid        icb8n...-go-github-com-petermattis-goid-0.0.0-0.3db12eb-checkout    0.03    0.01   19.7%
go-github-com-calmh-xdr               crqwx...-go-github-com-calmh-xdr-2.0.1-0.08e072f-checkout           0.07    0.01   14.1%
go-github-com-gobwas-glob             v3asr...-go-github-com-gobwas-glob-0.0.0-0.51eb1ee-checkout         0.21    0.02   10.1%
go-github-com-stathat-go              3w3df...-go-github-com-stathat-go-0.0.0-0.74669b9-checkout          0.03    0.01   17.4%
go-github-com-syndtr-goleveldb        49g50...-go-github-com-syndtr-goleveldb-0.0.0-2.34011bf-checkout    0.68    0.11   15.8%
go-github-com-kballard-go-shellquote  la1q1...-go-github-com-kballard-go-shellquote-0.0.0-0.cd60e84-checkout    0.02    0.00   20.8%
go-github-com-zillode-notify          28mfc...-go-github-com-zillode-notify-0.0.0-1.8fff849-checkout      0.23    0.05   20.0%
go-github-com-audriusbutkevicius-cli  c1isd...-go-github-com-audriusbutkevicius-cli-0.0.0-0.7f561c7-checkout    0.08    0.01   17.2%
go-github-com-golang-groupcache-lru   hzchs...-go-github-com-golang-groupcache-lru-0.0.0-1.84a468c-checkout    0.13    0.02   17.2%
go-github-com-vitrun-qart-gf256       v7sg9...-go-github-com-vitrun-qart-0.0.0-0.bf64b92-checkout         0.10    0.02   21.0%
go-github-com-rcrowley-go-metrics     3wsdr...-go-github-com-rcrowley-go-metrics-0.0.0-1.e181e09-checkout    0.22    0.03   13.0%
go-golang-org-x-time-rate             a3vcg...-go-golang-org-x-time-rate-0.0.0-1.6dc1736-checkout         0.04    0.01   19.9%
go-github-com-thejerf-suture          ikp7l...-go-github-com-thejerf-suture-2.0.1-0.87e298c-checkout      0.05    0.01   26.1%
go-golang-org-x-sys-unix              6jzmc...-go-golang-org-x-sys-unix-0.0.0-1.8380141-checkout          4.88    0.23    4.7%
go-github-com-ccding-go-stun          9ikw3...-go-github-com-ccding-go-stun-0.0.0-1.d9bbe8f-checkout      0.07    0.01   18.5%
go-github-com-templexxx-cpufeat       igsxi...-go-github-com-templexxx-cpufeat-0.0.0-0.3794dfb-checkout    0.01    0.00   18.4%
go-github-com-klauspost-reedsolomon   0vdn9...-go-github-com-klauspost-reedsolomon-1.6-0.0b30fa7-checkout    0.61    0.10   15.7%
go-github-com-xtaci-kcp-go            hqlkv...-go-github-com-xtaci-kcp-go-0.0.0-0.86eebd5-checkout        0.16    0.07   43.6%
go-golang.org-x-crypto-ssh-terminal   287i8...-go-golang.org-x-crypto-ssh-terminal-0.0.0-0.c78caca-checkout    3.14    1.10   35.0%
go-github.com-nsf-termbox-go          iqxvl...-go-github.com-nsf-termbox-go-0.0.0-0.4ed959e-checkout      0.13    0.03   19.2%
go-github.com-howeyc-gopass           w7047...-go-github.com-howeyc-gopass-0.0.0-0.bf9dde6-checkout       0.04    0.01   26.1%
termite                               f0zkv...-termite-13-checkout                                        0.14    0.02   17.4%
texlive-fonts-latex                   hq540...-svn-checkout                                               0.09    0.01   12.4%
texlive-metapost                      48djy...-svn-checkout                                               3.91    0.56   14.4%
texlive-fonts-rsfs                    i5yp4...-svn-checkout                                               0.05    0.01   15.8%
texlive-latex-trimspaces              4nqlk...-svn-checkout                                               0.01    0.00   15.7%
texlive-latex-psnfss                  2fann...-svn-checkout                                               0.07    0.02   21.3%
texlive-latex-subfigure               n3p72...-svn-checkout                                               0.16    0.03   21.1%
texlive-latex-tools                   dd6jq...-svn-checkout                                               0.94    0.21   22.0%
texlive-latex-colortbl                jhvrk...-svn-checkout                                               0.05    0.01   22.4%
texlive-fonts-amsfonts                j8k2p...-svn-checkout                                               0.69    0.06    9.4%
texlive-latex-float                   vwvqv...-svn-checkout                                               0.05    0.01   26.4%
texlive-latex-fncychap                xmw59...-texlive-latex-fncychap-44591-checkout                      0.03    0.00   17.5%
texlive-latex-oberdiek                3bh3f...-svn-checkout                                               4.66    0.48   10.3%
texlive-latex-cmap                    95ihl...-texlive-latex-cmap-44591-checkout                          0.03    0.00   14.7%
texlive-latex-filemod                 93qvd...-svn-checkout                                               0.02    0.00   12.8%
texlive-latex-wasysym                 nk2rk...-svn-checkout                                               0.03    0.01   22.2%
texlive-latex-anysize                 l3hns...-svn-checkout                                               0.01    0.00   10.5%
texlive-latex-seminar                 78cjs...-svn-checkout                                               0.17    0.03   16.8%
texlive-latex-wrapfig                 rzfx4...-texlive-latex-wrapfig-44591-checkout                       0.03    0.01   26.5%
texlive-tex-texinfo                   5dg1w...-svn-checkout                                               0.40    0.09   23.4%
texlive-latex-dinbrief                6b56j...-texlive-latex-dinbrief-44591-checkout                      0.25    0.05   21.5%
texlive-latex-acronym                 d52jm...-texlive-latex-acronym-44591-checkout                       0.07    0.01   21.6%
texlive-latex-changebar               zj29x...-svn-checkout                                               0.12    0.03   21.6%
texlive-fonts-cm                      15ld8...-svn-checkout                                               0.75    0.08   10.2%
texlive-latex-overpic                 9j1ln...-svn-checkout                                               0.01    0.00   12.4%
texlive-fonts-txfonts                 6j7ik...-texlive-fonts-txfonts-44591-checkout                       0.08    0.01   11.6%
texlive-latex-fontspec                bzwda...-svn-checkout                                               0.34    0.07   21.8%
texlive-latex-amsrefs                 7c149...-texlive-latex-amsrefs-44591-checkout                       0.47    0.10   20.6%
texlive-latex-footmisc                hz7f8...-svn-checkout                                               0.08    0.02   22.9%
texlive-latex-l3packages              qr8j9...-svn-checkout                                               0.36    0.06   16.3%
texlive-fonts-stmaryrd                f7hhc...-svn-checkout                                               0.03    0.01   18.9%
texlive-latex-jknapltx                ysq4b...-svn-checkout                                               0.04    0.01   21.4%
texlive-latex-bigfoot                 3zx72...-texlive-latex-bigfoot-44591-checkout                       0.18    0.04   23.5%
texlive-latex-url                     p7h0j...-svn-checkout                                               0.02    0.00   24.7%
texlive-latex-amscls                  vq5l7...-svn-checkout                                               0.35    0.09   24.3%
texlive-latex-galois                  59ksz...-texlive-latex-galois-44591-checkout                        0.03    0.01   19.6%
texlive-tex-plain                     7zfs9...-svn-checkout                                               3.79    0.38   10.0%
texlive-latex-ifplatform              5vvm4...-texlive-latex-ifplatform-44591-checkout                    0.02    0.01   22.5%
texlive-latex-blindtext               ya3xw...-texlive-latex-blindtext-44591-checkout                     0.09    0.02   19.2%
texlive-latex-enumitem                bzaa3...-texlive-latex-enumitem-44591-checkout                      0.05    0.01   21.7%
texlive-latex-etoolbox                1lfbc...-texlive-latex-etoolbox-44591-checkout                      0.05    0.01   14.6%
texlive-latex-appendix                115ic...-svn-checkout                                               0.05    0.01   21.4%
texlive-latex-amsmath                 7356r...-svn-checkout                                               0.28    0.06   21.6%
texlive-latex-preview                 q8r21...-texlive-latex-preview-44591-checkout                       0.09    0.02   24.9%
texlive-latex-base                    glsk0...-svn-checkout                                               2.73    0.49   18.1%
texlive-generic-babel-english         wg09a...-texlive-generic-babel-english-44591-checkout               0.03    0.01   19.6%
texlive-generic-dehyph-exptl          z0cr7...-svn-checkout                                               0.49    0.08   17.2%
texlive-latex-polyglossia             1dcz8...-texlive-latex-polyglossia-44591-checkout                   0.33    0.07   20.8%
texlive-latex-acmart                  s35zg...-texlive-latex-acmart-1.45-checkout                         0.19    0.04   20.8%
texlive-latex-varwidth                ad5p7...-texlive-latex-varwidth-44591-checkout                      0.02    0.00   20.7%
texlive-latex-mdwtools                5fbjd...-texlive-latex-mdwtools-44591-checkout                      0.42    0.10   23.5%
texlive-latex-parskip                 3a590...-texlive-latex-parskip-44591-checkout                       0.01    0.00   13.7%
texlive-latex-tabulary                gmybm...-texlive-latex-tabulary-44591-checkout                      0.04    0.01   21.8%
texlive-latex-pstool                  bv42n...-texlive-latex-pstool-44591-checkout                        0.05    0.01   26.6%
texlive-latex-expdlist                vj9bl...-texlive-latex-expdlist-44591-checkout                      0.06    0.01   16.6%
texlive-latex-multirow                w7pdh...-svn-checkout                                               0.09    0.02   20.0%
texlive-latex-draftwatermark          z85s4...-texlive-latex-draftwatermark-44591-checkout                0.03    0.01   21.6%
texlive-latex-framed                  bhks6...-texlive-latex-framed-44591-checkout                        0.03    0.01   24.8%
texlive-generic-epsf                  nlr8z...-svn-checkout                                               0.06    0.01   11.5%
texlive-latex-graphics                xb9k1...-svn-checkout                                               0.23    0.05   21.0%
texlive-latex-environ                 hzi8j...-texlive-latex-environ-44591-checkout                       0.03    0.01   22.3%
texlive-latex-babel                   ap2r5...-svn-checkout                                               0.38    0.09   23.6%
texlive-latex-filecontents            aczcl...-svn-checkout                                               0.02    0.01   22.6%
texlive-latex-fancybox                qz4p4...-svn-checkout                                               0.03    0.01   21.1%
texlive-latex-titlesec                151gi...-svn-checkout                                               0.10    0.02   19.2%
texlive-dvips                         q3g4s...-svn-checkout                                               6.92    1.10   15.8%
texlive-latex-psfrag                  z3n4p...-texlive-latex-psfrag-44591-checkout                        0.05    0.01   26.2%
texlive-latex-eepic                   nc1bw...-svn-checkout                                               0.05    0.01   21.1%
texlive-latex-ucs                     hcs8p...-texlive-latex-ucs-44591-checkout                           2.31    0.26   11.3%
texlive-latex-capt-of                 803vw...-texlive-latex-capt-of-44591-checkout                       0.01    0.00    9.3%
texlive-fonts-knuth-lib               a6d1m...-svn-checkout                                               0.05    0.01   24.2%
texlive-latex-lh                      1r3im...-svn-checkout                                               0.08    0.01   16.5%
texlive-generic-tex-ini-files         v31ah...-svn-checkout                                               0.02    0.00   18.7%
texlive-latex-xcolor                  9j8p8...-svn-checkout                                               0.34    0.07   21.9%
texlive-latex-eqparbox                lcsnm...-texlive-latex-eqparbox-44591-checkout                      0.07    0.02   22.7%
texlive-latex-fancyvrb                2kbdl...-svn-checkout                                               0.18    0.03   17.9%
texlive-metafont-base                 df56f...-svn-checkout                                               0.40    0.08   21.1%
texlive-latex-listings                kqch1...-svn-checkout                                               0.94    0.22   24.0%
texlive-latex-g-brief                 cbypr...-texlive-latex-g-brief-44591-checkout                       0.07    0.01   16.5%
texlive-generic-hyph-utf8             rc66j...-svn-checkout                                               8.22    1.04   12.7%
texlive-generic-ifxetex               77c0h...-svn-checkout                                               0.01    0.00   16.9%
texlive-latex-upquote                 h2dw6...-texlive-latex-upquote-44591-checkout                       0.01    0.00   19.3%
texlive-latex-fancyhdr                jnppk...-svn-checkout                                               0.02    0.00   16.0%
texlive-latex-amsfonts                rgyb8...-svn-checkout                                               0.19    0.05   24.1%
texlive-latex-cyrillic                9f9mz...-svn-checkout                                               0.17    0.02   12.2%
texlive-latex-pdfpages                gyzn9...-svn-checkout                                               0.18    0.03   18.5%
texlive-latex-gcite                   j41dd...-texlive-latex-gcite-44591-checkout                         0.03    0.01   26.1%
texlive-latex-supertabular            sp25p...-texlive-latex-supertabular-44591-checkout                  0.06    0.01   18.2%
texlive-latex-l3kernel                apk48...-svn-checkout                                               2.66    0.44   16.6%
texlive-latex-geometry                z88nk...-texlive-latex-geometry-44591-checkout                      0.16    0.03   20.0%
texlive-latex-type1cm                 8lny2...-texlive-latex-type1cm-44591-checkout                       0.02    0.00   12.2%
texlive-latex-eso-pic                 414np...-svn-checkout                                               0.03    0.01   18.8%
texlive-latex-natbib                  548pk...-svn-checkout                                               0.27    0.06   22.3%
texlive-latex-threeparttable          8v3mp...-svn-checkout                                               0.02    0.00   23.7%
texlive-generic-pdftex                8z4xr...-svn-checkout                                               0.22    0.03   11.5%
texlive-fontname                      6cvf0...-texlive-fontname-44591.tar.gz                              1.44    0.11    7.5%
texlive-fonts-ec                      yfs60...-svn-checkout                                               1.12    0.10    8.9%
texlive-generic-unicode-data          p4d5s...-svn-checkout                                               2.11    0.21   10.0%
kakoune                               rj6ks...-kakoune-0.0.0-1.125c8b7-checkout                           3.43    0.32    9.2%
go-github.com-mattn-go-runewidth      p19nc...-go-github.com-mattn-go-runewidth-0.0.2-checkout            0.08    0.01   17.5%
python-editdistance                   szas5...-git-checkout                                               0.11    0.02   14.3%
recode                                xx34z...-recode-3.7.0.201402-checkout                               6.87    0.98   14.2%
cityhash                              x8c1l...-cityhash-1.1-2.8af9b8c.tar.gz                              1.45    0.24   16.5%
tmux-themepack                        f0590...-tmux-themepack-0.0.0-1.03a3728-checkout                    0.10    0.00    4.3%
nyx                                   azabg...-nyx-1.9-1.fea2091-checkout                                 5.34    4.41   82.7%
git-remote-gcrypt                     lxnx6...-git-remote-gcrypt-1.0.1-checkout                           0.10    0.02   22.8%
git-test-sequence                     1hd23...-git-checkout                                               0.04    0.01   25.6%
git-flow                              lxd0v...-git-checkout                                               0.10    0.02   14.9%
myrepos                               1rpcr...-myrepos-1.20170129-checkout                                0.24    0.05   19.4%
ffmpeg-git                            ilml0...-ffmpeg-git-3.4-1.3f8874406-checkout                       60.70    8.23   13.6%
libsmpeg                              nl3nn...-libsmpeg-0.4.5-checkout                                    1.53    0.26   17.0%
vim-scheme                            4qzwm...-vim-scheme-0.0.0-1.9382798-checkout                        0.06    0.00    6.3%
vim-airline-themes                    26h5i...-vim-airline-themes-0.0.0-1.6026eb7-checkout                0.28    0.03    9.0%
vim-neosnippet                        8z74v...-vim-neosnippet-4.2-1.1bd7e23-checkout                      0.39    0.04    9.5%
vim-luna                              ih9qf...-vim-luna-0.0.0-1.6336199-checkout                          0.04    0.00   11.3%
vim-fugitive                          kpssr...-vim-fugitive-2.2-1.de6c057-checkout                        0.15    0.03   19.1%
vim-context-filetype                  m8xds...-vim-context-filetype-0.0.0-1.5e85f8c-checkout              0.07    0.01   10.0%
vim-neosnippet-snippets               5b0hf...-vim-neosnippet-snippets-0.0.0-1.8e2b1c0-checkout           0.15    0.03   19.9%
vpnc-scripts                          zy80j...-git-checkout                                               0.07    0.02   25.5%
w3m                                   827af...-w3m-0.5.3+git20170102-checkout                             9.35    1.11   11.9%
libwebsockets                         1k4c1...-libwebsockets-1.3                                          3.60    2.60   72.1%
nginx-documentation                   8f7pf...-nginx-documentation-2017-04-12-1961-dd4b6c564e10          10.43    1.54   14.8%
wget2                                 3c93r...-wget2-1.0.0-checkout                                      42.92    4.87   11.3%
python2-i3-py                         2ivq6...-python-i3-py-0.6.5-checkout                                0.06    0.01   21.1%
nxbelld                               h3a3x...-nxbelld-0.1.2-checkout                                     1.40    0.22   15.4%
dzen                                  jcc21...-dzen-0.9.5-1.488ab66                                       0.17    0.03   18.6%
xcompmgr                              h77gz...-xcompmgr-1.1.7                                             0.08    0.01   19.6%
xf86-video-intel                      p9fk4...-xf86-video-intel-2.99.917-9-af6d8e9                        9.07    0.91   10.1%
xf86-video-freedreno                  j21yc...-xf86-video-freedreno-1.4.0-1-ccba8f8                       0.28    0.05   16.6%
zile-on-guile                         qcjsz...-zile-on-guile-2.4.14-0.fd09781-checkout                   30.74    3.48   11.3%
------------------------------------------------------------------------------------------------------------------------------
TOTALS:                                                                                                3783.86  538.07   18.6% (median)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Compress VCS checkouts?
  2018-02-02 20:50 Compress VCS checkouts? Eric Bavier
@ 2018-02-03 20:10 ` Efraim Flashner
  2018-02-05 10:27 ` Ludovic Courtès
  1 sibling, 0 replies; 5+ messages in thread
From: Efraim Flashner @ 2018-02-03 20:10 UTC (permalink / raw)
  To: Eric Bavier; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 2424 bytes --]

On Fri, Feb 02, 2018 at 02:50:11PM -0600, Eric Bavier wrote:
> Hello Guix,
> 
> A while back I toyed with the idea of a "Sources DVD" for Guix, much
> like Trisquel has. While looking into the feasability of this, I
> noticed that the git/svn/cvs/hg checkouts our packages use are not
> compressed in the store.  So I wrote a simple script to compare the raw
> size of all source checkouts with their corresponding xz-compressed
> tarballs.  
> 
> At the time, I couldn't convince myself the trouble was worth the
> effort; we had only 385MiB of uncompressed source checkouts, which
> would compress to 135MiB.
> 
> Since that initial investigation, the number of packages using VCS
> checkouts has risen from 50 to 389, and the on-disk size of that source
> has risen to 3784MiB uncompressed.  We could compress that down to
> 538MiB.
> 
> Compression in not free, but I think it would be nice for our build
> servers who have to store and serve (which, iirc, requires them to
> compress it anyhow) this source.
> 
> What do people think?
> 
> `~Eric

> name                                  path                                                           size (MiB) tar.xz    rel%
> footswitch                            cwrqb...-footswitch-0.1-1.7cb0a93-checkout                          0.04    0.01   17.5%
< snip >
> ------------------------------------------------------------------------------------------------------------------------------
> TOTALS:                                                                                                3783.86  538.07   18.6% (median)

In case anyone else had to look at it twice, that's the relative size of
the new tarred source compared to the original. The sources don't change
very often, so the cost of the initial compression doesn't come up much,
and serving compressed data should be much faster. Interestingly, on
machines with the store on an external HDD, it might be faster to load
and unpack the source to ram than to just copy over USB2.

It does make grepping through the source in the store harder, but we
already couldn't do that with the tarballs. I think it would be overall
a positive change.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Compress VCS checkouts?
  2018-02-02 20:50 Compress VCS checkouts? Eric Bavier
  2018-02-03 20:10 ` Efraim Flashner
@ 2018-02-05 10:27 ` Ludovic Courtès
  2018-02-06  0:49   ` Eric Bavier
  1 sibling, 1 reply; 5+ messages in thread
From: Ludovic Courtès @ 2018-02-05 10:27 UTC (permalink / raw)
  To: Eric Bavier; +Cc: guix-devel

Hello!

Eric Bavier <ericbavier@centurylink.net> skribis:

> A while back I toyed with the idea of a "Sources DVD" for Guix, much
> like Trisquel has. While looking into the feasability of this, I
> noticed that the git/svn/cvs/hg checkouts our packages use are not
> compressed in the store.  So I wrote a simple script to compare the raw
> size of all source checkouts with their corresponding xz-compressed
> tarballs.  
>
> At the time, I couldn't convince myself the trouble was worth the
> effort; we had only 385MiB of uncompressed source checkouts, which
> would compress to 135MiB.
>
> Since that initial investigation, the number of packages using VCS
> checkouts has risen from 50 to 389, and the on-disk size of that source
> has risen to 3784MiB uncompressed.  We could compress that down to
> 538MiB.
>
> Compression in not free, but I think it would be nice for our build
> servers who have to store and serve (which, iirc, requires them to
> compress it anyhow) this source.

‘guix publish -C’ always compresses store items, unless they are regular
files ending in “.gz”, “.xz”, etc.  So there’s no problem here.

It seems to me that the only benefit of compressing checkouts would be
the use case you’re talking about.  For everyday use, it’s actually more
efficient to leave the uncompressed, so that people don’t end up doing
round of compression/decompression when they run “guix build”.

WDYT?

That said, at worst, this could be addressed by writing a specific tool
for the job.

Ludo’.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Compress VCS checkouts?
  2018-02-05 10:27 ` Ludovic Courtès
@ 2018-02-06  0:49   ` Eric Bavier
  2018-02-07 21:56     ` Ludovic Courtès
  0 siblings, 1 reply; 5+ messages in thread
From: Eric Bavier @ 2018-02-06  0:49 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

On Mon, 05 Feb 2018 11:27:10 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Hello!
> 
> Eric Bavier <ericbavier@centurylink.net> skribis:
> 
> > A while back I toyed with the idea of a "Sources DVD" for Guix, much
> > like Trisquel has. While looking into the feasability of this, I
> > noticed that the git/svn/cvs/hg checkouts our packages use are not
> > compressed in the store.  So I wrote a simple script to compare the raw
> > size of all source checkouts with their corresponding xz-compressed
> > tarballs.  
> >
> > At the time, I couldn't convince myself the trouble was worth the
> > effort; we had only 385MiB of uncompressed source checkouts, which
> > would compress to 135MiB.
> >
> > Since that initial investigation, the number of packages using VCS
> > checkouts has risen from 50 to 389, and the on-disk size of that source
> > has risen to 3784MiB uncompressed.  We could compress that down to
> > 538MiB.
> >
> > Compression in not free, but I think it would be nice for our build
> > servers who have to store and serve (which, iirc, requires them to
> > compress it anyhow) this source.  
> 
> ‘guix publish -C’ always compresses store items, unless they are regular
> files ending in “.gz”, “.xz”, etc.  So there’s no problem here.

I see, ok.

> 
> It seems to me that the only benefit of compressing checkouts would be
> the use case you’re talking about.  For everyday use, it’s actually more
> efficient to leave the uncompressed, so that people don’t end up doing
> round of compression/decompression when they run “guix build”.

Often, people would end up fetching the compressed source from a
substitute server, no?  And currently I already need to uncompress
what I get from a substitute server before building.

But now I'm wondering why we bother with the "repack" portion of
"patch-and-repack"?

> That said, at worst, this could be addressed by writing a specific tool
> for the job.

Yes. I was hoping it could be avoided though.

In the meantime I can continue to monitor the situation.

`~Eric

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Compress VCS checkouts?
  2018-02-06  0:49   ` Eric Bavier
@ 2018-02-07 21:56     ` Ludovic Courtès
  0 siblings, 0 replies; 5+ messages in thread
From: Ludovic Courtès @ 2018-02-07 21:56 UTC (permalink / raw)
  To: Eric Bavier; +Cc: guix-devel

Hi Eric,

Eric Bavier <ericbavier@centurylink.net> skribis:

> On Mon, 05 Feb 2018 11:27:10 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Hello!
>> 
>> Eric Bavier <ericbavier@centurylink.net> skribis:
>> 
>> > A while back I toyed with the idea of a "Sources DVD" for Guix, much
>> > like Trisquel has. While looking into the feasability of this, I
>> > noticed that the git/svn/cvs/hg checkouts our packages use are not
>> > compressed in the store.  So I wrote a simple script to compare the raw
>> > size of all source checkouts with their corresponding xz-compressed
>> > tarballs.  
>> >
>> > At the time, I couldn't convince myself the trouble was worth the
>> > effort; we had only 385MiB of uncompressed source checkouts, which
>> > would compress to 135MiB.
>> >
>> > Since that initial investigation, the number of packages using VCS
>> > checkouts has risen from 50 to 389, and the on-disk size of that source
>> > has risen to 3784MiB uncompressed.  We could compress that down to
>> > 538MiB.
>> >
>> > Compression in not free, but I think it would be nice for our build
>> > servers who have to store and serve (which, iirc, requires them to
>> > compress it anyhow) this source.  
>> 
>> ‘guix publish -C’ always compresses store items, unless they are regular
>> files ending in “.gz”, “.xz”, etc.  So there’s no problem here.
>
> I see, ok.
>
>> 
>> It seems to me that the only benefit of compressing checkouts would be
>> the use case you’re talking about.  For everyday use, it’s actually more
>> efficient to leave the uncompressed, so that people don’t end up doing
>> round of compression/decompression when they run “guix build”.
>
> Often, people would end up fetching the compressed source from a
> substitute server, no?

Specifically, they download a compressed nar that contains the
uncompressed checkout.

> And currently I already need to uncompress what I get from a
> substitute server before building.
>
> But now I'm wondering why we bother with the "repack" portion of
> "patch-and-repack"?

Good question!  It’s mostly because it “looked nicer” this way (and
maybe at the time some of the code did not properly handle non-tarball
sources.)  I hear we could apply the same “reasoning” (if we can call it
this way) to checkouts.  Hmm!

>> That said, at worst, this could be addressed by writing a specific tool
>> for the job.
>
> Yes. I was hoping it could be avoided though.

The tool could be relatively simple.

Ludo’.

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2018-02-07 21:56 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-02-02 20:50 Compress VCS checkouts? Eric Bavier
2018-02-03 20:10 ` Efraim Flashner
2018-02-05 10:27 ` Ludovic Courtès
2018-02-06  0:49   ` Eric Bavier
2018-02-07 21:56     ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).