[PATCH] scripts: hash: Add --git option. WIP

[PATCH] scripts: hash: Add --git option. WIP

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]

Hi!

Attached is a patch to get the hash of a git archive without having to
clean the tree or do a clean checkout.

Using

    guix hash -gr .

procudes the same hash as doing something like

    git clone . tmp && guix hash -rx tmp && rm -r tmp

I marked it as WIP because while it is already "handy" as it is, I
consider adding a commit argument and imply --recursive, like so

   guix hash --git HEAD
   guix hash --git v0.13

WDYT?

janneke


[-- Attachment #2: 0001-scripts-hash-Add-git-option.-WIP.patch --]
[-- Type: text/x-patch, Size: 5139 bytes --]

From cfc9e557db6fe6c9aece68cfc5153ec9481a45a4 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Thu, 23 Nov 2017 04:30:13 +0100
Subject: [PATCH] scripts: hash: Add --git option.  WIP

Using

    guix hash -gr .

procudes the same hash as doing something like

    git clone . tmp && guix hash -rx tmp && rm -r tmp

* guix/git.scm (git-ls-files): New function.
* guix/scripts/hash.scm (%options, show-help): Add `--git'.
(guix-hash)[git-file?]: New function.
---
 guix/git.scm          | 12 +++++++++++-
 guix/scripts/hash.scm | 33 +++++++++++++++++++++++++++++----
 2 files changed, 40 insertions(+), 5 deletions(-)

diff --git a/guix/git.scm b/guix/git.scm
index fc41e2ace..3fc6abcbc 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
+;;; Copyright © 2017 Jan Nieuwenhuizen <janneke@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -28,7 +29,8 @@
   #:use-module (ice-9 match)
   #:use-module (srfi srfi-1)
   #:export (%repository-cache-directory
-            latest-repository-commit))
+            latest-repository-commit
+            git-ls-files))
 
 (define %repository-cache-directory
   (make-parameter "/var/cache/guix/checkouts"))
@@ -132,3 +134,11 @@ Git repositories are kept in the cache directory specified by
      (copy-to-store store cache-dir
                     #:url url
                     #:repository repository))))
+
+(define (git-ls-files directory)
+  (with-libgit2
+   (let* ((repository (repository-open directory))
+          (oid (reference-target (repository-head repository)))
+          (commit (commit-lookup repository oid))
+          (tree (commit-tree commit)))
+     (tree-list tree))))
diff --git a/guix/scripts/hash.scm b/guix/scripts/hash.scm
index cae5d6bcd..261283b01 100644
--- a/guix/scripts/hash.scm
+++ b/guix/scripts/hash.scm
@@ -1,7 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2012, 2013, 2014, 2016, 2017 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
-;;; Copyright © 2016 Jan Nieuwenhuizen <janneke@gnu.org>
+;;; Copyright © 2016,2017 Jan Nieuwenhuizen <janneke@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -20,6 +20,7 @@
 
 (define-module (guix scripts hash)
   #:use-module (guix base32)
+  #:use-module (guix git)
   #:use-module (guix hash)
   #:use-module (guix serialization)
   #:use-module (guix ui)
@@ -52,6 +53,8 @@ and 'hexadecimal' can be used as well).\n"))
   (format #t (G_ "
   -x, --exclude-vcs      exclude version control directories"))
   (format #t (G_ "
+  -g, --git              consider git files only"))
+  (format #t (G_ "
   -f, --format=FMT       write the hash in the given format"))
   (format #t (G_ "
   -r, --recursive        compute the hash on FILE recursively"))
@@ -68,6 +71,9 @@ and 'hexadecimal' can be used as well).\n"))
   (list (option '(#\x "exclude-vcs") #f #f
                 (lambda (opt name arg result)
                   (alist-cons 'exclude-vcs? #t result)))
+        (option '(#\g "git") #f #f
+                (lambda (opt name arg result)
+                  (alist-cons 'git? #t result)))
         (option '(#\f "format") #t #f
                 (lambda (opt name arg result)
                   (define fmt-proc
@@ -117,6 +123,21 @@ and 'hexadecimal' can be used as well).\n"))
       (else
        #f)))
 
+  (define (git-file? directory)
+    (let* ((files (git-ls-files directory))
+           (directories (delete-duplicates (map dirname files)))
+           (prefix (if (string-suffix? "/" directory) directory
+                       (string-append directory "/")))
+           (prefix-length (string-length prefix)))
+      (lambda (file stat)
+        (case (stat:type stat)
+          ((directory)
+           (member (string-drop file prefix-length) directories))
+          ((regular)
+           (member (string-drop file prefix-length) files))
+          (else
+           #f)))))
+
   (let* ((opts (parse-options))
          (args (filter-map (match-lambda
                             (('argument . value)
@@ -124,9 +145,13 @@ and 'hexadecimal' can be used as well).\n"))
                             (_ #f))
                            (reverse opts)))
          (fmt  (assq-ref opts 'format))
-         (select? (if (assq-ref opts 'exclude-vcs?)
-                      (negate vcs-file?)
-                      (const #t))))
+         (select? (cond
+                   ((assq-ref opts 'exclude-vcs?)
+                    (negate vcs-file?))
+                   ((assq-ref opts 'git?)
+                    (git-file? (car args)))
+                   (else
+                    (const #t)))))
 
     (define (file-hash file)
       ;; Compute the hash of FILE.
-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com


[-- Attachment #3: Type: text/plain, Size: 152 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: [PATCH] scripts: hash: Add --git option. WIP

[Ludovic Courtès]

Hello,

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> Attached is a patch to get the hash of a git archive without having to
> clean the tree or do a clean checkout.
>
> Using
>
>     guix hash -gr .
>
> procudes the same hash as doing something like
>
>     git clone . tmp && guix hash -rx tmp && rm -r tmp

Very useful!

> I marked it as WIP because while it is already "handy" as it is, I
> consider adding a commit argument and imply --recursive, like so
>
>    guix hash --git HEAD
>    guix hash --git v0.13
>
> WDYT?

This can always be added in a later patch.

> From cfc9e557db6fe6c9aece68cfc5153ec9481a45a4 Mon Sep 17 00:00:00 2001
> From: Jan Nieuwenhuizen <janneke@gnu.org>
> Date: Thu, 23 Nov 2017 04:30:13 +0100
> Subject: [PATCH] scripts: hash: Add --git option.  WIP
>
> Using
>
>     guix hash -gr .
>
> procudes the same hash as doing something like
>
>     git clone . tmp && guix hash -rx tmp && rm -r tmp
>
> * guix/git.scm (git-ls-files): New function.
> * guix/scripts/hash.scm (%options, show-help): Add `--git'.
> (guix-hash)[git-file?]: New function.

[...]

> +(define (git-ls-files directory)

What about ‘git-file-list’ or ‘git-controlled-files’ rather?
With a docstring too.  :-)

> +  (define (git-file? directory)
> +    (let* ((files (git-ls-files directory))
> +           (directories (delete-duplicates (map dirname files)))
> +           (prefix (if (string-suffix? "/" directory) directory
> +                       (string-append directory "/")))
> +           (prefix-length (string-length prefix)))
> +      (lambda (file stat)
> +        (case (stat:type stat)
> +          ((directory)
> +           (member (string-drop file prefix-length) directories))
> +          ((regular)
> +           (member (string-drop file prefix-length) files))
> +          (else
> +           #f)))))

This appears to duplicate ‘git-predicate’ from (guix git-download),
which was carefully optimized for large trees by Chris.

What about:

  1. Rewrite ‘git-predicate’ to use the new ‘git-file-list’ (currently
     is shells out ‘git’);

  2. Moving ‘git-predicate’ to (guix git).

?

I understand this is probably more than you were willing to do ;-), but
it should be beneficial.

> +         (select? (cond
> +                   ((assq-ref opts 'exclude-vcs?)
> +                    (negate vcs-file?))
> +                   ((assq-ref opts 'git?)
> +                    (git-file? (car args)))
> +                   (else
> +                    (const #t)))))

I think we should use ‘match’ to gracefully handle any errors:

  ((assq-ref opts 'git?)
   (match args
     ((file) (git-predicate file))
     (_      (const #f))))

Thanks,
Ludo’.

Re: [PATCH] scripts: hash: Add --git option. WIP

[Andy Wingo]

On Thu 23 Nov 2017 04:54, Jan Nieuwenhuizen <janneke@gnu.org> writes:

> Hi!
>
> Attached is a patch to get the hash of a git archive without having to
> clean the tree or do a clean checkout.
>
> Using
>
>     guix hash -gr .
>
> procudes the same hash as doing something like
>
>     git clone . tmp && guix hash -rx tmp && rm -r tmp
>
> I marked it as WIP because while it is already "handy" as it is, I
> consider adding a commit argument and imply --recursive, like so
>
>    guix hash --git HEAD
>    guix hash --git v0.13
>
> WDYT?

Weird that we have done the same thing :)

https://git.savannah.gnu.org/gitweb/?p=guix.git;a=commit;h=572907daff98a77a4215861a88b81d2f30542c09

Andy

Re: [PATCH] scripts: hash: Add --git option. WIP

[Jan Nieuwenhuizen]

Andy Wingo writes:

>>    guix hash --git HEAD
>>    guix hash --git v0.13
>>
>> WDYT?
>
> Weird that we have done the same thing :)
>
> https://git.savannah.gnu.org/gitweb/?p=guix.git;a=commit;h=572907daff98a77a4215861a88b81d2f30542c09

:-) Nice!...on what branch is that?  You did see Ludo's suggestions?  I like
them and planned to look at them when my mind is less occupied with
bootstrapping.

janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: [PATCH] scripts: hash: Add --git option. WIP

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

On Tue, Nov 28, 2017 at 07:57:32PM +0100, Jan Nieuwenhuizen wrote:
> Andy Wingo writes:
> > Weird that we have done the same thing :)
> >
> > https://git.savannah.gnu.org/gitweb/?p=guix.git;a=commit;h=572907daff98a77a4215861a88b81d2f30542c09
> 
> :-) Nice!...on what branch is that?  You did see Ludo's suggestions?  I like
> them and planned to look at them when my mind is less occupied with
> bootstrapping.

I wasn't sure how to figure this out, so here's how to do it:

$ git branch --remote --contains 572907daff98a77a4215861a88b81d2f30
  origin/wip-potluck

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]