bug#26687: add git http support service

bug#26687: add git http support service

[Andy Wingo]

[-- Attachment #1: Type: text/plain, Size: 360 bytes --]

Attached pair of patches adds git http "support" to the manual.  I say
"support" because really what you usually want to do is extend an
existing nginx server with git support, not make an entirely new
server.  But right now we can't extend servers, so this support is a bit
preliminary.  Still, it's better than ad hoc logic in peoples'
configuration files.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0004-doc-Create-Version-Control-Services-section.patch --]
[-- Type: text/x-patch, Size: 6352 bytes --]

From 80d872ae8362b5e769c463599b03ddb0fb7482d2 Mon Sep 17 00:00:00 2001
From: Andy Wingo <wingo@igalia.com>
Date: Thu, 27 Apr 2017 10:59:44 +0200
Subject: [PATCH 4/5] doc: Create Version Control Services section.

* doc/guix.texi (Version Control Services): Move to its own node.
---
 doc/guix.texi | 124 ++++++++++++++++++++++++++++++----------------------------
 1 file changed, 64 insertions(+), 60 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index eeacf8833..cae497df3 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -220,6 +220,7 @@ Services
 * Network File System::         NFS related services.
 * Continuous Integration::      The Cuirass service.
 * Power management Services::   The TLP tool.
+* Version Control Services::    Providing remote access to Git repositories.
 * Miscellaneous Services::      Other services.
 
 Defining Services
@@ -8611,6 +8612,7 @@ declaration.
 * Network File System::         NFS related services.
 * Continuous Integration::      The Cuirass service.
 * Power management Services::   The TLP tool.
+* Version Control Services::    Providing remote access to Git repositories.
 * Miscellaneous Services::      Other services.
 @end menu
 
@@ -14639,6 +14641,68 @@ Defaults to @samp{#f}.
 
 @end deftypevr
 
+@node Version Control Services
+@subsubsection Version Control Services
+
+The @code{(gnu services version-control)} module provides a service to
+allow remote access to local Git repositories.
+
+@deffn {Scheme Procedure} git-daemon-service [#:config (git-daemon-configuration)]
+
+Return a service that runs @command{git daemon}, a simple TCP server to
+expose repositories over the Git protocol for anonymous access.
+
+The optional @var{config} argument should be a
+@code{<git-daemon-configuration>} object, by default it allows read-only
+access to exported@footnote{By creating the magic file
+"git-daemon-export-ok" in the repository directory.} repositories under
+@file{/srv/git}.
+
+@end deffn
+
+@deftp {Data Type} git-daemon-configuration
+Data type representing the configuration for @code{git-daemon-service}.
+
+@table @asis
+@item @code{package} (default: @var{git})
+Package object of the Git distributed version control system.
+
+@item @code{export-all?} (default: @var{#f})
+Whether to allow access for all Git repositories, even if they do not
+have the @file{git-daemon-export-ok} file.
+
+@item @code{base-path} (default: @file{/srv/git})
+Whether to remap all the path requests as relative to the given path.
+If you run git daemon with @var{(base-path "/srv/git")} on example.com,
+then if you later try to pull @code{git://example.com/hello.git}, git
+daemon will interpret the path as @code{/srv/git/hello.git}.
+
+@item @code{user-path} (default: @var{#f})
+Whether to allow @code{~user} notation to be used in requests.  When
+specified with empty string, requests to @code{git://host/~alice/foo} is
+taken as a request to access @code{foo} repository in the home directory
+of user @code{alice}.  If @var{(user-path "path")} is specified, the
+same request is taken as a request to access @code{path/foo} repository
+in the home directory of user @code{alice}.
+
+@item @code{listen} (default: @var{'()})
+Whether to listen on specific IP addresses or hostnames, defaults to
+all.
+
+@item @code{port} (default: @var{#f})
+Whether to listen on an alternative port, which defaults to 9418.
+
+@item @code{whitelist} (default: @var{'()})
+If not empty, only allow access to this list of directories.
+
+@item @code{extra-options} (default: @var{'()})
+Extra options will be passed to @code{git daemon}, please run
+@command{man git-daemon} for more information.
+
+@end table
+@end deftp
+
+
 @node Miscellaneous Services
 @subsubsection Miscellaneous Services
 
@@ -14771,66 +14835,6 @@ The following is an example @code{dicod-service} configuration.
                     %dicod-database:gcide))))
 @end example
 
-@subsubsection Version Control
-
-The @code{(gnu services version-control)} module provides the following services:
-
-@subsubheading Git daemon service
-
-@deffn {Scheme Procedure} git-daemon-service [#:config (git-daemon-configuration)]
-
-Return a service that runs @command{git daemon}, a simple TCP server to
-expose repositories over the Git protocol for anonymous access.
-
-The optional @var{config} argument should be a
-@code{<git-daemon-configuration>} object, by default it allows read-only
-access to exported@footnote{By creating the magic file
-"git-daemon-export-ok" in the repository directory.} repositories under
-@file{/srv/git}.
-
-@end deffn
-
-@deftp {Data Type} git-daemon-configuration
-Data type representing the configuration for @code{git-daemon-service}.
-
-@table @asis
-@item @code{package} (default: @var{git})
-Package object of the Git distributed version control system.
-
-@item @code{export-all?} (default: @var{#f})
-Whether to allow access for all Git repositories, even if they do not
-have the @file{git-daemon-export-ok} file.
-
-@item @code{base-path} (default: @file{/srv/git})
-Whether to remap all the path requests as relative to the given path.
-If you run git daemon with @var{(base-path "/srv/git")} on example.com,
-then if you later try to pull @code{git://example.com/hello.git}, git
-daemon will interpret the path as @code{/srv/git/hello.git}.
-
-@item @code{user-path} (default: @var{#f})
-Whether to allow @code{~user} notation to be used in requests.  When
-specified with empty string, requests to @code{git://host/~alice/foo} is
-taken as a request to access @code{foo} repository in the home directory
-of user @code{alice}.  If @var{(user-path "path")} is specified, the
-same request is taken as a request to access @code{path/foo} repository
-in the home directory of user @code{alice}.
-
-@item @code{listen} (default: @var{'()})
-Whether to listen on specific IP addresses or hostnames, defaults to
-all.
-
-@item @code{port} (default: @var{#f})
-Whether to listen on an alternative port, which defaults to 9418.
-
-@item @code{whitelist} (default: @var{'()})
-If not empty, only allow access to this list of directories.
-
-@item @code{extra-options} (default: @var{'()})
-Extra options will be passed to @code{git daemon}, please run
-@command{man git-daemon} for more information.
-
-@end table
-@end deftp
 
 @node Setuid Programs
 @subsection Setuid Programs
-- 
2.12.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0005-gnu-Add-Git-HTTP-S-service-support.patch --]
[-- Type: text/x-patch, Size: 7728 bytes --]

From cb9f33e2a70b304cc5712d8a369dc8db7a8ae95b Mon Sep 17 00:00:00 2001
From: Andy Wingo <wingo@igalia.com>
Date: Thu, 27 Apr 2017 15:02:19 +0200
Subject: [PATCH 5/5] gnu: Add Git HTTP(S) service support.

* doc/guix.texi (Version Control Services): Add documentation on the HTTP
backend for git.
* gnu/services/version-control.scm (<git-http-configuration>): New data type.
(git-http-nginx-location-configuration): New helper function.
---
 doc/guix.texi                    | 80 +++++++++++++++++++++++++++++++++++++++-
 gnu/services/version-control.scm | 54 ++++++++++++++++++++++++++-
 2 files changed, 132 insertions(+), 2 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index cae497df3..85235846d 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -14645,7 +14645,11 @@ Defaults to @samp{#f}.
 @subsubsection Version Control Services
 
 The @code{(gnu services version-control)} module provides a service to
-allow remote access to local Git repositories.
+allow remote access to local Git repositories.  There are two options:
+the @code{git-daemon-service}, which provides access to repositories via
+the @code{git://} unsecured TCP-based protocol, or extending the
+@code{nginx} web server to proxy some requests to
+@code{git-http-backend}.
 
 @deffn {Scheme Procedure} git-daemon-service [#:config (git-daemon-configuration)]
 
@@ -14702,6 +14706,80 @@ Extra options will be passed to @code{git daemon}, please run
 @end table
 @end deftp
 
+The @code{git://} protocol lacks authentication.  When you pull from a
+repository fetched via @code{git://}, you don't know that the data you
+receive was modified in transit or not.  It's better to use an
+authenticated transport, such as @code{https}.  Although Git allows you
+to serve repositories using unsophisticated file-based web servers,
+there is a faster protocol implemented by the @code{git-http-backend}
+program.  This program is the back-end of a proper Git web service.  It
+is designed to sit behind a FastCGI proxy.  @xref{Web Services}, for more
+on running the necessary @code{fcgiwrap} daemon.
+
+Guix has a separate configuration data type for serving Git repositories
+over HTTP.
+
+@deftp {Data Type} git-http-configuration
+Data type representing the configuration for @code{git-http-service}.
+
+@table @asis
+@item @code{package} (default: @var{git})
+Package object of the Git distributed version control system.
+
+@item @code{git-root} (default: @file{/srv/git})
+Directory containing the Git repositories to expose to the world.
+
+@item @code{export-all?} (default: @var{#f})
+Whether to expose access for all Git repositories in @var{git-root},
+even if they do not have the @file{git-daemon-export-ok} file.
+
+@item @code{uri-path} (default: @file{/git/})
+Path prefix for Git access.  With the default @code{/git/} prefix, this
+will map @code{http://@var{server}/git/@var{repo}.git} to
+@code{/srv/git/@var{repo}.git}.  Requests whose URI paths do not begin
+with this prefix are not passed on to this Git instance.
+
+@item @code{fcgiwrap-socket} (default: @code{127.0.0.1:9000})
+The socket on which the @code{fcgiwrap} daemon is listening.  @xref{Web
+Services}.
+@end table
+@end deftp
+
+There is no @code{git-http-service-type}, currently; instead you can
+create an @code{nginx-location-configuration} from a
+@code{git-http-configuration} and then add that location to a web
+server.
+
+@deffn {Scheme Procedure} git-http-nginx-location-configuration @
+       [config=(git-http-configuration)] Compute an
+@code{nginx-location-configuration} the corresponds with the given Git
+http configuration.  An example nginx service definition to serve the
+default @file{/srv/git} over HTTPS might be:
+
+@example
+(service nginx-service-type
+         (nginx-configuration
+          (server-blocks
+           (list
+            (nginx-server-configuration
+             (http-port #f)
+             (server-name "git.my-host.org")
+             (ssl-certificate
+              "/etc/letsencrypt/live/git.my-host.org/fullchain.pem")
+             (ssl-certificate-key
+              "/etc/letsencrypt/live/git.my-host.org/privkey.pem")
+             (locations
+              (list
+               (git-http-nginx-location-configuration
+                (git-http-configuration (uri-path "/"))))))))))
+@end example
+
+This example assumes that you are using Let's Encrypt to get your TLS
+certificate.  @xref{Certificate Services}.  The default @code{certbot}
+service will redirect all HTTP traffic on @code{git.my-host.org} to
+HTTPS.  You will also need to add an @code{fcgiwrap} proxy to your
+system services.  @xref{Web Services}.
+@end deffn
 
 @node Miscellaneous Services
 @subsubsection Miscellaneous Services
diff --git a/gnu/services/version-control.scm b/gnu/services/version-control.scm
index 107bc8e77..125d63653 100644
--- a/gnu/services/version-control.scm
+++ b/gnu/services/version-control.scm
@@ -21,6 +21,7 @@
   #:use-module (gnu services)
   #:use-module (gnu services base)
   #:use-module (gnu services shepherd)
+  #:use-module (gnu services web)
   #:use-module (gnu system shadow)
   #:use-module (gnu packages version-control)
   #:use-module (gnu packages admin)
@@ -32,7 +33,11 @@
   #:export (git-daemon-service
             git-daemon-service-type
             git-daemon-configuration
-            git-daemon-configuration?))
+            git-daemon-configuration?
+
+            git-http-configuration
+            git-http-configuration?
+            git-http-nginx-location-configuration))
 
 ;;; Commentary:
 ;;;
@@ -139,3 +144,50 @@ The optional @var{config} argument should be a
 @code{<git-daemon-configuration>} object, by default it allows read-only
 access to exported repositories under @file{/srv/git}."
   (service git-daemon-service-type config))
+
+
+\f
+;;;
+;;; HTTP access.  Add the result of calling
+;;; git-http-nginx-location-configuration to an nginx-server-configuration's
+;;; "locations" field.
+;;;
+
+(define-record-type* <git-http-configuration>
+  git-http-configuration
+  make-git-http-configuration
+  git-http-configuration?
+  (package          git-http-configuration-package        ;package
+                    (default git))
+  (git-root         git-http-configuration-git-root       ;string
+                    (default "/srv/git"))
+  (export-all?      git-http-configuration-export-all?    ;boolean
+                    (default #f))
+  (uri-path         git-http-configuration-uri-path       ;string
+                    (default "/git/"))
+  (fcgiwrap-socket  git-http-configuration-fcgiwrap-socket ;string
+                    (default "127.0.0.1:9000")))
+
+(define* (git-http-nginx-location-configuration #:optional
+                                                (config
+                                                 (git-http-configuration)))
+  (match config
+    (($ <git-http-configuration> package git-root export-all?
+                                 uri-path fcgiwrap-socket)
+     (nginx-location-configuration
+      (uri (string-append "~ /" (string-trim-both uri-path #\/) "(/.*)"))
+      (body
+       (list
+        (list "fastcgi_pass " fcgiwrap-socket ";")
+        (list "fastcgi_param SCRIPT_FILENAME "
+              package "/libexec/git-core/git-http-backend"
+              ";")
+        "fastcgi_param QUERY_STRING $query_string;"
+        "fastcgi_param REQUEST_METHOD $request_method;"
+        "fastcgi_param CONTENT_TYPE $content_type;"
+        "fastcgi_param CONTENT_LENGTH $content_length;"
+        (if export-all?
+            "fastcgi_param GIT_HTTP_EXPORT_ALL \"\";"
+            "")
+        (list "fastcgi_param GIT_PROJECT_ROOT " git-root ";")
+        "fastcgi_param PATH_INFO $1;"))))))
-- 
2.12.2

bug#26687: add git http support service

[Ludovic Courtès]

Andy Wingo <wingo@igalia.com> skribis:

> Attached pair of patches adds git http "support" to the manual.  I say
> "support" because really what you usually want to do is extend an
> existing nginx server with git support, not make an entirely new
> server.  But right now we can't extend servers, so this support is a bit
> preliminary.  Still, it's better than ad hoc logic in peoples'
> configuration files.

Agreed, that’s a nice addition.

>>From 80d872ae8362b5e769c463599b03ddb0fb7482d2 Mon Sep 17 00:00:00 2001
> From: Andy Wingo <wingo@igalia.com>
> Date: Thu, 27 Apr 2017 10:59:44 +0200
> Subject: [PATCH 4/5] doc: Create Version Control Services section.
>
> * doc/guix.texi (Version Control Services): Move to its own node.

OK.

>>From cb9f33e2a70b304cc5712d8a369dc8db7a8ae95b Mon Sep 17 00:00:00 2001
> From: Andy Wingo <wingo@igalia.com>
> Date: Thu, 27 Apr 2017 15:02:19 +0200
> Subject: [PATCH 5/5] gnu: Add Git HTTP(S) service support.
>
> * doc/guix.texi (Version Control Services): Add documentation on the HTTP
> backend for git.
> * gnu/services/version-control.scm (<git-http-configuration>): New data type.
> (git-http-nginx-location-configuration): New helper function.

[...]

> +The @code{git://} protocol lacks authentication.  When you pull from a
> +repository fetched via @code{git://}, you don't know that the data you
> +receive was modified in transit or not.  It's better to use an

Nitpick: “modified in transit” refers to integrity, not authenticity;
perhaps write “you don’t know that the data you receive is really coming
from the specified host”?

Also maybe “lacks authentication and confidentiality”.

> +(define* (git-http-nginx-location-configuration #:optional
> +                                                (config
> +                                                 (git-http-configuration)))

Perhaps “git-http-configuration->nginx-location”?

OK with changes along these lines!

Eventually we should consider adding tests for the Git services.

Thank you,
Ludo’.

bug#26687: add git http support service

[Ludovic Courtès]

Hello!

I pushed those patches as commits
4313125816c0cfc9ce15ea1fec8a941976012ee6 and
514baaf8b50005cedf11e2c2cc3bce0e9d90328f.

I also added a test for the Git-over-HTTP service as commit
58ff6e1e0b1f1c4d1099aa317a0445d5ffcbd662.

Thanks!

Ludo’.