From mboxrd@z Thu Jan 1 00:00:00 1970 From: Maxim Cournoyer Subject: Re: [PATCH] profiles: Generate database file for manpages Date: Fri, 31 Mar 2017 00:12:29 -0700 Message-ID: <87tw69538i.fsf@gmail.com> References: <87fuhvw4a8.fsf@gmail.com> <87a882wykx.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:60446) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ctqjf-0005V8-8U for guix-devel@gnu.org; Fri, 31 Mar 2017 03:12:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ctqjd-0007wc-DU for guix-devel@gnu.org; Fri, 31 Mar 2017 03:12:39 -0400 In-Reply-To: <87a882wykx.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Thu, 30 Mar 2017 17:53:18 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-devel --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludovic, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Hi Maxim, > > Maxim Cournoyer skribis: > >> I've managed to finalize a profile hooks which takes care of generating >> the manpages database file. This file is used by apropos or man -k to >> provide results. > > Awesome! I=E2=80=99ve been willing to have this for some time. > Thanks for providing feedback on this so quickly! >> I've tested and found it to be working for the regular user profile. To >> try it, apply the patch and use a guix command that causes a new profile >> generation. >> >> For example: guix package -r git -i git >> Then: "apropos git" should return plenty of results. > > For some reason I have to use =E2=80=9Cman -K=E2=80=9D (capital): > > ~$ apropos guile > guile: nothing appropriate. > ~$ man -k guile > guile: nothing appropriate. > ~$ man -K guile > --Man-- next: skribilo(1) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C)= ] > --Man-- next: guile(1) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C) ] > --Man-- next: guix-pull(1) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C= ) ] > --Man-- next: guix-package(1) [ view (return) | skip (Ctrl-D) | quit (Ctr= l-C) ] > --Man-- next: gv(3guile) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C) ] > ~$ env | grep MANPATH > MANPATH=3D/home/ludo/.guix-profile/share/man:/run/current-system/profile/= share/man:/home/ludo/.guix-profile/share/man:/run/current-system/profile/sh= are/man:/home/ludo/.opam/system/man > > > Any idea why? > Unfortunately some of the simplifications you made appear to break it :). I've tried your reworked patch and got the same result as you (no result found except when using man -K). When I use the original patch it works as I originally described. >> It requires extra processing time to do the work since the complete >> database (on my system, there are thousands or manpages being indexed) >> is recreated everytime a new profile is generated. > > On my laptop (with an SSD), I found it to be reasonable: > > $ time guix build /gnu/store/rkri628apz2a2i2jvav11ylv2736fvv3-manual-data= base.drv --check > @ build-started /gnu/store/rkri628apz2a2i2jvav11ylv2736fvv3-manual-databa= se.drv - x86_64-linux /var/log/guix/drvs/rk//ri628apz2a2i2jvav11ylv2736fvv3= -manual-database.drv.bz2 > /gnu/store/g7qxdk1cn7fwkamkaisi4428k6ijg0w2-manual-database > > real 0m2.647s > user 0m0.116s > sys 0m0.004s > $ guix package -I |wc -l > 229 > > >> For now it doesn't work for guix environments (yet), but this should be >> easy to fix (it seems the $GUIX_ENVIRONMENT/share/man should be merged >> with $HOME/.guix-profile/share/man, but I need to look at it more >> carefully). > > It seems to work but only with capital -K: > > $ ./pre-inst-env guix environment --ad-hoc guile man-db --pure -- man -K = guile > man: can't execute cat: No such file or directory > man: command exited with status 255: sed -e '/^[[:space:]]*$/{ N; /^[[:sp= ace:]]*\n[[:space:]]*$/D; }' | (cd && LESS=3D-ix8RmPm Manual page gu= ile(1) ?ltline %lt?L/%L.:byte %bB?s/%s..?e (END):?pB %pB\%.. (press h for h= elp or q to quit)$PM Manual page guile(1) ?ltline %lt?L/%L.:byte %bB?s/%s..= ?e (END):?pB %pB\%.. (press h for help or q to quit)$ MAN_PN=3Dguile(1) cat) > > (And we should hard-code the file name of =E2=80=98cat=E2=80=99 in =E2=80= =98man=E2=80=99=E2=80=A6) > >> >> The mandb database file (index.db) is used by the "apropos" (whatis) or >> "man -k" commands. This change introduces a profile hook to generate >> such database file. >> >> Co-authored by Ludovic Court=C3=A8s >> >> * guix/profiles.scm (manual-database): New procedure. >> (%default-profile-hooks): Add it. > > I made some simplifications to the patch. WDYT? > Looks good, except for the 2 problematic changes in the reworked patch identified below. > Thanks, > Ludo=E2=80=99. > > diff --git a/guix/profiles.scm b/guix/profiles.scm > index 795c9447f..f565f358b 100644 > --- a/guix/profiles.scm > +++ b/guix/profiles.scm > @@ -7,6 +7,7 @@ > ;;; Copyright =C2=A9 2016 Ricardo Wurmus > ;;; Copyright =C2=A9 2016 Chris Marusich > ;;; Copyright =C2=A9 2017 Huang Ying > +;;; Copyright =C2=A9 2017 Maxim Cournoyer > ;;; > ;;; This file is part of GNU Guix. > ;;; > @@ -946,10 +947,73 @@ files for the fonts of the @var{manifest} entries." > #:local-build? #t > #:substitutable? #f)) >=20=20 > +(define (manual-database manifest) > + (define man-db ;lazy reference > + (module-ref (resolve-interface '(gnu packages man)) 'man-db)) > + > + (define build > + #~(begin > + (use-modules (guix build utils) > + (srfi srfi-1)) > + > + (define entries > + (filter-map (lambda (directory) > + (let ((man (string-append directory "/share/man"= ))) > + (and (directory-exists? man) > + man))) > + '#$(manifest-inputs manifest))) > + > + (define manpages-collection-dir > + (string-append (getenv "PWD") "/manpages-collection")) > + > + (define man-directory > + (string-append #$output "/share/man")) > + > + (define (populate-manpages-collection-dir entries) > + (let ((manpages (append-map find-files entries))) > + (for-each (lambda (manpage) > + (let* ((dest-file (string-append > + manpages-collection-dir "/" > + (strip-store-file-name manpag= e)))) Unfortunately we cannot simply use strip-store-file-name, since what mandb expects to find in our manpages-collection-dir is a collection of subdirectories such as man1, man3, man7, etc. such as would be found under "/usr/share/man" on a traditional distro. This change instead makes it look like: bazaar-2.7.0/ gdbm-1.12/ libxau-1.0.8/ ... I guess this is what broke it. > + (mkdir-p (dirname dest-file)) > + (catch 'system-error > + (lambda () > + (symlink manpage dest-file)) > + (lambda args > + ;; Different packages may contain the same > + ;; manpage. Simply ignore the symlink err= or. > + #t)))) > + manpages))) > + > + (mkdir-p manpages-collection-dir) > + (populate-manpages-collection-dir entries) > + > + ;; Create a mandb config file which contains a custom made > + ;; manpath. The associated catpath is the location where the dat= abase > + ;; gets generated. > + (copy-file #+(file-append man-db "/etc/man_db.conf") > + "man_db.conf") > + > + (substitute* "man_db.conf" > + (("MANDB_MAP\\>.*$") > + (string-append "MANDB_MAP " manpages-collection-dir " " > + man-directory "\n"))) This change here will replace every MANDB_MAP entries with duplicated information: MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection /gnu/st= ore/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection /gnu/st= ore/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection /gnu/st= ore/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection /gnu/st= ore/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection /gnu/st= ore/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection /gnu/store/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man I don't think this is desirable. At first I was removing all the MANDB_MAP entries and then adding only our, but there's nothing to gain from doing this since the original man_db.conf file (bundled with mandb) is still somehow read and contains those entries anyway (which are harmless). > + > + (mkdir-p man-directory) > + (setenv "MANPATH" (string-join entries ":")) > + (zero? (system* #+(file-append man-db "/bin/mandb") > + "--quiet" "--create" > + "-C" "man_db.conf")))) > + > + (gexp->derivation "manual-database" build > + #:modules '((guix build utils)) > + #:local-build? #t)) > + > (define %default-profile-hooks > ;; This is the list of derivation-returning procedures that are called= by > ;; default when making a non-empty profile. > (list info-dir-file > + manual-database > fonts-dir-file > ghc-package-cache-file > ca-certificate-bundle I have reverted these two problematic changes and attached a new patch. Ple= ase try it out and let me know how it works :) Thanks, Maxim --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0001-profiles-Generate-database-file-for-manpages.patch Content-Transfer-Encoding: quoted-printable From=20c81fc96da953c867749b56b96324f500ba428f03 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Tue, 28 Mar 2017 09:25:21 -0700 Subject: [PATCH] profiles: Generate database file for manpages MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit The mandb database file (index.db) is used by the "apropos" (whatis) or "man -k" commands. This change introduces a profile hook to generate such database file. Co-authored by Ludovic Court=C3=A8s * guix/profiles.scm (manual-database): New procedure. (%default-profile-hooks): Add it. =2D-- guix/profiles.scm | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 67 insertions(+) diff --git a/guix/profiles.scm b/guix/profiles.scm index 795c9447fe..0c719987b8 100644 =2D-- a/guix/profiles.scm +++ b/guix/profiles.scm @@ -7,6 +7,7 @@ ;;; Copyright =C2=A9 2016 Ricardo Wurmus ;;; Copyright =C2=A9 2016 Chris Marusich ;;; Copyright =C2=A9 2017 Huang Ying +;;; Copyright =C2=A9 2017 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -946,10 +947,76 @@ files for the fonts of the @var{manifest} entries." #:local-build? #t #:substitutable? #f)) =20 +(define (manual-database manifest) + (define man-db ;lazy reference + (module-ref (resolve-interface '(gnu packages man)) 'man-db)) + + (define build + #~(begin + (use-modules (guix build utils) + (srfi srfi-1)) + + (define entries + (filter-map (lambda (directory) + (let ((man (string-append directory "/share/man"))) + (and (directory-exists? man) + man))) + '#$(manifest-inputs manifest))) + + (define manpages-collection-dir + (string-append (getenv "PWD") "/manpages-collection")) + + (define man-directory + (string-append #$output "/share/man")) + + (define (get-manpage-tail-path manpage-path) + (let ((index (string-contains manpage-path "/share/man/"))) + (substring manpage-path (+ index (string-length "/share/man/")= )))) + + (define (populate-manpages-collection-dir entries) + (let ((manpages (append-map find-files entries))) + (for-each (lambda (manpage) + (let* ((dest-file (string-append + manpages-collection-dir "/" + (get-manpage-tail-path manpage)= ))) + (mkdir-p (dirname dest-file)) + (catch 'system-error + (lambda () + (symlink manpage dest-file)) + (lambda args + ;; Different packages may contain the same + ;; manpage. Simply ignore the symlink error. + #t)))) + manpages))) + + (mkdir-p manpages-collection-dir) + (populate-manpages-collection-dir entries) + + ;; Create a mandb config file which contains a custom made + ;; manpath. The associated catpath is the location where the datab= ase + ;; gets generated. + (copy-file #+(file-append man-db "/etc/man_db.conf") + "man_db.conf") + (substitute* "man_db.conf" + (("MANDB_MAP /usr/man /var/cache/man/fsstnd") + (string-append "MANDB_MAP " manpages-collection-dir " " + man-directory))) + + (mkdir-p man-directory) + (setenv "MANPATH" (string-join entries ":")) + (zero? (system* #+(file-append man-db "/bin/mandb") + "--quiet" "--create" + "-C" "man_db.conf")))) + + (gexp->derivation "manual-database" build + #:modules '((guix build utils)) + #:local-build? #t)) + (define %default-profile-hooks ;; This is the list of derivation-returning procedures that are called by ;; default when making a non-empty profile. (list info-dir-file + manual-database fonts-dir-file ghc-package-cache-file ca-certificate-bundle =2D-=20 2.12.0 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEJ9WGpPiQCFQyn/CfEmDkZILmNWIFAljeAV4ACgkQEmDkZILm NWIsQxAAkK5U5OpwJLXM8f1bGJ6LB7w3ZjuReu4atyVnqtvD8LergKCymV5E5eFa PJpTzuzihkqE4c3ZsQaj5qgmIt5AIzVzFDRIBbIpIDTxgk2+nCKLA9dDDydLIoG5 s3KUFVUyW431nBnry7BO5G3PCQ7CQhbZ5Gw7FBoBFzRJCM83gBPk/idfJx0f3BGB X4S6DD1A2xhGKf9Ph0lr7n8sb8NmvwSF0DOl8WdXvcp5VVey91zpqyKQqquvQWHZ q+wpPM1b0EsdMyasl1O0MyMW9egO32OGeM1XjRhdpI3Bij+o0QFJXPtDkNpAS5iT tQKaBpNlL1bLnM0sfy6NV77hQjjispMG7QBD0EueyvlRJkDQBjo73ol5UgJlXuZx Gj1BSShuowi/Q7+Qb/JW8TLJlm4AGuWXb6wCey1lmSsnOCXpgN6D76zy+f7b36bn TWQJF8LyStpCabwNsMi+oaMMpiT7snDU4VLdlsGaDkOENRkR26Hw2XJALre/3IAQ fhZsYtRimwK5NG0WlfC5thPdDUGhv2yuw2hmfube3m6JJ7Y1khmjBQbxmrmC4HZO ha6+uvShJlkxonMDfQvd09nwVoGeXya5OclOcEggnmuZOMSQf8EYwKQ5Gc3o0hZE CBk2ZevOJIvzZxrmN4MpwbzJl54CifwX1SWb61nm6W5lEoZ0yu8= =Ow36 -----END PGP SIGNATURE----- --==-=-=--