From: "Ludovic Courtès" <ludo@gnu.org>
To: Brice Waegeneire <brice@waegenei.re>
Cc: Guix Devel <guix-devel@gnu.org>,
Maxim Cournoyer <maxim.cournoyer@gmail.com>
Subject: Re: Getting rid of the mandb profile hook?
Date: Fri, 02 Apr 2021 18:56:19 +0200 [thread overview]
Message-ID: <87lfa0zkuk.fsf@gnu.org> (raw)
In-Reply-To: <0170f58ece0b1bfd193f1566c37eddb8@waegenei.re> (Brice Waegeneire's message of "Wed, 03 Mar 2021 21:50:31 +0100")
[-- Attachment #1: Type: text/plain, Size: 1530 bytes --]
Hello!
Brice Waegeneire <brice@waegenei.re> skribis:
> On 2021-03-03 15:13, Ludovic Courtès wrote:
>>>> I’m thinking we could get rid of the mandb hook. However, the
[...]
> What about using mandoc¹, the manpage compiler from OpenBSD, instead of
> man-db? As from it's manual it support specifying the database location:
>
> “makewhatis -d dir [file ...]”²
I recently packaged it, but I’m not impressed; I’m not even sure how
that’s supposed to work:
$ guix environment --ad-hoc mandoc -- makewhatis -d /tmp/foo $(find -L ~/.guix-profile/share/man -name \*.[0-9].gz)
exits successfully but does nothing.
At this point my preference would be to build a custom tool (I’m not
aware of any existing tool like that, but if you do, please share) that
would lazily build a database, ideally full-text, and search through it;
attached a super rough example that uses Guile-Xapian and inserts man-db
synopses into a Xapian database.
The tool would index man pages and Info pages. It would be smart enough
to index only info/man files that have actually changed (it could look
at the inode number to determine in a way that avoids unnecessary cache
invalidation). I’m not sure how to implement this part though. It
sounds like a good hack for our Xapian experts—I’m looking at you Arun,
Ricardo, zimoun. :-)
Thoughts?
I’d really like to have a rough solution so we can remove the
‘manual-database’ hook in time for the release.
Thoughts?
Ludo’.
[-- Attachment #2: Xapian as a man-db replacement --]
[-- Type: text/plain, Size: 3425 bytes --]
(use-modules (xapian wrap)
(xapian xapian)
(ice-9 match)
(guix man-db)
(srfi srfi-1)
(srfi srfi-26))
;; eval: (put 'call-with-writable-database 'scheme-indent-function 1)
(define (index-mandb-entry db entry)
(define (mandb-entry-id-term entry)
(string-append "Q" "man:" (mandb-entry-name entry) "."
(number->string
(mandb-entry-section entry))))
(when (mandb-entry-name entry)
(let* ((idterm (mandb-entry-id-term entry))
(doc (make-document
#:data (object->string
`((name . ,(mandb-entry-name entry))
(section . ,(number->string
(mandb-entry-section entry)))
(file . ,(canonicalize-path
(mandb-entry-file-name entry)))))
#:terms `((,idterm . 0))))
(term-generator
(make-term-generator #:stem (make-stem "en")
#:document doc)))
(index-text! term-generator (mandb-entry-name entry) #:prefix "A")
(index-text! term-generator
(number->string (mandb-entry-section entry))
#:prefix "B")
(index-text! term-generator (mandb-entry-synopsis entry))
(replace-document! db idterm doc))))
(define (index-mandb-entries)
(call-with-writable-database "/tmp/db"
(lambda (db)
(for-each (cut index-mandb-entry db <>)
;; (mandb-entries "/run/current-system/profile/share/man")
(append-map mandb-entries
(string-split (getenv "MANPATH") #\:))
))))
(define* (parse-query* querystring #:key stemmer stemming-strategy
(prefixes '())
(boolean-prefixes '()))
(let ((queryparser (new-QueryParser)))
(QueryParser-set-stemmer queryparser stemmer)
(when stemming-strategy
(QueryParser-set-stemming-strategy queryparser stemming-strategy))
(for-each (match-lambda
((field . prefix)
(QueryParser-add-prefix queryparser field prefix)))
prefixes)
(for-each (match-lambda
((field . prefix)
(QueryParser-add-boolean-prefix queryparser field prefix)))
boolean-prefixes)
(let ((query (QueryParser-parse-query queryparser querystring)))
(delete-QueryParser queryparser)
query)))
(define* (search querystring #:key (pagesize 100))
(call-with-database "/tmp/db"
(lambda (db)
(let* ((query (parse-query querystring
#:stemmer (make-stem "en")
#:prefixes
'(("name" . "A")
("section" . "B"))))
(enq (enquire db query)))
;; (Enquire-set-sort-by-value enq 0 #f)
(reverse (mset-fold (lambda (item acc)
(cons (call-with-input-string
(document-data (mset-item-document item))
read)
acc))
'()
(enquire-mset enq
#:maximum-items pagesize)))))))
prev parent reply other threads:[~2021-04-02 16:56 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-05 18:31 Getting rid of the mandb profile hook? Ludovic Courtès
2020-12-05 18:51 ` Pierre Neidhardt
2020-12-05 20:18 ` Ryan Prior
2020-12-06 0:37 ` Ricardo Wurmus
2020-12-08 10:47 ` Ludovic Courtès
2021-02-27 13:05 ` Maxim Cournoyer
2021-03-03 14:13 ` Ludovic Courtès
2021-03-03 20:50 ` Brice Waegeneire
2021-03-10 10:20 ` Ludovic Courtès
2021-04-02 16:56 ` Ludovic Courtès [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87lfa0zkuk.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=brice@waegenei.re \
--cc=guix-devel@gnu.org \
--cc=maxim.cournoyer@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).