unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: "Ludovic Courtès" <ludo@gnu.org>
To: Brice Waegeneire <brice@waegenei.re>
Cc: Guix Devel <guix-devel@gnu.org>,
	Maxim Cournoyer <maxim.cournoyer@gmail.com>
Subject: Re: Getting rid of the mandb profile hook?
Date: Fri, 02 Apr 2021 18:56:19 +0200	[thread overview]
Message-ID: <87lfa0zkuk.fsf@gnu.org> (raw)
In-Reply-To: <0170f58ece0b1bfd193f1566c37eddb8@waegenei.re> (Brice Waegeneire's message of "Wed, 03 Mar 2021 21:50:31 +0100")

[-- Attachment #1: Type: text/plain, Size: 1530 bytes --]

Hello!

Brice Waegeneire <brice@waegenei.re> skribis:

> On 2021-03-03 15:13, Ludovic Courtès wrote:
>>>> I’m thinking we could get rid of the mandb hook.  However, the

[...]

> What about using mandoc¹, the manpage compiler from OpenBSD, instead of
> man-db? As from it's manual it support specifying the database location:
>
> “makewhatis -d dir [file ...]”²

I recently packaged it, but I’m not impressed; I’m not even sure how
that’s supposed to work:

  $ guix environment --ad-hoc mandoc -- makewhatis -d /tmp/foo $(find -L ~/.guix-profile/share/man -name \*.[0-9].gz)

exits successfully but does nothing.

At this point my preference would be to build a custom tool (I’m not
aware of any existing tool like that, but if you do, please share) that
would lazily build a database, ideally full-text, and search through it;
attached a super rough example that uses Guile-Xapian and inserts man-db
synopses into a Xapian database.

The tool would index man pages and Info pages.  It would be smart enough
to index only info/man files that have actually changed (it could look
at the inode number to determine in a way that avoids unnecessary cache
invalidation).  I’m not sure how to implement this part though.  It
sounds like a good hack for our Xapian experts—I’m looking at you Arun,
Ricardo, zimoun.  :-)

Thoughts?

I’d really like to have a rough solution so we can remove the
‘manual-database’ hook in time for the release.

Thoughts?

Ludo’.


[-- Attachment #2: Xapian as a man-db replacement --]
[-- Type: text/plain, Size: 3425 bytes --]

(use-modules (xapian wrap)
             (xapian xapian)
             (ice-9 match)
             (guix man-db)
             (srfi srfi-1)
             (srfi srfi-26))

;; eval: (put 'call-with-writable-database 'scheme-indent-function 1)

(define (index-mandb-entry db entry)
  (define (mandb-entry-id-term entry)
    (string-append "Q" "man:" (mandb-entry-name entry) "."
                   (number->string
                    (mandb-entry-section entry))))

  (when (mandb-entry-name entry)
    (let* ((idterm (mandb-entry-id-term entry))
           (doc (make-document
                 #:data (object->string
                         `((name . ,(mandb-entry-name entry))
                           (section . ,(number->string
                                        (mandb-entry-section entry)))
                           (file . ,(canonicalize-path
                                     (mandb-entry-file-name entry)))))
                 #:terms `((,idterm . 0))))
           (term-generator
            (make-term-generator #:stem (make-stem "en")
                                 #:document doc)))
      (index-text! term-generator (mandb-entry-name entry) #:prefix "A")
      (index-text! term-generator
                   (number->string (mandb-entry-section entry))
                   #:prefix "B")
      (index-text! term-generator (mandb-entry-synopsis entry))
      (replace-document! db idterm doc))))

(define (index-mandb-entries)
  (call-with-writable-database "/tmp/db"
    (lambda (db)
      (for-each (cut index-mandb-entry db <>)
                ;; (mandb-entries "/run/current-system/profile/share/man")
                (append-map mandb-entries
                            (string-split (getenv "MANPATH") #\:))
                ))))

(define* (parse-query* querystring #:key stemmer stemming-strategy
                       (prefixes '())
                       (boolean-prefixes '()))
  (let ((queryparser (new-QueryParser)))
    (QueryParser-set-stemmer queryparser stemmer)
    (when stemming-strategy
      (QueryParser-set-stemming-strategy queryparser stemming-strategy))
    (for-each (match-lambda
                ((field . prefix)
                 (QueryParser-add-prefix queryparser field prefix)))
              prefixes)
    (for-each (match-lambda
                ((field . prefix)
                 (QueryParser-add-boolean-prefix queryparser field prefix)))
              boolean-prefixes)
    (let ((query (QueryParser-parse-query queryparser querystring)))
      (delete-QueryParser queryparser)
      query)))

(define* (search querystring #:key (pagesize 100))
  (call-with-database "/tmp/db"
    (lambda (db)
      (let* ((query (parse-query querystring
                                  #:stemmer (make-stem "en")
                                  #:prefixes
                                  '(("name"    . "A")
                                    ("section" . "B"))))
             (enq (enquire db query)))
        ;; (Enquire-set-sort-by-value enq 0 #f)
        (reverse (mset-fold (lambda (item acc)
                              (cons (call-with-input-string
                                        (document-data (mset-item-document item))
                                      read)
                                    acc))
                            '()
                            (enquire-mset enq
                                          #:maximum-items pagesize)))))))

      parent reply	other threads:[~2021-04-02 16:56 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-05 18:31 Getting rid of the mandb profile hook? Ludovic Courtès
2020-12-05 18:51 ` Pierre Neidhardt
2020-12-05 20:18 ` Ryan Prior
2020-12-06  0:37 ` Ricardo Wurmus
2020-12-08 10:47   ` Ludovic Courtès
2021-02-27 13:05 ` Maxim Cournoyer
2021-03-03 14:13   ` Ludovic Courtès
2021-03-03 20:50     ` Brice Waegeneire
2021-03-10 10:20       ` Ludovic Courtès
2021-04-02 16:56       ` Ludovic Courtès [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87lfa0zkuk.fsf@gnu.org \
    --to=ludo@gnu.org \
    --cc=brice@waegenei.re \
    --cc=guix-devel@gnu.org \
    --cc=maxim.cournoyer@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).