From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Performance of the man page database generation Date: Tue, 16 May 2017 10:21:21 +0200 Message-ID: <87tw4lb4am.fsf@gnu.org> References: <20170405205638.12336.78854@vcs0.savannah.gnu.org> <20170405205640.15AE6210A4@vcs0.savannah.gnu.org> <86y3v9bicq.fsf@gmail.com> <864lxxghmm.fsf@gmail.com> <87a87ibjq1.fsf@gmail.com> <871ssn4ebg.fsf_-_@gnu.org> <87fug8j704.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:57122) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dAXjV-0006TA-P1 for guix-devel@gnu.org; Tue, 16 May 2017 04:21:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dAXjQ-000846-SH for guix-devel@gnu.org; Tue, 16 May 2017 04:21:29 -0400 In-Reply-To: <87fug8j704.fsf@gmail.com> (Maxim Cournoyer's message of "Sat, 13 May 2017 23:18:51 -0700") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Maxim Cournoyer Cc: guix-devel , myglc2 Hi Maxim, Maxim Cournoyer skribis: > 18 man subdirectories contained newer manual pages. > 3943 manual pages were added. > find-files: /gnu/store/5mk4lg1rdzq2p3148idjd4z0bvif0nf4-gimp-2.8.18/share= /man/man1/gimp-console.1: No such file or directory > guix build: error: build failed: derivation `/gnu/store/vk10zmd5nm8kppd0f= 655fraradr019fq-manual-database.drv' may not be deterministic: output `/gnu= /store/lmf9rfzj565m31xzhhlam3lk91bjbqsm-manual-database' differs > > real 0m39.890s > user 0m0.356s > sys 0m0.040s Not good! > As you can see, mandb stumbled more than once with some of our packages' > manpages. Those failures are probably attributable to bad packages > manual pages. For example, the gcc manual pages are known to be broken > for example. [0] > > By looking at the output as it was generated, it seemed as if those > failures caused mandb to take more time. That would be surprising no? > Another reason could be that mandb is just inefficient. There's an old > bug about mandb's use of processes being less than optimal ("strace > indicates that mandb is forking on the order of four processes per > page") [1]. It seems that Colin Watson had a good idea of how to improve > things but didn't get around to implementing it yet. Yeah, I noticed that it was forking a lot by looking at the process graph in gkrellm. That=E2=80=99s not good. The database creation is probably not rocket science, so I wonder if we could implement it more efficiently in Guile using the appropriate gdbm or bdb bindings. Certainly we could avoid the forks. Alternately we optimize mandb itself directly and contribute it upstream, which would be benefit everyone but might be trickier. Thoughts? Ludo=E2=80=99.