From mboxrd@z Thu Jan 1 00:00:00 1970 From: Erik Hetzner Subject: Re: John's amazing indexing posts Date: Sun, 26 Jul 2015 22:16:57 -0700 Message-ID: <55b5bed5.4731460a.680f0.ffffb455@mx.google.com> References: Reply-To: Erik Hetzner Mime-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:51164) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZJams-0000et-8C for emacs-orgmode@gnu.org; Mon, 27 Jul 2015 01:17:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZJamm-0004PI-2s for emacs-orgmode@gnu.org; Mon, 27 Jul 2015 01:17:18 -0400 Received: from mail-pd0-x236.google.com ([2607:f8b0:400e:c02::236]:35943) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZJaml-0004Oy-S1 for emacs-orgmode@gnu.org; Mon, 27 Jul 2015 01:17:12 -0400 Received: by pdjr16 with SMTP id r16so46203474pdj.3 for ; Sun, 26 Jul 2015 22:17:10 -0700 (PDT) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Org Mode Cc: John Kitchin Hi all, I previously hooked up org with recoll with pretty good results. I=E2=80=99= ve written this up for worg, but I have my ssh key on a different machine, so I can=E2=80=99t push now. Here is the info for the record. ** Recoll In order to index using the [[http://www.lesbonscomptes.com/recoll/][recoll= ]] search engine, you will want to add the following to your =3D~/.recoll/mimeinfo=3D file: #+BEGIN_SRC .org =3D text/x-org .org_archive =3D text/x-org #+END_SRC You will also need a shell script to convert your org mode files to HTML in batch mode. The script takes as an argument the file to convert and prints the output to stdout. Here is an example: #+BEGIN_SRC sh #!/bin/sh emacs --batch --eval "(progn (find-file \"$1\") (org-html-export-as-html) (= set-buffer \"*Org HTML Export*\") (princ (buffer-string)))" #+END_SRC You will also need the following in your =3D~/.recoll/mimeconf=3D: #+BEGIN_SRC [index] text/x-org =3D exec /home/egh/.recoll/rclorg ; mimetype =3D text/html #+END_SRC Now, rebuild your recoll index. Org mode files should be converted to HTML and indexed. It will take some time, because emacs will be launched for each conversion. An alternative is to use [[http://pandoc.org]= [pandoc]] to do the conversion. It can be configured as follows in your =3D~/.recoll/mimeconf=3D file: #+BEGIN_SRC [index] text/x-org =3D exec pandoc -s -f org -t html5 ; mimetype =3D text/html #+END_SRC If you want, you might change the pandoc template or org mode output to generate =3Dmeta=3D tags that will be recognized by recoll. See http://www.lesbonscomptes.com/recoll/usermanual/RCL.PROGRAM.html#RCL.PROGRA= M.FILTERS.HTML for details. On Mon, 13 Jul 2015 07:31:31 -0700, John Kitchin wrote: >=20 > Thanks Matt, >=20 > That is also my impression of where this will go. Eventually this will > move towards a database search engine, e.g. like Oleg's project at > https://github.com/wvxvw/sphinx-mode. I am not sure precisely which > direction though. Swish-e is nice, but at the moment you cannot > incrementally update the database, and full indexing is required every > time. I am not sure that is fixable, and swish-e does not do > unicode. There are half a dozen or so candidates to go forward on, and > they all have some pros and cons to think about. >=20 > It has a lot of other applications in org too, e.g. a file-system wide > agenda, tag search, etc... >=20 >=20 > Matt Price writes: >=20 > > Not sure if everyone has seen John's latest post about indexing org fil= es > > with swish-e: > > > > http://kitchingroup.cheme.cmu.edu/blog/2015/07/06/Indexing-headlines-in= -org-files-with-swish-e-with-laser-sharp-results/ > > > > It's very impressive. It strikes me as a step towards an incredibly > > ambitious project that would bring file indexing inside of Emacs -- so = it > > would not longer be necessary to go out to a shell or a Desktop Search = tool > > in order to find files that contain particular search terms. I'm looki= ng > > forward to your next steps, John! > > > > Matt >=20 > -- > Professor John Kitchin > Doherty Hall A207F > Department of Chemical Engineering > Carnegie Mellon University > Pittsburgh, PA 15213 > 412-268-7803 > @johnkitchin > http://kitchingroup.cheme.cmu.edu >=20 >=20