Organizing a collection of papers

Organizing a collection of papers

[Victor Miller]

I've just started using org-mode, and so far find it quite
useful. I have a very large collection of technical papers in a
directory tree, and I'd like to go through them and index them
through org-mode. What I'd like is to have a way of going through
them and look at the unannotated ones, and annotate them one by
one. I imagine doing this by first making up a file of links like
[[xxx.pdf][not done yet]], and then being presented with the not
done ones, glancing at them and deciding how what annotations to
put in. In addition I'd like to add tags. What I'd really like is
to be able to make up new tags on the fly. Has anyone done
anything like this in org-mode?

Victor

Re: Organizing a collection of papers

[BernardH]

Victor Miller <victorsmiller <at> gmail.com> writes:

> 
> I've just started using org-mode, and so far find it quite
> useful. I have a very large collection of technical papers in a
> directory tree, and I'd like to go through them and index them
> through org-mode. What I'd like is to have a way of going through
> them and look at the unannotated ones, and annotate them one by
> one. I imagine doing this by first making up a file of links like
> [[xxx.pdf][not done yet]], and then being presented with the not
> done ones, glancing at them and deciding how what annotations to
> put in. In addition I'd like to add tags. What I'd really like is
> to be able to make up new tags on the fly. Has anyone done
> anything like this in org-mode?
> 
> Victor
> 
> 
Hi,
For organizing papers, I've recently found
[[http://www-public.it-sudparis.eu/~berger_o/weblog/2012/03/23/how-to-manage-and-export-bibliographic-notesrefs-in-org-mode/][an
interesting blog post]].
Actually doing it is still on my TODO list unfortunately.
However, the use-case seems common enough amongst orgmode users that
if something is recognized as best-practice, maybe an entry in
[[http://orgmode.org/worg/org-tutorials/][the worg tutorials section]] would be
warranted.

Best Regards,

B.

Re: Organizing a collection of papers

[François Allisson]

Hi Victor,

Le lundi 04 jun 2012 à 00:57:53 (+0000), Victor Miller a écrit :
> I've just started using org-mode, and so far find it quite
> useful. I have a very large collection of technical papers in a
> directory tree, and I'd like to go through them and index them
> through org-mode. What I'd like is to have a way of going through
> them and look at the unannotated ones, and annotate them one by
> one. I imagine doing this by first making up a file of links like
> [[xxx.pdf][not done yet]], 

The script dir2org.zsh (located in the contrib/scripts directory) may
be a good starting point for you.

It transforms (recursively) a directory and all its files into an Org
mode file reproducing the directory tree hierarchy with one headline
per file, and it creates automatically the links to the files.

Briefly, if your files are located in the directory ~/my-big-database
(and its sub-directories), just type:

- cd path-to-org-mode-distribution-directory/
- cd contrib/scripts
- zsh    (if, like me, zsh is not your usual shell, you must call it,
          eventually install it; at the end, type "exit" to return to
          your favourite shell)
- ./dir2org.zsh ~/my-big-database > orgfile.org

And it's done. Your "orgfile.org" contains linked headlines for
all your files. You can then start playing with your files with all
the Org mode facilities (tags, ordering headlines, adding properties,
annotations, todos, etc.). HTH.

> and then being presented with the not
> done ones, glancing at them and deciding how what annotations to
> put in. In addition I'd like to add tags. What I'd really like is
> to be able to make up new tags on the fly. Has anyone done
> anything like this in org-mode?

I don't know what will be the better workflow to achieve this. I think
we are all looking for the best workflow to handle our electronic (and
non electronic) libraries. 

I cannot say yet what my workflow is, because I'm currently working on
it. FWIW, I /was/ handling my bibliographic resources in a big BibTeX
file, adding custom BibTeX fields for my personal needs. I can only
give hint whither I am going:

- a big biblio.org file with a lot of headlines (one per item),
  with three custom todo keyword (in a sequence TOREADLATER TOREAD |
  READ)

  - TOREAD: for items on which I still have something to do (read,
    annotate, decide whether it is worth reading or not, or if I
    should do something with it, etc.). These appear in my custom
    agenda-view as a block name "Currents readings".

  - TOREADLATER: idem. But these items do *not* appear in my custom
    agenda-view.

  - READ: for items on which I'm done.

- each item has a name "Author (year) Title of document", it contains
  tags as keywords, BibTeX properties (using org-bibtex.el), custom
  properties (physical location if not electronic, date of insert,
  etc.). It contains link to file (if file). And it contains my
  reading notes, quotes, comments, TODO items (if I have to check
  something, or discuss something with someone, I plan it). All this
  using sub-headings (for respecting chapters, or the papers
  structure, or other subdivision for my own needs. As this file is in
  my org-agenda-files, I can take advantage of all Org mode
  facilities.

I can export the whole database as a BibTeX file using only one
command (M-x org-bibtex, thanks to the excellent org-bibtex.el), and
can use it in my Org mode files (formerly LaTeX files), using the
(new) latex exporter.

I can also export a headline with the (new) exporter to share my notes
on one item with someone (and easily decide which parts of the notes
not to share with a few :noexport: headlines)

When I have free time (...) or when (I recommend doing this) I
scheduled reading hours in my week, I just call my custom agenda
command, showing me only TOREAD items, I'm one space key away from my
biblio.org file, and I can just fill my notes while reading.

I decided to restrict myself to 5 TOREAD keywords. Once a TOREAD is
done, it becomes READ and disappear from my agenda view. Once my
TOREAD list becomes empty, I decide which TOREADLATER becomes TOREAD
(again with the rule of 5). It helps me not starting dozens of things and
never finish them...

As for the capture of new items, I'm still working on it. I started to
use the command "M-x org-bibtex-create" for new items, and then
arrange manually my other needs, but I'm now in the process of
creating custom org-capture templates to better achieve my needs.

I would happily share a less confused state of my workflow when I'll
be more happy with it.

Sorry for being such OT.

> 
> Victor
> 
> 

Good luck with your collection of papers,

Cheers,
François

Re: Organizing a collection of papers

[Frank]

Victor Miller <victorsmiller <at> gmail.com> writes:

> 
> I've just started using org-mode, and so far find it quite
> useful. I have a very large collection of technical papers in a
> directory tree, and I'd like to go through them and index them
> through org-mode. What I'd like is to have a way of going through
> them and look at the unannotated ones, and annotate them one by
> one. I imagine doing this by first making up a file of links like
> [[xxx.pdf][not done yet]], and then being presented with the not
> done ones, glancing at them and deciding how what annotations to
> put in. In addition I'd like to add tags. What I'd really like is
> to be able to make up new tags on the fly. Has anyone done
> anything like this in org-mode?

I've used org-mode about a year and a half, and I don't think it's the best
application for what you describe.  There are several packages available to do
this; the one I've used most often and successfully is Zotero (zotero.org). 
This is a browswer plug-in which implements a very nice database for your
papers.  At this point, I've indexed 1092 papers (PDFs, web pages and other
materials).  In some cases, I have multiple files attached to each of them.

Zotero helps me manage the PDFs I have already, and makes it non unusual for me
to add a few dozen to my collection, all nicely cross-referenced and ready for
citations using Word or Latex.  Once you realize how easy it is to get from a
google scholar result to another PDF in your database, and the database entry is
completely ready to support insertion as a citation, as a footnote or
end-note,... this gets very addictive.  I can't imagine any org-mode package
accelerating the process of finding, displaying, indexing and citing sources as
much as Zotero does for me.

Re: Organizing a collection of papers

[John Wiegley]

>>>>> Frank  <fam@muenn.net> writes:

> I've used org-mode about a year and a half, and I don't think it's the best
> application for what you describe.  There are several packages available to
> do this; the one I've used most often and successfully is Zotero
> (zotero.org).  This is a browswer plug-in which implements a very nice
> database for your papers.  At this point, I've indexed 1092 papers (PDFs,
> web pages and other materials).  In some cases, I have multiple files
> attached to each of them.

If you happen to have a Mac, the application DEVONthink Pro is simply
wonderful for organizing collections like this.  I have about 7k PDFs in a
financial database, and it's always easy to find what I'm looking for.

John

Re: Organizing a collection of papers

[Thomas S. Dye]

"John Wiegley" <johnw@newartisans.com> writes:

>>>>>> Frank  <fam@muenn.net> writes:
>
>> I've used org-mode about a year and a half, and I don't think it's the best
>> application for what you describe.  There are several packages available to
>> do this; the one I've used most often and successfully is Zotero
>> (zotero.org).  This is a browswer plug-in which implements a very nice
>> database for your papers.  At this point, I've indexed 1092 papers (PDFs,
>> web pages and other materials).  In some cases, I have multiple files
>> attached to each of them.
>
> If you happen to have a Mac, the application DEVONthink Pro is simply
> wonderful for organizing collections like this.  I have about 7k PDFs in a
> financial database, and it's always easy to find what I'm looking for.
>
> John
>
>
I work in Org mode and LaTeX.  Ebib is integrated with both of them.
I follow an Org mode link or a LaTeX citation to the BibTeX entry in
Ebib, then open the pdf from Ebib using a link stored in the BibTeX
file.  I think Ebib is a splendid tool in this situation.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Organizing a collection of papers

[Christian Wittern]

On 2012-06-30 01:07, Thomas S. Dye wrote:
> "John Wiegley" <johnw@newartisans.com> writes:
>
>>>>>>> Frank  <fam@muenn.net> writes:
>>> I've used org-mode about a year and a half, and I don't think it's the best
>>> application for what you describe.  There are several packages available to
>>> do this; the one I've used most often and successfully is Zotero
>>> (zotero.org).  This is a browswer plug-in which implements a very nice
>>> database for your papers.  At this point, I've indexed 1092 papers (PDFs,
>>> web pages and other materials).  In some cases, I have multiple files
>>> attached to each of them.
>> If you happen to have a Mac, the application DEVONthink Pro is simply
>> wonderful for organizing collections like this.  I have about 7k PDFs in a
>> financial database, and it's always easy to find what I'm looking for.
>>
>> John
>>
>>
> I work in Org mode and LaTeX.  Ebib is integrated with both of them.
> I follow an Org mode link or a LaTeX citation to the BibTeX entry in
> Ebib, then open the pdf from Ebib using a link stored in the BibTeX
> file.  I think Ebib is a splendid tool in this situation.
Again, if you happen to be on a Mac, you can combine Zotero and BibTeX using 
the excellent Zot2Bib Firefox extension.  This will use the Zotero 
translators, but can be configured to transfer the citations to a BibTeX 
file.  This can be managed with BibDesk, but is also available within Emacs 
and can be used for citations in papers written in Org-mode.  I have started 
using this combo about two years ago and am quite pleased with the results.

All the best,

Christian

-- 
Christian Wittern, Kyoto