Publish atom feeds based on Org files

* Publish atom feeds based on Org files
@ 2010-03-19 22:00 David Maus
  2010-03-20  4:38 ` Austin Frank
                   ` (2 more replies)
  0 siblings, 3 replies; 10+ messages in thread
From: David Maus @ 2010-03-19 22:00 UTC (permalink / raw)
  To: Org Mode

[-- Attachment #1.1.1: Type: text/plain, Size: 10116 bytes --]

Hello all,

Carving another stone that might fit into the cathedral known as
Emacs/Orgmode I am glad to present attached file org-atom.el that
provides an exporting and a publishing function to create atom feeds
based on Org files.

To use org-atom.el you need the library atom-syndication.el, an
elisp implementation of the atom syndication format that can be found
in it's github repository.

http://github.com/dmj/atom-syndication/

Below you'll find a reduced ascii export of org-atom's project page
(currently at http://ictsoc.de/code/org-atom.html) that gives a brief
explanation of how to setup Org files for feed export.

Although both, atom-syndication.el and org-atom.el, are not quite
finshed yet, I decided to make the community aware of org-atom in the
hope some might try it out and give some feedback and ideas how to
extend org-atom's functionality.

My personal goal for org-atom is to make use of all functionality that
the atom syndication format provides: E.g. what popped into my mind is
that having an atom feed for Worg, especially the hacking-section
would be incredible.

Anyway: If you try org-atom and create feeds I'd like you to ask to
validate the feeds even if your feed reader accepts it.  Validating
the feed can be achived in two ways:

 - use the w3c feed validator at http://validator.w3.org/feed/

   It will not just validate the feed but also provide an short
   summary why a feed is not valid (if so) and suggestions on how to
   improve the feed.

 - use nxml-mode with the Relax NG Compact Syntax Grammar

   The syntax grammar is shipped with atom-syndication.el in the file
   atom-syndication.rnc.  Just open the feed file, maybe turn on nxml
   mode (M-x nxml-mode RET) and set the scheme file to
   atom-syndication.rnc (M-x rng-set-schema-file-and-validate RET
   /path/to/atom-syndication.rnc RET)

And here's the brief summary of the functions:

                Publish atom feeds based on Org files
                =====================================

Author: David Maus <dmaus@ictsoc.de>
Date: 2010-03-19 22:23:01 CET

Table of Contents
=================
1 About
2 Exporting and publishing
    2.1 New infile export options
    2.2 Mapping of other Org entities to atom metadata elements
    2.3 Exporting a single Org file
    2.4 Publishing a feed for multiple files in a publishing project

1 About
~~~~~~~~

  This addon for Emacs Org mode, an Emacs Mode for Notes, Project
  Planning, and Authoring, provides export and publishing functions to
  create an atom feed based on Org files.

  This file provides some information on the concept, the state of
  implementation and a maybe some ideas how to extend the exporter.

2 Exporting and publishing
~~~~~~~~~~~~~~~~~~~~~~~~~~~

2.1 New infile export options
==============================

   =#+FEED_URL=: URL of the published feed.

                     This option MUST be present when exporting an Org
                     file using =org-export-as-atom=.

   =#+FEED_CONTENT_URL=: URL of the published html file.

        When exporting an Org file using =org-export-as-atom= and the
        customization variable =org-atom-publish-content= is not set,
        this option MUST be present.

   =#+FEED_ID=: Unique identifier of the feed.

                    If not set, the feed url is used as id.

   =#+FEED_MAP_ENTRIES=: A TAGS/PROP/TODO query that matches entry
        headlines.

        If not set, all headlines are exported as atom entries.

2.2 Mapping of other Org entities to atom metadata elements
============================================================

   In addition org-atom maps other Org mode entities to atom metadata
   elements.  Following table provides a comprehensive list of all
   currently supported metadata elements and their mapped Org
   entities.

2.2.1 atom:feed metadata
-------------------------

      Element                  Description                       Org entity
     ------------------------+---------------------------------+-------------------------------------
      atom:author name         name of author                    #+AUTHOR
      atom:generator name      name of generator                 const: `org-atom-generator-name'
      atom:generator version   version of generator              const: `org-atom-generator-version'
      atom:id                  unique id of feed                 #+FEED_ID
      atom:link rel="self"     link to the published feed        #+FEED_URL
      atom:updated             time when feed was updated        (current-time) on export
      atom:subtitle            subtitle or description of feed   #+DESCRIPTION
      atom:title               title of feed                     #+TITLE

2.2.2 atom:entry metadata
--------------------------

      Element                     Description                     Org entity
     ---------------------------+-------------------------------+---------------------------------------------
      atom:author name            name of author                  #+AUTHOR
      atom:content                content of feed entry           subtree of feed headline
      atom:id                     unique id of entry              ID property
      atom:link rel="alternate"   link to entry content           #+FEED_CONTENT_URL
      atom:published              time when entry was published   property `org-atom-published-property-name'
      atom:title                  entry title                     headline title w/o TODO keywords and tags
      atom:updated                time when entry was updated     property `org-atom-updated-property-name'

2.3 Exporting a single Org file
================================

   A minimal setup of an Org file that will be published as atom feed
   could look like this:

   Calling =org-export-as-atom= on this file produces an atom feed:

,----
|   <feed xmlns="http://www.w3.org/2005/Atom"><title>Example file for feed export</title>
|   <generator version="0.1beta">Org/Atom</generator>
|   <id>urn:uuid:a20d6ebc-7bab-4307-8b1c-7e29ac49bf66</id>
|   <updated>2010-03-19T22:13:30+01:00</updated>
|   <link href="http://example.tld/~john/" rel="self" />
|   <author><name>John Doe</name></author>
|   <entry><published>2010-03-19T15:55:00+01:00</published>
|   <content type="html">&lt;div id="outline-container-1" class="outline-3">
|   &lt;h3 id="sec-1">&lt;a name="ID-ca13acc9-c14c-4707-b0e9-7f35687a4473" id="ID-ca13acc9-c14c-4707-b0e9-7f35687a4473">&lt;/a>Breaking news: Org now publishes atom feeds! &lt;/h3>
|   &lt;div class="outline-text-3" id="text-1">
|
|
|   &lt;p>
|   Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean
|   commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus
|   et magnis dis parturient montes, nascetur ridiculus mus. Donec quam
|   felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla
|   consequat massa quis enim. Donec pede justo, fringilla vel, aliquet
|   nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a,
|   venenatis vitae, justo. Nullam dictum felis eu pede mollis
|   pretium.
|   &lt;/p>&lt;/div>
|   &lt;/div>
|   </content>
|   <link href="http://example.tld/~john/news.html#ca13acc9-c14c-4707-b0e9-7f35687a4473" rel="alternate" />
|   <title type="html">
|   &lt;p>Breaking news: Org now publishes atom feeds!
|   &lt;/p></title>
|   <updated>2010-03-19T15:55:00+01:00</updated>
|   <id>urn:uuid:ca13acc9-c14c-4707-b0e9-7f35687a4473</id></entry></feed>
`----

2.4 Publishing a feed for multiple files in a publishing project
=================================================================

   To publish a feed for multiple Org files in a publishing project
   org-atom provides the function =org-atom-publish-feed-index= that
   can be plugged in a project's publishing alist as the
   =:index-function=:

,----
|   (setq org-publish-project-alist
|         '(
|           ...
|           ("news-feed"
|            :base-directory "~/website/"
|            :base-extension "org"
|            :publishing-directory "~/website/"
|            ;; new property :publishing-url
|            :publishing-url "http://example.tld/~john/"
|            :auto-index t
|            :index-filename "news.atom"
|            :index-title "News feed for john@example.tld"
|            :index-function org-atom-publish-feed-index
|            ;; feed specific settings
|            :feed-id "6ea57592-69f2-4ef8-b44d-b7a511bd2fe8")
|           ...
|           ))
`----

   Please note the new publishing property =:publishing-url=: Instead
   of setting the feed url (=:feed-url=) in this alist and the content
   urls in each file =:publishing-url= is used to derive the url of
   the feed (assuming the feed is published under the same url) and
   the content url of the files.

   Running =org-publish= lets =org-atom-publish-feed-index= interate
   over all files in the project alist and create a combined feed for
   all entries of all processed files.

 -- David

--
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... dmjena@jabber.org
Email..... dmaus@ictsoc.de

[-- Attachment #1.1.2: org-atom.el --]
[-- Type: application/octet-stream, Size: 10516 bytes --]

;;; org-atom.el --- Publish atom feeds based on Org files.
;;
;; Author: David Maus <dmaus@ictsoc.de>
;; Keywords: hypermedia, outline
;; Version: 0.1beta
;;
;; Copyright (C) 2010 by David Maus
;;
;; This file is NOT part of Gnu Emacs.
;;
;; This program is free software: you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.
;;
;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.
;;
;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
;;
;;; Commentary:
;;
;; This addon for Emacs Org mode, an Emacs Mode for Notes, Project
;; Planning, and Authoring, provides export and publishing functions to
;; create an atom feed based on Org files.
;;
;;; Code:
(require 'atom-syndication)

(defconst org-atom-uuid-regexp
  "^[[:xdigit:]]\\{8\\}\\(-[[:xdigit:]]\\{4\\}\\)\\{3\\}-[[:xdigit:]]\\{12\\}$"
  "Regular expression matching a uuid.")

(defconst org-atom-infile-options
  '(("FEED_MAP_ENTRIES" :feed-map-entries)
    ("FEED_ID" :feed-id)
    ("FEED_URL" :feed-url)
    ("FEED_CONTENT_URL" :feed-content-url)))

(defconst org-atom-generator-name "Org/Atom"
  "Name of the atom generator.")

(defconst org-atom-generator-version "0.1beta"
  "Version string of the atom generator.")

(defcustom org-atom-feed-extension "atom"
  "Extension of feed output file."
  :type 'string
  :group 'org-export-atom)

(defcustom org-atom-published-property-name "Created"
  "Name of property for publication date."
  :type 'string
  :group 'org-export-atom)

(defcustom org-atom-updated-property-name "Updated"
  "Name of property for date when entry was updated."
  :type 'string
  :group 'org-export-atom)

(defcustom org-atom-publish-content t
  "Publish feed content.  If unset only publish link to
  content."
  :type 'boolean
  :group 'org-export-atom)

(defcustom org-atom-prefer-urn-uuid t
  "Create iri with urn:uuid prefix when id looks like a uuid."
  :type 'boolean
  :group 'org-export-atom)

(defun org-export-as-atom (&optional ext-plist to-buffer body-only pub-dir)
  "Export outline as atom feed.

EXT-PLIST is a property list with external parameters overriding
org-mode's default settings, but still inferior to file-local
settings.
When TO-BUFFER is non-nil, create a buffer with that name and
export to that buffer.  If TO-BUFFER is the symbol `string',
don't leave any buffer behind but just return the resulting atom
feed as a string.
When BODY-ONLY is set, return only the atom:entry elements.
When PUB-DIR is set, use this as the publishing directory."
  (interactive)
  (let* ((opt-plist (org-combine-plists (org-default-export-plist)
					ext-plist
					(org-infile-export-plist)))
	 (feed-url (org-trim (or (plist-get opt-plist :feed-url) "")))
	 (feed-content-url (org-trim (or (plist-get opt-plist :feed-content-url)
					 "")))
	 (feed-id (org-trim (or (plist-get opt-plist :feed-id) feed-url)))
	 (feed-map-entries (org-trim (or (plist-get opt-plist :feed-map-entries)
					 "")))
	 (author (plist-get opt-plist :author))
	 (email (plist-get opt-plist :email))
	 (description (or (plist-get opt-plist :feed-description)
			  (plist-get opt-plist :description)))
	 (feed-title (or (plist-get opt-plist :feed-title)
			 (plist-get opt-plist :title)))
	 (feed-file (if (buffer-file-name)
			(concat (file-name-sans-extension
				 (file-truename (buffer-file-name)))
				"." org-atom-feed-extension)))
	 (body-only (or body-only (plist-get opt-plist :body-only)))
	 (atom-syndication-construct-text-html-function 'org-atom-htmlize)
	 entries feed)
    ;; check mandatory options
    (when (and (not body-only) (string= feed-url ""))
      (error "Missing url for feed"))
    ;; atom entry w/o content MUST have link pointing to the content
    (when (or (not org-atom-publish-content) (string= feed-content-url ""))
      (error "Missing url for feed content"))
    (unless to-buffer
      (setq to-buffer (if feed-file
			  (or (find-buffer-visiting feed-file)
			      (find-file-noselect feed-file))
			(error "Need a file name to be able to export")))
      (with-current-buffer to-buffer (erase-buffer)))
    ;; prepare headlines
    (when (> (length
	      (org-map-entries 'org-atom-prepare-headline feed-map-entries)) 0)
      (message "Exporting...")
      ;; there are entries in this file
      (setq entries
	    (org-map-entries '(lambda ()
				(append
				 (if body-only
				     (list (list 'author author)))
				 (org-atom-export-headline
				  feed-url
				  feed-content-url)))
			     feed-map-entries))
      (setq feed
	    (if body-only
		(mapconcat 'atom-syndication-element-entry entries "\n")
	      (atom-syndication-element-feed
	       (append
		(unless (string= description "")
		    (list (list 'subtitle (org-trim description))))
		(list
		 (list 'title (org-trim feed-title))
		 (list 'generator
		       org-atom-generator-name
		       org-atom-generator-version)
		 (list 'id (concat
			    (if (and org-atom-prefer-urn-uuid
				     (org-atom-looks-like-uuid-p feed-id))
				"urn:uuid:" "") feed-id))
		 (list 'updated (current-time))
		 (list 'link feed-url nil "self")
		 (list 'author author))
		(mapcar '(lambda (entry)
			   (cons 'entry (list entry))) entries)))))
      (if (eq to-buffer 'string)
	  feed
	(with-current-buffer to-buffer
	  (insert feed)
	  (if (buffer-file-name) (save-buffer)))))))

(defun org-atom-publish-feed-index (project &optional filename)
  "Publish feed for set of files in PROJECT.

Optional argument FILENAME is name of the output file."
  (let* ((project-plist (cdr project))
	 (dir (file-name-as-directory
	       (plist-get project-plist :base-directory)))
	 (exclude-regexp (plist-get project-plist :exclude))
	 (files (nreverse (org-publish-get-base-files project exclude-regexp)))
	 (index-filename (concat dir (or index-filename
					 (concat "feed."
						 org-atom-feed-extension))))
	 (index-title (or (plist-get project-plist :index-title)
			  (concat "Index for project " (car project))))
	 (pub-url (plist-get project-plist :publishing-url))
	 (feed-url (concat pub-url (if (string-match-p "/$" pub-url) "" "/")
			   index-filename))
	 (feed-id (or (plist-get project-plist :feed-id) feed-url))
	 (visiting (find-buffer-visiting index-filename))
	 file index-buffer)
    ;; maybe adjust publication url
    (unless (string-match-p "/$" pub-url)
      (setq pub-url (concat pub-url "/")))
    (setq project-plist (plist-put project-plist :feed-title index-title))
    (with-current-buffer (setq index-buffer
			       (or visiting (find-file index-filename)))
      (erase-buffer)
      (insert (concat "<?xml version=\"1.0\"?>\n"
		      (atom-syndication-element-feed
		       (append
			(list
			 (list 'title index-title)
			 (list 'id (concat
				    (if (and org-atom-prefer-urn-uuid
					     (org-atom-looks-like-uuid-p
					      feed-id))
					"urn:uuid:" "") feed-id))
			 (list 'updated (current-time))
			 (list 'link feed-url nil "self"))))))
      (re-search-backward "</feed>")
      (while (setq file (pop files))
	(let* ((entries-plist (org-combine-plists
			       project-plist
			       (plist-put nil :feed-content-url
					  (concat
					   pub-url
					   (file-relative-name
					    (file-name-sans-extension file) dir)
					   ".html"))))
	       (entries
		(with-current-buffer (or (find-buffer-visiting file)
					 (find-file-noselect file))
		  (org-export-as-atom entries-plist 'string t))))
	  (when entries (insert entries))))
      (save-buffer))
    (or visiting (kill-buffer index-buffer))))

(defun org-atom-export-headline (id-prefix content-url &optional pom)
  "Return atom:entry alist for headline.

ID-PREFIX is a string that is used as prefix for the atom:id
element.
CONTENT-URL is a url pointing on the published html file.
Optional argument POM is point or marker of headline.  If not
set, export headline at point."
  (save-excursion
    (goto-char (or pom (point)))
    (let* ((comps (org-heading-components))
	   (title (nth 4 comps))
	   (id (org-id-get))
	   (published (org-entry-get nil org-atom-published-property-name))
	   (updated (or (org-entry-get nil org-atom-updated-property-name)
			published))
	   (elist
	    (append
	     (if published
		 (list (list 'published (org-time-string-to-time published))))
	     (when org-atom-publish-content
	       (let (beg end content)
		 (save-excursion
		   (org-back-to-heading)
		   (setq beg (point))
		   (outline-end-of-subtree)
		   (setq end (point))
		   (setq content (buffer-substring-no-properties beg end))
		   (list (list 'content content "html")))))
	     (if (and content-url (not (string= content-url "")))
		 (list
		  (list 'link (concat content-url "#" id) nil "alternate")))
	     (list
	      (list 'title title (cons 'type "html"))
	      (list 'updated (org-time-string-to-time updated))
	      (list 'id (concat (if (and org-atom-prefer-urn-uuid
					 (org-atom-looks-like-uuid-p id))
				    "urn:uuid:" id-prefix) id))))))
      elist)))

(defun org-atom-prepare-headline (&optional pom)
  "Prepare headline at point or marker POM for export.

If POM is ommited, preapre headline at point."
  (save-excursion
    (goto-char (or pom (point)))
    (let ((id (org-id-get-create))
	  (dtime (or (org-entry-get nil org-atom-published-property-name)
		     (org-entry-get nil org-atom-updated-property-name))))
      (unless dtime
	(org-entry-put nil org-atom-published-property-name
		       (concat "["
			       (substring
				(format-time-string
				 (cdr org-time-stamp-formats))
				1 -1) "]"))))))

(defun org-atom-htmlize (string)
  "Return sanitized html markup for STRING."
  (with-temp-buffer
    (insert string)
    (org-mode)
    (atom-syndication-sanitize
     (org-export-region-as-html (point-min) (point-max) t 'string))))

(defun org-atom-looks-like-uuid-p (string)
  "Return non-nil if STRING looks like a uuid."
  (string-match-p org-atom-uuid-regexp string))

;; add infile options
(dolist (opt org-atom-infile-options)
  (unless (memq opt org-export-inbuffer-options-extra)
    (setq org-export-inbuffer-options-extra
	  (append (list opt) org-export-inbuffer-options-extra))))

(provide 'org-atom)

;;; org-atom.el ends here

[-- Attachment #1.2: Type: application/pgp-signature, Size: 230 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

^ permalink raw reply	[flat|nested] 10+ messages in thread