Re: [PATCH] org-export-generic, "text markup" -- and a request

Re: [PATCH] org-export-generic, "text markup" -- and a request

[tomas]

[-- Attachment #1.1.1: Type: text/plain, Size: 2618 bytes --]

Hi,

I tried contacting the author of org-export-generic, but don't know
whether I succeeded. Since this little patch might be useful, here it
is.

I'm in the enviable situation of transforming some docs (which I've
written in org) to mediawiki (why some people enjoy using their browser
as a doc editor escapes me, but that's for another day :)

For this, I'll have to transform the "text markup" (as I'd call that)
(what org calls "emphasis"), i.e. *foo* would become **foo**, =grumble=
would become ''grumble'' and +mumble+ would become <del>mumble</del>.

You get the idea.

I've come up with a little patch which "works for me". If that seems
useful for you, please accept it; I'd be willing to beat it into shape
if you give me some advice wrt which shape is desirable :-)

Some notes:

The "translation mapping" (org markup -> target markup) lives in the
org-generic-alist under the symbol :markup, like so (this would be for
doku-wiki):

   (org-set-generic-type
      "doku-wiki"
      '(:key-binding   ?K
  [... many lines elided ...]
	:markup (("*" . "**%s**")
                ("/" . "//%s//")
                ("_" . "__%s__")
                ("=" . "''%s''")
                ("~" . "''%s''")
                ("+" . "<del>%s</del>"))))

that is, an alist mapping the (org) markup char to a (target) format
string.

I generated the patch with the -b option. Because I wrapped a
considerable chunk of code with unwind-protect, the white space noise
obscures the patch. As I don't think the patch is final, I preferred to
keep it (human) readable.

The working is fairly straightforward: it tacks the function
org-export-generic-process-markup onto the org-export-preprocess-hook
for the whole duration off org-export-generic (the unwind-protect is
there to take this function off the hook when finished). This
org-e-g-process-markup then makes a pass through the buffer (with
org-emph-re), doing its substitutions. Some fiddling was necessary since
org-emph-re matches (possibly) a bit more than strictly the marked-up
span (I think it would have been better to use zero-width assertions for
the prefix and suffix part of org-emph-re).

Patch attached.

Now the request: I tried (several times) subscribing to this list -- but
didn't succeed (I used -- or rather tried to use -- the web interface at
<http://lists.gnu.org/mailman/listinfo/emacs-orgmode>). So please keep
me in Cc, at least as long as I wrangle with the subscriptions.

If anyone has a hint as to what I could try to subscribe, I'd be glad.

Thanks again, and regards
-- tomás

[-- Attachment #1.1.2: org-export-markup.patch --]
[-- Type: text/x-diff, Size: 0 bytes --]



[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: [PATCH] org-export-generic, "text markup" -- and a request

[Daniel Bausch]

Hi Tomas!

I have nothing to contribute to the real topic, but I wanted to inform you 
that there is a software called dokuwiki - so I got a bit irritated, whether 
you are reffering to that or to mediawiki, as you first stated.

And (at least on the mailinglist) your patch is delivered as an empty file.

Kind regards
Daniel

Re: Re: [PATCH] org-export-generic, "text markup" -- and a request

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, Jul 24, 2010 at 08:39:53AM +0200, Daniel Bausch wrote:
> Hi Tomas!
> 
> I have nothing to contribute to the real topic, but I wanted to inform you 
> that there is a software called dokuwiki - so I got a bit irritated, whether 
> you are reffering to that or to mediawiki, as you first stated.

No, that was a typo. In my special case it's actually dokuwiki (wasn't
my choice). But the patch itself isnt dependent on that.

> And (at least on the mailinglist) your patch is delivered as an empty file.

Whoops! thanks for the heads-up.

Here it is, inline (remember that I filtered out whitespace diffs -- the
indentation after patching isn't right, but the patch is more readable):

Usage is in my original mail.

- ------------------------------------------------------------------------
diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el
index 1b099dd..88c6169 100644
- --- a/contrib/lisp/org-export-generic.el
+++ b/contrib/lisp/org-export-generic.el
@@ -473,6 +473,8 @@ The prefix ARG specifies how many levels of the outline should become
 underlined headlines.  The default is 3."
   (interactive "P")
   (setq-default org-todo-line-regexp org-todo-line-regexp)
+  (unwind-protect
+      (add-hook 'org-export-preprocess-hook 'org-export-generic-process-markup)
   (let* ((opt-plist (org-combine-plists (org-default-export-plist)
 					(org-infile-export-plist)))
 	 (region-p (org-region-active-p))
@@ -541,6 +543,8 @@ underlined headlines.  The default is 3."
 		  (if (equal ass "default") org-generic-export-type ass)
 		  org-generic-alist))))
 
+	   (markup-table (plist-get export-plist :markup)) ; Need this early
+
 	 (custom-times org-display-custom-times)
 	 (org-generic-current-indentation '(0 . 0))
 	 (level 0) (old-level 0) line txt lastwastext
@@ -1021,8 +1025,23 @@ underlined headlines.  The default is 3."
 	(setq end (next-single-property-change beg 'org-cwidth))
 	(delete-region beg end)
 	(goto-char beg)))
- -    (goto-char (point-min))))
+      (goto-char (point-min)))
+    ;; Unwind:
+    (remove-hook 'org-export-preprocess-hook 'org-export-generic-process-markup)))
 
+(defun org-export-generic-process-markup ()
+  (save-excursion
+    (goto-char (point-min))
+    (while (re-search-forward org-emph-re nil t)
+      (let* ((mpre (match-string 1))  ; match prefix...
+	     (msuf (match-string 5))  ; and suffix: leave alone
+	     (mchar (match-string 3)) ; org's "markup charater"
+	     (mtext (match-string 4)) ; the marked-up text
+	     (fmt (or (cdr (assoc mchar markup-table))  ; found?
+		      (concat mchar "%s" mchar)))) ; no: leave alone
+	(replace-match
+	 (format (concat "%s" fmt "%s") mpre mtext msuf)))
+      (backward-char))))
 
 (defun org-export-generic-format (export-plist prop &optional len n reverse)
   "converts a property specification to a string given types of properties
- ------------------------------------------------------------------------

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFMSqMBBcgs9XrR2kYRArNHAJwKtIEHDAY3F6+o7mhT9pKLreJHPQCcDM7w
fgjepSJwjsZqm3lv7903Caw=
=pbVQ
-----END PGP SIGNATURE-----

Re: Re: [PATCH] org-export-generic, "text markup" -- and a request

[David Maus]

[-- Attachment #1.1: Type: text/plain, Size: 295 bytes --]


Hi Tomás,

Could I asked you to send the patch again as an attachment of type
text/plain?  If you do so Org mode's patchtracker is able to pick it
up for further review.

Thanks,
  -- David
-- 
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... dmjena@jabber.org
Email..... dmaus@ictsoc.de

[-- Attachment #1.2: Type: application/pgp-signature, Size: 230 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: [PATCH] org-export-generic, "text markup" -- and a request

[tomas]

[-- Attachment #1.1.1: Type: text/plain, Size: 414 bytes --]

On Sat, Jul 24, 2010 at 01:49:44PM +0200, David Maus wrote:
> 
> Hi Tomás,
> 
> Could I asked you to send the patch again as an attachment of type
> text/plain?  If you do so Org mode's patchtracker is able to pick it
> up for further review.

OK, I'll retry -- seems I made a mistake the first round. Here it
goes...

(BTW -- has anyone an idea why I can't subscribe to the list?)

Thanks
-- tomás

[-- Attachment #1.1.2: org-export-markup.patch --]
[-- Type: text/x-diff, Size: 2128 bytes --]

diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el
index 1b099dd..88c6169 100644
--- a/contrib/lisp/org-export-generic.el
+++ b/contrib/lisp/org-export-generic.el
@@ -473,6 +473,8 @@ The prefix ARG specifies how many levels of the outline should become
 underlined headlines.  The default is 3."
   (interactive "P")
   (setq-default org-todo-line-regexp org-todo-line-regexp)
+  (unwind-protect
+      (add-hook 'org-export-preprocess-hook 'org-export-generic-process-markup)
   (let* ((opt-plist (org-combine-plists (org-default-export-plist)
 					(org-infile-export-plist)))
 	 (region-p (org-region-active-p))
@@ -541,6 +543,8 @@ underlined headlines.  The default is 3."
 		  (if (equal ass "default") org-generic-export-type ass)
 		  org-generic-alist))))
 
+	   (markup-table (plist-get export-plist :markup)) ; Need this early
+
 	 (custom-times org-display-custom-times)
 	 (org-generic-current-indentation '(0 . 0))
 	 (level 0) (old-level 0) line txt lastwastext
@@ -1021,8 +1025,23 @@ underlined headlines.  The default is 3."
 	(setq end (next-single-property-change beg 'org-cwidth))
 	(delete-region beg end)
 	(goto-char beg)))
-    (goto-char (point-min))))
+      (goto-char (point-min)))
+    ;; Unwind:
+    (remove-hook 'org-export-preprocess-hook 'org-export-generic-process-markup)))
 
+(defun org-export-generic-process-markup ()
+  (save-excursion
+    (goto-char (point-min))
+    (while (re-search-forward org-emph-re nil t)
+      (let* ((mpre (match-string 1))  ; match prefix...
+	     (msuf (match-string 5))  ; and suffix: leave alone
+	     (mchar (match-string 3)) ; org's "markup charater"
+	     (mtext (match-string 4)) ; the marked-up text
+	     (fmt (or (cdr (assoc mchar markup-table))  ; found?
+		      (concat mchar "%s" mchar)))) ; no: leave alone
+	(replace-match
+	 (format (concat "%s" fmt "%s") mpre mtext msuf)))
+      (backward-char))))
 
 (defun org-export-generic-format (export-plist prop &optional len n reverse)
   "converts a property specification to a string given types of properties

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: [PATCH] org-export-generic, "text markup" -- and a request

[David Maus]

[-- Attachment #1.1: Type: text/plain, Size: 769 bytes --]

 wrote:
>[1  <multipart/mixed (7bit)>]
>[1.1  <text/plain; iso-8859-1 (quoted-printable)>]
>On Sat, Jul 24, 2010 at 01:49:44PM +0200, David Maus wrote:
>> 
>> Hi Tomás,
>> 
>> Could I asked you to send the patch again as an attachment of type
>> text/plain?  If you do so Org mode's patchtracker is able to pick it
>> up for further review.

>OK, I'll retry -- seems I made a mistake the first round. Here it
>goes...

Thanks, the patchtracker catched it[1].

>(BTW -- has anyone an idea why I can't subscribe to the list?)

Uh... What does it mean, you cannot subscribe to the list?

HTH,
  -- David

[1] http://patchwork.newartisans.com/patch/169/
-- 
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... dmjena@jabber.org
Email..... dmaus@ictsoc.de

[-- Attachment #1.2: Type: application/pgp-signature, Size: 230 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: [PATCH] org-export-generic, "text markup" -- and a request

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sun, Jul 25, 2010 at 04:47:24PM +0200, David Maus wrote:
>  wrote:
> >[1  <multipart/mixed (7bit)>]
> >[1.1  <text/plain; iso-8859-1 (quoted-printable)>]
> >On Sat, Jul 24, 2010 at 01:49:44PM +0200, David Maus wrote:
> >> 
> >> Hi Tomás,
> >> 
> >> Could I asked you to send the patch again as an attachment of type
> >> text/plain?  If you do so Org mode's patchtracker is able to pick it
> >> up for further review.
> 
> >OK, I'll retry -- seems I made a mistake the first round. Here it
> >goes...
> 
> Thanks, the patchtracker catched it[1].

Thank *you* :-)

> >(BTW -- has anyone an idea why I can't subscribe to the list?)
> 
> Uh... What does it mean, you cannot subscribe to the list?

(blush) oh, nevermind. I now found mailman's confirmation message It was
being filed in some place I didn't expect. Sorry for the noise...

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFMTGfcBcgs9XrR2kYRAqwpAJ43JGeulTTiMYIbtqcRNe/mbHMDkgCdHkp5
aPbx3RZlnt9i+45IN4r4YvI=
=iAC8
-----END PGP SIGNATURE-----

Re: [PATCH] org-export-generic, " text markup" -- and a request

[Robert Goldman]

FWIW, I have a number of substantial modifications to org-export-generic that
fix its original inability to handle typefaces across line boundaries.

However, my understanding is that all patches for org-export-generic wait on Wes
Hardaker to approve them.

If this would be a good time, I can ship the patches again.

Best,
r

Re: Re: [PATCH] org-export-generic, " text markup" -- and a request

[Bastien]

Robert Goldman <rpgoldman@sift.info> writes:

> FWIW, I have a number of substantial modifications to org-export-generic that
> fix its original inability to handle typefaces across line boundaries.
>
> However, my understanding is that all patches for org-export-generic wait on Wes
> Hardaker to approve them.
>
> If this would be a good time, I can ship the patches again.

Let's wait for Wes' comeback for a while.

Wes?

-- 
 Bastien

Re: Re: [PATCH] org-export-generic, " text markup" -- and a request

[Robert Goldman]

[-- Attachment #1: Type: text/plain, Size: 556 bytes --]

On 7/31/10 Jul 31 -3:29 AM, Bastien wrote:
> Robert Goldman <rpgoldman@sift.info> writes:
> 
>> FWIW, I have a number of substantial modifications to org-export-generic that
>> fix its original inability to handle typefaces across line boundaries.
>>
>> However, my understanding is that all patches for org-export-generic wait on Wes
>> Hardaker to approve them.
>>
>> If this would be a good time, I can ship the patches again.
> 
> Let's wait for Wes' comeback for a while.
> 
> Wes?
> 

OK, attached is my sequence of patches for easier reference.





[-- Attachment #2: 0007-Added-a-call-to-insert-any-bodynewline-paragraph-val.patch --]
[-- Type: text/plain, Size: 1063 bytes --]

From aefc56ca8b7ef9cf621a3833fa0100558f8823f0 Mon Sep 17 00:00:00 2001
From: Robert P. Goldman <rpgoldman@real-time.com>
Date: Sun, 30 May 2010 15:30:11 -0500
Subject: [PATCH 7/7] Added a call to insert any bodynewline-paragraph value before the start of a line item.

---
 contrib/lisp/org-export-generic.el |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el
index 114769b..f8e8c4a 100644
--- a/contrib/lisp/org-export-generic.el
+++ b/contrib/lisp/org-export-generic.el
@@ -932,9 +932,13 @@ underlined headlines.  The default is 3."
             (string-match "^\\([ \t]+\\)\\(\\*[ \t]*\\)" line))
 	;;
 	;; plain list item
-	;;
 	;; TODO: nested lists
 	;;
+        ;; first add a line break between any previous paragraph or line item and this
+        ;; one
+        (when bodynewline-paragraph
+          (insert bodynewline-paragraph))
+
         ;; I believe this gets rid of leading whitespace.
 	(setq line (replace-match "" nil nil line))
 
-- 
1.7.1


[-- Attachment #3: 0006-Add-a-test-code-file.patch --]
[-- Type: text/plain, Size: 2916 bytes --]

From eca1acab3ad7522f6dca2314c634c4b86fbb0a04 Mon Sep 17 00:00:00 2001
From: Robert P. Goldman <rpgoldman@real-time.com>
Date: Tue, 25 May 2010 09:08:04 -0500
Subject: [PATCH 6/7] Add a test code file.

---
 contrib/lisp/test-org-export-preproc.el |   39 +++++++++++++++++++++++++++++++
 1 files changed, 39 insertions(+), 0 deletions(-)
 create mode 100644 contrib/lisp/test-org-export-preproc.el

diff --git a/contrib/lisp/test-org-export-preproc.el b/contrib/lisp/test-org-export-preproc.el
new file mode 100644
index 0000000..3af8461
--- /dev/null
+++ b/contrib/lisp/test-org-export-preproc.el
@@ -0,0 +1,39 @@
+(require 'org-export-generic)
+
+(defun test-preproc ()
+  (interactive)
+  (let ((string
+         (let ((region
+                (buffer-substring
+                 (if (org-region-active-p) (region-beginning) (point-min))
+                 (if (org-region-active-p) (region-end) (point-max))))
+               (opt-plist (org-combine-plists (org-default-export-plist)
+                                              (org-infile-export-plist)))
+               (export-plist '("tikiwiki" :file-suffix ".txt" :key-binding 85 :header-prefix "" :header-suffix "" :title-format "-= %s =-\n" :date-export nil :toc-export nil :body-header-section-numbers nil :body-section-prefix "\n" :body-section-header-prefix
+                               ("! " "!! " "!!! " "!!!! " "!!!!! " "!!!!!! " "!!!!!!! ")
+                               :body-section-header-suffix
+                               (" \n" " \n" " \n" " \n" " \n" " \n")
+                               :body-line-export-preformated t :body-line-format "%s " :body-line-wrap nil :body-line-fixed-format " %s\n" :body-list-format "* %s\n" :body-number-list-format "# %s\n" :blockquote-start "\n^\n" :blockquote-end "^\n\n" :body-newline-paragraph "\n" :bold-format "__%s__" :italic-format "''%s''" :underline-format "===%s===" :strikethrough-format "--%s--" :code-format "-+%s+-" :verbatim-format "~pp~%s~/pp~")))
+    (org-export-preprocess-string
+		  region
+		  :for-ascii t
+		  :skip-before-1st-heading
+		  (plist-get opt-plist :skip-before-1st-heading)
+		  :drawers (plist-get export-plist :drawers-export)
+		  :tags (plist-get export-plist :tags-export)
+		  :priority (plist-get export-plist :priority-export)
+		  :footnotes (plist-get export-plist :footnotes-export)
+		  :timestamps (plist-get export-plist :timestamps-export)
+		  :todo-keywords (plist-get export-plist :todo-keywords-export)
+		  :verbatim-multiline t
+		  :select-tags (plist-get export-plist :select-tags-export)
+		  :exclude-tags (plist-get export-plist :exclude-tags-export)
+                  :emph-multiline t
+		  :archived-trees
+		  (plist-get export-plist :archived-trees-export)
+		  :add-text (plist-get opt-plist :text)))))
+    (save-excursion
+      (switch-to-buffer "*preproc-temp*")
+      (point-max)
+      (insert string))))
+
-- 
1.7.1


[-- Attachment #4: 0005-Expanded-docstring-for-org-emph-re.patch --]
[-- Type: text/plain, Size: 931 bytes --]

From 1f51cb4b8ce7c62699e6905eaf0aa47ef508cca0 Mon Sep 17 00:00:00 2001
From: Robert P. Goldman <rpgoldman@real-time.com>
Date: Mon, 24 May 2010 09:01:22 -0500
Subject: [PATCH 5/7] Expanded docstring for org-emph-re

---
 lisp/org.el |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 03399d7..63817d5 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3313,6 +3313,8 @@ When nil, the \\name form remains in the buffer."
 (defvar org-emph-re nil
   "Regular expression for matching emphasis.
 After a match, the match groups contain these elements:
+0  The match of the full regular expression, including the characters
+     before and after the proper match
 1  The character before the proper match, or empty at beginning of line
 2  The proper match, including the leading and trailing markers
 3  The leading marker like * or /, indicating the type of highlighting
-- 
1.7.1


[-- Attachment #5: 0004-Substantially-improved-org-export-generic-fontify-ba.patch --]
[-- Type: text/plain, Size: 3234 bytes --]

From d5f9d9793784f9e66af4d0c13736dc7e72eb7424 Mon Sep 17 00:00:00 2001
From: Robert P. Goldman <rpgoldman@real-time.com>
Date: Fri, 21 May 2010 09:14:09 -0500
Subject: [PATCH 4/7] Substantially improved org-export-generic-fontify based on help from Carsten.

---
 contrib/lisp/org-export-generic.el |   28 +++++++++++-----------------
 1 files changed, 11 insertions(+), 17 deletions(-)

diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el
index 3a7af40..114769b 100644
--- a/contrib/lisp/org-export-generic.el
+++ b/contrib/lisp/org-export-generic.el
@@ -1338,35 +1338,29 @@ conversions.")
   "Convert fontification according to generic rules."
   (if (string-match org-emph-re string)
         ;; The match goes one char after the *string*, except at the end of a line
-
-        ;; as far as I can tell from cargo-culting the code from
-        ;; the latex translation, we have the following:
-        ;; (match-string 1) is the material BEFORE the match
-        ;;          -- should be unchanged
-        ;; (match-string 3) is the actual markup character
-        ;; (match-string 4) is the material that is to be
-        ;;          marked up
-        ;; (match-string 5) is the remainder
         (let ((emph (assoc (match-string 3 string)
                            org-export-generic-emphasis-alist))
-              (beg (match-beginning 0)))
+              (beg (match-beginning 0))
+              (end (match-end 0)))
           (unless emph
             (message "`org-export-generic-emphasis-alist' has no entry for formatting triggered by \"%s\""
                      (match-string 3 string)))
           ;; now we need to determine whether we have strikethrough or
           ;; a list, which is a bit nasty
-          (if (and (equal (match-string 3 str) "+")
+          (if (and (equal (match-string 3 string) "+")
                    (save-match-data
-                     (string-match "\\`-+\\'" (match-string 4 str))))
-              ;; a list --- skip this match and recurse
-              (concat (substring str 0 (match-beginning 3))
-                      (org-export-generic-fontify (substring str (match-beginning 3))))
-              (concat (substring str 0 beg)
+                     (string-match "\\`-+\\'" (match-string 4 string))))
+              ;; a list --- skip this match and recurse on the point after the
+              ;; first emph char...
+              (concat (substring string 0 (1+ (match-beginning 3)))
+                      (org-export-generic-fontify (substring string (match-beginning 3))))
+              (concat (substring string 0 beg) ;; part before the match
                       (match-string 1 string)
                       (org-export-generic-emph-format (second emph)
                                                       (match-string 4 string)
                                                       (third emph))
-                      (org-export-generic-fontify (match-string 5 string)))))
+                      (or (match-string 5 string) "")
+                      (org-export-generic-fontify (substring string end)))))
         string))
 
 (defun org-export-generic-emph-format (format-varname string protect)
-- 
1.7.1


[-- Attachment #6: 0003-Partial-solution-to-the-fontification-problem.-Havin.patch --]
[-- Type: text/plain, Size: 7910 bytes --]

From 99f7bb368f73fcc5a6e2a4d05ce7dd4a4bca4919 Mon Sep 17 00:00:00 2001
From: Robert P. Goldman <rpgoldman@real-time.com>
Date: Thu, 20 May 2010 21:49:32 -0500
Subject: [PATCH 3/7] Partial solution to the fontification problem.  Having some trouble with the MATCH-STRING calls, but mostly ok.

---
 contrib/lisp/org-export-generic.el |  105 ++++++++++++++++++++++++++++++++++--
 1 files changed, 101 insertions(+), 4 deletions(-)

diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el
index 0933f19..3a7af40 100644
--- a/contrib/lisp/org-export-generic.el
+++ b/contrib/lisp/org-export-generic.el
@@ -1,4 +1,4 @@
-;;; org-export-generic.el --- Export frameworg with custom backends
+;; org-export-generic.el --- Export frameworg with custom backends
 
 ;; Copyright (C) 2009  Free Software Foundation, Inc.
 
@@ -466,6 +466,15 @@ preformatted text\).  A common non-nil value for this keyword
 is \"\\n\".  Should typically be combined with a value for 
 :body-line-format that does NOT end with a newline."
     :type string)
+
+;;; fontification keywords
+(def-org-export-generic-keyword :bold-format)
+(def-org-export-generic-keyword :italic-format)
+(def-org-export-generic-keyword :underline-format)
+(def-org-export-generic-keyword :strikethrough-format)
+(def-org-export-generic-keyword :code-format)
+(def-org-export-generic-keyword :verbatim-format)
+
     
   
 
@@ -623,6 +632,7 @@ underlined headlines.  The default is 3."
 		  :verbatim-multiline t
 		  :select-tags (plist-get export-plist :select-tags-export)
 		  :exclude-tags (plist-get export-plist :exclude-tags-export)
+                  :emph-multiline t
 		  :archived-trees
 		  (plist-get export-plist :archived-trees-export)
 		  :add-text (plist-get opt-plist :text))
@@ -671,6 +681,16 @@ underlined headlines.  The default is 3."
 	 (bodylineform  (or (plist-get export-plist :body-line-format) "%s"))
          (blockquotestart (or (plist-get export-plist :blockquote-start) "\n\n\t"))
          (blockquoteend (or (plist-get export-plist :blockquote-end) "\n\n"))
+
+         ;; dynamic variables used heinously in fontification
+         ;; not referenced locally...
+         (format-boldify (plist-get export-plist :bold-format))
+         (format-italicize (plist-get export-plist :italic-format))
+         (format-underline (plist-get export-plist :underline-format))
+         (format-strikethrough (plist-get export-plist :strikethrough-format))
+         (format-code (plist-get export-plist :code-format))
+         (format-verbatim (plist-get export-plist :verbatim-format))
+
          
 
 	 thetoc toctags have-headings first-heading-pos
@@ -854,7 +874,7 @@ underlined headlines.  The default is 3."
 	    (if org-export-generic-links-to-notes
 		(push (cons desc0 link) link-buffer)
 	      (setq rpl (concat rpl " (" link ")")
-		    wrap (+ (length line) (- (length (match-string 0) line))
+		    wrap (+ (length line) (- (length (match-string 0 line)))
 			    (length desc)))))
 	  (setq line (replace-match rpl t t line))))
       (when custom-times
@@ -936,7 +956,7 @@ underlined headlines.  The default is 3."
 			     listcheckhalfend)))
 	 )
 
-	(insert (format listformat line)))
+	(insert (format listformat (org-export-generic-fontify line))))
        ((string-match "^\\([ \t]+\\)\\([0-9]+\\.[ \t]*\\)" line)
 	;;
 	;; numbered list item
@@ -962,7 +982,7 @@ underlined headlines.  The default is 3."
 			     listcheckhalfend)))
 	 )
 
-	(insert (format numlistformat line)))
+	(insert (format numlistformat (org-export-generic-fontify line))))
 
        ((equal line "ORG-BLOCKQUOTE-START")
         (setq line blockquotestart))
@@ -978,6 +998,9 @@ underlined headlines.  The default is 3."
 	;;
 	(org-export-generic-check-section "body" bodytextpre bodytextsuf)
 
+        (setq line 
+              (org-export-generic-fontify line))
+
 	;; XXX: properties?  list?
 	(if (string-match "^\\([ \t]*\\)\\([-+*][ \t]+\\)\\(.*?\\)\\( ::\\)" line)
 	    (setq line (replace-match "\\1\\3:" t nil line)))
@@ -1284,6 +1307,80 @@ REVERSE means to reverse the list if the plist match is a list
     (and vl (setcar vl nil))
     vl))
 
+
+;;; FIXME: this should probably turn into a defconstant later [2010/05/20:rpg]
+(defvar org-export-generic-emphasis-alist
+  '(("*" format-boldify nil)
+    ("/" format-italicize nil)
+    ("_" format-underline nil)
+    ("+" format-strikethrough nil)
+    ("=" format-code t)
+    ("~" format-verbatim t))
+  "Alist of org format -> formatting variables for fontification.
+Each element of the list is a list of three elements.
+The first element is the character used as a marker for fontification.
+The second element is a variable name, set in org-export-generic.  That
+variable will be dereferenced to obtain a formatting string to wrap 
+fontified text with.
+The third element decides whether to protect converted text from other
+conversions.")
+
+;;; Cargo-culted from the latex translation.  I couldn't figure out how
+;;; to keep the structure since the generic export operates on lines, rather
+;;; than on a buffer as in the latex export, meaning that none of the
+;;; search forward code could be kept.  This led me to rewrite the
+;;; whole thing recursively.  A huge lose for efficiency (potentially),
+;;; but I couldn't figure out how to make the looping work.
+;;; Worse, it's /doubly/ recursive, because this function calls
+;;; org-export-generic-emph-format, which can call it recursively...
+;;; [2010/05/20:rpg]
+(defun org-export-generic-fontify (string)
+  "Convert fontification according to generic rules."
+  (if (string-match org-emph-re string)
+        ;; The match goes one char after the *string*, except at the end of a line
+
+        ;; as far as I can tell from cargo-culting the code from
+        ;; the latex translation, we have the following:
+        ;; (match-string 1) is the material BEFORE the match
+        ;;          -- should be unchanged
+        ;; (match-string 3) is the actual markup character
+        ;; (match-string 4) is the material that is to be
+        ;;          marked up
+        ;; (match-string 5) is the remainder
+        (let ((emph (assoc (match-string 3 string)
+                           org-export-generic-emphasis-alist))
+              (beg (match-beginning 0)))
+          (unless emph
+            (message "`org-export-generic-emphasis-alist' has no entry for formatting triggered by \"%s\""
+                     (match-string 3 string)))
+          ;; now we need to determine whether we have strikethrough or
+          ;; a list, which is a bit nasty
+          (if (and (equal (match-string 3 str) "+")
+                   (save-match-data
+                     (string-match "\\`-+\\'" (match-string 4 str))))
+              ;; a list --- skip this match and recurse
+              (concat (substring str 0 (match-beginning 3))
+                      (org-export-generic-fontify (substring str (match-beginning 3))))
+              (concat (substring str 0 beg)
+                      (match-string 1 string)
+                      (org-export-generic-emph-format (second emph)
+                                                      (match-string 4 string)
+                                                      (third emph))
+                      (org-export-generic-fontify (match-string 5 string)))))
+        string))
+
+(defun org-export-generic-emph-format (format-varname string protect)
+  "Return a string that results from applying the markup indicated by
+FORMAT-VARNAME to STRING."
+  (let ((format (symbol-value format-varname)))
+    (let ((string-to-emphasize
+           (if protect
+               string
+               (org-export-generic-fontify string))))
+      (if format
+          (format format string-to-emphasize)
+          string-to-emphasize))))
+
 (provide 'org-generic)
 (provide 'org-export-generic)
 
-- 
1.7.1


[-- Attachment #7: 0002-Make-newline-handling-more-flexible-per-WH-declare-k.patch --]
[-- Type: text/plain, Size: 1232 bytes --]

From 749ca33524ab0d0f96bb71e32bd0cc08070880d9 Mon Sep 17 00:00:00 2001
From: Robert P. Goldman <rpgoldman@real-time.com>
Date: Sun, 9 May 2010 10:20:15 -0500
Subject: [PATCH 2/7] Make newline-handling more flexible (per WH), declare keywords.

Followed Wes Hardaker's suggestion to make the translation of
newlines more flexible --- instead of making a boolean for
special translation of blank lines, I added the ability to
specify the translation.

Also added a macro for declaring generic translation keywords
with type information and documentation.  Hope this will make
the generic translator easier to use.
---
 contrib/lisp/org-export-generic.el |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el
index 23ecf9a..0933f19 100644
--- a/contrib/lisp/org-export-generic.el
+++ b/contrib/lisp/org-export-generic.el
@@ -465,7 +465,7 @@ newlines are interpreted as significant \(e.g., as indicating
 preformatted text\).  A common non-nil value for this keyword
 is \"\\n\".  Should typically be combined with a value for 
 :body-line-format that does NOT end with a newline."
-    :type string))
+    :type string)
     
   
 
-- 
1.7.1


[-- Attachment #8: 0001-Revision-to-handling-of-blank-lines.-Start-of-declar.patch --]
[-- Type: text/plain, Size: 2512 bytes --]

From 3d4d938009cf04d987ae4522ddd90558c48dc962 Mon Sep 17 00:00:00 2001
From: Robert P. Goldman <rpgoldman@real-time.com>
Date: Thu, 6 May 2010 14:16:37 -0500
Subject: [PATCH 1/7] Revision to handling of blank lines.  Start of declaration protocol.

Followed Wes Hardaker's idea of permitting alternative rewrites for blank
lines, instead of making the blank line handler be a boolean and
hard-wiring a newline character.

Also added a declaration form, with type and documentation options, for the
keywords used in defining a generic export method.
---
 contrib/lisp/org-export-generic.el |   27 ++++++++++++++++++++++++++-
 1 files changed, 26 insertions(+), 1 deletions(-)

diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el
index 1b099dd..23ecf9a 100644
--- a/contrib/lisp/org-export-generic.el
+++ b/contrib/lisp/org-export-generic.el
@@ -444,6 +444,31 @@ in this way, it will be wrapped."
 export definitions."
   (aput 'org-generic-alist type definition))
 
+;;; helper functions for org-set-generic-type
+(defvar org-export-generic-keywords nil)
+(defmacro* def-org-export-generic-keyword (keyword
+                                           &key documentation
+                                                type)
+  "Define KEYWORD as a legitimate element for inclusion in
+the body of an org-set-generic-type definition."
+  `(progn
+     (pushnew ,keyword org-export-generic-keywords)
+     ;; TODO: push the documentation and type information
+     ;; somewhere where it will do us some good.
+     ))
+
+(def-org-export-generic-keyword :body-newline-paragraph
+    :documentation "Bound either to NIL or to a pattern to be 
+inserted in the output for every blank line in the input.
+  The intention is to handle formats where text is flowed, and
+newlines are interpreted as significant \(e.g., as indicating
+preformatted text\).  A common non-nil value for this keyword
+is \"\\n\".  Should typically be combined with a value for 
+:body-line-format that does NOT end with a newline."
+    :type string))
+    
+  
+
 (defun org-export-generic-remember-section (type suffix &optional prefix)
   (setq org-export-generic-section-type type)
   (setq org-export-generic-section-suffix suffix)
@@ -946,7 +971,7 @@ underlined headlines.  The default is 3."
        ((string-match "^\\s-*$" line)
         ;; blank line
         (if bodynewline-paragraph
-            (insert "\n")))
+            (insert bodynewline-paragraph)))
        (t
 	;;
 	;; body
-- 
1.7.1


[-- Attachment #9: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Orgmode[PATCH] org-export-generic, "text markup" -- and a request

[Wes Hardaker]

>>>>> On Fri, 23 Jul 2010 13:19:31 +0200, tomas@tuxteam.de said:

t> I tried contacting the author of org-export-generic, but don't know
t> whether I succeeded. Since this little patch might be useful, here it
t> is.

You succeeded...  I'm just completely out of touch for the last two
weeks and it'll continue into next week minus this very very short break
when I actually can read mail...

Anyway, I haven't read the email chain yet but if others think the patch
is fine then it may certainly be applied!
-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: Orgmode[PATCH] org-export-generic, "text markup" -- and a request

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, Aug 02, 2010 at 06:54:44AM -0700, Wes Hardaker wrote:
> >>>>> On Fri, 23 Jul 2010 13:19:31 +0200, tomas@tuxteam.de said:
> 
> t> I tried contacting the author of org-export-generic, but don't know
> t> whether I succeeded. Since this little patch might be useful, here it
> t> is.
> 
> You succeeded...  I'm just completely out of touch for the last two
> weeks and it'll continue into next week minus this very very short break
> when I actually can read mail...

Do take your time. No impatience here.

> Anyway, I haven't read the email chain yet but if others think the patch
> is fine then it may certainly be applied!

I do hate myself how I did some things (the naming could take some
discussion. For example: "text markup" is a bad name. I don't know how
to subsume "strong", "emphasized"...). Besides, I'm unsure about this
hook-adding business within the unwind-protect. Maybe others could chime
in?

Thanks, regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFMVwCWBcgs9XrR2kYRAjTDAJ9X1jRrMJXKSnDkEjvotpPe/uiMPwCfRd66
/gJuF5E2ZI96o0cvLGqPpUU=
=oO1k
-----END PGP SIGNATURE-----

Re: Re: Orgmode[PATCH] org-export-generic, "text markup" -- and a request

[Carsten Dominik]

Hi everyone,

I am not sure what the status o this patch is now, is there anyone  
besides the author whw has tested it and can comment on it?

- Carsten

On Aug 2, 2010, at 3:54 PM, Wes Hardaker wrote:

>>>>>> On Fri, 23 Jul 2010 13:19:31 +0200, tomas@tuxteam.de said:
>
> t> I tried contacting the author of org-export-generic, but don't know
> t> whether I succeeded. Since this little patch might be useful,  
> here it
> t> is.
>
> You succeeded...  I'm just completely out of touch for the last two
> weeks and it'll continue into next week minus this very very short  
> break
> when I actually can read mail...
>
> Anyway, I haven't read the email chain yet but if others think the  
> patch
> is fine then it may certainly be applied!
> -- 
> Wes Hardaker
> My Pictures:  http://capturedonearth.com/
> My Thoughts:  http://pontifications.hardakers.net/
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

- Carsten

Re: Orgmode[PATCH] org-export-generic, "text markup" -- and a request

[Robert Goldman]

On 8/13/10 Aug 13 -3:35 PM, Carsten Dominik wrote:
> Hi everyone,
> 
> I am not sure what the status o this patch is now, is there anyone
> besides the author whw has tested it and can comment on it?

While we're at it, what's the status of my patches that add text markup
(and text markup across line boundaries) to org-export-generic.

I'm actually pretty anxious to see other patches be added before mine
goes in.  Mine had to revise the handling so that it wasn't
line-by-line, but now uses the same preprocessing as other org export
facilities.  I fear that other patches might clash with that pretty badly.

Best,
R

Re: Re: Orgmode[PATCH] org-export-generic, "text markup" -- and a request

[Carsten Dominik]

Hi Robert,

can you point me to that patch?  Any other testers?  Did Wes take a  
look and greed to apply it?

I have lost track and need help before I can act.

- Carsten

On Aug 19, 2010, at 5:04 AM, Robert Goldman wrote:

> On 8/13/10 Aug 13 -3:35 PM, Carsten Dominik wrote:
>> Hi everyone,
>>
>> I am not sure what the status o this patch is now, is there anyone
>> besides the author whw has tested it and can comment on it?
>
> While we're at it, what's the status of my patches that add text  
> markup
> (and text markup across line boundaries) to org-export-generic.
>
> I'm actually pretty anxious to see other patches be added before mine
> goes in.  Mine had to revise the handling so that it wasn't
> line-by-line, but now uses the same preprocessing as other org export
> facilities.  I fear that other patches might clash with that pretty  
> badly.
>
> Best,
> R
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

- Carsten

Re: Re: Orgmode[PATCH] org-export-generic, "text markup" -- and a request

[Robert Goldman]

On 8/20/10 Aug 20 -9:09 AM, Carsten Dominik wrote:
> Hi Robert,
> 
> can you point me to that patch?  Any other testers?  Did Wes take a look
> and greed to apply it?
> 

No, I'm afraid I have sent multiple emails and have heard nothing.

I don't have a good way to recruit any other testers, unfortunately.
I'm not sure how many people actually use org-generic-export; I do
because I have wikis I need to write to and I can never remember all
those oddball wiki syntaxen.

Here's a question:  what about setting up a branch on the core git repo
and letting me push that patch there?  Or you could push the patch to
the branch yourself.  That way people who are interested could test, and
those who aren't interested could just ignore it.

My patch seems really quite critical for anyone who DOES want to use
org-generic-export.  The existing generic export simply does not and
CANNOT work when markup spreads across line boundaries.  Generic export
processes the org-mode file line-by-line.  I fixed it to use the
preprocessing that is done by other org exporters, bringing together
multiple lines when there is markup (say italicization) that spreads
across line boundaries.

My next message will be a forwarding of a prior message containing the
patches.  I will send it only to you because I don't believe the mailing
list will take it.  Interested readers can find it at:

http://thread.gmane.org/gmane.emacs.orgmode/27959/focus=28255

Best,
r

Re: Re: Orgmode[PATCH] org-export-generic, "text markup" -- and a request

[Carsten Dominik]

On Aug 20, 2010, at 7:05 PM, Robert Goldman wrote:

> On 8/20/10 Aug 20 -9:09 AM, Carsten Dominik wrote:
>> Hi Robert,
>>
>> can you point me to that patch?  Any other testers?  Did Wes take a  
>> look
>> and greed to apply it?
>>
>
> No, I'm afraid I have sent multiple emails and have heard nothing.
>
> I don't have a good way to recruit any other testers, unfortunately.
> I'm not sure how many people actually use org-generic-export; I do
> because I have wikis I need to write to and I can never remember all
> those oddball wiki syntaxen.
>
> Here's a question:  what about setting up a branch on the core git  
> repo
> and letting me push that patch there?  Or you could push the patch to
> the branch yourself.  That way people who are interested could test,  
> and
> those who aren't interested could just ignore it.
>
> My patch seems really quite critical for anyone who DOES want to use
> org-generic-export.  The existing generic export simply does not and
> CANNOT work when markup spreads across line boundaries.  Generic  
> export
> processes the org-mode file line-by-line.  I fixed it to use the
> preprocessing that is done by other org exporters, bringing together
> multiple lines when there is markup (say italicization) that spreads
> across line boundaries.
>
> My next message will be a forwarding of a prior message containing the
> patches.  I will send it only to you because I don't believe the  
> mailing
> list will take it.  Interested readers can find it at:
>
> http://thread.gmane.org/gmane.emacs.orgmode/27959/focus=28255

Hi Robert,

I think the best way to proceed is that I simply apply the
patch.  This is definitely the best way to get people to test it.

I would like to ask that you - at least for a while - become
the responsible person for the generic exporter.
Wes is, I guess, too busy or not interested right now.  So if
you could please look at any additional patches for he general
exporter and make a recommendation as to wether to apply them
and what changes are necessary.

I have just applied al your 7 patches and pushed the result.

Can you please get in touch with tomas@tuxteam.de and see if
and what has to be done with his patch?

Thanks, in particular for your patience.

- Carsten

Re: Orgmode[PATCH] org-export-generic, "text markup" -- and a request

[Wes Hardaker]

>>>>> On Fri, 20 Aug 2010 12:05:46 -0500, Robert Goldman <rpgoldman@sift.info> said:

RG> No, I'm afraid I have sent multiple emails and have heard nothing.

As I mentioned, I had a huge number of "away trips" in the summer and
just got over the last of my deadlines last week.  I was hoping to look
at them this week, and I apologize for the delay.

That being said, I typically prefer to take the "people intend to do
good" approach to OSS development and applying them without a more
detailed review is just fine with me Carsten.  Please don't let my
in-and-out schedule hold up others!

-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: Orgmode[PATCH] org-export-generic, "text markup" -- and a request

[Wes Hardaker]

>>>>> On Sat, 21 Aug 2010 05:25:49 +0200, Carsten Dominik <carsten.dominik@gmail.com> said:

CD> I would like to ask that you - at least for a while - become
CD> the responsible person for the generic exporter.

I'm more than fine with letting someone else either co-manage or manage 
its development.  That means faster development to me :-)
-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: Orgmode[PATCH] org-export-generic, "text markup" -- and a request

[Robert Goldman]

On 8/23/10 Aug 23 -9:16 AM, Wes Hardaker wrote:
>>>>>> On Sat, 21 Aug 2010 05:25:49 +0200, Carsten Dominik <carsten.dominik@gmail.com> said:
> 
> CD> I would like to ask that you - at least for a while - become
> CD> the responsible person for the generic exporter.
> 
> I'm more than fine with letting someone else either co-manage or manage 
> its development.  That means faster development to me :-)

That's fine with me.  Note that I am on vacation this week.  That's why
I haven't sent a response earlier.  I'll be mostly unavailable until the
1 September.

I have a couple of further patches to push that involve adding new
export formats based on the generic exporter (a couple of wikis).

I'll see if I can squeeze in time to do some tests (export modules are
reasonably easy to test), but that won't be for a while.

Best,
Robert