Finer-grained control of published files

Finer-grained control of published files

[Brett Viren]

[-- Attachment #1: Type: text/plain, Size: 835 bytes --]

Hi,

I'm trying to set up org-publish and am looking for more fine-grained
control over what files get published than what (I think) I can get from
configuring org-publish-project-alist.  I've played with the
publishing-function of org-publish-attachment and :exclude/:include and
:base-extension regexps.

What I'm finding is that I want to control what type of files get
published on almost a per-directory basis and different directories may
have mutually conflicting file patterns to include/exclude.  Creating a
new org-publish-project-alist entry for each is tedious.

What I hope for is something equivalent to git's .gitignore
functionality where I can place, say, .orgignore files full of regexp
patterns anywhere in my org source tree and have org-publish honor them.

Is there anything in this direction?

Thanks,
-Brett.



[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Finer-grained control of published files

[Bastien]

[-- Attachment #1: Type: text/plain, Size: 547 bytes --]

Hi Brett,

Brett Viren <bv@bnl.gov> writes:

> What I hope for is something equivalent to git's .gitignore
> functionality where I can place, say, .orgignore files full of regexp
> patterns anywhere in my org source tree and have org-publish honor
> them.

I like this idea.

> Is there anything in this direction?

Please test the attached patch against the tip of the master branch
and let me know if it works: it checks against a .oxignore file, one
regexp on each line.  If you find it useful, I'll commit this for
the next version.

Thanks,


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: oxignore.patch --]
[-- Type: text/x-diff, Size: 3831 bytes --]

Changes in stash@{0}^2^..stash@{0}
1 file changed, 32 insertions(+), 19 deletions(-)
 lisp/ox-publish.el | 51 ++++++++++++++++++++++++++++++++-------------------

	Modified   lisp/ox-publish.el
diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el
index ad8eb32..6f20151 100644
--- a/lisp/ox-publish.el
+++ b/lisp/ox-publish.el
@@ -435,37 +435,50 @@ This splices all the components into the list."
     retval))
 
 (defun org-publish-get-base-files-1
-  (base-dir &optional recurse match skip-file skip-dir)
+  (base-dir &optional recurse match skip-file-regexp skip-dir-regexp)
   "Set `org-publish-temp-files' with files from BASE-DIR directory.
 If RECURSE is non-nil, check BASE-DIR recursively.  If MATCH is
 non-nil, restrict this list to the files matching the regexp
-MATCH.  If SKIP-FILE is non-nil, skip file matching the regexp
-SKIP-FILE.  If SKIP-DIR is non-nil, don't check directories
-matching the regexp SKIP-DIR when recursing through BASE-DIR."
+MATCH.  If SKIP-FILE-REGEXP is non-nil, skip file matching this
+regexp.  If SKIP-DIR-REGEXP is non-nil, don't check directories
+matching the regexp SKIP-DIR-REGEXP when recursing through BASE-DIR."
   (mapc (lambda (f)
 	  (let ((fd-p (file-directory-p f))
 		(fnd (file-name-nondirectory f)))
 	    (if (and fd-p recurse
 		     (not (string-match "^\\.+$" fnd))
-		     (if skip-dir (not (string-match skip-dir fnd)) t))
+		     (if skip-dir-regexp (not (string-match skip-dir-regexp fnd)) t))
 		(org-publish-get-base-files-1
-		 f recurse match skip-file skip-dir)
+		 f recurse match skip-file-regexp skip-dir-regexp)
 	      (unless (or fd-p ;; this is a directory
-			  (and skip-file (string-match skip-file fnd))
+			  (and skip-file-regexp (string-match skip-file-regexp fnd))
 			  (not (file-exists-p (file-truename f)))
 			  (not (string-match match fnd)))
-
 		(pushnew f org-publish-temp-files)))))
-	(let ((all-files (if (not recurse) (directory-files base-dir t match)
-			   ;; If RECURSE is non-nil, we want all files
-			   ;; matching MATCH and sub-directories.
-			   (org-remove-if-not
-			    (lambda (file)
-			      (or (file-directory-p file)
-				  (and match (string-match match file))))
-			    (directory-files base-dir t)))))
-	  (if (not org-publish-sitemap-requested) all-files
-	    (sort all-files 'org-publish-compare-directory-files)))))
+	(let* ((oxi (expand-file-name (concat base-dir ".oxignore")))
+	       reoxi
+	       (notmatch (when (file-exists-p oxi)
+			   (with-temp-buffer
+			     (insert-file-contents oxi)
+			     (goto-char (point-min))
+			     (while (re-search-forward "^\\(.+\\)$" nil t)
+			       (if (< 0 (length (org-trim (match-string 1))))
+				   (push (match-string 1) reoxi))))
+			   (if reoxi (mapconcat 'identity reoxi "\\|"))))
+	       (all-files (if (not recurse)
+			      (directory-files base-dir t match)
+			    ;; If RECURSE is non-nil, we want all files
+			    ;; matching MATCH and sub-directories.
+			    (org-remove-if-not
+			     (lambda (file)
+			       (or (file-directory-p file)
+				   (and match (string-match match file))))
+			     (directory-files base-dir t))))
+	       (all-files2 (org-remove-if (lambda (file)
+					    (and notmatch (string-match notmatch file)))
+					  all-files)))
+	  (if (not org-publish-sitemap-requested) all-files2
+	    (sort all-files2 'org-publish-compare-directory-files)))))
 
 (defun org-publish-get-base-files (project &optional exclude-regexp)
   "Return a list of all files in PROJECT.
@@ -512,7 +525,7 @@ matching filenames."
 		  org-publish-temp-files))
     (org-publish-get-base-files-1 base-dir recurse match
 				  ;; FIXME distinguish exclude regexp
-				  ;; for skip-file and skip-dir?
+				  ;; for skip-file-regexp and skip-dir-regexp?
 				  exclude-regexp exclude-regexp)
     (mapc (lambda (f)
 	    (pushnew


[-- Attachment #3: Type: text/plain, Size: 14 bytes --]


-- 
 Bastien

Re: Finer-grained control of published files

[Brett Viren]

[-- Attachment #1: Type: text/plain, Size: 765 bytes --]

Bastien <bzg@gnu.org> writes:

> Please test the attached patch against the tip of the master branch
> and let me know if it works: it checks against a .oxignore file, one
> regexp on each line.  If you find it useful, I'll commit this for
> the next version.

I had some unrelated trouble with the head of master so I applied your
patch to the 8.2.5h tag.  It applied cleanly with a little fuzz.

It seems to work great! I played with various ways to ignore and
couldn't make it fail.  

One thing, and I know it's going back on what I asked originally, but I
wonder if file globs would be the more natural pattern matching here
than regexp?  

Either way, this will really help publishing from my messy source
directories  

Thanks!
-Brett.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Finer-grained control of published files

[Bastien]

Brett Viren <bv@bnl.gov> writes:

> Either way, this will really help publishing from my messy source
> directories

Thanks for testing.

If nobody feels strongly against this, I'll add the feature to the
master branch.

-- 
 Bastien