How to apply a list of regex replaces to multiple files?

How to apply a list of regex replaces to multiple files?

[michael.l]

I've browsed the messages here and have googled but don't see a clear
solution yet. I have about 900 documents to which I need to apply a
list of maybe 40 separate regex search and replaces. I would like to
feed a list of the regex expressions and replacements to emacs and
have it applied to a directory of the files. Any solutions? Keyboard
macros don't seem like the right solution....

Re: How to apply a list of regex replaces to multiple files?

[David Kastrup]

"michael.l" <michael.lommel@gmail.com> writes:

> I've browsed the messages here and have googled but don't see a clear
> solution yet. I have about 900 documents to which I need to apply a
> list of maybe 40 separate regex search and replaces. I would like to
> feed a list of the regex expressions and replacements to emacs and
> have it applied to a directory of the files. Any solutions? Keyboard
> macros don't seem like the right solution....

dired-do-query-replace-regexp is an interactive compiled Lisp function
in `dired-aux.el'.

It is bound to Q, <menu-bar> <operate> <query-replace>.

(dired-do-query-replace-regexp FROM TO &optional DELIMITED)

Do `query-replace-regexp' of FROM with TO, on all marked files.
Third arg DELIMITED (prefix arg) means replace only word-delimited matches.
If you exit (C-g, RET or q), you can resume the query replace
with the command M-,.


-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: How to apply a list of regex replaces to multiple files?

[michael.l]

On Jun 30, 10:23 pm, David Kastrup <d...@gnu.org> wrote:
> "michael.l" <michael.lom...@gmail.com> writes:
> > I've browsed the messages here and have googled but don't see a clear
> > solution yet. I have about 900 documents to which I need to apply a
> > list of maybe 40 separate regex search and replaces. I would like to
> > feed a list of the regex expressions and replacements to emacs and
> > have it applied to a directory of the files. Any solutions? Keyboard
> > macros don't seem like the right solution....
>
> dired-do-query-replace-regexp is an interactive compiled Lisp function
> in `dired-aux.el'.
>
> It is bound to Q, <menu-bar> <operate> <query-replace>.
>
> (dired-do-query-replace-regexp FROM TO &optional DELIMITED)
>
> Do `query-replace-regexp' of FROM with TO, on all marked files.
> Third arg DELIMITED (prefix arg) means replace only word-delimited matches.
> If you exit (C-g, RET or q), you can resume the query replace
> with the command M-,.
>
> --
> David Kastrup, Kriemhildstr. 15, 44793 Bochum

Thank you David; hadn't come across that yet. Is there a way to input
the entire list of regex replacements rather than doing each from the
minibuffer? And will this mean typing "!" 900 times? I'm just learning
emacs.

Re: How to apply a list of regex replaces to multiple files?

[Tim X]

"michael.l" <michael.lommel@gmail.com> writes:

> I've browsed the messages here and have googled but don't see a clear
> solution yet. I have about 900 documents to which I need to apply a
> list of maybe 40 separate regex search and replaces. I would like to
> feed a list of the regex expressions and replacements to emacs and
> have it applied to a directory of the files. Any solutions? Keyboard
> macros don't seem like the right solution....

If all the files are in one directory, you could probably do this with
dired?

Tim

-- 
tcross (at) rapttech dot com dot au

Re: How to apply a list of regex replaces to multiple files?

[Phil Carmody]

"michael.l" <michael.lommel@gmail.com> writes:
> I've browsed the messages here and have googled but don't see a clear
> solution yet. I have about 900 documents to which I need to apply a
> list of maybe 40 separate regex search and replaces. I would like to
> feed a list of the regex expressions and replacements to emacs and
> have it applied to a directory of the files. Any solutions? Keyboard
> macros don't seem like the right solution....

Emacs is an interactive editor, most of the time in order
to achieve some modification of a file, you need to interact
with it.

Have you considered simply using `find' and `sed' (or even 
'perl') from the command line instead of an interactive
editor. They sound more like the right tool for the job.

Phil
-- 
Dear aunt, let's set so double the killer delete select all.
-- Microsoft voice recognition live demonstration

Re: How to apply a list of regex replaces to multiple files?

[Lennart Borgman (gmail)]

michael.l wrote:
> I've browsed the messages here and have googled but don't see a clear
> solution yet. I have about 900 documents to which I need to apply a
> list of maybe 40 separate regex search and replaces. I would like to
> feed a list of the regex expressions and replacements to emacs and
> have it applied to a directory of the files. Any solutions? Keyboard
> macros don't seem like the right solution....

Sounds like the best would be writing a small elisp function to do the job.

Re: How to apply a list of regex replaces to multiple files?

[michael.l]

On Jul 1, 1:38 am, "Lennart Borgman (gmail)"
<lennart.borg...@gmail.com> wrote:
> michael.l wrote:
> > I've browsed the messages here and have googled but don't see a clear
> > solution yet. I have about 900 documents to which I need to apply a
> > list of maybe 40 separate regex search and replaces. I would like to
> > feed a list of the regex expressions and replacements to emacs and
> > have it applied to a directory of the files. Any solutions? Keyboard
> > macros don't seem like the right solution....
>
> Sounds like the best would be writing a small elisp function to do the job.

I'll look into elisp...Struggled with perl to do this....

Re: How to apply a list of regex replaces to multiple files?

[Joel J. Adamson]

"michael.l" <michael.lommel@gmail.com> writes:

> On Jul 1, 1:38 am, "Lennart Borgman (gmail)"
> <lennart.borg...@gmail.com> wrote:
>> michael.l wrote:
>> > I've browsed the messages here and have googled but don't see a clear
>> > solution yet. I have about 900 documents to which I need to apply a
>> > list of maybe 40 separate regex search and replaces. I would like to
>> > feed a list of the regex expressions and replacements to emacs and
>> > have it applied to a directory of the files. Any solutions? Keyboard
>> > macros don't seem like the right solution....
>>
>> Sounds like the best would be writing a small elisp function to do the job.
>
> I'll look into elisp...Struggled with perl to do this....

Perl is a wrecking ball for this chainsaw problem.  Use sed.

On the other hand, if you want to use Emacs Lisp, look into the "cl"
(Common Lisp) macros, such as dolist.

This may be a good start, but you'd have to implement another function
or add to this to get it to call up each file into a buffer and save
it.  

(defun jedit-strip-regex (alist)
  "Takes a list of regex-replacement string pairs;
processes entire buffer."
  (interactive "sList: ")
  ;; for each cell in alist, define regex and replacement text
  (dolist (regex-cell alist)
      (let ((regex (car regex-cell))
	    (replacement (cadr regex-cell)))
	;; go to beginning of buffer
	(goto-char (point-min))
	;; when you find the search string, replace it with replacement
	;; text
	(while (re-search-forward regex nil t)
	  (replace-match replacement nil nil)))))


By the way, you can use Emacs non-interactively just as you would use
sed or awk, by using the --script argument.  For example

#!/usr/bin/emacs --script

[ ...

script = Emacs lisp with some imperative statements (do something)

... ]

Joel

-- 
Joel J. Adamson
(303) 880-3109
Public key: http://pgp.mit.edu
http://www.unc.edu/~adamsonj
http://trashbird1240.blogspot.com

Re: How to apply a list of regex replaces to multiple files?

[Tim X]

"michael.l" <michael.lommel@gmail.com> writes:

> On Jul 1, 1:38 am, "Lennart Borgman (gmail)"
> <lennart.borg...@gmail.com> wrote:
>> michael.l wrote:
>> > I've browsed the messages here and have googled but don't see a clear
>> > solution yet. I have about 900 documents to which I need to apply a
>> > list of maybe 40 separate regex search and replaces. I would like to
>> > feed a list of the regex expressions and replacements to emacs and
>> > have it applied to a directory of the files. Any solutions? Keyboard
>> > macros don't seem like the right solution....
>>
>> Sounds like the best would be writing a small elisp function to do the job.
>
> I'll look into elisp...Struggled with perl to do this....

Well, not meaning to sound rude, but if you had trouble doing this with
perl, I suspect you will have even a harder time with elisp. I only say
this as, from your description, perl would be my first choice and it
should be pretty straight forward. I personally find perl regexp a lot
easier to work with and a lot more powerful than elisp's
implementation. After perl, my second choice would
likely be sed (actually, sed may be my first choice if you have quite a
number of regexp because you could put them all in a file and then just
run sed on the files - essentially, no programming. However, sed can be
a little tricky to wrap your head around as it has a hell of a lot of
power.


Tim

-- 
tcross (at) rapttech dot com dot au

Re: How to apply a list of regex replaces to multiple files?

[David Combs]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1970 bytes --]

In article <8763ron1dr.fsf@lion.rapttech.com.au>,
Tim X  <timx@nospam.dev.null> wrote:
>"michael.l" <michael.lommel@gmail.com> writes:
>
>> On Jul 1, 1:38 am, "Lennart Borgman (gmail)"
>> <lennart.borg...@gmail.com> wrote:
>>> michael.l wrote:
>>> > I've browsed the messages here and have googled but don't see a clear
>>> > solution yet. I have about 900 documents to which I need to apply a
>>> > list of maybe 40 separate regex search and replaces. I would like to
>>> > feed a list of the regex expressions and replacements to emacs and
>>> > have it applied to a directory of the files. Any solutions? Keyboard
>>> > macros don't seem like the right solution....
>>>
>>> Sounds like the best would be writing a small elisp function to do the job.
>>
>> I'll look into elisp...Struggled with perl to do this....
>
>Well, not meaning to sound rude, but if you had trouble doing this with
>perl, I suspect you will have even a harder time with elisp. I only say
>this as, from your description, perl would be my first choice and it
>should be pretty straight forward. I personally find perl regexp a lot
>easier to work with and a lot more powerful than elisp's
>implementation. After perl, my second choice would
>likely be sed (actually, sed may be my first choice if you have quite a
>number of regexp because you could put them all in a file and then just
>run sed on the files - essentially, no programming. However, sed can be
>a little tricky to wrap your head around as it has a hell of a lot of
>power.
>
>
>Tim
>
>-- 
>tcross (at) rapttech dot com dot au


What I used to do is use "ex" (part of vi) or maybe
it was "ed".

Anyway, it had a "source" stmt, ie "source myRegexAndSubstituteCmds.ex",
plus a sh-script

ex foo.bar
source <something>

ex foo2.bar
source <same thing>

etc



something like that.  Anyway, it worked, for changing variable-names in
about 100 program files, totally hands-off.

Back then perl likely didn't even exist.


David

Re: How to apply a list of regex replaces to multiple files?

[Xah]

On Jun 30, 6:50 pm, "michael.l" <michael.lom...@gmail.com> wrote:
>... I have about 900 documents to which I need to apply a
> list of maybe 40 separate regex search and replaces. I would like to
> feed a list of the regex expressions and replacements to emacs and
> have it applied to a directory of the files. Any solutions? Keyboard
> macros don't seem like the right solution....

if you want to do it in Python, here's the full script:

http://xahlee.org/perl-python/findreplace_multi_pairs.html

if you want to do it in Perl, here's the full script:

http://xahlee.org/perl-python/find_replace_perl.html

If you want to do it in emacs, it's even easier.

If you only have 1 single find-replace pair, then it's is build in.
Just go to dired, mark the files, type Q (which calls dired-do-query-
replace-regexp). Aften done, type ibuffer then type “* U” to mark all
unsaved, then “S” to save them all, type “D” to close all opened
files. For some full detailed tutorial, see
 http://xahlee.org/emacs/find_replace_inter.html

If you have multiple pairs of find-replace, then you need a script.
Like this:

; open a file, process it, save, close it
(defun my-process-file (fpath)
  "process the file at fullpath fpath ..."
  (let (mybuffer)
    (setq mybuffer (find-file fpath))

    (goto-char (point-min))
    (while (search-forward-regexp "myStr1" nil t)
      (replace-match "myReplaceStr2"))

    (goto-char (point-min))
    (while (search-forward-regexp "myStr2" nil t)
      (replace-match "myReplaceStr2"))

;; ... more find-replace pairs

    (save-buffer)
    (kill-buffer mybuffer)))

;; and suppose you want to do this to all html files in a dir:
(require 'find-lisp)
(mapc 'my-process-file (find-lisp-find-files "~/web/emacs/" "\\.html
$"))

Save the above in a file “process-files.el”, then you can call it
either by eval-buffer or from command line “emacs --script process-
files.el”.

The beauty with elisp for text prcoessing is that many things are
buildin. i.e. backup, proper file decoding, file meta-data maintaince,
file saving, etc, and you dont have to code for interactivity since
elisp runs interactively in emacs. For example, if you need to do find-
replace by case-by-case basis with human eyeball, you can wrap the
following to each of the replace-match sexp above.

(when (y-or-n-p)
;; put the (replace-match ...) code here
)

For some explanation of the code, see:
 http://xahlee.org/emacs/elisp_text_processing.html

  Xah
∑ http://xahlee.org/

☄