Question about string-match and match-string

Question about string-match and match-string

[Suvayu Ali]

Hi,

I was writing a filter for Org mode and came across this confusion.  I
want to parse a string with LaTeX code and delete matching
sub-expressions.  As a test I tried this:

(let ((test "\\section{Heading{ignoreheading}}\nText\n"))
  (string-match "\\(\\\\\\\\section{.+{ignoreheading}}\\\\n\\)\\(.+\\)" test)
  (match-substitute-replacement "" t nil test 1)
  )

But evaluating the above I get the following backtrace.

Debugger entered--Lisp error: (args-out-of-range -1 -1)
  replace-match("" t nil "{" 1)
  match-substitute-replacement("" t nil "\\section{Heading{ignoreheading}}\nText\n" 1)
  (let ((test "\\section{Heading{ignoreheading}}\nText\n")) (string-match "\\(\\\\\\\\section{.+{ignoreheading}}\\\\n\\)\\(.+\\)" test) (match-substitute-replacement "" t nil test 1))
  eval((let ((test "\\section{Heading{ignoreheading}}\nText\n")) (string-match "\\(\\\\\\\\section{.+{ignoreheading}}\\\\n\\)\\(.+\\)" test) (match-substitute-replacement "" t nil test 1)) nil)
  eval-last-sexp-1(nil)
  eval-last-sexp(nil)
  call-interactively(eval-last-sexp nil nil)
  command-execute(eval-last-sexp)

I do not understand the error at all, since if I insert something like
(prin1 test) before the (match-substitute-replacement ..) in my example
code, I get the expected output (the value of test as bound by let).

Where am I going wrong?

Thank you for any guidance.

PS: I'm also not very clear why we need 4 \-s in the regex to match a
    single \ in the string.  I came up with it by trial and error with the
    regexp-builder.  Any explanation about that would also be great.

-- 
Suvayu

Open source is the future. It sets us free.

Re: Question about string-match and match-string

[Suvayu Ali]

On Wed, Jul 17, 2013 at 10:17:04AM +0200, Suvayu Ali wrote:
> 
> Debugger entered--Lisp error: (args-out-of-range -1 -1)
>   replace-match("" t nil "{" 1)

I forgot to mention, the string above, "{", keeps changing with every
subsequent evaluation!  I tried to extract the sub-expressions with
match-string with no luck.  I get nil!

(let ((test "\\section{Heading{ignoreheading}}\nText\n"))
  (string-match "\\(\\\\\\\\section{.+{ignoreheading}}\\\\n\\)\\(.+\\)" test)
  (match-string 1 test)
  )
nil

Thanks,

-- 
Suvayu

Open source is the future. It sets us free.

Re: Question about string-match and match-string

[Rasmus]

Suvayu Ali <fatkasuvayu+linux@gmail.com> writes:

> On Wed, Jul 17, 2013 at 10:17:04AM +0200, Suvayu Ali wrote:
>> 
>> Debugger entered--Lisp error: (args-out-of-range -1 -1)
>>   replace-match("" t nil "{" 1)
>
> I forgot to mention, the string above, "{", keeps changing with every
> subsequent evaluation!  I tried to extract the sub-expressions with
> match-string with no luck.  I get nil!
>
> (let ((test "\\section{Heading{ignoreheading}}\nText\n"))
>   (string-match "\\(\\\\\\\\section{.+{ignoreheading}}\\\\n\\)\\(.+\\)" test)
>   (match-string 1 test)
>   )

Are you just trying to implement an ignoreheading filter?

This is what I use.  I think Nicolas provided it in a post at some
point but I'm not sure.

  (defun rasmus/org-latex-ignore-heading (headline backend info)
     "Strip headline from HEADLINE if it has tag ignoreheading for
     certain headlines.  `info' is Ignored"
     (when (and (org-export-derived-backend-p backend 'latex 'html 'ascii)
                (string-match "\\`.*ignoreheading.*\n"
                              (downcase headline)))
       (replace-match "" nil nil headline)))

  (add-to-list 'org-export-filter-headline-functions
               'rasmus/org-latex-ignore-heading)


–Rasmus

-- 
Hvor meget poesi tror De kommer ud af et glas isvand?

Re: Question about string-match and match-string

[Suvayu Ali]

Hi Rasmus,

On Wed, Jul 17, 2013 at 10:44:45AM +0200, Rasmus wrote:
> Suvayu Ali <fatkasuvayu+linux@gmail.com> writes:
> 
> > On Wed, Jul 17, 2013 at 10:17:04AM +0200, Suvayu Ali wrote:
> >> 
> >> Debugger entered--Lisp error: (args-out-of-range -1 -1)
> >>   replace-match("" t nil "{" 1)
> >
> > I forgot to mention, the string above, "{", keeps changing with every
> > subsequent evaluation!  I tried to extract the sub-expressions with
> > match-string with no luck.  I get nil!
> >
> > (let ((test "\\section{Heading{ignoreheading}}\nText\n"))
> >   (string-match "\\(\\\\\\\\section{.+{ignoreheading}}\\\\n\\)\\(.+\\)" test)
> >   (match-string 1 test)
> >   )
> 
> Are you just trying to implement an ignoreheading filter?
> 

Yes I am!  :)

> This is what I use.  I think Nicolas provided it in a post at some
> point but I'm not sure.
> 
>   (defun rasmus/org-latex-ignore-heading (headline backend info)
>      "Strip headline from HEADLINE if it has tag ignoreheading for
>      certain headlines.  `info' is Ignored"
>      (when (and (org-export-derived-backend-p backend 'latex 'html 'ascii)
>                 (string-match "\\`.*ignoreheading.*\n"
>                               (downcase headline)))
>        (replace-match "" nil nil headline)))
> 
>   (add-to-list 'org-export-filter-headline-functions
>                'rasmus/org-latex-ignore-heading)

Thanks for the code, I will use it.  But I would still like to know what
I was doing wrong.

-- 
Suvayu

Open source is the future. It sets us free.

Re: Question about string-match and match-string

[Julien Cubizolles]

Rasmus <rasmus@gmx.us> writes:


>   (defun rasmus/org-latex-ignore-heading (headline backend info)
>      "Strip headline from HEADLINE if it has tag ignoreheading for
>      certain headlines.  `info' is Ignored"
>      (when (and (org-export-derived-backend-p backend 'latex 'html 'ascii)
>                 (string-match "\\`.*ignoreheading.*\n"
>                               (downcase headline)))
>        (replace-match "" nil nil headline)))
>
>   (add-to-list 'org-export-filter-headline-functions
>                'rasmus/org-latex-ignore-heading)

I'm also looking for a ignoreheading function and was looking forward to
using yours. However I get a (void-variable headline) error when
exporting to LaTeX.

Julien.

Re: Question about string-match and match-string

[Rasmus]

Julien Cubizolles <j.cubizolles@free.fr> writes:

> Rasmus <rasmus@gmx.us> writes:
>
>
>>  
>
> I'm also looking for a ignoreheading function and was looking forward to
> using yours.

As said, I'm pretty sure Nicholas posted it first!

>  However I get a (void-variable headline) error when exporting to
> LaTeX.


That's surprising.  First try the following, which I think will solve
the problem provided you're using org > 8 AND Emacs >24.4.    

;; add ignoreheading only after ox. 

(with-eval-after-load 'ox
  (defun rasmus/org-latex-ignore-heading (headline backend info)
    "Strip headline from HEADLINE if it has tag ignoreheading for
      certain headlines.  `info' is ignored"
    (when (and (org-export-derived-backend-p backend 'latex 'html 'ascii 'odt)
	       (string-match "\\`.*ignoreheading.*\n"
			     (downcase headline)))
      (replace-match "" nil nil headline)))  

  (add-to-list 'org-export-filter-headline-functions
	       'rasmus/org-latex-ignore-heading))

With Emacs < 24.4 load with

(eval-after-load 'ox
  '(progn 
     (defun ...)
     (add-to-list ...)
     ))

In general with-eval-after-load is a wonderful function that can
really speed up things around start.  The function is what you'd
guess: it stores the information until a certain file (a string) or
feature (a symbol) is loaded and then evals the body.

You can also just put a (require 'ox) before the defun. 



If the above ain't working follow this recipe and let me know when the
error occurs.

   1. start Emacs by issuing emacs -q in the terminal (don't load config)
   2. in the *scratch* buffer type (require 'org) (shouldn't be
      necessary, actually).
   3. Type M-x org-version RET and check that it's greater than 8.
      E.g. my output it:
         Org-mode version 8.0.3 (release_8.0.3-309-gabacff @
         /usr/share/emacs/site-lisp/org/)
   4. Load the filter above
   5. type C-x b test.org RET M-x org-mode RET
   6. input 
      * test :ignoreheading:
       test-text
   7. type C-x C-e lL

On my system this gives the desired output.  Can you reproduce?

-- 
May contains speling mistake

Re: Question about string-match and match-string

[Julien Cubizolles]

Rasmus <rasmus@gmx.us> writes:

> (with-eval-after-load 'ox
>   (defun rasmus/org-latex-ignore-heading (headline backend info)
>     "Strip headline from HEADLINE if it has tag ignoreheading for
>       certain headlines.  `info' is ignored"
>     (when (and (org-export-derived-backend-p backend 'latex 'html 'ascii 'odt)
> 	       (string-match "\\`.*ignoreheading.*\n"
> 			     (downcase headline)))
>       (replace-match "" nil nil headline)))  
>
>   (add-to-list 'org-export-filter-headline-functions
> 	       'rasmus/org-latex-ignore-heading))

It's working, thanks, but it's ignored if you use tags:nil option (which
is pretty common I guess). Consider the following 

--8<---------------cut here---------------start------------->8---
#+TITLE: Titre

* First Heading                                                         :tag:
blablabla
* Second Heading                                              :ignoreheading:
blablabla
--8<---------------cut here---------------end--------------->8---

The second heading is omitted and its text is present, just like I want
it. However, you get an (in my opinion) ugly \textsc{tag} in the first
heading (\section{First Heading\hfill{}\textsc{tag}})

If however you use

--8<---------------cut here---------------start------------->8---
#+TITLE: Titre
#+OPTIONS: tags:nil

* First Heading                                                         :tag:
blablabla
* Second Heading                                              :ignoreheading:
blablabla
--8<---------------cut here---------------end--------------->8---

Both the first and second heading are displayed (not what I want).

Finally, something weird with the title: if you don't define it with de
#+TITLE, it's set to name_of_thefile$_{\text{ignoreheading}}$...

Julien.

Re: Question about string-match and match-string

[Rasmus]

Julien Cubizolles <j.cubizolles@free.fr> writes:

> Rasmus <rasmus@gmx.us> writes:
>
>> (with-eval-after-load 'ox
>>   (defun rasmus/org-latex-ignore-heading (headline backend info)
>>     "Strip headline from HEADLINE if it has tag ignoreheading for
>>       certain headlines.  `info' is ignored"
>>     (when (and (org-export-derived-backend-p backend 'latex 'html 'ascii 'odt)
>> 	       (string-match "\\`.*ignoreheading.*\n"
>> 			     (downcase headline)))
>>       (replace-match "" nil nil headline)))  
>>
>>   (add-to-list 'org-export-filter-headline-functions
>> 	       'rasmus/org-latex-ignore-heading))
>
> It's working, thanks, but it's ignored if you use tags:nil option (which
> is pretty common I guess). 

Well. . .  If you never want tags perhaps the correct way is to never
have tag export e.g. via a filter or a derived backend. 

> If however you use
>
> #+TITLE: Titre
> #+OPTIONS: tags:nil
>
> * First Heading                                                         :tag:
> blablabla
> * Second Heading                                              :ignoreheading:
> blablabla
>
> Both the first and second heading are displayed (not what I want).

Try this then. . .  It's ugly.  Also, you can't easily do this with
html due to divs and tab of contents, manually numbering etc.  So
you'd need a derived backend.  I don't know about odt. 

It's an ugly hack.

  (defun rasmus/org-latex-ignore-heading (headline backend info)
    "Strip headline from HEADLINE if it has tag ignoreheading."
    (let* ((tags (org-element-property :tags
                                       (plist-get 
                                        (text-properties-at
                                         (- (string-match "\n" headline) 2) headline)
                                        :parent))))
      (when (and (member-ignore-case "ignoreheading" tags)
                 (org-export-derived-backend-p backend 'latex 'ascii))
        (string-match "\\`.*\n.*\n" headline)
        (replace-match "" nil nil headline))))

Alternatively, you can add hook removing lines before parsing which
would give nice results in all backend, but then this won't work:

* h1 :noexport:
* h2 :ignoreheading:
my protected section

If you never have such constructions this is a better solution.

(defun rasmus/remove-ignored-headlines-before-parse (backend)
  "Remove headlines with tag \"ignoreheading\" before parsing"
  (dolist (x 
           (org-element-map (org-element-parse-buffer 'headlines)
               'headline (lambda (h) (cons (org-element-property :tags h)
                                           (org-element-property :begin h)))))
    (save-excursion (when (member-ignore-case "ignoreheading" (car x))
                      (goto-char (cdr x)) (delete-region (line-beginning-position) (line-end-position))))))
(add-hook 'org-export-before-parsing-hook 'rasmus/remove-ignored-headlines-before-parse)


> Finally, something weird with the title: if you don't define it with de
> #+TITLE, it's set to name_of_thefile$_{\text{ignoreheading}}$...

I can't reproduce.

–Rasmus

-- 
A page of history is worth a volume of logic

Re: Question about string-match and match-string

[Rasmus]

Rasmus <rasmus@gmx.us> writes:

>
>   (defun rasmus/org-latex-ignore-heading (headline backend info)
>     "Strip headline from HEADLINE if it has tag ignoreheading."
>     (let* ((tags (org-element-property :tags
>                                        (plist-get 
>                                         (text-properties-at
>                                          (- (string-match "\n" headline) 2) headline)
>                                         :parent))))
>       (when (and (member-ignore-case "ignoreheading" tags)
>                  (org-export-derived-backend-p backend 'latex 'ascii))
>         (string-match "\\`.*\n.*\n" headline)
>         (replace-match "" nil nil headline))))

This one is better.  It also doesn't make sense to remove headlines
for plain text as they also has structure like numbers

  (defun rasmus/org-latex-ignore-heading (headline backend info)
    "Strip headline from HEADLINE if it has tag ignoreheading."
    (when (org-export-derived-backend-p backend 'latex)
      (let ((tags (org-element-property :tags
                                        (plist-get
                                         (text-properties-at
                                          (or (string-match ".?\\(\\\\hfill\\|}\n\\)" headline) 0) headline)
                                         :parent))))
        (when (and tags (member-ignore-case "ignoreheading" tags))
          (string-match "\\`.*\n.*\n" headline)
          (replace-match "" nil nil headline)))))

-- 
ツ

Re: Question about string-match and match-string

[Julien Cubizolles]

Rasmus <rasmus@gmx.us> writes:

> This one is better.

It's working fine, thanks a lot. There is still the strange
_\textsc{ignoreheading} added to the default title. It's not really
important since I always set the title.

>   (defun rasmus/org-latex-ignore-heading (headline backend info)
>     "Strip headline from HEADLINE if it has tag ignoreheading."
>     (when (org-export-derived-backend-p backend 'latex)
>       (let ((tags (org-element-property :tags
>                                         (plist-get
>                                          (text-properties-at
>                                           (or (string-match ".?\\(\\\\hfill\\|}\n\\)" headline) 0) headline)
>                                          :parent))))
>         (when (and tags (member-ignore-case "ignoreheading" tags))
>           (string-match "\\`.*\n.*\n" headline)
>           (replace-match "" nil nil headline)))))

Julien.

Re: Question about string-match and match-string

[Nicolas Richard]

Suvayu Ali <fatkasuvayu+linux@gmail.com> writes:
> I was writing a filter for Org mode and came across this confusion.  I
> want to parse a string with LaTeX code and delete matching
> sub-expressions.  As a test I tried this:
>
> (let ((test "\\section{Heading{ignoreheading}}\nText\n"))
>   (string-match "\\(\\\\\\\\section{.+{ignoreheading}}\\\\n\\)\\(.+\\)" test)
>   (match-substitute-replacement "" t nil test 1)
>   )

Easy : the string doesn't match.

(let ((test "\\section{Heading{ignoreheading}}\nText\n"))
  (if (string-match "\\(\\\\\\\\section{.+{ignoreheading}}\\\\n\\)\\(.+\\)" test)
      
      (match-substitute-replacement "" t nil test 1)
    "Error")) => "Error"

What you want is probably :
(let ((test "\\section{Heading{ignoreheading}}\nText\n"))
  (if (string-match "\\(\\\\section{.+{ignoreheading}}\n\\)\\(.+\\)" test)
      
      (match-substitute-replacement "" t nil test 1)
    "Error"))

Only \\\\ before "section", because elisp translates that into \\, which
is a regexp matching exactly one backslash char (and that's all there is
in your string, since \\ is translated to a single backslash).

\\\\n became \n, because the former would translate into the regexp \\n,
which matches a backslash followed by the character 'n', and you don't
have that in your string. Note that the dot doesn't match newlines by
default, so your group will not contain the final newline.

As a general rule, you should probably check if (string-match) worked
before using the match data ; otherwise it leads to unhelpful error
messages.

> PS: I'm also not very clear why we need 4 \-s in the regex to match a
>     single \ in the string.  I came up with it by trial and error with the
>     regexp-builder.  Any explanation about that would also be great.

I hope the above is an explanation of that. Just remember there are two
steps:
(i) emacs reads the string
(ii) the string is made into a regexp
so for matching on \, you  need the regexp \\, which is obtained by the
string "\\\\" (e.g. (insert "\\\\") inserts \\)

-- 
Nico.

Re: Question about string-match and match-string

[Suvayu Ali]

Hi Nicolas,

On Wed, Jul 17, 2013 at 10:53:14AM +0200, Nicolas Richard wrote:
> 
> > PS: I'm also not very clear why we need 4 \-s in the regex to match a
> >     single \ in the string.  I came up with it by trial and error with the
> >     regexp-builder.  Any explanation about that would also be great.
> 
> I hope the above is an explanation of that. Just remember there are two
> steps:
> (i) emacs reads the string
> (ii) the string is made into a regexp
> so for matching on \, you  need the regexp \\, which is obtained by the
> string "\\\\" (e.g. (insert "\\\\") inserts \\)

Thank you for the very clear explanation.  It was very helpful.  :)

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.