From: "Miguel V. S. Frasson" <mvsfrasson@gmail.com>
To: Stefan Monnier <monnier@iro.umontreal.ca>
Cc: emacs-devel@gnu.org
Subject: Re: Improving regexp-opt
Date: Fri, 12 Apr 2019 12:40:48 -0300 [thread overview]
Message-ID: <CAARdmY04bv2V1xJ+-C4_+B2WuyyuYNFE=1G=SiUFL4EF4BL_eg@mail.gmail.com> (raw)
In-Reply-To: <CAARdmY0VxaC9LdACyv_uzGZ-9Sinj3BmknwFGR6dWvGk21gtkQ@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 4464 bytes --]
Hi
Adding some thoughts on why current fa implementation is so slow.
It strives to keep well nested trees in all steps, so it spends a lot of
time on branch compassion. This ensures well nested trees for the fa and
eases final fa to regexp conversion.
If we drop well-formness for fa and manage to convert an not well-formed fa
to regexp, the process would be very fast...
Regards
Miguel
Em sex, 12 de abr de 2019 12:06, Miguel V. S. Frasson <mvsfrasson@gmail.com>
escreveu:
> Dear Stefan and others
>
> Some time ago I suggested an improvement in regexp-opt, factoring
> similarities at the end of groups. At the end, Stefan wrote:
>
> Em sex, 8 de fev de 2019 às 01:48, Stefan Monnier
> <monnier@iro.umontreal.ca> escreveu:
> > A much better approach is to go for a real "regexp to NFA/DFA
> > conversion". The `lex.el` package is one such example, but it's very
> > inefficient (in terms of building the FA and in the size of the FA, not
> > in terms of running the FA).
>
> After some time, I had an idea of simplification by FA. The base idea
> is implement FA as "nodes" being lists of ARROWi and arrows being
> (CHAR . NODE). For example the initial FA for strings ("abd" "acd") is
>
> >1 --a--> 2 --b--> 3 --d--> 4 --epsilon--> nil
> | ^
> +---c--> 5 --d--> 6 --epsilon-----|
> and inplemented as
> (?a (?b (?d (0))) (?c (?d (0))))
> = ((?a . ((?b . ((?d . ((0 . nil)))))
> (?c . ((?d . ((0 . nil)))))))
>
> Note that nodes 3 and 5 are `equal'. Simplification is to make them `eq'.
>
> Also, there is simplification where a node is equal to a subset of a
> parent node, resulting is a ? construction.
> For example,
> +---f--> 9 ----------epsilon -------------\
> | v
> >1 --a--> 2 --b--> 3 --c--> 4 --f--> 5 --epsilon--> nil
> | ^
> +---d--> 6 --e---/
> Node 4 is equal to a subnode derived from 2, resulting on
> +---epsilon-------+
> | v
> >1 --a--> 2 --b--> 3 --c--> 4 --f--> 5 --epsilon--> nil
> | ^
> +---d--> 6 --e--> 7
>
> I made an inplementation, a patch to regexp-opt.el
>
> The pros:
> If the resulting strings "came from" a regexp that is splittable, the
> FA implementation always simplifies to it. In pratice, these are
> uncommun, and in most cases, the results are equivalent.
>
> The cons:
> The algorithm for FA seams to have greater computation complexity,
> takes about 20 times to compute in average.
>
> Example, the case on the
>
> (setq strings '("cond" "if" "when" "unless" "while"
> "let" "let*" "progn" "prog1" "prog2"
> "save-restriction" "save-excursion" "save-window-excursion"
> "save-current-buffer" "save-match-data"
> "catch" "throw" "unwind-protect" "condition-case"))
>
> (regexp-opt strings) ->
>
> "\\(?:c\\(?:atch\\|ond\\(?:ition-case\\)?\\)\\|if\\|let\\*?\\|prog[12n]\\|save-\\(?:current-buffer\\|excursion\\|match-data\\|\\(?:restrict\\|window-excurs\\)ion\\)\\|throw\\|un\\(?:less\\|wind-protect\\)\\|wh\\(?:en\\|ile\\)\\)"
>
> (regexp-opt2 strings) ->
>
> "\\(?:c\\(?:atch\\|ond\\(?:ition-case\\)?\\)\\|if\\|let\\*?\\|prog[12n]\\|save-\\(?:current-buffer\\|match-data\\|\\(?:restrict\\|\\(?:window-\\)?excurs\\)ion\\)\\|throw\\|un\\(?:less\\|wind-protect\\)\\|wh\\(?:en\\|ile\\)\\)"
>
> The difference is that FA algorithm
>
> BUT my version takes 70 times more time to compute.
>
> Example 2:
> (setq strings2 '("car" "cdr"
> "caar" "cadr" "cdar" "cddr"
> "caaar" "caadr" "cadar" "caddr"
> "cdaar" "cdadr" "cddar" "cdddr"))
>
> (regexp-opt strings2) ->
>
> "\\(?:c\\(?:\\(?:a\\(?:a[ad]\\|d[ad]\\|[ad]\\)\\|d\\(?:a[ad]\\|d[ad]\\|[ad]\\)\\|[ad]\\)r\\)\\)"
>
> (regexp-opt2 strings2) ->
> "\\(?:c[ad]\\(?:[ad][ad]?\\)?r\\)"
>
> FA is 7 times slower here.
>
> If this implementation is useful, I would like very much to contribute it.
>
> I actually have the other implementation from previous idea. It is
> faster than FA, same complexity of current regexp-opt, a bit slower of
> course, but I like this better.
>
> Best regards
>
> Miguel Frasson
>
> --
> Miguel Vinicius Santini Frasson
> mvsfrasson@gmail.com
>
[-- Attachment #2: Type: text/html, Size: 6040 bytes --]
next prev parent reply other threads:[~2019-04-12 15:40 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-07 16:41 Improving regexp-opt Miguel V. S. Frasson
2019-02-08 3:48 ` Stefan Monnier
2019-04-12 15:06 ` Miguel V. S. Frasson
2019-04-12 15:40 ` Miguel V. S. Frasson [this message]
2019-04-12 16:53 ` Stefan Monnier
2019-04-18 0:59 ` Miguel V. S. Frasson
2019-04-18 3:49 ` Stefan Monnier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAARdmY04bv2V1xJ+-C4_+B2WuyyuYNFE=1G=SiUFL4EF4BL_eg@mail.gmail.com' \
--to=mvsfrasson@gmail.com \
--cc=emacs-devel@gnu.org \
--cc=monnier@iro.umontreal.ca \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.