From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ihor Radchenko Newsgroups: gmane.emacs.bugs Subject: bug#63225: Using char table-based finite-state machines as a replacement for re-search-forward (was: bug#63225: Compiling regexp patterns (and REGEXP_CACHE_SIZE in search.c)) Date: Tue, 09 May 2023 08:36:25 +0000 Message-ID: <87ednp296e.fsf@localhost> References: <63882A45-BD02-40D5-92FA-70175267BA3B@acm.org> <874jou7lsf.fsf@localhost> <37EED5F9-F1FE-46B6-B4FA-0B268B945123@gmail.com> <87wn1qqvj0.fsf@localhost> <34F4849A-CB39-4C96-9CC1-11ED723706DA@gmail.com> <87wn1psqny.fsf@localhost> <6DAF37F9-B236-4C33-8E30-0FCA47CCBCC5@gmail.com> <87zg6lfobh.fsf@localhost> <281B22C2-CD69-4495-A97C-E754446CA9A6@gmail.com> <87o7n1v1w3.fsf@localhost> <878E8D66-A548-42E6-B077-6068A8B131D8@gmail.com> <87ednvul22.fsf@localhost> <87y1m1oa01.fsf@localhost> <74CD5EF4-5424-40BA-8F80-D0FD89CB890F@gmail.com> <87zg6fjar6.fsf@localhost> <875y923964.fsf@localhost> <67AAB661-8B27-4D09-BF0D-6B76ABB54477@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="25056"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 63225@debbugs.gnu.org To: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue May 09 10:34:38 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pwInt-0006Kh-Sb for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 09 May 2023 10:34:38 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pwInN-0007m9-NW; Tue, 09 May 2023 04:34:05 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwInL-0007lz-6X for bug-gnu-emacs@gnu.org; Tue, 09 May 2023 04:34:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pwInK-0006O8-UK for bug-gnu-emacs@gnu.org; Tue, 09 May 2023 04:34:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pwInK-0003c5-DS for bug-gnu-emacs@gnu.org; Tue, 09 May 2023 04:34:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Ihor Radchenko Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 09 May 2023 08:34:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 63225 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 63225-submit@debbugs.gnu.org id=B63225.168362120013823 (code B ref 63225); Tue, 09 May 2023 08:34:02 +0000 Original-Received: (at 63225) by debbugs.gnu.org; 9 May 2023 08:33:20 +0000 Original-Received: from localhost ([127.0.0.1]:42361 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pwImd-0003at-Ng for submit@debbugs.gnu.org; Tue, 09 May 2023 04:33:20 -0400 Original-Received: from mout01.posteo.de ([185.67.36.65]:42549) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pwImc-0003ag-7M for 63225@debbugs.gnu.org; Tue, 09 May 2023 04:33:19 -0400 Original-Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id E8F3D240192 for <63225@debbugs.gnu.org>; Tue, 9 May 2023 10:33:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1683621191; bh=7pLCMG3Okis5kf8eJMbioJAZPiZeTUXwNG5057TTc/4=; h=From:To:Cc:Subject:Date:From; b=QT968cQC50A5TbS4tiIzKcXiOb2LDN9HTB6VszH/M8hD8k9pFD8Tuqtu+FbQUsW0m dJtLKuiewE2RJRsy8xdpSQHLBme7pKfHQS+psOXtAwm+LM1nezWxrDxFczk4JdmOGc Jt+3joLQdJ06qV9ZvLgNKuE97MisRePB/C+4XLK/LVLK9lwU6zZ6iaTwNXhqby+vSh 1mUpaBW/FE/kJRSeX4lCvno0+AwmsiB56+oRRRNvEHpsx60LE28y/vJZvpcJ87m/jM bIqIN5oqCjfe8Sf01lXqZKlGmztWbhERmftH/1NlAQ74AARUCqYSfiklkaM7wqoSdt xZJR+QLL/zg/g== Original-Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4QFryb0jvRz9rxB; Tue, 9 May 2023 10:33:11 +0200 (CEST) In-Reply-To: <67AAB661-8B27-4D09-BF0D-6B76ABB54477@gmail.com> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:261392 Archived-At: Mattias Engdeg=C3=A5rd writes: >> (I now start thinking that it might be more efficient to create a bunch >> or char tables and step over them to match "regexp", just like any >> finite automaton would) > > I wish elisp were fast enough that we could do that in general. I tried the following simple implementation for searching via char tables: (defun yant/make-char-re (string) "Create a regexp matching STRING using char tables. Return entry char table. The values are either char tables or return value = for terminals. This is a dumb version not trying to account for substring cycl= es." (let (root (table (make-char-table 'regexp-table)) (idx 0)) (setq root table) (set-char-table-range root t root) (seq-map (lambda (char) (let ((next-table (make-char-table 'regexp-table root))) (aset table char next-table) (setq table next-table))) string) (set-char-table-range table t t) root)) (defun yant/search-forward (regexp-table) "Move point after next string matching REGEXP-TABLE." (let ((pos (point-marker))) (while (and (< pos (point-max)) (char-table-p (setq regexp-table (aref regexp-table (char-after pos)))) (move-marker pos (1+ pos)))) (unless (char-table-p regexp-table) (goto-char pos)))) DEFUN ("auto-search-forward", Fauto_search_forward, Sauto_search_forward, 1= , 1, 0, doc: /* Search forward using CHAR-TABLE. (WIP) */) (Lisp_Object table) { CHECK_TYPE (CHAR_TABLE_P (table), Qchar_table_p, table); ptrdiff_t pos_byte =3D PT_BYTE; ptrdiff_t pos_char =3D PT; if (pos_byte < BEGV_BYTE || pos_byte >=3D ZV_BYTE) return Qnil; table =3D CHAR_TABLE_REF (table, FETCH_CHAR (pos_byte)); while (pos_byte < ZV_BYTE && CHAR_TABLE_P (table)) { pos_byte +=3D next_char_len(pos_byte); pos_char ++; table =3D CHAR_TABLE_REF (table, FETCH_CHAR (pos_byte)); } if (pos_byte >=3D ZV_BYTE) return Qnil; else { SET_PT (pos_char); return Qt; } } (setq yant/test (yant/make-char-re ":ID:")) (benchmark-run 10 (org-with-wide-buffer (goto-char 1) (while (yant/search-f= orward yant/test)))) (benchmark-run 10 (org-with-wide-buffer (goto-char 1) (while (auto-search-f= orward yant/test)))) (benchmark-run 10 (org-with-wide-buffer (goto-char (point-min)) (while (re-= search-forward ":ID:" nil t)))) - yant/search-forward :: (24.677463432 0 0.0) - re-search-forward :: (0.569693718 0 0.0) - auto-search-forward :: (1.177098129 0 0.0) The lisp version suffers from (1) Extra redirection when checking function symbol value; (2) moving markers, as there is no efficient way to move point forward from Elisp; `forward-char' is even slower. auto-search-forward is twice slower than re-search-forward, but I expect this to change if we ask for a more complex regexp. auto-search-forward will not care about the complexity of the finite-state machine provided. --=20 Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at