Lookarounds and recursion in Emacs regexes

Lookarounds and recursion in Emacs regexes

[Evan Aad]

Are there plans to extend Emacs regexes with lookarounds? With
recursion? If so, when are these enhancements expected?

Other text editors, such as Notepad++, EditPad, and Sublime Text
natively use regex flavors that support lookarounds and recursion. The
same applies to popular programming languages such as C#, Java, Perl,
and Python.

Compared with platforms whose regexes support lookarounds and
recursion, Emacs regexes have limited power, expressivity, and ease of
use.

Re: Lookarounds and recursion in Emacs regexes

[Marcin Borkowski]

On 2023-01-27, at 15:11, Evan Aad <oddeveneven@gmail.com> wrote:

> Are there plans to extend Emacs regexes with lookarounds? With
> recursion? If so, when are these enhancements expected?
>
> Other text editors, such as Notepad++, EditPad, and Sublime Text
> natively use regex flavors that support lookarounds and recursion. The
> same applies to popular programming languages such as C#, Java, Perl,
> and Python.
>
> Compared with platforms whose regexes support lookarounds and
> recursion, Emacs regexes have limited power, expressivity, and ease of
> use.

True, but what they don't have is Elisp;-), which makes up for the
limited regexen.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Lookarounds and recursion in Emacs regexes

[Emanuel Berg]

Marcin Borkowski wrote:

> True, but what they don't have is Elisp ;-), which makes up
> for the limited regexen.

But we want everything!

OP: Do it yourself?

[re: Elisp, I heard GIMP once used Lisp as its
 configuration/extension language, only at some point switched
 to Python for that. That's not gonna happen to Emacs, is it?
 *runs away*]

-- 
underground experts united
https://dataswamp.org/~incal

Re: Lookarounds and recursion in Emacs regexes

[tomas]

[-- Attachment #1: Type: text/plain, Size: 664 bytes --]

On Fri, Jan 27, 2023 at 08:30:56PM +0100, Emanuel Berg wrote:

> [... I heard GIMP once used Lisp as its
>  configuration/extension language, only at some point switched
>  to Python for that. That's not gonna happen to Emacs, is it?
>  *runs away*]

Actually a Scheme [0]. And actually, it's still in, besides Python,
Perl and Tcl [1]. Of course, I do prefer script-fu. Although somewhat
proficient, I never learnt to love Python.

Cheers
[0] I'm nearly certain it was derived from SIOD; the Wikipedia below
    says it's Tiny Scheme. Dunno which one is right.
[1] https://en.wikipedia.org/wiki/Script-fu#Automation,_scripts_and_plug-ins

-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Lookarounds and recursion in Emacs regexes

[Emanuel Berg]

tomas wrote:

>> I heard GIMP once used Lisp as its configuration/extension
>> language, only at some point switched to Python for that.
>> That's not gonna happen to Emacs, is it? *runs away*
>
> Actually a Scheme. And actually, it's still in, besides
> Python, Perl and Tcl. Of course, I do prefer script-fu.

I'm not following, what do you mean?

> Although somewhat proficient, I never learnt to love Python.

People don't love Python like they do Lisp, but no doubt it
has it's good sides - development speed not the least.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Lookarounds and recursion in Emacs regexes

[Jean Louis]

* Emanuel Berg <incal@dataswamp.org> [2023-02-04 17:54]:
> > Although somewhat proficient, I never learnt to love Python.
> 
> People don't love Python like they do Lisp, but no doubt it
> has it's good sides - development speed not the least.

Do you want to say that development speed in Lisp is slower than in
Python?

I think that development speed does not depend too much on the
language. 

One important issue for speed is functional programming, making one
function after the other. Python has it, Lisp has it.

But different paradigms are possible.

So when I program with small functions each doing something, then you
start doing higher level functions, and after some work done, it
becomes really speedy to make what you want.

Download the book of Paul Graham - On Lisp:
http://www.paulgraham.com/onlisptext.html

Ordering things in libraries, preparing functions in such way to be
useful to programmer, and having one's own utilities library, all that
contributes to really speedy programming.

69727 entries I have recently imported from Gutenberg free library,
the index of all the books. Now I can speedily search on my computer
for any works.

Let us say something about "Sweden", I get 119 results. Or something
like:

The Wanderings of Persiles and Sigismunda: A Northern Story
https://gutenberg.org/ebooks/61561

And I can quickly, within a second produce such hyperlinks, or just
press a key and come to the web page of that book to download it, or
to read it straight.

Few functions below are re-using previously made functions, and
multiple Emacs libraries.

Application usage can be seen here:
https://gnu.support/images/2023/02/2023-02-04/2023-02-04-16:41:08.ogv

And it is made on top of previous functions, by using these here below:

(defun rcd-db-gutenberg-search (&optional prefix query)
  (interactive "p")
  (let* ((query (or query (rcd-region-string) (rcd-ask-get "Query by name: ")))
	 (logic (cond (current-prefix-arg "OR") 
		      (t "AND")))
	 (title (rcd-sql-search-snippet-for-and-column 
		 "gutenberg_title" query nil logic))
	 (authors (rcd-sql-search-snippet-for-and-column 
		   "gutenberg_authors" query nil logic))
	 (subjects (rcd-sql-search-snippet-for-and-column 
		    "gutenberg_subjects" query nil logic))
	 (sql (format "SELECT gutenberg_id, gutenberg_title
                        FROM gutenberg
                       WHERE %s OR %s OR %s"
		      title authors subjects))
	 (title (format "Gutenberg entries for `%s'" (string-join (split-string query (concat " " logic " "))))))
    (rcd-db-sql-report title sql [("ID" 6 t) ("Name" 50 t)] "gutenberg" nil 
		       (lambda () 
			 (interactive)
			 (rcd-db-gutenberg-search prefix query)))))
	
(defun rcd-db-gutenberg-browse-url-1 (id)
  (format "https://gutenberg.org/ebooks/%d" id))

(defun rcd-db-gutenberg-read-html-images-1 (id)
  (format "https://gutenberg.org/ebooks/%d.html.images" id))

(defun rcd-db-gutenberg-browse-url (&optional id)
  "Browse URL for Gutenberg entry."
  (interactive nil rcd-db-list-mode)
  (when-tabulated-id "gutenberg"
      (let ((url (rcd-db-gutenberg-browse-url-1 id)))
	(browse-url url))))

(defun rcd-db-gutenberg-read-html-images (&optional id)
  "Read HTML with images for Gutenberg entry."
  (interactive nil rcd-db-list-mode)
  (when-tabulated-id "gutenberg"
      (let ((url (rcd-db-gutenberg-read-html-images-1 id)))
	(browse-url url))))

I have no idea of Python, but I am sure that same can be done with
Python in speedy time provided programmer re-uses existing libraries. 

Familiarity is what makes "speed" as well, user familiar with
programming language will be speedy as compared to one not as
familiar. 

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Lookarounds and recursion in Emacs regexes

[Emanuel Berg]

Jean Louis wrote:

>>> Although somewhat proficient, I never learnt to love Python.
>> 
>> People don't love Python like they do Lisp, but no doubt it
>> has it's good sides - development speed not the least.
>
> Do you want to say that development speed in Lisp is slower
> than in Python?

Lisp is a family of languages, if we talk Elisp then Elisp is
faster for anything Emacs related obviously, if we talk
everything else then Python is faster.

If we talk Common Lisp vs Python, then Python is, in
general, faster. We then consider the languages themselves,
the technology around, but also the huge spread of Python
while Lisp is either a fringe language or - actually that's
our common ground - the underground.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Lookarounds and recursion in Emacs regexes

[Jean Louis]

* Emanuel Berg <incal@dataswamp.org> [2023-02-05 16:15]:
> Jean Louis wrote:
> 
> >>> Although somewhat proficient, I never learnt to love Python.
> >> 
> >> People don't love Python like they do Lisp, but no doubt it
> >> has it's good sides - development speed not the least.
> >
> > Do you want to say that development speed in Lisp is slower
> > than in Python?
> 
> Lisp is a family of languages, if we talk Elisp then Elisp is
> faster for anything Emacs related obviously, if we talk
> everything else then Python is faster.

Faster for development?

Faster for speed processing by programming language?

Subject is development, not speed.

I do not know if it exists for Python, but for Emacs Lisp, all
references exists within Emacs. That helps for speed of development.

I cannot know for Python how can I see definition of the function, is
there any way to see that?

> If we talk Common Lisp vs Python, then Python is, in
> general, faster.

For Common Lisp I can access all functions and their definitions from
within the Common Lisp itself.

Great design!

For Python, I see that many things are not integrated in Emacs, and
getting symbol descriptions, functions, information, it is not easy,
and is error prone. I installed `jedi' package, but I see I get not
conclusive error messages and I cannot get information for Python
functions.

User is disabled by design.

That Python development would be faster, I can't say for Emacs editor.

Regarding language itself, maybe you could tell "why" do you consider
development with Python faster?

> We then consider the languages themselves, the technology around,
> but also the huge spread of Python while Lisp is either a fringe
> language or - actually that's our common ground - the underground.

You have not explained specifics. I cannot get you. I get opinion, but
not specific.

I gave you one specific on developing Emacs Lisp in Emacs, versus
developing Python, which function descriptions are not easily
accessible. 

Developing Emacs Lisp in Vim would become harder for that reason.

Editor is one important aspect of it.

"Huge spread" of Python is indication of something, I do not know what
you mean with it. Maybe you mean that number of people knowing Python
would be helpful in development? I can understand it from there.

If language is "fringe", I cannot see how that influences development,
apart from maybe not having other people to exchange with them.

For example, I could easily program in this programming language, much
"fringe", and I could find all references, books, just anything:

Icon (programming language) - Wikipedia:
https://en.wikipedia.org/wiki/Icon_(programming_language)

What are Icon's distinguishing characteristics:
https://www2.cs.arizona.edu/icon/uguide/faq.htm#features

I just guess I would have no problems with that one and speedy
development, it is for reason of being well documented.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Lookarounds and recursion in Emacs regexes

[Emanuel Berg]

Jean Louis wrote:

>>>>> Although somewhat proficient, I never learnt to
>>>>> love Python.
>>>> 
>>>> People don't love Python like they do Lisp, but no doubt
>>>> it has it's good sides - development speed not the least.
>>>
>>> Do you want to say that development speed in Lisp is
>>> slower than in Python?
>> 
>> Lisp is a family of languages, if we talk Elisp then Elisp
>> is faster for anything Emacs related obviously, if we talk
>> everything else then Python is faster.
>
> Faster for development?

Yes, a less skilled Python programmer can do a more advanced
application in less time ...

> Faster for speed processing by programming language?

Actually then Lisp is, or can be, much faster, as Lisp can be
as fast as any compiled language.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Lookarounds and recursion in Emacs regexes

[Jean Louis]

* Emanuel Berg <incal@dataswamp.org> [2023-02-07 15:41]:
> Jean Louis wrote:
> 
> >>>>> Although somewhat proficient, I never learnt to
> >>>>> love Python.
> >>>> 
> >>>> People don't love Python like they do Lisp, but no doubt
> >>>> it has it's good sides - development speed not the least.
> >>>
> >>> Do you want to say that development speed in Lisp is
> >>> slower than in Python?
> >> 
> >> Lisp is a family of languages, if we talk Elisp then Elisp
> >> is faster for anything Emacs related obviously, if we talk
> >> everything else then Python is faster.
> >
> > Faster for development?
> 
> Yes, a less skilled Python programmer can do a more advanced
> application in less time ...

OK, if you are the one less skilled Python programmer, could you help
me, I need something:

- do you know tabulated list mode?

- it provides ID of the line, so in Emacs I can specify key binding to
  execute function on the ID of the line

- tabulated list mode is similar to spreadsheet

- to start with it, I needed minutes, something like some coffee break
  period, the rest is history

- I have tried searching for GUI spreadsheet like widget in various
  programming languages, and I found it very complex.

Please show me how is that to be implemented in Python? If I get that
I would maybe like to liberate the Emacs version to some other GUI.

I give you example of usage:

- list all contacts as people names, with their people list
- click on single contact, get all phone numbers, emails, etc.
- click on single phone, send SMS
- mark phone, e-mail, XMPP username, write message once, send to all 
- search people by skill, mark, send email to all of them at once


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

the GLIMPs [GIMP Lisps] (was: Re: Lookarounds and recursion in Emacs regexes)

[Emanuel Berg]

tomas wrote:

> Actually a Scheme [0]. And actually, it's still in, besides
> Python, Perl and Tcl [1]. Of course, I do prefer script-fu.
> Although somewhat proficient, I never learnt to love Python.

Everyone says that ... my COMP-HIST automation [1] indicate
that this is about GIMP?

As Python (Gimp-Python) is now the main (only?) extension
language for GIMP - previously SIOD, TinyScheme, and
Script-Fu.

<incal> ,, hist Scheme, Perl, Tcl, SIOD, Python, TinyScheme, Script-Fu
  <sth> Scheme 1975 lexical Lisp. MIT, Guy Steele and Gerald Sussman [MacLisp]
  <sth> Perl 1987 Unix scripting language, Larry Wall
  <sth> Tcl 1988 ("tickle") interpreted, compact programming language
  <sth> SIOD 1988 Scheme In One Defun/Day, interpreted extension language
  <sth> Python 1991 interpreted OO. significant whitespace. name from UK show
  <sth> TinyScheme interpreted extension language, e.g. previously for GIMP
  <sth> Script-Fu 1998 GIMP extention language based on TinyScheme

[ Anyone knows when TinyScheme came? doesn't say here, or am
  I blind? https://en.wikipedia.org/wiki/TinyScheme ]

> I'm nearly certain it was derived from SIOD; the Wikipedia
> below says it's Tiny Scheme. Dunno which one is right.

Tomás, you are almost always right. Why don't you write
a book/booklet on computer history? If you do, I can edit it
in very neat LaTeX with diagrams and lists and stuff.
Including the last list, what is it called, the index?
But without the pretentious introduction (yuk).

[1] https://dataswamp.org/~incal/#sth
    https://dataswamp.org/~incal/sth/scripts/hist
    https://dataswamp.org/~incal/COMP-HIST

-- 
underground experts united
https://dataswamp.org/~incal

Re: the GLIMPs [GIMP Lisps] (was: Re: Lookarounds and recursion in Emacs regexes)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 385 bytes --]

On Sun, Feb 26, 2023 at 12:40:13PM +0100, Emanuel Berg wrote:
> tomas wrote:
> 
> > Actually a Scheme [0]. And actually, it's still in, besides
> > [...]

> Tomás, you are almost always right.

I'm just an old guy...

>                                      Why don't you write
> a book/booklet on computer history?

... but not very good at writing :)

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Lookarounds and recursion in Emacs regexes

[Stefan Monnier via Users list for the GNU Emacs text editor]

> [re: Elisp, I heard GIMP once used Lisp as its
>  configuration/extension language, only at some point switched
>  to Python for that. That's not gonna happen to Emacs, is it?
>  *runs away*]

There's a common confusion that ELisp is some kind of extension language
for Emacs.  That's not quite true.  ELisp is an *implementation*
language of Emacs (along with C for the other half).

It doesn't prevent adding other extension languages, nor even does it
necessarily prevent changing the implementation language to another one
in theory.

But in practice it has discouraged most attempts to provide
another language (except arguably for the "statically scoped dialect of
ELisp", because its interoperability with the original ELisp was
sufficiently smooth).


        Stefan

Re: Lookarounds and recursion in Emacs regexes

[Emanuel Berg]

Stefan Monnier via Users list for the GNU Emacs text editor wrote:

>> [re: Elisp, I heard GIMP once used Lisp as its
>>  configuration/extension language, only at some point switched
>>  to Python for that. That's not gonna happen to Emacs, is it?
>>  *runs away*]
>
> There's a common confusion that ELisp is some kind of
> extension language for Emacs. That's not quite true.
> ELisp is an *implementation* language of Emacs (along with
> C for the other half).

Okay, so Emacs is implementation, configuration _and_ extension.

C is to most implementation and that's it and unawarely so
probably most often, but with dynamic modules, it can be
extension as well, I suppose it could be configuration as well
but again in practice who does that and what benefit would
that be ...

Or am I wrong?

> It doesn't prevent adding other extension languages, nor
> even does it necessarily prevent changing the implementation
> language to another one in theory.

I know!

> But in practice it has discouraged most attempts to provide
> another language (except arguably for the "statically scoped
> dialect of ELisp", because its interoperability with the
> original ELisp was sufficiently smooth).

But didn't you have the idea of adding a third layer language,
don't remember what that idea was, the language was to be
created for this purpose so a brand new one was discussed,
maybe it stopped at discussions?

What would have been the idea with that?

People say Emacs Lisp is slow but to me it feels pretty fast?

But I think there are other problems - lack of standard
libraries - a lot of functions everywhere that do similar
things, do-something do-the-same-buffer do-the-same-region
do-the-same-no-properties etc. Too much looking up stuff all
the time is a problem. Also too long words all the time ...
reduce typing to increase development speed. so start -> beg,
`require' -> req, `expt' -> **, all that, also make all final
args of functions where nil makes sense optional so trailing
nils can be pruned, just reduce typing if one cares
about speed.

Its biggest problem? It's too sexy for it's shirt!

-- 
underground experts united
https://dataswamp.org/~incal

Re: Lookarounds and recursion in Emacs regexes

[Eli Zaretskii]

> Date: Sat, 04 Feb 2023 17:28:11 -0500
> From:  Stefan Monnier via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> 
> There's a common confusion that ELisp is some kind of extension language
> for Emacs.  That's not quite true.  ELisp is an *implementation*
> language of Emacs (along with C for the other half).

That depends on what one considers as "Emacs" and what as
"extensions".

And besides, given that Emacs extends itself, what exactly are the
significant and fundamental differences between "extension language"
and "implementation language"?


> It doesn't prevent adding other extension languages, nor even does it
> necessarily prevent changing the implementation language to another one
> in theory.

Indeed, but replacing ELisp would probably need to rewrite a lot of
internals currently implemented in C, since most, if not all, of them
are currently very tightly coupled with Lisp and assume the existence
and design/implementation of various aspects of the Lisp machine.

Re: Lookarounds and recursion in Emacs regexes

[Emanuel Berg]

Eli Zaretskii wrote:

> And besides, given that Emacs extends itself, what exactly
> are the significant and fundamental differences between
> "extension language" and "implementation language"?

Good point.

>> It doesn't prevent adding other extension languages, nor
>> even does it necessarily prevent changing the
>> implementation language to another one in theory.
>
> Indeed, but replacing ELisp would probably need to rewrite
> a lot of internals currently implemented in C, since most,
> if not all, of them are currently very tightly coupled with
> Lisp and assume the existence and design/implementation of
> various aspects of the Lisp machine.

I don't think anyone suggests replacing Elisp, and if one
added another language, one would have to find very good ways
to integrate it with everything else, and not just trivially
so with text insert and stuff like that, so optimally, one
wouldn't have to rewrite anything.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Lookarounds and recursion in Emacs regexes

[Emanuel Berg]

> I don't think anyone suggests replacing Elisp, and if one
> added another language, one would have to find very good
> ways to integrate it with everything else, and not just
> trivially so with text insert and stuff like that, so
> optimally, one wouldn't have to rewrite anything.

Maybe the org-mode people did stuff like this already BTW?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Lookarounds and recursion in Emacs regexes

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Mon, 06 Feb 2023 13:53:56 +0100
> 
> Eli Zaretskii wrote:
> 
> >> It doesn't prevent adding other extension languages, nor
> >> even does it necessarily prevent changing the
> >> implementation language to another one in theory.
> >
> > Indeed, but replacing ELisp would probably need to rewrite
> > a lot of internals currently implemented in C, since most,
> > if not all, of them are currently very tightly coupled with
> > Lisp and assume the existence and design/implementation of
> > various aspects of the Lisp machine.
> 
> I don't think anyone suggests replacing Elisp

Stefan did, in the part cited above.  he didn't suggest it, but he
said it's not prevented.  And I replied to that part.

Re: Lookarounds and recursion in Emacs regexes

[Emanuel Berg]

Eli Zaretskii wrote:

>>>> It doesn't prevent adding other extension languages, nor
>>>> even does it necessarily prevent changing the
>>>> implementation language to another one in theory.
>>>
>>> Indeed, but replacing ELisp would probably need to rewrite
>>> a lot of internals currently implemented in C, since most,
>>> if not all, of them are currently very tightly coupled
>>> with Lisp and assume the existence and
>>> design/implementation of various aspects of the
>>> Lisp machine.
>> 
>> I don't think anyone suggests replacing Elisp
>
> Stefan did, in the part cited above. he didn't suggest it,
> but he said it's not prevented. And I replied to that part.

It depends, I suppose as long as that's only part of
the reply?

-- 
underground experts united
https://dataswamp.org/~incal