Emacs-like file buffers

Emacs-like file buffers

[Jason Meade]

Under the "Cool Ideas" section of the Guile website, Emacs-like
buffers are mentioned. I'm thinking this would be a really useful
library, even independent of Guile. Once such a library is built, then
extending it into guile shouldn't be too much extra effort.

Is anyone actively pursuing this? If so, I'd be interested in
assisting. If not, then I plan to give a shot at this independently.
:)

As a start I see a need for:

* In-memory string (including binary string?) buffers
* buffer->string (and vice-versa) features (including splitting & tokenizing)
* Point based search/replace/insert/delete/copy facilities
* Full regular expression support (this touches upon another wish-list item)
* File I/O encapsulation

If anyone has any additional info on this topic, then I'd like to hear about it.

Thanks

-Jason


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Emacs-like file buffers

[Rob Browning]

"Jason Meade" <jemeade@gmail.com> writes:

> Under the "Cool Ideas" section of the Guile website, Emacs-like
> buffers are mentioned. I'm thinking this would be a really useful
> library, even independent of Guile. Once such a library is built,
> then extending it into guile shouldn't be too much extra effort.
>
> Is anyone actively pursuing this? If so, I'd be interested in
> assisting. If not, then I plan to give a shot at this independently.
> :)

If there's still a chance that Guile might eventually be used by
Emacs, then it would probably make sense to consider both Emacs
buffers and Emacs regular expressions when trying to decide what to
do, and even if there isn't a chance, Emacs is still a significant
precedent.

Also, you might want to look at Thien-Thi Nguyen's ttn-pers-scheme to
see if "Editing Buffers" are relevant.  I don't know enough to comment
either way.

> * Full regular expression support (this touches upon another
>   wish-list item)

I would really like to see Guile provide a fixed regular expression
format, one that doesn't vary depending on what the build platform has
available.  Without that, you cannot write scripts that are portable,
at least not with respect to regular expressions.

Relevant questions:

  - How is the addition of support for Unicode likely to affect any
    regular expression or buffer work?

  - What well-known form of regular expressions should be chosen -
    POSIX, Emacs, Perl?

  - Is it acceptable to add an external dependency for regular
    expression suppport, or does the implementation need to be in
    Guile's source tree?  Note that the latter choice doesn't
    necessarily rule out using an existing library if, for example,
    the author of the library is willing to allow the inclusion of the
    code in Guile.

Hope this helps.
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org; previously @cs.utexas.edu
GPG starting 2002-11-03 = 14DD 432F AE39 534D B592  F9A0 25C8 D377 8C7E 73A4


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Emacs-like file buffers

[Kevin Ryde]

Rob Browning <rlb@defaultvalue.org> writes:
>
> I would really like to see Guile provide a fixed regular expression
> format, one that doesn't vary depending on what the build platform has
> available.  Without that, you cannot write scripts that are portable,
> at least not with respect to regular expressions.

If you assume you've got mostly posix then it's not too bad is it?

>   - How is the addition of support for Unicode likely to affect any
>     regular expression or buffer work?

You actually get utf8 or whatever matching already if you "setlocale"
into such a charset.  A bit of a trap if your strings are in one
charset and it's being interpreted in another.

>   - What well-known form of regular expressions should be chosen -
>     POSIX, Emacs, Perl?

The perl way has its fans.  I guess an add-on that let you get that
instead or as well as the basic posix bits might be cute.

>   - Is it acceptable to add an external dependency for regular
>     expression suppport,

It might have to be LGPL to maintain that status on the resulting
program.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Emacs-like file buffers

[Rob Browning]

Kevin Ryde <user42@zip.com.au> writes:

> Rob Browning <rlb@defaultvalue.org> writes:
>>
>> I would really like to see Guile provide a fixed regular expression
>> format, one that doesn't vary depending on what the build platform has
>> available.  Without that, you cannot write scripts that are portable,
>> at least not with respect to regular expressions.
>
> If you assume you've got mostly posix then it's not too bad is it?

My impression was that Guile just uses whatever library it finds on
the system (if any), and that the library found might or might not be
POSIX compliant.  If that's correct, then I'd much rather have an
arrangement where we know exactly what kind of regular expressions
Guile will provide on *any* host system.

As motivation, imagine you're using Guile to write a cross-platform
tool that will need to use regular expressions to build a list of
files to pass to the equivalent of "rm -rf".

>>     POSIX, Emacs, Perl?
>
> The perl way has its fans.  I guess an add-on that let you get that
> instead or as well as the basic posix bits might be cute.

A while back I actually hacked up a preliminary (use-modules (pcre)).
(I probably still have the source around here somewhere.)

>>   - Is it acceptable to add an external dependency for regular
>>     expression suppport,
>
> It might have to be LGPL to maintain that status on the resulting
> program.

Right, though the main concern I was trying to raise was that, as it
stands now, Guile has very few external dependencies, and so we should
probably consider carefully the merit of adding any given new
dependency.

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org; previously @cs.utexas.edu
GPG starting 2002-11-03 = 14DD 432F AE39 534D B592  F9A0 25C8 D377 8C7E 73A4


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Emacs-like file buffers

[Kevin Ryde]

Rob Browning <rlb@defaultvalue.org> writes:
>
>   - How is the addition of support for Unicode likely to affect any
>     regular expression or buffer work?

Oh, I see what you mean.  Yes, no doubt a utf-8 aware regexp library
would be needed.

To turn around what I said about glibc, glibc in the right setlocale
does it, but I think it'd depend on having and finding a utf-8 locale
that's been enabled in locale.gen (or wherever).  Switching back and
forward with setlocale would no doubt be as horrible as the same thing
we do with tzset, though when I measured it setlocale was pretty fast
(unlike tzset).

> My impression was that Guile just uses whatever library it finds on
> the system (if any), and that the library found might or might not be
> POSIX compliant.  If that's correct, then I'd much rather have an
> arrangement where we know exactly what kind of regular expressions
> Guile will provide on *any* host system.

I'd probably only go so far as help someone link to a decent library,
a "--with" option on configure or whatever.  Non-posix libc would
probably hurt many programs so you could argue guile needn't try too
hard to overcome it.

> A while back I actually hacked up a preliminary (use-modules (pcre)).
> (I probably still have the source around here somewhere.)

That sounds good as an add-on.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Emacs-like file buffers

[Rob Browning]

Kevin Ryde <user42@zip.com.au> writes:

> Rob Browning <rlb@defaultvalue.org> writes:

>> My impression was that Guile just uses whatever library it finds on
>> the system (if any), and that the library found might or might not be
>> POSIX compliant.  If that's correct, then I'd much rather have an
>> arrangement where we know exactly what kind of regular expressions
>> Guile will provide on *any* host system.
>
> I'd probably only go so far as help someone link to a decent library,
> a "--with" option on configure or whatever.  Non-posix libc would
> probably hurt many programs so you could argue guile needn't try too
> hard to overcome it.

I still think there's quite a lot to be said for the fact that in Perl
(and I assume Python) you can write the equivalent of

    (if (regex-match FOO filename)
      (rm-rf filename))

and know for *sure* that according to the official documentation
(barring an implementation bug) you will get the exact same behavior
on every supported platform.

Frankly, without some guarantee like that, I would think twice about
trying to write regular expression based, cross-platform scripts (in
particular, destructive programs) using Guile.

Of course if we could add a mandatory, full POSIX regular expression
test suite to make check (whenever regular expressions are available),
then that, along with suitable documentation, would be one way to
solve the problem.

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org; previously @cs.utexas.edu
GPG starting 2002-11-03 = 14DD 432F AE39 534D B592  F9A0 25C8 D377 8C7E 73A4


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Fwd: Emacs-like file buffers

[Jason Meade]

On 5/13/06, Rob Browning <rlb@defaultvalue.org> wrote:
> "Jason Meade" <jemeade@gmail.com> writes:
>
> > Since Perl's source is available, why not just grab its regex engine
> > and drop it into guile? If that's no good for licensing or other
> > reasons, then there's always PCRE (http://www.pcre.org/).
>
> Either of those would be a possibility, but first we'd have to come to
> a decision about whether or not we wanted Guile's official regular
> expressions to be Perl format.

...

>
> And of course there are also arguments in favor of the other formats
> (POSIX or Emacs).  I'm OK with any of them; I'd just like to have a
> standard.
>

~~ *sigh!* I forgot to cc the group... :( ~~

Is there a standard for regular expressions somewhere? It seems like
most implements (from grep through tcl, perl, and beyond) seem to
agree on simple stuff like ^[a-c]?002*[^b]+$ etc... (not all
understand ? and + I've noticed....) but very few seem to share an
affinity for \w or \< ... \> etcetera.

I don't use regular expressions (outside of grep) everyday, but when I
do use them, it's usually for some serious pattern matching, with
variable capture (via $1, or \#1 etc). I'm starting to incorporate
search-forward and search-backwards into my filebuffer lib. For that
kind of work, even strchr will work (sorta. binary data is a problem).
However, eventually I'll want to be able to regex my buffers, and once
I add the guile hooks, it would be nice to share the same regex syntax
with everyone else. After all, the last thing you want is to load a
module, only to find that it's not syntax compatible with the rest of
the language!

-Jason


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Fwd: Emacs-like file buffers

[Bruce Korb]

Jason Meade wrote:
> Is there a standard for regular expressions somewhere?

"a standard"?  No.  "are standards"?  Yes.  Many.  Naturally,
that is the problem.

> It seems like
> most implements (from grep through tcl, perl, and beyond) seem to
> agree on simple stuff like ^[a-c]?002*[^b]+$ etc... (not all
> understand ? and + I've noticed....) but very few seem to share an
> affinity for \w or \< ... \> etcetera.
> 
> I don't use regular expressions (outside of grep) everyday, but when I
> do use them, it's usually for some serious pattern matching, with
> variable capture (via $1, or \#1 etc). I'm starting to incorporate
> search-forward and search-backwards into my filebuffer lib. For that
> kind of work, even strchr will work (sorta. binary data is a problem).
> However, eventually I'll want to be able to regex my buffers, and once
> I add the guile hooks, it would be nice to share the same regex syntax
> with everyone else. After all, the last thing you want is to load a
> module, only to find that it's not syntax compatible with the rest of
> the language!

The most comprehensible and still useful thing would be to provide
a "regcomp" funtion that had a usable implementation of the REG_BASIC
flag and, perhaps, added such tweaks as REG_PERL et al.  The reason
I said, "usable implementation" is because in my implementation of
regexec(3C), REG_BASIC is not defined so there is no way to tell
the regex compiler that I want a basic re.  SO, into the guile.h
family of headers, add:

   #ifndef REG_BASIC
   #  define GUILE_REG_BASIC   0x10000
   #else
   #  define GUILE_REG_BASIC   REG_BASIC
   #endif

then overload "make-regexp" and "regexp-exec" to cope with this and
"I'd-rather-use-Perl-syntax" flags.  Or even a "I-want-fnmatch-
style-globbing" match flag.  It brings all the pattern matching
stuff together making it findable (very important in a huge interface)
and relatively easy to use (i.e. remember).  :)  That's my nickel.

Cheers - Bruce

P.S. one thing missing from the ice-9 regex functions seems to be a way
to pass such flags through to the implementing libguile functions.
Perhaps I just missed how to do it?

(define (string-match pattern str . args)
   (let ((rx (make-regexp pattern))
         (start (if (pair? args) (car args) 0)))
     (regexp-exec rx str start)))


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Fwd: Emacs-like file buffers

[Kevin Ryde]

Bruce Korb <bkorb@gnu.org> writes:
>
>   #ifndef REG_BASIC
>   #  define GUILE_REG_BASIC   0x10000
>   #else
>   #  define GUILE_REG_BASIC   REG_BASIC
>   #endif

Hmm.  Yes, if you're calling scm_make_regexp you need that or
something similar.  (To tell it not to force REG_EXTENDED by default.)

> P.S. one thing missing from the ice-9 regex functions seems to be a way
> to pass such flags through to the implementing libguile functions.
> Perhaps I just missed how to do it?
>
> (define (string-match pattern str . args)
>   (let ((rx (make-regexp pattern))
>         (start (if (pair? args) (car args) 0)))
>     (regexp-exec rx str start)))

Alas yes, you can't get compilation flags like regexp/newline through
with string-match.  I wanted that the other day and did

	(regexp-exec (make-regexp "..." regexp/newline) str)

Maybe string-match could separate comp flags vs exec flags among its
args, though I guess that'd depend on the two sets of constants being
distinct ...


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Fwd: Emacs-like file buffers

[Kevin Ryde]

"Jason Meade" <jemeade@gmail.com> writes:
>
> Is there a standard for regular expressions somewhere?

Posix at

    http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html

> an affinity for \w

I think in posix [[:word:]] (or whatever the syntax actually is), but
yes other stuff is pretty divergant, every program though it was a
good idea to add more bits.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Regular expressions

[Ludovic Courtès]

Hi,

"Jason Meade" <jemeade@gmail.com> writes:

> Is there a standard for regular expressions somewhere?

Some Scheme implementations (e.g., Bigloo [0]) use S-expressions to
represent regexps.  It has the advantage of being (i)
syntax-independent, (ii) easier to understand, and (iii) it could be
used as a back-end to a variety of different syntaxes.

Actually, Bigloo uses `pregexp' [1] which should work on any R5RS
implementation.  It uses the S-exp representation at its core but can
convert a "typical" regexp string into it.  The issue is that it's
unclear whether this syntax is compatible with POSIX, Perl, or only with
itself.  However, one could implement a converter from any particular
regexp syntax to S-exps.

In practice, `pregexp' on Guile may end up being much slower than the
available native implementations, though...

Thanks,
Ludovic.

[0] http://www-sop.inria.fr/mimosa/fp/Bigloo/doc/bigloo.pdf
[1] http://www.ccs.neu.edu/home/dorai/pregexp/pregexp.html


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel