Sting abstraction 2

Sting abstraction 2

[Mike Gran]

Hi-

I've been working on the Unicode problem in Guile, and I had been too
ambitious.  I hacked it up pretty harsh and started getting errors that
were tough to debug.

Anyway, I backtracked a bit.  Today I pushed a new tree
(string_abstraction2) which should be the same as master except with all
unnecessary calls to scm_i_string_chars, scm_i_symbol_chars, and
scm_i_string_writable_chars removed.  I largely avoided other
unnecessary modifications.  It is still an 8-bit string build, but, now
the string internals have been confined to strings.c and strports.c,
with very few exceptions.

>From here, I'm more confident that I can fold in changes for unicode
without breakage.

Re: Sting abstraction 2

[Ludovic Courtès]

Hello!

Mike Gran <spk121@yahoo.com> writes:

> Anyway, I backtracked a bit.  Today I pushed a new tree
> (string_abstraction2) which should be the same as master except with all
> unnecessary calls to scm_i_string_chars, scm_i_symbol_chars, and
> scm_i_string_writable_chars removed.  I largely avoided other
> unnecessary modifications.  It is still an 8-bit string build, but, now
> the string internals have been confined to strings.c and strports.c,
> with very few exceptions.

How does it differ from the approach you took a while back (e.g.,
http://thread.gmane.org/gmane.lisp.guile.devel/8436)?

(The commit notification message is hard to follow because it contains
many commits in addition to the relevant ones.)

Thanks,
Ludo'.

Re: Sting abstraction 2

[Mike Gran]

> From: Ludovic Courtès <ludo@gnu.org>
> 
> Hello!
> 
> Mike Gran writes:
> 
> > Anyway, I backtracked a bit.  Today I pushed a new tree
> > (string_abstraction2) which should be the same as master except with all
> > unnecessary calls to scm_i_string_chars, scm_i_symbol_chars, and
> > scm_i_string_writable_chars removed.  I largely avoided other
> > unnecessary modifications.  It is still an 8-bit string build, but, now
> > the string internals have been confined to strings.c and strports.c,
> > with very few exceptions.
> 
> How does it differ from the approach you took a while back (e.g.,
> http://thread.gmane.org/gmane.lisp.guile.devel/8436)?
> 

It is exactly the same idea as before, but, more militantly applied.

In my private build of my first attempt, I'm basically done.  All the pieces are there.  But it is buggy as hell.  In the first pass, I tried to do too many things at once: ports, locales, R6RS escapes.  I started getting some errors that were tough to debug (double-frees, SIGSEGV).  Since valgrind and mcheck are useless here, I got frustrated.  So, what I'm doing now, basically, is just adding the patches I made before in a more logical order and testing along the way.

So, same patches, different order, better testing.  Just try to git 'er done.

The locale conversion will still happen at scm_lfwrite and scm_getc.

(Did I ever mention this backtrace tree pic?  http://www.lonelycactus.com/uploaded_images/test[1]-765536.PNG  It shows that for all the scripts the test suite, all of the calls to low-level read and write pass through those two functions.)

I have changed my opinion on one issue. I don't believe that Guile ports should have a specific encoding: they should just use the locale.  This is just pragmatism.  Guile ports and the default reader are annoying things to hack.  I am loathe to touch them more than is necessary. 

The R6RS ports have the nice transcoder idea.  It might be more fun to push port-specific encodings to that library.  

The one weird side-effect of doing the locale conversion at scm_getc and scm_lfwrite is that the ports' buffers -- including string ports --  contain locale-encoded data. 

Since all the pieces have already been coded once already, it should come together quickly.

-Mike

Re: Sting abstraction 2

[Ludovic Courtès]

Hey,

Mike Gran <spk121@yahoo.com> writes:

> So, same patches, different order, better testing.  Just try to git
> er done.

OK, thanks for the explanation.

> (Did I ever mention this backtrace tree
> pic?  http://www.lonelycactus.com/uploaded_images/test[1]-765536.PNG 
> It shows that for all the scripts the test suite, all of the calls to
> low-level read and write pass through those two functions.)

Nice.  I suppose you had breakpoints in GDB, captured the output of
"bt", frobbed it and fed it to `dot'?

> I have changed my opinion on one issue. I don't believe that Guile
> ports should have a specific encoding: they should just use the
> locale.  This is just pragmatism.  Guile ports and the default reader
> are annoying things to hack.  I am loathe to touch them more than is
> necessary. 
>
> The R6RS ports have the nice transcoder idea.  It might be more fun to
> push port-specific encodings to that library. 

Are you saying that we'd have an implementation of the R6 port API that
DTRT, whereas Guile's current API would remain encoding-oblivious?

Thanks!

Ludo'.

Re: Sting abstraction 2

[Mike Gran]

On Thu, 2009-05-21 at 01:32 +0200, Ludovic Courtès wrote:

> > (Did I ever mention this backtrace tree
> > pic?  http://www.lonelycactus.com/uploaded_images/test[1]-765536.PNG 
> > It shows that for all the scripts the test suite, all of the calls to
> > low-level read and write pass through those two functions.)
> 
> Nice.  I suppose you had breakpoints in GDB, captured the output of
> "bt", frobbed it and fed it to `dot'?
> 
Pretty much.

> > I have changed my opinion on one issue. I don't believe that Guile
> > ports should have a specific encoding: they should just use the
> > locale.  This is just pragmatism.  Guile ports and the default reader
> > are annoying things to hack.  I am loathe to touch them more than is
> > necessary. 
> >
> > The R6RS ports have the nice transcoder idea.  It might be more fun to
> > push port-specific encodings to that library. 
> 
> Are you saying that we'd have an implementation of the R6 port API that
> DTRT, whereas Guile's current API would remain encoding-oblivious?
> 

I am thinking that the setlocale character encoding becomes the encoding
for all legacy Guile port IO, and if you want to operate on a port that
has a different locale than your setlocale character encoding, you need
to do that through an R6RS port.

> Thanks!
> 
> Ludo'.

-Mike

Re: Sting abstraction 2

[Andy Wingo]

On Wed 20 May 2009 17:24, Mike Gran <spk121@yahoo.com> writes:

> I've been working on the Unicode problem in Guile, and I had been too
> ambitious.  I hacked it up pretty harsh and started getting errors that
> were tough to debug.
>
> Anyway, I backtracked a bit.

FWIW this has happened to me a number of times. Bite off something big,
understand the problem better, then come back and move from known-good
state to known-good state. Yay git! :)

Happy hacking,

Andy
-- 
http://wingolog.org/