Issues in ia64

Issues in ia64

[Ying Luo]

Dear Guile Experts:

I met some problems during running guile in ia64. When the program are compiled
in debug mode, everything is okay except that when we run the scheme command
(call-with-current-continuation ...) ; but when we compile the program with
optimize flag, it will catch a crash after several gabage collectings. The
breakpoint locates in function scm_gc_mark(...) , which seemed the problem was
caused by the mis-allignment during guile's stack structure. (P.S.  The machine's
version is ia64-redhat-linux)

Would someone give me a hand to help me out? I searched quite a few days in codes
but without results.  :(

Thanks a lot!

Best Regards,
Pavane



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[Thien-Thi Nguyen]

   From: Ying Luo <yluo@synopsys.com>
   Date: Tue, 02 Jul 2002 08:37:44 +0800

   I met some problems during running guile in ia64. When the program
   are compiled in debug mode, everything is okay except that when we
   run the scheme command (call-with-current-continuation ...) ;

i'd like to develop some test programs for call/cc (in C) that we can
run as part of "make check".  perhaps you have a fragment already that i
can use?

   but when we compile the program with optimize flag, it will catch a
   crash after several gabage collectings. The breakpoint locates in
   function scm_gc_mark(...) , which seemed the problem was caused by
   the mis-allignment during guile's stack structure. (P.S.  The
   machine's version is ia64-redhat-linux)

guile's type system holds an assumption:

 * This in turn means that cells must be aligned on a 8 byte boundary,
 * which is just right for two 32bit numbers (surprise, surprise).

alignment for 64 bit machines and such introduces to the specification
an MBZ field, let's call it machine-alignment-immediate-slack.  the
current code (libguile/struct.c:317) does not guarantee this:

  /* Adjust it even further so it's aligned on an eight-byte boundary.  */
  p = (scm_t_bits *) (((scm_t_bits) p + 7) & ~7);

it specifies the minimum and omits the slack.  ignoring slack altogether
we can probably reformulate purely using scm_t_bits, the result being:

  #define space (sizeof (scm_t_bits) * 2 - 1)
  p = (scm_t_bits *) (((scm_t_bits) p + space) & ~space);

this satisfies/implements the documented constraint directly.  it looks
like there are more of these types of changes required in struct.c (from
counting 7s), and possibly elsewhere.  and this is just for structs,
which are special-cased in practice.

   Would someone give me a hand to help me out? I searched quite a few
   days in codes but without results.  :(

it's a pretty cheesy suggestion, but: try the above fix first and see if
the problem goes away or is somehow statistically diminished.  in the
meantime, i'll go through struct.c and enforce this abstraction and will
post a diff when done.

thi

_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[Thien-Thi Nguyen]

see below for a diff that addresses this lack of slack in three places.

thi


_______________________________________________________
*** struct.c.~1.1.1.1.~	Thu Jun  6 02:36:37 2002
--- struct.c	Tue Jul  2 02:51:41 2002
***************
*** 59,64 ****
--- 59,77 ----
  #include <string.h>
  #endif
  
+ /* +------------------+----+-----+
+    | POINTER          | 0* | 000 |
+    +------------------+----+-----+
+                        ^^^^
+ 
+    This is the machine-alignment-immediate-slack field, which may have width 0
+    (but no less ;-), and Must Be Zero (MBZ).
+ 
+    For structs, we actually use two words.  */
+ 
+ #define two_word_size (sizeof (scm_t_bits) * 2)
+ #define space (two_word_size - 1)
+ 
  \f
  
  static SCM required_vtable_fields = SCM_BOOL_F;
***************
*** 306,319 ****
  scm_bits_t *
  scm_alloc_struct (int n_words, int n_extra, char *who)
  {
!   int size = sizeof (scm_bits_t) * (n_words + n_extra) + 7;
    void * block = scm_must_malloc (size, who);
  
    /* Adjust the pointer to hide the extra words.  */
    scm_bits_t * p = (scm_bits_t *) block + n_extra;
  
    /* Adjust it even further so it's aligned on an eight-byte boundary.  */
!   p = (scm_bits_t *) (((scm_bits_t) p + 7) & ~7);
  
    /* Initialize a few fields as described above.  */
    p[scm_struct_i_free] = (scm_bits_t) scm_struct_free_standard;
--- 319,332 ----
  scm_bits_t *
  scm_alloc_struct (int n_words, int n_extra, char *who)
  {
!   int size = sizeof (scm_bits_t) * (n_words + n_extra) + space;
    void * block = scm_must_malloc (size, who);
  
    /* Adjust the pointer to hide the extra words.  */
    scm_bits_t * p = (scm_bits_t *) block + n_extra;
  
    /* Adjust it even further so it's aligned on an eight-byte boundary.  */
!   p = (scm_bits_t *) (((scm_bits_t) p + space) & ~space);
  
    /* Initialize a few fields as described above.  */
    p[scm_struct_i_free] = (scm_bits_t) scm_struct_free_standard;
***************
*** 341,347 ****
  scm_struct_free_standard (scm_bits_t * vtable, scm_bits_t * data)
  {
    size_t n = (data[scm_struct_i_n_words] + scm_struct_n_extra_words)
! 	     * sizeof (scm_bits_t) + 7;
    scm_must_free ((void *) data[scm_struct_i_ptr]);
    return n;
  }
--- 354,360 ----
  scm_struct_free_standard (scm_bits_t * vtable, scm_bits_t * data)
  {
    size_t n = (data[scm_struct_i_n_words] + scm_struct_n_extra_words)
! 	     * sizeof (scm_bits_t) + space;
    scm_must_free ((void *) data[scm_struct_i_ptr]);
    return n;
  }
***************
*** 350,356 ****
  scm_struct_free_entity (scm_bits_t * vtable, scm_bits_t * data)
  {
    size_t n = (data[scm_struct_i_n_words] + scm_struct_entity_n_extra_words)
! 	     * sizeof (scm_bits_t) + 7;
    scm_must_free ((void *) data[scm_struct_i_ptr]);
    return n;
  }
--- 363,369 ----
  scm_struct_free_entity (scm_bits_t * vtable, scm_bits_t * data)
  {
    size_t n = (data[scm_struct_i_n_words] + scm_struct_entity_n_extra_words)
! 	     * sizeof (scm_bits_t) + space;
    scm_must_free ((void *) data[scm_struct_i_ptr]);
    return n;
  }

_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[Marius Vollmer]

Thien-Thi Nguyen <ttn@giblet.glug.org> writes:

> alignment for 64 bit machines and such introduces to the specification
> an MBZ field, let's call it machine-alignment-immediate-slack.

There are two kinds of minimum alignment involved: the one needed for
our tag system to work, and the one needed by the machine.

We only need 8-byte alignment for the tag system, even when scm_t_cell
is larger than 8 byte.  On machines with 64 bit pointers, where
scm_t_cell is 16 byte, we _could_ make use of the additional bit
afforded by aligning scm_t_cell:s on 16-byte boundaries, but we don't
need to.  Given that we want to support 32-bit machines in the
foreseeable future, it is probably easier to let the extra bit go to
waste than to use two different tagging schemes on 32-bit and 64-bit
platforms.  Thus, for our tagging scheme, we don't need to align cells
on 16 byte boundaries on 64-bit machines.

The alignment needed by the machine is a different issue, we need to
guarantee it no matter what.  The machine alignment is guaranteed by
malloc, and thus also by scm_gc_malloc, etc.  Since we are using the
memeory returned by malloc only as arrays of scm_t_bits and thus only
use offsets that are (in bytes) a whole multiple of
sizeof(scm_t_bits), we are not violating the alignment when creating
the strange 'struct' layout.

Or, with numbers: on a 32-bit platform the tag alignment is 8, the
machine alignment is at most 4, which gives a total alignment of 8.
On 64-bit platforms, the tag alignment is still 8, and the machine
alignment is at most 8, which gives again a total of 8.  So we are
safe.

This

>   /* Adjust it even further so it's aligned on an eight-byte boundary.  */
>   p = (scm_t_bits *) (((scm_t_bits) p + 7) & ~7);

is only there the satisfy the alignment for the tag system, which is 8
bytes, regardless of pointer size.

Therefore, I don't see the point of your patch.  Did I get this wrong?

_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[Thien-Thi Nguyen]

   From: Marius Vollmer <mvo@zagadka.ping.de>
   Date: 02 Jul 2002 21:01:25 +0200

   On 64-bit platforms, the tag alignment is still 8, and the machine
   alignment is at most 8, which gives again a total of 8.  So we are
   safe.

probably we should find a reliable way to find out machine pointer
alignment requirements in case it is not as we assume.  any assumptions
we have should be made into a test for "make check" (although you could
consider building guile such a test it's not very fine-grained).

in any case, there is this to play w/ now:

  http://www.glug.org/tmp/2002-07/guile-1.4.1.0.20020706.tar.gz

this is a pre-vacation snapshot/work-in-progress release, about halfway
though the 1.4.2 tasks:

  http://www.glug.org/snap/w/tasks/TODO

i'd appreciate it if people could bang on it (see if you can spot the
planted bugs ;-) for a few weeks so when i get back i can finish 1.4.2
w/ the learned feedback.

ciao for now,
thi

_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[Marius Vollmer]

Ying Luo <yluo@synopsys.com> writes:

> I met some problems during running guile in ia64.

What version of Guile did you use?

There is some stuff for IA64 in 1.5.x while 1.4 will probably not work.
You can get 1.5.6 from

    ftp://alpha.gnu.org/gnu/guile/

Also, there are snapshots of both development series in

    ftp://ftp.dt.e-technik.uni-dortmund.de/pub/guile/snapshots/

_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[Rob Browning]

Marius Vollmer <mvo@zagadka.ping.de> writes:

> What version of Guile did you use?
>
> There is some stuff for IA64 in 1.5.x while 1.4 will probably not work.
> You can get 1.5.6 from

After the last fixes I put in (a while back) I believe the 1.5 tree
was passing make check on both ia64 and the alpha, at least under then
current debian.  I haven't checked again in the past few weeks, though.

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[cr88192 sydney]

>
>Dear Guile Experts:
>
>I met some problems during running guile in ia64. When the program are 
>compiled
>in debug mode, everything is okay except that when we run the scheme 
>command
>(call-with-current-continuation ...) ; but when we compile the program with
>optimize flag, it will catch a crash after several gabage collectings. The
>breakpoint locates in function scm_gc_mark(...) , which seemed the problem 
>was
>caused by the mis-allignment during guile's stack structure. (P.S.  The 
>machine's
>version is ia64-redhat-linux)
>
>Would someone give me a hand to help me out? I searched quite a few days in 
>codes
>but without results.  :(
>
>Thanks a lot!
>
this sounds more like it is probably more of a compiler issue than a guile 
issue, as it varies based on compilation flags. it it were a guile error it 
would more likely manifest as an error that occures regardless of compiler 
options.

I may be wrong though...

misc: I don't really use guile myself as I maintain my own scheme 
implementation which I find easier to work with, though it is quite a bit 
more basic... guile however is far better for running code in userspace, but 
I find its external interface annoying and dislike that it has been 
"labotomized" in my experience when used for its intended purpose... also 
that it holds external posix dependancies (what I have looked at), thus I 
can't use it elsewhere (in an os kernel of mine...).
other than that it is pretty good.


_________________________________________________________________
Chat with friends online, try MSN Messenger: http://messenger.msn.com


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[Thien-Thi Nguyen]

   From: "cr88192 sydney" <cr88192@hotmail.com>
   Date: Tue, 02 Jul 2002 02:53:27 +0000

   it has been "labotomized" in my experience when used for its intended
   purpose... also that it holds external posix dependancies (what I
   have looked at), thus I can't use it elsewhere (in an os kernel of
   mine...).

posix can be autoloaded w/ some SMOP.

what brain-slicing do you refer to?

thi

_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[cr88192 sydney]

>
>posix can be autoloaded w/ some SMOP.
>
err?...

>what brain-slicing do you refer to?
>
what I had done with it would was using it as a lib, but all the "normal" 
extensions loaded in (ice-9, guile-gtk, ...). didn't work and couldn't be 
loaded without putting them in the current directory...

really though I like maintaining my own implementation anyways as I have 
quite a bit of control over the workings of the interpreter and I know the 
details of the system anyways...

I had also considered cool security features, but instead decided a 
different kind of security was more in order. at first I considered to use 
tag/predicate locking but then I came up with the idea that it would be 
easier just to run code in reduced environments (ie: an "unlet" form that 
can hide bindings, or by composing the top level environment from peices and 
making restrictions by which peices are included...).
ok, the real issue is that I don't know guile's workings in detail...

I have for a while been considering writting a compile system, I had written 
a general spec for the calling convention and how to handle precise 
gc/relocation/swizzling for the stack, if wanted I may be able to make the 
partial spec available. are there any real important issues for this?
in my spec call/cc is accomplished by copying the pages from the stack 
region into a continuation region and setting the real stack copy-on-write, 
is this good?

question: in guile anyways is it possible to supply your own mm and to 
inquire about the structure of objects? is there a spec on the structure of 
objects as well?...




_________________________________________________________________
MSN Photos is the easiest way to share and print your photos: 
http://photos.msn.com/support/worldwide.aspx


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[Thien-Thi Nguyen]

   From: "cr88192 sydney" <cr88192@hotmail.com>
   Date: Tue, 02 Jul 2002 21:18:08 +0000

   err?...

for example, getpwent binding (glue code) can be put into a shared
object library and then we can arrange for guile to load that library
automatically on first reference to `getpwent'.  all this is "can"
because no one has actually done it yet.  part of the problem is
settling on methodology to make this particular separation an instance
of the larger task of "supporting modular development".  everything can
be done by hand but that gets old...

   what I had done with it would was using it as a lib, but all the
   "normal" extensions loaded in (ice-9, guile-gtk, ...). didn't work
   and couldn't be loaded without putting them in the current
   directory...

did you load boot-9.scm?  this defines the module system, which is what
searches `%load-path' for those things.

   ok, the real issue is that I don't know guile's workings in detail...

i'm in the same boat (whatever level of detail i know, there's always a
deeper level).

   I have for a while been considering writting a compile system, I had
   written a general spec for the calling convention and how to handle
   precise gc/relocation/swizzling for the stack, if wanted I may be
   able to make the partial spec available. are there any real important
   issues for this?

important issues for compilation are everywhere.  important issues for
your spec are that it be complete, consistent and implementable.  less
clearly important (i.e., may be important to some people but not to you)
is that it interoperate w/ current designs, to aid experimentation.

   in my spec call/cc is accomplished by copying the pages from the
   stack region into a continuation region and setting the real stack
   copy-on-write, is this good?

i don't know enough about the particulars to answer this.

   question: in guile anyways is it possible to supply your own mm and
   to inquire about the structure of objects? is there a spec on the
   structure of objects as well?...

guile implements a mini language for specifying object layout in memory.
see libguile/struct.c `make-struct-layout'.  user-level docs are in the
manual under node `Structures':

 http://www.glug.org/docbits/guile-doc-out/html/guile-1.7.0/guile_25.html#SEC237        
                                               
i don't know what "your own memory management" means, precisely.
certainly guile abstracts malloc and free for its own use internally.

thi

_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Issues in ia64

[cr88192 sydney]

>
>for example, getpwent binding (glue code) can be put into a shared
>object library and then we can arrange for guile to load that library
>automatically on first reference to `getpwent'.  all this is "can"
>because no one has actually done it yet.  part of the problem is
>settling on methodology to make this particular separation an instance
>of the larger task of "supporting modular development".  everything can
>be done by hand but that gets old...
>
in my implementation (except for ports and a few other things), all 
interaction with the ouside world is done by "sa_*" functions, everything 
else is provided by the vm. ports sadly are tied into an external io 
subsystem (mostly), luckily that io subsystem is also part of my code base 
and is usually used with my vm. I still don't like this, but the only real 
alternative would be to strip off file interaction with the outside world or 
wrap the whole io subsystem with more sa_* functions (which would be 
annoying as my io subsystem is a little bulky and this could only provide a 
dumbed down interface, though that is all r5rs asks...).

>    what I had done with it would was using it as a lib, but all the
>    "normal" extensions loaded in (ice-9, guile-gtk, ...). didn't work
>    and couldn't be loaded without putting them in the current
>    directory...
>
>did you load boot-9.scm?  this defines the module system, which is what
>searches `%load-path' for those things.
>
oh, ok. I guess this would fix it.

>    ok, the real issue is that I don't know guile's workings in detail...
>
>i'm in the same boat (whatever level of detail i know, there's always a
>deeper level).
>
I know my vm, as I wrote the whole thing...

>    I have for a while been considering writting a compile system, I had
>    written a general spec for the calling convention and how to handle
>    precise gc/relocation/swizzling for the stack, if wanted I may be
>    able to make the partial spec available. are there any real important
>    issues for this?
>
>important issues for compilation are everywhere.  important issues for
>your spec are that it be complete, consistent and implementable.  less
>clearly important (i.e., may be important to some people but not to you)
>is that it interoperate w/ current designs, to aid experimentation.
>
well it should be all this (except maybe the last one). the spec also allows 
code to "break from the spec". though the spec can be broken it could be 
considered less safe (thus when dealing with certain parts of the stack the 
collector would need to be conservative and it would be pinned). my spec is 
fussy but it should not be impossible.
it will need to be redesigned for other archs though, but even then it will 
not be too much different. I think x86 is about the lower limit of usable 
registers (where only eax, ecx, and edx are defined as being usable for 
data...). I am considering changing the design to allow a "scratchpad" 
space, which would work like extended registers though data there would be 
conservative (requiring precise data to be on the stack...).

>    in my spec call/cc is accomplished by copying the pages from the
>    stack region into a continuation region and setting the real stack
>    copy-on-write, is this good?
>
>i don't know enough about the particulars to answer this.
>
ok then, seems good to me in cases where continuations are stack allocated.

>    question: in guile anyways is it possible to supply your own mm and
>    to inquire about the structure of objects? is there a spec on the
>    structure of objects as well?...
>
>guile implements a mini language for specifying object layout in memory.
>see libguile/struct.c `make-struct-layout'.  user-level docs are in the
>manual under node `Structures':
>
ok cool, is the whole object system done this way?
my object system is a little more ad-hoc though, but it needs a good 
cleaning...
>  
>http://www.glug.org/docbits/guile-doc-out/html/guile-1.7.0/guile_25.html#SEC237
>
>i don't know what "your own memory management" means, precisely.
>certainly guile abstracts malloc and free for its own use internally.
>
I mean, can you replace the cons/object allocator with your own functions, 
ie: so you could allocate from a store. actually mine does this but it is 
currently hard coded (though I later hope to allow user defined memory 
managment...).

sadly in my spec at least some external mm is needed, though probably for 
init of something like an os kernel a small pool could be used.

I have changed projects so sadly the compile system is not really 
needed/usable...




_________________________________________________________________
Send and receive Hotmail on your mobile device: http://mobile.msn.com


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user