Uniform vector lengths -- how long?

Uniform vector lengths -- how long?

[Rob Browning]

Right now the srfi-4 code stores the length of uniform vectors in
SCM_CELL_WORD_2, and sometimes retrieves these lengths directly into
an int:

  #define SCM_UVEC_LENGTH(u) (SCM_CELL_WORD_2(u))
  ...
  int i = SCM_UVEC_LENGTH (uvec);

which is probably wrong for architectures where sizeof(int) <
sizeof(long), and in all cases, limits array sizes to 2GB.  Other code
uses SCM_VALIDATE_INUM to check the lengths given to "make-FOOvector",
which limits the maximum array size even further.

So the thing I'm wondering about is how do we want our uniform vectors
to behave?  Is it OK for their lengths to be limited to a fixnum?
What if someone wants to allocate a multi-GB s8 array on a 64-bit
machine, or a much smaller, but still larger than fixnum array on a
32-bit machine?

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org; previously @cs.utexas.edu
GPG starting 2002-11-03 = 14DD 432F AE39 534D B592  F9A0 25C8 D377 8C7E 73A4


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Uniform vector lengths -- how long?

[Kevin Ryde]

Rob Browning <rlb@defaultvalue.org> writes:
>
> Is it OK for their lengths to be limited to a fixnum?

I'd think so.

> What if someone wants to allocate a multi-GB s8 array on a 64-bit
> machine,

Yes, lifting it from an int to a long (or a size_t) would ensure
64-bits can be fully used.  (Eg. on alpha, I think.)


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Uniform vector lengths -- how long?

[Rob Browning]

Kevin Ryde <user42@zip.com.au> writes:

>> What if someone wants to allocate a multi-GB s8 array on a 64-bit
>> machine,
>
> Yes, lifting it from an int to a long (or a size_t) would ensure
> 64-bits can be fully used.  (Eg. on alpha, I think.)

OK, so it sounds like you'd argue we stick with SCM_CELL_WORD_2 for
the length (which I believe can be treated as an unsigned long), and
then just be consistent in the rest of the code.  If so, then I
suppose we should use scm_num2ulong in make-FOOvector and friends,
rather then SCM_INUM.

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org; previously @cs.utexas.edu
GPG starting 2002-11-03 = 14DD 432F AE39 534D B592  F9A0 25C8 D377 8C7E 73A4


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Uniform vector lengths -- how long?

[Kevin Ryde]

Rob Browning <rlb@defaultvalue.org> writes:
>
> I suppose we should use scm_num2ulong in make-FOOvector and friends,
> rather then SCM_INUM.

I suppose, though personally I wouldn't feel any great urgency for
that.  A positive inum ends up only, what, 3 bits less than a ulong,
and a 512Mb array would be more than likely bigger than one could do
real work with on a 32-bit machine.

What is worth worrying about though would be making sure the "len *
uvec_sizes[type]" in make_uvec doesn't overflow, perhaps wrapping
around to a small size which is successfully malloced.  I guess
currently with a maximum 8 in uvec_sizes and maximum
MOST_POSITIVE_INUM for the count it's ok.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Uniform vector lengths -- how long?

[Rob Browning]

Kevin Ryde <user42@zip.com.au> writes:

> I suppose, though personally I wouldn't feel any great urgency for
> that.  A positive inum ends up only, what, 3 bits less than a ulong,
> and a 512Mb array would be more than likely bigger than one could do
> real work with on a 32-bit machine.
>
> What is worth worrying about though would be making sure the "len *
> uvec_sizes[type]" in make_uvec doesn't overflow, perhaps wrapping
> around to a small size which is successfully malloced.  I guess
> currently with a maximum 8 in uvec_sizes and maximum
> MOST_POSITIVE_INUM for the count it's ok.

OK, I've fixed both of these.  After talking to Marius, I reworked
srfi-4 to use size_t for the length, since (as he pointed out) that's
what you'll eventually pass to malloc, and we will just make the
assertion that size_t must fit in a cell.

I also added checking to make_uvec to avoid the possibility of
overflow with types bigger than 1 byte.  Although the current solution
depends on SIZE_MAX being defined, and we wondered if it's
sufficiently common.

  if (len > ((size_t) SIZE_MAX / uvec_sizes[type])) ...

(I'll commit after I have a chance to double check the work.)

Thanks
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org; previously @cs.utexas.edu
GPG starting 2002-11-03 = 14DD 432F AE39 534D B592  F9A0 25C8 D377 8C7E 73A4


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Uniform vector lengths -- how long?

[Kevin Ryde]

Rob Browning <rlb@defaultvalue.org> writes:
>
> SIZE_MAX being defined

I think, but don't quote me, that limits.h and SIZE_MAX are ANSI C,
and guile already demands that.

>   if (len > ((size_t) SIZE_MAX / uvec_sizes[type])) ...

Since uvec_sizes are all powers of 2, perhaps change to a
uvec_shift[type] array and use a shift instead of a division.  Ditto
shift instead of multiply for the len -> bytes.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: Uniform vector lengths -- how long?

[Rob Browning]

Kevin Ryde <user42@zip.com.au> writes:

> I think, but don't quote me, that limits.h and SIZE_MAX are ANSI C,
> and guile already demands that.
>
>>   if (len > ((size_t) SIZE_MAX / uvec_sizes[type])) ...
>
> Since uvec_sizes are all powers of 2, perhaps change to a
> uvec_shift[type] array and use a shift instead of a division.  Ditto
> shift instead of multiply for the len -> bytes.

Good suggestion.  I've changed it.

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org; previously @cs.utexas.edu
GPG starting 2002-11-03 = 14DD 432F AE39 534D B592  F9A0 25C8 D377 8C7E 73A4


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel