unsigned-int

unsigned-int

[Catonano]

I can't extract correct values from unsigned-int's

I can extract correct values from int, unsigned-short

but NOT form an unsigned-int

In that case the number that comes out is plainly wrong

This is how I extract a number from an int (and it works)

(bytevector-uint-ref (pointer->bytevector
          outcome-ptr (sizeof int)) 0
          (endianness big) 1) )

This is an unsigned-short (and it works)

(bytevector-uint-ref (pointer->bytevector
                   columns-ptr (sizeof unsigned-short)) 0
                   (endianness big) 1)

This is an unsigned-int and it DOESN'T work

(bytevector-uint-ref (pointer->bytevector
                    rows-ptr (sizeof unsigned-int)) 0
                   (endianness big) 1)

Thanks in advance

Re: unsigned-int

[Catonano]

The relevant code is here
https://gitlab.com/humanitiesNerd/guile-freexl/blob/master/freexl/common.scm

and the not working function is "freexl-worksheet-dimensions" on line 132

Re: unsigned-int

[Mark H Weaver]

Catonano <catonano@gmail.com> writes:

> I can't extract correct values from unsigned-int's
>
> I can extract correct values from int, unsigned-short
>
> but NOT form an unsigned-int
>
> In that case the number that comes out is plainly wrong
>
> This is how I extract a number from an int (and it works)
>
> (bytevector-uint-ref (pointer->bytevector
>           outcome-ptr (sizeof int)) 0
>           (endianness big) 1) )

You need to use 'bytevector-int-ref', not 'bytevector-uint-ref', to
extract a signed integer.

> This is an unsigned-short (and it works)
>
> (bytevector-uint-ref (pointer->bytevector
>                    columns-ptr (sizeof unsigned-short)) 0
>                    (endianness big) 1)
>
> This is an unsigned-int and it DOESN'T work
>
> (bytevector-uint-ref (pointer->bytevector
>                     rows-ptr (sizeof unsigned-int)) 0
>                    (endianness big) 1)

There's also a problem with all three of your examples above.  You're
passing '1' as the final argument to 'bytevector-uint-ref'.  That's the
width in bytes of the numeric field to access.  In all cases, those 1s
should be replaced with (sizeof <type>).

Also, I'm not sure why you're specifying (endianness big) here.  I would
think (native-endianness) would be appropriate here.  Given this, and
the fact that you're passing the wrong width, makes me surprised that
this is working for you at all.

       Mark

Re: unsigned-int

[Catonano]

2017-06-22 18:20 GMT+02:00 Mark H Weaver <mhw@netris.org>:

> Catonano <catonano@gmail.com> writes:
>
> > I can't extract correct values from unsigned-int's
> >
> > I can extract correct values from int, unsigned-short
> >
> > but NOT form an unsigned-int
> >
> > In that case the number that comes out is plainly wrong
> >
> > This is how I extract a number from an int (and it works)
> >
> > (bytevector-uint-ref (pointer->bytevector
> >           outcome-ptr (sizeof int)) 0
> >           (endianness big) 1) )
>
> You need to use 'bytevector-int-ref', not 'bytevector-uint-ref', to
> extract a signed integer.
>

I am extracting an unsigned-int here.
(sizeof int) and (sizeof unsigned-int) happen to be te same


>
> > This is an unsigned-short (and it works)
> >
> > (bytevector-uint-ref (pointer->bytevector
> >                    columns-ptr (sizeof unsigned-short)) 0
> >                    (endianness big) 1)
> >
> > This is an unsigned-int and it DOESN'T work
> >
> > (bytevector-uint-ref (pointer->bytevector
> >                     rows-ptr (sizeof unsigned-int)) 0
> >                    (endianness big) 1)
>
> There's also a problem with all three of your examples above.  You're
> passing '1' as the final argument to 'bytevector-uint-ref'.  That's the
> width in bytes of the numeric field to access.  In all cases, those 1s
> should be replaced with (sizeof <type>).
>

Ah. I had misunderstood the signature of bytevector-*-ref
Fixed.


> Also, I'm not sure why you're specifying (endianness big) here.


I just attempted with (endianness big). It worked, so I assumed it had to
be right


> I would
> think (native-endianness) would be appropriate here.


Ah ! I hadn't noticed (native-endianness) existed !
Fixed


> Given this, and
> the fact that you're passing the wrong width, makes me surprised that
> this is working for you at all.


Mark, thank you so much for your review. I appreciate that

I apologize if my questions are naive.
I wrote my last scrap of C code in about 2004 and it never was my thing
Also the manual is a great reference but not a great tutorial and I'm not a
great reader, probably.

Still I'd like to use guile-freexl to extract data from a bunch of xls
files and store such data in a Postgresql db.

I believe that this could be useful to other guilers too

Last time I used a Clojure wrap around an Apache Foundation Java library.
But guile-freexl would be free software !

So, yeah, I'm sorry for the fuss

All that said, I applied the fixes that you suggested and I still get
completely wrong number of rows of the spreadsheets and correct number of
columns

Exactly as before your correctons

So maybe this thingie of mine could use some further attention.

Thanks again

Re: unsigned-int

[Catonano]

2017-06-22 21:13 GMT+02:00 Catonano <catonano@gmail.com>:
>
>
> I apologize if my questions are naive.
> I wrote my last scrap of C code in about 2004 and it never was my thing
> Also the manual is a great reference but not a great tutorial and I'm not
a great reader, probably.

Something that is concerning me is that in the example  a "rows" variable
and a "columns" variable get declared as

unsigned int rows;
unsigned short columns;

and then they get passed as arguments to freexl_worksheet_dimensions as

&rows, &columns

like this

ret = freexl_worksheet_dimensions (handle, &rows, &columns);

In scheme I am NOT declaring anything because I don't now how to
I am just passing void pointers to freexl_worksheet_dimensions

But when you declare a variable of some type, some memory gets reserved for
the value to be contained in that variable, right ?

But I am passing void pointers here, so is any memory region being reserved
? I'm afraid not

That might be the reason why I get garbage rows numbers. Because I am
making it read some random memory region that has been already reserved for
something else, that's already populated with God knows what and I am
making it interpreting that stuff as numbers.

I don't know, I'm speaking freely, here, this is a wild hypothesis

I can't make so much sense of this code

Please bear with me :-/

Thanks in advance

Re: unsigned-int

[Catonano]

2017-06-22 21:33 GMT+02:00 Catonano <catonano@gmail.com>:

> 2017-06-22 21:13 GMT+02:00 Catonano <catonano@gmail.com>:
> >
> >
> > I apologize if my questions are naive.
> > I wrote my last scrap of C code in about 2004 and it never was my thing
> > Also the manual is a great reference but not a great tutorial and I'm
> not a great reader, probably.
>
> Something that is concerning me is that in the example  a "rows" variable
> and a "columns" variable get declared as
>
> unsigned int rows;
> unsigned short columns;
>
> and then they get passed as arguments to freexl_worksheet_dimensions as
>
> &rows, &columns
>
> like this
>
> ret = freexl_worksheet_dimensions (handle, &rows, &columns);
>
> In scheme I am NOT declaring anything because I don't now how to
>

Maybe you make a bytevector the size of <type>

and then you use bytevector->pointer ?

Ah !

Re: unsigned-int

[Catonano]

2017-06-22 21:55 GMT+02:00 Catonano <catonano@gmail.com>:

> 2017-06-22 21:33 GMT+02:00 Catonano <catonano@gmail.com>:
>
>> 2017-06-22 21:13 GMT+02:00 Catonano <catonano@gmail.com>:
>> >
>> >
>> > I apologize if my questions are naive.
>> > I wrote my last scrap of C code in about 2004 and it never was my thing
>> > Also the manual is a great reference but not a great tutorial and I'm
>> not a great reader, probably.
>>
>> Something that is concerning me is that in the example  a "rows" variable
>> and a "columns" variable get declared as
>>
>> unsigned int rows;
>> unsigned short columns;
>>
>> and then they get passed as arguments to freexl_worksheet_dimensions as
>>
>> &rows, &columns
>>
>> like this
>>
>> ret = freexl_worksheet_dimensions (handle, &rows, &columns);
>>
>> In scheme I am NOT declaring anything because I don't now how to
>>
>
> Maybe you make a bytevector the size of <type>
>
> and then you use bytevector->pointer ?
>
> Ah !
>

No :-/

I tried. The number of rows keeps being completely off

Re: unsigned-int

[Catonano]

For now I renounced to extract  numbers from my bytevectors

The culprit is: #vu8(232 3 0 0)

There are 27 lines in the sheet, so this bytevector should represent 27,
somehow.

The good bytevector is: #vu8(5 0)

In fact, the sheet has 5 columns

On another sheet in the same file, this is what happens:

(#vu8(118 0 0 0) #vu8(17 0))

The sheet has 17 columns indeed. And it has 23 rows :-/

This is where I'm at, right now

I even made a commit out of this
https://gitlab.com/humanitiesNerd/guile-freexl/commit/bddc8bb33c5dc5fbe5acf4b954dfb7d42453f956

Re: unsigned-int

[Mark H Weaver]

Catonano <catonano@gmail.com> writes:

> 2017-06-22 21:13 GMT+02:00 Catonano <catonano@gmail.com>:
>>
>>
>> I apologize if my questions are naive.
>> I wrote my last scrap of C code in about 2004 and it never was my thing
>> Also the manual is a great reference but not a great tutorial and I'm not
> a great reader, probably.
>
> Something that is concerning me is that in the example  a "rows" variable
> and a "columns" variable get declared as
>
> unsigned int rows;
> unsigned short columns;
>
> and then they get passed as arguments to freexl_worksheet_dimensions as
>
> &rows, &columns
>
> like this
>
> ret = freexl_worksheet_dimensions (handle, &rows, &columns);
>
> In scheme I am NOT declaring anything because I don't now how to
> I am just passing void pointers to freexl_worksheet_dimensions

No, you need to allocate the memory where 'rows' and 'columns' will be
stored, and pass pointers to that memory.  freexl_worksheet_dimensions
will then store the values in those locations.

I looked at your code, and I see you've updated the code since you
posted these messages, but it's still not right:

> (define freexl-worksheet-dimensions
>   (let* ((ptr     (freexl-func "freexl_worksheet_dimensions"))
>          (proc    (pointer->procedure int ptr (list '* '* '*))))
>     (lambda (handle-ptr)
>       (let* ((rows-ptr (bytevector->pointer (make-bytevector (sizeof '*))))
>              (columns-ptr (bytevector->pointer (make-bytevector (sizeof '*))))

In the two lines above, instead of (sizeof '*), it should be
(sizeof unsigned-int) for the rows bytevector, and
(sizeof unsigned-short) for the columns bytevector.

>              (result (proc (unwrap-handler handle-ptr) rows-ptr columns-ptr)))
>         (if (not (= result 0))
>             (throw 'get-worksheet-dimensions-error 'error-code result)
>             ;;(list (dereference-pointer rows-ptr) (dereference-pointer columns-ptr))
>             (list
>              (pointer->bytevector
>               rows-ptr (sizeof unsigned-int)) 
>              
>               
>                                    
>              (pointer->bytevector
>               columns-ptr (sizeof unsigned-short))  
>               ))

This returns a list of two bytevectors.  Wouldn't it be better to return
the actual values?  Also, instead of using 'pointer->bytevector' here,
it would be better to have saved the bytevectors that you created in the
first place.  Maybe something like this (untested):

(define freexl-worksheet-dimensions
  (let* ((ptr     (freexl-func "freexl_worksheet_dimensions"))
         (proc    (pointer->procedure int ptr (list '* '* '*))))
    (lambda (handle-ptr)
      (let* ((rows-bv    (make-bytevector (sizeof unsigned-int)))
             (columns-bv (make-bytevector (sizeof unsigned-short)))
             (result     (proc (unwrap-handler handle-ptr)
                               (bytevector->pointer rows-bv)
                               (bytevector->pointer columns-bv))))
        (unless (zero? result)
          (throw 'get-worksheet-dimensions-error 'error-code result))
        (list (bytevector-uint-ref rows-bv 0 (native-endianness)
                                   (sizeof unsigned-int))
              (bytevector-uint-ref columns-bv 0 (native-endianness)
                                   (sizeof unsigned-short)))))))

       Mark

Re: unsigned-int

[Mark H Weaver]

Catonano <catonano@gmail.com> writes:

> 2017-06-22 18:20 GMT+02:00 Mark H Weaver <mhw@netris.org>:
>
>> Given this, and
>> the fact that you're passing the wrong width, makes me surprised that
>> this is working for you at all.
>
>
> Mark, thank you so much for your review. I appreciate that
>
> I apologize if my questions are naive.

No, not at all.  I appreciate your work on guile-freexl.  My surprise
that it was working at all was not a judgement of you, rather it was a
sincere confusion, but now I understand.

The combination of two mistakes: (endianness big) and width 1, actually
had the effect of making this work for numbers small enough to fit in 1
byte, which means unsigned numbers between 0 and 255 and signed numbers
between -128 and 127.

> So, yeah, I'm sorry for the fuss

No need to apologize.

> All that said, I applied the fixes that you suggested and I still get
> completely wrong number of rows of the spreadsheets and correct number of
> columns
>
> Exactly as before your correctons

Except that now it will work for larger numbers, and on architectures
that are truly big endian, whereas before it wouldn't have.

> For now I renounced to extract  numbers from my bytevectors
> 
> The culprit is: #vu8(232 3 0 0)
> 
> There are 27 lines in the sheet, so this bytevector should represent 27,
> somehow.

On a little-endian architecture (which includes Intel and most other
popular architectures), #vu8(232 3 0 0) represents 1000.  If that's the
number you're getting, then that part of the code is working correctly.

> On another sheet in the same file, this is what happens:
> 
> (#vu8(118 0 0 0) #vu8(17 0))
> 
> The sheet has 17 columns indeed. And it has 23 rows :-/

#vu8(118 0 0 0) represents 118.

I'm not sure why it's a different number than you're expecting, but I
can tell you that's the number in that bytevector.

       Mark

Re: unsigned-int

[Catonano]

2017-06-23 14:16 GMT+02:00 Mark H Weaver <mhw@netris.org>:

>
>
No need to apologize.
>
Cool, tanks :-)




>
> #vu8(118 0 0 0) represents 118.
>
> I'm not sure why it's a different number than you're expecting, but I
> can tell you that's the number in that bytevector.
>

I was expecting a different numbr of rows because I opened the spreadsheet
with Librreoffice and counted the rows with my eyes

In my opinion, tey are not 118. They are way less

BUT I tried to run an example C program that comes with Freexl that
extracts the same information

On most sheets I agree with what tis test program reports

Only on 1 of them I disagree

The good news is that the numbers that are coming out form my scheme
functions are the same numbers that are coming out from the test C program

They output the same numbers

As for why I disagree on the number of rows on one sheet I don't understand
why 118 rows come up. Maybe there are some cells filled with spaces

So there's no glitch in my scheme wrap. Not anymore

This is all I needed to know

Than you Mark.