I was looking at writing something to interact with gpsd. There is a shared memory interface, and a Unix socket interface. The protocol spoken by gpsd is JSON, so I figured that I would just open up a socket and start talking ... however ... really early on in the man page, it says "it is a bad idea for clients to speak the protocol directly" ( https://gpsd.gitlab.io/gpsd/gpsd_json.html) presumably since the protocol was likely to change? They didn't elaborate ... There is a C interface described at (https://gpsd.gitlab.io/gpsd/libgps.html), which has the familiar "open-and-get-opaque-thing" and "close-opaque-thing-when-done" design. I'd rather not compile anything in C, and just use the tools in Guile to interact with libgps. Is there a way to get Guile's garbage collector to call "gps_close" on the opaque structure returned by "gps_open"? Or is this something that would be more handily implemented in a C extension? Right now, because I'm just not concerned about sloppiness, I've just been speaking the wire protocol, but I do feel bad about being sloppy and wanted to clean it up for others to use perhaps.
Hello,
Le samedi 04 septembre 2021 à 07:41 -0500, Tim Meehan a écrit :
> I'd rather not compile anything in C, and just use the tools in Guile to
> interact with libgps. Is there a way to get Guile's garbage collector to
> call "gps_close" on the opaque structure returned by "gps_open"?
I think it would be best to do it yourself. Maybe the garbage collector will not run immediately (if at all), and if each creation uses "precious" resources (such as file descriptors), you might run
into a shortage before the garbage collector is triggered. This is not considered by people writing the C API of course, because they assume you would close the thing as soon as possible. (Also, if
calling the close function is mandatory, for instance to run code that’s not just freeing resources, and the garbage collector does not have a chance to run, then it’s another problem. However, I
doubt your library does something like that).
If you want to avoid the problem, you should explicitely bind and call the gps-close function and not rely on the garbage collector to do it for you. You can use dynamic-wind to open and close
resources as needed.
That being said, make-pointer (from (system foreign-library)) is probably what you are expecting. It should work with gps_close.
Best regards,
Vivien
Thanks Vivien, I had not considered dynamic-wind. I'll look into that a bit
more.
Cheers,
Tim
On Sat, Sep 4, 2021 at 8:35 AM Vivien Kraus <vivien@planete-kraus.eu> wrote:
> Hello,
>
> Le samedi 04 septembre 2021 à 07:41 -0500, Tim Meehan a écrit :
> > I'd rather not compile anything in C, and just use the tools in Guile to
> > interact with libgps. Is there a way to get Guile's garbage collector to
> > call "gps_close" on the opaque structure returned by "gps_open"?
> I think it would be best to do it yourself. Maybe the garbage collector
> will not run immediately (if at all), and if each creation uses "precious"
> resources (such as file descriptors), you might run
> into a shortage before the garbage collector is triggered. This is not
> considered by people writing the C API of course, because they assume you
> would close the thing as soon as possible. (Also, if
> calling the close function is mandatory, for instance to run code that’s
> not just freeing resources, and the garbage collector does not have a
> chance to run, then it’s another problem. However, I
> doubt your library does something like that).
>
> If you want to avoid the problem, you should explicitely bind and call the
> gps-close function and not rely on the garbage collector to do it for you.
> You can use dynamic-wind to open and close
> resources as needed.
>
> That being said, make-pointer (from (system foreign-library)) is probably
> what you are expecting. It should work with gps_close.
>
> Best regards,
>
> Vivien
>
>
Il giorno sab, 04/09/2021 alle 15.35 +0200, Vivien Kraus via General Guile related discussions ha scritto: > Hello, > > Le samedi 04 septembre 2021 à 07:41 -0500, Tim Meehan a écrit : > > I'd rather not compile anything in C, and just use the tools in > > Guile to > > interact with libgps. Is there a way to get Guile's garbage > > collector to > > call "gps_close" on the opaque structure returned by "gps_open"? > I think it would be best to do it yourself. Maybe the garbage > collector will not run immediately (if at all), and if each creation > uses "precious" resources (such as file descriptors), you might run > into a shortage before the garbage collector is triggered. This is > not considered by people writing the C API of course, because they > assume you would close the thing as soon as possible. (Also, if > calling the close function is mandatory, for instance to run code > that’s not just freeing resources, and the garbage collector does not > have a chance to run, then it’s another problem. However, I > doubt your library does something like that). > > If you want to avoid the problem, you should explicitely bind and > call the gps-close function and not rely on the garbage collector to > do it for you. You can use dynamic-wind to open and close > resources as needed. It'd be so nice to have an example > That being said, make-pointer (from (system foreign-library)) is > probably what you are expecting. It should work with gps_close. Another exmple of what you mean, here, would be so nice :-)
On Sun, 19 Sep 2021, adriano <randomlooser@riseup.net> wrote: >> If you want to avoid the problem, you should explicitely bind and >> call the gps-close function and not rely on the garbage collector to >> do it for you. You can use dynamic-wind to open and close >> resources as needed. > > It'd be so nice to have an example (define (with-my-resource token proc) (let ((resource #f)) (dynamic-wind (lambda () (set! resource (open-my-resource% token))) (proc resource) (lambda () (when resource (close-my-resource% resource)))))) (with-my-resource "some-internal-token" (lambda ())) > >> That being said, make-pointer (from (system foreign-library)) is >> probably what you are expecting. It should work with gps_close. > > Another exmple of what you mean, here, would be so nice :-) Says you have `open_my_resource()` and `close_my_resource()` in C in library "libfoo.so" where open_my_resource takes a C string and returns an integer for the resource while close_my_resource takes the integer of the resource: (define open-my-resource% (eval-when (eval load compile) (let ((this-lib (load-foreign-library "libfoo"))) (foreign-library-function this-lib "open_my_resource" #:return-type int #:arg-types (list '*)))) (define open-my-resource% (eval-when (eval load compile) (let ((this-lib (load-foreign-library "libfoo"))) (foreign-library-function this-lib "open_my_resource" #:return-type int #:arg-types (list int))))) Note that you probably need to do a wrapper named `open-my-resource` to do the conversion of scm_string to C raw pointer before calling `open-my-resource%` in this particular case. This is just an example, but it shows you that you can call foreign C primitives easily without any C code, and that you have to use a dynamic context to manage the lifetime of the C resources. -- Olivier Dion Polymtl
[-- Attachment #1: Type: text/plain, Size: 1546 bytes --] Olivier Dion via General Guile related discussions schreef op zo 19-09-2021 om 14:11 [-0400]: > On Sun, 19 Sep 2021, adriano <randomlooser@riseup.net> wrote: > > > If you want to avoid the problem, you should explicitely bind and > > > call the gps-close function and not rely on the garbage collector to > > > do it for you. You can use dynamic-wind to open and close > > > resources as needed. > > > > It'd be so nice to have an example > > (define (with-my-resource token proc) > (let ((resource #f)) > (dynamic-wind > (lambda () > (set! resource (open-my-resource% token))) > > (proc resource) > > (lambda () > (when resource > (close-my-resource% resource)))))) > > (with-my-resource "some-internal-token" (lambda ())) FWIW, this doesn't work well with continuations, e.g. if you use guile-fibers for concurrency. I'm not familiar with 'libgps', so I'm not sure if it would work for you, but another method for implementing with-my-resource could be to use the exception handling mechanism to call 'close-my-resource' on both normal exits and abnormal exits. Note that this alternative method does _not_ close the resource when calling an escape continuation (let/ec (with-my-resource .. (lambda (r) (ec) (unreachable)))), so you might want to use something like guardians and after-gc-hook to eventually free the resource. (Here, close-my-resource is like close-my-resource% except it doesn't do anything if the resource is already closed.) Greetings, Maxime. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 260 bytes --]
Hi Olivier, thank you very much for your reply Il giorno dom, 19/09/2021 alle 14.11 -0400, Olivier Dion ha scritto: > On Sun, 19 Sep 2021, adriano <randomlooser@riseup.net> wrote: > > > > > > It'd be so nice to have an example > > (define (with-my-resource token proc) > (let ((resource #f)) > (dynamic-wind > (lambda () > (set! resource (open-my-resource% token))) > > (proc resource) > > (lambda () > (when resource > (close-my-resource% resource)))))) > > (with-my-resource "some-internal-token" (lambda ())) Oh my, thank you for this ! This should be included in the manual ! The example that's there currently is totally indequate, in my opinion > > > > Says you have `open_my_resource()` and `close_my_resource()` in C in > library "libfoo.so" where open_my_resource takes a C string and returns > an integer for the resource while close_my_resource takes the integer > of > the resource: > > (define open-my-resource% > (eval-when (eval load compile) > (let ((this-lib (load-foreign-library "libfoo"))) > (foreign-library-function this-lib "open_my_resource" > #:return-type int > #:arg-types (list '*)))) > > (define open-my-resource% > (eval-when (eval load compile) > (let ((this-lib (load-foreign-library "libfoo"))) > (foreign-library-function this-lib "open_my_resource" > #:return-type int > #:arg-types (list int))))) Uhmm... I see 2 versions of open-my-resource% The only slight difference I see is in the #:arg-types The first one has (list '*) and the second one has (list int) Maybe you you got confused while editing ? This would be my version of close-my-resource% (define close-my-resource% (eval-when (eval load compile) (let ((this-lib (load-foreign-library "libfoo"))) (foreign-library-function this-lib "close_my_resource" #:return-type int #:arg-types (int))))) > > Note that you probably need to do a wrapper named `open-my-resource` > to > do the conversion of scm_string to C raw pointer before calling > `open-my-resource%` in this particular case. Ok > > This is just an example, but it shows you that you can call foreign C > primitives easily without any C code, and that you have to use a > dynamic context to manage the lifetime of the C resources. > Thank you again
On Tue, 21 Sep 2021, adriano <randomlooser@riseup.net> wrote: > Hi Olivier, > > thank you very much for your reply > > Il giorno dom, 19/09/2021 alle 14.11 -0400, Olivier Dion ha scritto: >> On Sun, 19 Sep 2021, adriano <randomlooser@riseup.net> wrote: >> > >> > > > > > >> > It'd be so nice to have an example >> >> (define (with-my-resource token proc) >> (let ((resource #f)) >> (dynamic-wind >> (lambda () >> (set! resource (open-my-resource% token))) >> >> (proc resource) >> >> (lambda () >> (when resource >> (close-my-resource% resource)))))) >> >> (with-my-resource "some-internal-token" (lambda ())) > > Oh my, thank you for this ! > > This should be included in the manual ! > > The example that's there currently is totally indequate, in my opinion > >> > >> >> Says you have `open_my_resource()` and `close_my_resource()` in C in >> library "libfoo.so" where open_my_resource takes a C string and returns >> an integer for the resource while close_my_resource takes the integer >> of >> the resource: >> >> (define open-my-resource% >> (eval-when (eval load compile) >> (let ((this-lib (load-foreign-library "libfoo"))) >> (foreign-library-function this-lib "open_my_resource" >> #:return-type int >> #:arg-types (list '*)))) >> >> (define open-my-resource% >> (eval-when (eval load compile) >> (let ((this-lib (load-foreign-library "libfoo"))) >> (foreign-library-function this-lib "open_my_resource" >> #:return-type int >> #:arg-types (list int))))) > > Uhmm... I see 2 versions of open-my-resource% > The only slight difference I see is in the #:arg-types > > The first one has > > (list '*) > > and the second one has > > (list int) > > > Maybe you you got confused while editing ? You're right. I copy paste the form two times and forget to edit the second binding name. I meant `close-my-resource%` for the second define. > > Thank you again Just make sure to read what Maxime says about escape continuation. Dynamic wind are not bullet proof. As the manual says: If, any time during the execution of THUNK, the dynamic extent of the ‘dynamic-wind’ expression is escaped non-locally, OUT_GUARD is called. If the dynamic extent of the dynamic-wind is re-entered, IN_GUARD is called. Thus IN_GUARD and OUT_GUARD may be called any number of times. Thus, you could also do a `(set! resource #f)` after freeing it, so that the overall dynamic-wind is reentrant and you don't end up with double free of resource. -- Olivier Dion Polymtl
Hi Maxime,
Il giorno dom, 19/09/2021 alle 20.23 +0200, Maxime Devos ha scritto:
> Olivier Dion via General Guile related discussions schreef op zo 19-
> 09-2021 om 14:11 [-0400]:
> > On Sun, 19 Sep 2021, adriano <randomlooser@riseup.net> wrote:
> > > > If you want to avoid the problem, you should explicitely bind
> > > > and
> > > > call the gps-close function and not rely on the garbage
> > > > collector to
> > > > do it for you. You can use dynamic-wind to open and close
> > > > resources as needed.
> > >
> > > It'd be so nice to have an example
> >
> > (define (with-my-resource token proc)
> > (let ((resource #f))
> > (dynamic-wind
> > (lambda ()
> > (set! resource (open-my-resource% token)))
> >
> > (proc resource)
> >
> > (lambda ()
> > (when resource
> > (close-my-resource% resource))))))
> >
> > (with-my-resource "some-internal-token" (lambda ()))
>
> FWIW, this doesn't work well with continuations, e.g. if you use
> guile-fibers
> for concurrency. I'm not familiar with 'libgps', so I'm not sure if
> it would
> work for you, but another method for implementing with-my-resource
> could be to
> use the exception handling mechanism to call 'close-my-resource' on
> both
> normal exits and abnormal exits.
>
> Note that this alternative method does _not_ close the resource when
> calling
> an escape continuation (let/ec (with-my-resource .. (lambda (r) (ec)
> (unreachable)))),
> so you might want to use something like guardians and after-gc-hook
> to eventually
> free the resource.
>
> (Here, close-my-resource is like close-my-resource% except it doesn't
> do anything
> if the resource is already closed.)
>
> Greetings,
> Maxime.
Well, it'd be wonderful to have an example of what you're describing
here ! 🙂️
The new exceptions system is still quite obscure 🤷️
Il giorno mar, 21/09/2021 alle 10.25 -0400, Olivier Dion ha scritto:
> On Tue, 21 Sep 2021, adriano <randomlooser@riseup.net> wrote:
> > Hi Olivier,
> >
> > thank you very much for your reply
> >
> > Il giorno dom, 19/09/2021 alle 14.11 -0400, Olivier Dion ha
> > scritto:
> > > On Sun, 19 Sep 2021, adriano <randomlooser@riseup.net> wrote:
> > > >
> > > >
> >
> >
> >
> >
> > > > It'd be so nice to have an example
> > >
> > > (define (with-my-resource token proc)
> > > (let ((resource #f))
> > > (dynamic-wind
> > > (lambda ()
> > > (set! resource (open-my-resource% token)))
> > >
> > > (proc resource)
> > >
> > > (lambda ()
> > > (when resource
> > > (close-my-resource% resource))))))
> > >
> > > (with-my-resource "some-internal-token" (lambda ()))
> >
> > Oh my, thank you for this !
> >
> > This should be included in the manual !
> >
> > The example that's there currently is totally indequate, in my
> > opinion
> >
> > > >
> > >
> > > Says you have `open_my_resource()` and `close_my_resource()` in C
> > > in
> > > library "libfoo.so" where open_my_resource takes a C string and
> > > returns
> > > an integer for the resource while close_my_resource takes the
> > > integer
> > > of
> > > the resource:
> > >
> > > (define open-my-resource%
> > > (eval-when (eval load compile)
> > > (let ((this-lib (load-foreign-library "libfoo")))
> > > (foreign-library-function this-lib "open_my_resource"
> > > #:return-type int
> > > #:arg-types (list '*))))
> > >
> > > (define open-my-resource%
> > > (eval-when (eval load compile)
> > > (let ((this-lib (load-foreign-library "libfoo")))
> > > (foreign-library-function this-lib "open_my_resource"
> > > #:return-type int
> > > #:arg-types (list int)))))
> >
> > Uhmm... I see 2 versions of open-my-resource%
> > The only slight difference I see is in the #:arg-types
> >
> > The first one has
> >
> > (list '*)
> >
> > and the second one has
> >
> > (list int)
> >
> >
> > Maybe you you got confused while editing ?
>
> You're right. I copy paste the form two times and forget to edit the
> second
> binding name. I meant `close-my-resource%` for the second define.
>
> >
> > Thank you again
>
> Just make sure to read what Maxime says about escape continuation.
> Dynamic wind are not bullet proof.
>
> As the manual says:
>
> If, any time during the execution of THUNK, the dynamic
> extent of
> the ‘dynamic-wind’ expression is escaped non-locally,
> OUT_GUARD is
> called. If the dynamic extent of the dynamic-wind is re-
> entered,
> IN_GUARD is called. Thus IN_GUARD and OUT_GUARD may be
> called any
> number of times.
>
> Thus, you could also do a `(set! resource #f)` after freeing it, so
> that
> the overall dynamic-wind is reentrant and you don't end up with
> double
> free of resource.
>
After seeing your example of usage of dinamic-wind, I understand what
IN_GUARD and OUT_GUARD are supposed to be
Before that it was quite nebulous
Thanks again
[-- Attachment #1: Type: text/plain, Size: 750 bytes --] adriano schreef op di 21-09-2021 om 16:26 [+0200]: > > [...] > Well, it'd be wonderful to have an example of what you're describing > here ! 🙂️ > > The new exceptions system is still quite obscure 🤷️ Ignoring with-throw-handler, raise-continuable and pre-unwind handlers, it's the same mechanism as in Java or Python, though with different syntax, and unlike Java, Guile doesn't have ‘finally’: ;; Can also be implemented with the old 'catch' -- 'catch' isn't disappearing. (guard (c (#t (close-resource-unless-closed the-resource) (raise-exception c))) (define-values return-values (do-stuff)) (close-the-resource-unless-closed the-resource) (apply values return-vales)) Greetings, Maxime [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 260 bytes --]