From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Matt Wette Newsgroups: gmane.lisp.guile.devel,gmane.lisp.guile.user Subject: ffi-help: status to 19 Aug 2017 Date: Sat, 19 Aug 2017 08:30:54 -0700 Message-ID: <25A19914-FADD-46DC-AEFA-F290210C33DF@gmail.com> References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1503156675 4513 195.159.176.226 (19 Aug 2017 15:31:15 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 19 Aug 2017 15:31:15 +0000 (UTC) Cc: Guile User To: guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Sat Aug 19 17:31:11 2017 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dj5iO-0000nF-3g for guile-devel@m.gmane.org; Sat, 19 Aug 2017 17:31:08 +0200 Original-Received: from localhost ([::1]:33575 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dj5iU-0000j8-JN for guile-devel@m.gmane.org; Sat, 19 Aug 2017 11:31:14 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:33512) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dj5iH-0000hK-2Q for guile-devel@gnu.org; Sat, 19 Aug 2017 11:31:02 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dj5iD-0000p7-TF for guile-devel@gnu.org; Sat, 19 Aug 2017 11:31:01 -0400 Original-Received: from mail-pg0-x231.google.com ([2607:f8b0:400e:c05::231]:33773) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dj5iD-0000mz-Ix; Sat, 19 Aug 2017 11:30:57 -0400 Original-Received: by mail-pg0-x231.google.com with SMTP id t3so50789373pgt.0; Sat, 19 Aug 2017 08:30:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=GnY8Ih5MErKqjlMB5s7oHb/Pej5sdu7lrAxjZvI/0+E=; b=kCxRLw2Db75VXBqCmgACijD191Hl94lg8j7r+HSFkjdULM2TGT1XVJoFrgWrpgKRZa p0zfzddpjRSUVikN4TUrlfOKhMkOAuG98q9Zzc2ThwiS+ol10lfBtRGEIWCK/Mm59KnA O5oyh/RJXSOjJQGo5i/MUkIuJ4UijXnL/gExD0Ohm52wpzLDP9yBFSR9FuvjtqGvt8mZ Dj2SEuzTEUuPOP4iBN3uao3KZlNFZo8n7lGlPPL9PIvucvFzpVorlrOUk5cPtN1MAtrB vRw5BttXtDBbKrRFcczJXtI0O01UJ1jyOGpKWFRRXE7g7UI08+WfgULoiUS19eqBuDM+ Vp+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=GnY8Ih5MErKqjlMB5s7oHb/Pej5sdu7lrAxjZvI/0+E=; b=OBIK0946FbHHfiLpWvfLsTClMAPm6jVUiak/7d7X/BX1fJbKas2e6MDrpvLRPqwPy1 vSjqAGbwJ7QIAktlFyFsTluMVsvmGooxqNwMZc0Ej2CzCt0McXqqQjpPx/HfN+hiD8Qo DoBH9nrVtPGhP0BgbF+cquavAhnDdCkZ7wlK0y5p26/Mn27Xj1I88xRQVEzfH1QsF0JS plhQgxxR/wSQX6riHwPUAayBIHLDqEmFgqwRL1PkicpjhM+5rRCx3oTv4usn2X27wwKf TgPpULI6njmB2P9cicsx5Zx+iHRjICzOc1omg9HIzrqfGdEaT8tQfE5D3b+FaZnaaHyR I2Tw== X-Gm-Message-State: AHYfb5hh8m8ClDlak0TzcJN2ypONQuVscVPowze4shJKlfNf8vlmbSoB WuT2hos/4X0D0vjY9ek= X-Received: by 10.84.132.76 with SMTP id 70mr13551088ple.7.1503156656011; Sat, 19 Aug 2017 08:30:56 -0700 (PDT) Original-Received: from nautilus.championbroadband.com (216-165-229-229.championbroadband.com. [216.165.229.229]) by smtp.gmail.com with ESMTPSA id u12sm14889207pgq.24.2017.08.19.08.30.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Aug 2017 08:30:55 -0700 (PDT) In-Reply-To: X-Mailer: Apple Mail (2.3273) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::231 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19261 gmane.lisp.guile.user:14038 Archived-At: Hi All, I am working on a ffi-helper: a program that will read in a C dot-h file = and generate a Guile dot-scm file=20 which defines a module to provide hooks into the associated C library. = Goal is to release something in Oct 2017 but that date is likely to slip. Current shortcomings: 1) Guile does not currently handle some types (e..g, long long, = uintptr_t). 2) Guile does not have support for varargs (e.g., printf(char *, ...) ). I may take a look at this. One idea I have is to use '... in the = call interface spec and use (type . value) pairs in the calls. 3) The bytestructures module does not support function declarations. 4) ... (probably more) Someone asked to have libgit2 converted and this, and some others, = turned out to give visibility to several limitations and bugs in my C parser. For one, how #include = is interpreted=20 is not specified by the language: it is implementation defined, and I = had to track down how libgit2=20 was including files. I also had to add some GNUC extensions (e.g., asm, = statement-block expressions, include_next) to the parser and preprocessor. As one can see from the = file listing below, libgit2=20 has a large number of files, and declarations. mwette$ ls include/git2 annotated_commit.h indexer.h repository.h attr.h inttypes.h reset.h blame.h merge.h revert.h blob.h message.h revparse.h branch.h net.h revwalk.h buffer.h notes.h signature.h checkout.h object.h stash.h cherrypick.h odb.h status.h clone.h odb_backend.h stdint.h commit.h oid.h strarray.h common.h oidarray.h submodule.h config.h pack.h sys/ cred_helpers.h patch.h tag.h describe.h pathspec.h trace.h diff.h proxy.h transaction.h errors.h rebase.h transport.h filter.h refdb.h tree.h global.h reflog.h types.h graph.h refs.h version.h ignore.h refspec.h worktree.h index.h remote.h mwette$ cat libgit2.ffi (define-ffi-module (libgit2) #:include '("git2.h") #:inc-filter (lambda (file-spec path-spec) (string-contains path-spec "git2/" 0)) #:library '("libgit2")) The following command takes 10.5 seconds on my new macbook pro=20 and produces a libgit2.scm file that is 14,546 lines long. mwette$ guild compile-ffi libgit2.ffi The following command takes 66 seconds on my mac (Guile 2.0.13). mwette$ guile -c '(use-modules (libgit2))'=20 (However, I'm still getting some unresolved references: that's work to = go.) (The compile takes significantly longer using Guile 2.2.) Here are some excerpts from (100% autogenerated) libgit2.scm: ;; typedef struct git_repository git_repository; (define-fh-pointer-type git_repository*) ;; typedef struct git_remote git_remote; (define-fh-pointer-type git_remote*) ;; extern git_repository *git_remote_owner(const git_remote *remote); (define git_remote_owner (let ((~f (ffi:pointer->procedure '* (dynamic-func "git_remote_owner" (dynamic-link)) (list '*)))) (lambda (remote) (let ((~remote (unwrap-git_remote* remote))) (wrap-git_repository* (~f ~remote)))))) (export git_remote_owner) ;; typedef enum { ;; GIT_REF_INVALID =3D 0, ;; GIT_REF_OID =3D 1, ;; GIT_REF_SYMBOLIC =3D 2, ;; GIT_REF_LISTALL =3D GIT_REF_OID | GIT_REF_SYMBOLIC, ;; } git_ref_t; (define-fh-enum git_ref_t '((GIT_REF_INVALID . 0) (GIT_REF_OID . 1) (GIT_REF_SYMBOLIC . 2) (GIT_REF_LISTALL . 3)) ) ;; typedef struct git_remote_callbacks git_remote_callbacks; ;; struct git_remote_callbacks { ;; unsigned int version; ;; /** ;; * Textual progress from the remote. Text send over the ;; * progress side-band will be passed to this function (this is ;; * the 'counting objects' output). ;; */ ;; git_transport_message_cb sideband_progress; ;; /** ;; * Completion is called when different parts of the download ;; * process are done (currently unused). ;; */ ;; int (*completion)(git_remote_completion_type type, void *data); ;; /** ;; * This will be called if the remote host requires ;; * authentication in order to connect to it. ;; * ;; * Returning GIT_PASSTHROUGH will make libgit2 behave as ;; * though this field isn't set. ;; */ ;; git_cred_acquire_cb credentials; ;; /** ;; * If cert verification fails, this will be called to let the ;; * user make the final decision of whether to allow the ;; * connection to proceed. Returns 1 to allow the connection, 0 ;; * to disallow it or a negative value to indicate an error. ;; */ ;; git_transport_certificate_check_cb certificate_check; ;; /** ;; * During the download of new data, this will be regularly ;; * called with the current count of progress done by the ;; * indexer. ;; */ ;; git_transfer_progress_cb transfer_progress; ;; /** ;; * Each time a reference is updated locally, this function ;; * will be called with information about it. ;; */ ;; int (*update_tips)(const char *refname, const git_oid *a, const = git_oid * ;; b, void *data); ;; /** ;; * Function to call with progress information during pack ;; * building. Be aware that this is called inline with pack ;; * building operations, so performance may be affected. ;; */ ;; git_packbuilder_progress pack_progress; ;; /** ;; * Function to call with progress information during the ;; * upload portion of a push. Be aware that this is called ;; * inline with pack building operations, so performance may be ;; * affected. ;; */ ;; git_push_transfer_progress push_transfer_progress; ;; /** ;; * Called for each updated reference on push. If `status` is ;; * not `NULL`, the update was rejected by the remote server ;; * and `status` contains the reason given. ;; */ ;; int (*push_update_reference)(const char *refname, const char = *status,=20 ;; void *data); ;; /** ;; * Called once between the negotiation step and the upload. It ;; * provides information about what updates will be performed. ;; */ ;; git_push_negotiation push_negotiation; ;; /** ;; * Create the transport to use for this operation. Leave NULL ;; * to auto-detect. ;; */ ;; git_transport_cb transport; ;; /** ;; * This will be passed to each of the callbacks in this struct ;; * as the last parameter. ;; */ ;; void *payload; ;; }; (define git_remote_callbacks-desc (bs:struct (list `(version ,unsigned-int) `(sideband_progress ,git_transport_message_cb-desc) `(completion ,(bs:pointer intptr_t)) `(credentials ,git_cred_acquire_cb-desc) `(certificate_check ,git_transport_certificate_check_cb-desc) `(transfer_progress ,git_transfer_progress_cb-desc) `(update_tips ,(bs:pointer intptr_t)) `(pack_progress ,git_packbuilder_progress-desc) `(push_transfer_progress ,git_push_transfer_progress-desc) `(push_update_reference ,(bs:pointer intptr_t)) `(push_negotiation ,git_push_negotiation-desc) `(transport ,git_transport_cb-desc) `(payload ,(bs:pointer intptr_t))))) (export git_remote_callbacks-desc) (define-fh-compound-type/p git_remote_callbacks = git_remote_callbacks-desc) (define struct-git_remote_callbacks git_remote_callbacks) ;; extern int git_reference_create_matching(git_reference **out,=20 ;; git_repository *repo, const char *name, const git_oid *id, int = force,=20 ;; const git_oid *current_id, const char *log_message); (define git_reference_create_matching (let ((~f (ffi:pointer->procedure ffi:int (dynamic-func "git_reference_create_matching" (dynamic-link)) (list '* '* '* '* ffi:int '* '*)))) (lambda (out repo name id force current_id log_message) (let ((~out (unwrap~pointer out)) (~repo (unwrap-git_repository* repo)) (~name (unwrap~pointer name)) (~id (unwrap-git_oid* id)) (~force (unwrap~fixed force)) (~current_id (unwrap-git_oid* current_id)) (~log_message (unwrap~pointer log_message))) (~f ~out ~repo ~name ~id ~force ~current_id ~log_message))))) (export git_reference_create_matching)