unofficial mirror of guile-user@gnu.org 
 help / color / mirror / Atom feed
* Guile AWS
@ 2021-04-12 15:47 Ricardo Wurmus
  2021-04-12 16:02 ` [EXT] " Thompson, David
  0 siblings, 1 reply; 3+ messages in thread
From: Ricardo Wurmus @ 2021-04-12 15:47 UTC (permalink / raw)
  To: guile-user

Hi Guilers,

this is not quite a release announcement, but the code turns out to be
usable enough that I thought I’d share what I’ve got at this point.

Guile AWS is a library that lets you talk to Amazon Web Services such as
S3, the Elastic Compute Cloud (EC2), Elastic File System (EFS), Route53
(Amazon’s DNS server), etc.  The implementation is probably more
interesting than the library itself.

Guile AWS is little more than a language specification for the Guile
compiler tower, which compiles the JSON specification of the AWS APIs to
Scheme.  There’s also a bit of inelegant plumbing to actually make
requests to an API endpoint.

I’ve been hacking on this library on and off again, testing only those
bits that I needed, so I would be happy if others were to play with it
and/or improve it.

Here’s what a session might look like:

--8<---------------cut here---------------start------------->8---
(import (aws api elasticfilesystem-2015-02-01))

;; You can also use the parameters %aws-default-region, %aws-access-key,
;; and %aws-secret-access-key
(setenv "AWS_DEFAULT_REGION" "eu-central-1")
(setenv "AWS_SECRET_ACCESS_KEY" "…")
(setenv "AWS_ACCESS_KEY_ID" "AKIA…")

;; Create a file system with this unique creation token.
(CreateFileSystem
 #:CreationToken "my-guile-aws-filesystem"
 #:Tags (list (Tag #:Key "project" #:Value "guile-aws")
              (Tag #:Key "type" #:Value "test")))

;; => returns some JSON, including the fs-id …


(import (aws api ec2-2016-11-15))
(RunInstances #:MaxCount 1
              #:MinCount 1
              #:ImageId %my-ami
              #:KeyName %my-ssh-key-name
              #:SubnetId %my-subnet-id
              #:InstanceType type
              #:InstanceInitiatedShutdownBehavior "terminate"
              #:BlockDeviceMappings
              (list (BlockDeviceMapping
                     #:DeviceName "/dev/xvda"
                     #:Ebs
                     (EbsBlockDevice
                      #:DeleteOnTermination #true
                      #:VolumeType "gp3"
                      #:VolumeSize 2))
                    (BlockDeviceMapping
                     #:DeviceName "/dev/sdf"
                     #:Ebs
                     (EbsBlockDevice
                      #:DeleteOnTermination #true
                      #:SnapshotId "snapshot-whatever"
                      #:VolumeType "gp3"
                      #:Iops 6000)))
              #:TagSpecifications
              (list (TagSpecification
                     #:ResourceType "instance"
                     #:Tags (list (Tag #:Key "created-by"
                                       #:Value "guile-aws")
                                  (Tag #:Key "name"
                                       #:Value "guile-aws-test"))))
              #:SecurityGroupIds
              %my-security-groups
              #:UserData %encoded-userdata)

;; => returns some SXML response
--8<---------------cut here---------------end--------------->8---

The biggest flaw in Guile AWS is handling of responses.  Currently, the
responses are raw JSON or SXML (dependent on the API protocol).  I left
it this way because I found that converting the response to an opaque
record type just isn’t very nice; it made processing of responses quite
a bit harder.  If you have a good idea how to best represent the
responses in a more Schemey way, please do let me know!

Another open issue is plumbing.  A conversation with AWS APIs is
reminiscent of monadic computations (in that computations can fail at
any point and the flow should not be burdened with error handling), and
it also feels like prompts and delimited continuations would be a good
fit.  Currently, there is nothing here to simplify a conversation with
the API, e.g. to repeat API calls until a certain condition is met, to
time out requests or to abort polling after n retries, or even to just
handle errors.

The biggest obstacle was the variability in AWS APIs.  Some expect XML
payloads, others JSON, yet others expect a weirdly mangled request
string (some sort of custom stringy representation of the request
payload) — it’s a zoo!  I’m still pretty sure that there are a bunch of
ridiculous bugs in the code, and it’s difficult for me to come up with
good tests.  I’d appreciate any help!

Having said all that: this really works for my limited use cases.  I can
create EC2 instances, initialize them, tag them, terminate them, attach
volumes, create and configure EFS file systems, attach them, update DNS
records for newly generated machines, etc.

The code is here:

   https://git.elephly.net/gitweb.cgi?p=software/guile-aws.git

and here’s a Guix package definition:

--8<---------------cut here---------------start------------->8---
(use-modules ((guix licenses) #:prefix license:)
             (guix packages)
             (guix git-download)
             (guix utils)
             (guix build-system gnu)
             (gnu packages)
             (gnu packages autotools)
             (gnu packages databases)
             (gnu packages gnupg)
             (gnu packages guile)
             (gnu packages guile-xyz)
             (gnu packages pkg-config)
             (gnu packages web))

(define guile-aws
  (let ((commit "bed1a6624e0faac0830346e3bb1ac3581e8bb34b")
        (revision "20210409.1"))
    (package
      (name "guile-aws")
      (version (git-version "0" revision commit))
      (source (origin
                (method git-fetch)
                (uri (git-reference
                      (url "https://git.elephly.net/software/guile-aws.git")
                      (commit commit)))
                (file-name (git-file-name name version))
                (sha256
                 (base32
                  "129xxv4xq2pn1bax8gvq0h3p4bsfdflipv85bsq94pinks9jb0cd"))))
      (build-system gnu-build-system)
      (native-inputs
       `(("autoconf" ,autoconf)
         ("automake" ,automake)
         ("pkg-config" ,pkg-config)))
      (inputs
       `(("guile" ,guile-3.0-latest)))
      (propagated-inputs
       (let ((p (package-input-rewriting
                 `((,guile-3.0 . ,guile-3.0-latest))
                 #:deep? #false)))
         `(("guile-json" ,(p guile-json-3))
           ("guile-gcrypt" ,(p guile-gcrypt)))))
      (home-page "https://git.elephly.net/software/guile-aws.git")
      (synopsis "AWS client for Guile")
      (description "Guile AWS is pre-alpha software.  At the very
least it’s yet another demonstration that Guile’s compiler tower can
be used to generate an embedded domain specific language from JSON
specifications.")
      (license license:gpl3+))))
--8<---------------cut here---------------end--------------->8---

-- 
Ricardo



^ permalink raw reply	[flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-04-12 16:19 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-12 15:47 Guile AWS Ricardo Wurmus
2021-04-12 16:02 ` [EXT] " Thompson, David
2021-04-12 16:19   ` Ricardo Wurmus

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).