From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christopher Baines Subject: Re: Getting build information in to the Guix Data Service (draft patch) Date: Mon, 28 Oct 2019 08:33:19 +0000 Message-ID: <87pnihjmls.fsf@cbaines.net> References: <87mudvrhim.fsf@cbaines.net> <87tv7zsezm.fsf@gnu.org> <8736fjqbrd.fsf@cbaines.net> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:53458) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP0St-00016Y-Fv for guix-devel@gnu.org; Mon, 28 Oct 2019 04:33:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP0Sr-0007Vn-Se for guix-devel@gnu.org; Mon, 28 Oct 2019 04:33:27 -0400 In-reply-to: <8736fjqbrd.fsf@cbaines.net> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-devel@gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Christopher Baines writes: > Ludovic Court=C3=A8s writes: > >> Christopher Baines skribis: >> >> Should =E2=80=98BuildEvents=E2=80=99 be more generic and have =E2=80=98e= vent=E2=80=99 be an sexp or JSON >> string that could describe any kind of event? >> >> If we did that, we could keep =E2=80=98derivation=E2=80=99 but remove = =E2=80=9CNOT NULL=E2=80=9D so that >> non-derivation events can exist but we can still query >> derivation-related events quickly. Does that make sense? > > Yep, that makes sense. > > This seems to be the general decision about the way you use a relational > database, do you have specific tables (types) for the data, or do you > have a more freeform structure (columns containing sexp or JSON). > > There's quite a few factors to consider here, the internals of Cuirass, > how these events are exposed through the HTTP API, how these events one > day might be published to a WebSub hub and then what kind of > subscriptions you might support in Cuirass (events for an individual > derivation, all builds for an evaluation, all builds, ...). I'll think > about it further and see if I can form an opinion either way. So I had a think about this, and also re-read the WebSub spec [1], along with JSON Feed which is a Atom/RSS alternative with JSON [2]. I'm still a bit undecided about how closely to follow WebSub, as it seems more concerned with applications like blogging, and less with moving data between services, but I still think the principles are good. For now, I think it's best to pick the bits of the standard that work, and leave any that don't particularly apply. I think a generic approach to storing the events in the database will work better though, at least with the way the events are currently exposed. It does make it potentially harder to expose and support subscriptions for events for individual builds, but that can probably be addressed later if desired. 1: https://www.w3.org/TR/websub/ 2: https://jsonfeed.org/version/1 >>> +CREATE TABLE BuildEventsOutbox ( >>> + build_event_id INTEGER NOT NULL, >>> + FOREIGN KEY (build_event_id) REFERENCES BuildEvents (id) >>> +); >> >> These are events that have not yet been sent, right? > > Yep, exactly. > >> Thanks! > > Thanks for taking a look. I'll neaten up the patch a bit, add in some > error handling and retrying for sending out the events, and think a bit > more about the data model, then hopefully send an updated patch soon! I've now sent an updated set of patches. The first to add events for builds, and the second to extend this to evaluations. I've also added in error handling to the sending of the events. Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl22p89fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9Xdwmg//ZB/QyMZ2r9mQQu5FWv+aI9nY+Su36PjEAv1uC7C0Wfp2WWdNfXiY7eTS +TdGw4TjKT5K4kuYQi+CEOUktODpc6KbMTw9aHazE0D5rKqk/+r4V19BzIVjCFz8 qCHPidRE802sxVAsqMy3HRmrLXhMwx5aeYpf6ahIjBQO3HDt5UHgRRxZxH6Qr1AL v8DzkmjXOFFvd1wlnmrYGTgIgKUyxorRIksjAN/2K+gAvBBhQCv6YVnWDwuEG/Qc E05kx8ck/JfXwJ7K8PI7M580jm/LHRemIWvh72rzyvjL6AET7PfN/hwe/PBIBFcY NUtJK1gm06djAbgRF31ZoXrSM4UJ9dgiZQEZQfSsJDr9vM+HoD6w7rtk1EH0rYNJ te/Eeyt4hT2My4OGFSfAfGEu1A8EDEsoWhkmvrKdlEpWJfkOXdmet6PHDUI34Ogr fwzGJC8Q7PE8EDJvjUnsBJsYJxOy7ZhpfRlznQye5sg540VvGGugE0jCbhM9OA5q tBr/shtrsxL88wTR2SwUqdzXIL0ArYU/a6P0yVgShEvGP+++RXTklg2ci9X5w8nq yhpPXMWAVgxEStY5KyUNHHkO4EgQe1lm77++t8mTp2uXaAD/6qvKNgkJJOXaFfwZ rmhMD6UUJFNoXM7MIKVYMPMYn1nWwrpMgof+OzYrdaY744ZPjWU= =UjBZ -----END PGP SIGNATURE----- --=-=-=--