From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id MCNCGht4d1+bZQAA0tVLHw (envelope-from ) for ; Fri, 02 Oct 2020 18:57:31 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id qOE3Fht4d1/HRAAA1q6Kng (envelope-from ) for ; Fri, 02 Oct 2020 18:57:31 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A15349406F7 for ; Fri, 2 Oct 2020 18:57:27 +0000 (UTC) Received: from localhost ([::1]:34034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kOQFC-00036a-Eq for larch@yhetil.org; Fri, 02 Oct 2020 14:57:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60096) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kOQEx-00036B-M7 for guix-devel@gnu.org; Fri, 02 Oct 2020 14:57:11 -0400 Received: from mira.cbaines.net ([212.71.252.8]:46722) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kOQEr-00044b-Sk; Fri, 02 Oct 2020 14:57:10 -0400 Received: from localhost (188.28.111.33.threembb.co.uk [188.28.111.33]) by mira.cbaines.net (Postfix) with ESMTPSA id EAD6527BBE8; Fri, 2 Oct 2020 19:57:03 +0100 (BST) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id b973af86; Fri, 2 Oct 2020 18:57:01 +0000 (UTC) References: <87o8m062r2.fsf@gnu.org> User-agent: mu4e 1.4.13; emacs 27.1 From: Christopher Baines To: Marius Bakke Subject: Re: Running service migrations during upgrades In-reply-to: <87o8m062r2.fsf@gnu.org> Date: Fri, 02 Oct 2020 19:56:59 +0100 Message-ID: <87ft6w1mo4.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 14:36:47 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Spam-Score: -3.11 X-TUID: Mh/6TpZpysvw --=-=-= Content-Type: text/plain Marius Bakke writes: > Some services require administrator interaction when they are updated. > The most prominent examples here are MySQL/MariaDB and PostgresQL. ... > Another approach is adding a 'herd upgrade' Shepherd action along with a > news entry describing what to do. Of course it is possible to do both, > having 'auto-upgrade?' _and_ a Shepherd action for manual upgrades. > > Thoughts? > > While that works for MariaDB, I'm not sure what to do about Postgres. > For those unfamiliar, the procedure for upgrading from PostgreSQL 10 > (current default) to 11 (available in Guix) is roughly: > > sudo cp -a /var/lib/postgresql/data /var/lib/postgresql/data10 > sudo -u postgres $(guix build postgresql)/bin/pg_upgrade \ > --old-bindir=$(guix build postgresql@10)/bin \ > --new-bindir=$(guix build postgresql)/bin \ > --old-datadir=/var/lib/postgresql/data10 \ > --new-datadir=/var/lib/postgresql/data > > In order to automate it, we need to somehow preserve the "previous" > version of PostgreSQL so that we can reach it when the major version > changes. Or add an 'upgrade-from' parameter to > postgresql-service-type. Hi! I'm really glad you're thinking about this Marius, I've got a few PostgreSQL clusters running with postgresql@10 that I'd like to upgrade, so this is very interesting. Reading through the documentation on pg_upgrade, it suggests the new cluster should be initialised as part of the process [1]. Maybe leaving the old files in place is sometimes/always sufficient, I'm unsure? 1: https://www.postgresql.org/docs/11/pgupgrade.html#id-1.9.5.11.7 One approach I have in mind: - Make the postgresql package explicit in the configuration - This avoids the current situation where changing the major version of the postgresql package would cause PostgreSQL not to start when reconfigured. - Somehow include the major version in the data directory name - So rather than /var/lib/postgresql/data it should be /var/lib/postgresql/10, /var/lib/postgresql/data/10 or something like that. - Add a upgrade-from field to the postgresql service configuration, this takes a postgresql package, which is used for the old-bindir when doing the upgrade - Add a shepherd action to call pg_upgrade I think if all the above things are done, you could end up with a process like: (service postgresql-service-type (postgresql-configuration (postgresql postgresql-10))) -> (service postgresql-service-type (postgresql-configuration (postgresql postgresql-11) (upgrade-from postgresql-10))) Assuming the data directory default includes the major version, when reconfiguring you'd go from running version 10, to an empty version 11. Then running herd upgrade postgres would stop the service, and run pg_upgrade. I started sketching out some patches to at least make the postgresql package explicit in the configuration [2. 2: https://issues.guix.info/43771 Thanks again, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl93d/tfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XczNBAAp3HGQsasQ5nG3P1ZZ76IjoMyV5ah6jJo x4DhU1nLCw4F6a/xIzwCxqTQl2JMbeJUqbl/McY7jRLEieAZh1bhAXeoivtJOLHg E+/tuBA1FPZ4Pv/s2yLt+eeuNY8EfhSTom5b/gCjKoUx0ZrDpYkEn2fZXDRuMAAi jkDj4q87DcBK/XsVx8ijXfLliCQXBoQHlNLgs3NlisbtDwITmFFJ62I5J3l8AO4p ROclXvxJ747IoxvDki9eDhYROZtjhYrBOMGZ+IVyvsvdaH259zPxwwEfDpLOzG5v 2I3SMyXLCOU3pHMNOTsXEV7reLvWBSOUWZJItPlBIWZpwNFTk42aeRwB6oHH+2tl OUZxmL4xppfc1p6PRj/DH5qCC8yq2GdpezBAPQzaHdjoTIxftp/AZfplkzoqEiuR 68tUFrDjmP3HOJxX9vO1PqmdqT5Yp3J2G4Vq34NL31MaRW6XTgaM6FAYwMOyM1bU fCgPkcCtno5EQBsODC2/jUKkQUJ2SWM+cV4IZPEJUyg5VGtQxutp4UzcBR3TQ0uY ckxkhsmFOYdQsCkkeTnn4t73oRBCuwpVwf0dZruEjCArVsNjz82NDaZA6D+LPbjz 3+9Fs1w9F3sbFs+9l5j1Uix32R6eL3xnFC65WmCMbNo/Oo45V0859DY/jv+zjImm VBAizjiG+ks= =bmzZ -----END PGP SIGNATURE----- --=-=-=--