From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christopher Baines Subject: Potential for a race condition with latest-repository-commit Date: Sat, 29 Feb 2020 21:22:18 +0000 Message-ID: <87blphw0l1.fsf@cbaines.net> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:55824) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j89Z5-0000rU-0F for guix-devel@gnu.org; Sat, 29 Feb 2020 16:22:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j89Z3-0001QZ-Uh for guix-devel@gnu.org; Sat, 29 Feb 2020 16:22:26 -0500 Received: from mira.cbaines.net ([212.71.252.8]:45236) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1j89Z3-0001QP-PP for guix-devel@gnu.org; Sat, 29 Feb 2020 16:22:25 -0500 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id 2901027BBE4 for ; Sat, 29 Feb 2020 21:22:24 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id a4511c93 for ; Sat, 29 Feb 2020 21:22:20 +0000 (UTC) List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane-mx.org@gnu.org Sender: "Guix-devel" To: guix-devel@gnu.org --=-=-= Content-Type: text/plain Hey, So one of the things that's currently restricted to doing by one job at a time in the Guix Data Service is running latest-repository-commit from the (guix git) module. Previously this was more of a problem for the Guix Data Service, as a large section of the work for loading information about a revision was serialised to avoid the potential for contention over the cached checkout this procedure uses. There's still a lock used now, but I realised when looking in to this further that it's only necessary to lock around this specific call, not the larger section that was restricted previously. I think that add-to-store which this procedure uses isn't atomic for a directory, so there's a risk of weird results if the repository is changed after the required revision is checked out. While it isn't common to run guix pull twice, I think this could happen there if you ran guix pull concurrently for the same repository, but two different profiles. I added a sleep call just prior to the add-to-store call in latest-repository-commit, and tested running guix pull twice at roughly the same time, with different branches and profiles, and I did see both profiles then reflecting a single branch, so one profile was mistakenly referring to the wrong branch. Is this something that is worth guarding against? Maybe latest-repository-commit could double check the Git repo state after add-to-store completes, and raise an error if it's different to what it expects. Or perhaps individual worktrees could be used for each process, which would hopefully avoid the race condition entirely. Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl5a1gpfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9Xf3UA/+NdCxMpW7+FdotUCjaBooPSemcBKvU1T9prVnwowrNZz/adpRNHp1tfKD x+AkdMBFedoxF7wmNCR1wW8PcAp1T65ZBz/vo4y3Q3f9lF45cYcW3jDr7AoGqfqd FM4icF0htO1EVut7DImGK5tFEGe5wXo5KhlSNN99FFp7aBEnPfo0S5jzBbC3Ajw5 GtUSOePFiuzmaJCgsO+R1hRyg/YPOYMFrefIfDk1ZZVx6857cz5W0/+6LHO362bY HKnlESNNpHCZnjcvWDsTTZ92PluvgcyBreqJJs+kWWrCITHmdhFbhPlCkuN/8dY7 hNmpdXxZAbba60rjjkQjE/QkeeYWvKawLA5Tvs7pvdhzJenH4oIAOXD8Mp3EuaLl mV2pZGoHFD03q6OtGUZHZBkUGw5im4x7iIDZIMNWcVew14V2S12EVywky2B82lmp UI3R2d6UACuKQUYXLCO0uEGT1PC0yjH6i98RhYFNFfM8RoQsljUbSD0KM0ZJrped Qf7G0/eXUd8c9+JErIuE1yQV6a/Q5i7hscsaxp6MZwkkufrf1LalrdhEfMAPnQxN zOOrL2XIFvTvOyuoyUypLTQCt7KVIuzKn83yIWPRpaPMN7TMjNDUZXiECdI6YEdF 0X9n92XxcTL8gvOgrTkniLeIywTy6n6EP2xgw2Ty+5kZL4JLCeg= =SILE -----END PGP SIGNATURE----- --=-=-=--