From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id KBc1NkBCUGdP5QAAe85BDQ:P1 (envelope-from ) for ; Wed, 04 Dec 2024 11:51:29 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id KBc1NkBCUGdP5QAAe85BDQ (envelope-from ) for ; Wed, 04 Dec 2024 12:51:29 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=f5gMBThH; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1733313088; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=xkQU3c9df9S4lpXUs+XbZbEoZbpbNrxlCX5nlFuWoFw=; b=Y5ibd2VmSxVSsyB8yrF7O9AFtz3mYfsqapSnudzgH3Lpi7jVLLMfYPI94w679dFdKFPiXH qAqqhMnrAAdfhM5OLkknB0/BtRxmNmeCN97Cakb3oiigcCN6tFnMsuN4sb9Yz94in9hkkx IdfoU9DE8SiJBf+duZDhsQoE+xVCs79QHEb+tMrcOouXygYflUe2RSWiK2D7I/VFybBKgp 7CUabVELY5fBGvP0SKXF1NGEreS6Z/t+pY+psFO2pl9VWH3jiqxwQgHlJbqz2KDhI8t3gz Sm/W6NEw+CFhRbURthVEPF3fSmcM7oOFEOTnP0EzEP1vylsJG2HefiBN+vqfGw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=f5gMBThH; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=key1; d=yhetil.org; t=1733313088; a=rsa-sha256; cv=none; b=EugEphBPUNJ5ylqm92/WiYIWfwhkIhk95CDSqKV/JsKb1k539VyjHOhLdzLdyawjTZD8ex ts+6v0HTYjr0SHAJffzYlm9BsdXSUHSEx+qD9QMQlipJlnCCMIVEx8ixZnnjvxwo3JXZHK B8bYg2H4iiQklKNhH6qhWh5SKhO3zd6OUpcEbhEkISL+C6xKrAaTWK1KkH8GAtVOVHOklj xOK6cOeAGS5GCv/ItPyhGPPczD6idh3LKmNplueoLdHKY1qHGMvmi92AiptaJemNVLuiPo 34kecvgQHNYbtLkWmE2RdVE+fB6Z7gjc0OKn+0DJxV+0B3bwXTPMFuncYJvA4A== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 751AE7CF0B for ; Wed, 04 Dec 2024 12:51:28 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tInuK-0004Eb-8l; Wed, 04 Dec 2024 06:51:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tInuF-0004EJ-RS for help-guix@gnu.org; Wed, 04 Dec 2024 06:50:59 -0500 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tInuE-0001Sn-8C for help-guix@gnu.org; Wed, 04 Dec 2024 06:50:59 -0500 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-7242f559a9fso6527393b3a.1 for ; Wed, 04 Dec 2024 03:50:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733313055; x=1733917855; darn=gnu.org; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=xkQU3c9df9S4lpXUs+XbZbEoZbpbNrxlCX5nlFuWoFw=; b=f5gMBThHr2cZYlvyAyWJZvmHWuqEj+gszFjACog7CT9mUZHfvpBDNr4KcsEDGE2JMy TVtK3HCyRmqygiZ5you2hqqeHNjqI8F7wTD5b8hjDx8XzbBScss1/pO1cga7zgT2jQY7 eJK4YO4jPDVeIaTYGziNxeSl5hEnMAXwwUkX5pEMdCm0fdY5sNYbvtKhfrDBZD5OXjie YkFZSswhcaMx5jUV2GYb7feTqUUIpeentzlkZr6e9nRokq96TNNLkioLFm9Ct6AJkbti hQOpnnibhXsL0uDVM7EIF3OL0wDjtoOvYQt+lylTl+1HtMM4ExSsWtDRR1aJy5j5kjiI 8pUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733313055; x=1733917855; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xkQU3c9df9S4lpXUs+XbZbEoZbpbNrxlCX5nlFuWoFw=; b=gRgS8EM7rYYyAN4p1Py7kgsWGTo1UpBpCYVkNSmzf/EcpkQ4f6Ievc54bZMH9FQwTQ BHXO2QFfkFXDp2VmmVk/oKEyLG+rpEhcnd6PX2KLFq1Wrw2+PL+Btun1ZHCKIk0aPLp/ f2fMNqiF/CzKU19cZUUt5aGYISENvTz+x+O6FB+7Jv0Kei9C+drCVaOHVeyiwlJE3JtU 9jYRirjA5wf+D7VyxksBOOTfJ5XUAUOJ6GbXOoz1LSAnPYjzYIC7rOj5FjDkrisl8sx4 61QXQ3Jfro0/27EAEIDiYlIhr2csSycv5tWCMVruNVLxOAoo9Zeogk0sWMkOLR6uG6gj zqWA== X-Forwarded-Encrypted: i=1; AJvYcCVJDNN91lt2mvI3r+0gpb/DSz0sn0SNpGu1x8YVMDR+gwyXKdGSbmwbM09jj+3T5qwi58qn9wYDoCs=@gnu.org X-Gm-Message-State: AOJu0YzPlf7QIy/s7dlleT1y5tgCh3Nf1efkI2A5pJpxzMoWBCw7FBhC K1rMQAo3HHF49lPxceRgH0q7EgdlhdOyTH8vJrbIhXZVzKxrZZF+jbQXUw== X-Gm-Gg: ASbGncvSAZWbHfLMyiWd79iblhFt9Pq6HdcIFrl56UOVQP54LBfrCWcCU3oI9rlm1MK a7O/7/AEd4nBbV80DxGgnyDz19bBeGE+kmL2bj/WI0Y5WlCYWhay1Q413jt/BgHTZ/J5U1qum2p 86o7/0AfqxLfCIcj3ClsKFMPO77c41LVX10MW+pfVWCIbp7WEONST24DXOpGmUI2ZZgNxg9e08O hIYLEGAW3ZwT+NbmcesgNRaHMwpp73SB3gq+El7pNPL4D0= X-Google-Smtp-Source: AGHT+IGHv6XSniLVmFW8VjQTxc0upEFu4MMH8YpyXgGD38BXxneQVHIe9E3TEDd2YUqCn4AeAioV8g== X-Received: by 2002:a05:6a00:3998:b0:71e:5e04:be9b with SMTP id d2e1a72fcca58-7257fa70dc6mr7896545b3a.12.1733313054828; Wed, 04 Dec 2024 03:50:54 -0800 (PST) Received: from terra ([2405:6586:be0:0:c8ff:1707:9b9:af89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417614basm12190681b3a.11.2024.12.04.03.50.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2024 03:50:53 -0800 (PST) From: Maxim Cournoyer To: Christoph Buck Cc: Zack Weinberg , Efraim Flashner , help-guix@gnu.org Subject: Re: ABI mismatch on boot on arm32 system In-Reply-To: <87jzdg4q1y.fsf@icepic.de> (Christoph Buck's message of "Wed, 06 Nov 2024 11:25:29 +0100") References: <87y12opdbh.fsf@icepic.de> <87iktmhk6v.fsf@icepic.de> <86b5aa0a-35ce-40ee-84a6-de58ea153fbf@app.fastmail.com> <87a5eyheme.fsf@icepic.de> <87o73dvkz6.fsf@icepic.de> <87wmhqesw0.fsf@icepic.de> <87plnh67w5.fsf@icepic.de> <87jzdg4q1y.fsf@icepic.de> Date: Wed, 04 Dec 2024 20:50:44 +0900 Message-ID: <87y10vll9n.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=maxim.cournoyer@gmail.com; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: help-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx11.migadu.com X-Migadu-Spam-Score: -2.38 X-Spam-Score: -2.38 X-Migadu-Queue-Id: 751AE7CF0B X-TUID: 9w41yUfxtV2B Hi Christoph, Sorry for my delayed reply. Christoph Buck writes: > Hi Guix! > > So i looked into the guile source code and, as expected, the `scm_hash` > function (see hash.c in guile) uses `unsigned long` wich is 8 bytes on > x64 and 4 bytes on arm32/i868. If `string-hash` is called with the size > parameter `n`, the hash value is limited to size by calculating the > modulo `n` of the hash value, see scm_ihash in hash.c:440, namely > >> (unsigned long) scm_raw_ihash (obj, 10) % n > > (The `10` can be ignored as far as i can tell). Since the hash values > are different on different platforms the modulo is different as well. > > However, if one steps through the call stack of `string-hash` you can > see that the actual hash value is calculated by the > `JENKINS_LOOKUP3_HASHWORD2` macro, which contains are rather > interesting comment and a possible workaround for the abi problem, > namely > > /* Scheme can access symbol-hash, which exposes this value. For \ > cross-compilation reasons, we ensure that the high 32 bits of \ > the hash on a 64-bit system are equal to the hash on a 32-bit \ > system. The low 32 bits just add more entropy. */ \ > if (sizeof (ret) == 8) \ > ret = (((unsigned long) c) << 32) | b; \ > else \ > ret = c; \ > > > in hash.c:82. > > Meaning, if executed on a x64 platform, the higher 32bit of the > resulting 64bit hash result are equal to the hash value on a 32bit > platform. A simple test case in c++ looks like this: > > int main(int args, char** argv) > { > scm_init_guile(); > auto strToHash = scm_from_locale_string ("((device) (mount-point))"); > auto maxULong = scm_from_ulong(ULONG_MAX); > auto hashResult = scm_hash(strToHash,maxULong); > auto hashResultUL = scm_to_ulong(hashResult); > std::cout << "Max ULONG_MAX: " << ULONG_MAX < std::cout << "Original hashResult ulong: " << hashResultUL << std::endl; > > if(sizeof(hashResultUL) == 8) > { > std::cout << "Corrected for 32bit: " << (hashResultUL >> 32) << std::endl; > } > } > > > which results on x64 in > >> Max ULONG_MAX: 18446744073709551615 >> Original hashResult ulong: 10454028974864831 >> Corrected for 32bit: 2434018 > > and on arm32 to > >> Max ULONG_MAX: 4294967295 >> Original hashResult ulong: 2434018 > > This suggest the following workaround. Always limit the hash size to > 32bit even if executed on a 64bit platform (or to be more specific a > platform where ulong is 8bytes big). Do this by right shift the hash > value 32bits and don't rely on the size parameter of the `string-hash` > function. > > In code it could look something like this > > (define (compute-abi-cookie field-specs) > ;; Compute an "ABI cookie" for the given FIELD-SPECS. We use > ;; 'string-hash' because that's a better hash function that 'hash' on a > ;; list of symbols. > (let ((hash > (syntax-case field-specs () > (((field get properties ...) ...) > (let ((hash-value (string-hash (object->string > (syntax->datum #'((field properties ...) ...)))))) > (if (= (native-word-size) 8) > (ash hash-value -32) > hash-value))))) > (fd (syntax-case field-specs () > (((field get properties ...) ...) > (object->string > (syntax->datum #'((field properties ...) ...))))))) > > (format #t "Compute-abi-cookie: ~a~%" hash) > hash)) > > > where `native-word-size` is define by > > (define (native-word-size) > ((@ (system foreign) sizeof) '*)) This is a thorough investigation, and the above fix/workaround that can be applied to Guix is the cherry on the cake! Thank you for producing it. > (taken from `cross-compilation.test`). There might be a cleaner way to > formulate this, but you get the point. > > This seems to work for all combinations on my machine. I tested > x64 -> arm, x64 -> i868, i868 -> x64... > > I can only think of two drawbacks. > > 1) Lost entropy on 64 bit machines > 2) Abi break because on new compilation the hash values on 64bit > platforms will change. > > 1) is imho irrelevant, because it is not cryptophically important. For > 2) i am not sure how important this is. > > Any thoughts on this? I don't think 1 or 2 are too important; about 2 for example, we break the ABI every time we add or remove a new record field. > Might this be something worth fixing and sending a patch in? Totally, if you haven't done so already. -- Thanks, Maxim