From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Bj=C3=B6rn?= Bidar via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#74413: [PATCH] Allow to store and read repository information of VCS builds Date: Tue, 26 Nov 2024 17:35:58 +0200 Message-ID: <87mshmdn35.fsf@thaodan.de> References: <86frnovhg8.fsf@gnu.org> <867c90vbfk.fsf@gnu.org> <673c31c6.a70a0220.18f62b.10d8SMTPIN_ADDED_BROKEN@mx.google.com> Reply-To: =?UTF-8?Q?Bj=C3=B6rn?= Bidar Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="15979"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: luangruo@yahoo.com, 74413@debbugs.gnu.org, Eli Zaretskii To: Stefan Kangas Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Nov 26 16:37:30 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tFxd3-00042K-U7 for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 26 Nov 2024 16:37:30 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tFxcm-0007xp-80; Tue, 26 Nov 2024 10:37:12 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tFxce-0007wz-HW for bug-gnu-emacs@gnu.org; Tue, 26 Nov 2024 10:37:04 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tFxcc-0001K6-3I for bug-gnu-emacs@gnu.org; Tue, 26 Nov 2024 10:37:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:References:In-Reply-To:From:To:Subject; bh=bNg4C+gbd34d4cYs1LnpstJTPz06KZgqApeK6ulXJJs=; b=kcplX5k7rZMqpXfxcgf6DopvWeAtdU9z4VFJ0g0/rJymHNtNeChkCuShyHwE7U+S7cPBeCrh1NrlNHZlFd5Vm/WiHmdzgZiZCq+kffb0ikCvGwfTLbleargzWW7yXdtzg4hlPuTpTtauOiCubpUECiguQBZxCy2ehhfl3gburWpYC/wR9F4TB2swBDuus09J/21JJJsRfABavSMLgK+Se8Hml0JYUQGAWw+B/IQzTt9TCysurstn1Vvs60EWLzuRyAu8WhfM095GjpatJD6sU/9b5tCzYxS3DHbRiL6PVEUL9y4/sLqF+WsdU7Kmj31kt5GxMxMxvqaGvr5NOlWMvQ==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tFxcb-0000B3-UO for bug-gnu-emacs@gnu.org; Tue, 26 Nov 2024 10:37:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: =?UTF-8?Q?Bj=C3=B6rn?= Bidar Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 26 Nov 2024 15:37:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 74413 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 74413-submit@debbugs.gnu.org id=B74413.1732635365605 (code B ref 74413); Tue, 26 Nov 2024 15:37:01 +0000 Original-Received: (at 74413) by debbugs.gnu.org; 26 Nov 2024 15:36:05 +0000 Original-Received: from localhost ([127.0.0.1]:50081 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tFxbh-00009h-3P for submit@debbugs.gnu.org; Tue, 26 Nov 2024 10:36:05 -0500 Original-Received: from thaodan.de ([185.216.177.71]:54770) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tFxbd-00009F-RS for 74413@debbugs.gnu.org; Tue, 26 Nov 2024 10:36:04 -0500 Original-Received: from NordStern (unknown [185.252.118.71]) by thaodan.de (Postfix) with ESMTPSA id 5E18BD00057; Tue, 26 Nov 2024 17:36:00 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=thaodan.de; s=mail; t=1732635360; bh=ACnm+D0O9gr4NK+1s/xB/Kzl6npHxrRuBehCW6GuGYM=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=g7/kJLcm2UJict3z807scTLQWtR7o3CmSYcQCH5ntOa+67u/v0Kkj/y9lM+uL740U 38TNZJo5e1yAQMeGwboO1Mk7xCiocArz8g+KrOvAGPrXlRnnwYJfaLYapWNOelhoFe 4/qWWFW2wLvG7DJ65GzTFstqn0s1D66WljBhS7pglYNsZLiyThpoUN/vu4obWFEPTG bqiLIHtporitBbCkgpUMMUqb6YnRfnUYls89E6mHMTuIb90/8S0qmnmBm1mpYAYTCo GrgTb56kHm4ggjjg3wwHYyNULQHyrth5Tgjtm9m7IQw7vgz650paYJGhNXNzev64SU UhnOcHh4qrBRlhm+7MFA79NiDGV16t0tR1XiFclaFtvtXsiw96O1kYj+xbGTf+WUlT cW9Xn+cyxIQeoN0qAmv37KX4aBS9Mmlpy3jQ2q6CfICZ7YaFY/Vqhl1H4l1hBKc4dC a2EjurnRp1qpg9jdHoaQn+5GpgfdSXMg4ccdB6DdohDPf4e6zi/6zeebROCob/6Gsj eBJ6cr2nGjRRYQw06F/1ee63lG3hTUsQVrNs3IGKvpMXX8xmCLqS4ktqB8eAu/Uczo PJubEBBh6tz96mERHVsBM0GMNrTChPO1i87WLM5AEWMc9s2BlUxTp4hhPi4wmXgBYF CLHWy+ZfkD1s05LIusHk2uE0= In-Reply-To: (Stefan Kangas's message of "Mon, 25 Nov 2024 15:43:41 -0800") Autocrypt: addr=bjorn.bidar@thaodan.de; prefer-encrypt=nopreference; keydata= mDMEZNfpPhYJKwYBBAHaRw8BAQdACBEmr+0xwIIHZfIDlZmm7sa+lHHSb0g9FZrN6qE6ru60JUJq w7ZybiBCaWRhciA8Ympvcm4uYmlkYXJAdGhhb2Rhbi5kZT6IlgQTFgoAPgIbAwULCQgHAgIiAgYV CgkICwIEFgIDAQIeBwIXgBYhBFHxdut1RzAepymoq1wbdKFlHF9oBQJk1/YmAhkBAAoJEFwbdKFl HF9oB9cBAJoIIGQKXm4cpap+Flxc/EGnYl0123lcEyzuduqvlDT0AQC3OlFKm/OiqJ8IMTrzJRZ8 phFssTkSrrFXnM2jm5PYDoiTBBMWCgA7FiEEUfF263VHMB6nKairXBt0oWUcX2gFAmTX6T4CGwMF CwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQXBt0oWUcX2hbCQEAtru7kvM8hi8zo6z9ux2h K+B5xViKuo7Z8K3IXuK5ugwA+wUfKzomzdBPhfxDsqLcEziGRxoyx0Q3ld9aermBUccHtBxCasO2 cm4gQmlkYXIgPG1lQHRoYW9kYW4uZGU+iJMEExYKADsCGwMFCwkIBwICIgIGFQoJCAsCBBYCAwEC HgcCF4AWIQRR8XbrdUcwHqcpqKtcG3ShZRxfaAUCZNf2FQAKCRBcG3ShZRxfaCzSAP4hZ7cSp0YN XYpcjHdsySh2MuBhhoPeLGXs+2kSiqBiOwD/TP8AgPEg/R+SI9GI9on7fBJJ0mp2IT8kZ2rhDOjg gA6IkwQTFgoAOxYhBFHxdut1RzAepymoq1wbdKFlH X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:295993 Archived-At: Stefan Kangas writes: > Bj=C3=B6rn Bidar writes: > >>> I don't fully understand how you can have a situation where you can get >>> this information in the Makefile, but you can't also get it when dumping >>> using `emacs-repository-get-version` and `emacs-repository-get-branch` >>> (lisp/loadup.el:474). Could you please elaborate on this? >> >> I added the Makefile part so it would work outside of my environment if >> one builds Emacs and then distributes/install it without the sources or >> without them containing the VCS information while git is also installed >> in their build environment. > > I don't think I understand what you're saying here. Is it possible that > there are some typos above? I'm thinking of the part that contains the > tautology "without the sources or without them". I meant to say that if a user builds with Git installed and Emacs sources=20 checked out from git. Then packages Emacs but only puts the checkout sources without the VCS information into the source package/tarball. > There are also some words where the meaning is not entirely clear to me, > such as "distributes" and "installs". Distributed in this case meant to be the package build put into a repository and then installed using a package manager. > Maybe it will be clarified by the answers to my below questions. > >>> Do you mean that you have one containerized process with Git that clones >>> emacs.git into a directory, and then an entirely separate containerized >>> process, without Git, builds Emacs from that very same directory? Or >>> something along those lines? >> >> Yes more or less. >> I'm using an obs source service that generates a tarball from VCS's and >> then creates a tarball. [1] > > I took a look at the linked documentation, but it seems to be on a very > detailed level, and does not provide a good overview. So I don't think > I understood much from it. > > I'm not familiar with the Open Build System, so this is quite hard for > me to follow. Please be patient with me. > > In your description above, could you explain what is the difference > between the two steps "generates a tarball from VCS" and "creates a > tarball"? It should have be more detailed or used the proper word. The program that I use to generate the sources fetches the sources from a VCS such as git and then creates a cpio archive (for storing the sources more efficiently but that's beyond this topic). The cpio archive is then converted to a tar archive and compressed. > Do you mean to say that it: > 1) generates a tarball from emacs.git, excluding the .git directory > 2) copies that tarball to a completely different environment > 3) builds Emacs Essentially yes. The point of this is that the environment that builds the packages is disconnected from internet or other external factors for security reasons. The tarball is copied into the build environment while it prepares it so that a minimal linux vm or chroot can start to build software. > The part that I do not understand is in which step you run `make`, and > how. Is it run in the first step, the third, the first and the third, > or something else? Is it run automatically by OBS, or do you have to > specify it? I run make in in the 3 step right after the environment is setup up inside rpmbuild. Before make is called I parse the file that contains the reference hash and the file that contains the configuration for the program that generates the sources which contains the branch that the source where generated from into the file that Emacs then can read during the build process and after that when the package was installed. The only part that's special here is that I generate the version file from the information gathered while generating the source tarball as opposed to using git to gather those. > Basically, I still don't understand the answer to my below question: > >>> In this very particular scenario, isn't it enough to add this >>> additional step to your CI pipeline: >>> >>> ./src/emacs -Q --batch --eval '(princ (format "%s:%s" \ >>> emacs-repository-version emacs-repository-branch))' > version.info >> The obs source service already contains that information see the >> emacs.obsinfo file I mentioned earlier. > > Hmm. Would an alternative solution be to read the version information > from that file then, if it exists? > >>> In other words, is it really necessary for us to support this use case >>> in our Makefile? Do we expect that building Emacs in such CI pipelines >>> using non-released development version of Emacs will be very common? >> >> The Makefile target is there already for the Android builds all I did I >> added it to other platforms too. If the target would be shared then the >> hole process is reused. > > If we want this for other builds, then reusing what we do for the > Android build sounds prudent. It is clear that this is needed for > Android. What remains to clarify is why it is needed elsewhere. > >> I think it would become more common to build from VCS sources would be >> become more common where the information is useful. > > Building from VCS sources is already quite common. I'm asking something > more specific: will it be increasingly common to build using an > environment very similar to OBS? I think so. It's already quite common to build software in a much simpler CI such as Gitlab CI. Fedora has their own CI similar to OBS for ex= ample. > If it is not going to be very common, this should perhaps be fixed in > the OBS pipeline directly, instead of complicating our Makefile. But > I'm not yet clear on whether that is true. The makefile only has the rule add or moved for the version file. To me this just looks like refactoring so the logic that was added for Android can used on other platforms. >> To get more users to test earlier development builds to have faster >> general feedback and bug reports reports it is very useful to have >> non-released development builds. There are other instances outside my >> own usecase where this is frequent such as for example the AUR emacs-git= , guix >> emacs-next or the Android Fdroid builds. > > Are you saying that AUR emacs-git, guix emacs-next, and the Android > Fdroid builds also have no way to know which revision Emacs was built > from without your patch? The Android builds obviously have the revision already integrated as the logic that I'm trying to reuse here have been already implemented for Android. I don't know about the Guix build process but for Arch git isn't into the change-root when building Emacs from the tarball that has been generated before hand. The OBS can build packages for all kinds of Linux distributions such as Flatpak, Debian, Fedora, SUSE, Ubuntu etc. So this change can help building test builds for all these distributions in theory. Personally just building for SUSE. >>> BTW, what is the name of that CI system that you're using here? For >>> what purpose are you building Emacs: to test Elisp packages, to test >>> Emacs, or something else? Finally, why are you not using officially >>> tagged versions, either from us or from some distro, in this context? >> >> I'm build Emacs on the open build service. I'm building Emacs on the obs >> to provide development builds that can be used to test Emacs and submit >> bug reports. >> The package is build using the official RPM packaging just >> with the newer sources from git. >> >> The initial reason reason was that I noticed that even if you build from >> git directly without a CI/or something similar to the source service >> mentioned earlier that the functions to retrieve the repository >> information don't work since the source path either doesn't exist on any >> the machine that installs the final package or that it doesn't contain >> the VCS information anymore. >> The setup is very similar to the Android builds. > > Thanks. No problem.