unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* [GCD] Set search paths without program wrappers
@ 2025-02-02  4:57 宋文武 via Development of GNU Guix and the GNU System distribution.
  2025-02-03 16:02 ` Simon Tournier
  0 siblings, 1 reply; 3+ messages in thread
From: 宋文武 via Development of GNU Guix and the GNU System distribution. @ 2025-02-02  4:57 UTC (permalink / raw)
  To: guix-devel; +Cc: Maxim Cournoyer

[-- Attachment #1: Type: text/plain, Size: 241 bytes --]

Hello, when trying to fix some issues caused by program wrappers in
<https://issues.guix.gnu.org/75688>, it was suggested that I should
discussed the idea and implementation more broadly through a GCD
document.

So here is a draft version:


[-- Attachment #2: 003-set-search-paths-without-program-wrappers.md --]
[-- Type: text/plain, Size: 6500 bytes --]

title: Set search paths without program wrappers
id: 003
status: draft
discussion: https://issues.guix.gnu.org/<number assigned by issue tracker>
authors: 宋文武 <iyzsong@envs.net>
sponsors: Maxim Cournoyer <maxim.cournoyer@gmail.com>
date-submitted: <date when you send the first draft>
date: <date when the discussion period starts>
SPDX-License-Identifier: CC-BY-SA-4.0 OR GFDL-1.3-no-invariants-only
---

# Summary

Currently program wrappers are widely used to set search paths via
environment variables.  Those wrappers have some problems:

  - environment variables leakage from a process to its child
    processes;
  - duplicate entries in environment variables;
  - obscured process names.

To address those problems, we propose a way to set search paths with
some per-output configuration files, reduce the need of creating
program wrappers.


# Motivation

To make sure programs work out-of-the-box rather then depend on some
external settings, Guix encourages the use of [program wrappers](https://guix.gnu.org/manual/en/html_node/Build-Utilities.html#Wrappers)
when define packages.  In particular,  both `glib-or-gtk-build-system`
and `qt-build-system` includes a wrap phase to make program wrappers for every
GNOME and KDE program.

Those wrappers have some unsolved issues:

- [Program crash due to leaked environment variables](https://issues.guix.gnu.org/63203)
- [Duplicate entries in various environment variables](https://issues.guix.gnu.org/23118)
- [Ansible & others' problems with wrapped '.ansible-real' scripts](https://issues.guix.gnu.org/26752)

If we managed to find a way to set search paths without using program
wappers, then programs will be more robust.


# Detailed Design

In addition to environment variables, some programs also allow to set search
paths via configuration files, for example you can use
[path configuration files](https://docs.python.org/3/library/site.html)
to set `sys.path` for Python, and we have a per-output [`ld.so.cache`](https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/)
to load shared libraries efficiently.  Inspired by them, we are going to patch
some programs and libraries, so that when they build a search path from an
environment variable, would also honor a per-output search path configuration
file.  The details are how to make those search path configuration files and
how to find them when an executable is running.

## Search path configuration files

We'll create search path configuration files under the `etc/search-path.d`
directory of each package output, with each file specify a search path.
The file name and its content are same to the corresponding environment
variable.  For example the output of the `gnome-console` package would have:

```
bin
  kgx
etc
  ld.so.cache
  search-paths.d
    GUIX_XDG_DATA_DIRS
    GUIX_GIO_EXTRA_MODULES
    GUIX_GTK4_PATH
lib
share
```

The content of its `GUIX_XDG_DATA_DIRS` file would be:
```
gnu/store/qdrx44hbnnhd979higdjrm4mi87ipn9f-shared-mime-info-2.3/share:/gnu/store/2266wqwigxmsm3vh8vjqhqbmxdacy9c0-glib-2.78.0/share:/gnu/store/7g2cz0pna9ndvxx4d8sm8lnkxjxg4vnz-gsettings-desktop-schemas-44.0/share:/gnu/store/nak2cv7r3rmsnqjzqy8116xn43rvh8c0-libadwaita-1.5.2/share:/gnu/store/y9ig70jqr1j99q69akvqhv0ij44hmfii-gtk-4.14.5/share:/gnu/store/7h0xh6rd88g37s10g67gym7cd5l0lb77-gnome-console-44.4/share
```

Those search path configuration files would be created by the package builder,
after the `install` phase, replace usages of `wrap-program` when possible.


## Find the location of the current executable

To find its search path configuration files when an executable is running,
we can first find the location of the executable.  Conveniently, Linux
provides a pseudo-file `/proc/self/exe` for this exact purpose, which works
well for ELF executables.   But for an interpreter script, `/proc/self/exe`
would return the file name of its interpreter instead of the script, so
we patch interpreters to set 2 environment variables:

  - `GUIX_INTERPRETER_FILE`: absolute file name of the interpreter
  - `GUIX_MAIN_SCRIPT_FILE`: absolute file name of the script

And when the executable's `/proc/self/exe` matches `GUIX_INTERPRETER_FILE`,
we can get the script file name from `GUIX_MAIN_SCRIPT_FILE`.  Alternatively,
we can try to construct the script file name from command line arguments, but
that won't work when you run a script using a relative file name and its
current working directory changed before we figure out the script file name.


## Implementation plan

A WIP implementation can be found in <https://issues.guix.gnu.org/75688>.

- Add a new function `g_guix_build_search_path_dirs` to GLib, which returns a
  search path as a list of file or directory names from a search path
  configuration file and an environment variable.
- Patch GLib to use `g_guix_build_search_path_dirs` for `GUIX_XDG_DATA_DIRS`,
  `GUIX_XDG_CONFIG_DIRS`, `GUIX_GIO_EXTRA_MODULES` and
  `GUIX_GSETTINGS_SCHEMA_DIR`.
- Patch Python to set `GUIX_INTERPRETER_FILE` and `GUIX_MAIN_SCRIPT_FILE`.
- Patch Qt to use `g_guix_build_search_path_dirs` for `GUIX_XDG_DATA_DIRS`,
  `GUIX_XDG_CONFIG_DIRS`, `GUIX_QT_PLUGIN_PATH`, `GUIX_QML_IMPORT_PATH`,
  `GUIX_QML2_IMPORT_PATH`, `GUIX_QTWEBENGINEPROCESS_PATH`.
- Modify `glib-or-gtk-build-system` to get rid of `wrap-program`.
- Modify `qt-build-system` to get rid of `wrap-program`.


# The Cost Of Reverting

By limiting the creation of search path configuration files in build systems,
we can revert to program wrappers by manually adding wrap phases on a case by
case basic, if needed.


# Drawbacks or Open Questions

If implemented, we would likely carry several custom patches for GLib,
Qt, Python, etc. forever, and programs could fail in some subtle ways.

This proposal focuses solving problems caused by program wrappers in desktop
environments, namely GNOME and KDE.  Individual `wrap-progam` usages are not
addressed.  We plan to handle that in build systems later, for example:

  - Handle `GUIX_GI_TYPELIB_PATH` and `GUIX_GDK_PIXBUF_MODULE_FILES` in
    `glib-or-gtk-build-system` without wrappers.
  - Handle `GUIX_PYTHONPATH` in `python-build-system` without wrappers.


There are still ABI problems caused by environment variables from
profiles, which may be addressed later as suggested by Maxime Devos in
<https://issues.guix.gnu.org/63203#5>.

[-- Attachment #3: Type: text/plain, Size: 32 bytes --]



Feedbacks welcome, thank you!

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [GCD] Set search paths without program wrappers
  2025-02-02  4:57 [GCD] Set search paths without program wrappers 宋文武 via Development of GNU Guix and the GNU System distribution.
@ 2025-02-03 16:02 ` Simon Tournier
  2025-02-04  1:23   ` Maxim Cournoyer
  0 siblings, 1 reply; 3+ messages in thread
From: Simon Tournier @ 2025-02-03 16:02 UTC (permalink / raw)
  To: 宋文武, guix-devel; +Cc: Maxim Cournoyer

Hi,

On Sun, 02 Feb 2025 at 12:57, 宋文武 via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> wrote:

> Hello, when trying to fix some issues caused by program wrappers in
> <https://issues.guix.gnu.org/75688>, it was suggested that I should
> discussed the idea and implementation more broadly through a GCD
> document.

I’m happy that GCD covers a need!  And that it’s widely supported.  But,
could we wait a couple of days and thus have all in place before
shooting? :-)

The date for the final acceptance is [1]:

    The Deliberation Period ends on February, 5th everywhere on Earth.

Therefore, let wait to have the GCD repository in place with the proper
README and so on; all based from the experience of the GCD 001
itself. :-)

Somehow, Ludo’s been too eager for discussing “forge“ on a concrete
proposal.  Heh!

Again, I’m really happy and we’re speaking about days. :-)  I hope to
complete all for the end of this week (between 6th and 7th Feb).

Cheers,
simon


1: [bug#74736] [FWD] Guix Consensus Document process – deliberation
Simon Tournier <zimon.toutoune@gmail.com>
Wed, 22 Jan 2025 20:49:55 +0100
id:87msfiljbw.fsf@gmail.com
https://issues.guix.gnu.org/74736
https://issues.guix.gnu.org/msgid/87msfiljbw.fsf@gmail.com
https://yhetil.org/guix/87msfiljbw.fsf@gmail.com


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [GCD] Set search paths without program wrappers
  2025-02-03 16:02 ` Simon Tournier
@ 2025-02-04  1:23   ` Maxim Cournoyer
  0 siblings, 0 replies; 3+ messages in thread
From: Maxim Cournoyer @ 2025-02-04  1:23 UTC (permalink / raw)
  To: Simon Tournier; +Cc: 宋文武, guix-devel

Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi,
>
> On Sun, 02 Feb 2025 at 12:57, 宋文武 via "Development of GNU Guix and
> the GNU System distribution." <guix-devel@gnu.org> wrote:
>
>> Hello, when trying to fix some issues caused by program wrappers in
>> <https://issues.guix.gnu.org/75688>, it was suggested that I should
>> discussed the idea and implementation more broadly through a GCD
>> document.
>
> I’m happy that GCD covers a need!  And that it’s widely supported.  But,
> could we wait a couple of days and thus have all in place before
> shooting? :-)
>
> The date for the final acceptance is [1]:
>
>     The Deliberation Period ends on February, 5th everywhere on Earth.

[...]

Sure, that's coming soon; consider this one a tentative draft if that
helps :-).

-- 
Thanks,
Maxim


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2025-02-04  1:24 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-02  4:57 [GCD] Set search paths without program wrappers 宋文武 via Development of GNU Guix and the GNU System distribution.
2025-02-03 16:02 ` Simon Tournier
2025-02-04  1:23   ` Maxim Cournoyer

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).