From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: sbaugh@catern.com Newsgroups: gmane.emacs.devel Subject: Re: Managing environments (Python venv, guix environment, etc.) Date: Wed, 20 Jul 2016 20:32:02 -0400 Message-ID: <874m7jygot.fsf@earth.catern.com> References: <87y453sy0n.fsf@earth.catern.com> <87r3arripr.fsf@earth.catern.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1469061158 30930 80.91.229.3 (21 Jul 2016 00:32:38 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 21 Jul 2016 00:32:38 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Jul 21 02:32:29 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bQ1ud-0000Qb-DF for ged-emacs-devel@m.gmane.org; Thu, 21 Jul 2016 02:32:27 +0200 Original-Received: from localhost ([::1]:37595 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQ1uc-0004zH-4I for ged-emacs-devel@m.gmane.org; Wed, 20 Jul 2016 20:32:26 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:40832) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQ1uT-0004yx-4q for emacs-devel@gnu.org; Wed, 20 Jul 2016 20:32:18 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bQ1uN-0006nU-2c for emacs-devel@gnu.org; Wed, 20 Jul 2016 20:32:16 -0400 Original-Received: from plane.gmane.org ([80.91.229.3]:50836) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQ1uM-0006mm-Mp for emacs-devel@gnu.org; Wed, 20 Jul 2016 20:32:10 -0400 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1bQ1uK-0000HL-No for emacs-devel@gnu.org; Thu, 21 Jul 2016 02:32:08 +0200 Original-Received: from 71-46-80-67.res.bhn.net ([71.46.80.67]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 21 Jul 2016 02:32:08 +0200 Original-Received: from sbaugh by 71-46-80-67.res.bhn.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 21 Jul 2016 02:32:08 +0200 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 299 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 71-46-80-67.res.bhn.net User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) Cancel-Lock: sha1:MJFwH8ftJfMTu9dk5QW8QBHkwZE= X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 80.91.229.3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:205929 Archived-At: --=-=-= Content-Type: text/plain >> Does this approach (putting project root directories into >> file-name-handler-alist) sound like something that could be accepted >> into core Emacs? If so I'll get started. OK, I implemented a very rough prototype of this. It is attached. environment.el is the API that would go into Emacs, guix-environment.el provides a command to use an guix-provided environment, and test-environment.el is just a sequence of expressions to evaluate to demonstrate them. (Not tests) --=-=-= Content-Type: application/emacs-lisp Content-Disposition: attachment; filename=environment.el Content-Transfer-Encoding: quoted-printable Content-Description: environment.el ;;; environment.el --- Manage and enable specialized process environments = -*- lexical-binding: t; -*- ;; Copyright (C) 2016=20=20 ;; Author: ;; Keywords: unix, processes, tools ;; This program is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with this program. If not, see . ;;; Commentary: ;; BUG: If a file operation is called on something with a ~ in it (or ;; something otherwise unexpanded but still corresponding to part of ;; an environmentized directory) then we won't get called. ;; That's problematic, but can we maybe rely on things always being ;; called on canonical filenames? ;;; Code: (require 'cl-lib) (require 'cl) (defvar environment-path-alist nil "Alist of elements (PATH . ENV) for paths and special environments.") (defun environment-lookup-path (path) "Returns the environment for this path." (assoc-default path environment-path-alist #'string-prefix-p)) (defun environment-unassociate-dir (dir) "Remove any environments that might affect this dir." (setq environment-path-alist (delete-if (lambda (x) (string-prefix-p (car x) dir)) environment-path-ali= st)) (setq file-name-handler-alist (delete-if (lambda (x) (and (string-match (car x) dir) (eq #'environment-magic-handler (cdr x)))) file-name-handler-alist))) (defun environment-magic-handler (func &rest args) "Handler for file-name-handler-alist" (let* ((inhibit-file-name-handlers (cons #'environment-magic-handler inhi= bit-file-name-handlers)) (inhibit-file-name-operation 'directory-file-name) (env (environment-lookup-path (directory-file-name default-directory))) (inhibit-file-name-operation func)) (if env (let ((exec-path (cdr (assoc 'exec-path env))) (process-environment (cdr (assoc 'process-environment env)))) (apply func args)) (message "environment-magic-handler: Was invoked, but couldn't find a= n environment for %s" default-directory) (apply func args)))) (defun environment--add-handler (dir) "Add handler to file-name-handler-alist for DIR and its children." (let* ((dir (directory-file-name dir)) (regexp (rx-to-string `(and string-start ,dir) t)) (handler (cons regexp #'environment-magic-handler))) (push handler file-name-handler-alist))) (defun environment-associate-dir (dir environment) "Associate given directory with given environment." (let* ((dir (directory-file-name (expand-file-name dir)))) (environment-unassociate-dir dir) (push (cons dir environment) environment-path-alist) (push (cons (abbreviate-file-name dir) environment) environment-path-al= ist) (environment--add-handler dir) (environment--add-handler (abbreviate-file-name dir)) )) (defun environment-current-as-alist () "Returns the current process-environment as an alist." (cl-remove-if-not #'identity (mapcar (lambda (varval) (let ((index (string-match "=3D" varval))) (when index (cons (substring varval 0 index) (substring varval (1+ i= ndex)))))) process-environment))) (defun environment-create (alist) "Takes an alist of (VAR . VAL) and turns it into our internal environment= object. This is a pure function." (letf* (((symbol-function 'ensure-list) (lambda (x) (if (stringp x) (spli= t-string x ":") x))) ((symbol-function 'ensure-string) (lambda (x) (if (consp x) (mapconcat #= 'identity x ":") x))) (my-exec-path (ensure-list (cdr (assoc "PATH" alist)))) (my-process-environment (mapcar (lambda (x) (concat (car x) "=3D" (ensur= e-string (cdr x)))) alist))) (list (cons 'exec-path my-exec-path) (cons 'process-environment my-process-environment)))) (provide 'environment) ;;; environment.el ends here --=-=-= Content-Type: application/emacs-lisp Content-Disposition: attachment; filename=guix-environment.el Content-Transfer-Encoding: quoted-printable Content-Description: guix-environment.el ;gnus;; guix-environment.el --- Enter and manage guix profile environments = -*- lexical-binding: t; -*- ;; Copyright (C) 2016=20=20 ;; Author: ;; Keywords: unix, processes, tools ;; This program is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with this program. If not, see . ;;; Commentary: ;;=20 ;;; Code: (require 'cl-lib) (require 'environment) (require 'guix-profiles) (require 'guix-geiser) (require 'guix-backend) (defun guix-environment-get (profile) "Get the variable bindings for this profile" (guix-eval-read "(use-modules (guix search-paths))") (guix-eval-read (format "(let* ((profile \"%s\" ) (search-paths (delete-duplicates (cons $PATH (append-map manifest-entry-search-paths (manifest-entries (profile-manifest profile))))))) (filter-map (lambda (x) (cons (search-path-specification-variable (car x)= ) (cdr x))) (evaluate-search-paths search-paths (list profile) getenv)))" profil= e))) (defvar guix-environment-augment-path nil "When setting an environment, augment the path instead of replacing it") (defun guix-environment-set-dir (profile dir) "Get the variable bindings for this profile" (interactive (let ((profile (guix-profile-prompt))) (list profile (read-file-name (concat "Directory to associate with pro= file " profile " : ") nil nil 'confirm)))) (let ((env (guix-environment-get profile))) (when guix-environment-augment-path (setq env (cons (cons "PATH" (concat (getenv "PATH") ":" (cdr (assoc = "PATH" env)))) env))) (setq env (environment-create (cl-remove-duplicates (append env (enviro= nment-current-as-alist)) :key #'car))) (environment-associate-dir dir env))) (provide 'guix-environment) ;;; guix-environment.el ends here --=-=-= Content-Type: application/emacs-lisp Content-Disposition: attachment; filename=test-environment.el Content-Transfer-Encoding: quoted-printable Content-Description: test-environment.el (progn (setq my-special-dir "~/special") (setq my-special-path '(,(concat (getenv "HOME") "/bin/") "/usr/bin/" "/b= in/")) (setq my-special-env (environment-create `(("PATH" . ,my-special-path) ("HOME" . ,(getenv "HOME")))))) (environment-lookup-path my-special-dir) (environment-lookup-path "~/special/test-environment.el") (environment-associate-dir my-special-dir my-special-env) (let ((exec-path nil) (process-environment nil)) (shell-command "env")) (environment-unassociate-dir my-special-dir) (guix-environment-set-dir "/var/guix/profiles/per-user/sbaugh/guix-profile"= my-special-dir) (let ((exec-path nil) (process-environment nil)) (shell-command "env")) --=-=-= Content-Type: text/plain I have run into a problem though with the file-name-handler-alist approach. I don't think it will work in the long run. To recap, I am using file-name-handler-alist to make sure that all operations on files that are descendants in the filesystem tree of some project directories, are specially handled by my environment code. The problem is that paths passed to find-file-name-handler, such as default-directory, are not necessarily canonical paths. But file-name-handler-alist is an list of (REGEX . HANDLER), so I am only able to textually match against the input path. So if a path passed to find-file-name-handler contains ~ or .. or ., it might be that it is in fact a descendant of a project directory, but it won't be matched by the regex, and so it won't get specially handled. I don't think path resolution can be done with regular expressions, so this approach will never really work. >My impression is that this will have to be fixed case-by-case. There is >a precedent for that, which is the handling of buffer-local settings. >E.g. vc-diff is expected to obey the vc-diff-switches settings of the >"original buffer" rather than those of the buffer where the process >runs. So we sometimes go through the trouble to read some variables >before switching to the destination buffer. Environments would "simply" >fall into this case. But yes, those issues will have to fixed >one-by-one. That treatment of vc-diff-switches does look very similar to the correct treatment of environments. Since there are other variables that need to be treated in this way, beyond just environments, maybe a generic solution is possible? Perhaps the "new kind of dynamic buffer-local variable" I mentioned earlier: >- Make exec-path and process-environment a different kind of > buffer-local dynamic variable > > The behavior of this variable would be that when it is defined locally > in the current buffer, it behaves like a normal dynamic variable. And > when it is not defined locally in the current buffer, looking up the > variable will look up the variable dynamically instead of looking for > a global value set with setq-default. > This would mean that when M-x compile is invoked from some buffer, if > exec-path and process-environment are not buffer-local in the compile > buffer, those variables will be looked up dynamically, and found in > the buffer-locals of the buffer that was current at the time of the > M-x compile invocation. > I don't know if this is possible given the implementation of dynamic > variables in Emacs Lisp. This could be used for vc-diff-switches too, and other variables like it. Having now read more about how dynamic variables are implemented in Emacs, I guess this would require a fair bit of work though. If a generic solution like this isn't possible, I can start working on fixing things case-by-case as soon as a good design is worked out. >I'm not maintainer any more so I don't need to make those decisions. >But if I were maintainer, I'd need to see the code before I could make >a decision. OK, I'll wait until John expresses his thoughts. (Possibly after the release?) --=-=-=--