bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[immerrr again]

I have recently tried to play with a clean .emacs.d directory by setting
HOME=clean/home/dir, but faced an error coming from deep inside
package-install. The error boils down to the fact that
`expand-file-name' for some reason doubles the `default-directory'
prefix when it is relative:


$ emacs -Q -batch --eval '(let ((default-directory "DEFAULT-DIR/"))
(print (expand-file-name "EXPANDED-DIR/")))'

"DEFAULT-DIR/DEFAULT-DIR/EXPANDED-DIR/"


Interestingly, if you run `expand-file-name' twice you end up with four
prefixes:


$ emacs -Q -batch --eval '(let ((default-directory "DEFAULT-DIR/"))
(print (expand-file-name (expand-file-name "EXPANDED-DIR/"))))'

"DEFAULT-DIR/DEFAULT-DIR/DEFAULT-DIR/DEFAULT-DIR/EXPANDED-DIR/"


I couldn't find any requirement for DEFAULT-DIRECTORY parameter or
`default-directory' variable to be absolute, so I would assume this is a
bug in expand-file-name.


In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
 of 2018-07-09 built on mmrcomp
Repository revision: 3307353e13a9226d477c9b1a39baae76584b90b9
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 16.04.5 LTS

Recent messages:
Mark set
Saving file /home/immerrr/.cask/cask-cli.el...
Wrote /home/immerrr/.cask/cask-cli.el
Saving file /home/immerrr/.cask/cask-cli.el...
Wrote /home/immerrr/.cask/cask-cli.el
Saving file /home/immerrr/.cask/cask-cli.el...
Wrote /home/immerrr/.cask/cask-cli.el
Saving file /home/immerrr/.cask/cask-cli.el...
Wrote /home/immerrr/.cask/cask-cli.el
Mark set

Configured using:
 'configure --prefix=/home/immerrr/.local'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 THREADS JSON LCMS2

Important settings:
  value of $LC_TIME: en_GB.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: el

Minor modes in effect:
  magit-auto-revert-mode: t
  auto-revert-mode: t
  global-git-commit-mode: t
  async-bytecomp-package-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  diff-auto-refine-mode: t
  nameless-mode: t
  guide-key-mode: t
  whitespace-mode: t
  flycheck-mode: t
  shell-dirtrack-mode: t
  counsel-projectile-mode: t
  projectile-mode: t
  ivy-mode: t
  global-company-mode: t
  company-mode: t
  pyvenv-mode: t
  paredit-mode: t
  auto-compile-on-save-mode: t
  auto-compile-mode: t
  recentf-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  save-place-mode: t
  show-paren-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  override-global-mode: t
  bar-cursor-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/immerrr/.emacs.d/elpa/ample-regexps-20151023.300/init-tryout
hides /home/immerrr/.emacs.d/elpa/lua-mode-20180104.626/init-tryout

Features:
(shadow sort mail-extr emacsbug sendmail macrostep-c cmacexp macrostep
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs ielm tar-mode git-rebase rect bug-reference py-isort
executable vc vc-dispatcher magit-bookmark magit-obsolete magit-blame
magit-stash magit-bisect magit-remote magit-commit magit-sequence
magit-notes magit-worktree magit-branch magit-collab ghub url-auth
magit-files magit-refs magit-status magit dired-x magit-repos
magit-apply magit-wip magit-log magit-diff smerge-mode magit-core
magit-autorevert autorevert filenotify magit-process magit-margin
magit-mode git-commit magit-git magit-section magit-utils ido crm
magit-popup log-edit message rfc822 mml mml-sec epa epg gnus-util rmail
rmail-loaddefs mailabbrev gmm-utils mailheader pcvs-util add-log
with-editor term ehelp esh-var esh-cmd esh-opt esh-io esh-ext esh-proc
esh-arg esh-groups eshell esh-module esh-mode async-bytecomp async
server imenu undo-tree diff edebug network-stream puny nsm rmc
company-jedi jedi-core epc ctable concurrent cap-words superword subword
pipenv f eieio-opt speedbar sb-image ezimage dframe warnings pulse
cl-print debug vc-git diff-mode bookmark pp company-elisp nameless
guide-key popwin face-remap disp-table whitespace flycheck find-func rx
jka-compr let-alist colir mmr-org-settings python tramp-sh tramp
trampver tramp-compat tramp-loaddefs ucs-normalize shell pcomplete
parse-time format-spec counsel-projectile projectile skeleton ibuf-macs
wgrep grep ibuf-ext ibuffer ibuffer-loaddefs counsel dired
dired-loaddefs compile esh-util swiper ivy-rich ivy delsel ivy-overlay
ffap thingatpt traad virtualenvwrapper gud comint ansi-color s
request-deferred request mail-utils url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap popup
python-environment deferred subr-x company-oddmuse company-keywords
company-etags etags xref project company-gtags company-dabbrev-code
company-dabbrev company-files company-capf company-cmake company-xcode
company-clang company-semantic company-eclim company-template
company-css company-nxml company-bbdb company pyvenv paredit
auto-compile packed recentf tree-widget wid-edit yasnippet derived
elec-pair saveplace hydra ring lv paren solarized-dark-theme
solarized-theme solarized color dash display-line-numbers
mmr-dir-local-env mmr-bootstrap quelpa-use-package cl-extra advice pcase
quelpa mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
mm-util ietf-drums mail-prsvr help-fns radix-tree help-mode
package-recipe-mode edmacro kmacro package-build-badges package-build
lisp-mnt use-package use-package-ensure use-package-delight
use-package-diminish use-package-bind-key use-package-core bind-key
diminish cl linum-ex bar-cursor easy-mmode mmr-minimal mmr-ediff
windmove time-date mule-util info finder-inf package easymenu epg-config
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 1357726 145867)
 (symbols 48 50837 2)
 (miscs 40 6306 4056)
 (strings 32 121140 10207)
 (string-bytes 1 4278168)
 (vectors 16 68224)
 (vector-slots 8 1809895 106986)
 (floats 8 422 1779)
 (intervals 56 116849 2490)
 (buffers 992 53)
 (heap 1024 76825 5884))

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Noam Postavsky]

immerrr again <immerrr@gmail.com> writes:

> I have recently tried to play with a clean .emacs.d directory by setting
> HOME=clean/home/dir, but faced an error coming from deep inside
> package-install. The error boils down to the fact that
> `expand-file-name' for some reason doubles the `default-directory'
> prefix when it is relative:
[...]
> I couldn't find any requirement for DEFAULT-DIRECTORY parameter or
> `default-directory' variable to be absolute, so I would assume this is a
> bug in expand-file-name.

(elisp) File Name Expansion says:

 -- Variable: default-directory
     The value of this buffer-local variable is the default directory
     for the current buffer.  It should be an absolute directory name;
     it may start with `~'.[...]

And expand-file-name is supposed to expand file names relative to
default-directory, so if default-directory is itself relative I don't
see how it could possibly work.  So we should just mention the
absoluteness requirement in the docstring as well.

I think it would make sense for Emacs to expand HOME against the current
directory when it starts up, if HOME is relative.  That would let your
motivating example succeed without error.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Glenn Morris]

Noam Postavsky wrote:

> immerrr again <immerrr@gmail.com> writes:
>
>> I have recently tried to play with a clean .emacs.d directory by setting
>> HOME=clean/home/dir,

I think a non-absolute HOME is a user error.
Eg cd $HOME may then not be idempotent.
I would use $PWD/clean/home/dir.

> I think it would make sense for Emacs to expand HOME against the current
> directory when it starts up, if HOME is relative.  

I would rather see Emacs abort with an error.
Otherwise you have to eg fix up process-environment too, else child
processes may behave oddly.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 691 bytes --]

> I think a non-absolute HOME is a user error.
> Eg cd $HOME may then not be idempotent.
That's true. However, POSIX says that sh treats ~/foo like $HOME/foo 
even when HOME is not absolute, and it's better if Emacs is consistent 
with POSIX as much as possible within the Emacs constraint that 
expand-file-name must expand to an absolute file name. So I implemented 
something along the line of Noam's suggestion by installing the attached 
patch into master; this should fix the bug originally reported.

Unlike Noam's suggestion, this patch causes Emacs to look at the current 
value of HOME, not the value HOME had when Emacs started up, as that 
corresponds more closely to POSIX sh.


[-- Attachment #2: 0001-Act-like-POSIX-sh-if-HOME-is-relative.patch --]
[-- Type: text/x-patch, Size: 14613 bytes --]

From a2df04a05a68d5a4afa39cfe463f72108b31bc25 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 13 Nov 2018 09:29:14 -0800
Subject: [PATCH] Act like POSIX sh if $HOME is relative

POSIX says sh ~/foo should act like $HOME/foo even if $HOME is
relative, so be consistent with that (Bug#33255).
* admin/merge-gnulib (GNULIB_MODULES): Add dosname.
* src/buffer.c (init_buffer): Use emacs_wd to get
initial working directory with slash appended if needed.
(default-directory): Say it must be absolute.
* src/emacs.c (emacs_wd): New global variable.
(init_cmdargs): Dir arg is now char const *.
(main): Set emacs_wd.
* src/emacs.c (main) [NS_IMPL_COCOA]:
* src/fileio.c (Fexpand_file_name):
Use get_homedir instead of egetenv ("HOME").
* src/fileio.c: Include dosname.h, for IS_ABSOLUTE_FILE_NAME.
(splice_dir_file, get_homedir): New functions.
* src/xrdb.c (gethomedir): Remove.  All callers changed
to use get_homedir and splice_dir_file.
* test/src/fileio-tests.el (fileio-tests--relative-HOME): New test.
---
 admin/merge-gnulib       |  2 +-
 src/buffer.c             | 28 ++++++------------
 src/emacs.c              | 20 ++++++++-----
 src/fileio.c             | 62 +++++++++++++++++++++++++++++++++++++---
 src/lisp.h               |  3 ++
 src/xrdb.c               | 54 +++++++---------------------------
 test/src/fileio-tests.el |  8 ++++++
 7 files changed, 103 insertions(+), 74 deletions(-)

diff --git a/admin/merge-gnulib b/admin/merge-gnulib
index 575e3fa74a..84dcb0b875 100755
--- a/admin/merge-gnulib
+++ b/admin/merge-gnulib
@@ -30,7 +30,7 @@ GNULIB_MODULES=
   careadlinkat close-stream
   count-leading-zeros count-one-bits count-trailing-zeros
   crypto/md5-buffer crypto/sha1-buffer crypto/sha256-buffer crypto/sha512-buffer
-  d-type diffseq dtoastr dtotimespec dup2
+  d-type diffseq dosname dtoastr dtotimespec dup2
   environ execinfo explicit_bzero faccessat
   fcntl fcntl-h fdatasync fdopendir
   filemode filevercmp flexmember fpieee fstatat fsusage fsync
diff --git a/src/buffer.c b/src/buffer.c
index ac2de7d19f..90ef886b22 100644
--- a/src/buffer.c
+++ b/src/buffer.c
@@ -5268,9 +5268,7 @@ init_buffer_once (void)
 void
 init_buffer (int initialized)
 {
-  char *pwd;
   Lisp_Object temp;
-  ptrdiff_t len;
 
 #ifdef USE_MMAP_FOR_BUFFERS
   if (initialized)
@@ -5324,7 +5322,7 @@ init_buffer (int initialized)
   if (NILP (BVAR (&buffer_defaults, enable_multibyte_characters)))
     Fset_buffer_multibyte (Qnil);
 
-  pwd = emacs_get_current_dir_name ();
+  char const *pwd = emacs_wd;
 
   if (!pwd)
     {
@@ -5336,22 +5334,16 @@ init_buffer (int initialized)
     {
       /* Maybe this should really use some standard subroutine
          whose definition is filename syntax dependent.  */
-      len = strlen (pwd);
-      if (!(IS_DIRECTORY_SEP (pwd[len - 1])))
-        {
-          /* Grow buffer to add directory separator and '\0'.  */
-          pwd = realloc (pwd, len + 2);
-          if (!pwd)
-            fatal ("get_current_dir_name: %s\n", strerror (errno));
-          pwd[len] = DIRECTORY_SEP;
-          pwd[len + 1] = '\0';
-          len++;
-        }
+      ptrdiff_t len = strlen (pwd);
+      bool add_slash = ! IS_DIRECTORY_SEP (pwd[len - 1]);
 
       /* At this moment, we still don't know how to decode the directory
          name.  So, we keep the bytes in unibyte form so that file I/O
          routines correctly get the original bytes.  */
-      bset_directory (current_buffer, make_unibyte_string (pwd, len));
+      Lisp_Object dirname = make_unibyte_string (pwd, len + add_slash);
+      if (add_slash)
+	SSET (dirname, len, DIRECTORY_SEP);
+      bset_directory (current_buffer, dirname);
 
       /* Add /: to the front of the name
          if it would otherwise be treated as magic.  */
@@ -5372,8 +5364,6 @@ init_buffer (int initialized)
 
   temp = get_minibuffer (0);
   bset_directory (XBUFFER (temp), BVAR (current_buffer, directory));
-
-  free (pwd);
 }
 
 /* Similar to defvar_lisp but define a variable whose value is the
@@ -5706,8 +5696,8 @@ visual lines rather than logical lines.  See the documentation of
   DEFVAR_PER_BUFFER ("default-directory", &BVAR (current_buffer, directory),
 		     Qstringp,
 		     doc: /* Name of default directory of current buffer.
-It should be a directory name (as opposed to a directory file-name).
-On GNU and Unix systems, directory names end in a slash `/'.
+It should be an absolute directory name; on GNU and Unix systems,
+these names start with `/' or `~' and end with `/'.
 To interactively change the default directory, use command `cd'. */);
 
   DEFVAR_PER_BUFFER ("auto-fill-function", &BVAR (current_buffer, auto_fill_function),
diff --git a/src/emacs.c b/src/emacs.c
index 512174d562..acb4959bfe 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -204,6 +204,9 @@ HANDLE w32_daemon_event;
 char **initial_argv;
 int initial_argc;
 
+/* The name of the working directory, or NULL if this info is unavailable.  */
+char const *emacs_wd;
+
 static void sort_args (int argc, char **argv);
 static void syms_of_emacs (void);
 
@@ -406,7 +409,7 @@ terminate_due_to_signal (int sig, int backtrace_limit)
 /* Code for dealing with Lisp access to the Unix command line.  */
 
 static void
-init_cmdargs (int argc, char **argv, int skip_args, char *original_pwd)
+init_cmdargs (int argc, char **argv, int skip_args, char const *original_pwd)
 {
   int i;
   Lisp_Object name, dir, handler;
@@ -694,7 +697,7 @@ main (int argc, char **argv)
   char *ch_to_dir = 0;
 
   /* If we use --chdir, this records the original directory.  */
-  char *original_pwd = 0;
+  char const *original_pwd = 0;
 
   /* Record (approximately) where the stack begins.  */
   stack_bottom = (char *) &stack_bottom_variable;
@@ -794,6 +797,8 @@ main (int argc, char **argv)
       exit (0);
     }
 
+  emacs_wd = emacs_get_current_dir_name ();
+
   if (argmatch (argv, argc, "-chdir", "--chdir", 4, &ch_to_dir, &skip_args))
     {
 #ifdef WINDOWSNT
@@ -804,13 +809,14 @@ main (int argc, char **argv)
       filename_from_ansi (ch_to_dir, newdir);
       ch_to_dir = newdir;
 #endif
-      original_pwd = emacs_get_current_dir_name ();
       if (chdir (ch_to_dir) != 0)
         {
           fprintf (stderr, "%s: Can't chdir to %s: %s\n",
                    argv[0], ch_to_dir, strerror (errno));
           exit (1);
         }
+      original_pwd = emacs_wd;
+      emacs_wd = emacs_get_current_dir_name ();
     }
 
 #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN)
@@ -1289,21 +1295,21 @@ Using an Emacs configured with --with-x-toolkit=lucid does not have this problem
     {
 #ifdef NS_IMPL_COCOA
       /* Started from GUI? */
-      /* FIXME: Do the right thing if getenv returns NULL, or if
+      /* FIXME: Do the right thing if get_homedir returns "", or if
          chdir fails.  */
       if (! inhibit_window_system && ! isatty (STDIN_FILENO) && ! ch_to_dir)
-        chdir (getenv ("HOME"));
+        chdir (get_homedir ());
       if (skip_args < argc)
         {
           if (!strncmp (argv[skip_args], "-psn", 4))
             {
               skip_args += 1;
-              if (! ch_to_dir) chdir (getenv ("HOME"));
+              if (! ch_to_dir) chdir (get_homedir ());
             }
           else if (skip_args+1 < argc && !strncmp (argv[skip_args+1], "-psn", 4))
             {
               skip_args += 2;
-              if (! ch_to_dir) chdir (getenv ("HOME"));
+              if (! ch_to_dir) chdir (get_homedir ());
             }
         }
 #endif  /* COCOA */
diff --git a/src/fileio.c b/src/fileio.c
index 7fb865809f..e178c39fc1 100644
--- a/src/fileio.c
+++ b/src/fileio.c
@@ -96,6 +96,7 @@ along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.  */
 #include <acl.h>
 #include <allocator.h>
 #include <careadlinkat.h>
+#include <dosname.h>
 #include <fsusage.h>
 #include <stat-time.h>
 #include <tempname.h>
@@ -1093,8 +1094,7 @@ the root directory.  */)
 	{
 	  Lisp_Object tem;
 
-	  if (!(newdir = egetenv ("HOME")))
-	    newdir = newdirlim = "";
+	  newdir = get_homedir ();
 	  nm++;
 #ifdef WINDOWSNT
 	  if (newdir[0])
@@ -1109,7 +1109,7 @@ the root directory.  */)
 #endif
 	    tem = build_string (newdir);
 	  newdirlim = newdir + SBYTES (tem);
-	  /* `egetenv' may return a unibyte string, which will bite us
+	  /* get_homedir may return a unibyte string, which will bite us
 	     if we expect the directory to be multibyte.  */
 	  if (multibyte && !STRING_MULTIBYTE (tem))
 	    {
@@ -1637,7 +1637,6 @@ See also the function `substitute-in-file-name'.")
 }
 #endif
 \f
-/* If /~ or // appears, discard everything through first slash.  */
 static bool
 file_name_absolute_p (const char *filename)
 {
@@ -1650,6 +1649,61 @@ file_name_absolute_p (const char *filename)
      );
 }
 
+/* Put into BUF the concatenation of DIR and FILE, with an intervening
+   directory separator if needed.  Return a pointer to the null byte
+   at the end of the concatenated string.  */
+char *
+splice_dir_file (char *buf, char const *dir, char const *file)
+{
+  char *e = stpcpy (buf, dir);
+  *e = DIRECTORY_SEP;
+  e += ! (buf < e && IS_DIRECTORY_SEP (e[-1]));
+  return stpcpy (e, file);
+}
+
+/* Get the home directory, an absolute file name.  Return the empty
+   string on failure.  The returned value does not survive garbage
+   collection, calls to this function, or calls to the getpwnam class
+   of functions.  */
+char const *
+get_homedir (void)
+{
+  char const *home = egetenv ("HOME");
+  if (!home)
+    {
+      static char const *userenv[] = {"LOGNAME", "USER"};
+      struct passwd *pw = NULL;
+      for (int i = 0; i < ARRAYELTS (userenv); i++)
+	{
+	  char *user = egetenv (userenv[i]);
+	  if (user)
+	    {
+	      pw = getpwnam (user);
+	      if (pw)
+		break;
+	    }
+	}
+      if (!pw)
+	pw = getpwuid (getuid ());
+      if (pw)
+	home = pw->pw_dir;
+      if (!home)
+	return "";
+    }
+  if (IS_ABSOLUTE_FILE_NAME (home))
+    return home;
+  if (!emacs_wd)
+    error ("$HOME is relative to unknown directory");
+  static char *ahome;
+  static ptrdiff_t ahomesize;
+  ptrdiff_t ahomelenbound = strlen (emacs_wd) + 1 + strlen (home) + 1;
+  if (ahomesize <= ahomelenbound)
+    ahome = xpalloc (ahome, &ahomesize, ahomelenbound + 1 - ahomesize, -1, 1);
+  splice_dir_file (ahome, emacs_wd, home);
+  return ahome;
+}
+
+/* If /~ or // appears, discard everything through first slash.  */
 static char *
 search_embedded_absfilename (char *nm, char *endp)
 {
diff --git a/src/lisp.h b/src/lisp.h
index f8ffb33a64..7e7dba631f 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -4061,6 +4061,8 @@ extern void syms_of_marker (void);
 
 /* Defined in fileio.c.  */
 
+extern char *splice_dir_file (char *, char const *, char const *);
+extern char const *get_homedir (void);
 extern Lisp_Object expand_and_dir_to_file (Lisp_Object);
 extern Lisp_Object write_region (Lisp_Object, Lisp_Object, Lisp_Object,
 				 Lisp_Object, Lisp_Object, Lisp_Object,
@@ -4185,6 +4187,7 @@ extern void syms_of_frame (void);
 /* Defined in emacs.c.  */
 extern char **initial_argv;
 extern int initial_argc;
+extern char const *emacs_wd;
 #if defined (HAVE_X_WINDOWS) || defined (HAVE_NS)
 extern bool display_arg;
 #endif
diff --git a/src/xrdb.c b/src/xrdb.c
index 4abf1ad84e..87c2faf659 100644
--- a/src/xrdb.c
+++ b/src/xrdb.c
@@ -202,35 +202,6 @@ magic_db (const char *string, ptrdiff_t string_len, const char *class,
 }
 
 
-static char *
-gethomedir (void)
-{
-  struct passwd *pw;
-  char *ptr;
-  char *copy;
-
-  if ((ptr = getenv ("HOME")) == NULL)
-    {
-      if ((ptr = getenv ("LOGNAME")) != NULL
-	  || (ptr = getenv ("USER")) != NULL)
-	pw = getpwnam (ptr);
-      else
-	pw = getpwuid (getuid ());
-
-      if (pw)
-	ptr = pw->pw_dir;
-    }
-
-  if (ptr == NULL)
-    return xstrdup ("/");
-
-  ptrdiff_t len = strlen (ptr);
-  copy = xmalloc (len + 2);
-  strcpy (copy + len, "/");
-  return memcpy (copy, ptr, len);
-}
-
-
 /* Find the first element of SEARCH_PATH which exists and is readable,
    after expanding the %-escapes.  Return 0 if we didn't find any, and
    the path name of the one we found otherwise.  */
@@ -316,12 +287,11 @@ get_user_app (const char *class)
   if (! db)
     {
       /* Check in the home directory.  This is a bit of a hack; let's
-	 hope one's home directory doesn't contain any %-escapes.  */
-      char *home = gethomedir ();
+	 hope one's home directory doesn't contain ':' or '%'.  */
+      char const *home = get_homedir ();
       db = search_magic_path (home, class, "%L/%N");
       if (! db)
 	db = search_magic_path (home, class, "%N");
-      xfree (home);
     }
 
   return db;
@@ -346,10 +316,9 @@ get_user_db (Display *display)
   else
     {
       /* Use ~/.Xdefaults.  */
-      char *home = gethomedir ();
-      ptrdiff_t homelen = strlen (home);
-      char *filename = xrealloc (home, homelen + sizeof xdefaults);
-      strcpy (filename + homelen, xdefaults);
+      char const *home = get_homedir ();
+      char *filename = xmalloc (strlen (home) + 1 + sizeof xdefaults);
+      splice_dir_file (filename, home, xdefaults);
       db = XrmGetFileDatabase (filename);
       xfree (filename);
     }
@@ -380,13 +349,12 @@ get_environ_db (void)
       if (STRINGP (system_name))
 	{
 	  /* Use ~/.Xdefaults-HOSTNAME.  */
-	  char *home = gethomedir ();
-	  ptrdiff_t homelen = strlen (home);
-	  ptrdiff_t filenamesize = (homelen + sizeof xdefaults
-				    + 1 + SBYTES (system_name));
-	  p = filename = xrealloc (home, filenamesize);
-	  lispstpcpy (stpcpy (stpcpy (filename + homelen, xdefaults), "-"),
-		      system_name);
+	  char const *home = get_homedir ();
+	  p = filename = xmalloc (strlen (home) + 1 + sizeof xdefaults
+				  + 1 + SBYTES (system_name));
+	  char *e = splice_dir_file (p, home, xdefaults);
+	  *e++ = '/';
+	  lispstpcpy (e, system_name);
 	}
     }
 
diff --git a/test/src/fileio-tests.el b/test/src/fileio-tests.el
index 5d12685fa1..b7b78bbda0 100644
--- a/test/src/fileio-tests.el
+++ b/test/src/fileio-tests.el
@@ -95,3 +95,11 @@ fileio-tests--symlink-failure
   (should (equal (file-name-as-directory "d:/abc/") "d:/abc/"))
   (should (equal (file-name-as-directory "D:\\abc/") "d:/abc/"))
   (should (equal (file-name-as-directory "D:/abc//") "d:/abc//")))
+
+(ert-deftest fileio-tests--relative-HOME ()
+  "Test that expand-file-name works even when HOME is relative."
+  (let ((old-home (getenv "HOME")))
+    (setenv "HOME" "a/b/c")
+    (should (equal (expand-file-name "~/foo")
+                   (expand-file-name "a/b/c/foo")))
+    (setenv "HOME" old-home)))
-- 
2.19.1

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 13 Nov 2018 10:26:43 -0800
> Cc: 33255-done@debbugs.gnu.org, immerrr again <immerrr@gmail.com>,
> 	Noam Postavsky <npostavs@gmail.com>
> 
> > I think a non-absolute HOME is a user error.
> > Eg cd $HOME may then not be idempotent.
> That's true. However, POSIX says that sh treats ~/foo like $HOME/foo 
> even when HOME is not absolute, and it's better if Emacs is consistent 
> with POSIX as much as possible within the Emacs constraint that 
> expand-file-name must expand to an absolute file name. So I implemented 
> something along the line of Noam's suggestion by installing the attached 
> patch into master; this should fix the bug originally reported.
> 
> Unlike Noam's suggestion, this patch causes Emacs to look at the current 
> value of HOME, not the value HOME had when Emacs started up, as that 
> corresponds more closely to POSIX sh.

Thanks.  I think this needs to be called out in NEWS as an
incompatible change, and probably also documented in the manuals.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Glenn Morris]

Paul Eggert wrote:

> However, POSIX says that sh treats ~/foo like $HOME/foo even when HOME
> is not absolute [...]

Can you point me to the citation for that please (I'm not doubting you,
just interested to read the wording, which I could not find).

I'm a bit disappointed to see that a relative HOME is now documented
in the Emacs manuals, since IMO it lends legitimacy to a questionable
usage. I don't see it mentioned in eg the bash manual.
I also don't see why it is in NEWS since it is described as
a bug fix, and those aren't normally mentioned.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Paul Eggert]

On 11/14/18 10:10 AM, Glenn Morris wrote:
> Can you point me to the citation for that please

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_01

> I'm a bit disappointed to see that a relative HOME is now documented
> in the Emacs manuals, since IMO it lends legitimacy to a questionable
> usage. I don't see it mentioned in eg the bash manual.
> I also don't see why it is in NEWS since it is described as
> a bug fix, and those aren't normally mentioned.

My initial reaction was the same as yours, which is why my original fix 
didn't document the change in the manual or in NEWS. Eli felt otherwise, 
though.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 14 Nov 2018 10:17:59 -0800
> Cc: 33255@debbugs.gnu.org, immerrr again <immerrr@gmail.com>,
> 	Noam Postavsky <npostavs@gmail.com>
> 
> > I'm a bit disappointed to see that a relative HOME is now documented
> > in the Emacs manuals, since IMO it lends legitimacy to a questionable
> > usage. I don't see it mentioned in eg the bash manual.
> > I also don't see why it is in NEWS since it is described as
> > a bug fix, and those aren't normally mentioned.
> 
> My initial reaction was the same as yours, which is why my original fix 
> didn't document the change in the manual or in NEWS. Eli felt otherwise, 
> though.

I agree that we should discourage relative $HOME, so I added text to
the documentation to that effect.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Glenn Morris]

Paul Eggert wrote:

> On 11/14/18 10:10 AM, Glenn Morris wrote:
>> Can you point me to the citation for that please
>
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_01

I had looked at that page before. Looking again, I still don't see
anything as clear-cut as "POSIX says that sh treats ~/foo like $HOME/foo
even when HOME is not absolute".

There are no matches for "relative" on that page.
There are three for "absolute". One is for ENV, and says that the
results are unspecified if it is not absolute. The other two are for
PWD, and say that it must be absolute.

What am I missing?
Just that the section on ~ expansion says nothing one way or the other?
I was really looking for a reference about HOME.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Glenn Morris]

Eli Zaretskii wrote:

> I agree that we should discourage relative $HOME, so I added text to
> the documentation to that effect.

I really think it would be better not to mention it at all.
I have never heard of it in many years of using Unix (I know this is an
awful kind of statement), and see no need for it. Again, the GNU Bash
manual says nothing about this.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: Paul Eggert <eggert@cs.ucla.edu>,  33255@debbugs.gnu.org,  immerrr@gmail.com,  npostavs@gmail.com
> Date: Tue, 20 Nov 2018 14:11:00 -0500
> 
> Eli Zaretskii wrote:
> 
> > I agree that we should discourage relative $HOME, so I added text to
> > the documentation to that effect.
> 
> I really think it would be better not to mention it at all.

Sorry, I disagree.  We should not hide from the users what Emacs does
in such corner cases.  Hiding won't work anyway, because someone
determined enough will find out by reading the code.  We just punish
those who aren't determined enough.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Paul Eggert]

On 11/20/18 11:08 AM, Glenn Morris wrote:
>> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_01
> I had looked at that page before. Looking again, I still don't see
> anything as clear-cut as "POSIX says that sh treats ~/foo like $HOME/foo
> even when HOME is not absolute".

It needs to be read in context. Section 2.6.1 is about tilde expansion 
in the shell, e.g., how to treat commands like this:

cd ~eggert/xxx
cd ~/yyy

2.6.1 says "If the login name is null (that is, the tilde-prefix 
contains only the tilde), the tilde-prefix is replaced by the value of 
the variable /HOME."/ This is talking about the second "echo" example 
which uses a null login name, and it means that the second example is 
treated like this:

cd "$HOME"/yyy/
/

(The quotes are because of the last sentence in that section of the 
spec.) This occurs regardless of whether $HOME starts with /"/".//
/

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Glenn Morris]

Paul Eggert wrote:

> It needs to be read in context. Section 2.6.1 is about tilde expansion
> in the shell, e.g., how to treat commands like this:
>
> cd ~eggert/xxx
> cd ~/yyy
>
> 2.6.1 says "If the login name is null (that is, the tilde-prefix
> contains only the tilde), the tilde-prefix is replaced by the value of
> the variable /HOME."/ This is talking about the second "echo" example
> which uses a null login name, and it means that the second example is
> treated like this:
>
> cd "$HOME"/yyy/

This seems to be telling me what tilde expansion is.

I am at this point looking for any documentation (not even from POSIX,
any shell or frankly any Unix utility will do) that says "HOME need not
be absolute, if not, here's how that is handled". So far all I see from
POSIX is that it doesn't say anything about whether HOME is absolute or
not.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Paul Eggert]

Glenn Morris wrote:
> So far all I see from POSIX is that it doesn't say anything about whether HOME is absolute or not.

By not saying anything POSIX gives permission to the application to set HOME to 
a relative name. When POSIX intends the requirement that an environment 
variable's value must be absolute (e.g., PWD), it says so. When it doesn't 
intend such a requirement (e.g., HOME, PATH, SHELL, TMPDIR) it says nothing.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Glenn Morris]

Paul Eggert wrote:

> By not saying anything POSIX gives permission to the application to
> set HOME to a relative name. When POSIX intends the requirement that
> an environment variable's value must be absolute (e.g., PWD), it says
> so. When it doesn't intend such a requirement (e.g., HOME, PATH,
> SHELL, TMPDIR) it says nothing.

So the justification for implementing this is indeed "it's not
explicitly forbidden".

Here's an example of how this can be confusing:

cd /tmp
mkdir foo
echo hi > foo/bar
HOME=foo emacs
C-x C-f ~/bar   ; works
M-: (shell-command "ls ~/bar")  ; fails

So, Emacs and external processes it spawns interpret ~ differently;
ie external processes are likely to fail in odd ways. All this would be
avoided if the user had just said HOME=$PWD/foo to start with.

>> I am at this point looking for any documentation (not even from POSIX,
>> any shell or frankly any Unix utility will do) that says "HOME need
>> not be absolute, if not, here's how that is handled".

AFAICS Emacs is the only thing documenting this scenario.

bug#33255: 27.0.50; expand-file-name: default directory expanded twice if relative

[Paul Eggert]

On 11/26/18 9:42 PM, Glenn Morris wrote:
> Here's an example of how this can be confusing:
>
> cd /tmp
> mkdir foo
> echo hi > foo/bar
> HOME=foo emacs
> C-x C-f ~/bar   ; works
> M-: (shell-command "ls ~/bar")  ; fails

Yes, and there's a similar confusion in ordinary POSIX shells:

$ cd /tmp
$ mkdir foo
$ echo hi > foo/bar
$ HOME=foo sh
$ cat ~/bar
hi
$ cd
$ cat ~/bar
cat: foo/bar: No such file or directory

The moral of this story in POSIX is "Don't set HOME to a relative file 
name, as it's trouble for any application that chdirs." In your example 
Emacs chdirs; in mine, sh chdirs.

The question is whether Emacs should attempt to insulate users from this 
trouble, presumably by replacing HOME with an absolute directory name 
when HOME's value is relative (or is absent or empty, for that matter). 
The POSIX shell doesn't do that, which is an argument for Emacs not 
doing it either.