bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

[vuori]

configure on NetBSD forces using the native libterminfo library, even
when ncurses is available. When started with a TERM such xterm-256color,
this results in abort() in tputs1 (TERM=xterm works).

Hacking configure to use ncurses when available provides stable
operation on xterm-256color.

The current special configure logic originates from this issue:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=6190 . This check is
probably not a good idea today. Either NetBSD should do a) what the
other BSDs do (= force ncurses) or b) what gnu* does (use what
configure found, otherwise force ncurses).

My vote goes to b (possibly with a warning from configure if libterminfo
is picked): keep it possible to build emacs without installing ncurses,
but use ncurses for terminfo when available.

To reproduce:

1. Compile Emacs 29 on NetBSD 9.3 (same logic is present on master, so I
assume that won't work any better).

2. Note that configure prints "-lncurses" if ncurses is installed, but
actually emacs is linked with libterminfo.

3. Run TERM=xterm-256color emacs -nw. You should get an abort in tputs1.


In GNU Emacs 29.0.92 (build 4, x86_64-unknown-netbsd9.3, X toolkit,
 cairo version 1.16.0) of 2023-07-10 built on netbsd
Repository revision: 419b4d44914701cedf308df58c3e5789715c87e7
Repository branch: emacs-29
System Description: NetBSD netbsd 9.3_STABLE NetBSD 9.3_STABLE (VUORI) #4: Sat Jun  3 09:32:53 UTC 2023  vuori@netbsd:/usr/src/sys/arch/amd64/compile/VUORI amd64

Configured using:
 'configure --prefix=/home/vuori/emacs --with-x-toolkit=athena
 --without-sound --with-modules --with-json
 --without-toolkit-scroll-bars CFLAGS=-I/usr/pkg/include
 'LDFLAGS=-L/usr/pkg/lib -R/usr/pkg/lib -R/usr/X11R7/lib''

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBXML2 MODULES NOTIFY KQUEUE PDUMPER PNG SQLITE3 THREADS TIFF
WEBP X11 XAW3D XDBE XIM XINPUT2 XPM LUCID ZLIB

Important settings:
  value of $LC_COLLATE: C.UTF-8
  value of $LC_TIME: C.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils term/xterm xterm byte-opt gv bytecomp byte-compile
rmc iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind kqueue lcms2 dynamic-setting system-font-setting
font-render-setting cairo x-toolkit xinput2 x multi-tty
make-network-process emacs)

Memory information:
((conses 16 46279 8331)
 (symbols 48 5824 0)
 (strings 32 14914 1175)
 (string-bytes 1 424404)
 (vectors 16 8323)
 (vector-slots 8 105495 7095)
 (floats 8 24 247)
 (intervals 56 217 0)
 (buffers 976 10))

bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

[Eli Zaretskii]

> From: vuori@notcom.org
> Date: Tue, 11 Jul 2023 20:37:19 +0000
> 
> 
> configure on NetBSD forces using the native libterminfo library, even
> when ncurses is available. When started with a TERM such xterm-256color,
> this results in abort() in tputs1 (TERM=xterm works).
> 
> Hacking configure to use ncurses when available provides stable
> operation on xterm-256color.
> 
> The current special configure logic originates from this issue:
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=6190 . This check is
> probably not a good idea today. Either NetBSD should do a) what the
> other BSDs do (= force ncurses) or b) what gnu* does (use what
> configure found, otherwise force ncurses).
> 
> My vote goes to b (possibly with a warning from configure if libterminfo
> is picked): keep it possible to build emacs without installing ncurses,
> but use ncurses for terminfo when available.

I'm not sure I understand.  The configure script first looks for a
library with tputs, and it tries tinfo, ncurses, terminfo, curses,
termcap, tinfow, ncursesw, in that order.  I understand that in your
case it finds ncurses, which is good, right?

After that, the configure script decides whether to use terminfo or
termcap.  It is here that for netbsd we decide to use termcap in
certain situations instead of terminfo.  In your case, since ncurses
was found, LIBS_TERMCAP should have been set to -lncurses, and this
code:

  netbsd)
    if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
      TERMINFO=no
      LIBS_TERMCAP="-ltermcap"
    fi

should have set TERMINFO=no, which disables linking against terminfo.

So I don't think I understand what is going on there.

As I don't have access to a netbsd system, could you please post the
config.log you get when configuring with a fresh tree of
emacs-29.0.92, and also show the "set -x" expansion of the relevant
parts of the configure script?  (Please compress those when posting if
they are larger than, say, 0.5MB.)  I would like to use those to
understand what happens on your system and why the configure script
decides to use terminfo.

Once we understand the exact reason, it will be easier to discuss
potential solutions.

Also, do you happen to know when terminfo on NetBSD started to be
buggy?  I would think that the change in bug#6190 did work at the
time, so at least back then NetBSD did have a workable terminfo.  When
did it start failing?

Thanks.

bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

[Valtteri Vuorikoski]

[-- Attachment #1: Type: text/plain, Size: 3254 bytes --]

On Wed, Jul 12, 2023 at 03:11:02PM +0300, Eli Zaretskii wrote:
> > configure on NetBSD forces using the native libterminfo library, even
> > when ncurses is available. When started with a TERM such xterm-256color,
> > this results in abort() in tputs1 (TERM=xterm works).
> > 
> > Hacking configure to use ncurses when available provides stable
> > operation on xterm-256color.
> 
> I'm not sure I understand.  The configure script first looks for a
> library with tputs, and it tries tinfo, ncurses, terminfo, curses,
> termcap, tinfow, ncursesw, in that order.  I understand that in your
> case it finds ncurses, which is good, right?
> 
> After that, the configure script decides whether to use terminfo or
> termcap.  It is here that for netbsd we decide to use termcap in
> certain situations instead of terminfo.  In your case, since ncurses
> was found, LIBS_TERMCAP should have been set to -lncurses, and this
> code:
> 
>   netbsd)
>     if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
>       TERMINFO=no
>       LIBS_TERMCAP="-ltermcap"
>     fi
> 
> should have set TERMINFO=no, which disables linking against terminfo.
> 
> So I don't think I understand what is going on there.

Finding ncurses is good, but unfortunately ncurses isn't what ends up
getting used. configure says:

checking for library containing tputs... -lncurses

but this happens:

$ ldd src/emacs | egrep 'term|curses'
     -lterminfo.1 => /usr/lib/libterminfo.so.1

Disabling the test above results in src/emacs being linked with
ncurses. That is, when I remove the whole "netbsd" special case
from configure this is the result:

$ ldd ~/emacs/bin/emacs | egrep 'term|curses'
   -lncurses.6 => /usr/pkg/lib/libncurses.so.6

> As I don't have access to a netbsd system, could you please post the
> config.log you get when configuring with a fresh tree of
> emacs-29.0.92, and also show the "set -x" expansion of the relevant
> parts of the configure script?  (Please compress those when posting if
> they are larger than, say, 0.5MB.)  I would like to use those to
> understand what happens on your system and why the configure script
> decides to use terminfo.

These are attached.

After some further investigation, it turns out that in NetBSD 9.3,
libtermcap is a symlink to libterminfo. So the configure test ends up
forcing -ltermcap and TERMINFO=no because LIBS_TERMCAP=-lncurses (so
!= -lterminfo), which actually links with libterminfo due to the
symlink.

So with current NetBSD, I don't think the test as it stands could ever
do anything useful, since there is only terminfo and apparently a
termcap-like compatibility wrapper around terminfo.

Looking at a git mirror of NetBSD system sources, this seems to have
been the case since at least 2010.

> Also, do you happen to know when terminfo on NetBSD started to be
> buggy?  I would think that the change in bug#6190 did work at the
> time, so at least back then NetBSD did have a workable terminfo.  When
> did it start failing?

I have a build of emacs 27.0.91 on another system built with a similar
environment (ncurses should be available but emacs is linked with
libterminfo) and it also aborts in tparam1 with TERM=xterm-256color,
so looks like this has been happening for quite a while.

 -Valtteri
 

[-- Attachment #2: config.log.gz --]
[-- Type: application/gzip, Size: 72810 bytes --]

[-- Attachment #3: config.x --]
[-- Type: text/plain, Size: 8180 bytes --]

+ printf '%s\n' 'configure:22791: checking for library containing tputs' >&5
+ printf '%s' 'checking for library containing tputs... ' >&6
checking for library containing tputs... + test
+ as_fn_nop
+ return 1
+ test netbsd = mingw32
+ tputs_libraries='tinfo ncurses terminfo curses termcap tinfow ncursesw'
+ for tputs_library=''
+ OLIBS='-lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo'
+ test -z ''
+ emacs_cv_tputs_lib='none required'
+ test no = yes
+ as_fn_nop
+ return 1
+ cat confdefs.h - <<... >conftest.c
+ ac_fn_c_try_run 22845
+ as_lineno=22845 as_lineno_stack=as_lineno_stack=
+ using redirections: >&5 do {
+ ac_try='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5 '
+ ac_try_echo='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5 '
+ eval 'ac_try_echo="$as_me:22845: $CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5 "'
+ ac_try_echo='configure:22845: gcc -o conftest -isystem /usr/X11R7/include -I/usr/pkg/include -O -isystem /usr/pkg/include -D_REENTRANT -isystem /usr/pkg/include/cairo -isystem /usr/pkg/include -isystem /usr/pkg/include/freetype2 -isystem /usr/pkg/include -isystem /usr/pkg/include/libpng16 -isystem /usr/X11R7/include/pixman-1 -isystem /usr/X11R7/include -isystem /usr/X11R7/include    -L/usr/pkg/lib -R/usr/pkg/lib -R/usr/X11R7/lib -L/usr/X11R7/lib conftest.c -lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo >&5 '
+ printf '%s\n' 'configure:22845: gcc -o conftest -isystem /usr/X11R7/include -I/usr/pkg/include -O -isystem /usr/pkg/include -D_REENTRANT -isystem /usr/pkg/include/cairo -isystem /usr/pkg/include -isystem /usr/pkg/include/freetype2 -isystem /usr/pkg/include -isystem /usr/pkg/include/libpng16 -isystem /usr/X11R7/include/pixman-1 -isystem /usr/X11R7/include -isystem /usr/X11R7/include    -L/usr/pkg/lib -R/usr/pkg/lib -R/usr/X11R7/lib -L/usr/X11R7/lib conftest.c -lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo >&5 '
+ } done
+ using redirections: 2>&5 do subshell (
+ ) done subshell
+ ac_status=1
+ printf '%s\n' 'configure:22845: $? = 1' >&5
+ test 1 = 0
+ as_fn_nop
+ return 1
+ printf '%s\n' 'configure: program exited with status 1' >&5
+ printf '%s\n' 'configure: failed program was:' >&5
+ sed 's/^/| /' conftest.c >&5
+ ac_retval=1
+ rm -rf conftest.dSYM conftest_ipa8_conftest.oo
+ eval as_lineno_stack=
+ as_lineno_stack=''
+ unset as_lineno
+ as_fn_set_status 1
+ return 1
+ as_fn_nop
+ return 1
+ emacs_cv_tputs_lib=no
+ rm -f core '*.core' 'core.conftest.*' gmon.out bb.out conftest conftest.o conftest.beam conftest.c
+ LIBS='-lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo'
+ test Xno '!=' Xno
+ for tputs_library=tinfo
+ OLIBS='-lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo'
+ test -z tinfo
+ emacs_cv_tputs_lib=-ltinfo
+ LIBS='-ltinfo -lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo'
+ test no = yes
+ as_fn_nop
+ return 1
+ cat confdefs.h - <<... >conftest.c
+ ac_fn_c_try_run 22845
+ as_lineno=22845 as_lineno_stack=as_lineno_stack=
+ using redirections: >&5 do {
+ ac_try='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5 '
+ ac_try_echo='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5 '
+ eval 'ac_try_echo="$as_me:22845: $CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5 "'
+ ac_try_echo='configure:22845: gcc -o conftest -isystem /usr/X11R7/include -I/usr/pkg/include -O -isystem /usr/pkg/include -D_REENTRANT -isystem /usr/pkg/include/cairo -isystem /usr/pkg/include -isystem /usr/pkg/include/freetype2 -isystem /usr/pkg/include -isystem /usr/pkg/include/libpng16 -isystem /usr/X11R7/include/pixman-1 -isystem /usr/X11R7/include -isystem /usr/X11R7/include    -L/usr/pkg/lib -R/usr/pkg/lib -R/usr/X11R7/lib -L/usr/X11R7/lib conftest.c -ltinfo -lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo >&5 '
+ printf '%s\n' 'configure:22845: gcc -o conftest -isystem /usr/X11R7/include -I/usr/pkg/include -O -isystem /usr/pkg/include -D_REENTRANT -isystem /usr/pkg/include/cairo -isystem /usr/pkg/include -isystem /usr/pkg/include/freetype2 -isystem /usr/pkg/include -isystem /usr/pkg/include/libpng16 -isystem /usr/X11R7/include/pixman-1 -isystem /usr/X11R7/include -isystem /usr/X11R7/include    -L/usr/pkg/lib -R/usr/pkg/lib -R/usr/X11R7/lib -L/usr/X11R7/lib conftest.c -ltinfo -lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo >&5 '
+ } done
+ using redirections: 2>&5 do subshell (
+ ) done subshell
+ ac_status=1
+ printf '%s\n' 'configure:22845: $? = 1' >&5
+ test 1 = 0
+ as_fn_nop
+ return 1
+ printf '%s\n' 'configure: program exited with status 1' >&5
+ printf '%s\n' 'configure: failed program was:' >&5
+ sed 's/^/| /' conftest.c >&5
+ ac_retval=1
+ rm -rf conftest.dSYM conftest_ipa8_conftest.oo
+ eval as_lineno_stack=
+ as_lineno_stack=''
+ unset as_lineno
+ as_fn_set_status 1
+ return 1
+ as_fn_nop
+ return 1
+ emacs_cv_tputs_lib=no
+ rm -f core '*.core' 'core.conftest.*' gmon.out bb.out conftest conftest.o conftest.beam conftest.c
+ LIBS='-lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo'
+ test Xno '!=' Xno
+ for tputs_library=ncurses
+ OLIBS='-lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo'
+ test -z ncurses
+ emacs_cv_tputs_lib=-lncurses
+ LIBS='-lncurses -lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo'
+ test no = yes
+ as_fn_nop
+ return 1
+ cat confdefs.h - <<... >conftest.c
+ ac_fn_c_try_run 22845
+ as_lineno=22845 as_lineno_stack=as_lineno_stack=
+ using redirections: >&5 do {
+ ac_try='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5 '
+ ac_try_echo='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5 '
+ eval 'ac_try_echo="$as_me:22845: $CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5 "'
+ ac_try_echo='configure:22845: gcc -o conftest -isystem /usr/X11R7/include -I/usr/pkg/include -O -isystem /usr/pkg/include -D_REENTRANT -isystem /usr/pkg/include/cairo -isystem /usr/pkg/include -isystem /usr/pkg/include/freetype2 -isystem /usr/pkg/include -isystem /usr/pkg/include/libpng16 -isystem /usr/X11R7/include/pixman-1 -isystem /usr/X11R7/include -isystem /usr/X11R7/include    -L/usr/pkg/lib -R/usr/pkg/lib -R/usr/X11R7/lib -L/usr/X11R7/lib conftest.c -lncurses -lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo >&5 '
+ printf '%s\n' 'configure:22845: gcc -o conftest -isystem /usr/X11R7/include -I/usr/pkg/include -O -isystem /usr/pkg/include -D_REENTRANT -isystem /usr/pkg/include/cairo -isystem /usr/pkg/include -isystem /usr/pkg/include/freetype2 -isystem /usr/pkg/include -isystem /usr/pkg/include/libpng16 -isystem /usr/X11R7/include/pixman-1 -isystem /usr/X11R7/include -isystem /usr/X11R7/include    -L/usr/pkg/lib -R/usr/pkg/lib -R/usr/X11R7/lib -L/usr/X11R7/lib conftest.c -lncurses -lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo >&5 '
+ } done
+ using redirections: 2>&5 do subshell (
+ ) done subshell
+ ac_status=0
+ printf '%s\n' 'configure:22845: $? = 0' >&5
+ test 0 = 0
+ ac_try='./conftest$ac_exeext'
+ using redirections: >&5 do {
+ ac_try_echo='./conftest$ac_exeext'
+ eval 'ac_try_echo="$as_me:22845: ./conftest$ac_exeext"'
+ ac_try_echo='configure:22845: ./conftest'
+ printf '%s\n' 'configure:22845: ./conftest'
+ } done
+ using redirections: 2>&5 do subshell (
+ ) done subshell
+ ac_status=0
+ printf '%s\n' 'configure:22845: $? = 0' >&5
+ test 0 = 0
+ ':'
+ ac_retval=0
+ rm -rf conftest.dSYM conftest_ipa8_conftest.oo
+ eval as_lineno_stack=
+ as_lineno_stack=''
+ unset as_lineno
+ as_fn_set_status 0
+ return 0
+ ':'
+ rm -f core '*.core' 'core.conftest.*' gmon.out bb.out conftest conftest.o conftest.beam conftest.c
+ LIBS='-lsqlite3 -lX11   -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lcairo'
+ test X-lncurses '!=' Xno
+ break
+ printf '%s\n' 'configure:22862: result: -lncurses' >&5
+ printf '%s\n' -lncurses >&6
-lncurses
+ ':'
+ LIBS_TERMCAP=-lncurses
+ TERMINFO=yes
+ test x-lncurses '!=' x-lterminfo
+ TERMINFO=no
+ LIBS_TERMCAP=-ltermcap
+ TERMCAP_OBJ=tparam.o
+ test no = yes
+ test X-ltermcap = X-lncurses

bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

[Eli Zaretskii]

> Date: Wed, 12 Jul 2023 15:57:29 +0300
> From: Valtteri Vuorikoski <vuori@notcom.org>
> Cc: 64577@debbugs.gnu.org
> 
> >   netbsd)
> >     if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
> >       TERMINFO=no
> >       LIBS_TERMCAP="-ltermcap"
> >     fi
> > 
> > should have set TERMINFO=no, which disables linking against terminfo.
> > 
> > So I don't think I understand what is going on there.
> 
> Finding ncurses is good, but unfortunately ncurses isn't what ends up
> getting used. configure says:
> 
> checking for library containing tputs... -lncurses
> 
> but this happens:
> 
> $ ldd src/emacs | egrep 'term|curses'
>      -lterminfo.1 => /usr/lib/libterminfo.so.1
> 
> Disabling the test above results in src/emacs being linked with
> ncurses. That is, when I remove the whole "netbsd" special case
> from configure this is the result:
> 
> $ ldd ~/emacs/bin/emacs | egrep 'term|curses'
>    -lncurses.6 => /usr/pkg/lib/libncurses.so.6

So does the patch below fix the issue?

diff --git a/configure.ac b/configure.ac
index 19575e8..21c396f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -5212,7 +5212,7 @@ AC_DEFUN
     ;;
 
   netbsd)
-    if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
+    if test "x$LIBS_TERMCAP" != "x-lncurses"; then
       TERMINFO=no
       LIBS_TERMCAP="-ltermcap"
     fi

bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

[Valtteri Vuorikoski]

On Wed, Jul 12, 2023 at 04:32:39PM +0300, Eli Zaretskii wrote:
> So does the patch below fix the issue?
> 
> diff --git a/configure.ac b/configure.ac
> index 19575e8..21c396f 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -5212,7 +5212,7 @@ AC_DEFUN
>      ;;
>  
>    netbsd)
> -    if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
> +    if test "x$LIBS_TERMCAP" != "x-lncurses"; then
>        TERMINFO=no
>        LIBS_TERMCAP="-ltermcap"
>      fi

It fixes the immediate problem, yes. But I don't think this special
case should stick around, since it seems relevant only to a very
ancient version of NetBSD.

I tried out the following patch and it turns out that the thing that
causes tparam1 aborts is actually TERMINFO=no:

--- a/configure.ac
+++ b/configure.ac
@@ -5184,7 +5184,7 @@ AC_DEFUN
   ##  option to use it.
   darwin) LIBS_TERMCAP="-lncurses" ;;

-  gnu*) test -z "$LIBS_TERMCAP" && LIBS_TERMCAP="-lncurses" ;;
+  gnu*|netbsd) test -z "$LIBS_TERMCAP" && LIBS_TERMCAP="-lncurses" ;;

   freebsd)
     AC_MSG_CHECKING([whether FreeBSD is new enough to use terminfo])
@@ -5211,13 +5211,6 @@ AC_DEFUN
     LIBS_TERMCAP=
     ;;

-  netbsd)
-    if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
-      TERMINFO=no
-      LIBS_TERMCAP="-ltermcap"
-    fi
-    ;;
-
   openbsd | dragonfly) LIBS_TERMCAP="-lncurses" ;;

   ## hpux: Make sure we get select from libc rather than from libcurses

Without ncurses installed, this produces emacs linked with libterminfo
that no longer crashes with TERM=xterm-256color on NetBSD 9.3 and
shows 256 colors. With ncurses installed, libncurses is used and the
result is equally functional. Seems like pure win to me.

If users with ancient NetBSD versions experience problems with
libterminfo (that the original special case logic presumably avoided),
they can install ncurses as a workaround. 

 -Valtteri
 

bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

[Eli Zaretskii]

> Date: Wed, 12 Jul 2023 17:06:08 +0300
> From: Valtteri Vuorikoski <vuori@notcom.org>
> Cc: 64577@debbugs.gnu.org
> 
> > diff --git a/configure.ac b/configure.ac
> > index 19575e8..21c396f 100644
> > --- a/configure.ac
> > +++ b/configure.ac
> > @@ -5212,7 +5212,7 @@ AC_DEFUN
> >      ;;
> >  
> >    netbsd)
> > -    if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
> > +    if test "x$LIBS_TERMCAP" != "x-lncurses"; then
> >        TERMINFO=no
> >        LIBS_TERMCAP="-ltermcap"
> >      fi
> 
> It fixes the immediate problem, yes. But I don't think this special
> case should stick around, since it seems relevant only to a very
> ancient version of NetBSD.

We don't want to drop support of old versions just because we can, and
we don't really know how "ancient" those ancient versions should
actually be to require this special case.  We know that in May 2010 it
was TRT, but not when it stopped being TRT.  Without knowing that, I'd
like to avoid removing this, at least until we hear a real problem
with it.  Asking NetBSD users to install ncurses doesn't sound like a
problem to me.

> I tried out the following patch and it turns out that the thing that
> causes tparam1 aborts is actually TERMINFO=no:

That's strange, since termcap and terminfo are the same on NetBSD.
Maybe the problem is that TERMINFO=no means we don't compile
terminfo.c into Emacs, and/or also don't define TERMINFO, which could
bypass more code that's needed for terminfo?  In that case, just
avoiding TERMINFO=no, even if ncurses were not found, could be another
good measure.

> If users with ancient NetBSD versions experience problems with
> libterminfo (that the original special case logic presumably avoided),
> they can install ncurses as a workaround. 

That argument goes both ways, doesn't it?

bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

[Valtteri Vuorikoski]

On Wed, Jul 12, 2023 at 07:06:42PM +0300, Eli Zaretskii wrote:
> > It fixes the immediate problem, yes. But I don't think this special
> > case should stick around, since it seems relevant only to a very
> > ancient version of NetBSD.
> 
> We don't want to drop support of old versions just because we can, and
> we don't really know how "ancient" those ancient versions should
> actually be to require this special case.  We know that in May 2010 it
> was TRT, but not when it stopped being TRT.  Without knowing that, I'd
> like to avoid removing this, at least until we hear a real problem
> with it.  Asking NetBSD users to install ncurses doesn't sound like a
> problem to me.

Can't say I'm happy about keeping the magical special case, which will
result in a different thing happening than what configure prints out.
But at least ncurses-having users will win and the ncurses-preferring
logic will mostly bring things in line with other BSDs.

The reason not much has been heard about this is probably that most
users likely use packaged emacs, and the pkgsrc build system forcibly sets
TERMINFO=yes if it determines that this is a NetBSD with the relevant
headers available. See this package build makefile:
http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/editors/emacs21/Makefile.common?rev=1.21&content-type=text/x-cvsweb-markup
"CPPFLAGS+=-DTERMINFO" (similarly check exists for emacsen up to 28).

> That's strange, since termcap and terminfo are the same on NetBSD.
> Maybe the problem is that TERMINFO=no means we don't compile
> terminfo.c into Emacs, and/or also don't define TERMINFO, which could
> bypass more code that's needed for terminfo?  In that case, just
> avoiding TERMINFO=no, even if ncurses were not found, could be another
> good measure.

Looks like removing the TERMINFO=no line inside the if clause has the
same result as unconditionally using -lterminfo, which is to say that
emacs is linked with libterminfo, uses terminfo (and not the termcap
compatibility wrappers) and hence doesn't abort itself.

This seems to be what effectively happens in pkgsrc-packaged
emacs: TERMINFO=no is overridden and things largely proceed as if the
configure special case logic didn't exist, modulo the fact that
libterminfo is always used in preference to ncurses.

How about this then, this way should preserve the intent of the
original special casing while honoring the library selection configure
ended up with:

--- a/configure.ac
+++ b/configure.ac
@@ -5212,9 +5212,8 @@ AC_DEFUN
     ;;

   netbsd)
-    if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
+    if test "x$LIBS_TERMCAP" = "x-ltermcap"; then
       TERMINFO=no
-      LIBS_TERMCAP="-ltermcap"
     fi
     ;;

On ancient NetBSD (without ncurses) LIBS_TERMCAP should be detected as
-ltermcap, and this will ensure that TERMINFO stays undefined
(probably a no-op). On a newer system where someone actually wants to use
termcap and forces LIBS_TERMCAP, this will make sure that TERMINFO is
"no" even though libtermcap is a symlink to libterminfo (= only
the termcap compatibility functions will be used and terminfo stuff
will be ignored even though configure sees tputs() and friends).

The end result should be the same binary as pkgsrc packaging has
provided for years, except that on a system with ncurses installed
it will be used.

> > If users with ancient NetBSD versions experience problems with
> > libterminfo (that the original special case logic presumably avoided),
> > they can install ncurses as a workaround. 
> 
> That argument goes both ways, doesn't it?

Not really. With the netbsd = gnu* version, users with NetBSD versions
released in last ~10 years i.e. most NetBSD users get a good
experience out of the box regardless of whether ncurses is installed,
and users of ancient versions _might_ have problems without ncurses.

With the LIBS_TERMCAP!=-lncurses version, everyone without ncurses
gets a bad experience, assuming TERMINFO=no isn't removed.

 -Valtteri
 

bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

[Eli Zaretskii]

> Date: Wed, 12 Jul 2023 23:22:16 +0300
> From: Valtteri Vuorikoski <vuori@notcom.org>
> Cc: 64577@debbugs.gnu.org
> 
> On Wed, Jul 12, 2023 at 07:06:42PM +0300, Eli Zaretskii wrote:
> > > It fixes the immediate problem, yes. But I don't think this special
> > > case should stick around, since it seems relevant only to a very
> > > ancient version of NetBSD.
> > 
> > We don't want to drop support of old versions just because we can, and
> > we don't really know how "ancient" those ancient versions should
> > actually be to require this special case.  We know that in May 2010 it
> > was TRT, but not when it stopped being TRT.  Without knowing that, I'd
> > like to avoid removing this, at least until we hear a real problem
> > with it.  Asking NetBSD users to install ncurses doesn't sound like a
> > problem to me.
> 
> Can't say I'm happy about keeping the magical special case, which will
> result in a different thing happening than what configure prints out.

That happens on other systems as well, look at the code around the
netbsd case.  The fact that the configure script says "ncurses" is not
a promise that ncurses will be used, it's just a status report of that
particular test.

> The reason not much has been heard about this is probably that most
> users likely use packaged emacs, and the pkgsrc build system forcibly sets
> TERMINFO=yes if it determines that this is a NetBSD with the relevant
> headers available. See this package build makefile:
> http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/editors/emacs21/Makefile.common?rev=1.21&content-type=text/x-cvsweb-markup
> "CPPFLAGS+=-DTERMINFO" (similarly check exists for emacsen up to 28).

If you could find out since when did the NetBSD pkgsrc system start
using this override, it could be a data point that we could consider
in making these decisions.

> How about this then, this way should preserve the intent of the
> original special casing while honoring the library selection configure
> ended up with:
> 
> --- a/configure.ac
> +++ b/configure.ac
> @@ -5212,9 +5212,8 @@ AC_DEFUN
>      ;;
> 
>    netbsd)
> -    if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
> +    if test "x$LIBS_TERMCAP" = "x-ltermcap"; then
>        TERMINFO=no
> -      LIBS_TERMCAP="-ltermcap"
>      fi
>      ;;
> 
> On ancient NetBSD (without ncurses) LIBS_TERMCAP should be detected as
> -ltermcap, and this will ensure that TERMINFO stays undefined
> (probably a no-op). On a newer system where someone actually wants to use
> termcap and forces LIBS_TERMCAP, this will make sure that TERMINFO is
> "no" even though libtermcap is a symlink to libterminfo (= only
> the termcap compatibility functions will be used and terminfo stuff
> will be ignored even though configure sees tputs() and friends).
> 
> The end result should be the same binary as pkgsrc packaging has
> provided for years, except that on a system with ncurses installed
> it will be used.

I don't mind installing the above instead of my proposal, thanks.

bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

[Valtteri Vuorikoski]

On Thu, Jul 13, 2023 at 08:19:21AM +0300, Eli Zaretskii wrote:
> > The reason not much has been heard about this is probably that most
> > users likely use packaged emacs, and the pkgsrc build system forcibly sets
> > TERMINFO=yes if it determines that this is a NetBSD with the relevant
> > headers available. See this package build makefile:
> > http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/editors/emacs21/Makefile.common?rev=1.21&content-type=text/x-cvsweb-markup
> > "CPPFLAGS+=-DTERMINFO" (similarly check exists for emacsen up to 28).
> 
> If you could find out since when did the NetBSD pkgsrc system start
> using this override, it could be a data point that we could consider
> in making these decisions.

Looks like the TERMINFO override was installed in CVS version 1.7 in
May 2010 for emacs21:
http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/editors/emacs21/Makefile.common?rev=1.7&content-type=text/x-cvsweb-markup

According to the comments, NetBSD 6.0 released the same year was the
first version to ship with libterminfo. From looking at the platform
repo, the old termcap library was completely removed in this release
and got replaced by the current libterminfo.

> > How about this then, this way should preserve the intent of the
> > original special casing while honoring the library selection configure
> > ended up with:
> > 
> > --- a/configure.ac
> > +++ b/configure.ac
> > @@ -5212,9 +5212,8 @@ AC_DEFUN
> >      ;;
> > 
> >    netbsd)
> > -    if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
> > +    if test "x$LIBS_TERMCAP" = "x-ltermcap"; then
> >        TERMINFO=no
> > -      LIBS_TERMCAP="-ltermcap"
> >      fi
> >      ;;
> > 
> > On ancient NetBSD (without ncurses) LIBS_TERMCAP should be detected as
> > -ltermcap, and this will ensure that TERMINFO stays undefined
> > (probably a no-op). On a newer system where someone actually wants to use
> > termcap and forces LIBS_TERMCAP, this will make sure that TERMINFO is
> > "no" even though libtermcap is a symlink to libterminfo (= only
> > the termcap compatibility functions will be used and terminfo stuff
> > will be ignored even though configure sees tputs() and friends).
> > 
> > The end result should be the same binary as pkgsrc packaging has
> > provided for years, except that on a system with ncurses installed
> > it will be used.
> 
> I don't mind installing the above instead of my proposal, thanks.

Great. I looked at NetBSD 5.1 sources and expanded the check somewhat:

--- a/configure.ac
+++ b/configure.ac
@@ -5212,7 +5212,11 @@ AC_DEFUN
     ;;

   netbsd)
-    if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
+    # NetBSD versions prior to 6.0 lack native terminfo, but have a
+    # tputs() built on top of termcap in these libraries. Use native
+    # termcap instead in this case. NetBSD >= 6.0 has native terminfo
+    # implementation in -lterminfo.
+    if test "x$LIBS_TERMCAP" = "x-ltermcap" -o "x$LIBS_TERMCAP" = "x-lcurses"; then
       TERMINFO=no
       LIBS_TERMCAP="-ltermcap"
     fi

On 5.1 and older (no ncurses installed), configure's scan through
tputs_libraries matches either -ltermcap or -lcurses (both seem to
include tputs() in 5.1, but situation may differ in older versions),
but this will be termcap under the hood. Therefore force TERMINFO=no
and also force -ltermcap (as before) to avoid pulling in curses
unnecessarily.

On 6.0 and newer (again no ncurses) the order of tputs_libraries
should always cause -lterminfo to match first, so the branch will not
be taken and TERMINFO stays enabled.

Installing ncurses should now enable full terminfo support on any
version.

Tested on 9.3 with and without ncurses installed and the result is
full functionality with TERM=xterm-256color.

 -Valtteri
 

bug#64577: 29.0.92; configure on NetBSD forces unstable terminfo library

[Eli Zaretskii]

> Date: Thu, 13 Jul 2023 12:35:51 +0300
> From: Valtteri Vuorikoski <vuori@notcom.org>
> Cc: 64577@debbugs.gnu.org
> 
> > I don't mind installing the above instead of my proposal, thanks.
> 
> Great. I looked at NetBSD 5.1 sources and expanded the check somewhat:
> 
> --- a/configure.ac
> +++ b/configure.ac
> @@ -5212,7 +5212,11 @@ AC_DEFUN
>      ;;
> 
>    netbsd)
> -    if test "x$LIBS_TERMCAP" != "x-lterminfo"; then
> +    # NetBSD versions prior to 6.0 lack native terminfo, but have a
> +    # tputs() built on top of termcap in these libraries. Use native
> +    # termcap instead in this case. NetBSD >= 6.0 has native terminfo
> +    # implementation in -lterminfo.
> +    if test "x$LIBS_TERMCAP" = "x-ltermcap" -o "x$LIBS_TERMCAP" = "x-lcurses"; then
>        TERMINFO=no
>        LIBS_TERMCAP="-ltermcap"
>      fi
> 
> On 5.1 and older (no ncurses installed), configure's scan through
> tputs_libraries matches either -ltermcap or -lcurses (both seem to
> include tputs() in 5.1, but situation may differ in older versions),
> but this will be termcap under the hood. Therefore force TERMINFO=no
> and also force -ltermcap (as before) to avoid pulling in curses
> unnecessarily.
> 
> On 6.0 and newer (again no ncurses) the order of tputs_libraries
> should always cause -lterminfo to match first, so the branch will not
> be taken and TERMINFO stays enabled.
> 
> Installing ncurses should now enable full terminfo support on any
> version.
> 
> Tested on 9.3 with and without ncurses installed and the result is
> full functionality with TERM=xterm-256color.

Thanks, installed on the emacs-29 branch, and closing the bug.