"user with UID not found" error

"user with UID not found" error

[Christopher Batten]

Hello GUIX community!

I am a new guix user, and we are enjoying using guix our research and teaching. However, ever since I installed guix on our server I have been seeing this error:

  % guix pull
  user with UID 317241 not found

Things still worked so I didn't think much of it. However, whenever I try to use --container with guix shell I get this odd error:

  % guix shell -D -C hello
  user with UID 317241 not found
  Backtrace:
          14 (primitive-load "/home/cb535/.config/guix/current/bin/g?")
In guix/ui.scm:
   2238:7 13 (run-guix . _)
  2201:10 12 (run-guix-command _ . _)
In ice-9/boot-9.scm:
  1752:10 11 (with-exception-handler _ _ #:unwind? _ # _)
  1752:10 10 (with-exception-handler _ _ #:unwind? _ # _)
In guix/store.scm:
   659:37  9 (thunk)
   1298:8  8 (call-with-build-handler _ _)
   1298:8  7 (call-with-build-handler #<procedure 7f76707d57e0 at g?> ?)
In guix/status.scm:
    815:4  6 (call-with-status-report _ _)
In guix/scripts/environment.scm:
   953:13  5 (_)
In guix/store.scm:
  2168:25  4 (run-with-store #<store-connection 256.99 7f767075e7d0> ?)
In guix/scripts/environment.scm:
   638:34  3 (_ _)
In unknown file:
           2 (getpw 317241)
In ice-9/boot-9.scm:
  1685:16  1 (raise-exception _ #:continuable? _)
  1685:16  0 (raise-exception _ #:continuable? _)

  ice-9/boot-9.scm:1685:16: In procedure raise-exception:
  In procedure getpw: entry not found

The error messages mention of (getpw 317241) resulting in an "entry not found" makes me think these two issues are related? Maybe this is some kind of bad guix install? I am using CentOS 7.9.2009. Any thoughts or advice?

A maybe related but maybe unrelated issue is now I am seeing this:

  % guix pull
  user with UID 317241 not found
  guix pull: error: failed to load '/home/cb535/.config/guix/channels.scm': Permission denied

This didn't use to happen so I am not sure why this is happening now? I can definitely access channels.scm just fine directly.

  % ls -la /home/cb535/.config/guix/channels.scm
 -rw-rw-r--. 1 cb535 pug-cb535 288 Jun 24 09:26 /home/cb535/.config/guix/channels.scm

  % cat /home/cb535/.config/guix/channels.scm
  (use-modules (guix ci))
  (list
  (channel
   (name 'gn-bioinformatics)
    (url (string-append "https://git.genenetwork.org/"
          "guix-bioinformatics/guix-bioinformatics.git"))
   (branch "master"))
  (channel-with-substitutes-available
    %default-guix-channel "https://ci.guix.gnu.org"))

Any thoughts or advice greatly appreciated!

Best,
Chris

----------------------

Christopher F. Batten

Associate Professor
Computer Systems Laboratory
School of Electrical and Computer Engineering
Cornell University

email : cbatten@cornell.edu
web   : http://www.csl.cornell.edu/~cbatten
addr  : 323 Rhodes Hall, Ithaca, NY 14853
cell  : (617) 780-6222

Re: "user with UID not found" error

[Ricardo Wurmus]

Hi Christopher,

> I am a new guix user, and we are enjoying using guix our research and teaching. However, ever since I installed guix on our server I have been seeing this error:
>
>   % guix pull
>   user with UID 317241 not found

You need to run the system’s nscd daemon.

-- 
Ricardo

Re: "user with UID not found" error

[Christopher Batten]

Hi Ricardo,

Turns out we are using SSSD not NCSD on our Linux systems. It seems to be working fine:

  % service sssd status
  ...
   Active: active (running) since Sat 2022-01-15 15:02:04 EST; 6 months 9 days ago

Note that this works fine:

  % id --user
  317241
  % id -nu 317241
  cb535

Do you think there is any issue with the fact that my user ID is 317241, and that is greater than 65535?

Hmmm ... I tried this too:

  % guile
  scheme@(guile-user)> (getpw 317241)
  $1 = #("cb535" "*" 317241 123123 "Christopher Batten" "/home/cb535" "/bin/bash")

And that works fine.

Does this have something to do with how guix interacts with LDAP?

I am pretty confused why getpw works in guile but guix is saying "user with UID 317241 not found". I don't know enough about the guix internals to try and debug it ... maybe this has something to do with this UID is not available when the guix daemon is doing some of its magic?

Thoughts?
Chris

> On Jul 22, 2022, at 3:25 PM, Ricardo Wurmus <rekado@elephly.net> wrote:
> 
> 
> Hi Christopher,
> 
>> I am a new guix user, and we are enjoying using guix our research and teaching. However, ever since I installed guix on our server I have been seeing this error:
>> 
>>  % guix pull
>>  user with UID 317241 not found
> 
> You need to run the system’s nscd daemon.
> 
> -- 
> Ricardo

Re: "user with UID not found" error

[Ricardo Wurmus]

Hi Christopher,

> Turns out we are using SSSD not NCSD on our Linux systems. It seems to be working fine:

You will need to run NSCD if you want Guix software to be able to talk
to the system’s SSSD.  The reason is that SSSD works by having
applications load a library whereas NSCD implements a network protocol.
Software built with Guix cannot load the system libary due to ABI
conflict, but it *can* talk to NSCD over the network.

The system NSCD talks to the system SSSD, so there’s no problem there.

-- 
Ricardo

Re: "user with UID not found" error

[Christopher Batten]

Hmmm ... this made me think that running both was not a great idea?

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system-level_authentication_guide/usingnscd-sssd

But I am not an expert on any of this :) I guess the key is to just make sure you only enable caching for hosts in /etc/nscd.conf?

Best,
Chris

> On Jul 27, 2022, at 8:44 AM, Ricardo Wurmus <rekado@elephly.net> wrote:
> 
> 
> Hi Christopher,
> 
>> Turns out we are using SSSD not NCSD on our Linux systems. It seems to be working fine:
> 
> You will need to run NSCD if you want Guix software to be able to talk
> to the system’s SSSD.  The reason is that SSSD works by having
> applications load a library whereas NSCD implements a network protocol.
> Software built with Guix cannot load the system libary due to ABI
> conflict, but it *can* talk to NSCD over the network.
> 
> The system NSCD talks to the system SSSD, so there’s no problem there.
> 
> -- 
> Ricardo

Re: "user with UID not found" error

[Chris Marusich]

FYI: https://lists.gnu.org/archive/html/guix-devel/2022-02/msg00263.html

Sorry for the top post; I'm not on my usual computer.

On Wed, Jul 27, 2022 at 12:51 PM Christopher Batten <cbatten@cornell.edu>
wrote:

>
> Hmmm ... this made me think that running both was not a great idea?
>
>
> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system-level_authentication_guide/usingnscd-sssd
>
> But I am not an expert on any of this :) I guess the key is to just make
> sure you only enable caching for hosts in /etc/nscd.conf?
>
> Best,
> Chris
>
> > On Jul 27, 2022, at 8:44 AM, Ricardo Wurmus <rekado@elephly.net> wrote:
> >
> >
> > Hi Christopher,
> >
> >> Turns out we are using SSSD not NCSD on our Linux systems. It seems to
> be working fine:
> >
> > You will need to run NSCD if you want Guix software to be able to talk
> > to the system’s SSSD.  The reason is that SSSD works by having
> > applications load a library whereas NSCD implements a network protocol.
> > Software built with Guix cannot load the system libary due to ABI
> > conflict, but it *can* talk to NSCD over the network.
> >
> > The system NSCD talks to the system SSSD, so there’s no problem there.
> >
> > --
> > Ricardo
>
>

Re: "user with UID not found" error

[Christopher Batten]

>>> > On Jul 27, 2022, at 8:44 AM, Ricardo Wurmus <rekado@elephly.net> wrote:
>>> > 
>>> > Hi Christopher,
>>> > 
>>> >> Turns out we are using SSSD not NCSD on our Linux systems. It seems to be working fine:
>>> > 
>>> > You will need to run NSCD if you want Guix software to be able to talk
>>> > to the system’s SSSD.  The reason is that SSSD works by having
>>> > applications load a library whereas NSCD implements a network protocol.
>>> > Software built with Guix cannot load the system libary due to ABI
>>> > conflict, but it *can* talk to NSCD over the network.
>>> > 
>>> > The system NSCD talks to the system SSSD, so there’s no problem there.
>>> > 
>>> > -- 
>>> > Ricardo
>> 
>> On Wed, Jul 27, 2022 at 12:51 PM Christopher Batten <cbatten@cornell.edu> wrote:
>> 
>> Hmmm ... this made me think that running both was not a great idea?
>> 
>> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system-level_authentication_guide/usingnscd-sssd
>> 
>> But I am not an expert on any of this :) I guess the key is to just make sure you only enable caching for hosts in /etc/nscd.conf?
>> 
>> Best,
>> Chris
> 
> On Jul 28, 2022, at 9:14 PM, Chris Marusich <cmmarusich@gmail.com> wrote:
> 
> FYI: https://lists.gnu.org/archive/html/guix-devel/2022-02/msg00263.html

Thanks for everyone's help! I think I understand now: the key is if we want to use guix, then we really do want to run NSCD and SSSD on these RHEL/CentOS systems if at all possible.

So I think we just now need to follow the instructions here?

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system-level_authentication_guide/usingnscd-sssd

Or do you recommend a different way to run NSCD and SSSD at the same time on RHEL/CentOS systems? 

Once we figure this out maybe we should add a note about this in the Guix docs here:

https://guix.gnu.org/manual/en/html_node/Application-Setup.html

Especially if Fedora is intending to remove NSCD?

Thanks!
Chris

Re: "user with UID not found" error

[Jack Hill]

On Mon, 1 Aug 2022, Christopher Batten wrote:

>>>>> On Jul 27, 2022, at 8:44 AM, Ricardo Wurmus <rekado@elephly.net> wrote:
>>>>>
>>>>> Hi Christopher,
>>>>>
>>>>>> Turns out we are using SSSD not NCSD on our Linux systems. It seems to be working fine:
>>>>>
>>>>> You will need to run NSCD if you want Guix software to be able to talk
>>>>> to the system’s SSSD.  The reason is that SSSD works by having
>>>>> applications load a library whereas NSCD implements a network protocol.
>>>>> Software built with Guix cannot load the system libary due to ABI
>>>>> conflict, but it *can* talk to NSCD over the network.
>>>>>
>>>>> The system NSCD talks to the system SSSD, so there’s no problem there.
>>>>>
>>>>> --
>>>>> Ricardo
>>>
>>> On Wed, Jul 27, 2022 at 12:51 PM Christopher Batten <cbatten@cornell.edu> wrote:
>>>
>>> Hmmm ... this made me think that running both was not a great idea?
>>>
>>> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system-level_authentication_guide/usingnscd-sssd
>>>
>>> But I am not an expert on any of this :) I guess the key is to just make sure you only enable caching for hosts in /etc/nscd.conf?
>>>
>>> Best,
>>> Chris
>>
>> On Jul 28, 2022, at 9:14 PM, Chris Marusich <cmmarusich@gmail.com> wrote:
>>
>> FYI: https://lists.gnu.org/archive/html/guix-devel/2022-02/msg00263.html
>
> Thanks for everyone's help! I think I understand now: the key is if we want to use guix, then we really do want to run NSCD and SSSD on these RHEL/CentOS systems if at all possible.
>
> So I think we just now need to follow the instructions here?
>
> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system-level_authentication_guide/usingnscd-sssd
>
> Or do you recommend a different way to run NSCD and SSSD at the same time on RHEL/CentOS systems?
>
> Once we figure this out maybe we should add a note about this in the Guix docs here:
>
> https://guix.gnu.org/manual/en/html_node/Application-Setup.html
>
> Especially if Fedora is intending to remove NSCD?
>
> Thanks!
> Chris

I think I've run into a similar issue trying to run packs as well (in my 
case on AlmaLinux 9). I produced a pack with

`guix pack -f tarball -RR -S /bin=bin screen sssd glibc`

and running screen from the pack reports:

"getpwuid() can't identify your account!"

screen might be a special-case of program, but it seems the problem exists 
in some way even when not trying to install Guix.

Best,
Jack

Re: "user with UID not found" error

[Ricardo Wurmus]

>Or do you recommend a different way to run NSCD and SSSD at the same time on RHEL/CentOS systems? 

You only need to start it. We don't use it as a cache. We are only interested in its network interface for use with glibc.


-- 
Ricardo

Re: "user with UID not found" error

[Christopher Batten]

Hi Ricardo,

>> Or do you recommend a different way to run NSCD and SSSD at the same time on RHEL/CentOS systems? 
> 
> You only need to start it. We don't use it as a cache. We are only interested in its network interface for use with glibc.

OK! We started NSCD:

 %  service nscd status
 Redirecting to /bin/systemctl status nscd.service
* nscd.service - Name Service Cache Daemon
   Loaded: loaded (/usr/lib/systemd/system/nscd.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2022-08-02 12:20:13 EDT; 1h 24min ago
  Process: 87689 ExecStart=/usr/sbin/nscd $NSCD_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 87691 (nscd)
    Tasks: 7
   Memory: 596.0K
   CGroup: /system.slice/nscd.service
           `-87691 /usr/sbin/nscd

We turned off all NSCD caching:

 % grep enable-cache /etc/nscd.conf
#       enable-cache		<service> <yes|no>
	enable-cache		passwd		no
	enable-cache		group		no
	enable-cache		hosts		no
	enable-cache		services	no
	enable-cache		netgroup	no

Still no luck though?

 % guix pull
 user with UID 317241 not found

and

 % guix shell -D -C hello
 user with UID 317241 not found
 ...
 ice-9/boot-9.scm:1685:16: In procedure raise-exception:
 In procedure getpw: entry not found

Thoughts? Sorry for all the questions ...
Chris

Re: "user with UID not found" error

[Ricardo Wurmus]

Hi Chris,

>>> Or do you recommend a different way to run NSCD and SSSD at the same time on RHEL/CentOS systems? 
>> 
>> You only need to start it. We don't use it as a cache. We are only interested in its network interface for use with glibc.
[…]
> We turned off all NSCD caching:
>
>  % grep enable-cache /etc/nscd.conf
> #       enable-cache		<service> <yes|no>
> 	enable-cache		passwd		no
> 	enable-cache		group		no
> 	enable-cache		hosts		no
> 	enable-cache		services	no
> 	enable-cache		netgroup	no

Sorry, it appears that I was wrong about the role of caching.  Our
cluster nodes (running CentOS) have this nscd config:

--8<---------------cut here---------------start------------->8---
enable-cache passwd yes
enable-cache group yes
enable-cache hosts no
enable-cache netgroup no
--8<---------------cut here---------------end--------------->8---

So while we don’t rely on caching per se, nscd needs to be configured to
cache passwd and group so that it actually fetches this type of
information from the system directories (e.g. LDAP).

Sorry for the confusion!

-- 
Ricardo

Re: "user with UID not found" error

[Christopher Batten]

>>>> Or do you recommend a different way to run NSCD and SSSD at the same time on RHEL/CentOS systems? 
>>> 
>>> You only need to start it. We don't use it as a cache. We are only interested in its network interface for use with glibc.
> […]
>> We turned off all NSCD caching:
>> 
>> % grep enable-cache /etc/nscd.conf
>> #       enable-cache		<service> <yes|no>
>> 	enable-cache		passwd		no
>> 	enable-cache		group		no
>> 	enable-cache		hosts		no
>> 	enable-cache		services	no
>> 	enable-cache		netgroup	no
> 
> Sorry, it appears that I was wrong about the role of caching.  Our
> cluster nodes (running CentOS) have this nscd config:
> 
> --8<---------------cut here---------------start------------->8---
> enable-cache passwd yes
> enable-cache group yes
> enable-cache hosts no
> enable-cache netgroup no
> --8<---------------cut here---------------end--------------->8---
> 
> So while we don’t rely on caching per se, nscd needs to be configured to
> cache passwd and group so that it actually fetches this type of
> information from the system directories (e.g. LDAP).
> 
> Sorry for the confusion!

OK! But this RHEL doc:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system-level_authentication_guide/usingnscd-sssd

Seems to suggest turning on NSCD caching for passwd and group while also running SSSD can cause subtle issues? It says:

"To avoid this problem, enable caching only for hosts in the the /etc/nscd.conf file and rely on the SSSD cache for the passwd, group, services, and netgroup entries."

So my sysadmin and I are worried about turning on caching in NSCD and SSSD at the same time? Are you running both and have you seen any issues?

Best,
Chris

Re: "user with UID not found" error

[Ricardo Wurmus]

Christopher Batten <cbatten@cornell.edu> writes:

> So my sysadmin and I are worried about turning on caching in NSCD and
> SSSD at the same time? Are you running both and have you seen any
> issues?

We have been running SSSD and NSCD together since a number of years on
all cluster nodes and all RHEL servers.  There haven’t been any problems
due to caching.

I suppose you could also configure NSCD to cache entries only for a
second to force it to talk to SSSD more often.  We never felt the need
to change the default configuration.

-- 
Ricardo

Re: "user with UID not found" error

[Christopher Batten]

Well ... we tried running NSCD and set it up to cache passwd and group but I am still getting the "user with UID 317241 not found" warnings and cannot use --container with shell.

We must not have something setup correctly ... I will have to look into this more later this fall. We can still use guix we just have that warning and cannot take advantage of the --container argument to shell or guix pack.

I really appreciate everyone's help and we definitely have learned something!
Chris

> On Aug 2, 2022, at 4:53 PM, Ricardo Wurmus <rekado@elephly.net> wrote:
> 
> 
> Christopher Batten <cbatten@cornell.edu> writes:
> 
>> So my sysadmin and I are worried about turning on caching in NSCD and
>> SSSD at the same time? Are you running both and have you seen any
>> issues?
> 
> We have been running SSSD and NSCD together since a number of years on
> all cluster nodes and all RHEL servers.  There haven’t been any problems
> due to caching.
> 
> I suppose you could also configure NSCD to cache entries only for a
> second to force it to talk to SSSD more often.  We never felt the need
> to change the default configuration.
> 
> -- 
> Ricardo