In that case, I suspect that some of your customizations somehow
temporarily reset display-line-numbers to nil. You could try to use
Lisp watchpoints (see "Watching Variables" in the ELisp manual) for a
single buffer where this happens, to find out which code does that.
Alternatively, instrument your functions that reset
display-line-numbers to log messages to some buffer with useful
information, such as the Lisp backtrace, and when the problem happens,
examine the log for clues.
Kaushal Modi