Re: --batch-size and --jobs combination

[Konstantin Ryabitsev <konstantin@linuxfoundation.org>]

On Thu, Jul 29, 2021 at 10:06:29PM +0000, Eric Wong wrote:
> My gut says 1g batch-size seems too high (Xapian has extra
> overhead) and could still eat too much into the kernel cache
> (and slow down reads). 100m might be a more reasonable limit
> for jobs=4 and 128G RAM.

Okay, I have things up and running on one of the 4 edge nodes. You can access
it and kick the tires at https://x-lore.kernel.org/. Initial observations:

- I can't give any kind of reliable numbers for initial importing/indexing, as
  I was doing it piecemeal for a while to make sure that the indexer hooks
  were doing the right thing. Besides, this is a live system serving a lot of
  (static) content from the same partition where the indexing was done, so I/O
  was routinely under high and unpredictable load. Final import/index took 40+
  hours, but I'll have more reliable numbers once I do it on 3 other systems.

- Performance in /all/ seems laggy at times, probably depending on whether
  lvmcache has Xapian DBs in SSD cache or not. After a period of laggy
  performance, speed seems to dramatically improve, which is probably when
  most of the backend is in cache.

- URLs are mapped a bit wonkily right now -- / redirects to /all/, since I
  expect that would be what most devs would want (pending feedback, I could be
  totally wrong). Wwwlisting is mapped to https://x-lore.kernel.org/lists.html
  since that's the URL currently containing the full archive. All of this may
  and will probably change.

- I will bring up the rest of the nodes throughout the week, so
  x-lore.kernel.org will become more geoip-balanced. I will share any other
  observations once I have more data. Once all 4 nodes are up, I will share
  this more widely with kernel devs so they can kick some tires and report
  whether they are seeing decreased performance compared to current
  lore.kernel.org. It's entirely possible that my plan to use
  mirrors.edge.kernel.org nodes for this isn't one of my brightest ideas, in
  which case I may bring up several dedicated instances in multiple clouds
  instead.

Thanks for all your work, Eric.

Best regards,
-K