I think we should remove that, and leave behind an alias that uses
utf-8-hfs, which is provided by Emacs. There's no reason to maintain
2 identical definitions.
When you set file-name-coding-system to nil, Emacs uses
default-file-name-system, which is utf-8, so it doesn't
compose/decompose characters, and that's why you see what you see.
IOW, using nil is a step backward.
What does this return:
M-: (file-name-all-completion "åäö" "/that/empty/directory/") RET
Also, what is your value of completion-ignore-case?