Hi Colton, what you're describing is not the bug in eshell but rather a
bash feature; no other shell will perform expansion on literal tilde in
PATH (you can disable it in Bash using POSIXLY_CORRECT environment
variable).  If you drop the quotes then it's going to work as expected
because the shell will expand the tilde to HOME before storing it in PATH:

PATH=~/test-bin:$PATH emacs -Q

Best,
Jakub Ječmínek