I'm running GNU Emacs 24.3.1, with the -nw flag. If have TERM=xterm or TERM=xterm-256color, it takes 2 or 3 seconds to start up. Setting TERM=ansi (or vt100, or vt102), I get instant startups. I'm running on goobuntu (Google's own flavor of ubuntu linux)
Tracing the emacs process as it starts up, I see that all the time is spent in a set of select() calls (fd 3 is "/dev/tty" at this point):
15215 pselect6(4, [3], [], NULL, {1, 996360390}, {NULL, 8}) = 1 (in [3], left {1, 980201052}) <0.016169>
15215 pselect6(4, [3], [], NULL, {1, 999612379}, {NULL, 8}) = 0 (Timeout) <2.001673>
15215 pselect6(4, [3], [], NULL, {0, 499948991}, {NULL, 8}) = 0 (Timeout) <0.500516>
15215 pselect6(4, [3], [], NULL, {27, 497597665}, {NULL, 8}) = 1 (in [3], left {27, 83731691}) <0.413916>
15215 pselect6(4, [3], [], NULL, {0, 499864020}, {NULL, 8}) = 1 (in [3], left {0, 287729575}) <0.212174>