Try logging in through another machine. If it feels sluggish then there is something wrong with the network. But its just my assumption so run MTR. And try to boot the machine in rescue mode and log in via ssh so that you can be sure its not the network.
Drop into sh, exec sh - speedier? Is terminal compression enabled in your client? It’s enabled by default in /etc/ssh/sshd_config, but client needs to implement as well. Which part is slow? Acknowledgement on the keystroke from the terminal or running a command in terminal?
Very possible, always could setup smokeping and monitor it to various endpoints and monitor it over a period of time.
On another note, if your dealing with ~100+ms connections all the time and you want a silky smooth SSH session, you could look into using/setting up mosh (https://mosh.org/).
It’s also very handy for road warrior users as it handles dropped connections well (think of tweaked screen or tmux). Simple to setup and no real difference once its setup then a regular ssh session (ssh runs on top).
Won’t bore everyone here why its smoother/faster, but have a little research/read about it, if the SSH delay gets annoying.