Sure. You just have to balance it against convenience, as well as consider effort vs value.
Changing the SSH port is fine, and having to remember/teach people that it's a non-standard port is pretty easy. Although it is security-by-obscurity, it's decent value because you're significantly less likely to get dictionary attacks. Fairly low value, but also very low effort.
Restricting SSH connections to specific source IPs further reduces your risk, but adds inconvenience: you have to be in a specific place or use a VPN first, or remember to add new IPs for new people that need to connect. If you have only your office white-listed, and something happens to your office, now what?
Using port knocking can provide even more security-by-obscurity, but is much more inconvenient to connect to, harder to train new people on, etc. I've not used this myself but I'd also be worried about possibility of it not working.
When you consider these in the context of effort vs value, I'm not sure they're really there. They definitely add some security value, but it's a pretty tiny amount compared to something like using key-based authentication. Arguably both are a bit more secure than just changing the SSH port, but that comes at a significantly higher effort.
Changing the SSH port is fine, and having to remember/teach people that it's a non-standard port is pretty easy. Although it is security-by-obscurity, it's decent value because you're significantly less likely to get dictionary attacks. Fairly low value, but also very low effort.
Restricting SSH connections to specific source IPs further reduces your risk, but adds inconvenience: you have to be in a specific place or use a VPN first, or remember to add new IPs for new people that need to connect. If you have only your office white-listed, and something happens to your office, now what?
Using port knocking can provide even more security-by-obscurity, but is much more inconvenient to connect to, harder to train new people on, etc. I've not used this myself but I'd also be worried about possibility of it not working.
When you consider these in the context of effort vs value, I'm not sure they're really there. They definitely add some security value, but it's a pretty tiny amount compared to something like using key-based authentication. Arguably both are a bit more secure than just changing the SSH port, but that comes at a significantly higher effort.