It's 2024! Please avoid writing SSH commands like that.
Instead, configure your ~/.ssh/config with LocalForward, RemoteForward, and ProxyJump. This can save you a significant amount of time, especially when using ssh, scp, or rsync to transfer data from a remote server that requires multiple intermediate SSH connections.
e.g:
Host jump-host-1
HostName jump1.example.com
User your_username
IdentityFile ~/.ssh/id_rsa
Host jump-host-2
HostName jump2.example.com
User your_username
IdentityFile ~/.ssh/id_rsa
ProxyJump jump-host-1
Host jump-host-3
HostName jump3.example.com
User your_username
IdentityFile ~/.ssh/id_rsa
ProxyJump jump-host-2
Host target-server
HostName target.example.com
User your_username
IdentityFile ~/.ssh/id_rsa
ProxyJump jump-host-3
LocalForward 0.0.0.0:8080 0.0.0.0:80
RemoteForward 0.0.0.0:9022 0.0.0.0:22
# after this:
# - you can ssh/scp/rsync to your target-server via an alias
# - forward traffic FROM port 80 on your target-server to port 8080 on your local machine
# - forward ssh requests TO port 9022 on your target-server to port 22 on your local machine
# - remember, for LocalForward & RemoteForward :
# + left is target-server
# + right is your local
# + use 0.0.0.0 instead of localhost or 127.0.0.1
While we're sharing neat ssh_config tricks, here's my favorite trick I use:
My home network is set up so that if I'm home or on my self-hosted VPN, I can SSH directly to my various things. But if I'm away from home and not on the VPN, I can SSH into my home systems through a jump host.
In the ssh_config file, I have it configured to detect how/where I am and optionally use a jump host.
Host jump jump.example.org
HostName jump.example.org
Port 41444
User mmh
UserKnownHostsFile /dev/null
ChallengeResponseAuthentication no
CheckHostIP no
Compression yes
ForwardX11 no
GSSAPIAuthentication no
LogLevel ERROR
PreferredAuthentications publickey,keyboard-interactive
ProxyJump none
PermitLocalCommand yes
# Order here matters. Detect VPN first, then home network.
# If connecting to a *.example.org host and router.example.org = 10.0.0.1, must be home/vpn.
Match host *.example.org exec "getent ahosts router.example.org | grep -q ^10.0.0.1"
ProxyJump none
# If connecting to a *.example.org host and the macaddr of 10.0.0.1 is NOT 2a:70:ff:ff:ff:ff, then use jump.example.org:
Match host *.example.org exec "! arp -ne 10.0.0.1 | grep -Fq 2a:70:ff:ff:ff:ff"
ProxyJump jump.example.org
## Define the things
Host tv tv.example.org
HostName tv.example.org
User mmh
I think that using 0.0.0.0 it's a bad idea. That is supposedly opening the port in all network interfaces, including the external ones. So, if you don't have a firewall (especially on the remote server) you are exposing something to the world.
OTOH if I'm going to use some tunnelling/port forwarding quite often, I would use the config file option, but for an one time or sporadic use, the command line option is better IMHO.
Nice catch. You're right.
At my company all servers operate inside a complex & heavily-guarded intranet, so I usually use 0.0.0.0 instead of localhost / 127.0.0.1. Sometimes, only using the former worked (e.g: using Code-Server or Jupyter Notebook), and I'm not so good at networking to dive into iptables and firewall things.
SSH tunnelling is an utter necessity in the ridiculous corporate environment I work in. Incredible amounts of bureaucracy and sometimes weeks of waiting to get access to stuff, get ports opened, get some exception in their firewalls and vpn so someone can access a thing they need to do their job.
This guide mentions -D but doesn't really articulate quite how powerful it is if you don't know what it does.
ssh -D 8888 someserver, set your browser's SOCKS proxy to localhost:8888 (firefox still lets you set this without altering system defaults). Now all your browser's traffic is routed via someserver.
That was pretty much my standard way to browse the web away from home in the mid 2000s. But when I actually got a corporate job they had whitelisted IP addresses so I couldn't even get an SSH connection to some random box on the net. I was so miserable I started to look into setting up http tunnel and somehow getting a box I controlled whitelisted. But instead of going that far I just changed jobs.
It isn't a good idea to circumvent corporate environment networks. they're there for a reason, and doing it shows a lack of professionalism and dis-respect for the organization process, procedures, and security. Yes it takes weeks/months to get access, then it takes weeks/months to get access. You don't want to be held liable for opening a backdoor to confidential information, or compromising their security.
Exactly. It's not a good idea to bypass policies at work. Just because you don't know why the policy is there or you disagree with the reason, it doesn't mean you can ignore the policy.
If you can't get your job done, then escalate the issue to your manager. You not being able to get your work done because of other teams is the kind of problem they're supposed to be solving.
Bypassing corporate policy at work is risky. You might bring down negative consequences on yourself or your workplace. You have to understand what you are doing. You have to understand likely reactions.
But also, bypassing corporate policy can have benefits. If I'm more productive or get a reputation as the guy who gets things done or don't get seen as a complainer or just generally produce results because I bypassed a policy, those are all benefits. If I can transform "hey boss, it's gonna be another week on this project because I'm waiting on a policy exemption" to "here it is", that's a benefit.
You have to weigh whether the benefits outweigh the risks for you.
"Waiting for corporate to punch a hole through three firewalls for me to get access to the test server :P"
I was on a project once where a consultant had dropped their laptop and it had taken a week or two to get fixed. After that everyone had to use a laptop provided by the client. When we scaled up the project with 3 more developers the project manager who had set up this policy discovered that the lead time for 3 dev laptops meant that the new developers got to be bored for a month at a fairly high hourly rate.
Sometimes they are. Sometimes that reason is long forgotten, or isn't really valid anymore, or is an overprotective measure and not really a good reason in the first place. Quite often it doesn't justify waiting weeks or months to get it changed.
That is an awfully specific question. Here are a few examples of what could happen though:
- Malicious code on a webpage compromises your computer.
- You download unauthorized software to install, which possibly even comes from a known-bad source.
- Your employer could have trouble establishing that their patent is legitimate because you accessed documentation from a competitor.
Even if the worker avoids liability for costly mistakes, the company will be set back. You can also be fired for breaking rules like that even when there are no actual damages.
The filthiest SSH tunneling hack that I've ever done was at 3AM while in a three-way... datacenter connection. The interesting part of that, while the three facilities, spaced out over a single metro area had upstream transit connectivity to the rest of the net, only two pairs were able to reach the other due to some odd routing policies that weren't able to be resolved in time.
That meant that A could connect to B, and only B could connect to C. The data I had to move from facility A to facility C via B in the most ridiculous rsync+ssh tunnel+keys+routing shenanigan mashup I've ever done. It took a few tries to get the incantation exactly right, but it was magical seeing it all move as one.
Looking back it is super obvious how I'd do it now, but back then being green, was a huge accomplishment. I still remember the exhilaration when I confirmed everything was synced up.
> It lowers the throughput due to more overhead and increases the latency. On connections with packet loss or high latencies (e.x. satellite) it can cause a TCP meltdown.
This actually isn't a problem with SSH tunnels unless you're using TAP/TUN, because It unpacks and forwards the TCP streams. But you can still get reduced performance with multiple channels due to head of line blocking.
I love the extra detail in the visualizations. My wish is for networking to have much more visual representation of traffic, especially at lower level connections.
Hi.. Check out the diagrams here: https://www.nathanhandy.blog/articles/osi-model-revisited.ht... .. obviously this is only a static conceptual representation. Most network vendors will have some form of visual representation of traffic, but it's tyipcally only discreet metrics / graphs.
That is a fantastic example. I will definitely try your HandyDash as well. We should have had the ability to see this detailed traffic breakdown 20 years ago.
SSH tunnels are an excellent tool, but nowadays you often want TLS and reverse proxy functionality built in. I maintain a list of such tools here: https://github.com/anderspitman/awesome-tunneling
Your suggested solution Cloudflare Tunnels man in the middles the traffic and it’s not an end to end tunnel. It’s a tunnel to Cloudflare! The users should be warned about this!
I learned how to use ssh tunnels when wanting to bypass a firewall in my university network around 15 years ago, had to change the default port to 443.
Been using it ever since for so much more than just bypassing firewalls.
I use it for proxying general internet traffic (such as from your web browser) using the SOCKS5 proxy described in the article. Combined with FoxyProxy or similar it's nice if you want certain traffic (such as to a certain domain which only allows certain IP blocks) to flow from a certain host based on things like the domain.
I think you could do that with a virtual IP. For some reason my firewall/router doesn't communicate DHCP option 67 correctly, it sends its own address no matter what I do so I had to set up a a virtual IP/rule to route all PXE boot traffic on whatever port that is going to the routers IP, over to the real PXE boot server instead.
I've found VS Code can setup port forwarding tunnels if you remote into a host and its been very useful. Its graphical, no command line incantations to remember and I usually have it running anyways.
It's 2024! Please avoid writing SSH commands like that.
Instead, configure your ~/.ssh/config with LocalForward, RemoteForward, and ProxyJump. This can save you a significant amount of time, especially when using ssh, scp, or rsync to transfer data from a remote server that requires multiple intermediate SSH connections.
e.g:
While we're sharing neat ssh_config tricks, here's my favorite trick I use:
My home network is set up so that if I'm home or on my self-hosted VPN, I can SSH directly to my various things. But if I'm away from home and not on the VPN, I can SSH into my home systems through a jump host.
In the ssh_config file, I have it configured to detect how/where I am and optionally use a jump host.
You should probably replace that "arp -ne" command with "ip neigh show" instead (assuming this is Linux).
Wow. Nice trick! I didn't know SSH Config can do that exec control flow.
This is really cool, I didnt know you could use "exec".
I think that using 0.0.0.0 it's a bad idea. That is supposedly opening the port in all network interfaces, including the external ones. So, if you don't have a firewall (especially on the remote server) you are exposing something to the world.
OTOH if I'm going to use some tunnelling/port forwarding quite often, I would use the config file option, but for an one time or sporadic use, the command line option is better IMHO.
Nice catch. You're right. At my company all servers operate inside a complex & heavily-guarded intranet, so I usually use 0.0.0.0 instead of localhost / 127.0.0.1. Sometimes, only using the former worked (e.g: using Code-Server or Jupyter Notebook), and I'm not so good at networking to dive into iptables and firewall things.
This is interesting. I always made a bash function to proxyjump. This seems cleaner. I'll try it.
SSH tunnelling is an utter necessity in the ridiculous corporate environment I work in. Incredible amounts of bureaucracy and sometimes weeks of waiting to get access to stuff, get ports opened, get some exception in their firewalls and vpn so someone can access a thing they need to do their job.
This guide mentions -D but doesn't really articulate quite how powerful it is if you don't know what it does.
ssh -D 8888 someserver, set your browser's SOCKS proxy to localhost:8888 (firefox still lets you set this without altering system defaults). Now all your browser's traffic is routed via someserver.
I find that to be incredibly useful.
That was pretty much my standard way to browse the web away from home in the mid 2000s. But when I actually got a corporate job they had whitelisted IP addresses so I couldn't even get an SSH connection to some random box on the net. I was so miserable I started to look into setting up http tunnel and somehow getting a box I controlled whitelisted. But instead of going that far I just changed jobs.
It isn't a good idea to circumvent corporate environment networks. they're there for a reason, and doing it shows a lack of professionalism and dis-respect for the organization process, procedures, and security. Yes it takes weeks/months to get access, then it takes weeks/months to get access. You don't want to be held liable for opening a backdoor to confidential information, or compromising their security.
Exactly. It's not a good idea to bypass policies at work. Just because you don't know why the policy is there or you disagree with the reason, it doesn't mean you can ignore the policy.
If you can't get your job done, then escalate the issue to your manager. You not being able to get your work done because of other teams is the kind of problem they're supposed to be solving.
I think that statement is pretty short-sighted.
Bypassing corporate policy at work is risky. You might bring down negative consequences on yourself or your workplace. You have to understand what you are doing. You have to understand likely reactions.
But also, bypassing corporate policy can have benefits. If I'm more productive or get a reputation as the guy who gets things done or don't get seen as a complainer or just generally produce results because I bypassed a policy, those are all benefits. If I can transform "hey boss, it's gonna be another week on this project because I'm waiting on a policy exemption" to "here it is", that's a benefit.
You have to weigh whether the benefits outweigh the risks for you.
New version of https://xkcd.com/303/ ?
"Waiting for corporate to punch a hole through three firewalls for me to get access to the test server :P"
I was on a project once where a consultant had dropped their laptop and it had taken a week or two to get fixed. After that everyone had to use a laptop provided by the client. When we scaled up the project with 3 more developers the project manager who had set up this policy discovered that the lead time for 3 dev laptops meant that the new developers got to be bored for a month at a fairly high hourly rate.
Sometimes they are. Sometimes that reason is long forgotten, or isn't really valid anymore, or is an overprotective measure and not really a good reason in the first place. Quite often it doesn't justify waiting weeks or months to get it changed.
Can you cite any examples of damage resulting from personal browsing over an SSH tunnel that the worker was held liable for?
That is an awfully specific question. Here are a few examples of what could happen though:
- Malicious code on a webpage compromises your computer.
- You download unauthorized software to install, which possibly even comes from a known-bad source.
- Your employer could have trouble establishing that their patent is legitimate because you accessed documentation from a competitor.
Even if the worker avoids liability for costly mistakes, the company will be set back. You can also be fired for breaking rules like that even when there are no actual damages.
The filthiest SSH tunneling hack that I've ever done was at 3AM while in a three-way... datacenter connection. The interesting part of that, while the three facilities, spaced out over a single metro area had upstream transit connectivity to the rest of the net, only two pairs were able to reach the other due to some odd routing policies that weren't able to be resolved in time.
That meant that A could connect to B, and only B could connect to C. The data I had to move from facility A to facility C via B in the most ridiculous rsync+ssh tunnel+keys+routing shenanigan mashup I've ever done. It took a few tries to get the incantation exactly right, but it was magical seeing it all move as one.
Looking back it is super obvious how I'd do it now, but back then being green, was a huge accomplishment. I still remember the exhilaration when I confirmed everything was synced up.
just check my comment in in this post using `~/.ssh/config` with ProxyJump, you can virtually jump between A B C D E ... or whatever.
Yep, had I known that nearly 20 years ago I would have done just that. That's the ideal way to do it now!
If it makes you feel any better, ProxyJump was only added in 2016
> TCP-over-TCP
> It lowers the throughput due to more overhead and increases the latency. On connections with packet loss or high latencies (e.x. satellite) it can cause a TCP meltdown.
This actually isn't a problem with SSH tunnels unless you're using TAP/TUN, because It unpacks and forwards the TCP streams. But you can still get reduced performance with multiple channels due to head of line blocking.
I love the extra detail in the visualizations. My wish is for networking to have much more visual representation of traffic, especially at lower level connections.
Hi.. Check out the diagrams here: https://www.nathanhandy.blog/articles/osi-model-revisited.ht... .. obviously this is only a static conceptual representation. Most network vendors will have some form of visual representation of traffic, but it's tyipcally only discreet metrics / graphs.
That is a fantastic example. I will definitely try your HandyDash as well. We should have had the ability to see this detailed traffic breakdown 20 years ago.
Thanks for the feedback. I hope you find HandyDash useful. If you find any issues feel free to raise here: https://github.com/Nathan-Handy/HandyDash/issues
sshuttle go have a read much nicer for tunnelling... sshuttle -r user@host 10.0.0.0/8
Anything on 10/8 automatic tunnel it's pretty much a vpn over ssh
SSH tunnels are an excellent tool, but nowadays you often want TLS and reverse proxy functionality built in. I maintain a list of such tools here: https://github.com/anderspitman/awesome-tunneling
Your suggested solution Cloudflare Tunnels man in the middles the traffic and it’s not an end to end tunnel. It’s a tunnel to Cloudflare! The users should be warned about this!
I learned how to use ssh tunnels when wanting to bypass a firewall in my university network around 15 years ago, had to change the default port to 443.
Been using it ever since for so much more than just bypassing firewalls.
What purpose have you enjoyed it for beyond bypassing firewalls and exposing local services across a network?
I use it for proxying general internet traffic (such as from your web browser) using the SOCKS5 proxy described in the article. Combined with FoxyProxy or similar it's nice if you want certain traffic (such as to a certain domain which only allows certain IP blocks) to flow from a certain host based on things like the domain.
In essence it is what you mentioned, these are a few practical uses:
- Streaming region locked content from overseas.
- Permanent reverse-tunnel for remote-access with autossh.
- Increased security compared to making services visible to the internet.
- Downloading scientific articles using my university's connection as a proxy.
Kind of related, but I was wondering if there is some kind of redirect functionality in SSH itself. Something like:
- A wants to SSH into B
- B tells A that it must connect to C instead
- A transparently connects to C directly
- B is not a part of the critical data path anymore
Does something like this exist?
B could port forward (as in route packets?) to C, but I don't think there are any HTTP Permanent Redirect equivalents, no.
Maybe you can explain the problem more and perhaps there's a more suitable solution?
If you have a host that's somewhat embedded, you can have DNS handle the "routing" for you. You will have to handle fingerprint verification.
I think you could do that with a virtual IP. For some reason my firewall/router doesn't communicate DHCP option 67 correctly, it sends its own address no matter what I do so I had to set up a a virtual IP/rule to route all PXE boot traffic on whatever port that is going to the routers IP, over to the real PXE boot server instead.
It would be misleading if A doesn't know that the real target is C.
Otherwise you can use jump functionality
From A:
If B doesn't need to be part of the path, just connect to C directly if it's doable. If it's not, then B will have to be a hop either way.I've found VS Code can setup port forwarding tunnels if you remote into a host and its been very useful. Its graphical, no command line incantations to remember and I usually have it running anyways.
Also setting up code tunnels on the workstations I use regularly saves me a lot of headaches.
Sure it's a crutch, ugly and probably unsafe but I'm not a networking guy and need to get my actual stuff done.
I've used tunneling quite a lot over the years but never knew about -J option.
What I'd really like is just some visual tool to configure my tunnels instead of spending 30 minutes very few months when I need to use a tunnel.
See also:
A Visual Guide to SSH Tunnels: Local and Remote Port Forwarding https://iximiuz.com/en/posts/ssh-tunnels/
Bookmarked, thank you.