(For more resources on Ubuntu, see here.)
The basic setup
This article will be based around a typical workplace or school you'll find almost anywhere. There is a proxy server wedged between users and the internet, as well as Windows throughout for the servers, meaning Windows-based NTLM authentication. With the advent of Linux servers and even Linux operating systems being deployed on desktops, the problems with proxy servers may not become non-issue quite soon.
Outgoing connections are blocked on all ports but 80 and 443 (those defaulting for HTTP and HTTPS) to 'help' security, creating a problem for those using various services such as games, chat and peer-to-peer sharing.
How it all works
Any packets sent from a computer that is headed towards an external host will reach the proxy server first, which will check if you are authenticated by passing on login information to the authentication server. Authentication using the 'basic' method is rare nowadays where NTLMv2 is widespread among large, internal domain networks. If the user is authenticated (and hence allowed to use the internet) and the port is allowed, then the packet will be passed on to the target host.
When a computer makes a HTTP request using Firefox, for example, everything works as expected. Firefox 'understands' the NTLMv2 protocol, and the request returns successfully. However, most other programs, especially those using the command line and not integrated with Gnome, generally only support basic authentication and things go haywire.
A proxy server for a proxy server
The solution to this authentication problem lies with NTLMaps, which is a proxy server that installs on the computer locally. It can handle the NTLMv2 protocol smoothly, and handles this for programs that can't. Once it's installed, you can point programs to connect through this proxy (without needing to supply authentication) and packets will pass through this proxy, and then be transferred to the 'real' proxy server, with authentication. NTLMaps was originally written to allow wget to make requests on a problematic network like this, and it works very well.
NTLMaps is available in the Ubuntu repository:
apt-get install ntlmaps
Debconf will then ask you for some information for NTLMaps. For the port number, enter any port that isn't used by listened on by another daemon on your computer. 8080, 5865, or even 12345 work just fine. After this, enter the 'real' proxy server which NTLMaps will connect to. For example, 'proxy', or '10.148.88.13'. Do not enter the 'real' proxy's port number.
In the next step, you can provide the port number that the 'real' proxy listens on. This is usually port 8080, and defaults as such. In the following steps, enter the domain name, user name and password that you wish to authenticate NTLMaps with. If you have Windows computers on the domain, you can see the domain name by checking the dropdown on the login dialog (e.g. 'CURRIC4126').
If you wish to configure NTLMaps again to add or change these settings, you can either edit the NTLMaps configuration file or use Debconf to do this process again:
nano /etc/ntlmaps/server.cfg; service ntlmaps restart
Now that NTLMaps is installed and running, you may point your programs to use the local proxy server. The Gnome 'Network proxy' window has a bug in which the authentication user name and password did not carry through to the environment variables when set.
Having NTLMaps brings the added bonus of not having this problem, as no 'client-side' authentication information needs to be entered.
Downloading packages through a proxy server
After pointing programs to use the NTLMaps proxy server by using the Gnome 'Network proxy' dialog (gnome-network-properties), the proxy environment variables (HTTP_PROXY, http_proxy, etc.) should be set to something like http://localhost:12345/. You should make this setting system-wide (click "Apply System-Wide...") so that these environment variables are set when logged in as root.
You should now be able to download and install or upgrade packages using Synaptic, apt-get or any other package management suite without any problems. If, however, the Gnome proxy settings don't set the environment variables, or take effect, you can force the proxy server by adding this line to /etc/apt/apt.conf (this is quite inconvenient as there's no "location" support and you must remove this line if you use the internet at home):
Remember to make sure you change the port number if you are using a different one for NTLMaps, though.
(For more resources on Ubuntu, see here.)
Remotely accessing an external computer with ssh
After all this success, logging in to an outside computer using ssh (secure shell) still isn't quite possible. For a start, sshd (the server) usually runs on port 22 by default, and it's most likely the situation that port 22 is blocked from outgoing connections. To change this, edit the configuration file on the target computer:
and add this lines after the line with "Port 22":
This will make sshd listen on both ports 22 and 443. Please note that you will have to sacrifice the use of a daemon on these ports, such as Apache, by stopping these, or moving them to other port numbers. Remember to restart sshd to allow these changes to take effect:
ssh supports working through a proxy with the "ProxyCommand" option. You will need to install corkscrew to allow these packets to be routed through a proxy server. Corkscrew doesn't support NTLMv2 authentication, which is not a problem now that NTLMaps is installed. First off, install corkscrew:
apt-get install corkscrew
After corkscrew is installed, edit either the system-wide ssh configuration or your own:
and add this line:
ProxyCommand corkscrew localhost 12345 %h %p
Remember that you don't need to provide authentication as NTLMaps handles this, and remember to change 12345 to another port if NTLMaps doesn't run on port 12345. Once this is set up, try connecting to another server (your home computer, maybe):
ssh -p443 delan.ath.cx
If you want to use a proxy 'one-off' without writing the setting in a configuration file, you can use the following:
ssh -o 'ProxyCommand corkscrew localhost 12345 %h %p' -p443 delan.ath.cx
Copying files to and from an external host
scp is a powerful and very useful program that allows the remote copying of files over a ssh connection. Because of this, ssh configuration options take effect here too, so if you've set up ssh to connect through NTLMaps, scp will do so too. The '-o' option for overriding a setting once works when using scp too. To copy a file from an outside computer:
scp firstname.lastname@example.org:/home/delan/Documents/invoice /home/delan/Documents/invoice
Conversely, to copy to an outside computer, use:
scp /home/delan/Documents/results email@example.com:/home/delan/Documents/results
Using the X desktop remotely
ssh supports X forwarding, a technique that allows you to open graphical applications or a whole X session over a network or the internet. Ideally, you should have a connection of a few megabytes a second, so naturally, X forwarding works best over a LAN.
However, X forwarding works quite well over the internet too; and it's still much more efficient than VNC. X forwarding is enabled by default on ssh clients, but you should switch this on at the target computer before it'll work. To do this, edit the sshd configuration file:
then find this line:
and uncomment it by removing the hash at the start. Save the file and restart sshd:
Now you'll be able to open X applications just by running their command. To check if X forwarding is working without launching an application, use:
If you see a line, such as "localhost:10.0", everything is working. A blank line indicates that X forwarding is disbaled.
At this point, you can run any program individually on your current X server (e.g. "gcalctool") just fine. However, you can take X forwarding to the next level by running the full Gnome session on another tty. To do this, switch to a real tty terminal (for no apparent reason, you can't start an X server using a windowed terminal) and log in. Launch a new X server:
xinit -- :1
The X server will launch on the next tty (usually tty8 or tty9) and you'll see a small xterm. Now ssh in and open a Gnome session:
ssh -p443 delan.ath.cx gnome-session
You will now log in graphically; exactly as if you were sitting at the target computer. Please note that closing the xterm window will kill the Gnome session and return you back to just the xterm.
This article explained in detail how to get the most out of restricted internet on Ubuntu (or any other Linux operating system), using package managers, ssh, scp, and X forwarding.
- Jailbreaking the iPad - in Ubuntu
- Network Based Ubuntu Installations
- Ubuntu 9.10: How To Upgrade
- Install GNOME-Shell on Ubuntu 9.10 Karmic Koala
- Five Years of Ubuntu
- Ubuntu User Interface Tweaks
- What's New In Ubuntu 9.10 "Karmic Koala"
- Securely Encrypt Removable Media with Ubuntu
- Folding @ Home on Ubuntu: Cancer Research Made Easy
- Securing Network Services with FreeBSD Jails
- Create a Local Ubuntu Repository using Apt-Mirror and Apt-Cacher