[Unixcert] W00t! I got the FTP Tunnel lab to work!!!

Glen Jarvis glen at glenjarvis.com
Wed Sep 5 08:18:39 UTC 2012


I *finally* got the ftp tunnel lab to work. Several people asked me about
this but I couldn't give very good help. In fact, now I know the answer, I
think I hurt instead of helped... Let me try to rectify this...


I've taken this bigger lab and broken it down into three smaller labs (to
help get all of the concepts):

1) ssh tunnels in general

2) ftp protocol

3) putting them together (our lab for class)



SSH Tunnels

===========

For those of us who just can't let it go. Here is a series of smaller labs
that builds the concepts and gets us to use ftp over SSH tunnels.


Part 1 - The Basics

===============

    The goal of this mini exercise is just to get the basics for the SSH
tunnel syntax. There are three parameters to SSH that create tunnels:

a) Normal tunnel (-L option)

b) Reverse tunnel (-R option)

c) Dynamic tunnels (-D option)


We just want to create a basic -L ssh tunnel for now. This web link is a
great resource to show how to do that:

http://chamibuddhika.wordpress.com/2012/03/21/ssh-tunnelling-explained/


1) Just like our normal lab write-up,

*You will need 2 computers to perform this exercise. If you are completing
this in the lab, choose a partner to work with. Each of you must complete
the following steps.*

I'm going to connect from my Macbook (I will call my "client") to a SuSE
virtual machine (I will call my "server"). The ssh daemon has to be running
on the server (my SuSE box in my example). This is important because:

a) You'll be connecting to that ssh daemon, so therefore the daemon has to
be running

b) The ssh daemon is doing the tunneling work behind the scenes.


2) We're going to deviate for quite a while to build up this concept. Our
immediate objective is to set-up a scenario when this may be appropriate.
(To show a why). It will take us a few steps to show this, however.


2a) Turn off the firewall on the server

2b) Start apache on the server

2c) From the client go to server's web page (e.g., http://server/). You
should see a page that you'll recognize later -- this is what we'll use to
see if this worked when the firewall is back up.

2d) Now, start the firewall

2e) Configure the firewall so that all ports are closed EXCEPT ssh (22)


3) Try to go to the server's web page again (e.g., http://server/). it
should fail.

4) Go to the server and start a web browser there. Go to this page:

    http://127.0.0.1/

   This should work since the firewall was configured to protect external
ports, not loopback ports.


Now, we're in a situation where there is a webserver running on the server.
We can see it running when on the server. But, from the client, we can't
get to it (The firewall blocks us). Let's hack it! :)


5) We're going to work around the firewall by using our ssh port (and some
port forwarding on the server).


Type this command in the client (substituting the IP address of your client
and server). (This command will appear to 'hang', but in reality is
working… This behavior is the -N option that we added).


ssh <your username>@<your server> -L 8030:localhost:80 -N


For example, if my username was 'glenjarvis' and my server IP was
'192.168.1.126', I'd type this:


ssh glenjarvis at 192.168.1.126 -L 8030:localhost:80 -N


There's nothing special about port 8030 -- I made it up. You can make up
any number that you want. You'll just have to remember to use it instead of
8030 when we get to the next part.


6) In a web browser, go to this address:


     http://127.0.0.1:8030


    (Or substitute the port number you used in part 5 above).


You should have been able to see your web page again (even though the
firewall blocked the Apache port). How does this work?


We connect to the server via port 22. We additionally say anything that, ON
THE SERVER, we are going to connect to our loop back IP address, port 80
(the part that worked in part 4 above). That connection will magically look
like it's on the client's port 8030.


So, when the client goes to 8030, that really gets tunneled to the server.
The server instead redirects that traffic to it's local 127.0.0.1 machine
and the port of 80. This is the local web server. Thus, the client looks
like it has a webserver on it's port 8030; but, in reality, this is on the
server -- the client web browser never knows.



Part 2 - FTP

=========

FTP is a tricky protocol. It's tricky because:


a) It opens up two ports to talk on (instead of the traditional one). If
you think about it, this makes sense. One is the actual data pipe. And, the
other is the control pipe. If you were downloading a 2 Gig file over a
modem that may take 2 hours to download, you'd like to interrupt it and
make it stop.


But, how does the stop signal get up there? It would be fighting with all
the data in the pipe. Instead, this way, it has it's own port so commands
won't be logged or lost.


b) It opens some of these ports up on the fly when it needs them. And,
knowing what it is doing can be tricky.


A good way to see this is to analyze Wireshark or tcpdump traffic just for
a FTP communication. This is so painstakingly tedious and can take 2-3
hours of tedium to do. However, I think after doing this just once, you'll
have such a solid understanding of what FTP is doing.


1) Turn off firewall on the server again

2) Turn on FTP daemon on the server

3) on the client type this:

    ftp <server IP>

4) Change your password to something that you won't be embarrassed to see
on the network (i.e., don't use a real password; this lab will be insecure).

5) Start wireshark

6) from the client type: ftp <servers IP address>. Output should be similar
to this (answering questions as appropriate below):


prompt> ftp 192.168.1.126

Connected to 192.168.1.126.

220-Welcome to Pure-FTPd.

220-You are user number 1 of 50 allowed.

220-Local time is now 23:17. Server port: 21.

220-IPv6 connections are also welcome on this server.

220 You will be disconnected after 15 minutes of inactivity.

500 This security scheme is not implemented

Name (192.168.1.126:gjarvis_old): glenjarvis

331 User glenjarvis OK. Password required

Password:

230-User glenjarvis has group access to:  users      video

230 OK. Current directory is /home/glenjarvis

Remote system type is UNIX.

Using binary mode to transfer files.

ftp> ls

200 PORT command successful

150 Connecting to port 54004

drwxr-xr-x    2 glenjarvis users            4096 Jul 22 02:53 Desktop

drwxr-xr-x    2 glenjarvis users            4096 Jul 22 02:51 Downloads

-rw-r--r--    1 glenjarvis users              22 Sep  4 23:16 README.txt

-rw-r--r--    1 glenjarvis users              43 Sep  4 23:16 sudo

-rw-r--r--    1 glenjarvis users          253019 Aug 25 14:40
typescript_20120825_process_accounting.txt

-rw-r--r--    1 glenjarvis users              43 Sep  4 23:16 yast

226-Options: -l

226 6 matches total

ftp> get README.txt

local: README.txt remote: README.txt

200 PORT command successful

150 Connecting to port 54005

226-File successfully transferred

226 0.003 seconds (measured here), 7.06 Kbytes per second

22 bytes received in 0.0098 seconds (2.2 Kbytes/s)

ftp> quit

221-Goodbye. You uploaded 0 and downloaded 1 kbytes.

221 Logout.


7) Analyze/Review each packet between your client and your server. Can you
reconstruct the communication between them? What does the Data portion of
the packets look like? What does the PORT data look like? How does this
information tie to the IP addresses and port numbers of your client's and
servers?

I did a similar analysis and the instructor forwarded it in an email to
everyone. It's tedious to go through, but *really* helps because you can
see exactly what FTP is doing.

In the above output, I did an "ls" command first, and then I did a "get."
How many different communication channels got built? (A very good hint if
you do an ls).



Part 3 - Putting 1 and 2 together

=========================

This picks back up where we left off regarding the class lab

Turn the firewall off if it isn't already. We should have these three steps
completed now:

*1. You will need 2 computers to perform this exercise. If you are
completing this in the lab, choose a partner to work with. Each of you must
complete the following steps.*

*2. Turn off the Firewall*

*3. Make sure FTP is turned on*


Now, to start on the monsterly hard step:

*4. Set-up an SSH tunnel to run FTP between your system and you neighbor’s
system*


Client box:

sudo ssh glenjarvis@<server IP address> -L 21:localhost:21 -N


So, we are connecting the clients port 21 to really shoot over to the
server, and then go through the server's localhost (loopback interface) on
ports 21.

Note, that it can be confusing with both a sudo and a password. For
example, this command has sudo so it needs the client's account password to
execute the sudo. The, when this is successful, it asks the server's
password for glenjarvis's account (unless you already have ssh keys
established to connect without passwords).

You might see output asking for a password twice. If they are different, as
they were on my computer, that can be a source of confusion.


prompt> sudo ssh glenjarvis at 192.168.1.126 -L 21:localhost:21 -N

Password:

Password:



*5. Use Active mode so you know the ports that will be used*


I didn't really have to do anything with this. I had active by default.


*6. Use two tunnels (ports 21 and 20)*

I think I cheated a little -- or maybe not -- depending on what was
intended. Now that I'm writing this up after I had a successful connection,
I think I could reduce (but add complexity) to the number of steps that I
took. Since I got this to work, I'll document what I did and we can come
back and experiment.


We made a tunnel to port 21 already. Let's do the same for port 20 (data
transfer). Instead of "crossing the streams", it made more sense to me to
have the client accessing it's localhost and then having the forwards also
access their localhost. So, the client should just look like he's sitting
on the server.

Again, there may be an extra inefficient step.… but, let's stick to what
worked for now....


sudo ssh glenjarvis@<server ip address> -L 20:localhost:20 -N


The problem now comes to predicting the next port. I spun my wheels here
forever. But, I learned a few good things:

* I thought I was so clever in using wireshark to find the predicted port
number. I thought I was even more clever for figuring out to listen on the
loopback device with wireshark because the ethernet was encrypted. I
thought I was so clever to figure out how to multiply the data in the port
command (256 * fifth number in the stream + sixth number in the stream).
I'm an idiot…  The ftp client output tells you what port it's trying to
connect to. Wireshark was overkill, confusing and unnecessary. Just look at
the ftp output when you try to retrieve the file.

* You know, why not let one port fail -- this way you can see what port
number that it tried. Then, you can predict the next one from this number.

* Use the -v (verbose flags) when using ssh. They can help you see what is
going on:

ssh -vvv glenjarvis at 192.168.1.126 -L 54161:localhost:54161 -N

Output:

debug1: channel 2: free: direct-tcpip: listening port 54172 for localhost
port 54172, connect from 127.0.0.1 port 20, nchannels 3

debug3: channel 2: status: The following connections are open:

  #2 direct-tcpip: listening port 54172 for localhost port 54172, connect
from 127.0.0.1 port 20 (t4 r0 i3/0 o3/0 fd 6/6 cc -1)



Finally -- the way I did my second port was probably silly. But, because I
was making a port-to-port -- local-to-local connection between my client
and my server, to make the prediction actually work (after a billion
failures, let me tell ya ;), I had to make my port forward command execute
from the SERVER back to the client (since that's what the FTP server does
when it builds the port):

This worked for me:

server_prompt> ssh -vvv glenjarvis@<clients IP address> -L
54172:localhost:54172 -N


I previously trying to make Reverse connections (-R flag) from the client
machine to the server. But, this isn't the same thing and it kept
failing..  I *think* (but am not certain) that I can combine this above
step and the following step into one statement (it's a theory only):

client_prompt> sudo ssh glenjarvis at 192.168.1.126 -L 20:localhost:20 -N


*7. Transfer a file*

This part just works if the above ports are configured properly. And, it
fails miserably if it does not. I tried setting up 30 ports in advance of
this one to catch the one that I was missing. It would just eventually skip
over them. It was so frustrating. But, once you get the ports set up, this
just works.

Also, because I knew that an "ls" used up this port and a new one would be
used for the data transfer. I avoided doing anything but a simple 'get' of
the file name that I knew would work:

prompt> ftp localhost

Connected to localhost.

220-Welcome to Pure-FTPd.

220-You are user number 1 of 50 allowed.

220-Local time is now 00:08. Server port: 21.

220-IPv6 connections are also welcome on this server.

220 You will be disconnected after 15 minutes of inactivity.

500 This security scheme is not implemented

Name (localhost:gjarvis_old): glenjarvis

331 User glenjarvis OK. Password required

Password:

230-User glenjarvis has group access to:  users      video

230 OK. Current directory is /home/glenjarvis

Remote system type is UNIX.

Using binary mode to transfer files.

ftp> get README.txt

local: README.txt remote: README.txt

200 PORT command successful

425 Could not open data connection to port 54131: Connection refused

ftp> get README.txt

local: README.txt remote: README.txt

200 PORT command successful

425 Could not open data connection to port 54134: Connection refused

ftp> get README.txt

local: README.txt remote: README.txt

200 PORT command successful

425 Could not open data connection to port 54135: Connection refused


<snip out SO MANY failed attempts>


ftp> get README.txt

local: README.txt remote: README.txt

200 PORT command successful

425 Could not open data connection to port 54171: Connection refused

ftp> get README.txt

local: README.txt remote: README.txt

200 PORT command successful

150 Connecting to port 54172

226-File successfully transferred

226 0.037 seconds (measured here), 0.58 Kbytes per second

22 bytes received in 0.039 seconds (0.55 Kbytes/s)


W00t! It worked!!!


*8. Try SFTP instead – what is the difference?*


<I'll leave this one for an academic exercise>




Cheers,


Glen
-- 

"Pursue, keep up with, circle round and round your life as a dog does his
master's chase. Do what you love. Know your own bone; gnaw at it, bury it,
unearth it, and gnaw it still."

--Henry David Thoreau
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.noisebridge.net/pipermail/unixcert/attachments/20120905/71a0de8c/attachment-0002.html>


More information about the Unixcert mailing list