UDP/TCP Analysis

Exclusive offer: get 50% off this eBook here
Network Analysis using Wireshark Cookbook

Network Analysis using Wireshark Cookbook — Save 50%

Over 80 recipes to analyze and troubleshoot network problems using Wireshark with this book and ebook

€23.99    €12.00
by Yoram Orzach | December 2013 | Architecture & Analysis Cookbooks Networking & Telephony Open Source

In this article by Yoram Orzach, the author of Network Analysis Using Wireshark Cookbook, covers the following recipes:

  • Configuring TCP and UDP preferences for troubleshooting
  • TCP connection problems
  • TCP retransmissions – where they come from and why

(For more resources related to this topic, see here.)

Configuring TCP and UDP preferences for troubleshooting

In most of the cases you can use the default Wireshark parameters for TCP and UDP network analysis, but there are also some changes that can be configured. The changes will be configured in the Preferences window.

Getting ready

For TCP or UDP configuration:

  1. Start Wireshark, and from the Edit menu, choose Statistics.
  2. Under Protocols, choose TCP or UDP.

How to do it...

We will see how to configure TCP and UDP preferences.

UDP parameters

Let's see some parameters that can influence the capture of UDP:

You can configure the following parameters in UDP:

  • Show UDP summary in protocol tree: Mark this button if you want the UDP summary line to be shown in the protocol tree (set by default)
  • Try heuristic sub-dissectors first: Try to decode a packet using the heuristic method before using a sub-dissector registered to the specific port
  • Validate the UDP checksum if possible: Validates the UDP checksum
  • Collect process flow information: Collects process flow information

By default only the first parameter is set. In most of the cases it is enough.

TCP parameters

You can configure the following parameters in TCP:

  • Show TCP summary in protocol tree: Mark this button if you want the TCP summary line to be shown in the protocol tree (set by default).

  • Validate the TCP checksum if possible: This feature can slow down performance. In most cases it is not required.

  • Allow subdissector to reassemble TCP streams: This option is for stream analysis (set by default).

  • Relative sequence numbers: When this is set, Wireshark will show you every TCP connection that starts from Seq=0.

  • Track number of bytes in flight: This setting enables Wireshark to track the number of unacknowledged bytes flowing on the network (set by default).

  • Calculate conversation timestamps: This feature enables the calculations of TCP timestamps option.

  • Try heuristic sub-dissectors first: Try to decode a packet using heuristic method before using a sub-dissector registered to the specific port.

  • Ignore TCP Timestamps in summary: Ignore the timestamp option in the TCP header.

  • Do not call subdissector for error packets: This option does not analyze erroneous TCP packets.

How it works...

There are some parameters in the TCP preferences that I would like to say a few words about.

Referring to relative sequence numbers, when you look at a TCP connection you see that it always starts with sequence numbers equal to zero. These are the relative numbers that are normalized to zero by Wireshark. The real numbers are numbers between 0 and 232, picked by the TCP process, which are difficult to follow. The TCP standard does not set any rule for picking this number.

The calculating conversations timestamps refers to the timestamp option of the TCP packet. The TCP timestamps option carries two 4-byte timestamp fields, as seen in the next illustration:

The problem that the timestamp option comes to solve is the sensitivity of TCP to delay variations. The solution, and written in RFC 1323, is to use TCP options in the following ways (for every TCP connection):

  • The sender places a timestamp in each data segment that it sends (the Tsval field)
  • The receiver reflects these timestamps in ACK segments (the Tsecr field)

Then, a single subtraction gives the sender an accurate RTT measurement for every ACK segment (which will correspond to every other data segment, with a sensible receiver). This mechanism is called Round Trip Time Measurement (RTTM).

There's more…

UDP is a very simple protocol with a very simple header that includes only four fields: source port, destination port, packet length, and checksum. Checksum is used by the receiver to check whether to accept the packet or drop it. In case of a packet drop, there is no recovery mechanism. In some cases, the application will recover it (for example, DNS that sends the request again), and in some cases it won't.

TCP is more sophisticated. It is a connection-oriented, reliable protocol, with sequencing mechanism, flow, and congestion control. These subjects will be discussed later in this article

SCTP is a reliable, connection-oriented protocol that allows the transfer of multiple streams per connection, optional bundling of multiple user messages into a single SCTP packet, support for cookies, multi-homing, and other mechanisms. It was initially developed for carrying signaling messages in cellular networks, and later implemented with other application protocols.

Network Analysis using Wireshark Cookbook Over 80 recipes to analyze and troubleshoot network problems using Wireshark with this book and ebook
Published: December 2013
eBook Price: €23.99
Book Price: €38.99
See more
Select your format and quantity:

TCP connection problems

When two TCP processes wish to communicate, they open the connection, send the data, and then close the connection. This happens when you open a browser to the Internet, connect from your mail client to the mail server, or connect with Telnet to your router or any other application that works over TCP.

When TCP opens the connection, it sends a request for open connection from the source port to destination port.

Some problems can occur during the establishment or closing of the application. Using Wireshark to locate and solve these problems is the goal of this recipe.

Getting ready

If you experience one of the following problems, use Wireshark in order to find out what is the reason for it.

These problems can be of many types. Of these:

  • You try to run an application and it does not work. You try to browse the Internet and you don't get any response.
  • You try to use your mail but you don't have a connection to the mail server.
  • Problems can be due to simple reasons, such as the server being down, the application is not running on the server, or the network is down somewhere on the way to the server.
  • Problems can be also due to more complicated reasons, such as DNS problems, insufficient memory on the server that does not enable you to connect (due to high memory consumption by an application, for example), duplicate IPs, and many other.

In this recipe we focus on these GO/NO-GO problems that are usually quite easy to solve.

How to do it...

Here you will see some indicators and what you can see when you use Wireshark for debugging TCP connectivity problems. Usually these problems results in trying to run an application and getting no results.

When you try to run an application, for example, a database client, a mail client, watching cameras servers, and so on, and you don't get any output, follow these steps:

  1. Verify that the server and applications are running.
  2. Verify if your client is running, you have an IP address configured (manually or by DHCP), and you are connected to the network.
  3. Ping the server and verify you have connectivity to it.

    In some cases, you will not have Ping to the server, but still have connectivity to the application. This can happen because a firewall is blocking the ICMP messages, so if you don't have Ping to a destination it doesn't necessarily mean that something is wrong. The firewall can be a dedicated device in the network or a windows (or Linux/UNIX) firewall installed on the end device.

  4. In the capture file, look for one of the following patterns:
    • Triple SYN messages with no response (in the following screenshot)
    • SYN messages with a reset (RST) response

In both cases it can be that a firewall is blocking the specific application or the application is not running.

In the following screenshot, we see a simple case in which we simply don't get access to web server 81.218.31.171 (packets 61, 62, and 63). It can be because it is not permitted by a firewall, or simply because there is a problem with the server. We can also see that we have a connection to another website (108.160.163.43, packets 65, 66, and 67), so the connection problem is only to 81.218.31.171.

In the next illustration we see a slightly more complex case of the same situation. In this case, we've had a cameras server that the customer wanted to log in to and watch the cameras on a remote site. The camera's server had the IP address 135.82.12.1 and the problem was that the customer was able to get the main web page of the server with the login window, but couldn't log into the system. In the following screenshot, we can see that we open a connection to the IP address 135.82.12.1. We see that a TCP connection is opened to the HTTP server, and at first it looks like there are no connectivity problems:

The problems arise when we filter all traffic to the IP address 135.82.12.1, that is, the cameras server.

Here we see that when we try to connect to TCP port 6036, we get an RST/ACK response, which can be:

  • A firewall that blocks port 6036 (that was the case here)
  • When port address translation (PAT) is configured, and we translate only port 80 and not 6036
  • The authentication of the username and password were done on TCP port 6036, the firewall allowed only port 80, the authentication was blocked, and the application didn't work

To summarize, when you don't have connectivity to a server, check the server and the client if all TCP/UDP ports are forwarded throughout the network, and if you have any ports that you don't know about.

In some cases when you install new applications in your network, it is good to connect Wireshark on the client and the server, and check what is actually running between them. The software house will not always tell you what they are actually transferring over the network (sometimes this is because they are not aware of it!), and firewalls can block information that you are not aware of.

How it works...

Starting a TCP connection, as seen in the next illustration, happens in three steps:

  1. The TCP process on the client side sends an SYN packet. This is a packet with the SYN flag set to 1. In this packet the client:
    • Specifies its initial sequence number. This is the number of the first byte that the client sends to the server.
    • Specifies its window size. This is the buffer the clients allocate to the process (the place in the client's RAM).
    • Sets the options that will be used by it: MSS, Selective ACK, and so on.
  2. When the server receives the request to establish a connection, the server:
    • Sends an SYN/ACK packet to the client, confirming the acceptance of the SYN request.
    • Specifies the server's initial sequence number. This is the number of the first byte that the server sends to the client.
    • Specifies the server's window size. This is the buffer size that the server allocates to the process (the place in the server's RAM).
    • Responds to the options requested and sets the options on the server side.
  3. When receiving the server's SYN/ACK, the client:
    • Sends an ACK packet to the server, confirming the acceptance of the SYN/ACK packet from the server.
    • Specifies the client's window size. This is the buffer size that the client allocates to the process. Although this parameter was defined in the first packet (the SYN packet), the server will refer to this one since it is the latest window size received by the server.

In the options field of the TCP header, we have the following main options:

  • Maximum Segment Size (MSS): This is the maximum size of the TCP datagram, that is, the number of bytes from the beginning of the TCP header to the end of the entire packet.
  • Windows Size (WSopt): This factor is multiplied with the Window Size field in the TCP header to notify the receiver on a larger size buffer. Since the maximum window size in the header is 64 KB, a factor of 4 gives us 64 KB multiplied by 4, that is, a 256 KB window size.
  • SACK: Selective ACK is an option that enables the two parties of a connection to acknowledge specific packets, so when a single packet is lost, only this packet will be sent again. Both parties of the connection have to agree on SACK in the connection establishment.
  • Timestamps options (TSopt): This parameter was explained earlier in this article, and refers to measurement of the delay between client and the server.

By this stage, both sides:

  • Agree to establish a connection
  • Know the other side's initial sequence number
  • Know the other side's window size

Anything but a full three-way handshake while establishing a connection should be considered as a problem. This includes SYN without a response, SYN and then SYN/ACK and no last ACK, SYN which is answered with a reset (RST flag equal 1), and so on.

There's more…

Some rules of thumb are as follows:

  • In case an SYN packet is answered with RST, look for the firewall that blocks the port numbers
  • Triple SYN without any answer occurs either due to an application that didn't respond, or a firewall that blocks the request on a specific port
  • Always verify if you have Network Address Translation (NAT), port forwarding, and mechanisms that play with TCP or UDP ports. These mechanisms can interrupt with the standard operation of TCP.
Network Analysis using Wireshark Cookbook Over 80 recipes to analyze and troubleshoot network problems using Wireshark with this book and ebook
Published: December 2013
eBook Price: €23.99
Book Price: €38.99
See more
Select your format and quantity:

TCP retransmission – where do they come from and why

When TCP sends a packet or a group of packets (refer to the How it works… section at the end of this recipe), it waits for acknowledgment to confirm the acceptance of these packets. Retransmissions, obviously, happen due to a packet that has not arrived, or acknowledgment that has not arrived on time. There can be various reasons for this, and finding the reason is the goal of this recipe.

Getting ready

When you see that the network becomes slow, one of the reasons for this can be retransmissions. Connect Wireshark in the port mirror to the suspicious client or server, and watch the results.

In this recipe, we will see some common problems that we encounter with Wireshark, and what they indicate.

How to do it...

Let's get started:

  1. Start capturing data on the relevant interface.
  2. Go to the Analyze | Expert Info menu.
  3. Under Notes, look for Retransmissions.
  4. You can click on the (+) sign and a list of retransmissions will open. A single mouse click on every line will bring you the retransmission in the packet capture pane.
  5. Now comes the important question: how to locate the problem.

    When you capture packets over a communication line, server interface, link to the Internet, or any other line, you can have traffic from many IP addresses, many applications, and even specific procedures on every application, for example, accessing a specific table in a database application. The important thing here is to locate the TCP connections on which the retransmissions happen.

  6. You can see where the retransmissions come from by:
    • Moving packet-by-packet in the Expert Info window, and looking for what packets does it take you in the packet capture pane (good for experienced users)
    • In the packet pane, configure the display filter expert.message == "Retransmission (suspected)", and you will get all retransmissions in the capture file
    • By applying the filter, and then checking the Limit to display filter section to the right-bottom corner of the window in the Statistics à Conversations window

Case 1 – retransmissions to many destinations

In the following screenshot, you see that we've got many retransmissions, spread between many servers, with destination ports 80 (HTTP). What we can also see from here is the 10.0.0.5 port sends the retransmission, so packets were lost on the way to the Internet, or acknowledgement was not sent back on time from the web servers.

Well, obviously something is wrong on the line to the Internet. How can we know what it is?

  1. From the Statistics menu, open IO Graph.
  2. In this case (case 1), we see that the line is nearly empty. Probably it is an error, or another loaded line on the way to the Internet.
  3. You can check packet losses and errors that cause them by logging into the communications equipment or by any SNMP browser (when the SNMP agent is configured on the equipment). Check the following screenshot for reference:

Case 2 – retransmissions on a single connection

If all retransmissions will be on a single IP, with a single TCP port number, it will be a slow application. We can see this in the following screenshot:

For retransmissions on a single connection, perform the following steps:

  1. We can also verify this by opening Conversations from the Statistics menu, and by selecting the Limit to display filter checkbox, we will get all the conversations that have retransmissions, in this case, a single conversation.
  2. By choosing the IPv4:1 tab as shown in the following screenshot we will see from which IP addresses we get the retransmissions:

  3. By choosing the TCP:6 tab as shown in the following screenshot we will see from which port numbers (or applications) we get the retransmissions:

To isolate the problem, perform the following steps:

  1. Look at the IO graph, and make sure that the line is not busy.

    An indication to a busy communication line will be a straight line very close to the maximum bandwidth of the line. For example, if you have a 10 Mbps communication line, you port mirror it, and see in the IO graph a straight line which is close to the 10 Mbps, this is a good indication to a loaded line. A non-busy communication line will have many ups and downs, peeks and empty intervals.

  2. If the line is not busy, it can be a problem on the server for the IP address 10.1.1.200 (10.90.30.12 is sending most of the retransmissions, so it can be that 10.1.1.200 responds slowly).
  3. From the packet pane we see that the application is FTP-DATA. It is possible that the FTP server works in an active mode. Hence we've opened a connection on one port (2350), and the server changed the port to 1972, so it can be a slow non-responsive FTP software (that was the problem here eventually).

Case 3 – retransmission patterns

An important thing to watch in TCP retransmissions is if the retransmissions have any pattern that you can see.

In the following screenshot, we see that all retransmissions are coming from a single connection, between a single client and NetBIOS Session Service (TCP port 139) on the server.

Looks like a simple server/application problem, but when we look at the packet capture pane, we see something interesting (refer the following screenshot):

The interesting thing is that when we look at the pattern of retransmissions, we see that they occur cyclically every 30 milliseconds. The time format here is seconds, since the previously displayed packet and the time scale is in seconds.

The problem in this case was a client that performed a financial procedure in the software that caused the software to slow down every 30-36 milliseconds.

Case 4 – retransmission due to a non-responsive application

Another reason for retransmissions can be when a client or a server does not answer to requests. In this case, you will see five retransmissions, with an increasing time difference. After these five consecutive retransmissions, the connection is considered to be lost by the sending side (in some cases, reset will be sent to close the connection, depending on the software implementation). After the disconnection, two things may happen:

  1. An SYN request will be sent by the client, in order to open a new connection. What the user will see in this case is a freeze in the application, and after 15-20 seconds it will start to work again
  2. No SYN will be sent, and the user will have to run the application (or a specific part of it) again

In the following screenshot we can see a case in which a new connection is opened:

Case 5 – retransmission due to delayed variations

TCP is a protocol that is quite tolerant to delays, as long as the delay does not vary. When you have variations in delay, you can expect retransmissions. The way to find out if this is the problem is as follows:

  1. The first thing to do is of course to ping the destination, and get the first information of the communications line delay. Look at the How it works… section to see how it should be.
  2. Check for the delay variations, which can happen due to the following reasons:
    • A delay can happen due to a non-stable or busy communication line. In this case, you will see delay variations using the Ping command. Usually it will happen on lines with a narrow bandwidth, and in some cases on cellular lines.
    • A delay can happen due to a loaded or inefficient application. In this case, you will see many retransmissions on this specific application only.
    • A delay can happen due to a loaded communication equipment (CPU load, buffer load, and so on.). You can check this by accessing the communication equipment directly.
  3. Use the Wireshark tools

The bottom line with TCP retransmissions is that retransmissions are a natural behavior of TCP as long as we don't have too many of them. Degradation in performance will start when the retransmissions are around 0.5 percent, and disconnections will start around 5 percent. It also depends on the application and its sensitivity to retransmissions.

Finding what it is

When you see retransmissions on a communication link (to the Internet, on a server, between sites, or any other link), perform for the following:

  1. Locate the problem—is it a specific IP address, specific connection, specific application, or some other problem.
  2. Check if the problem is because of the communication link, packet loss, or a slow server or PC. Check if the application is slow.
  3. If it is not due to any of the preceding reasons, check for delay variations.

How it works...

Let's see the regular operation of TCP, and what are the causes for problems that might happen.

Regular operation of the TCP Sequence/Acknowledge mechanism

One of the mechanisms that is built into TCP is the retransmission mechanism. This mechanism enables the recovery of data that is damaged, lost, duplicated, or delivered out of order.

This is achieved by assigning a sequence number to every transmitted byte, and expecting an acknowledgment (ACK) from the receiving party. If the ACK is not received within a timeout interval, the data is retransmitted.

At the receiver end, the sequence numbers are used in order to verify that the information comes in the order that it was sent. If not, rearrange it to its previous state.

This mechanism works as follows:

  1. At the connection establishment, both sides tell each other what will be their initial sequence number.
  2. When data is sent, every packet has a sequence number. The sequence number indicates the number of the first byte in the TCP payload. The next packet that is sent will have the sequence number of the previous one, plus the number of bytes in the previous packet, plus one (in the next screenshot).
  3. When a packet is sent, the RTO (Retransmission Timeout) counter starts to count the time from the moment it was sent.

    The Retransmission Timeout timer is based on the Van Jacobson congestion avoidance and control algorithm, which basically says the TCP is tolerant to high delays, but not to fast delay variations.

  4. When the receiver receives the packet, it answers with an ACK (Acknowledge) packet that tells the sender to send the next packet. In the following screenshot you will see how it works:
    1. You can see from here that 10.0.0.7 is downloading a file from 62.219.24.171. The file is downloaded via HTTP (the Wireshark window was configured to show tcp.seq and tcp.ack from the Edit | Preferences columns configuration).

    2. You can see from here that 62.219.24.171 sends packets with sequence numbers that end with 85101, 86557, after which 10.0.0.7 sends an ACK back telling him to send the packet that ends with 88009. The sender then sends it and so on.

      You can see an illustration for this.

What are TCP retransmissions and what do they cause

When a packet acknowledgment is lost, or when an ACK does not arrive on time, the sender will perform two things:

  1. Send the packet again, as described earlier in this recipe.
  2. Decrease the throughput.

    In the next screenshot we see an example of retransmissions that reduces the sender throughput (red thin lines added for clarity):

There's more...

TCP is tolerant to high delays, as long as they are reasonably stable. The algorithm that defines the TCP behavior under delay variations (among other things) is called the Van Jacobson algorithm (on the name of its inventor). The Van Jacobson algorithm enables tolerance of up to 3-4 times the average delay, so if for example, you have a delay of 100 milliseconds, TCP will be tolerant to delays of up to 300-400 milliseconds as long as they are not frequently changed.

See also

You can check the Van Jacobson algorithm at http://ee.lbl.gov/papers/congavoid.pdf.

Summary

In this article we covered how to configure TCP and UDP preferences for troubleshooting, looked at TCP connection problems, and also saw TCP retransmissions – where they come from and why.

Resources for Article:


Further resources on this subject:


About the Author :


Yoram Orzach

Yoram Orzach gained his Bachelor's degree in Science from the Technion in Haifa, Israel, and worked in Bezeq as a systems engineer in the fields of transmission and access networks from 1991 to 1995. In 1995, he joined Netplus from the Leadcom group as technical manager, and since 1999 he has worked as the CTO of NDI Communications (www.ndi-com.com), involved in the design, implementation, and troubleshooting of data communication networks worldwide. Yoram's experience is both with corporate networks, service providers, and Internet service provider's networks, and among his customers are companies such as Comverse, Motorola, Intel, Ceragon networks, Marvel, HP, and others. Yoram's experience is in design, implementation, and troubleshooting, along with training for R&D, engineering, and IT groups.

Books From Packt


Instant Cytoscape Complex Network Analysis How-to [Instant]
Instant Cytoscape Complex Network Analysis How-to [Instant]

Instant Wireshark Starter [Instant]
Instant Wireshark Starter [Instant]

GNS3 Network Simulation Guide
GNS3 Network Simulation Guide

Zenoss Core Network and System Monitoring
Zenoss Core Network and System Monitoring

Network Graph Analysis and Visualization with Gephi
Network Graph Analysis and Visualization with Gephi

BackTrack - Testing Wireless Network Security
BackTrack - Testing Wireless Network Security

Software Defined Networking with OpenFlow
Software Defined Networking with OpenFlow


Cacti 0.8 Network Monitoring

Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software