FIT – MGT5157 – Week 2 – Discussion Post
What is the reason behind packet loss? What can protocols do to limit packet loss? Are there tools available for providers and consumers that identify the source of packet loss?
What is the reason behind packet loss?
“A primary cause of packet loss is the finite size of the buffer involved” (Wu & Irwin, 2017, p. 15)
Link congestion can cause packet loss. This is where one of the devices in the packet’s path has no room in the buffer to queue the packet, so the packet has to be discarded. Increasing available bandwidth can be a resolution to link congestion, this allows buffers can empty quicker to reduce or eliminate queuing. The use of QoS to prioritize traffic like voice and video can lower the probability of a dropped packet for traffic that does not tolerate packet loss and retransmission.
Bandwidth constraints, too much data on too small of a pipe creates congestion and packet loss.
As you can see above congestion is just like this is like a four-lane road merging into one lane road. Packet loss can be an intentional thing where packets are dropped because a rule is in place to drop packets at a certain limit, hosting providers use this to control how customers use available bandwidth. Packet loss can also just occur because of unintentional congestion, where the traffic just exceeds the available bandwidth.
Device performance can also cause packet loss. This occurs in a situation where you may increase the bandwidth of the route the packet will take, but the device (router, switch, firewall, etc…) is not able to handle the load. In this case, a new device is likely required to support the network load.
For example, a Cisco ASA 5505 (Links to an external site.) is meant to handle 150 Mbps of throughput, if the device is will likely begin to have issues, maybe the CPU of the device can’t process the throughput and then device experiences congestion and begins dropping packets.
Faulty hardware, software or misconfiguration. Issues can occur here from a faulty component, like an SFP (small form-factor pluggable), a cable, a bug in the device software, or a configuration issue like a duplex mismatch, can cause packet loss.
Examples of software issues which have caused packet loss: (Links to an external site.)
- Cisco Bug: CSCub04965 – TCP Session hung causing Packet loss (Links to an external site.)
- Cisco Bug: CSCej43682 – packet loss with CEF and GRE (Links to an external site.)
- Cisco Bug: CSCtx55513 – ASA: Packet loss during phase 2 rekey (Links to an external site.)
Network attacks like a Denial of Service (DoS) attack can result in packets being dropped because the attack is overwhelming a device with traffic.
What can protocols do to limit packet loss?
TCP (Transmission Control Protocol) is a connection-oriented protocol which is built to detect packet loss and to retransmit data. The protocol itself is built to handle packet loss.
UDP (User Datagram Protocol) is a connectionless protocol that will not detect packet loss and will not retransmit. We see UDP used for streaming connect like stock ticker data, video feeds, etc… UDP is often used in conjunction with multicast where data is transmitted one-to-many or many-to-many. You can probably visualize the use cases here, and how packet loss can impact the user experience. With UDP data is lost rather than the system experiencing slow or less than optimal response times.
Layer 4 Transport Optimizations
- RIP (Routing Information Protocol) and BGP (Border Gateway Protocol) make routing (pathing) decisions based on paths, policies, and rules.
- TCP Proxy and TFO (Traffic Flow Optimization)
- DRE (Data Redundancy Elimination): A technique used to reduce traffic by removing redundant data transmission. This can be extremely useful for chatty protocols like CIFS (SMB).
Layer 2 and 3 Optimizations
- OSPF (Open Shortest Path First) and IS-IS (Intermediate System to Intermediate System) use link state routing (LSR) algorithms to determine the best route (path).
- EIGRP (Enhanced Interior Gateway Routing Protocol) is an advanced distance-vectoring routing protocol to automate network routing decisions.
- Network Segmentation and QoS (Quality of Service): Network congestion is a common cause of packet loss, network segmentation and QoS can ensure that the right traffic is given priority on the network while less critical traffic is dropped.
Are there tools available for providers and consumers that identify the source of packet loss?
There are no hard and fast rules for detecting packet loss on a network but there are tools and an approach that can be followed.
Some tools I use for diagnosis and troubleshooting:
- Ping (Links to an external site.): Send an ICMP (Internet Control Message Protocol) echo request packet from a source to a destination and wait for the reply.
- Traceroute (Links to an external site.): Display the route (path) and transmit time of packets from a source to a destination.
- MTR (Matt’s or My Traceroute) (Links to an external site.): an open-source utility that combines both traceroute and ping. Users can send ICMP traffic to a target and watch the path of the packets to see where packets are being dropped. This is a little tricking because ICMP traffic is often blocked by firewalls and it is also the lowest priority traffic and it’s common for an ICMP packet
- iPerf (Links to an external site.): iPerf is a tool to create and measure network bandwidth. iPerf is a client and server application that can test TCP, UDP and SCTP on but IPV4 and IPV6 networks.
- Network Sniffers: Many network sniffers work in a similar fashion, sniffers like Microsoft Network Monitor (Links to an external site.) and Capsa network analyzer (Links to an external site.) capture packets and allow for inspection of traffic, but undisputed leader here is Wireshark (Links to an external site.) (formerly known as ethereal).
Wireshark is a sniffer which can be used to capture traffic, isolate a TCP session and analyze the session. Once the session is isolated a filter can be applied in wireshark to view lost packets (tcp.analysis.lost_segment).
- Tcpdump and tcpreplay: tcpdump (Links to an external site.) can be used to capture packets on a network, just like Wireshared and saved them to a pcap file. tcpreplay (Links to an external site.) can then be used to replay this pcap. The use of tcpdump and tcpreplay is a great way to troubleshoot an issue with an application where the app is experiencing an issue and you want to troubleshoot and test in an automated fashion.
- Netcat (Links to an external site.): Allows for user controlled reading and writing to network connections using TCP or UDP. Can be a very valuable tools for troubleshooting.
- Host based application inspection: Tools like TCPView (Links to an external site.) which map the host based application communication very useful.
- Etc., etc., etc.
Bocchinfuso, R. (2008, January 15). Fs Cisco Event V6 Rjb. Retrieved July 13, 2018, from https://www.slideshare.net/rbocchinfuso/fs-cisco-event-v6-rjb
Hurley, M. (2015, April 28). 4 Causes of Packet Loss and How to Fix Them. Retrieved July 13, 2018, from https://www.annese.com/blog/what-causes-packet-loss
Packet Loss – What is it, How to Diagnose and Fix It in your Network. (2018, May 01). Retrieved July 13, 2018, from https://www.pcwdld.com/packet-loss
Wu, Chwan-Hwa (John). Irwin, J. David. (2013). Introduction to computer networks and cybersecurity. Hoboken: CRC Press.
FIT – MGT5157 – Week 2 – Discussion Response 1
For anyone looking to play with packet sniffing, regardless of the sniffer it is always good to capture a quality workload, be able to modify your lab environment and replay the workload to see what happens. Windump (tcpdump (Links to an external site.) for Windows) is great tools to capture traffic to a pcap file, but I would also become familiar with tcpreplay (Links to an external site.). You probably want to trade in that Windows box for Linux, my distro of choice for this sort of work is Parrot Security OS (Links to an external site.). There is one Windows tool I really like, called NetworkMiner (Links to an external site.), check it out. I would also get familiar with GNS3 (Links to an external site.) and the NETem (Links to an external site.) appliance. So many great tools out there but GNS3 is a critical tool for learning. Capturing a quality workload to a pcap, modifying your lab network with GNS3 and using tcpreplay to replay the workload while observing behavior provides a great way to experiment and see the impact. Looking ahead, GNS3 provides a way to apply the routing and subnetting theory that it looks like we’ll be diving into in week three.
FIT – MGT5157 – Week 2 – Discussion Response 2
Andrew, good post. The only comment I would make is to be careful with using ping as the method to diagnose packet loss, great place to start if the problem is really overt, but often the issues are more complex and dropped ICMP packets can be expected behavior because they typically are prioritized out by QoS.
I typically recommend the use of paping (https://code.google.com/archive/p/paping/ (Links to an external site.)) or hping3 (https://tools.kali.org/information-gathering/hping3 (Links to an external site.)) to send a TCP vs ICMP request.
If you are going to use ping I would also suggest increasing the ICMP payload size, assuming the target is not rejecting ICMP requests or dropping them because of a QoS policy.
Lastly, there are lots of hops between your computer and using MTR is a great way to see where packets are being dropped, where the latency is, etc.
FIT – MGT5157 – Week 2 – Discussion Response 3
Jonathan, a couple of comments on your post. While TCP packet loss and dropped packets have the same result, a discarded packet requiring retransmission. Packet loss has an implied context stating that the discarded packet was unintentional because of the reasons you mention above. Dropped packets can also be intentional, for example, ICMP (ping) traffic is often traffic deprioritized by QoS so these packets are intentionally dropped so they do not impact higher priority traffic.
UDP is a connectionless protocol so there is no ACK from the receiver. Packets are sent and if they are lost there is no retransmit because there is no way for the protocol to know the packet was not delivered, with UDP data can be lost or delivered out of order. There are implementations or UDP (e.g. – RUDP) where checks can be added to UDP to increase the reliability of protocol. UDP is often used in conjunction with multicast, if you think about multicast and how TCP and UDP work it becomes obvious why multicast works with a connectionless protocol like UDP and why TCP can only be used in unicast applications.