Once you start using Fastvue Sophos Reporter with Site Clean, one of the first things you may notice is the bulk of your traffic typically comes from YouTube. The Site Clean functionality provides a clear picture of how much bandwidth is being used because it consolidates all the YouTube traffic instead of splitting it among youtube.com, googlevideo.com, ytimg.com, and other domains.
In the days of old, YouTube primarily contained cute cat videos. Now the site has expanded to include vast amounts of valuable content that could legitimately be required by a company. The big question is: How can a company allow YouTube traffic while limiting its ability to slow down Internet connectivity?
One approach is to simply limit the number of users who have access to streaming media, but this restriction could legitimately affect all of a company’s employees. The other approach is to impose some clever flexible limits.
Quality of Service
Sophos UTM has the capability of providing Quality Of Service (QoS) for the traffic that passes through it. Because QoS is a generic term, let’s start with what it means. Wikipedia defines QoS as:
The overall performance of a telephony or computer network, particularly the performance seen by the users of the network.
This definition implies you should get what you expect. If you’ve bought and paid for a certain level of service, you expect to have that delivered to you by the network or ISP. If you do not get that expected level, the QoS has not been met.
Network administrators need to deliver multiple services on a network, and for them, QoS means much more. Your network may be oversubscribed at certain times of the day and almost idle during others. During the idle time, there will usually be no problem maintaining the expected QoS, but during times of congestion, you need to ensure critical applications function as expected.
There are various kinds of traffic, and they have different tolerances for latency and bandwidth starvation.
An example of traffic that needs a high QoS is a video conference call or VOIP application. If insufficient QoS is available, the video will drop frames or scale down the quality of the feed. An example of traffic that can sustain some reduction in QoS is normal web browsing. Web pages still load, just perhaps a little slower.
On the other hand, YouTube traffic is somewhere in between. Leaving your video quality on automatic will allow it to fairly aggressively move between the quality modes, seeking the best available quality that can be sustained. This not only applies to scaling up and using higher stream quality but also to scaling down when required.
Without some sort of QoS control, YouTube and web browsing could squash the performance of a video conference call. This would negatively impact video streaming users, but web browsing users may not even notice. YouTube users might have to drop quality, but the videos still would play perfectly fine.
This is what is meant by QoS in the Sophos UTM context. It is the various configurations enabling us to use selective QoS control.
Step 1: Define Quality of Service (Qos) Interface Settings
For QoS to be effective over a network interface, you first need to specify the available bandwidth. Then, you need to explicitly enable QoS on the interface. Because QoS does incur some overhead, it is not advisable to turn it on for a network interface that does not require it.
As an example, the WAN link typically is significantly slower than the internal link. It would therefore be impossible to saturate the internal link with traffic from the WAN because having QoS on the WAN implicitly means you have QoS on the internal network.
These steps will allows you to configure a very basic QoS deployment that you can use to limit YouTube traffic:
- Open the Sophos Management Console and select Interfaces and Routing | Quality of Service (QoS) | Status.
- Edit your WAN (or other external) interface.
- Specify the maximum download and uplink speed (note that this is in mbits).
- Check the box for Limit uplink.
- Check the box for Download Equalizer.
- Check the box for Upload Optimizer.
There is some confusion in the non-technical audience when it comes to kb vs KB. The capital B is a byte, and a byte is 8 bits. Data size is measured in bytes, while network speeds are measured in bits/second. What this means is that a 10Mb/s Internet line can transfer 1.25 MB/s, but, practically speaking and factoring in network overhead, I always work on a 10 to 1 ratio. 10Mb speed equals 1MB.
At this stage, you will have already made an improvement to your network. After specifying the limits for the interface, the UTM will now enforce fairness and equality when the limits are reached.
Download Equalizer: If enabled, Stochastic Fairness Queuing (SFQ) and Random Early Detection (RED) queuing algorithms will avoid network congestion. If the configured downlink speed is reached, packets from the most downlink consuming stream will be dropped.Upload Optimizer: If enabled, this option will automatically prioritize outgoing TCPClosed connection establishments (TCP packets with SYN flag set), acknowledgement packets of TCP connections (TCP packets with ACK flag set and a packet length between 40 and 60 bytes), and DNSClosed lookups (UDPClosed packets on port 53).
Step 2: Define a Traffic Selector to Limit YouTube
As mentioned above, different classes of traffic have different QoS requirements. Sophos UTM uses traffic selectors to allow you to select and group traffic types. There are several ways to use traffic selectors:
- You can use a Traffic Selector, which allows you to specify a service or protocol (like Citrix ICA or RDP).
- You also can use an Application Selector, which allows you to define any of the applications specified in Applications Control (for example, Skype).
- You could also use Dynamic Application, which allows you to group applications into different categories, such as social media or streaming video.
- You could also create groups of traffic selectors to simplify your QoS rules.
This demonstration will use an Application selector with to limit YouTube bandwidth. If you don’t already have application control enabled, you should do so by completing the following steps:
- Open the Sophos Management Console and select Web Protection | Application Control | Network Visibility.
- Toggle the On/Off Switch to turn it on.
Now configure a traffic selector for YouTube:
- Open the Sophos Management Console and select Interfaces and Routing | Quality of Service (QoS) | Traffic Selectors.
- Click + New Traffic Selector.
- Name YouTube.
- Selector Type Application Selector.
- Source Any.
- Destination Any.
- Control by Applications.
- Control these applications YouTube.
- Click Save.
Step 3: Define a Bandwidth Pool
Bandwidth pool is a bit of a vague term to use—a better name is “interface bandwidth guarantees and optional limits,” but that’s a bit of a mouthful. In this step, you can use one of the traffic selectors you created and guarantee a set amount of bandwidth for it, specific to an interface. By defining a bandwidth pool, you are reserving capacity. You also have the option to set a maximum amount of bandwidth that can be used.
- Open the Sophos Management Console and select Interfaces and Routing | Quality of Service (QoS) |Bandwidth Pools.
- Select the Bound to Interface you want to use (WAN).
- Click +New Bandwidth Pools.
- Name YouTube.
- Position Top.
- Bandwidth 128kbits (Note that kbits are used, as opposed to mbits, when defining the interface).
- Check the box for Specify upper bandwidth limits 1024kbit (optional).
- Traffic Selectors select YouTube.
- Click Save.
- Turn on the pool with the toggle switch.
The limits here guarantees YouTube will always have 128kb/s available but can never exceed 1024 kb/s. If you do not specify an upper limit, the traffic will be allowed to burst out and consume all available bandwidth.
Step 4: Download throttling
The term that is often used to describe throttling by ISPs is shaping. Personally, I think it is just an attempt to make the phrase sound less brutal. Throttling gives you the ability to limit or choke traffic that can handle lower bandwidth and higher latency.
- Open the Sophos Management Console and select Interfaces and Routing | Quality of Service (QoS) | Download Throttling.
- Select the Bound to Interface you want to use (WAN).
- Click + New Download Throttling Rule.
- Name YouTube.
- Position Top.
- Limit (kbits/s) 256.
- Limit each source address.
- Traffic Selectors YouTube.
- Click Save.
- Turn on the throttle with the toggle switch.
There are a few options available to enforce the limit. Selecting shared achieves the same result as setting the upper bandwidth limit in the bandwidth pool.
In this example, we selected each source address to make sure that no single IP can consume more than 256 kbits/s for YouTube, meaning that any YouTube stream is limited to 256 kbits per second and multiple streams from the same IP would have to share the limits.
The combination of the bandwidth pool and the download throttle gives you a global limit and individual limits. I did some basic calculations to determine approximately how much bandwidth is required for each of the different YouTube quality streams. The results are as follows:
- 144P = 29kb/s
- 240P = 76kb/s
- 360p = 128kb/s
- 480p = 252/kb/s
- 720p = 285kb/s
- 1080p = 512kb/s
From this list, we can see that the network QoS setting defined above would allow video quality of up to 480P per user for up to 4 concurrent users.
Conclusion
QoS does not magically give you more bandwidth, and any benefits you see from it are achieved by stealing from Peter and giving to Paul. Trying to enforce overly complex QoS rules will not have the desired result. Having a clear, simple view of what you want to achieve is the most prudent way to proceed. All that remains now is for you to test your deployment and see whether it is working as expected.
Because we specified throttles per source IP, you might want to use a few machines in your testing. You should now be able to start YouTube videos, and they should run fairly smoothly until you start adding more than 4 streams from 4 separate sources and IPs. The YouTube player should then start stepping down the quality automatically or start experiencing buffering delays. This is because, despite how hard you might be trying, the bandwidth pool will keep the YouTube traffic pegged to 1mbit/s.
This means that even with everyone trying to view YouTube at the same time, you should be able to have a decent-quality Skype video call!
I hope this article has been useful for you, and I’d love to hear about your experience with Sophos UTM QoS in the comments!
Good article.
One correction though: The bandwidth pools is actually only to guarantee the UPLINK speed and NOT the down link speed. I found this out when I was trying give one of my applications a guarantee download speed. My ISP is 15Mbit download and 1Mbit upload. I created a bandwidth pool and put a value higher than 1Mbit and I got a warning saying the value can not be more than 90% of the external interface UPLINK speed.
I hope i’m wrong but that’s exactly the issue i’m facing. It doesn’t seem like you can get a guarantee speed for download.
Hi John
You can control the down speed but it is be way of controlling the responses goign back via the uplink. Yes it sounds confusing but here is a great article that explains a lot http://www.linksysinfo.org/index.php?threads/using-qos-tutorial-and-discussion.28349/
I ran into the same issue you did during testing. I don’t set asymmetric values on uplinks, even if they are. It rather rely on the QOS rules to try and control things.
Hope that helps
Regards
Etienne
It seems like bandwidth pools is only to guarantee UPLINK (outgoing traffic). I found this out when trying to set a bandwidth pool and set a value higher than my external interface uplink.
I come from the router world. With regards to QoS and traffic shaping/guarantees, you typically only do that in the egress direction.The only thing you typically do on routers (and apparently with the Sophos UTM in this case) in the ingress direction is set a hard rate limit. However, if you want to set up QoS in the other direction, you must think in terms of where to control it. In this case, in order to set a QoS policy to guarantee traffic from the WAN to the internal network, you need to apply an outbound QoS policy on the internal (up) interface. Don’t forget to edit your internal network interface with the downlink and uplink speeds. If you have an upload speed of 20 and download of 200 on the WAN, set the internal interface for an upload speed of 200 and download of 20.This all assumes your Sophos UTM only has two ports – one towards WAN and one towards the internal network. If you have others that could send traffic to your internal interface/network, then take that into consideration as well when picking your downlink/uplink speeds. In an enterprise environment, you wouldn’t normally do this.You would typically only do an external link policy for traffic guarantees. Rate limits and throttles are sometimes done on enterprise LANs, but it’s best to let the network switches or routers handle that if you can.
If you want to set up a traffic guarantee or hard limit from the WAN to the LAN within Sophos UTM, apply the policy to the internal (up) interface.
Thanks for your great blog posts. However I am struggling to control traffic. I’ve set up several tests using youtube and speedtest. I’ve also tried using the internal interface to bind the QOS rules against. So far I cant limit the bandwidth to either site.
I’m also noticing a lot of unclassified traffic on the external interface so I’m wondering if the utm’s appliance app classifications become dated and thus dont include newer addresses these providers might use?
Hi Glen
Thank for the feedback. Getting the QOS to work for a test can be tricky. The article was written for an implementation where Transparent proxy was the access method. If you have the option of using transparent I would suggest trying it out there first. This is the only time when application control correctly functions. The Application identification should be continually updated with Up2Date so even if there is a lag in adding additional servers it should be short term.
Hi Etienne
i have a problem quite diferent, i do not whant to limit the outgoin trafic so i dont have any pool, and i dont want to ” limit ” the incomming trafic, exactly what i need and i cant understand yet, is how to set Download Throttling for keep alway the same quality in 480p no matter how many devices were conected
the tricky part is we are in a schooll with 7 computer labs with 35 Pc each one, and 30 classrooms with the same right to have youtube, and at least 45 offices ,they are not alway all of them connected at the same time on youtube but they are so many, and even with 500 kbit/s we cant sustain the 480p, in some devices i get 144p and in some moments in some devices i can get HD quality , is turning in some thing really confussing for me
the test i made it was turn on an entire lab and play a random 480p video,,, it was a failure !!
so the question is, what i am doing wrong? or should i try some kind of diferent test ?
thanks in advance and best regards.
Sorry I thought I could leave an image with the configuration but I could not, so I leave the summary of the configuration I have
Position: 4
Limit (kbit / s): 500 (right now)
Limit: each source / destination
Traffic Selectors: googlevideo and youtube