About admin

This author has not yet filled in any details.
So far has created 33 blog entries.

The Ins and Outs of Bi-directional Firewall Rules

When I look at firewalls rule sets maintained by other companies, I often notice the same common mistakes. The one is see most often is potentially the worst. I can speculate on a number of reasons how these rules actually get defined and implemented, but it all comes down to the same thing.  They way traffic is evaluated and processed by a firewall is not always understood correctly.  The result is a rule that looks like this.

The thinking being that the client needs a way to connect to the web server and that the web server needs a way to connect back to the client.  Let me explain why this rule is bad and some of it unnecessary.
Dissecting a Firewall Rule
The very essence of a firewall is to limit or restrict unwanted traffic, it does this by evaluating specific criteria. At its most basic, a firewall rule consists of 5 objects:

Source IP address
Source Port
Destination IP address
Destination Port

For a TCP rule such as HTTP, the following three step handshake applies:

The source or client  is the computer initiating the conversation with a SYN packet. A port is dynamically allocated on the source machine and the request is sent to the destination on the predefined static service port.
The destination or server is the computer receiving the SYN conversation request on the specified static service port. The destination machine sends back a SYN-ACK packet.
The client machine receives the SYN-ACK packet from the destination and sends back a final ACK packet.

This completes the three-way handshake. At this point you have a TCP socket or conversation pair.  During the lifespan of the socket, the port number on the source and destination will not change.  This socket is now a two way […]

By |November 14th, 2013|tmgreporter|0 Comments

How To Solve “13 The Data Is Invalid” Error in Forefront TMG

Last week we had a problem with one of our customers that was trying to access a new web site recently built and hosted at their cloud provider. When running a query in Forefront TMG Logs and Reports view, I found the error below:

Failed Connection  Attempt Log type: Web  Proxy (Forward)
Status: 13 The data is invalid.
Rule: Source: Internal
Destination: External  
Request: GET  http://
Filter information: Req ID: 0fdbab91;  Compression: client=No, server=Yes, compress rate=0% decompress  rate=0%
Protocol: http
User:  anonymous
This error occurs when the destination Web Server is using Chunked Encoding but it is not passing the correct TransferEnconding header.

When researching about this problem I found a great post from my friend Yuri Diógenes that explains why this problem happens:
According to RFC if a server is using chunked encoding it must set the Transfer-Encoding header to “chunked”. In order to compress the content we need to accumulate all the chucks and then compress. When it works, TMG knows that all that content is part of the same HTTP request since it says in the HTTP Response Header; therefore it waits for the entire content, compress and send it back to the client. On the failing server we receive the first answer that doesn’t say that the content is chucked and right after that we receive other chucks, since HTTP Compression is enabled it fails to reassemble all the content since it doesn’t know that they belong to the same content.” – Yuri Diógenes at:
If possible, the best way to solve it is by asking the web site administrator to change the “TransferEncoding header” parameter on the web server to “chunked”. See How to enable chunked transfer encoding with IIS.
If you cannot wait for the website administrator to change this, you can workaround the problem […]

By |October 7th, 2013|tmgreporter|2 Comments

Make The World A Better Place with Fastvue and Microsoft Reputation Services (MRS)

Uncle Ben told Peter Parker “With great power comes great responsibility”. With TMG Reporter and Webspy Vantage you gain total visibility of what your users are up to. This allows you to make informed decisions that can be backed up with hard data.

One great feature of Forefront TMG is URL filtering. Simply put, it knows which sites belong to which category and based on your rules, allows or denies access. Forefront TMG and other products use Microsoft Reputation Services to lookup the site category.

This is a great system as it relies on centralized / partnered and crowd sourced data to populate and keep the lists up to date. Having said that, it is not perfect and some sites do slip through the cracks especially new or small sites.

Fortunately, TMG Reporter can help you find any site that has not been categorized by Forefront TMG’s URL Filtering service. You can then use this information to manually override the site and assign it to the correct category, but you can also submit the site to Microsoft Reputation Services (MRS), so everyone can benefit from your discovery!
Real-world Example
I was going through a daily report and saw that the bulk for the top user came from a single site. By simply doing a mouse-over I could see what site it was. Looking up the site it became clear that this was a remote proxy site. This kind of site should normally be categorized as Anonimizers.  Looking up the site in Forefront TMG did not identify the site at all.

Finding Uncategorized Sites
You can easily find all sites that have not been categorized by Forefront TMG. Simply run an Overview Report, click the Filters button, and enter the filter “Category Equal […]

By |September 18th, 2013|tmgreporter|0 Comments

How To Recover Forefront TMG From a Corrupt Configuration Database

We all know it is good practice to keep regular Forefront TMG configuration backups as they help you recover your deployment quickly and accurately in case of a failure or miss configuration.  There is however a scenario where these backups cannot be restored to bail you out.  When Forefront TMG has a corrupt configuration database, the backup and restore mechanism itself is broken and as such you need to fix this first before you can recover from backup.
Symptoms of a Corrupt Configuration
In the cases of corrupt configuration that I have seen, Forefront TMG generally keeps working as per normal, but you do not have any ability to change anything. Another symptom you may notice is an empty Firewall policy screen.

In Monitoring | Configuration, you may also see an error about not having synchronized since 1999/11/30.

Under Monitoring | Alerts and in the Windows event logs (Application log), you may also see the following errors:

Level: Error
Event ID:  21209
Source: Microsoft Forefront TMG Control
Description:  The Forefront TMG configuration agent was unable to upload the configuration to the forefornt TMG services. This could be due to a corrupt configuration.  The Forefront TMG configuration agent is reverting the configuration back to the last known configuration.
Level: Error
Event ID: 14016
Source: Microsoft Forefront TMG Firewall
Description:  Forefront TMG failed to load the firewall policy configuration
Level: Error
Event ID: 21177
Source: Microsoft Forefront TMG Web Proxy
Description:  The <service> failed to reload its configuration.  If you recently applied changes to the configuration, verify that these changes are configured properly.
You may also get pop up warnings when selecting the different screens in the TMG Management Console, or if you are attempting to restore from a backup.
So what is broken?
Forefront TMG stores it configuration in an Active Directory Lightweight Directory Services (AD-LDS) database.  AD-LDS […]

By |August 16th, 2013|tmgreporter|1 Comment

Forefront TMG Configuration Backup Scripts For Standalone and Enterprise Arrays

It is good practice to keep regular backups of your Forefront TMG Configuration files. Even if you have a Forefront TMG Array with the configuration in multiple locations, this simply gives you fault tolerance, and should not be regarded as a backup.

Backups can be performed manually or automated with a script. This article explains both TMG configuration backup methods.
Forefront TMG Deployment Options
There are two types of Forefront TMG Arrays: Standalone arrays and Enterprise arrays.
Standalone Array
A Standalone Forefront TMG Array consists of two or more TMG Servers. Technically, a standalone array a can have up to 50 members but since most TMG arrays are deployed with Windows Network Load Balancer (WNLB), the deployment most likely caps out at the WNLB limit of 32 nodes.

In a Standalone Array configuration, one of the members in the array is selected as the Array Manager. This server’s configuration is the master config and it is replicated to the other members. Each server in turns keeps a local cache of the configuration.
Enterprise Arrays
When you have more than one Standalone Array, it makes sense to start using an Enterprise Array managed by an Enterprise Management Server (EMS). An Enterprise Array can contain up to 200 TMG Arrays. This scenario is typically for multi-site deployments.

In an Enterprise deployment, a dedicated server, the Enterprise Management Server (EMS) keeps the master configuration. The EMS keeps both the Enterprise wide and Array level configuration for all of the Arrays. The EMS then replicates the Enterprise and the appropriate Array configurations down to the Arrays and members who in turn all keep a local cache of the configuration.
Why are configuration backups a good idea?
Arrays and Enterprise Arrays have multiple copies of the configuration making TMG deployments fault tolerant with respect to losing the Array Master or even […]

By |August 15th, 2013|tmgreporter|0 Comments

Understanding Hyper-V CPU Usage (Physical and Virtual)

The latest version of Fastvue TMG Reporter has some enhancements when it comes to processing multiple reports.  It will now schedule multiple report jobs sequentially based on the amount of processor cores available to the server.

Scheduled reports are now processed sequentially instead of concurrently.
For every three CPU cores available beyond an initial three cores, an extra report can be run concurrently. (e.g. 12 core system can run 4 reports concurrently).
These changes are to prevent a large number of scheduled reports from maxing out system resources.

So what is the best way to plan, allocate and monitor CPU resources in your Hyper-V environment?

Understanding Hyper-V CPU Usage
When running virtual servers in Hyper-V, there is often some confusion when determining how much CPU is actually being used.  The typical first approach is to log into the Hyper-V host and open up task manager.  Here most people are surprised to see that their CPU usage is minimal, this despite the indication that the VM’s CPU’s are 100% utilized.

Truly understanding Hyper-V virtual and physical CPU usage first requires us to dive into a little bit of theory.

Virtual CPU’s can be allocated to a virtual machine.  The amount of virtual processors available are determined by the number of cores available on the hardware. So as an example, if you have a 4 socket server where each processor has 8 cores, this will present 32 logical processors. As a result you will be able to allocate a maximum of 32 virtual CPUs to a virtual server.  Each virtual CPU maps down to a physical core.
Configuring Your Virtual Machine’s Processors
To understand how this all maps together, let’s have a look at the Hyper-V VM processor configuration.  Before we start allocating CPU’s, let consider the […]

By |August 15th, 2013|tmgreporter|1 Comment

Deploying Winfrasoft Forefront TMG Virtual Appliances

a img {
max-width: 100%;
height: auto;

I am frequently asked by users of Forefront TMG, how to continue using the product now that Microsoft is no longer selling licences as of January 2013. The most common decision is to look for a third part proxy or secure web gateway solution.

However, many third party solutions do not cover all the functions of Forefront TMG, and often come with a much higher price tag. As Forefront TMG will be supported through to 2015 (and through to 2020 via extended support), swapping out your existing TMG infrastructure may not be a high priority concern.

As we reported in March, the OEM loophole is one way you can still purchase new or additional Forefront TMG licenses, and Winfrasoft is the only vendor making Forefront TMG available as a virtual appliance. 

Deploying the Winfrasoft  TMG virtual appliance can save you deployment time and costs by utilizing your existing VMWare or Hyper-V infrastructure.

Having now deployed several Winfrasoft TMG Virtual Appliances, I thought I would document the process to show you how quick and easy it is!
Requirements for Winfrasoft Forefront TMG Virtual Appliance
Winfrasoft’s TMG Virtual Appliance is a customized Windows Server installation with Forefront TMG (either Standard or Enterprise) pre-installed.

Winfrasoft offer three deployment options:

Physical appliance
Virtual Appliance for Hyper-V
Virtual Appliance for VMWare (any version)

The virtual appliances have the following minimum requirements:

CPU – 1 (Multi CPU / Core is ideal)
RAM – 4 GB  (will function with 2 GB but it’s not recommended)
Hard disk – 40 GB
Network Interfaces – 2 (for TMG Firewall and proxy) or 1 (for TMG proxy only)

It’s also recommended to use the Forefront TMG Capacity Planning tool Worksheet to plan your deployment.

Winfrasoft virtual appliances are supported on the following platforms supporting 64bit guest architecture:


Microsoft Windows Server […]

By |July 24th, 2013|tmgreporter|0 Comments

Monitoring, Alerting and Blocking Countries with Forefront TMG

Recently I posted an article on ISAserver.org on using Forefront Threat Management Gateway (TMG) 2010 to monitor and optionally block network access based on the country/geography of the source or destination IP address. There are certainly some compelling reasons to consider blocking countries, as bad actors are often associated with specific regions of the world. Depending on your organization’s business and appetite for risk, monitoring and blocking traffic from certain countries may be interesting, or it might just be essential to protecting your intellectual property.

In the original article I demonstrated how to create firewall policy to identify traffic based on geography. In addition I showed how to view this activity using the native Forefront TMG 2010 monitoring tools. The challenge to using this method, however, is that it requires the administrator to be watching the logs in real time using TMG’s live logging feature. It also suffers from the flaw that you can’t create alerts when network traffic matches a specific access rule, nor can you generate a report for this access either. Thankfully we can address these shortcomings using TMG Reporter from Fastvue.
Reporting on Country Access
Generating a report to identify traffic processed by a specific access rule is quite simple. In the TMG Reporter management console click Reports and choose Activity Report. For the Field select Rule and for the Operator choose Contains and enter any unique text that will identify your access rule for this traffic. On my TMG firewall the rule is called China .

Enter the appropriate date range for your needs and click Run Report to view any traffic that was handled by this access rule. If you’ve identified workstations that have access resources using this rule, it might be a […]

By |June 20th, 2013|tmgreporter|0 Comments

Troubleshooting Forefront TMG Web Proxy Auto Discovery (WPAD) Issues

It was almost 6:00 pm on Friday evening and I was setting my mind on what to do with my weekend when suddenly the phone rings.
“Hey Uilson, please help us! We are getting an error to access the internet!”
OK! Time to stop dreaming about weekend plans and find out what is going on!

I quickly confirmed from my notebook that internet access was down and Internet Explorer was returning the error message below:

FW-1 at fw6057: Access denied.

Requests were being redirected to our edge firewall.
Network Configuration
The network used two Forefront TMG in Network Load Balanced (NLB) configuration and all browsers received proxy details via WPAD.DAT script, delivered by GPO from our Active Directory servers.
When setting the web proxy details manually in Internet Explorer using the IP and port of the Forefront TMG proxy server,  Internet access was restored! This narrowed the problem down to an issue with the WPAD.DAT script.
Investigating WPAD
I went to Internet Explorer and tried to download the WPAD script by typing its address into my browser:

I found I could not access this link. Then, remembering some advice I received from one of our Field Analysts, I tried accessing the script via port 8080:

Success! I could download script.

I tried manually setting one of the workstations to download the script using port 8080, and it was able to access the internet again!

OK my friends! I’ve found what was wrong! The Forefront TMG Server was refusing requests to the WPAD.dat script on port 80.
The reason why Internet access suddenly dropped was that someone made a change to Forefront TMG’s Internal network properties and disabled the access via port 80 by unchecking the “Publish automatic discovery information for this network” option, as shown in the image below:

When checking this option again, all users got their […]

By |June 6th, 2013|tmgreporter|0 Comments

What everyone should know about HTTPS, SSL, TLS and Certificates

Growing concerns about Internet security spurred the development of secure encrypted protocols to deliver web content.  The Secure Sockets Layer Protocol was developed and released by Netscape in 1994, 19 years ago.  SSL and its superseding technology Transport Layer Security (TLS) is the primary method of securing web based data transfer today.

Amazingly, very few people know anything about it and even fewer people including some IT professionals know how it works. The aim of this articles is to briefly explain the concepts of how this technology works. With your new understanding you should be able to detect and avoid SSL related problems and warnings.
There are a few common terms used to refer to different aspects of the technology, but in general they are all interchangeable and refer to an encrypted data session between a client browser and a secure web server:

HTTPS – Hyper Text Transfer Protocol Secure
SSL – Secure Sockets Layer
TLS – Transport Layer Security

Other terms related to this technology include:

Keys – Text cypher
Public Key -Known text cypher
Private Key – unknown text cyper
Symmetrical or Session Key – Pair of matching keys on either side
Certificates – Text containers containing cypher and other identifying information

The Basic Steps
Below are the main steps involved in creating and maintaining an encrypted data session. The initial SSL hand shake is covered in steps 1 through 5, and the data transmission that continually reoccurs is covered in steps 6 and 7.

 –> Browser requests secure site using HTTPS header
<– Secure web server sends certificate containing its public key
–> Browser validates the certificate  with request to validation servers
–> Browser uses the public key and creates a symmetric key that will only be valid for that session and sends it to the web server
Web server decrypts the symmetric key with it’s private key
<– Web server returns data […]

By |May 7th, 2013|tmgreporter|0 Comments