HttpWebRequest.GetResponse() taking a long time (Microsoft Security Essentials, DefaultProxy)

I wrote a simple VB.NET command-line (Console) program to grab a web page using HttpWebRequest and HttpWebResponse.

The following code snippet (req is an HttpWebRequest) would take 13 to 17 seconds to grab a simple 4k web page that would come up instantly under the IE or Chrome browser.

   1:  SWATCH = Stopwatch.StartNew()

   2:  Dim resp As HttpWebResponse = DirectCast(req.GetResponse(), HttpWebResponse)

   3:  SWATCH.Stop()

   4:  Console.WriteLine("OK - Elapsed Time: " & SWATCH.Elapsed.ToString())

   5:   

I did some fiddling around and found TWO areas that I had to make modifications.

The first modification I made was inside the settings for my anti-virus program: Microsoft Security Essentials.  After playing around, I found that one particular setting was causing 10 seconds of the 13+ second delay: the Enable behavior monitoring setting.

The next culprit was the fact that HttpWebRequest will use the default proxy set in IE.  I do not have a proxy set, but it was still causing about a 3 second delay.  The solution was to create a .config file for my program that turned this behavior off.

   1:  <?xml version="1.0" encoding="utf-8"?>

   2:  <configuration>

   3:      <system.net>

   4:          <defaultProxy>

   5:               <proxy autoDetect="false" />

   6:          </defaultProxy>

   7:      </system.net>

   8:  </configuration>

 

Another method is to disable the Automatic Proxy Detection in Internet Explorer:

#WINNING!

The offending 13+ seconds was reduced to 0.12 seconds!