Being responsible for a site with slowly loading pages is simply not good. Having twitchy managers look over your shoulder, angry phone calls from advertisers, and other such nastiness is simply no fun. Unfortunately, it can happen to even the best developer"one day your site is cruising along just fine and the next morning, it is really slow. Of course, it may not be completely your fault. Moving the site to that cheap hosting plan may have a bearing on it, or the advertising campaign that marketing just launched without telling you, driving massive traffic to the site. In the end, it doesn't matter"all eyes are on you to make the site faster, and do it quickly. Of course, if you slip in performance improvements in regular releases, there is a better chance you'll avert that sort of crisis.
The problem is that generating even a moderately complex web page involves many lines of code and many different files. Seeing that time is always short, you'll want to zero in on those lines of code and those files that have the biggest impact on performance, leaving everything else for another day. Do this in a structured, top-down way, and you'll get it done more quickly and with a greater chance of success.
In this chapter, you'll learn the following:
The overall process of assuring good performance, and how performance tuning fits into this.
How to generate Waterfall charts as a way to identify those aspects of your site that cause web page performance issues.
A number of scenarios of poor performance and how to recognize each scenario using the Waterfall charts you generated. With each scenario, you'll find a reference to the specific chapter that shows how to further pinpoint the bottleneck, and then eliminate it.
In this chapter, we'll stay at a fairly general level; in the subsequent chapters, we'll dive deeply into the technical aspects of improving your website's performance. This will include performance monitoring (Chapter 2, Reducing Time to First Byte), IIS thread usage (Chapter 6, Thread Usage), database access (Chapter 8, Speeding up Database Access), and on-the-fly JavaScript compression and on-demand loading (Chapter 13,
To see how to load test your code without touching your production environment, refer to Chapter 14,
Before you start diagnosing and fixing performance issues, let's first have a quick look at how this fits in to the overall process of assuring good performance. This process consists of these parts:
Continuous monitoring
Setting performance goals
Iterative improvements
Let's go through each part of the process one-by-one.
The performance of your website is affected by both the things you control, such as code changes, and the things you cannot control such as increases in the number of visitors or server problems. Because of this, it makes sense to monitor the performance of your site continuously. That way, you find out that the site is becoming too slow before your manager does.
At the end of this section, you'll find a list of monitoring services that will keep an eye on your site.
What do you mean by "site is becoming too slow"? Glad you asked.
When this books talks about performance, it ultimately talks about page load time that's what your visitors and potential customers experience.
It pays to have a clear understanding of how fast is fast enough. Spending time and effort on making your site faster is time and effort not spent on other improvements. There is little point in making your site faster if it is already fast enough.
Your manager, client, or marketing department may have ideas about what constitutes "fast enough". Alternatively, visit the page Website Response Times at http://www.useit.com/alertbox/response-times.html to gain inspiration for setting performance goals.
Ok, you've been monitoring your site, and found it is getting too slow. What now?
Because websites, web servers, and browsers are such complex beasts, it pays to make improvements in a deliberate fashion. Hence, these two golden rules:
One change at a time: Because multiple changes can affect each other in surprising ways, life will be much easier if you implement and evaluate changes one-by-one.
Test, test, test: You don't know whether your change really is an improvement until you have tested it thoroughly. Make sure that performance after the change is better than performance before the change, and be ready to roll back your changes if it isn't; what works for other sites may not work for your site.
This process makes your site run faster. An iterative process is shown as following:
1. Record the current performance.
2. Diagnose the performance problem identify the single most important bottleneck.
3. Fix that one bottleneck (remember, one change at a time).
4. Test your change. Has performance improved? If it didn't, roll back your change and come up with a better fix. If it did, go back to step 1.
As a result of this, making your site faster is an iterative process:
If you find bottlenecks that currently do not pose a problem, but will come back to bite you in the future, it may be good to fix them, now that they are clear in your head.
We can capture all this in a flowchart:
 |
If you search Google for "site monitoring service", you'll find lots of companies that will monitor your website's performance for you. They also alert you when the response time gets too high, indicating your site is down.
Here are a few of the monitoring services that offer a free plan at the time of this writing. Note that the descriptions below relate to the free plan. If you are prepared to spend a bit of money, check their websites for their offerings:
Pingdom:
Checking frequency: Once every 1, 5, 15, 30, or 60 minutes
Checks from: United States, Asia, Europe
Sends alerts via: E-mail, SMS, Twitter or iPhone
Reporting: Detailed downloadable stats showing the result of each check
Mon.itor.us:
Pages you can monitor: Unlimited (1 per contact e-mail address)
Checking frequency: Once every 30 minutes
Checks from: United States, Europe
Sends alerts via: E-mail, SMS, Twitter, or IM
Reporting: Weekly reports, real-time graph
dotcom-monitor:
Pages you can monitor: Unlimited
Checking frequency: Once per minute
Checks from: United States, Europe, Israel
Sends alerts via: E-mail, Phone, SMS, Pager, SNMP
Reporting: Daily, weekly, and monthly e-mails, extensive online reports, and graphs
247webmonitoring:
Checking frequency: Once every 15, 30, or 60 minutes
Checks from: United States
Sends alerts via: E-mail
Reporting: Real-time graph
Now that we've seen the process of assuring satisfactory performance, let's focus on how to diagnose and fix performance problems. After all, that is what this book is all about.
Before you start diagnosing and fixing performance issues, let's first have a quick look at how this fits in to the overall process of assuring good performance. This process consists of these parts:
Continuous monitoring
Setting performance goals
Iterative improvements
Let's go through each part of the process one-by-one.
The performance of your website is affected by both the things you control, such as code changes, and the things you cannot control such as increases in the number of visitors or server problems. Because of this, it makes sense to monitor the performance of your site continuously. That way, you find out that the site is becoming too slow before your manager does.
At the end of this section, you'll find a list of monitoring services that will keep an eye on your site.
What do you mean by "site is becoming too slow"? Glad you asked.
When this books talks about performance, it ultimately talks about page load time that's what your visitors and potential customers experience.
It pays to have a clear understanding of how fast is fast enough. Spending time and effort on making your site faster is time and effort not spent on other improvements. There is little point in making your site faster if it is already fast enough.
Your manager, client, or marketing department may have ideas about what constitutes "fast enough". Alternatively, visit the page Website Response Times at http://www.useit.com/alertbox/response-times.html to gain inspiration for setting performance goals.
Ok, you've been monitoring your site, and found it is getting too slow. What now?
Because websites, web servers, and browsers are such complex beasts, it pays to make improvements in a deliberate fashion. Hence, these two golden rules:
One change at a time: Because multiple changes can affect each other in surprising ways, life will be much easier if you implement and evaluate changes one-by-one.
Test, test, test: You don't know whether your change really is an improvement until you have tested it thoroughly. Make sure that performance after the change is better than performance before the change, and be ready to roll back your changes if it isn't; what works for other sites may not work for your site.
This process makes your site run faster. An iterative process is shown as following:
1. Record the current performance.
2. Diagnose the performance problem identify the single most important bottleneck.
3. Fix that one bottleneck (remember, one change at a time).
4. Test your change. Has performance improved? If it didn't, roll back your change and come up with a better fix. If it did, go back to step 1.
As a result of this, making your site faster is an iterative process:
If you find bottlenecks that currently do not pose a problem, but will come back to bite you in the future, it may be good to fix them, now that they are clear in your head.
We can capture all this in a flowchart:
|
If you search Google for "site monitoring service", you'll find lots of companies that will monitor your website's performance for you. They also alert you when the response time gets too high, indicating your site is down.
Here are a few of the monitoring services that offer a free plan at the time of this writing. Note that the descriptions below relate to the free plan. If you are prepared to spend a bit of money, check their websites for their offerings:
Pingdom:
Checking frequency: Once every 1, 5, 15, 30, or 60 minutes
Checks from: United States, Asia, Europe
Sends alerts via: E-mail, SMS, Twitter or iPhone
Reporting: Detailed downloadable stats showing the result of each check
Mon.itor.us:
Pages you can monitor: Unlimited (1 per contact e-mail address)
Checking frequency: Once every 30 minutes
Checks from: United States, Europe
Sends alerts via: E-mail, SMS, Twitter, or IM
Reporting: Weekly reports, real-time graph
dotcom-monitor:
Pages you can monitor: Unlimited
Checking frequency: Once per minute
Checks from: United States, Europe, Israel
Sends alerts via: E-mail, Phone, SMS, Pager, SNMP
Reporting: Daily, weekly, and monthly e-mails, extensive online reports, and graphs
247webmonitoring:
Checking frequency: Once every 15, 30, or 60 minutes
Checks from: United States
Sends alerts via: E-mail
Reporting: Real-time graph
Now that we've seen the process of assuring satisfactory performance, let's focus on how to diagnose and fix performance problems. After all, that is what this book is all about.
Because our goal is to make a given web page load quicker, let's enumerate the components that make up a web page:
The file with the HTML: In the ASP.NET world, this file normally has the extension
.aspx. Because of this, in the remainder of this book, I'll refer to this file as the main.aspxfile. Without this file, there is no page. Generating this file most often takes most of the server resources required to generate the overall page.Images and flash files: These files are often large and numerous.
JavaScript and CSS files: These files can block rendering of the page.
Based on this, we can categorize bottlenecks that slow down the page-loading into these broad categories:
Main
.aspxfile takes too long to generateMain
.aspxfile takes too long to transfer from server to browserImages (and flash files) take too long to transfer
JavaScript and CSS files block page rendering
In order to speed up loading of a page, we need to know in which of these categories the bottleneck falls. This is the focus of the remainder of this chapter. Once you know the broad category, you can further pinpoint the bottleneck and fix it. You'll see how in the subsequent chapters.
How do I figure out in which broad category the bottleneck falls? A simple way to do that is with a Waterfall chart.
A Waterfall chart shows the components that make up the page, in which order they get loaded by the browser, and more importantly, how much time they take to load. It looks similar to the following:
 |
If for example, the main .aspx file takes too long to load, the Waterfall chart will show a very long bar for that file.
In a little while, we'll see how to figure out the broad category of the bottleneck in a Waterfall chart. But first, let's see how to produce such a chart.
There are many free tools that will generate a Waterfall chart for you. One of those is Firebug, the popular add-on for the Firefox browser. Obviously, this tool works only for Firefox and not Internet Explorer or other browsers; so at the end of this chapter, you'll find other free tools that generate Waterfall charts for those browsers. But given the popularity of Firebug, we'll stick with that for now.
To use Firebug, you need to have the Firefox browser installed on your computer. Get it for free at http://www.mozilla.com/.
Once you have Firefox running, use it to open the Firebug home page at http://getfirebug.com/.
Here you'll find a link that installs Firebug.
Now that you have Firebug installed, create a Waterfall chart:
1. In Firefox, open the Tools menu, choose Firebug, and then choose Open Firebug or press F12. The Firebug bar appears at the bottom of the browser window.
2. In the Firebug bar, click on the Net drop-down and select Enable. Also, make sure that the All option is selected below the Firebug bar.
3. Browse to a page in Firefox. You will see the Waterfall chart of the various components on the page appear.
That chart is certainly very colorful, but what does it all mean? Actually, Firebug provides a lot of information in the Waterfall chart, including full request and response headers and timeline information. Also, it's easy to save all this information to a file.
Click the little + sign to the left of a filename in the Waterfall chart. You'll see a drop-down with the full request and response headers that were sent when the file was retrieved.
 |
But wait, there is more:
In the drop-down, click on the Response tab to see the contents of the file.
If the URL of the file had query string parameters, they are shown in a Params tab.
If this was a POST or PUT request, there is a tab with the sent data.
If it is an HTML file, the HTML tab shows the actual page created by the HTML, which is very handy for iframes.
If the response came in the form of a JSON file, you'll find a tab with the data in the form of an expandable tree. For more information about JSON, visit http://www.json.org/.
Hover the mouse over a colored bar to see detailed timing information:
 |
Here is what the numbers mean:
|
DNS Lookup |
Time taken to find the IP address of the server holding the file, based on the file's URL. |
|
Connecting |
Time spent creating a TCP connection. |
|
Queuing |
Firefox limits the number of outstanding requests. If it finds an image or some other file in the |
|
Waiting For Response |
Time spent waiting for the first byte of the file from the server. |
|
Receiving Data |
Time spent in receiving the file. |
|
'DOMContentLoaded' (event) |
Time between the start of the DNS lookup of this request and the entire DOM (Document Object Model) being ready. The DOM is the browser's internal representation of the HTML on your page. When the DOM is ready, the visitor can look around the entire page, even while some items are still loading. This number can be negative if the request started after the DOM became ready. |
|
'load' (event) |
Time between the start of the DNS lookup of this request and the page having been fully loaded, including the images. |
In addition to this timing popup, there are two more. Hover over the file size to see the precise size of the file in bytes. To see the full URL of a file, hover over its name.
At the very bottom of the Waterfall chart, you will find the total number of files making up the page, their total size and the total page-load time:
 |
To save the headers to the clipboard, right-click on a file to get a popup with a save option. You can then paste the headers to a text file.
To save the entire chart to a file, use the NetExport extension for Firebug available at http://getfirebug.com/releases/extensions.html.
Once you have this installed, an Export button appears above the Waterfall chart, next to the All button. This exports the chart to a .har file.
If you open the .har file in your favorite text editor, you'll find that the file stores the information in JSON format.
The easiest way to view your .har files as a chart is with the online viewer at http://www.softwareishard.com/har/viewer/.
This not only shows the Waterfall chart saved in the .har file, but also throws in some additional statistics:
How much of the total wait time was taken by the various components of each request (DNS lookup, receiving the page, and so on)
The proportion of the total page weight in bytes taken by the
.aspxfile, images, CSS files, and so onThe proportion of the total page weight that came from the internal browser cache rather than over the Internet
The size of the request and response headers as compared to the request and response bodies
 |
Before we saw how to create a Waterfall chart with Firebug, we discussed the four broad categories of bottlenecks that slow down page loading:
Main
.aspxfile taking too long to generateMain
.aspxfile taking too long to loadImages (and flash files) taking too long to load
JavaScript and CSS files blocking page rendering
The following sections show how each broad category expresses itself in a Waterfall chart generated by Firebug. Each section also refers you to the chapter where you'll see how to further pinpoint the bottleneck and then fix it.
In the chart below, the top line is for the main .aspx page. The purple section shows how long the browser waited until it received the first byte. It shows a situation where the browser has to wait a long time for the first byte of the .aspx file. Obviously, this delays loading of all other components, because it is the .aspx file that tells the browser which images, CSS files, and so on to load.
 |
If your web page produces a similar Waterfall chart, you will want to focus on reducing the time it takes the web server to generate the .aspx page. Refer to Chapter 2, Reducing Time to First Byte, where you'll see how to pinpoint web server issues, code issues, and so on.
In the chart below, the top line still refers to the .aspx file. In this scenario, the purple section shows that the browser didn't have to wait long for the first byte of the .aspx file. However, the grey section shows it took a long time to load the entire file over the Internet. This, in turn, delays loading of those images and other files that appear towards the end of the .aspx file.
 |
To see how to deal with this scenario, refer to Chapter 9,
This chart shows a common scenario, where the .aspx file is quick to arrive and load, but the images on the page take a long time to load. Fortunately, browsers will load a number of images in parallel rather than one-by-one, speeding up the process. However, the number of images loaded in parallel is limited by the browser to avoid overloading the connection; the exact limit depends on the browser. As a result, on pages with lots of images, some images will wait for other images to load first, thereby increasing the page load time, as shown in the following screenshot:
 |
Refer to Chapter 12, Reducing Image Load Times to see how to get your images loaded sooner in a variety of ways, including browser caching and removing unused bytes to reduce image file sizes.
This chart shows a less common scenario, where a JavaScript file blocks rendering of the rest of the page. This can have a big impact on overall page load time.
When an external JavaScript file starts loading, rendering of the rest of the page is stopped until the file has completed loading and executing. This ensures that the JavaScript coming later in the page can use JavaScript libraries that were loaded and executed earlier on. CSS files also block page rendering in order to ensure that the visitor is not exposed to the raw , unstyled version of the page.
 |
The following chart shows a different scenario where JavaScript code blocks rendering of the rest of the page, not by being slow to load but by taking a long time to execute. This could be caused by both external JavaScript, which is loaded from an external file, and by internal JavaScript which sits in the main .aspx file.
 |
Solutions for this scenario include reducing the download time of JavaScript files, loading them after the page has been rendered, and loading them on demand. Chapter 13, Improving JavaScript Loading goes into the details. It also shows how to improve loading of your CSS files, and how to ensure that the advertisements served by external advertisement networks do not block rendering of your page.
The Waterfall chart generated by Firebug as introduced earlier in this chapter obviously only works with Firefox, because Firebug is a Firefox add-on. This is not convenient if Firefox is not your favorite browser, or if you want to see how other browsers load content. Here are a few free alternatives for other browsers.
Fiddler is a proxy which logs all HTTP and HTTPS traffic between your browser and the Internet. It also allows you to change the incoming and outgoing data. Because it doesn't integrate with the browser itself as Firebug does, Fiddler works with any browser, including Internet Explorer.
Download at Fiddler from http://www.fiddler2.com/fiddler2/.
To generate a Waterfall chart with Fiddler, follow these steps:
 |
3. Start your favorite browser and load a page. It is the easiest if you have only one page open at a time, because Fiddler will record all traffic to all browser windows.
4. You will see all the files loaded as part of the page in Fiddler, in the left-hand Web Sessions window.
5. Press F12 to stop capturing any more traffic.
6. From the Edit menu, choose Select All. This way, all files will be shown in the Waterfall chart.
7. Click on the Timeline tab on the right-hand side. This will show the Waterfall chart. To see how to interpret the chart, visit http://www.fiddler2.com/fiddler/help/timeline.asp.
If the bars on the timeline look hatched rather than solid, it means that Fiddler has used buffering, which distorted the chart. Click on the Streaming button to switch off buffering and start again.
8. If you want to analyze another page, clear out the Web Sessions window first by clicking on the Remove button. Press F12 again to start capturing traffic again and visit another page in your browser.
This free, web-based service uses Internet Explorer to produce Waterfall charts. Simply visit their site at http://www.webpagetest.org/test and enter the URL of a page you're interested in. You can run tests from the United States, the UK, and New Zealand.
Because this is a web-based service, there is no software to install. On the other hand, it won't work for you if your site is behind a firewall, which probably applies to your development server.
Also, keep in mind that it runs Internet Explorer to produce its charts, so the charts you see apply to that browser even if you are visiting WebPagetest with Firefox!
Google's Chrome browser lets you generate a simple Waterfall chart right out of the box. You also get headers and other details about each file.
Download Google Chrome fromhttp://www.google.com/chrome.
To generate a Waterfall chart:
1. Open a web page in Chrome.
2. Right-click anywhere on the page and choose Inspect element. The Web Inspector window appears.
3. In the new window, click on the Resources button.
4. Choose to enable Resource Tracking. Your page reloads and the Waterfall chart appears.
To see request and response headers, parameters, and so on, click on a filename to the left of the Waterfall chart. This replaces the Waterfall chart with detailed information for that file.
 |
Apple Safari has the same underlying code base as Google Chrome, including the Web Inspector.
Download Apple Safari from http://www.apple.com/safari/.
However, before you can use the Web Inspector on Safari, it needs to be enabled first:
 |
3. Check Show Develop menu in menu bar.
With the Web Inspector enabled, you can now generate a Waterfall chart in the same way, as with Google Chrome:
1. Open a web page in Safari.
2. Right-click anywhere on the page and choose Inspect element. The Web Inspector window appears.
3. In the new window, click on the Resources button.
4. Choose to enable Resource Tracking. Your page reloads and the Waterfall chart appears.
To see request and response headers, parametes, and so on, click on a filename to the left of the Waterfall chart. This replaces the Waterfall chart with detailed information for that file.
HttpWatch: This is paid software. A free evaluation version is also available. It provides more detailed information than Fiddler. Also, it is slightly more advanced than Firebug and works with both Firefox and Internet Explorer.
IBM Page Detailer: This Waterfall chart generator is proxy-based, similar to Fiddler. It provides more details than Fiddler, but is more difficult to use. Visit http://www.alphaworks.ibm.com/tech/pagedetailer for more details.
Site-Perf.com: This is a web-based Waterfall chart generator, similar to WebPagetest. It allows you to run tests from both Europe and the United States. Visit http://site-perf.com/ for more details.
Pingdom Page Test: This is a web-based Waterfall chart generator, with some nice sorting features. Visit http://tools.pingdom.com/ for more details.
Visual Round Trip Analyzer: This is another proxy-based generator, similar to Fiddler. However, instead of files, this shows TCP packets. It is strictly hardcore only. Visit http://www.microsoft.com/downloads/details.aspx?FamilyID=119f3477-dced-41e3-a0e7-d8b5cae893a3&displaylang=en for more details.
Here are some more online resources:
Response Times: Check the 3 Important Limitshttp://www.useit.com/papers/responsetime.html
Performance Research, Part 1: Check what the 80/20 rule tells us about reducing HTTP requests http://yuiblog.com/blog/2006/11/28/performance-research-part-1/
High Performance Websites: Check the Importance of Front-End Performance http://yuiblog.com/blog/2006/11/28/performance-research-part-1/
In this chapter, we saw that performance tuning is a part of a continuous effort to assure satisfactory performance for your website. The two golden rules are, as we saw earlier in the chapter, introducing changes one-by-one, and testing every change to see if it really improves the performance of your website.
Then we saw a detailed description of the Waterfall charts generated by Firebug, an add-on for the popular Firefox browser. This was followed by a series of possible scenarios of poor performance, plus references to a specific chapter in this book for each scenario.
A list of tools to generate Waterfall charts for browsers other than Firefox was provided at the end, including tools which are independent of the browser being used.
In the next chapter, we'll see how to reduce the time it takes for the server to generate the page. You'll learn about the Windows performance monitor, and you'll see how to improve the use of system resources.