Squid Proxy Server 3.1: Beginner's Guide


Squid Proxy Server 3.1: Beginner's Guide
eBook: $26.99
Formats: PDF, PacktLib, ePub and Mobi formats
$22.94
save 15%!
Print + free eBook + free PacktLib access to the book: $71.98    Print cover: $44.99
$44.99
save 37%!
Free Shipping!
UK, US, Europe and selected countries in Asia.
Also available on:
Overview
Table of Contents
Author
Reviews
Support
Sample Chapters
  • Get the most out of your network connection by customizing Squid's access control lists and helpers
  • Set up and configure Squid to get your website working quicker and more efficiently
  • No previous knowledge of Squid or proxy servers is required
  • Part of Packt's Beginner's Guide series: lots of practical, easy-to-follow examples accompanied by screenshots

Book Details

Language : English
Paperback : 332 pages [ 235mm x 191mm ]
Release Date : February 2011
ISBN : 1849513902
ISBN 13 : 9781849513906
Author(s) : Kulbir Saini
Topics and Technologies : All Books, Networking and Servers, Beginner's Guides, Open Source, Web Development

Table of Contents

Preface
Chapter 1: Getting Started with Squid
Chapter 2: Configuring Squid
Chapter 3: Running Squid
Chapter 4: Getting Started with Squid's Powerful ACLs and Access Rules
Chapter 5: Understanding Log Files and Log Formats
Chapter 6: Managing Squid and Monitoring Traffic
Chapter 7: Protecting your Squid Proxy Server with Authentication
Chapter 8: Building a Hierarchy of Squid Caches
Chapter 9: Squid in Reverse Proxy Mode
Chapter 10: Squid in Intercept Mode
Chapter 11: Writing URL Redirectors and Rewriters
Chapter 12: Troubleshooting Squid
Pop Quiz Answers
Index
  • Chapter 1: Getting Started with Squid
    • Proxy server
    • Reverse proxy
    • Getting Squid
    • Time for action – identifying the right version
      • Methods of obtaining Squid
        • Using source archives
    • Time for action – downloading Squid
      • Obtaining the latest source code from Bazaar VCS
  • Time for action – using Bazaar to obtain source code
    • Using binary packages
  • Installing Squid
    • Installing Squid from source code
      • Compiling Squid
      • Uncompressing the source archive
      • Configure or system check
  • Time for action – running the configure command
  • Time for action – compiling the source
  • Time for action – installing Squid
  • Time for action – exploring Squid files
    • Installing Squid from binary packages
      • Fedora, CentOS or Red Hat
      • Debian or Ubuntu
      • FreeBSD
      • OpenBSD or NetBSD
      • Dragonfly BSD
      • Gentoo
      • Arch Linux
  • Summary
    • Chapter 2: Configuring Squid
      • Quick start
      • Syntax of the configuration file
        • Types of directives
      • HTTP port
      • Time for action – setting the HTTP port
      • Access control lists
      • Time for action – constructing simple ACLs
      • Controlling access to the proxy server
        • HTTP access control
      • Time for action – combining ACLs and HTTP access
        • HTTP reply access
        • ICP access
        • HTCP access
        • HTCP CLR access
        • Miss access
        • Ident lookup access
      • Cache peers or neighbors
        • Declaring cache peers
      • Time for action – adding a cache peer
        • Quickly restricting access to domains using peers
        • Advanced control on access using peers
      • Caching web documents
        • Using main memory (RAM) for caching
          • In-transit objects or current requests
          • Hot or popular objects
          • Negatively cached objects
          • Specifying cache space in RAM
      • Time for action – specifying space for memory caching
        • Maximum object size in memory
        • Memory cache mode
      • Using hard disks for caching
        • Specifying the storage space
    • Time for action – creating a cache directory
      • Configuring the number of sub directories
    • Time for action – adding a cache directory
      • Cache directory selection
      • Cache object size limits
      • Setting limits on object replacement
    • Cache replacement policies
      • Least recently used (LRU)
      • Greedy dual size frequency (GDSF)
      • Least frequently used with dynamic aging (LFUDA)
    • Tuning Squid for enhanced caching
      • Selective caching
    • Time for action – preventing the caching of local content
      • Refresh patterns for cached objects
    • Time for action – calculating the freshness of cached objects
      • Options for refresh pattern
    • Aborting the partial retrievals
    • Caching the failed requests
    • Playing around with HTTP headers
      • Controlling HTTP headers in requests
      • Controlling HTTP headers in responses
      • Replacing the contents of HTTP headers
    • DNS server configuration
      • Specifying the DNS program path
      • Controlling the number of DNS client processes
      • Setting the DNS name servers
    • Time for action – adding DNS name servers
      • Setting the hosts file
      • Default domain name for requests
      • Timeout for DNS queries
      • Caching the DNS responses
      • Setting the size of the DNS cache
    • Logging
      • Log formats
      • Log file rotation or log file backups
      • Log access
      • Buffered logs
      • Strip query terms
    • URL rewriters and redirectors
    • Other configuration directives
      • Setting the effective user for running Squid
      • Configuring hostnames for the proxy server
        • Hostname visible to everyone
        • Unique hostname for the server
      • Controlling the request forwarding
        • Always direct
        • Never direct
        • Hierarchy stoplist
      • Broken posts
      • TCP outgoing address
      • PID filename
      • Client netmask
    • Summary
      • Chapter 3: Running Squid
        • Command line options
          • Getting a list of available options
        • Time for action – listing the options
          • Getting information about our Squid installation
        • Time for action – finding out the Squid version
          • Creating cache or swap directories
        • Time for action – creating cache directories
          • Using a different configuration file
          • Getting verbose output
        • Time for action – debugging output in the console
          • Full debugging output on the terminal
          • Running as a normal process
          • Parsing the Squid configuration file for errors or warnings
        • Time for action – testing our configuration file
          • Sending various signals to a running Squid process
            • Reloading a new configuration file in a running process
            • Shutting down the Squid process
            • Interrupting or killing a running Squid process
            • Checking the status of a running Squid process
            • Sending a running process in to debug mode
            • Rotating the log files
          • Forcing the storage metadata to rebuild
          • Double checking swap during rebuild
        • Automatically starting Squid at system startup
          • Adding Squid command to /etc/rc.local file
          • Adding init script
        • Time for action – adding the init script
        • Summary
        • Chapter 4: Getting Started with Squid's Powerful ACLs and Access Rules
          • Access control lists
            • Fast and slow ACL types
            • Source and destination IP address
          • Time for action – constructing ACL lists using IP addresses
          • Time for action – using a range of IP addresses to build ACL lists
            • Source and destination domain names
          • Time for action – constructing ACL lists using domain names
            • Destination port
          • Time for action – building ACL lists using destination ports
            • HTTP methods
            • Identifying requests using the request protocol
          • Time for action – using a request protocol to construct access rules
            • Time-based ACLs
            • URL and URL path-based identification
            • Matching client usernames
            • Proxy authentication
          • Time for action – enforcing proxy authentication
            • User limits
            • Identification based on various HTTP headers
            • HTTP reply status
            • Identifying random requests
          • Access list rules
            • Access to HTTP protocol
            • Access to other ports
            • Enforcing limited access to neighbors
          • Time for action – denying miss_access to neighbors
            • Requesting neighbor proxy servers
            • Forwarding requests to remote servers
            • Ident lookup access
            • Controlled caching of web documents
            • URL rewrite access
            • HTTP header access
            • Custom error pages
            • Maximum size of the reply body
            • Logging requests selectively
          • Mixing ACL lists and rules – example scenarios
            • Handling caching of local content
          • Time for action – avoiding caching of local content
            • Denying access from external networks
            • Denying access to selective clients
            • Blocking the download of video content
          • Time for action – blocking video content
            • Special access for certain clients
          • Time for action – writing rules for special access
            • Limited access during working hours
            • Allowing some clients to connect to special ports
          • Testing access control with squidclient
          • Time for action – testing our access control example with squidclient
          • Time for action – testing a complex access control
          • Summary
          • Chapter 5: Understanding Log Files and Log Formats
            • Log messages
            • Cache log or debug log
            • Time for action – understanding the cache log
            • Access log
              • Understanding the access log
            • Time for action – understanding the access log messages
              • Access log syntax
            • Time for action – analyzing a syntax to specify access log
              • Log format
            • Time for action – learning log format and format codes
              • Log formats provided by Squid
          • Time for action – customizing the access log with a new log format
          • Selective logging of requests
          • Time for action – using access_log to control logging of requests
          • Referer log
          • Time for action – enabling the referer log
          • Time for action – translating the referer logs to a human-readable format
          • User agent log
          • Time for action – enabling user agent logging
          • Emulating HTTP server-like logs
          • Time for action – enabling HTTP server log emulation
          • Log file rotation
          • Other log related features
            • Cache store log
          • Summary
            • Chapter 6: Managing Squid and Monitoring Traffic
              • Cache manager
                • Installing the Apache Web server
              • Time for action – installing Apache Web server
                • Configuring Apache for providing the cache manager web interface
              • Time for action – configuring Apache to use cachemgr.cgi
                • Accessing the cache manager web interface
                  • Configuring Squid
                  • Log in to cache manger
                  • General Runtime Information
                  • IP Cache Stats and Contents
                  • FQDN Cache Statistics
                  • HTTP Header Statistics
                  • Traffic and Resource Counters
                  • Request Forwarding Statistics
                  • Cache Client List
                  • Memory Utilization
                  • Internal DNS Statistics
              • Log file analyzers
                • Calamaris
                  • Installing Calamaris
              • Time for action – installing Calamaris
                • Using Calamaris to generate statistics
            • Time for action – generating stats in plain text format
            • Time for action – generating graphical reports with Calamaris
            • Summary
              • Chapter 7: Protecting your Squid Proxy Server with Authentication
                • HTTP authentication
                • Basic authentication
                • Time for action – exploring Basic authentication
                  • Database authentication
                    • Configuring database authentication
                  • NCSA authentication
                • Time for action – configuring NCSA authentication
                  • NIS authentication
                  • LDAP authentication
                  • SMB authentication
                  • PAM authentication
                • Time for action – configuring PAM service
                  • MSNT authentication
                • Time for action – configuring MSNT authentication
                  • MSNT multi domain authentication
                  • SASL authentication
                • Time for action – configuring Squid to use SASL authentication
                  • getpwnam authentication
                  • POP3 authentication
                  • RADIUS authentication
                • Time for action – configuring RADIUS authentication
                  • Fake Basic authentication
                • Digest authentication
                • Time for action – configuring Digest authentication
                  • File authentication
                  • LDAP authentication
                  • eDirectory authentication
                • Microsoft NTLM authentication
                  • Samba's NTLM authentication
                  • Fake NTLM authentication
                • Negotiate authentication
                • Time for action – configuring Negotiate authentication
                • Using multiple authentication schemes
                • Writing a custom authentication helper
                • Time for action – writing a helper program
                • Making non-concurrent helpers concurrent
                • Common issues with authentication
                • Summary
                • Chapter 8: Building a Hierarchy of Squid Caches
                  • Cache hierarchies
                  • Reasons to use hierarchical caching
                  • Problems with hierarchical caching
                  • Joining a cache hierarchy
                  • Time for action – joining a cache hierarchy
                    • ICP options
                    • HTCP options
                    • Peer or neighbor selection
                      • Options for peer selection methods
                    • Other cache peer options
                  • Controlling communication with peers
                    • Domain-based forwarding
                  • Time for action – configuring Squid for domain-based forwarding
                    • Cache peer access
                  • Time for action – forwarding requests to cache peers using ACLs
                    • Switching peer relationship
                  • Time for action – configuring Squid to switch peer relationship
                    • Controlling request redirects
                  • Peer communication protocols
                    • Internet Cache Protocol
                    • Cache digests
                      • Squid and cache digest configuration
                    • Hypertext Caching Protocol
                  • Summary
                  • Chapter 9: Squid in Reverse Proxy Mode
                    • What is reverse proxy mode?
                      • Exploring reverse proxy mode
                    • Configuring Squid as a server surrogate
                    • HTTP port
                      • HTTP options in reverse proxy mode
                    • HTTPS port
                      • HTTPS options in reverse proxy mode
                      • Adding backend web servers
                        • Cache peer options for reverse proxy mode
                    • Time for action – adding backend web servers
                      • Support for surrogate protocol
                        • Understanding the surrogate protocol
                        • Configuration options for surrogate support
                      • Support for ESI protocol
                        • Configuring Squid for ESI support
                    • Logging messages in web server log format
                      • Ignoring the browser reloads
                    • Time for action – configuring Squid to ignore the browser reloads
                    • Access controls in reverse proxy mode
                      • Squid in only reverse proxy mode
                      • Squid in reverse proxy and forward proxy mode
                      • Example configurations
                      • Web server and Squid server on the same machine
                      • Accelerating multiple backend web servers hosting one website
                      • Accelerating multiple web servers hosting multiple websites
                    • Summary
                    • Chapter 10: Squid in Intercept Mode
                      • Interception caching
                      • Time for action – understanding interception caching
                      • Advantages of interception caching
                      • Problems with interception caching
                      • Diverting HTTP traffic to Squid
                        • Using a router's policy routing to divert requests
                        • Using rule-based switching to divert requests
                        • Using Squid server as a bridge
                        • Using WCCP tunnel
                        • Implementing interception caching
                        • Configuring the network devices
                        • Configuring the operating system
                      • Time for action – enabling IP forwarding
                      • Time for action – redirecting HTTP traffic to Squid
                        • Configuring Squid
                        • Configuring HTTP port
                      • Summary
                      • Chapter 11: Writing URL Redirectors and Rewriters
                        • URL redirectors and rewriters
                          • Understanding URL redirectors
                            • HTTP status codes for redirection
                          • Understanding URL rewriters
                          • Issues with URL rewriters
                        • Squid, URL redirectors, and rewriters
                          • Communication interface
                        • Time for action – exploring the message flow between Squid and redirectors
                        • Time for action – writing a simple URL redirector program
                          • Concurrency
                          • Handling whitespace in URLs
                            • Using the uri_whitespace directive
                            • Making redirector programs intelligent
                        • Writing our own URL redirector program
                        • Time for action – writing our own template for a URL redirector
                        • Configuring Squid
                          • Specifying the URL redirector program
                          • Controlling redirector children
                          • Controlling requests passed to the redirector program
                          • Bypassing URL redirector programs when under heavy load
                          • Rewriting the Host HTTP header
                        • A special URL redirector – deny_info
                        • Popular URL redirectors
                          • SquidGuard
                          • Squirm
                          • Ad Zapper
                        • Summary
                        • Chapter 12: Troubleshooting Squid
                          • Some common issues
                            • Cannot write to log files
                          • Time for action – changing the ownership of log files
                            • Could not determine hostname
                            • Cannot create swap directories
                          • Time for action – fixing cache directory permissions
                            • Failed verification of swap directories
                          • Time for action – creating swap directories
                            • Address already in use
                          • Time for action – finding the program listening on a specific port
                            • URLs with underscore results in an invalid URL
                              • Enforce hostname checks
                              • Allow underscore
                            • Squid becomes slow over time
                            • The request or reply is too large
                            • Access denied on the proxy server
                            • Connection refused when reaching a sibling proxy server
                          • Debugging problems
                          • Time for action – debugging HTTP requests
                          • Time for action – debugging access control
                            • Getting help online and reporting bugs
                          • Summary

                          Kulbir Saini

                          Kulbir Saini is an entrepreneur based in Hyderabad, India. He has had extensive experience in managing systems and network infrastructure. Apart from his work as a freelance developer, he provides services to a number of startups. Through his blogs, he has been an active contributor of documentation for various open source projects, most notable being The Fedora Project and Squid. Besides computers, which his life practically revolves around, he loves travelling to remote places with his friends. For more details, please check http://saini.co.in/.

                          Code Downloads

                          Download the code and support files for this book.


                          Submit Errata

                          Please let us know if you have found any errors not listed on this list by completing our errata submission form. Our editors will check them and add them to this list. Thank you.


                          Errata

                          - 2 submitted: last submission 12 Nov 2012

                          Errata type: Typo | Page number: 15

                          "Use the following the command to see the available options along with a brief description." should be "Use the following command to see the available options along with a brief description."

                           

                          Errata type: Typo | Page number: 72

                          "2. Consider the following lines from the Squid configuration file:
                          acl exapmile_sites dstdomain .example.com .example.net"

                          should be

                          "2. Consider the following lines from the Squid configuration file:
                          acl example_sites dstdomain .example.com .example.net"

                           

                          Sample chapters

                          You can view our sample chapters and prefaces of this title on PacktLib or download sample chapters in PDF format.

                          Frequently bought together

                          Squid Proxy Server 3.1: Beginner's Guide +    Haskell Data Analysis Cookbook =
                          50% Off
                          the second eBook
                          Price for both: €31.75

                          Buy both these recommended eBooks together and get 50% off the cheapest eBook.

                          What you will learn from this book

                          • Discover which configuration option would best suit your network
                          • Gain better control over Squid with command-line options that help you to debug Squid
                          • Devise an Access Control List (ACL) to decide which users are granted access to different ports
                          • Understand logfiles and log format and how to customize them to suit your needs
                          • Learn about Squid's Cache Manager web interface so that you can monitor your traffic in real time to prevent any problems before they happen
                          • Implement a cache hierarchy to use in a large network
                          • Use Squid in Accelerator Mode to quickly boost the performance of a very slow website
                          • Write your own URL rewriters to customize the behavior of Squid
                          • Learn how to troubleshoot Squid

                          In Detail

                          Squid Proxy Server enables you to cache your web content and return it quickly on subsequent requests. System administrators often struggle with delays and too much bandwidth being used, but Squid solves these problems by handling requests locally. By deploying Squid in accelerator mode, requests are handled faster than on normal web servers making your site perform quicker than everyone else's!

                          Squid Proxy Server 3.1 Beginner's Guide will help you to install and configure Squid so that it is optimized to enhance the performance of your network. The Squid Proxy Server reduces the amount of effort that you will have to put in, saving your time to get the most out of your network. Whether you only run one site, or are in charge of a whole network, Squid is an invaluable tool that improves performance immeasurably. Caching and performance optimization usually requires a lot of work on the developer's part, but Squid does all that for you. This book will show you how to get the most out of Squid by customizing it for your network. You will learn about the different configuration options available and the transparent and accelerated modes that enable you to focus on particular areas of your network.

                          Applying proxy servers to large networks can be a lot of work as you have to decide where to place restrictions and who should have access, but the straightforward examples in this book will guide you through step by step so that you will have a proxy server that covers all areas of your network by the time you finish the book.

                          A practical guide to implementing the Squid Proxy Server in your network or for your website

                          Approach

                          Part of Packt's Beginner's Guide Series, this book has lots of screenshots and step-by-step instructions to help you get to grips with the techniques as quickly as possible. Each chapter is dedicated to a different aspect of the Squid proxy server, so you will have a thorough understanding of how everything works and how it is connected by the end of the book.

                          Who this book is for

                          If you are a Linux or Unix system administrator and you want to enhance the performance of your network or you are a web developer and want to enhance the performance of your website, this book is for you. You are expected to have some basic knowledge of networking concepts, but may not have used caching systems or proxy servers before now.

                          Code Download and Errata
                          Packt Anytime, Anywhere
                          Register Books
                          Print Upgrades
                          eBook Downloads
                          Video Support
                          Contact Us
                          Awards Voting Nominations Previous Winners
                          Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
                          Resources
                          Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software