Log In
Or create an account ->
Imperial Library
Home
About
News
Upload
Forum
Help
Login/SignUp
Index
Squid: The Definitive Guide
SPECIAL OFFER: Upgrade this ebook with O’Reilly
Preface
About This Book
Topics Not Covered
Recommended Reading
Conventions Used in This Book
Comments and Questions
Acknowledgments
1. Introduction
1.1. Web Caching
1.2. A Brief History of Squid
1.3. Hardware and Operating System Requirements
1.4. Squid Is Open Source
1.5. Squid's Home on the Web
1.6. Getting Help
1.6.1. Frequently Asked Questions
1.6.2. Mailing Lists
1.6.2.1. squid-users
1.6.2.2. squid-announce
1.6.2.3. squid-dev
1.6.3. Professional Support
1.7. Getting Started with Squid
1.8. Exercises
2. Getting Squid
2.1. Versions and Releases
2.2. Use the Source, Luke
2.3. Precompiled Binaries
2.4. Anonymous CVS
2.5. devel.squid-cache.org
2.6. Exercises
3. Compiling and Installing
3.1. Before You Start
3.2. Unpacking the Source
3.3. Pretuning Your Kernel
3.3.1. File Descriptors
3.3.1.1. FreeBSD, NetBSD, OpenBSD
3.3.1.2. Linux
3.3.1.3. Solaris
3.3.2. Mbuf Clusters
3.3.3. Ephemeral Port Range
3.4. The configure Script
3.4.1. configure Options
3.4.2. Running configure
3.5. make
3.6. make Install
3.7. Applying a Patch
3.8. Running configure Later
3.9. Exercises
4. Configuration Guide for the Eager
4.1. The squid.conf Syntax
4.2. User IDs
4.3. Port Numbers
4.4. Log File Pathnames
4.5. Access Controls
4.6. Visible Hostname
4.7. Administrative Contact Information
4.8. Next Steps
4.9. Exercises
5. Running Squid
5.1. Squid Command-Line Options
5.2. Check Your Configuration File for Errors
5.3. Initializing Cache Directories
5.4. Testing Squid in a Terminal Window
5.5. Running Squid as a Daemon Process
5.5.1. The squid_start Script
5.6. Boot Scripts
5.6.1. /etc/rc.local
5.6.2. init.d and rc.d
5.6.3. /etc/inittab
5.7. A chroot Environment
5.8. Stopping Squid
5.9. Reconfiguring a Running Squid Process
5.10. Rotating the Log Files
5.11. Exercises
6. All About Access Controls
6.1. Access Control Elements
6.1.1. A Few Base ACL Types
6.1.1.1. IP addresses
6.1.1.2. Domain names
6.1.1.3. Usernames
6.1.1.4. Regular expressions
6.1.1.5. TCP port numbers
6.1.1.6. Autonomous system numbers
6.1.2. ACL Types
6.1.2.1. src
6.1.2.2. dst
6.1.2.3. myip
6.1.2.4. dstdomain
6.1.2.5. srcdomain
6.1.2.6. port
6.1.2.7. myport
6.1.2.8. method
6.1.2.9. proto
6.1.2.10. time
6.1.2.11. ident
6.1.2.12. proxy_auth
6.1.2.13. src_as
6.1.2.14. dst_as
6.1.2.15. snmp_community
6.1.2.16. maxconn
6.1.2.17. arp
6.1.2.18. srcdom_regex
6.1.2.19. dstdom_regex
6.1.2.20. url_regex
6.1.2.21. urlpath_regex
6.1.2.22. browser
6.1.2.23. req_mime_type
6.1.2.24. rep_mime_type
6.1.2.25. ident_regex
6.1.2.26. proxy_auth_regex
6.1.3. External ACLs
6.1.4. Dealing with Long ACL Lists
6.1.5. How Squid Matches Access Control Elements
6.2. Access Control Rules
6.2.1. Access Rule Syntax
6.2.2. How Squid Matches Access Rules
6.2.3. Access List Style
6.2.4. Delayed Checks
6.2.5. Slow and Fast Rule Checks
6.3. Common Scenarios
6.3.1. Allowing Local Clients Only
6.3.2. Blocking a Few Misbehaving Clients
6.3.3. Denying Pornography
6.3.4. Restricting Usage During Working Hours
6.3.5. Preventing Squid from Talking to Non-HTTP Servers
6.3.6. Giving Certain Users Special Access
6.3.7. Preventing Abuse from Siblings
6.3.8. Denying Requests with IP Addresses
6.3.9. An http_reply_access Example
6.3.10. Preventing Cache Hits for Local Sites
6.4. Testing Access Controls
6.5. Exercises
7. Disk Cache Basics
7.1. The cache_dir Directive
7.1.1. Scheme
7.1.2. Directory
7.1.3. Size
7.1.3.1. Inodes
7.1.3.2. The relationship between disk space and process size
7.1.4. L1 and L2
7.1.5. Options
7.1.5.1. read-only
7.1.5.2. max-size
7.2. Disk Space Watermarks
7.3. Object Size Limits
7.4. Allocating Objects to Cache Directories
7.5. Replacement Policies
7.6. Removing Cached Objects
7.6.1. Removing Individual Objects
7.6.2. Removing a Group of Objects
7.6.3. Removing All Objects
7.7. refresh_pattern
7.8. Exercises
8. Advanced Disk Cache Topics
8.1. Do I Have a Disk I/O Bottleneck?
8.2. Filesystem Tuning Options
8.3. Alternative Filesystems
8.4. The aufs Storage Scheme
8.4.1. How aufs Works
8.4.2. aufs Issues
8.4.3. Monitoring aufs Operation
8.5. The diskd Storage Scheme
8.5.1. How diskd Works
8.5.2. Compiling and Configuring diskd
8.5.3. Monitoring diskd
8.6. The coss Storage Scheme
8.6.1. How coss Works
8.6.2. Compiling and Configuring coss
8.6.3. coss Issues
8.7. The null Storage Scheme
8.8. Which Is Best for Me?
8.9. Exercises
9. Interception Caching
9.1. How It Works
9.2. Why (Not) Intercept?
9.3. The Network Device
9.3.1. Inline Squid
9.3.2. Layer Four Switches
9.3.2.1. Alteon/Nortel
9.3.2.2. Foundry
9.3.2.3. Extreme Networks
9.3.2.4. Cisco Arrowpoint
9.3.2.5. A comment on HTTP servers and health checks
9.3.3. Cisco Policy Routing
9.3.4. Web Cache Coordination Protocol
9.3.4.1. WCCPv1
9.3.4.2. WCCPv2
9.3.4.3. Debugging
9.4. Operating System Tweaks
9.4.1. Linux
9.4.1.1. Linux and WCCP
9.4.2. FreeBSD
9.4.2.1. FreeBSD and WCCP
9.4.3. OpenBSD
9.4.3.1. OpenBSD and WCCP
9.4.4. IPFilter on NetBSD and Others
9.4.4.1. NetBSD and WCCP
9.5. Configure Squid
9.5.1. Configuring WCCPv1
9.6. Debugging Problems
9.7. Exercises
10. Talking to Other Squids
10.1. Some Terminology
10.2. Why (Not) Use a Hierarchy?
10.3. Telling Squid About Your Neighbors
10.3.1. cache_peer Options
10.3.2. Neighbor State
10.3.3. Altering the Relationship
10.4. Restricting Requests to Neighbors
10.4.1. cache_peer_access
10.4.2. cache_peer_domain
10.4.3. never_direct
10.4.4. always_direct
10.4.5. hierarchy_stoplist
10.4.6. nonhierarchical_direct
10.4.7. prefer_direct
10.5. The Network Measurement Database
10.6. Internet Cache Protocol
10.6.1. Being an ICP Server
10.6.1.1. The icp_hit_stale directive
10.6.1.2. The ICP_MISS_NOFETCH feature
10.6.1.3. The test_reachability directive
10.6.2. Being an ICP Client
10.6.2.1. cache_peer options for ICP clients
10.6.2.2. ICP and netdb
10.6.3. Multicast ICP
10.6.3.1. Multicast ICP server
10.6.3.2. Multicast ICP client
10.6.3.3. Multicast ICP example
10.7. Cache Digests
10.7.1. Configuring Squid for Cache Digests
10.8. Hypertext Caching Protocol
10.8.1. Configuring Squid for HTCP
10.9. Cache Array Routing Protocol
10.9.1. Configuring Squid for CARP
10.10. Putting It All Together
10.10.1. Step 1: Determine Direct Options
10.10.2. Step 2: Neighbor Selection Protocols
10.10.3. Step 2a: ICP/HTCP Reply Processing
10.10.4. Step 3: Secondary Parent Selection
10.10.5. Retrying
10.11. How Do I ...
10.11.1. Send All Requests Through Another Proxy?
10.11.2. Send All Requests Through Another Proxy Unless It's Down?
10.11.3. Make Sure Squid Doesn't Use Neighbors for Some Requests?
10.11.4. Send Some Requests Through a Parent to Bypass Local Filters?
10.12. Exercises
11. Redirectors
11.1. The Redirector Interface
11.1.1. Handling URIs That Contain Whitespace
11.1.2. Generating HTTP Redirect Messages
11.2. Some Sample Redirectors
11.3. The Redirector Pool
11.4. Configuring Squid
11.4.1. redirect_program
11.4.2. redirect_children
11.4.3. redirect_rewrites_host_header
11.4.4. redirector_access
11.4.5. redirector_bypass
11.5. Popular Redirectors
11.5.1. Squirm
11.5.2. Jesred
11.5.3. squidGuard
11.5.4. AdZapper
11.6. Exercises
12. Authentication Helpers
12.1. Configuring Squid
12.2. HTTP Basic Authentication
12.2.1. NCSA
12.2.2. LDAP
12.2.3. MSNT
12.2.4. Multi-domain-NTLM
12.2.5. PAM
12.2.6. SASL
12.2.7. SMB
12.2.8. YP
12.2.9. getpwnam
12.2.10. winbind
12.2.11. The Basic Auth API
12.3. HTTP Digest Authentication
12.3.1. password
12.3.2. Digest Authentication API
12.4. Microsoft NTLM Authentication
12.4.1. SMB
12.4.2. winbind
12.4.3. NTLM Authentication API
12.5. External ACLs
12.5.1. ip_user
12.5.2. ldap_group
12.5.3. unix_group
12.5.4. wbinfo_group
12.5.5. winbind_group
12.5.6. Write Your Own
12.6. Exercises
13. Log Files
13.1. cache.log
13.1.1. Debugging Levels
13.1.2. Forwarding cache.log Messages to the System Log
13.1.3. Dumping cache.log Messages to Your Terminal
13.2. access.log
13.2.1. access.log Result Codes
13.2.2. HTTP Response Status Codes
13.2.3. access.log Peering Codes
13.2.4. Configuration Directives That Affect access.log
13.2.4.1. log_icp_queries
13.2.4.2. emulate_httpd_log
13.2.4.3. log_mime_hdrs
13.2.4.4. log_fqdn
13.2.4.5. ident_lookup_access
13.2.4.6. log_ip_on_direct
13.2.4.7. client_netmask
13.2.4.8. strip_query_terms
13.2.4.9. uri_whitespace
13.2.4.10. buffered_logs
13.2.5. access.log Analysis Tools
13.3. store.log
13.3.1. Mapping File Numbers to Pathnames
13.4. referer.log
13.5. useragent.log
13.6. swap.state
13.7. Rotating the Log Files
13.8. Privacy and Security
13.9. Exercises
14. Monitoring Squid
14.1. cache.log Warnings
14.2. The Cache Manager
14.2.1. Cache Manager Pages
14.2.1.1. leaks: Memory Leak Tracking
14.2.1.2. mem: Memory Utilization
14.2.1.3. cbdata: Callback Data Registry Contents
14.2.1.4. events: Event Queue
14.2.1.5. squidaio_counts: Async IO Function Counters
14.2.1.6. diskd: DISKD Stats
14.2.1.7. config: Current Squid Configuration*
14.2.1.8. comm_incoming: comm_incoming( ) Stats
14.2.1.9. ipcache: IP Cache Stats and Contents
14.2.1.10. fqdncache: FQDN Cache Stats and Contents
14.2.1.11. idns: Internal DNS Statistics
14.2.1.12. dns: Dnsserver Statistics
14.2.1.13. redirector: URL Redirector Stats
14.2.1.14. basicauthenticator: Basic User Authenticator Stats
14.2.1.15. digestauthenticator: Digest User Authenticator Stats
14.2.1.16. ntlmauthenticator: NTLM User Authenticator Stats
14.2.1.17. external_acl: External ACL Stats
14.2.1.18. http_headers: HTTP Header Statistics
14.2.1.19. via_headers: Via Request Headers
14.2.1.20. forw_headers: X-Forwarded-For Request Headers
14.2.1.21. menu: This Cache Manager Menu
14.2.1.22. shutdown: Shut Down the Squid Process*
14.2.1.23. offline_toggle: Toggle offline_mode Setting*
14.2.1.24. info: General Runtime Information
14.2.1.25. filedescriptors: Process File Descriptor Allocation
14.2.1.26. objects: All Cache Objects
14.2.1.27. vm_objects: In-Memory and In-Transit Objects
14.2.1.28. openfd_objects: Objects with Swapout Files Open
14.2.1.29. io: Server-Side Network read( ) Size Histograms
14.2.1.30. counters: Traffic and Resource Counters
14.2.1.31. peer_select: Peer Selection Algorithms
14.2.1.32. digest_stats: Cache Digest and ICP Blob
14.2.1.33. 5min: 5 Minute Average of Counters
14.2.1.34. 60min: 60 Minute Average of Counters
14.2.1.35. utilization: Cache Utilization
14.2.1.36. histograms: Full Histogram Counts
14.2.1.37. active_requests: Client-Side Active Requests
14.2.1.38. store_digest: Store Digest
14.2.1.39. storedir: Store Directory Stats
14.2.1.40. store_check_cachable_stats: storeCheckCachable( ) Stats
14.2.1.41. store_io: Store IO Interface Stats
14.2.1.42. pconn: Persistent Connection Utilization Histograms
14.2.1.43. refresh: Refresh Algorithm Statistics
14.2.1.44. delay: Delay Pool Levels
14.2.1.45. forward: Request Forwarding Statistics
14.2.1.46. client_list: Cache Client List
14.2.1.47. netdb: Network Measurement Database
14.2.1.48. asndb: AS Number Database
14.2.1.49. carp: CARP Information
14.2.1.50. server_list: Peer Cache Statistics
14.2.1.51. non_peers: List of Unknown Sites Sending ICP messages
14.2.2. Cache Manager Access Controls
14.2.2.1. http_access
14.2.2.2. cachemgr_passwd
14.2.2.3. cachemgr.cgi
14.2.3. Reasons to Dislike the Cache Manager
14.2.4. Squid-RRD
14.3. Using SNMP
14.3.1. Using snmpwalk and snmpget
14.3.2. The Squid MIB
14.4. Exercises
15. Server Accelerator Mode
15.1. Overview
15.2. Configuring Squid
15.2.1. http_port
15.2.2. https_port
15.2.3. httpd_accel_host
15.2.4. httpd_accel_port
15.2.5. httpd_accel_uses_host_header
15.2.6. httpd_accel_single_host
15.2.7. httpd_accel_with_proxy
15.3. Gee, That Was Confusing!
15.3.1. One Box, One Server Name
15.3.2. One Box, Many Server Names
15.3.3. Many Boxes, One Server Name
15.3.4. Many Boxes, Many Server Names
15.4. Access Controls
15.5. Content Negotiation
15.6. Gotchas
15.6.1. Logging
15.6.2. Ignoring Reloads
15.6.3. Uncachable Content
15.6.4. Errors
15.6.5. Purging Objects
15.6.6. Neighbors
15.7. Exercises
16. Debugging and Troubleshooting
16.1. Some Common Problems
16.1.1. "Failed to make swap directory"
16.1.2. "Address already in use"
16.1.3. "Could not determine fully qualified hostname"
16.1.4. "DNS name lookup tests failed"
16.1.5. "Illegal character in hostname"
16.1.6. "Running out of filedescriptors"
16.1.7. "icmpRecv: Connection refused"
16.1.8. Squid Becomes Slow After Running for Some Time
16.1.9. Debugging Access Controls
16.2. Debugging via cache.log
16.3. Core Dumps, Assertions, and Stack Traces
16.3.1. Can't Find the Core File?
16.4. Replicating Problems
16.5. Reporting a Bug
16.6. Exercises
A. Config File Reference
http_port
https_port
ssl_unclean_shutdown
icp_port
htcp_port
mcast_groups
udp_incoming_address
udp_outgoing_address
cache_peer
cache_peer_domain
neighbor_type_domain
icp_query_timeout
maximum_icp_query_timeout
mcast_icp_query_timeout
dead_peer_timeout
hierarchy_stoplist
no_cache
cache_access_log
cache_log
cache_store_log
cache_swap_log
emulate_httpd_log
log_ip_on_direct
cache_dir
cache_mem
cache_swap_low
cache_swap_high
maximum_object_size
minimum_object_size
maximum_object_size_in_memory
cache_replacement_policy
memory_replacement_policy
store_dir_select_algorithm
mime_table
ipcache_size
ipcache_low
ipcache_high
fqdncache_size
log_mime_hdrs
useragent_log
referer_log
pid_filename
debug_options
log_fqdn
client_netmask
ftp_user
ftp_list_width
ftp_passive
ftp_sanitycheck
cache_dns_program
dns_children
dns_retransmit_interval
dns_timeout
dns_defnames
dns_nameservers
hosts_file
diskd_program
unlinkd_program
pinger_program
redirect_program
redirect_children
redirect_rewrites_host_header
redirector_access
redirector_bypass
auth_param
authenticate_ttl
authenticate_cache_garbage_interval
authenticate_ip_ttl
external_acl_type
wais_relay_host
wais_relay_port
request_header_max_size
request_body_max_size
refresh_pattern
quick_abort_min
quick_abort_max
quick_abort_pct
negative_ttl
positive_dns_ttl
negative_dns_ttl
range_offset_limit
connect_timeout
peer_connect_timeout
read_timeout
request_timeout
persistent_request_timeout
client_lifetime
half_closed_clients
pconn_timeout
ident_timeout
shutdown_lifetime
acl
http_access
http_reply_access
icp_access
miss_access
cache_peer_access
ident_lookup_access
tcp_outgoing_tos
tcp_outgoing_address
reply_body_max_size
cache_mgr
cache_effective_user
cache_effective_group
visible_hostname
unique_hostname
hostname_aliases
announce_period
announce_host
announce_file
announce_port
httpd_accel_host
httpd_accel_port
httpd_accel_single_host
httpd_accel_with_proxy
httpd_accel_uses_host_header
dns_testnames
logfile_rotate
append_domain
tcp_recv_bufsize
err_html_text
deny_info
memory_pools
memory_pools_limit
forwarded_for
log_icp_queries
icp_hit_stale
minimum_direct_hops
minimum_direct_rtt
cachemgr_passwd
store_avg_object_size
store_objects_per_bucket
client_db
netdb_low
netdb_high
netdb_ping_period
query_icmp
test_reachability
buffered_logs
reload_into_ims
always_direct
never_direct
header_access
header_replace
icon_directory
error_directory
maximum_single_addr_tries
snmp_port
snmp_access
snmp_incoming_address
snmp_outgoing_address
as_whois_server
wccp_router
wccp_version
wccp_incoming_address
wccp_outgoing_address
delay_pools
delay_class
delay_access
delay_parameters
delay_initial_bucket_level
incoming_icp_average
incoming_http_average
incoming_dns_average
min_icp_poll_cnt
min_dns_poll_cnt
min_http_poll_cnt
max_open_disk_fds
offline_mode
uri_whitespace
broken_posts
mcast_miss_addr
mcast_miss_ttl
mcast_miss_port
mcast_miss_encode_key
nonhierarchical_direct
prefer_direct
strip_query_terms
coredump_dir
ignore_unknown_nameservers
digest_generation
digest_bits_per_entry
digest_rebuild_period
digest_rewrite_period
digest_swapout_chunk_size
digest_rebuild_chunk_percentage
chroot
client_persistent_connections
server_persistent_connections
pipeline_prefetch
extension_methods
request_entities
high_response_time_warning
high_page_fault_warning
high_memory_warning
ie_refresh
vary_ignore_expire
sleep_after_fork
B. The Memory Cache
C. Delay Pools
C.1. Overview
C.2. Configuring Squid
C.2.1. delay_pools
C.2.2. delay_class
C.2.3. delay_parameters
C.2.4. delay_initial_bucket_level
C.2.5. delay_access
C.2.6. cache_peer no-delay Option
C.3. Examples
C.4. Issues
C.4.1. Fairness
C.4.2. Application Versus Transport Layer
C.4.3. Fixed Subnetting Scheme
C.5. Monitoring Delay Pools
D. Filesystem Performance Benchmarks
D.1. The Benchmark Environment
D.1.1. Hardware for Squid
D.1.2. Squid Version and Configuration
D.1.3. Web Polygraph Workload
D.2. General Comments
D.3. Linux
D.4. FreeBSD
D.5. OpenBSD
D.6. NetBSD
D.7. Solaris
D.8. Number of Disk Spindles
E. Squid on Windows
E.1. Cygwin
E.1.1. Installing Cygwin
E.1.2. The Squid Package
E.1.3. Compiling Squid
E.1.4. Configuring and Running
E.2. SquidNT
F. Configuring Squid Clients
F.1. Manually
F.1.1. Netscape/Mozilla
F.1.2. Explorer
F.1.3. Konqueror
F.1.4. Opera
F.1.5. Lynx
F.1.6. Environment Variables
F.2. Proxy Auto-Configuration
F.3. WPAD
F.4. Summary
About the Author
Colophon
SPECIAL OFFER: Upgrade this ebook with O’Reilly
← Prev
Back
Next →
← Prev
Back
Next →