Cover image for HTTP : the definitive guide
HTTP : the definitive guide
Gourley, David.
Personal Author:
First edition.
Publication Information:
Beijing ; Cambridge [Mass.] : O'Reilly, [2002]

Physical Description:
xviii, 635 pages : illustrations ; 24 cm
Added Author:
Format :


Call Number
Material Type
Home Location
Item Holds
QA76.76.H94 G68 2002 Adult Non-Fiction Central Closed Stacks

On Order



Behind every web transaction lies the Hypertext Transfer Protocol (HTTP) --- the language of web browsers and servers, of portals and search engines, of e-commerce and web services. Understanding HTTP is essential for practically all web-based programming, design, analysis, and administration.While the basics of HTTP are elegantly simple, the protocol's advanced features are notoriously confusing, because they knit together complex technologies and terminology from many disciplines. This book clearly explains HTTP and these interrelated core technologies, in twenty-one logically organized chapters, backed up by hundreds of detailed illustrations and examples, and convenient reference appendices. HTTP: The Definitive Guide explains everything people need to use HTTP efficiently -- including the "black arts" and "tricks of the trade" -- in a concise and readable manner.In addition to explaining the basic HTTP features, syntax and guidelines, this book clarifies related, but often misunderstood topics, such as: TCP connection management, web proxy and cache architectures, web robots and robots.txt files, Basic and Digest authentication, secure HTTP transactions, entity body processing, internationalized content, and traffic redirection.Many technical professionals will benefit from this book. Internet architects and developers who need to design and develop software, IT professionals who need to understand Internet architectural components and interactions, multimedia designers who need to publish and host multimedia, performance engineers who need to optimize web performance, technical marketing professionals who need a clear picture of core web architectures and protocols, as well as untold numbers of students and hobbyists will all benefit from the knowledge packed in this volume.There are many books that explain how to use the Web, but this is the one that explains how the Web works. Written by experts with years of design and implementation experience, this book is the definitive technical bible that describes the "why" and the "how" of HTTP and web core technologies. HTTP: The Definitive Guide is an essential reference that no technically-inclined member of the Internet community should be without.

Author Notes

is the Chief Technology Officer of Endeca, where he leads the research and development of Endeca's knowledge navigation products. Prior to working at Endeca, David designed and developed core components of Inktomi's Internet-scale search database and was a senior developer of Inktomi's web caching products. David earned a Computer Science from the University of California at Berkeley.

was a member of Inktomi Corporation's founding engineering team, and served as Inktomi's Vice President of R&D where he co-developed Inktomi's search engine database, and led the development of Inktomi's web caching and streaming media technologies. Formerly, he was a scientist at Silicon Graphics and at Apple Computer's Advanced Technology Group. Brian Totty has received several awards for research and teaching excellence, and holds a Ph.D. in Computer Science from the University of Illinois, Urbana-Champaign, and a B.S. in Computer Science from M.I.T.

Table of Contents

Prefacep. xiii
Part I. HTTP: The Web's Foundation
1. Overview of HTTPp. 3
HTTP: The Internet's Multimedia Courierp. 3
Web Clients and Serversp. 4
Resourcesp. 4
Transactionsp. 8
Messagesp. 10
Connectionsp. 11
Protocol Versionsp. 16
Architectural Components of the Webp. 17
The End of the Beginningp. 21
For More Informationp. 21
2. URLs and Resourcesp. 23
Navigating the Internet's Resourcesp. 24
URL Syntaxp. 26
URL Shortcutsp. 30
Shady Charactersp. 35
A Sea of Schemesp. 38
The Futurep. 40
For More Informationp. 41
3. HTTP Messagesp. 43
The Flow of Messagesp. 43
The Parts of a Messagep. 44
Methodsp. 53
Status Codesp. 59
Headersp. 67
For More Informationp. 73
4. Connection Managementp. 74
TCP Connectionsp. 74
TCP Performance Considerationsp. 80
HTTP Connection Handlingp. 86
Parallel Connectionsp. 88
Persistent Connectionsp. 90
Pipelined Connectionsp. 99
The Mysteries of Connection Closep. 101
For More Informationp. 104
Part II. HTTP Architecture
5. Web Serversp. 109
Web Servers Come in All Shapes and Sizesp. 109
A Minimal Perl Web Serverp. 111
What Real Web Servers Dop. 113
Step 1 Accepting Client Connectionsp. 115
Step 2 Receiving Request Messagesp. 116
Step 3 Processing Requestsp. 120
Step 4 Mapping and Accessing Resourcesp. 120
Step 5 Building Responsesp. 125
Step 6 Sending Responsesp. 127
Step 7 Loggingp. 127
For More Informationp. 127
6. Proxiesp. 129
Web Intermediariesp. 129
Why Use Proxies?p. 131
Where Do Proxies Go?p. 137
Client Proxy Settingsp. 141
Tricky Things About Proxy Requestsp. 144
Tracing Messagesp. 150
Proxy Authenticationp. 156
Proxy Interoperationp. 157
For More Informationp. 160
7. Cachingp. 161
Redundant Data Transfersp. 161
Bandwidth Bottlenecksp. 161
Flash Crowdsp. 163
Distance Delaysp. 163
Hits and Missesp. 164
Cache Topologiesp. 168
Cache Processing Stepsp. 171
Keeping Copies Freshp. 175
Controlling Cachabilityp. 182
Setting Cache Controlsp. 186
Detailed Algorithmsp. 187
Caches and Advertisingp. 194
For More Informationp. 196
8. Integration Points: Gateways, Tunnels, and Relaysp. 197
Gatewaysp. 197
Protocol Gatewaysp. 200
Resource Gatewaysp. 203
Application Interfaces and Web Servicesp. 205
Tunnelsp. 206
Relaysp. 212
For More Informationp. 213
9. Web Robotsp. 215
Crawlers and Crawlingp. 215
Robotic HTTPp. 225
Misbehaving Robotsp. 228
Excluding Robotsp. 229
Robot Etiquettep. 239
Search Enginesp. 242
For More Informationp. 246
10. HTTP-NGp. 247
HTTP's Growing Painsp. 247
HTTP-NG Activityp. 248
Modularize and Enhancep. 248
Distributed Objectsp. 249
Layer 1 Messagingp. 250
Layer 2 Remote Invocationp. 250
Layer 3 Web Applicationp. 251
WebMUXp. 251
Binary Wire Protocolp. 252
Current Statusp. 252
For More Informationp. 253
Part III. Identification, Authorization, and Security
11. Client Identification and Cookiesp. 257
The Personal Touchp. 257
HTTP Headersp. 258
Client IP Addressp. 259
User Loginp. 260
Fat URLsp. 262
Cookiesp. 263
For More Informationp. 276
12. Basic Authenticationp. 277
Authenticationp. 277
Basic Authenticationp. 281
The Security Flaws of Basic Authenticationp. 283
For More Informationp. 285
13. Digest Authenticationp. 286
The Improvements of Digest Authenticationp. 286
Digest Calculationsp. 291
Quality of Protection Enhancementsp. 299
Practical Considerationsp. 300
Security Considerationsp. 303
For More Informationp. 306
14. Secure HTTPp. 307
Making HTTP Safep. 307
Digital Cryptographyp. 309
Symmetric-Key Cryptographyp. 313
Public-Key Cryptographyp. 315
Digital Signaturesp. 317
Digital Certificatesp. 319
HTTPS: The Detailsp. 322
A Real HTTPS Clientp. 328
Tunneling Secure Traffic Through Proxiesp. 335
For More Informationp. 336
Part IV. Entities, Encodings, and Internationalization
15. Entities and Encodingsp. 341
Messages Are Crates, Entities Are Cargop. 342
Content-Length: The Entity's Sizep. 344
Entity Digestsp. 347
Media Type and Charsetp. 348
Content Encodingp. 351
Transfer Encoding and Chunked Encodingp. 354
Time-Varying Instancesp. 359
Validators and Freshnessp. 360
Range Requestsp. 363
Delta Encodingp. 365
For More Informationp. 369
16. Internationalizationp. 370
HTTP Support for International Contentp. 370
Character Sets and HTTPp. 371
Multilingual Character Encoding Primerp. 376
Language Tags and HTTPp. 384
Internationalized URIsp. 389
Other Considerationsp. 392
For More Informationp. 392
17. Content Negotiation and Transcodingp. 395
Content-Negotiation Techniquesp. 395
Client-Driven Negotiationp. 396
Server-Driven Negotiationp. 397
Transparent Negotiationp. 400
Transcodingp. 403
Next Stepsp. 405
For More Informationp. 406
Part V. Content Publishing and Distribution
18. Web Hostingp. 411
Hosting Servicesp. 411
Virtual Hostingp. 413
Making Web Sites Reliablep. 419
Making Web Sites Fastp. 422
For More Informationp. 423
19. Publishing Systemsp. 424
FrontPage Server Extensions for Publishing Supportp. 424
WebDAV and Collaborative Authoringp. 429
For More Informationp. 446
20. Redirection and Load Balancingp. 448
Why Redirect?p. 449
Where to Redirectp. 449
Overview of Redirection Protocolsp. 450
General Redirection Methodsp. 452
Proxy Redirection Methodsp. 462
Cache Redirection Methodsp. 469
Internet Cache Protocolp. 473
Cache Array Routing Protocolp. 475
Hyper Text Caching Protocolp. 478
For More Informationp. 481
21. Logging and Usage Trackingp. 483
What to Log?p. 483
Log Formatsp. 484
Hit Meteringp. 492
A Word on Privacyp. 495
For More Informationp. 495
Part VI. Appendixes
A. URI Schemesp. 499
B. HTTP Status Codesp. 505
C. HTTP Header Referencep. 508
D. MIME Typesp. 533
E. Base-64 Encodingp. 570
F. Digest Authenticationp. 574
G. Language Tagsp. 581
H. MIME Charset Registryp. 602
Indexp. 617