HTTPWTF. Unusual in conventional protocol

Approx. transl. : This article was written by the author of the Open Source HTTP Toolkit , designed to research and modify HTTP (S) traffic for debugging and testing purposes. The material contains the remarkable features of the HTTP standard, which have lived with us for many years, but not everyone guesses about their existence.





The HTTP protocol is critical to all modern development, from frontend to backend to mobile systems. But like any other widespread mature standard, it has a few curious skeletons tucked away in its closet.





— , , — , ( ), . , …





No-cache «» 

, - HTTP . —  no-cache



private



. , HTTP- ?





Cache-Control: private, no-cache
      
      



« », — ? --, !





: «, , ». , , .





, no-cache



, , , CDN , If-Match



If-Modified-Since



, , . private



, , , CDN -.





, , , , «» - . no-store



.





Cache-Control: no-store



, , . , — , . , max-age=0



.





, Twitter . Pragma: no-cache



( ) Cache-Control: no-store



, (DM) . , , , . .





HTTP Trailers

, HTTP- (headers). HTTP- , URL ( ) / ( ), / , (headers), (body).





, trailer' ?





, , , ( ). , , .





API- gRPC . , trailer' Server-Timing, . . Trailer' , , HTTP-.





, , , . , :





  • trailer' TE: trailers



    .





  • trailer', : Trailer: <field names>



    .





  • trailer', Content-Length



    , Cache-Control



    , Authorization



    , Host



    , , .





HTTP/1.1 chunked



. , HTTP/2 , .





trailer' HTTP/1.1 :





HTTP/1.1 200 OK
Transfer-Encoding: chunked
Trailer: My-Trailer-Field

[...chunked response body...]

My-Trailer-Field: some-extra-metadata
      
      



HTTP 1XX

, HTTP- ? 1 (200, 404 ). .





1 : 100, 101, 102, 103. , :





HTTP 100

HTTP 100 — , .





. , , . , .





, Expect: 100-continue



. , 100 , .





Expect: 100-continue



, ( «» ). URL- , (, ), HTTP 100 — . , 100, . .





HTTP 101

HTTP 101 . : « URL , ». — .





-. , :





Connection: upgrade
Upgrade: websocket
      
      



, :





HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: upgrade
      
      



HTTP raw- - .





101 HTTP/1.1 HTTP/2 . HTTP- TCP.





, HTTP/2 : - ( — - HTTP/1.1).





HTTP 102

HTTP 102 , . 100 , , .





, , - ( , , «»). HTTP, .





, , .





HTTP 103

, HTTP 103 — ( ) , push- HTTP/2 ( Chrome).





HTTP 103 — , . , Link: </style.css>; rel=preload; as=style



, ( , JS- -), .





, , . HTTP 103 , .





Referer

HTTP- Referer



, URL- . , .





referer — . , Unix referer referrer ( ). , , , , , .





, ( , , ), .





, /, , Referrer-Policy



, .





«» UUID -

XKCD to the topic.  The comment in the code reads: “Obtained by a toss of a bone.  This is guaranteed to be random. "
XKCD . : « . »

, HTTP 101 -. :





GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: upgrade
Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13
Origin: http://example.com
      
      



… , -, — :





HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: upgrade
Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=
Sec-WebSocket-Protocol: chat
      
      



 Sec-WebSocket-Accept



. websocket-, , , , . :





  • -, base64;





  • UUID 258EAFA5-E914-47DA-95CA-C5AB0DC85B11;





  • , base64 .





. UUID, -? , base64, , base64?





, , , -. , , websocket- .





. , ( ), , websocket- «» UUID.





- CORS

-: , CORS single-origin, HTTP-?





CORS , JavaScript a.com b.com, .





. , ( - ) ( cookies ).





, - CORS, , websocket- , Origin



. , , , , .





, .





, WebSocket API, Origin



/ CSRF, .





X-*

- ( 1982-) RFC , X-



— .





, HTTP-.





— HTTP-:





  • X-Shenanigans: none



    — API Twilio. , , , .





  • X-Clacks-Overhead: GNU Terry Pratchett



    ; « ».





  • X-Requested-With: XMLHttpRequest



    — JS-, jQuery, AJAX- ( ).





  • X-Recruiting: <- >



    — , , HTTP.





  • X-Powered-By: <>



    — , ( ). , .





  • X-Http-Method-Override



    — , - ( /). , .





  • X-Forwarded-For: <ip>



    — - IP- upstream-.





- , , RFC (2011) .





, . , ( X-



), «» , X-



( ).





, :





  • - , Content-Type: application/x-www-form-url-encoded



    .





  • RFC HTTP 1997 , content-encoding



    , , x-gzip



    x-compress



    gzip



    compress



    .





  • -  X-Frame-Options



    Frame-Options



    .





  • X-Content-Type-Options



    , X-DNS-Prefetch-Control



    , X-XSS-Protection



    X-Forwarded-*



    CDN/. .





, , - . , , (namespacing).






, HTTP ( ). / Twitter.





P.S.

  • «3 Linux»;





  • « swap' [ Linux]: »;





  • « ».








All Articles