Just like with our TCP client earlier in the chapter, we can perform some basic validation of our code with netcat. We’ll again open a listening socket with nc -v -l 127.0.0.1 8081, then connect to it with our client like this:
| | $ ./target/scala-2.11/httpclient-out 127.0.0.1 8081 / |
| | looking up address: 127.0.0.1 port: 8081 |
| | about to perform lookup |
| | lookup returned 0 |
| | got addrinfo: flags 2, family 0, socktype 1, protocol 6 |
| | creating socket |
| | socket returned fd 3 |
| | connecting |
| | connect returned 0 |
| | wrote request |
| | reading status line? |
It will then hang to await a response. And then on the server we’ll see this (as long as we made sure to start the server first):
| | $ nc -v -l 127.0.0.1 8081 |
| | GET / HTTP/1.1 |
| | Host: 127.0.0.1 |
Because netcat isn’t an HTTP server, it won’t give any kind of response on its own, and the client should just hang while awaiting an answer to its request. That said, the request itself looks to be formatted correctly. If we were to paste in a placeholder response in the netcat window, we could test out the header and response parsing a bit—but I think we’re ready to test this with a real website. Let’s try www.example.com again:
| | $ ./target/scala-2.11/httpclient-out www.example.com 80 / |
| | <html> |
| | <head> |
| | <title>Example Domain</title> |
| | ... |
| | </head> |
| | |
| | <body> |
| | <div> |
| | <h1>Example Domain</h1> |
| | <p>This domain is established to be used for illustrative examples ... |
| | domain in examples without prior coordination or asking for permission.</p> |
| | <p><a href="http://www.iana.org/domains/example">More information...</a></p> |
| | </div> |
| | </body> |
| | </html> |
Woohoo! Some celebration is in order. We now have a simple HTTP client that can make requests to many common web servers. But as we noted at the beginning of this section, HTTP is a large specification, and there are some cases we haven’t handled:
Messages using chunked transfer, in which the length of the response is included in the body.
Messages without a content length indication at all.
Message pipelining, with multiple request/responses on a single persistent connection.
None of these are especially difficult to implement, but they aren’t very interesting, either, so I’ve included them in the code for this chapter. On the other hand, if you try out the client on a few sites in the wild, you’re likely to encounter something like this, from http://pragprog.com:
| | HTTP/1.1 302 Found |
| | Content-Type: text/html; charset=utf-8 |
| | Date: Sat, 03 Feb 2018 18:44:15 GMT |
| | Location: https://pragprog.com/ |
| | Server: nginx + Phusion Passenger 5.0.27 |
| | Status: 302 Found |
| | X-Powered-By: Phusion Passenger 5.0.27 |
| | X-Request-Id: 810d84407858b4561fc170afd8588aff |
| | X-Runtime: 0.004691 |
| | X-UA-Compatible: IE=Edge,chrome=1 |
| | Content-Length: 87 |
| | Connection: keep-alive |
This is a valid HTTP response, containing a 302 Found message, which is redirecting us to https://pragprog.com. Note the protocol in the URL: like more and more of the public internet, Pragmatic’s main site has switched entirely from HTTP to HTTPS.
In some ways, HTTPS is a misnomer, because it’s not a separate application protocol at all. The content and structure of HTTPS is identical to HTTP; the only thing that is different is that a cryptographic protocol, TLS (Transport Layer Security) is inserted seamlessly on top of the TCP transport. TLS encrypts all communication between the client and the server, but it behaves just like a plain TCP socket from the point of view of application code.
As more and more of the business of daily life is conducted over the web, this sort of robust and ubiquitous encryption is essential for privacy and safety. Encryption is notoriously difficult to get right, and even experts may make errors in implementation or specification that can leak sensitive information to attackers; and for that reason I cannot recommend implementing it oneself.
Fortunately, we have many resources at our disposal if we want to support secure communication in our Scala Native programs. As you saw back in Chapter 1, The Basics: Input and Output, Scala Native has powerful facilities for invoking external C code, and there are many high-quality libraries available to us. For example, libsodium[22] provides high-quality cryptographic primitives, mbed TLS[23] offers a fully featured TLS connection library, and libcurl[24] provides a full-featured, high-performance HTTP/HTTPS client. Later in this book we’ll look at how to work with these C libraries in our Scala Native code. But first, we’re going to take a deep dive into Scala Native’s memory management and allocation capabilities so that we have the full complement of techniques necessary for working with idiomatic C code.