§Streaming HTTP responses

§Standard responses and Content-Length header

Since HTTP 1.1, to keep a single connection open to serve several HTTP requests and responses, the server must send the appropriate Content-Length HTTP header along with the response.

By default, you are not specifying a Content-Length header when you send back a simple result, such as:

def index = Action {
  Ok("Hello World")
}

Of course, because the content you are sending is well-known, Play is able to compute the content size for you and to generate the appropriate header.

Note that for text-based content it is not as simple as it looks, since the Content-Length header must be computed according the character encoding used to translate characters to bytes.

Actually, we previously saw that the response body is specified using a play.api.libs.iteratee.Enumerator:

def index = Action {
  SimpleResult(
    header = ResponseHeader(200),
    body = Enumerator("Hello World")
  )
}

This means that to compute the Content-Length header properly, Play must consume the whole enumerator and load its content into memory.

§Sending large amounts of data

If it’s not a problem to load the whole content into memory for simple Enumerators, what about large data sets? Let’s say we want to return a large file to the web client.

Let’s first see how to create an Enumerator[Array[Byte]] enumerating the file content:

val file = new java.io.File("/tmp/fileToServe.pdf")
val fileContent: Enumerator[Array[Byte]] = Enumerator.fromFile(file)

Now it looks simple right? Let’s just use this enumerator to specify the response body:

def index = Action {

  val file = new java.io.File("/tmp/fileToServe.pdf")
  val fileContent: Enumerator[Array[Byte]] = Enumerator.fromFile(file)    
    
  SimpleResult(
    header = ResponseHeader(200),
    body = fileContent
  )
}

Actually we have a problem here. As we don’t specify the Content-Length header, Play will have to compute it itself, and the only way to do this is to consume the whole enumerator content and load it into memory, and then compute the response size.

That’s a problem for large files that we don’t want to load completely into memory. So to avoid that, we just have to specify the Content-Length header ourself.

def index = Action {

  val file = new java.io.File("/tmp/fileToServe.pdf")
  val fileContent: Enumerator[Array[Byte]] = Enumerator.fromFile(file)    
    
  SimpleResult(
    header = ResponseHeader(200, Map(CONTENT_LENGTH -> file.length.toString)),
    body = fileContent
  )
}

This way Play will consume the body enumerator in a lazy way, copying each chunk of data to the HTTP response as soon as it is available.

§Serving files

Of course, Play provides easy-to-use helpers for common task of serving a local file:

def index = Action {
  Ok.sendFile(new java.io.File("/tmp/fileToServe.pdf"))
}

This helper will also compute the Content-Type header from the file name, and add the Content-Disposition header to specify how the web browser should handle this response. The default is to ask the web browser to download this file by adding the header Content-Disposition: attachment; filename=fileToServe.pdf to the HTTP response.

You also provide your own file name:

def index = Action {
  Ok.sendFile(
    content = new java.io.File("/tmp/fileToServe.pdf"),
    fileName = _ => "termsOfService.pdf"
  )
}

If you want to serve this file inline:

def index = Action {
  Ok.sendFile(
    content = new java.io.File("/tmp/fileToServe.pdf"),
    inline = true
  )
}

Now you don’t have to specify a file name since the web browser will not try to download it, but will just display the file content in the web browser window. This is useful for content types supported natively by the web browser, such as text, HTML or images.

§Chunked responses

For now, it works well with streaming file content since we are able to compute the content length before streaming it. But what about dynamically computed content, with no content size available?

For this kind of response we have to use Chunked transfer encoding.

Chunked transfer encoding is a data transfer mechanism in version 1.1 of the Hypertext Transfer Protocol (HTTP) in which a web server serves content in a series of chunks. It uses the Transfer-Encoding HTTP response header instead of the Content-Length header, which the protocol would otherwise require. Because the Content-Length header is not used, the server does not need to know the length of the content before it starts transmitting a response to the client (usually a web browser). Web servers can begin transmitting responses with dynamically-generated content before knowing the total size of that content.

The size of each chunk is sent right before the chunk itself, so that a client can tell when it has finished receiving data for that chunk. Data transfer is terminated by a final chunk of length zero.

http://en.wikipedia.org/wiki/Chunked_transfer_encoding

The advantage is that we can serve the data live, meaning that we send chunks of data as soon as they are available. The drawback is that since the web browser doesn’t know the content size, it is not able to display a proper download progress bar.

Let’s say that we have a service somewhere that provides a dynamic InputStream computing some data. First we have to create an Enumerator for this stream:

val data = getDataStream
val dataContent: Enumerator[Array[Byte]] = Enumerator.fromStream(data)

We can now stream these data using a ChunkedResult:

def index = Action {

  val data = getDataStream
  val dataContent: Enumerator[Array[Byte]] = Enumerator.fromStream(data)
  
  ChunkedResult(
    header = ResponseHeader(200),
    chunks = dataContent
  )
}

As always, there are helpers available to do this:

def index = Action {

  val data = getDataStream
  val dataContent: Enumerator[Array[Byte]] = Enumerator.fromStream(data)
  
  Ok.stream(dataContent)
}

Of course, we can use any Enumerator to specify the chunked data:

def index = Action {
  Ok.stream(
    Enumerator("kiki", "foo", "bar").andThen(Enumerator.eof)
  )
}

Tip: Enumerator.callbackEnumerator and Enumerator.pushEnumerator are two convenient way of creating reactive non-blocking enumerators in an imperative style.

We can inspect the HTTP response sent by the server:

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked

4
kiki
3
foo
3
bar
0

We get three chunks followed by one final empty chunk that closes the response.

Next: Comet sockets