§Body parsers
§What is a body parser?
An HTTP request is a header followed by a body. The header is typically small - it can be safely buffered in memory, hence in Play it is modelled using the RequestHeader
class. The body however can be potentially very long, and so is not buffered in memory, but rather is modelled as a stream. However, many request body payloads are small and can be modelled in memory, and so to map the body stream to an object in memory, Play provides a BodyParser
abstraction.
Since Play is an asynchronous framework, the traditional InputStream
can’t be used to read the request body - input streams are blocking, when you invoke read
, the thread invoking it must wait for data to be available. Instead, Play uses an asynchronous streaming library called Akka Streams. Akka Streams is an implementation of Reactive Streams, a SPI that allows many asynchronous streaming APIs to seamlessly work together, so though traditional InputStream
based technologies are not suitable for use with Play, Akka Streams and the entire ecosystem of asynchronous libraries around Reactive Streams will provide you with everything you need.
§More about Actions
Previously we said that an Action
was a Request => Result
function. This is not entirely true. Let’s have a more precise look at the Action
trait:
trait Action[A] extends (Request[A] => Result) {
def parser: BodyParser[A]
}
First we see that there is a generic type A
, and then that an action must define a BodyParser[A]
. With Request[A]
being defined as:
trait Request[+A] extends RequestHeader {
def body: A
}
The A
type is the type of the request body. We can use any Scala type as the request body, for example String
, NodeSeq
, Array[Byte]
, JsonValue
, or java.io.File
, as long as we have a body parser able to process it.
To summarize, an Action[A]
uses a BodyParser[A]
to retrieve a value of type A
from the HTTP request, and to build a Request[A]
object that is passed to the action code.
§Using the built in body parsers
Most typical web apps will not need to use custom body parsers, they can simply work with Play’s built in body parsers. These include parsers for JSON, XML, forms, as well as handling plain text bodies as Strings and byte bodies as ByteString
.
§The default body parser
The default body parser that’s used if you do not explicitly select a body parser will look at the incoming Content-Type
header, and parse the body accordingly. So for example, a Content-Type
of type application/json
will be parsed as a JsValue
, while a Content-Type
of application/x-www-form-urlencoded
will be parsed as a Map[String, Seq[String]]
.
The default body parser produces a body of type AnyContent
. The various types supported by AnyContent
are accessible via as
methods, such as asJson
, which returns an Option
of the body type:
def save: Action[AnyContent] = Action { (request: Request[AnyContent]) =>
val body: AnyContent = request.body
val jsonBody: Option[JsValue] = body.asJson
// Expecting json body
jsonBody
.map { json => Ok("Got: " + (json \ "name").as[String]) }
.getOrElse {
BadRequest("Expecting application/json request body")
}
}
The following is a mapping of types supported by the default body parser:
- text/plain:
String
, accessible viaasText
. - application/json:
JsValue
, accessible viaasJson
. - application/xml, text/xml or application/XXX+xml:
scala.xml.NodeSeq
, accessible viaasXml
. - application/x-www-form-urlencoded:
Map[String, Seq[String]]
, accessible viaasFormUrlEncoded
. - multipart/form-data:
MultipartFormData
, accessible viaasMultipartFormData
. - Any other content type:
RawBuffer
, accessible viaasRaw
.
The default body parser tries to determine if the request has a body before it tries to parse. According to the HTTP spec, the presence of either the Content-Length
or Transfer-Encoding
header signals the presence of a body, so the parser will only parse if one of those headers is present, or on FakeRequest
when a non-empty body has explicitly been set.
If you would like to try to parse a body in all cases, you can use the anyContent
body parser, described below.
§Choosing an explicit body parser
If you want to explicitly select a body parser, this can be done by passing a body parser to the Action
apply
or async
method.
Play provides a number of body parsers out of the box, this is made available through the PlayBodyParsers
trait, which can be injected into your controller.
So for example, to define an action expecting a json body (as in the previous example):
def save: Action[JsValue] = Action(parse.json) { (request: Request[JsValue]) =>
Ok("Got: " + (request.body \ "name").as[String])
}
Note this time that the type of the body is JsValue
, which makes it easier to work with the body since it’s no longer an Option
. The reason why it’s not an Option
is because the json body parser will validate that the request has a Content-Type
of application/json
, and send back a 415 Unsupported Media Type
response if the request doesn’t meet that expectation. Hence we don’t need to check again in our action code.
This of course means that clients have to be well behaved, sending the correct Content-Type
headers with their requests. If you want to be a little more relaxed, you can instead use tolerantJson
, which will ignore the Content-Type
and try to parse the body as json regardless:
def save: Action[JsValue] = Action(parse.tolerantJson) { (request: Request[JsValue]) =>
Ok("Got: " + (request.body \ "name").as[String])
}
Here is another example, which will store the request body in a file:
def save: Action[File] = Action(parse.file(to = new File("/tmp/upload"))) { (request: Request[File]) =>
Ok("Saved the request content to " + request.body)
}
§Combining body parsers
In the previous example, all request bodies are stored in the same file. This is a bit problematic isn’t it? Let’s write another custom body parser that extracts the user name from the request Session, to give a unique file for each user:
val storeInUserFile = parse.using { request =>
request.session
.get("username")
.map { user => parse.file(to = new File("/tmp/" + user + ".upload")) }
.getOrElse {
sys.error("You don't have the right to upload here")
}
}
def save: Action[File] = Action(storeInUserFile) { request => Ok("Saved the request content to " + request.body) }
Note: Here we are not really writing our own BodyParser, but just combining existing ones. This is often enough and should cover most use cases. Writing a
BodyParser
from scratch is covered in the advanced topics section.
§Max content length
Text based body parsers (such as text, json, xml or formUrlEncoded) use a max content length because they have to load all the content into memory. By default, the maximum content length that they will parse is 100KB. It can be overridden by specifying the play.http.parser.maxMemoryBuffer
property in application.conf
:
play.http.parser.maxMemoryBuffer=128K
For parsers that buffer content on disk, such as the raw parser or multipart/form-data
, the maximum content length is specified using the play.http.parser.maxDiskBuffer
property, it defaults to 10MB. The multipart/form-data
parser also enforces the text max length property for the aggregate of the data fields.
You can also override the default maximum length for a given action:
// Accept only 10KB of data.
def save: Action[String] = Action(parse.text(maxLength = 1024 * 10)) { (request: Request[String]) =>
Ok("Got: " + text)
}
You can also wrap any body parser with maxLength
:
// Accept only 10KB of data.
def save: Action[Either[MaxSizeExceeded, File]] = Action(parse.maxLength(1024 * 10, storeInUserFile)) {
request =>
Ok("Saved the request content to " + request.body)
}
§Writing a custom body parser
A custom body parser can be made by implementing the BodyParser
trait. This trait is simply a function:
trait BodyParser[+A] extends (RequestHeader => Accumulator[ByteString, Either[Result, A]])
The signature of this function may be a bit daunting at first, so let’s break it down.
The function takes a RequestHeader
. This can be used to check information about the request - most commonly, it is used to get the Content-Type
, so that the body can be correctly parsed.
The return type of the function is an Accumulator
. An accumulator is a thin layer around an Akka Streams Sink
. An accumulator asynchronously accumulates streams of elements into a result, it can be run by passing in an Akka Streams Source
, this will return a Future
that will be redeemed when the accumulator is complete. It is essentially the same thing as a Sink[E, Future[A]]
, in fact it is nothing more than a wrapper around this type, but the big difference is that Accumulator
provides convenient methods such as map
, mapFuture
, recover
etc. for working with the result as if it were a promise, where Sink
requires all such operations to be wrapped in a mapMaterializedValue
call.
The accumulator that the apply
method returns consumes elements of type ByteString
- these are essentially arrays of bytes, but differ from byte[]
in that ByteString
is immutable, and many operations such as slicing and appending happen in constant time.
The return type of the accumulator is Either[Result, A]
- it will either return a Result
, or it will return a body of type A
. A result is generally returned in the case of an error, for example, if the body failed to be parsed, if the Content-Type
didn’t match the type that the body parser accepts, or if an in memory buffer was exceeded. When the body parser returns a result, this will short circuit the processing of the action - the body parsers result will be returned immediately, and the action will never be invoked.
§Directing the body elsewhere
One common use case for writing a body parser is for when you actually don’t want to parse the body, rather, you want to stream it elsewhere. To do this, you may define a custom body parser:
import javax.inject._
import scala.concurrent.ExecutionContext
import akka.util.ByteString
import play.api.libs.streams._
import play.api.libs.ws._
import play.api.mvc._
class MyController @Inject() (ws: WSClient, val controllerComponents: ControllerComponents)(
implicit ec: ExecutionContext
) extends BaseController {
def forward(request: WSRequest): BodyParser[WSResponse] = BodyParser { req =>
Accumulator.source[ByteString].mapFuture { source =>
request
.withBody(source)
.execute("POST")
.map(Right.apply)
}
}
def myAction: Action[WSResponse] = Action(forward(ws.url("https://example.com"))) { req => Ok("Uploaded") }
}
§Custom parsing using Akka Streams
In rare circumstances, it may be necessary to write a custom parser using Akka Streams. In most cases it will suffice to buffer the body in a ByteString
first, this will typically offer a far simpler way of parsing since you can use imperative methods and random access on the body.
However, when that’s not feasible, for example when the body you need to parse is too long to fit in memory, then you may need to write a custom body parser.
A full description of how to use Akka Streams is beyond the scope of this documentation - the best place to start is to read the Akka Streams documentation. However, the following shows a CSV parser, which builds on the Parsing lines from a stream of ByteStrings documentation from the Akka Streams cookbook:
import akka.stream.scaladsl._
import akka.util.ByteString
import play.api.libs.streams._
import play.api.mvc.BodyParser
val Action = inject[DefaultActionBuilder]
val csv: BodyParser[Seq[Seq[String]]] = BodyParser { req =>
// A flow that splits the stream into CSV lines
val sink: Sink[ByteString, Future[Seq[Seq[String]]]] = Flow[ByteString]
// We split by the new line character, allowing a maximum of 1000 characters per line
.via(Framing.delimiter(ByteString("\n"), 1000, allowTruncation = true))
// Turn each line to a String and split it by commas
.map(_.utf8String.trim.split(",").toSeq)
// Now we fold it into a list
.toMat(Sink.fold(Seq.empty[Seq[String]])(_ :+ _))(Keep.right)
// Convert the body to a Right either
Accumulator(sink).map(Right.apply)
}
§Deferred body parsing
By default body parsing takes place before action composition happens. It’s however possible to defer body parsing after some (or all) actions defined via action composition have been processed. More details can be found here.
Next: Actions composition