§The Play WS API
Sometimes we would like to call other HTTP services from within a Play application. Play supports this via its WS library, which provides a way to make asynchronous HTTP calls.
There are two important parts to using the WS API: making a request, and processing the response. We’ll discuss how to make both GET and POST HTTP requests first, and then show how to process the response from WS. Finally, we’ll discuss some common use cases.
§Making a Request
To use WS, first add ws
to your build.sbt
file:
libraryDependencies ++= Seq(
ws
)
Now any controller or component that wants to use WS will have to declare a dependency on the WSClient
:
import javax.inject.Inject
import scala.concurrent.Future
import scala.concurrent.duration._
import play.api.mvc._
import play.api.libs.ws._
import play.api.http.HttpEntity
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.util.ByteString
import scala.concurrent.ExecutionContext
class Application @Inject() (ws: WSClient) extends Controller {
}
We’ve called the WSClient
instance ws
, all the following examples will assume this name.
To build an HTTP request, you start with ws.url()
to specify the URL.
val request: WSRequest = ws.url(url)
This returns a WSRequest that you can use to specify various HTTP options, such as setting headers. You can chain calls together to construct complex requests.
val complexRequest: WSRequest =
request.withHeaders("Accept" -> "application/json")
.withRequestTimeout(10000.millis)
.withQueryString("search" -> "play")
You end by calling a method corresponding to the HTTP method you want to use. This ends the chain, and uses all the options defined on the built request in the WSRequest
.
val futureResponse: Future[WSResponse] = complexRequest.get()
This returns a Future[WSResponse]
where the Response contains the data returned from the server.
§Request with authentication
If you need to use HTTP authentication, you can specify it in the builder, using a username, password, and an AuthScheme. Valid case objects for the AuthScheme are BASIC
, DIGEST
, KERBEROS
, NTLM
, and SPNEGO
.
ws.url(url).withAuth(user, password, WSAuthScheme.BASIC).get()
§Request with follow redirects
If an HTTP call results in a 302 or a 301 redirect, you can automatically follow the redirect without having to make another call.
ws.url(url).withFollowRedirects(true).get()
§Request with query parameters
Parameters can be specified as a series of key/value tuples.
ws.url(url).withQueryString("paramKey" -> "paramValue").get()
§Request with additional headers
Headers can be specified as a series of key/value tuples.
ws.url(url).withHeaders("headerKey" -> "headerValue").get()
If you are sending plain text in a particular format, you may want to define the content type explicitly.
ws.url(url).withHeaders("Content-Type" -> "application/xml").post(xmlString)
§Request with virtual host
A virtual host can be specified as a string.
ws.url(url).withVirtualHost("192.168.1.1").get()
§Request with timeout
If you wish to specify a request timeout, you can use withRequestTimeout
to set a value. An infinite timeout can be set by passing Duration.Inf
.
ws.url(url).withRequestTimeout(5000.millis).get()
§Submitting form data
To post url-form-encoded data a Map[String, Seq[String]]
needs to be passed into post
.
ws.url(url).post(Map("key" -> Seq("value")))
§Submitting multipart/form data
To post multipart-form-encoded data a Source[play.api.mvc.MultipartFormData.Part[Source[ByteString, Any]], Any]
needs to be passed into post
.
ws.url(url).post(Source.single(DataPart("key", "value")))
To upload a file you need to pass a play.api.mvc.MultipartFormData.FilePart[Source[ByteString, Any]]
to the Source
:
ws.url(url).post(Source(FilePart("hello", "hello.txt", Option("text/plain"), FileIO.fromFile(tmpFile)) :: DataPart("key", "value") :: List()))
§Submitting JSON data
The easiest way to post JSON data is to use the JSON library.
import play.api.libs.json._
val data = Json.obj(
"key1" -> "value1",
"key2" -> "value2"
)
val futureResponse: Future[WSResponse] = ws.url(url).post(data)
§Submitting XML data
The easiest way to post XML data is to use XML literals. XML literals are convenient, but not very fast. For efficiency, consider using an XML view template, or a JAXB library.
val data = <person>
<name>Steve</name>
<age>23</age>
</person>
val futureResponse: Future[WSResponse] = ws.url(url).post(data)
§Streaming data
It’s also possible to stream data.
For example, imagine you have executed a database query that is returning a large image, and you would like to forward that data to a different endpoint for further processing. Ideally, if you can send the data as you receive it from the database, you will reduce latency and also avoid problems resulting from loading in memory a large set of data. If your database access library supports Reactive Streams (for instance, Slick does), here is an example showing how you could implement the described behavior:
val wsResponse: Future[WSResponse] = ws.url(url)
.withBody(StreamedBody(largeImageFromDB)).execute("PUT")
The largeImageFromDB
in the code snippet above is an Akka Streams Source[ByteString, _]
.
§Request Filters
You can do additional processing on a WSRequest by adding a request filter. A request filter is added by extending the play.api.libs.ws.WSRequestFilter
trait, and then adding it to the request with request.withRequestFilter(filter)
.
A sample request filter that logs the request in cURL format to SLF4J has been added in play.api.libs.ws.ahc.AhcCurlRequestLogger
.
ws.url(s"http://localhost:$testServerPort")
.withRequestFilter(AhcCurlRequestLogger())
.withBody(Map("param1" -> Seq("value1")))
.put(Map("key" -> Seq("value")))
will output:
curl \
--verbose \
--request PUT \
--header 'Content-Type: application/x-www-form-urlencoded; charset=utf-8' \
--data 'key=value' \
'http://localhost:19001/
§Processing the Response
Working with the Response is easily done by mapping inside the Future.
The examples given below have some common dependencies that will be shown once here for brevity.
Whenever an operation is done on a Future
, an implicit execution context must be available - this declares which thread pool the callback to the future should run in. You can inject the default Play execution context in your DI-ed class by declaring an additional dependency to ExecutionContext
in the class’ constructor:
class PersonService @Inject()(implicit context: ExecutionContext) {
// ...
}
If you are not using DI, you can still access the default Play execution context:
implicit val context = play.api.libs.concurrent.Execution.Implicits.defaultContext
The examples also use the following case class for serialization/deserialization:
case class Person(name: String, age: Int)
§Processing a response as JSON
You can process the response as a JSON object by calling response.json
.
val futureResult: Future[String] = ws.url(url).get().map {
response =>
(response.json \ "person" \ "name").as[String]
}
The JSON library has a useful feature that will map an implicit Reads[T]
directly to a class:
import play.api.libs.json._
implicit val personReads = Json.reads[Person]
val futureResult: Future[JsResult[Person]] = ws.url(url).get().map {
response => (response.json \ "person").validate[Person]
}
§Processing a response as XML
You can process the response as an XML literal by calling response.xml
.
val futureResult: Future[scala.xml.NodeSeq] = ws.url(url).get().map {
response =>
response.xml \ "message"
}
§Processing large responses
Calling get()
, post()
or execute()
will cause the body of the response to be loaded into memory before the response is made available. When you are downloading a large, multi-gigabyte file, this may result in unwelcomed garbage collection or even out of memory errors.
WS
lets you consume the response’s body incrementally by using an Akka Streams Sink
. The stream()
method on WSRequest
returns a Future[StreamedResponse]
. A StreamedResponse
is a simple container holding together the response’s headers and body.
Here is a trivial example that uses a folding Sink
to count the number of bytes returned by the response:
// Make the request
val futureResponse: Future[StreamedResponse] =
ws.url(url).withMethod("GET").stream()
val bytesReturned: Future[Long] = futureResponse.flatMap {
res =>
// Count the number of bytes returned
res.body.runWith(Sink.fold[Long, ByteString](0L){ (total, bytes) =>
total + bytes.length
})
}
Alternatively, you could also stream the body out to another location. For example, a file:
// Make the request
val futureResponse: Future[StreamedResponse] =
ws.url(url).withMethod("GET").stream()
val downloadedFile: Future[File] = futureResponse.flatMap {
res =>
val outputStream = java.nio.file.Files.newOutputStream(file.toPath)
// The sink that writes to the output stream
val sink = Sink.foreach[ByteString] { bytes =>
outputStream.write(bytes.toArray)
}
// materialize and run the stream
res.body.runWith(sink).andThen {
case result =>
// Close the output stream whether there was an error or not
outputStream.close()
// Get the result or rethrow the error
result.get
}.map(_ => file)
}
Another common destination for response bodies is to stream them back from a controller’s Action
:
def downloadFile = Action.async {
// Make the request
ws.url(url).withMethod("GET").stream().map {
case StreamedResponse(response, body) =>
// Check that the response was successful
if (response.status == 200) {
// Get the content type
val contentType = response.headers.get("Content-Type").flatMap(_.headOption)
.getOrElse("application/octet-stream")
// If there's a content length, send that, otherwise return the body chunked
response.headers.get("Content-Length") match {
case Some(Seq(length)) =>
Ok.sendEntity(HttpEntity.Streamed(body, Some(length.toLong), Some(contentType)))
case _ =>
Ok.chunked(body).as(contentType)
}
} else {
BadGateway
}
}
}
As you may have noticed, before calling stream()
we need to set the HTTP method to use by calling withMethod
on the request. Here follows another example that uses PUT
instead of GET
:
val futureResponse: Future[StreamedResponse] =
ws.url(url).withMethod("PUT").withBody("some body").stream()
Of course, you can use any other valid HTTP verb.
§Common Patterns and Use Cases
§Chaining WS calls
Using for comprehensions is a good way to chain WS calls in a trusted environment. You should use for comprehensions together with Future.recover to handle possible failure.
val futureResponse: Future[WSResponse] = for {
responseOne <- ws.url(urlOne).get()
responseTwo <- ws.url(responseOne.body).get()
responseThree <- ws.url(responseTwo.body).get()
} yield responseThree
futureResponse.recover {
case e: Exception =>
val exceptionData = Map("error" -> Seq(e.getMessage))
ws.url(exceptionUrl).post(exceptionData)
}
§Using in a controller
When making a request from a controller, you can map the response to a Future[Result]
. This can be used in combination with Play’s Action.async
action builder, as described in Handling Asynchronous Results.
def wsAction = Action.async {
ws.url(url).get().map { response =>
Ok(response.body)
}
}
status(wsAction(FakeRequest())) must_== OK
§Using WSClient with Future Timeout
If a chain of WS calls does not complete in time, it may be useful to wrap the result in a timeout block, which will return a failed Future if the chain does not complete in time. The best way to do this is with Play’s non-blocking Timeout feature, using play.api.libs.concurrent.Timeout
.
§Directly creating WSClient
We recommend that you get your WSClient
instances using dependency injection as described above. WSClient
instances created through dependency injection are simpler to use because they are automatically created when the application starts and cleaned up when the application stops.
However, if you choose, you can instantiate a WSClient
directly from code and use this for making requests or for configuring underlying AsyncHttpClient
options. If you create a WSClient manually then you must call client.close()
to clean it up when you’ve finished with it. Each client creates its own thread pool. If you fail to close the client or if you create too many clients then you will run out of threads or file handles -— you’ll get errors like “Unable to create new native thread” or “too many open files” as the underlying resources are consumed.
import com.typesafe.config.ConfigFactory
import play.api._
import play.api.libs.ws._
val configuration = Configuration.reference ++ Configuration(ConfigFactory.parseString(
"""
|ws.followRedirects = true
""".stripMargin))
// If running in Play, environment should be injected
val environment = Environment(new File("."), this.getClass.getClassLoader, Mode.Prod)
val parser = new WSConfigParser(configuration, environment)
val config = new AhcWSClientConfig(wsClientConfig = parser.parse())
val builder = new AhcConfigBuilder(config)
val logging = new AsyncHttpClientConfig.AdditionalChannelInitializer() {
override def initChannel(channel: io.netty.channel.Channel): Unit = {
channel.pipeline.addFirst("log", new io.netty.handler.logging.LoggingHandler("debug"))
}
}
val ahcBuilder = builder.configure()
ahcBuilder.setHttpAdditionalChannelInitializer(logging)
val ahcConfig = ahcBuilder.build()
val wsClient = AhcWSClient(ahcConfig)
You can also use play.api.test.WsTestClient.withTestClient
to create an instance of WSClient
in a functional test. See ScalaTestingWebServiceClients for more details.
Or, you can run the WSClient
completely standalone without involving a running Play application at all:
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import play.api.libs.ws._
import play.api.libs.ws.ahc.AhcWSClient
import scala.concurrent.Future
object Main {
import scala.concurrent.ExecutionContext.Implicits._
def main(args: Array[String]): Unit = {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
val wsClient = AhcWSClient()
call(wsClient)
.andThen { case _ => wsClient.close() }
.andThen { case _ => system.terminate() }
}
def call(wsClient: WSClient): Future[Unit] = {
wsClient.url("http://www.google.com").get().map { response =>
val statusText: String = response.statusText
println(s"Got a response $statusText")
}
}
}
Again, once you are done with your custom client work, you must close the client:
wsClient.close()
Ideally, you should close a client after you know all requests have been completed. Be careful of using an automatic resource management pattern to close the client, because WSClient logic is asynchronous and many ARM solutions may be designed for a single threaded synchronous solution.
§Accessing AsyncHttpClient
You can get access to the underlying AsyncHttpClient from a WSClient
.
import org.asynchttpclient.AsyncHttpClient
val client: AsyncHttpClient = ws.underlying
This is important in a couple of cases. WS has a couple of limitations that require access to the underlying client:
WS
does not support streaming body upload. In this case, you should use theFeedableBodyGenerator
provided by AsyncHttpClient.
§Configuring WS
Use the following properties in application.conf
to configure the WS client:
play.ws.followRedirects
: Configures the client to follow 301 and 302 redirects (default is true).play.ws.useProxyProperties
: To use the JVM system’s HTTP proxy settings (http.proxyHost, http.proxyPort) (default is true).play.ws.useragent
: To configure the User-Agent header field.play.ws.compressionEnabled
: Set it to true to use gzip/deflater encoding (default is false).
§Configuring WS with SSL
To configure WS for use with HTTP over SSL/TLS (HTTPS), please see Configuring WS SSL.
§Configuring Timeouts
There are 3 different timeouts in WS. Reaching a timeout causes the WS request to interrupt.
play.ws.timeout.connection
: The maximum time to wait when connecting to the remote host (default is 120 seconds).play.ws.timeout.idle
: The maximum time the request can stay idle (connection is established but waiting for more data) (default is 120 seconds).play.ws.timeout.request
: The total time you accept a request to take (it will be interrupted even if the remote host is still sending data) (default is 120 seconds).
The request timeout can be overridden for a specific connection with withRequestTimeout()
(see “Making a Request” section).
§Configuring AsyncHttpClientConfig
The following advanced settings can be configured on the underlying AsyncHttpClientConfig.
Please refer to the AsyncHttpClientConfig Documentation for more information.
Note:
allowPoolingConnection
andallowSslConnectionPool
are combined in AsyncHttpClient 2.0 into a singlekeepAlive
variable. As such,play.ws.ning.allowPoolingConnection
andplay.ws.ning.allowSslConnectionPool
are not valid and will throw an exception if configured.
play.ws.ahc.keepAlive
play.ws.ahc.maxConnectionsPerHost
play.ws.ahc.maxConnectionsTotal
play.ws.ahc.maxConnectionLifetime
play.ws.ahc.idleConnectionInPoolTimeout
play.ws.ahc.maxNumberOfRedirects
play.ws.ahc.maxRequestRetry
play.ws.ahc.disableUrlEncoding