servant – A Type-Level Web DSL¶

Static strings¶

As you’ve already seen, you can use type-level strings (enabled with the DataKinds language extension) for static path fragments. Chaining them amounts to /-separating them in a URL.

type UserAPI3 = "users" :> "list-all" :> "now" :> Get '[JSON] [User]
              -- describes an endpoint reachable at:
              -- /users/list-all/now

Delete, Get, Patch, Post and Put¶

The Get combinator is defined in terms of the more general Verb:

data Verb method (statusCode :: Nat) (contentType :: [*]) a
type Get = Verb 'GET 200

There are other predefined type synonyms for other common HTTP methods, such as e.g.:

type Delete = Verb 'DELETE 200
type Patch  = Verb 'PATCH 200
type Post   = Verb 'POST 200
type Put    = Verb 'PUT 200

There are also variants that do not return a 200 status code, such as for example:

type PostCreated  = Verb 'POST 201
type PostAccepted = Verb 'POST 202

An endpoint always ends with a variant of the Verb combinator (unless you write your own combinators). Examples:

type UserAPI4 = "users" :> Get '[JSON] [User]
           :<|> "admins" :> Get '[JSON] [User]

Capture¶

URL captures are segments of the path of a URL that are variable and whose actual value is captured and passed to the request handlers. In many web frameworks, you’ll see it written as in /users/:userid, with that leading : denoting that userid is just some kind of variable name or placeholder. For instance, if userid is supposed to range over all integers greater or equal to 1, our endpoint will match requests made to /users/1, /users/143 and so on.

The Capture combinator in servant takes a (type-level) string representing the “name of the variable” and a type, which indicates the type we want to decode the “captured value” to.

data Capture (s :: Symbol) a
-- s :: Symbol just says that 's' must be a type-level string.

In some web frameworks, you use regexes for captures. We use a FromText class, which the captured value must be an instance of.

Examples:

type UserAPI5 = "user" :> Capture "userid" Integer :> Get '[JSON] User
                -- equivalent to 'GET /user/:userid'
                -- except that we explicitly say that "userid"
                -- must be an integer

           :<|> "user" :> Capture "userid" Integer :> DeleteNoContent '[JSON] NoContent
                -- equivalent to 'DELETE /user/:userid'

In the second case, DeleteNoContent specifies a 204 response code, JSON specifies the content types on which the handler will match, and NoContent says that the response will always be empty.

QueryParam, QueryParams, QueryFlag¶

QueryParam, QueryParams and QueryFlag are about parameters in the query string, i.e., those parameters that come after the question mark (?) in URLs, like sortby in /users?sortby=age, whose value is set to age. QueryParams lets you specify that the query parameter is actually a list of values, which can be specified using ?param=value1&param=value2. This represents a list of values composed of value1 and value2. QueryFlag lets you specify a boolean-like query parameter where a client isn’t forced to specify a value. The absence or presence of the parameter’s name in the query string determines whether the parameter is considered to have the value True or False. For instance, /users?active would list only active users whereas /users would list them all.

Here are the corresponding data type declarations:

data QueryParam (sym :: Symbol) a
data QueryParams (sym :: Symbol) a
data QueryFlag (sym :: Symbol)

Examples:

type UserAPI6 = "users" :> QueryParam "sortby" SortBy :> Get '[JSON] [User]
                -- equivalent to 'GET /users?sortby={age, name}'

Again, your handlers don’t have to deserialize these things (into, for example, a SortBy). servant takes care of it.

ReqBody¶

Each HTTP request can carry some additional data that the server can use in its body, and this data can be encoded in any format – as long as the server understands it. This can be used for example for an endpoint for creating new users: instead of passing each field of the user as a separate query string parameter or something dirty like that, we can group all the data into a JSON object. This has the advantage of supporting nested objects.

servant‘s ReqBody combinator takes a list of content types in which the data encoded in the request body can be represented and the type of that data. And, as you might have guessed, you don’t have to check the content type header, and do the deserialization yourself. We do it for you. And return Bad Request or Unsupported Content Type as appropriate.

Here’s the data type declaration for it:

data ReqBody (contentTypes :: [*]) a

Examples:

type UserAPI7 = "users" :> ReqBody '[JSON] User :> Post '[JSON] User
                -- - equivalent to 'POST /users' with a JSON object
                --   describing a User in the request body
                -- - returns a User encoded in JSON

           :<|> "users" :> Capture "userid" Integer
                        :> ReqBody '[JSON] User
                        :> Put '[JSON] User
                -- - equivalent to 'PUT /users/:userid' with a JSON
                --   object describing a User in the request body
                -- - returns a User encoded in JSON

Request Headers¶

Request headers are used for various purposes, from caching to carrying auth-related data. They consist of a header name and an associated value. An example would be Accept: application/json.

The Header combinator in servant takes a type-level string for the header name and the type to which we want to decode the header’s value (from some textual representation), as illustrated below:

data Header (sym :: Symbol) a

Here’s an example where we declare that an endpoint makes use of the User-Agent header which specifies the name of the software/library used by the client to send the request.

type UserAPI8 = "users" :> Header "User-Agent" Text :> Get '[JSON] [User]

Content types¶

So far, whenever we have used a combinator that carries a list of content types, we’ve always specified '[JSON]. However, servant lets you use several content types, and also lets you define your own content types.

Four content types are provided out-of-the-box by the core servant package: JSON, PlainText, FormUrlEncoded and OctetStream. If for some obscure reason you wanted one of your endpoints to make your user data available under those 4 formats, you would write the API type as below:

type UserAPI9 = "users" :> Get '[JSON, PlainText, FormUrlEncoded, OctetStream] [User]

(There are other packages that provide other content types. For example servant-lucid and servant-blaze allow to generate html pages (using lucid and blaze-html) and both come with a content type for html.)

We will further explain how these content types and your data types can play together in the section about serving an API.

Response Headers¶

Just like an HTTP request, the response generated by a webserver can carry headers too. servant provides a Headers combinator that carries a list of Header types and can be used by simply wrapping the “return type” of an endpoint with it.

data Headers (ls :: [*]) a

If you want to describe an endpoint that returns a “User-Count” header in each response, you could write it as below:

type UserAPI10 = "users" :> Get '[JSON] (Headers '[Header "User-Count" Integer] [User])

Basic Authentication¶

Once you’ve established the basic routes and semantics of your API, it’s time to consider protecting parts of it. Authentication and authorization are broad and nuanced topics; as servant began to explore this space we started small with one of HTTP’s earliest authentication schemes: Basic Authentication.

When protecting endpoints with basic authentication, we need to specify two items:

1. The realm of authentication as per the Basic Authentictaion spec.
2. The datatype returned by the server after authentication is verified. This is usually a User or Customer type datatype.

With those two items in mind, servant provides the following combinator:

data BasicAuth (realm :: Symbol) (userData :: *)

Which is used like so:

type ProtectedAPI12
     = UserAPI                             -- this is public
 :<|> BasicAuth "my-real" User :> UserAPI2 -- this is protected by auth

Interoperability with wai: Raw¶

Finally, we also include a combinator named Raw that provides an escape hatch to the underlying low-level web library wai. It can be used when you want to plug a wai Application into your webservice:

type UserAPI11 = "users" :> Get '[JSON] [User]
                 -- a /users endpoint

            :<|> Raw
                 -- requests to anything else than /users
                 -- go here, where the server will try to
                 -- find a file with the right name
                 -- at the right path

One example for this is if you want to serve a directory of static files along with the rest of your API. But you can plug in everything that is an Application, e.g. a whole web application written in any of the web frameworks that support wai.

The truth behind JSON¶

What exactly is JSON (the type as used in Get '[JSON] User)? Like the 3 other content-types provided out of the box by servant, it’s a really dumb data type.

data JSON
data PlainText
data FormUrlEncoded
data OctetStream

Obviously, this is not all there is to JSON, otherwise it would be quite pointless. Like most of the data types in servant, JSON is mostly there as a special symbol that’s associated with encoding (resp. decoding) to (resp. from) the JSON format. The way this association is performed can be decomposed into two steps.

The first step is to provide a proper MediaType (from http-media) representation for JSON, or for your own content-types. If you look at the haddocks from this link, you can see that we just have to specify application/json using the appropriate functions. In our case, we can just use (//) :: ByteString -> ByteString -> MediaType. The precise way to specify the MediaType is to write an instance for the Accept class:

-- for reference:
class Accept ctype where
    contentType   :: Proxy ctype -> MediaType

instance Accept JSON where
    contentType _ = "application" // "json"

The second step is centered around the MimeRender and MimeUnrender classes. These classes just let you specify a way to encode and decode values into or from your content-type’s representation.

class Accept ctype => MimeRender ctype a where
    mimeRender :: Proxy ctype -> a -> ByteString
    -- alternatively readable as:
    mimeRender :: Proxy ctype -> (a -> ByteString)

Given a content-type and some user type, MimeRender provides a function that encodes values of type a to lazy ByteStrings.

In the case of JSON, this is easily dealt with! For any type a with a ToJSON instance, we can render values of that type to JSON using Data.Aeson.encode.

instance ToJSON a => MimeRender JSON a where
  mimeRender _ = encode

And now the MimeUnrender class, which lets us extract values from lazy ByteStrings, alternatively failing with an error string.

class Accept ctype => MimeUnrender ctype a where
    mimeUnrender :: Proxy ctype -> ByteString -> Either String a

We don’t have much work to do there either, Data.Aeson.eitherDecode is precisely what we need. However, it only allows arrays and objects as toplevel JSON values and this has proven to get in our way more than help us so we wrote our own little function around aeson and attoparsec that allows any type of JSON value at the toplevel of a “JSON document”. Here’s the definition in case you are curious.

eitherDecodeLenient :: FromJSON a => ByteString -> Either String a
eitherDecodeLenient input = do
    v :: Value <- parseOnly (Data.Aeson.Parser.value <* endOfInput) (cs input)
    parseEither parseJSON v

This function is exactly what we need for our MimeUnrender instance.

instance FromJSON a => MimeUnrender JSON a where
    mimeUnrender _ = eitherDecodeLenient

And this is all the code that lets you use JSON with ReqBody, Get, Post and friends. We can check our understanding by implementing support for an HTML content-type, so that users of your webservice can access an HTML representation of the data they want, ready to be included in any HTML document, e.g. using jQuery’s load function, simply by adding Accept: text/html to their request headers.

Case-studies: servant-blaze and servant-lucid¶

These days, most of the haskellers who write their HTML UIs directly from Haskell use either blaze-html or lucid. The best option for servant is obviously to support both (and hopefully other templating solutions!). We’re first going to look at lucid:

data HTMLLucid

Once again, the data type is just there as a symbol for the encoding/decoding functions, except that this time we will only worry about encoding since lucid doesn’t provide a way to extract data from HTML.

instance Accept HTMLLucid where
    contentType _ = "text" // "html" /: ("charset", "utf-8")

Note that this instance uses the (/:) operator from http-media which lets us specify additional information about a content-type, like the charset here.

The rendering instances call similar functions that take types with an appropriate instance to an “abstract” HTML representation and then write that to a ByteString.

instance ToHtml a => MimeRender HTMLLucid a where
    mimeRender _ = renderBS . toHtml

-- let's also provide an instance for lucid's
-- 'Html' wrapper.
instance MimeRender HTMLLucid (Html a) where
    mimeRender _ = renderBS

For blaze-html everything works very similarly:

-- For this tutorial to compile 'HTMLLucid' and 'HTMLBlaze' have to be
-- distinct. Usually you would stick to one html rendering library and then
-- you can go with one 'HTML' type.
data HTMLBlaze

instance Accept HTMLBlaze where
    contentType _ = "text" // "html" /: ("charset", "utf-8")

instance ToMarkup a => MimeRender HTMLBlaze a where
    mimeRender _ = renderHtml . Text.Blaze.Html.toHtml

-- while we're at it, just like for lucid we can
-- provide an instance for rendering blaze's 'Html' type
instance MimeRender HTMLBlaze Text.Blaze.Html.Html where
    mimeRender _ = renderHtml

Both servant-blaze and servant-lucid let you use HTMLLucid and HTMLBlaze in any content-type list as long as you provide an instance of the appropriate class (ToMarkup for blaze-html, ToHtml for lucid).

We can now write a webservice that uses servant-lucid to show the HTMLLucid content-type in action. We will be serving the following API:

type PersonAPI = "persons" :> Get '[JSON, HTMLLucid] [Person]

where Person is defined as follows:

data Person = Person
  { firstName :: String
  , lastName  :: String
  } deriving Generic -- for the JSON instance

instance ToJSON Person

Now, let’s teach lucid how to render a Person as a row in a table, and then a list of Persons as a table with a row per person.

-- HTML serialization of a single person
instance ToHtml Person where
  toHtml person =
    tr_ $ do
      td_ (toHtml $ firstName person)
      td_ (toHtml $ lastName person)

  -- do not worry too much about this
  toHtmlRaw = toHtml

-- HTML serialization of a list of persons
instance ToHtml [Person] where
  toHtml persons = table_ $ do
    tr_ $ do
      th_ "first name"
      th_ "last name"

    -- this just calls toHtml on each person of the list
    -- and concatenates the resulting pieces of HTML together
    foldMap toHtml persons

  toHtmlRaw = toHtml

We create some Person values and serve them as a list:

people :: [Person]
people =
  [ Person "Isaac"  "Newton"
  , Person "Albert" "Einstein"
  ]

personAPI :: Proxy PersonAPI
personAPI = Proxy

server4 :: Server PersonAPI
server4 = return people

app2 :: Application
app2 = serve personAPI server4

And we’re good to go:

$ curl http://localhost:8081/persons
[{"lastName":"Newton","firstName":"Isaac"},{"lastName":"Einstein","firstName":"Albert"}]
$ curl -H 'Accept: text/html' http://localhost:8081/persons
<table><tr><td>first name</td><td>last name</td></tr><tr><td>Isaac</td><td>Newton</td></tr><tr><td>Albert</td><td>Einstein</td></tr></table>
# or just point your browser to http://localhost:8081/persons

Performing IO¶

Another important instance from the list above is MonadIO m => MonadIO (ExceptT e m). MonadIO is a class from the transformers package defined as:

class Monad m => MonadIO m where
  liftIO :: IO a -> m a

The IO monad provides a MonadIO instance. Hence for any type e, ExceptT e IO has a MonadIO instance. So if you want to run any kind of IO computation in your handlers, just use liftIO:

type IOAPI1 = "myfile.txt" :> Get '[JSON] FileContent

newtype FileContent = FileContent
  { content :: String }
  deriving Generic

instance ToJSON FileContent

server5 :: Server IOAPI1
server5 = do
  filecontent <- liftIO (readFile "myfile.txt")
  return (FileContent filecontent)

Failing, through ServantErr¶

If you want to explicitly fail at providing the result promised by an endpoint using the appropriate HTTP status code (not found, unauthorized, etc) and some error message, all you have to do is use the throwError function mentioned above and provide it with the appropriate value of type ServantErr, which is defined as:

data ServantErr = ServantErr
    { errHTTPCode     :: Int
    , errReasonPhrase :: String
    , errBody         :: ByteString -- lazy bytestring
    , errHeaders      :: [Header]
    }

Many standard values are provided out of the box by the Servant.Server module. If you want to use these values but add a body or some headers, just use record update syntax:

failingHandler :: ExceptT ServantErr IO ()
failingHandler = throwError myerr

  where myerr :: ServantErr
        myerr = err503 { errBody = "Sorry dear user." }

Here’s an example where we return a customised 404-Not-Found error message in the response body if “myfile.txt” isn’t there:

server6 :: Server IOAPI1
server6 = do
  exists <- liftIO (doesFileExist "myfile.txt")
  if exists
    then liftIO (readFile "myfile.txt") >>= return . FileContent
    else throwError custom404Err

  where custom404Err = err404 { errBody = "myfile.txt just isn't there, please leave this server alone." }

Here’s how that server looks in action:

$ curl --verbose http://localhost:8081/myfile.txt
[snip]
* Connected to localhost (127.0.0.1) port 8081 (#0)
> GET /myfile.txt HTTP/1.1
> User-Agent: curl/7.30.0
> Host: localhost:8081
> Accept: */*
>
< HTTP/1.1 404 Not Found
[snip]
myfile.txt just isnt there, please leave this server alone.

$ echo Hello > myfile.txt

$ curl --verbose http://localhost:8081/myfile.txt
[snip]
* Connected to localhost (127.0.0.1) port 8081 (#0)
> GET /myfile.txt HTTP/1.1
> User-Agent: curl/7.30.0
> Host: localhost:8081
> Accept: */*
>
< HTTP/1.1 200 OK
[snip]
< Content-Type: application/json
[snip]
{"content":"Hello\n"}

Natural transformations¶

If we have a function that gets us from an m a to an n a, for any a, what do we have?

newtype m :~> n = Nat { unNat :: forall a. m a -> n a}

For example:

listToMaybeNat :: [] :~> Maybe
listToMaybeNat = Nat listToMaybe  -- from Data.Maybe

(Nat comes from “natural transformation”, in case you’re wondering.)

So if you want to write handlers using another monad/type than ExceptT ServantErr IO, say the Reader String monad, the first thing you have to prepare is a function:

readerToHandler :: Reader String :~> ExceptT ServantErr IO

Let’s start with readerToHandler'. We obviously have to run the Reader computation by supplying it with a String, like "hi". We get an a out from that and can then just return it into ExceptT. We can then just wrap that function with the Nat constructor to make it have the fancier type.

readerToHandler' :: forall a. Reader String a -> ExceptT ServantErr IO a
readerToHandler' r = return (runReader r "hi")

readerToHandler :: Reader String :~> ExceptT ServantErr IO
readerToHandler = Nat readerToHandler'

We can write some simple webservice with the handlers running in Reader String.

type ReaderAPI = "a" :> Get '[JSON] Int
            :<|> "b" :> Get '[JSON] String

readerAPI :: Proxy ReaderAPI
readerAPI = Proxy

readerServerT :: ServerT ReaderAPI (Reader String)
readerServerT = a :<|> b

  where a :: Reader String Int
        a = return 1797

        b :: Reader String String
        b = ask

We unfortunately can’t use readerServerT as an argument of serve, because serve wants a Server ReaderAPI, i.e., with handlers running in ExceptT ServantErr IO. But there’s a simple solution to this.

Enter enter¶

That’s right. We have just written readerToHandler, which is exactly what we would need to apply to all handlers to make the handlers have the right type for serve. Being cumbersome to do by hand, we provide a function enter which takes a natural transformation between two parametrized types m and n and a ServerT someapi m, and returns a ServerT someapi n.

In our case, we can wrap up our little webservice by using enter readerToHandler on our handlers.

readerServer :: Server ReaderAPI
readerServer = enter readerToHandler readerServerT

app4 :: Application
app4 = serve readerAPI readerServer

This is the webservice in action:

$ curl http://localhost:8081/a
1797
$ curl http://localhost:8081/b
"hi"