Interface

Introduction

Gun is an HTTP client for Erlang/OTP.

Gun supports the HTTP/2, HTTP/1.1 and Websocket protocols.

Prerequisites

Knowledge of Erlang, but also of the HTTP/1.1, HTTP/2 and Websocket protocols is required in order to read this guide.

Supported platforms

Gun is tested and supported on Linux, FreeBSD, Windows and OSX.

Gun is developed for Erlang/OTP 19.0 and newer.

License

Gun uses the ISC License.

Copyright (c) 2013-2018, Loïc Hoguin <essen@ninenines.eu>

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

Versioning

Gun uses Semantic Versioning 2.0.0.

Conventions

In the HTTP protocol, the method name is case sensitive. All standard method names are uppercase.

Header names are case insensitive. Gun converts all the header names to lowercase, and expects your application to provide lowercase header names.

The same applies to any other case insensitive value.

Starting and stopping

This chapter describes how to start and stop the Gun application.

Setting up

Specify Gun as a dependency to your application in your favorite build tool.

With Erlang.mk this is done by adding gun to the DEPS variable in your Makefile.

Adding Gun as an Erlang.mk dependency

Starting

Gun is an OTP application. It needs to be started before you can use it.

Starting Gun in an Erlang shell

Stopping

You can stop Gun using the application:stop/1 function, however only Gun will be stopped. This is the reverse of application:start/1. The application_ensure_all_started/1 function has no equivalent for stopping all applications.

Stopping Gun

Supported protocols

This chapter describes the protocols supported and the operations available to them.

HTTP/1.1

HTTP/1.1 is a text request-response protocol. The client sends a request, the server sends back a response.

Gun provides convenience functions for performing GET, HEAD, OPTIONS, POST, PATCH, PUT, and DELETE requests. All these functions are aliases of gun:request/4,5,6 for the respective methods. Gun also provides a gun:data/4 function for streaming the request body.

Gun will send a gun_inform message for every intermediate informational responses received. They will always be sent before the gun_response message.

Gun will send a gun_response message for every response received, followed by zero or more gun_data messages for the response body, which is optionally terminated by a gun_trailers message. If something goes wrong, a gun_error will be sent instead.

Gun provides convenience functions for dealing with messages. The gun:await/2,3,4 function waits for a response to the given request, and the gun:await_body/2,3,4 function for the response body. The gun:flush/1 function can be used to clear all messages related to a request or a connection from the mailbox of the calling process.

The function gun:cancel/2 can be used to silence the response to a request previously sent if it is no longer needed. When using HTTP/1.1 there is no multiplexing so Gun will have to receive the response fully before any other responses can be received.

Finally, Gun can upgrade an HTTP/1.1 connection to Websocket. It provides the gun:ws_upgrade/2,3,4 function for that purpose. A gun_upgrade message will be sent on success; a gun_response message otherwise.

HTTP/2

HTTP/2 is a binary protocol based on HTTP, compatible with the HTTP semantics, that reduces the complexity of parsing requests and responses, compresses the HTTP headers and allows the server to push additional resources along with the normal response to the original request.

The HTTP/2 interface is very similar to HTTP/1.1, so this section instead focuses on the differences in the interface for the two protocols.

Gun will send gun_push messages for every push received. They will always be sent before the gun_response message. They can be ignored safely if they are not needed, or they can be canceled.

The gun:cancel/2 function will use the HTTP/2 stream cancellation mechanism which allows Gun to inform the server to stop sending a response for this particular request, saving resources.

It is not currently possible to upgrade an HTTP/2 connection to Websocket. Support for this will be added in a future release.

Websocket

Websocket is a binary protocol built on top of HTTP that allows asynchronous concurrent communication between the client and the server. A Websocket server can push data to the client at any time.

Websocket is only available as a connection upgrade over an HTTP/1.1 connection.

Once the Websocket connection is established, the only operation available on this connection is sending Websocket frames using gun:ws_send/2.

Gun will send a gun_ws message for every frame received.

Summary

The two following tables summarize the supported operations and the messages Gun sends depending on the connection’s current protocol.

Table 1. Supported operations per protocol
Operation HTTP/1.1 HTTP/2 Websocket

delete

yes

yes

no

get

yes

yes

no

head

yes

yes

no

options

yes

yes

no

patch

yes

yes

no

post

yes

yes

no

put

yes

yes

no

request

yes

yes

no

data

yes

yes

no

await

yes

yes

no

await_body

yes

yes

no

flush

yes

yes

no

cancel

yes

yes

no

ws_upgrade

yes

no

no

ws_send

no

no

yes
Table 2. Messages sent per protocol
Message HTTP/1.1 HTTP/2 Websocket

gun_push

no

yes

no

gun_inform

yes

yes

no

gun_response

yes

yes

no

gun_data

yes

yes

no

gun_trailers

yes

yes

no

gun_error

yes

yes

yes

gun_upgrade

yes

no

no

gun_ws

no

no

yes

Connection

This chapter describes how to open, monitor and close a connection using the Gun client.

Gun connections

Gun is designed with the HTTP/2 and Websocket protocols in mind. They are built for long-running connections that allow concurrent exchange of data, either in the form of request/responses for HTTP/2 or in the form of messages for Websocket.

A Gun connection is an Erlang process that manages a socket to a remote endpoint. This Gun connection is owned by a user process that is called the owner of the connection, and is managed by the supervision tree of the gun application.

The owner process communicates with the Gun connection by calling functions from the module gun. All functions perform their respective operations asynchronously. The Gun connection will send Erlang messages to the owner process whenever needed.

When the remote endpoint closes the connection, Gun attempts to reconnect automatically.

Opening a new connection

The gun:open/2,3 function must be used to open a connection.

Opening a connection to example.org on port 443

If the port given is 443, Gun will attempt to connect using TLS. The protocol will be selected automatically using the ALPN extension for TLS. By default Gun supports HTTP/2 and HTTP/1.1 when connecting using TLS.

For any other port, Gun will attempt to connect using plain TCP and will use the HTTP/1.1 protocol.

The transport and protocol used can be overriden via options. The manual documents all available options.

Options can be provided as a third argument, and take the form of a map.

Opening a TLS connection to example.org on port 8443

Waiting for the connection to be established

When Gun successfully connects to the server, it sends a gun_up message with the protocol that has been selected for the connection.

Gun provides the functions gun:await_up/1,2,3 that wait for the gun_up message. They can optionally take a monitor reference and/or timeout value. If no monitor is provided, one will be created for the duration of the function call.

Synchronous opening of a connection

Handling connection loss

When the connection is lost, Gun will send a gun_down message indicating the current protocol, the reason the connection was lost and two lists of stream references.

The first list indicates open streams that may have been processed by the server. The second list indicates open streams that the server did not process.

Monitoring the connection process

Because software errors are unavoidable, it is important to detect when the Gun process crashes. It is also important to detect when it exits normally. Erlang provides two ways to do that: links and monitors.

Gun leaves you the choice as to which one will be used. However, if you use the gun:await/2,3 or gun:await_body/2,3 functions, a monitor may be used for you to avoid getting stuck waiting for a message that will never come.

If you choose to monitor yourself you can do it on a permanent basis rather than on every message you will receive, saving resources. Indeed, the gun:await/3,4 and gun:await_body/3,4 functions both accept a monitor argument if you have one already.

Monitoring the connection process

This monitor reference can be kept and used until the connection process exits.

Handling DOWN messages

What to do when you receive a DOWN message is entirely up to you.

Closing the connection abruptly

The connection can be stopped abruptly at any time by calling the gun:close/1 function.

Immediate closing of the connection

The process is stopped immediately without having a chance to perform the protocol’s closing handshake, if any.

HTTP

This chapter describes how to use the Gun client for communicating with an HTTP/1.1 or HTTP/2 server.

Streams

Every time a request is initiated, Gun creates a stream. A stream reference uniquely identifies a set of request and response and must be used to perform additional operations with a stream or to identify its messages.

Stream references use the Erlang reference data type and are therefore unique.

Streams can be canceled at any time. This will stop any further messages from being sent to the owner process. Depending on its capabilities, the server will also be instructed to cancel the request.

Canceling a stream may result in Gun dropping the connection temporarily, to avoid uploading or downloading data that will not be used.

Cancelling a stream

Sending requests

Gun provides many convenient functions for performing common operations, like GET, POST or DELETE. It also provides a general purpose function in case you need other methods.

The availability of these methods on the server can vary depending on the software used but also on a per-resource basis.

Gun will automatically set a few headers depending on the method used. For all methods however it will set the host header if it has not been provided in the request arguments.

This section focuses on the act of sending a request. The handling of responses will be explained further on.

GET and HEAD

Use gun:get/2,3,4 to request a resource.

GET "/organizations/ninenines"
GET "/organizations/ninenines" with custom headers

Note that the list of headers has the field name as a binary. The field value is iodata, which is either a binary or an iolist.

Use gun:head/2,3,4 if you don’t need the response body.

HEAD "/organizations/ninenines"
HEAD "/organizations/ninenines" with custom headers

It is not possible to send a request body with a GET or HEAD request.

POST, PUT and PATCH

HTTP defines three methods to create or update a resource.

POST is generally used when the resource identifier (URI) isn’t known in advance when creating a resource. POST can also be used to replace an existing resource, although PUT is more appropriate in that situation.

PUT creates or replaces a resource identified by the URI.

PATCH provides instructions on how to modify the resource.

Both POST and PUT send the entire resource representation in their request body. The PATCH method can be used when this is not desirable. The request body of a PATCH method may be a partial representation or a list of instructions on how to update the resource.

The gun:post/4,5, gun:put/4,5 and gun:patch/4,5 functions take a body as their fourth argument. These functions do not require any body-specific header to be set, although it is always recommended to set the content-type header. Gun will set the other headers automatically.

In this and the following examples in this section, gun:post can be replaced by gun:put or gun:patch for performing a PUT or PATCH request, respectively.

POST "/organizations/ninenines"

The gun:post/3, gun:put/3 and gun:patch/3 functions do not take a body in their arguments. If a body is to be provided later on, using the gun:data/4 function, then the request headers must indicate this. This can be done by setting the content-length or content-type request headers. If these headers are not set then Gun will assume the request has no body.

It is recommended to send the content-length header if you know it in advance, although this is not required. If it is not set, HTTP/1.1 will use the chunked transfer-encoding, and HTTP/2 will continue normally as it is chunked by design.

POST "/organizations/ninenines" with delayed body

The atom fin indicates this is the last chunk of data to be sent. You can call the gun:data/4 function as many times as needed until you have sent the entire body. The last call must use fin and all the previous calls must use nofin. The last chunk may be empty.

Streaming the request body

DELETE

Use gun:delete/2,3,4 to delete a resource.

DELETE "/organizations/ninenines"
DELETE "/organizations/ninenines" with custom headers

OPTIONS

Use gun:options/2,3 to request information about a resource.

OPTIONS "/organizations/ninenines"
OPTIONS "/organizations/ninenines" with custom headers

You can also use this function to request information about the server itself.

OPTIONS "*"

Requests with an arbitrary method

The gun:request/4,5,6 function can be used to send requests with a configurable method name. It is mostly useful when you need a method that Gun does not understand natively.

Example of a TRACE request

Processing responses

All data received from the server is sent to the owner process as a message. First a gun_response message is sent, followed by zero or more gun_data messages. If something goes wrong, a gun_error message is sent instead.

The response message will inform you whether there will be data messages following. If it contains fin there will be no data messages. If it contains nofin then one or more data messages will follow.

When using HTTP/2 this value is sent with the frame and simply passed on in the message. When using HTTP/1.1 however Gun must guess whether data will follow by looking at the response headers.

You can receive messages directly, or you can use the await functions to let Gun receive them for you.

Receiving a response using receive

While it may seem verbose, using messages like this has the advantage of never locking your process, allowing you to easily debug your code. It also allows you to start more than one connection and concurrently perform queries on all of them at the same time.

You can also use Gun in a synchronous manner by using the await functions.

The gun:await/2,3,4 function will wait until it receives a response to, a pushed resource related to, or data from the given stream.

When calling gun:await/2,3 and not passing a monitor reference, one is automatically created for you for the duration of the call.

The gun:await_body/2,3,4 works similarly, but returns the body received. Both functions can be combined to receive the response and its body sequentially.

Receiving a response using await

Handling streams pushed by the server

The HTTP/2 protocol allows the server to push more than one resource for every request. It will start sending those extra resources before it starts sending the response itself, so Gun will send you gun_push messages before gun_response when that happens.

You can safely choose to ignore gun_push messages, or you can handle them. If you do, you can either receive the messages directly or use await functions.

The gun_push message contains both the new stream reference and the stream reference of the original request.

Receiving a pushed response using receive

If you use the gun:await/2,3,4 function, however, Gun will use the original reference to identify the message but will return a tuple that doesn’t contain it.

Receiving a pushed response using await

The PushedStreamRef variable can then be used with gun:await/2,3,4 and gun:await_body/2,3,4.

Flushing unwanted messages

Gun provides the function gun:flush/1 to quickly get rid of unwanted messages sitting in the process mailbox. You can use it to get rid of all messages related to a connection, or just the messages related to a stream.

Flush all messages from a Gun connection
Flush all messages from a specific stream

Redirecting responses to a different process

Gun allows you to specify which process will handle responses to a request via the reply_to request option.

GET "/organizations/ninenines" to a different process

Websocket

This chapter describes how to use the Gun client for communicating with a Websocket server.

HTTP upgrade

Websocket is a protocol built on top of HTTP. To use Websocket, you must first request for the connection to be upgraded. Only HTTP/1.1 connections can be upgraded to Websocket, so you might need to restrict the protocol to HTTP/1.1 if you are planning to use Websocket over TLS.

You must use the gun:ws_upgrade/2,3,4 function to upgrade to Websocket. This function can be called anytime after connection, so you can send HTTP requests before upgrading to Websocket.

Upgrade to Websocket

Gun will set all the necessary headers for performing the Websocket upgrade, but you can specify additional headers if needed. For example you can authenticate.

Upgrade to Websocket using HTTP authentication

You can pass the Websocket options as part of the gun:open/2,3 call when opening the connection, or using the gun:ws_upgrade/4. The fourth argument is those same options.

Gun can negotiate the protocol to be used for the Websocket connection. The protocols option can be given with a list of protocols accepted and the corresponding handler module. Note that the interface for handler modules is currently undocumented and must be set to gun_ws_h.

Upgrade to Websocket with protocol negotiation

The upgrade will fail if the server cannot satisfy the protocol negotiation.

When the upgrade succeeds, a gun_upgrade message is sent. If the server does not understand Websocket or refused the upgrade, a gun_response message is sent. If Gun couldn’t perform the upgrade due to an error (for example attempting to upgrade to Websocket on an HTTP/1.0 connection) then a gun_error message is sent.

When the server does not understand Websocket, it may send a meaningful response which should be processed. In the following example we however ignore it:

Sending data

Once the Websocket upgrade has completed successfully, you no longer have access to functions for performing requests. You can only send and receive Websocket messages.

Use gun:ws_send/2 to send messages to the server.

Send a text frame

Note that if you send a close frame, Gun will close the connection cleanly but will attempt to reconnect afterwards.

Receiving data

Gun sends an Erlang message to the owner process for every Websocket message it receives.

Additional information

Appendix A: Changes since Gun 1.3

The following patch versions were released since Gun 1.3:

Gun 1.3.3

This release updates Erlang.mk to include features useful to RabbitMQ.

It also updates Cowlib to 2.7.0.

Gun 1.3.2

This release backports a fix that will be included in the upcoming Gun 2.0 release:

  • Gun will now properly identify HTTP/2 preface errors, for example when attempting to connect to an HTTP/1.1 server or a non-HTTP server. Gun will give a different error message when it thinks it received an HTTP/1.1 response.

Gun 1.3.1

This release backports a fix that will be included in the upcoming Gun 2.0 release:

  • POTENTIAL SECURITY VULNERABILITY: Fix transfer-encoding precedence over content-length in responses. This bug may contribute to a response smuggling security vulnerability when Gun is used inside a proxy.

Appendix B: Migrating from Gun 1.2 to 1.3

Gun 1.3 improves the support for CONNECT requests introduced in the previous version and documents Websocket protocol negotiation.

Features added

  • The protocols CONNECT destination option has been added as a replacement for the now deprecated protocol option.

  • Add built-in support for Websocket protocol negotiation through the Websocket option protocols. The interface of the handler module currently remains undocumented and must be set to gun_ws_h.

  • Add the h2specd HTTP/2 test suite from the h2spec project.

Bugs fixed

  • Fix connecting to HTTP/2 over TLS origin servers via HTTP/1.1 CONNECT proxies.

  • Do not send the HTTP/1.1 keepalive while waiting for a response to a CONNECT request.

  • Do not crash on HTTP/2 HEADERS frames with the PRIORITY flag set.

  • Do not crash on HTTP/2 HEADERS frames when the END_HEADERS flag is not set.

  • Do not crash on unknown HTTP/2 frame types.

  • Reject HTTP/2 WINDOW_UPDATE frames when they would cause the window to overflow.

  • Send a GOAWAY frame on closing the HTTP/2 connection.

Appendix C: Migrating from Gun 1.1 to 1.2

Gun 1.2 adds support for the CONNECT request over HTTP/1.1 connections.

Features added

  • CONNECT requests can now be issued on HTTP/1.1 connections. The tunneled connection can use any of the protocols Gun supports: HTTP/1.1, HTTP/2 and Websocket over both TCP and TLS transports. Note that Gun currently does not support tunneling a TLS connection over a TLS connection due to limitations in Erlang/OTP.

  • Gun supports sending multiple CONNECT requests, allowing the tunnel to the origin server to go through multiple proxies.

  • Gun supports sending CONNECT requests with authorization credentials using the Basic authentication mechanism.

  • Update Cowlib to 2.6.0

Functions added

  • The functions gun:connect/2,3,4 have been added. They can be used to initiate CONNECT requests on HTTP/1.1 connections.

Appendix D: Migrating from Gun 1.0 to 1.1

Gun 1.1 updates the Cowlib dependency to 2.5.1 and fixes a few problems with experimental features.

Features added

  • Update Cowlib to 2.5.1

Bugs fixed

  • A bug in the experimental gun_sse_h where lone id lines were not propagated has been fixed by updating the Cowlib dependency.

  • The status code was incorrectly given to the experimental content handlers as a binary. It has been fixed an an integer is now given as was intended.

  • A number of Dialyzer warnings have been fixed.