Interface

Introduction

Gun is an HTTP client for Erlang/OTP.

Gun supports the HTTP/2, HTTP/1.1 and Websocket protocols.

Prerequisites

Knowledge of Erlang, but also of the HTTP/1.1, HTTP/2 and Websocket protocols is required in order to read this guide.

Supported platforms

Gun is tested and supported on Linux, FreeBSD, Windows and OSX.

Gun is developed for Erlang/OTP 22.0 and newer.

License

Gun uses the ISC License.

Copyright (c) 2013-2025, Loïc Hoguin <essen@ninenines.eu>

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

Versioning

Gun uses Semantic Versioning 2.0.0.

Conventions

In the HTTP protocol, the method name is case sensitive. All standard method names are uppercase.

Header names are case insensitive. Gun converts all the header names to lowercase, including request headers provided by your application.

Starting and stopping

This chapter describes how to start and stop the Gun application.

Setting up

Specify Gun as a dependency to your application in your favorite build tool.

With Erlang.mk this is done by adding gun to the DEPS variable in your Makefile.

Adding Gun as an Erlang.mk dependency

Starting

Gun is an OTP application. It needs to be started before you can use it.

Starting Gun in an Erlang shell

1> application:ensure_all_started(gun).
{ok,[crypto,cowlib,asn1,public_key,ssl,gun]}

Stopping

You can stop Gun using the application:stop/1 function, however only Gun will be stopped. This is the reverse of application:start/1. The application_ensure_all_started/1 function has no equivalent for stopping all applications.

Stopping Gun

application:stop(gun).

Supported protocols

This chapter describes the protocols supported and the operations available to them.

HTTP/1.1

HTTP/1.1 is a text request-response protocol. The client sends a request, the server sends back a response.

Gun provides convenience functions for performing GET, HEAD, OPTIONS, POST, PATCH, PUT, and DELETE requests. All these functions are aliases of gun:headers/4,5 or gun:request/5,6 for the respective methods. Gun also provides a gun:data/4 function for streaming the request body.

Gun will send a gun_inform message for every intermediate informational responses received. They will always be sent before the gun_response message.

Gun will send a gun_response message for every response received, followed by zero or more gun_data messages for the response body, which is optionally terminated by a gun_trailers message. If something goes wrong, a gun_error will be sent instead.

Gun provides convenience functions for dealing with messages. The gun:await/2,3,4 function waits for a response to the given request, and the gun:await_body/2,3,4 function for the response body. The gun:flush/1 function can be used to clear all messages related to a request or a connection from the mailbox of the calling process.

The function gun:cancel/2 can be used to silence the response to a request previously sent if it is no longer needed. When using HTTP/1.1 there is no multiplexing so Gun will have to receive the response fully before any other responses can be received.

Finally, Gun can upgrade an HTTP/1.1 connection to Websocket. It provides the gun:ws_upgrade/2,3,4 function for that purpose. A gun_upgrade message will be sent on success; a gun_response message otherwise.

HTTP/2

HTTP/2 is a binary protocol based on HTTP, compatible with the HTTP semantics, that reduces the complexity of parsing requests and responses, compresses the HTTP headers and allows the server to push additional resources along with the normal response to the original request.

The HTTP/2 interface is very similar to HTTP/1.1, so this section instead focuses on the differences in the interface for the two protocols.

Gun will send gun_push messages for every push received. They will always be sent before the gun_response message. They can be ignored safely if they are not needed, or they can be canceled.

The gun:cancel/2 function will use the HTTP/2 stream cancellation mechanism which allows Gun to inform the server to stop sending a response for this particular request, saving resources.

Note that because HTTP/2 headers are compressed, there are scenarios where it is possible to probe or extract data, creating security risks. One scenario being the use of Gun as a proxy to create a single connection to an origin, with requests coming from multiple mutually distrustful entities. Gun will provide configuration options to restrict headers that can be compressed in a future release.

Websocket

Websocket is a binary protocol built on top of HTTP that allows asynchronous concurrent communication between the client and the server. A Websocket server can push data to the client at any time.

Once the Websocket connection is established over an HTTP/1.1 connection, the only operation available on this connection is sending Websocket frames using gun:ws_send/3.

Gun will send a gun_ws message for every frame received.

It is recommended to disable automatic reconnect when Websocket is used because Gun cannot automatically upgrade to Websocket on reconnect, and so an undetected disconnect may lead to many error messages from Gun.

Summary

The two following tables summarize the supported operations and the messages Gun sends depending on the connection’s current protocol.

Table 1. Supported operations per protocol
Operation	HTTP/1.1	HTTP/2	Websocket
delete	yes	yes	no
get	yes	yes	no
head	yes	yes	no
options	yes	yes	no
patch	yes	yes	no
post	yes	yes	no
put	yes	yes	no
request	yes	yes	no
data	yes	yes	no
await	yes	yes	no
await_body	yes	yes	no
flush	yes	yes	no
cancel	yes	yes	no
ws_upgrade	yes	yes	no
ws_send	no	no	yes

Table 2. Messages sent per protocol
Message	HTTP/1.1	HTTP/2	Websocket
gun_push	no	yes	no
gun_inform	yes	yes	no
gun_response	yes	yes	no
gun_data	yes	yes	no
gun_trailers	yes	yes	no
gun_error	yes	yes	yes
gun_upgrade	yes	yes	no
gun_ws	no	no	yes

Connection

This chapter describes how to open, monitor and close a connection using the Gun client.

Gun connections

Gun is designed with the HTTP/2 and Websocket protocols in mind. They are built for long-running connections that allow concurrent exchange of data, either in the form of request/responses for HTTP/2 or in the form of messages for Websocket.

A Gun connection is an Erlang process that manages a socket to a remote endpoint. This Gun connection is owned by a user process that is called the owner of the connection, and is managed by the supervision tree of the gun application.

Any process can communicate with the Gun connection by calling functions from the module gun. All functions perform their respective operations asynchronously. The Gun connection will send Erlang messages to the calling process whenever needed.

When the remote endpoint closes the connection, Gun attempts to reconnect automatically.

Opening a new connection

The gun:open/2,3 function must be used to open a connection.

Opening a connection to example.org on port 443

{ok, ConnPid} = gun:open("example.org", 443).

If the port given is 443, Gun will attempt to connect using TLS. The protocol will be selected automatically using the ALPN extension for TLS. By default Gun supports HTTP/2 and HTTP/1.1 when connecting using TLS.

For any other port, Gun will attempt to connect using plain TCP and will use the HTTP/1.1 protocol.

The transport and protocol used can be overriden via options. The manual documents all available options.

Options can be provided as a third argument, and take the form of a map.

Opening a TLS connection to example.org on port 8443

{ok, ConnPid} = gun:open("example.org", 8443, #{transport => tls}).

When using TLS you may want to tweak the configuration for the ssl application, in particular the session_lifetime and session_cache_client_max to limit the amount of memory used for the TLS sessions cache.

Waiting for the connection to be established

When Gun successfully connects to the server, it sends a gun_up message with the protocol that has been selected for the connection.

Gun provides the functions gun:await_up/1,2,3 that wait for the gun_up message. They can optionally take a monitor reference and/or timeout value. If no monitor is provided, one will be created for the duration of the function call.

Synchronous opening of a connection

{ok, ConnPid} = gun:open("example.org", 443),
{ok, Protocol} = gun:await_up(ConnPid).

Handling connection loss

When the connection is lost, Gun will send a gun_down message indicating the current protocol, the reason the connection was lost and two lists of stream references.

The first list indicates open streams that may have been processed by the server. The second list indicates open streams that the server did not process.

Monitoring the connection process

Because software errors are unavoidable, it is important to detect when the Gun process crashes. It is also important to detect when it exits normally. Erlang provides two ways to do that: links and monitors.

Gun leaves you the choice as to which one will be used. However, if you use the gun:await/2,3 or gun:await_body/2,3 functions, a monitor may be used for you to avoid getting stuck waiting for a message that will never come.

If you choose to monitor yourself you can do it on a permanent basis rather than on every message you will receive, saving resources. Indeed, the gun:await/3,4 and gun:await_body/3,4 functions both accept a monitor argument if you have one already.

Monitoring the connection process

{ok, ConnPid} = gun:open("example.org", 443).
MRef = monitor(process, ConnPid).

This monitor reference can be kept and used until the connection process exits.

Handling DOWN messages

receive
    %% Receive Gun messages here...
    {'DOWN', Mref, process, ConnPid, Reason} ->
        error_logger:error_msg("Oops!"),
        exit(Reason)
end.

What to do when you receive a DOWN message is entirely up to you.

Closing the connection abruptly

The connection can be stopped abruptly at any time by calling the gun:close/1 function.

Immediate closing of the connection

gun:close(ConnPid).

The process is stopped immediately without having a chance to perform the protocol’s closing handshake, if any.

HTTP

This chapter describes how to use the Gun client for communicating with an HTTP/1.1 or HTTP/2 server.

Streams

Every time a request is initiated, Gun creates a stream. A stream reference uniquely identifies a set of request and response and must be used to perform additional operations with a stream or to identify its messages.

Stream references use the Erlang reference data type and are therefore unique.

Streams can be canceled at any time. This will stop any further messages from being sent to the calling process. Depending on its capabilities, the server will also be instructed to cancel the request.

Canceling a stream may result in Gun dropping the connection temporarily, to avoid uploading or downloading data that will not be used.

Cancelling a stream

gun:cancel(ConnPid, StreamRef).

Sending requests

Gun provides many convenient functions for performing common operations, like GET, POST or DELETE. It also provides a general purpose function in case you need other methods.

The availability of these methods on the server can vary depending on the software used but also on a per-resource basis.

Gun will automatically set a few headers depending on the method used. For all methods however it will set the host header if it has not been provided in the request arguments.

This section focuses on the act of sending a request. The handling of responses will be explained further on.

GET and HEAD

Use gun:get/2,3,4 to request a resource.

GET "/organizations/ninenines"

StreamRef = gun:get(ConnPid, "/organizations/ninenines").

GET "/organizations/ninenines" with custom headers

StreamRef = gun:get(ConnPid, "/organizations/ninenines", [
    {<<"accept">>, "application/json"},
    {<<"user-agent">>, "revolver/1.0"}
]).

Note that the list of headers has the field name as a binary. The field value is iodata, which is either a binary or an iolist.

Use gun:head/2,3,4 if you don’t need the response body.

HEAD "/organizations/ninenines"

StreamRef = gun:head(ConnPid, "/organizations/ninenines").

HEAD "/organizations/ninenines" with custom headers

StreamRef = gun:head(ConnPid, "/organizations/ninenines", [
    {<<"accept">>, "application/json"},
    {<<"user-agent">>, "revolver/1.0"}
]).

It is not possible to send a request body with a GET or HEAD request.

POST, PUT and PATCH

HTTP defines three methods to create or update a resource.

POST is generally used when the resource identifier (URI) isn’t known in advance when creating a resource. POST can also be used to replace an existing resource, although PUT is more appropriate in that situation.

PUT creates or replaces a resource identified by the URI.

PATCH provides instructions on how to modify the resource.

Both POST and PUT send the entire resource representation in their request body. The PATCH method can be used when this is not desirable. The request body of a PATCH method may be a partial representation or a list of instructions on how to update the resource.

The functions gun:post/4,5, gun:put/4,5 and gun:patch/4,5 take a body as their fourth argument. These functions do not require any body-specific header to be set, although it is always recommended to set the content-type header. Gun will set the other headers automatically.

In this and the following examples in this section, gun:post can be replaced by gun:put or gun:patch for performing a PUT or PATCH request, respectively.

POST "/organizations/ninenines"

Body = "{\"msg\": \"Hello world!\"}",
StreamRef = gun:post(ConnPid, "/organizations/ninenines", [
    {<<"content-type">>, "application/json"}
], Body).

The functions gun:post/3,4, gun:put/3,4 and gun:patch/3,4 do not take a body in their arguments: the body must be provided later on using the gun:data/4 function.

It is recommended to send the content-length header if you know it in advance, although this is not required. If it is not set, HTTP/1.1 will use the chunked transfer-encoding, and HTTP/2 will continue normally as it is chunked by design.

POST "/organizations/ninenines" with delayed body

Body = "{\"msg\": \"Hello world!\"}",
StreamRef = gun:post(ConnPid, "/organizations/ninenines", [
    {<<"content-length">>, integer_to_binary(length(Body))},
    {<<"content-type">>, "application/json"}
]),
gun:data(ConnPid, StreamRef, fin, Body).

The atom fin indicates this is the last chunk of data to be sent. You can call the gun:data/4 function as many times as needed until you have sent the entire body. The last call must use fin and all the previous calls must use nofin. The last chunk may be empty.

Streaming the request body

sendfile(ConnPid, StreamRef, Filepath) ->
    {ok, IoDevice} = file:open(Filepath, [read, binary, raw]),
    do_sendfile(ConnPid, StreamRef, IoDevice).

do_sendfile(ConnPid, StreamRef, IoDevice) ->
    case file:read(IoDevice, 8000) of
        eof ->
            gun:data(ConnPid, StreamRef, fin, <<>>),
            file:close(IoDevice);
        {ok, Bin} ->
            gun:data(ConnPid, StreamRef, nofin, Bin),
            do_sendfile(ConnPid, StreamRef, IoDevice)
    end.

DELETE

Use gun:delete/2,3,4 to delete a resource.

DELETE "/organizations/ninenines"

StreamRef = gun:delete(ConnPid, "/organizations/ninenines").

DELETE "/organizations/ninenines" with custom headers

StreamRef = gun:delete(ConnPid, "/organizations/ninenines", [
    {<<"user-agent">>, "revolver/1.0"}
]).

OPTIONS

Use gun:options/2,3 to request information about a resource.

OPTIONS "/organizations/ninenines"

StreamRef = gun:options(ConnPid, "/organizations/ninenines").

OPTIONS "/organizations/ninenines" with custom headers

StreamRef = gun:options(ConnPid, "/organizations/ninenines", [
    {<<"user-agent">>, "revolver/1.0"}
]).

You can also use this function to request information about the server itself.

OPTIONS "*"

StreamRef = gun:options(ConnPid, "*").

Requests with an arbitrary method

The functions gun:headers/4,5 or gun:request/5,6 can be used to send requests with a configurable method name. It is mostly useful when you need a method that Gun does not understand natively.

Example of a TRACE request

gun:request(ConnPid, "TRACE", "/", [
    {<<"max-forwards">>, "30"}
], <<>>).

Processing responses

All data received from the server is sent to the calling process as a message. First a gun_response message is sent, followed by zero or more gun_data messages. If something goes wrong, a gun_error message is sent instead.

The response message will inform you whether there will be data messages following. If it contains fin there will be no data messages. If it contains nofin then one or more data messages will follow.

When using HTTP/2 this value is sent with the frame and simply passed on in the message. When using HTTP/1.1 however Gun must guess whether data will follow by looking at the response headers.

You can receive messages directly, or you can use the await functions to let Gun receive them for you.

Receiving a response using receive

print_body(ConnPid, MRef) ->
    StreamRef = gun:get(ConnPid, "/"),
    receive
        {gun_response, ConnPid, StreamRef, fin, Status, Headers} ->
            no_data;
        {gun_response, ConnPid, StreamRef, nofin, Status, Headers} ->
            receive_data(ConnPid, MRef, StreamRef);
        {'DOWN', MRef, process, ConnPid, Reason} ->
            error_logger:error_msg("Oops!"),
            exit(Reason)
    after 1000 ->
        exit(timeout)
    end.

receive_data(ConnPid, MRef, StreamRef) ->
    receive
        {gun_data, ConnPid, StreamRef, nofin, Data} ->
            io:format("~s~n", [Data]),
            receive_data(ConnPid, MRef, StreamRef);
        {gun_data, ConnPid, StreamRef, fin, Data} ->
            io:format("~s~n", [Data]);
        {'DOWN', MRef, process, ConnPid, Reason} ->
            error_logger:error_msg("Oops!"),
            exit(Reason)
    after 1000 ->
        exit(timeout)
    end.

While it may seem verbose, using messages like this has the advantage of never locking your process, allowing you to easily debug your code. It also allows you to start more than one connection and concurrently perform queries on all of them at the same time.

You can also use Gun in a synchronous manner by using the await functions.

The gun:await/2,3,4 function will wait until it receives a response to, a pushed resource related to, or data from the given stream.

When calling gun:await/2,3 and not passing a monitor reference, one is automatically created for you for the duration of the call.

The gun:await_body/2,3,4 works similarly, but returns the body received. Both functions can be combined to receive the response and its body sequentially.

Receiving a response using await

StreamRef = gun:get(ConnPid, "/"),
case gun:await(ConnPid, StreamRef) of
    {response, fin, Status, Headers} ->
        no_data;
    {response, nofin, Status, Headers} ->
        {ok, Body} = gun:await_body(ConnPid, StreamRef),
        io:format("~s~n", [Body])
end.

Handling streams pushed by the server

The HTTP/2 protocol allows the server to push more than one resource for every request. It will start sending those extra resources before it starts sending the response itself, so Gun will send you gun_push messages before gun_response when that happens.

You can safely choose to ignore gun_push messages, or you can handle them. If you do, you can either receive the messages directly or use await functions.

The gun_push message contains both the new stream reference and the stream reference of the original request.

Receiving a pushed response using receive

receive
    {gun_push, ConnPid, OriginalStreamRef, PushedStreamRef,
            Method, Host, Path, Headers} ->
        enjoy()
end.

If you use the gun:await/2,3,4 function, however, Gun will use the original reference to identify the message but will return a tuple that doesn’t contain it.

Receiving a pushed response using await

{push, PushedStreamRef, Method, URI, Headers}
    = gun:await(ConnPid, OriginalStreamRef).

The PushedStreamRef variable can then be used with gun:await/2,3,4 and gun:await_body/2,3,4.

Flushing unwanted messages

Gun provides the function gun:flush/1 to quickly get rid of unwanted messages sitting in the process mailbox. You can use it to get rid of all messages related to a connection, or just the messages related to a stream.

Flush all messages from a Gun connection

gun:flush(ConnPid).

Flush all messages from a specific stream

gun:flush(StreamRef).

Redirecting responses to a different process

Gun allows you to specify which process will handle responses to a request via the reply_to request option.

GET "/organizations/ninenines" to a different process

StreamRef = gun:get(ConnPid, "/organizations/ninenines", [],
    #{reply_to => Pid}).

Websocket

This chapter describes how to use the Gun client for communicating with a Websocket server.

HTTP upgrade

Websocket is a protocol built on top of HTTP. To use Websocket, you must first request for the connection to be upgraded. Only HTTP/1.1 connections can be upgraded to Websocket, so you might need to restrict the protocol to HTTP/1.1 if you are planning to use Websocket over TLS.

You must use the gun:ws_upgrade/2,3,4 function to upgrade to Websocket. This function can be called anytime after connection, so you can send HTTP requests before upgrading to Websocket.

Upgrade to Websocket

gun:ws_upgrade(ConnPid, "/websocket").

Gun will set all the necessary headers for performing the Websocket upgrade, but you can specify additional headers if needed. For example you can authenticate.

Upgrade to Websocket using HTTP authentication

gun:ws_upgrade(ConnPid, "/websocket", [
    {<<"authorization">>, "Basic dXNlcm5hbWU6cGFzc3dvcmQ="}
]).

You can pass the Websocket options as part of the gun:open/2,3 call when opening the connection, or using the gun:ws_upgrade/4. The fourth argument is those same options.

Gun can negotiate the protocol to be used for the Websocket connection. The protocols option can be given with a list of protocols accepted and the corresponding handler module. Note that the interface for handler modules is currently undocumented and must be set to gun_ws_h.

Upgrade to Websocket with protocol negotiation

StreamRef = gun:ws_upgrade(ConnPid, "/websocket", []
    #{protocols => [{<<"xmpp">>, gun_ws_h}]}).

The upgrade will fail if the server cannot satisfy the protocol negotiation.

When the upgrade succeeds, a gun_upgrade message is sent. If the server does not understand Websocket or refused the upgrade, a gun_response message is sent. If Gun couldn’t perform the upgrade due to an error (for example attempting to upgrade to Websocket on an HTTP/1.0 connection) then a gun_error message is sent.

When the server does not understand Websocket, it may send a meaningful response which should be processed. In the following example we however ignore it:

receive
    {gun_upgrade, ConnPid, StreamRef, [<<"websocket">>], Headers} ->
        upgrade_success(ConnPid, StreamRef);
    {gun_response, ConnPid, _, _, Status, Headers} ->
        exit({ws_upgrade_failed, Status, Headers});
    {gun_error, ConnPid, StreamRef, Reason} ->
        exit({ws_upgrade_failed, Reason})
    %% More clauses here as needed.
after 1000 ->
    exit(timeout)
end.

Sending data

Once the Websocket upgrade has completed successfully, you no longer have access to functions for performing requests. You can only send and receive Websocket messages.

Use gun:ws_send/3 to send messages to the server.

Send a text frame

gun:ws_send(ConnPid, StreamRef, {text, "Hello!"}).

Send a text frame, a binary frame and then close the connection

gun:ws_send(ConnPid, StreamRef, [
    {text, "Hello!"},
    {binary, BinaryValue},
    close
]).

Note that if you send a close frame, Gun will close the connection cleanly but may attempt to reconnect afterwards depending on the retry configuration.

Receiving data

Gun sends an Erlang message to the owner process for every Websocket message it receives.

receive
    {gun_ws, ConnPid, StreamRef, Frame} ->
        handle_frame(ConnPid, StreamRef, Frame)
end.

Automatic reconnect gotchas

This can be done by setting the retry option to 0 when opening a connection:

{ok, ConnPid} = gun:open("localhost", 12345, #{
    retry ⇒ 0
}).

Advanced

Internals: TLS over TLS

The ssl application that comes with Erlang/OTP implements TLS using an interface equivalent to the gen_tcp interface: you get and manipulate a socket. The TLS encoding and decoding is applied transparently to the data sent and received.

In order to have a TLS layer inside another TLS layer we need a way to encode the data of the inner layer before we pass it to the outer layer. We cannot do this with a socket interface. Thankfully, the ssl application comes with options that allow to transform an sslsocket() into an encoder/decoder.

The implementation is however a little convoluted as a result. This chapter aims to give an overview of how it all works under the hood.

gun_tls_proxy

The module gun_tls_proxy implements an intermediary process that sits between the Gun process and the TLS process. It is responsible for routing data from the Gun process to the TLS process, and from the TLS process to the Gun process.

In order to obtain the TLS encoded data the cb_info option is given to the ssl:connect/3 function. This replaces the default TCP outer socket module with our own custom module. Gun uses the gun_tls_proxy_cb module instead. This module will forward all messages to the gun_tls_proxy process.

The resulting operations looks like this:

Gun process <-> gun_tls_proxy <-> sslsocket() <-> gun_tls_proxy <-> "inner socket"

The "inner socket" is the socket for the Gun connection. The gun_tls_proxy process decides where to send or receive the data based on where it’s coming from. This is how it knows whether the data has been encoded/decoded or not.

Because the ssl:connect/3 function call is blocking, a temporary process is used while connecting. This is required because the gun_tls_proxy needs to forward data even while performing the TLS handshake, otherwise the ssl:connect/3 call will not complete.

The result of the ssl:connect/3 call is forward to the Gun process, along with the negotiated protocols when the connection was successful.

The gun_tls_proxy_cb module does not actually implement {active,N} as requested by the ssl application. Instead it uses {active,true}.

The behavior of the gun_tls_proxy process will vary depending on whether the TLS over TLS is done connection-wide or only stream-wide.

Connection-wide TLS over TLS

When used for the entire connection, the gun_tls_proxy process will act like a real socket once connected. The only difference is how the connection is performed. As mentioned earlier, the result of the ssl:connect/3 call is sent back to the Gun process.

When doing TLS over TLS the processes will end up looking like this:

Gun process <-> gun_tls_proxy <-> "inner socket"

The details of the interactions between gun_tls_proxy and its associated sslsocket() have been removed in order to better illustrate the concept.

When adding another layer this becomes:

Gun process <-> gun_tls_proxy <-> gun_tls_proxy <-> sslsocket()

This is what is done when only HTTP/1.1 and SOCKS proxies are involved.

Stream-wide TLS over TLS

The same cannot be done for HTTP/2 proxies. This is because the HTTP/2 CONNECT method does not apply to the entire connection, but only to a stream. The proxied data must be wrapped inside a DATA frame. It cannot be sent directly. This is what must be done:

Gun process -> gun_tls_proxy -> Gun process -> "inner socket"

The "inner socket" is the socket for the HTTP/2 connection.

In order to tell Gun to continue processing the data, the handle_continue mechanism is introduced. When gun_tls_proxy has TLS encoded the data it sends it back to the Gun process, wrapped in a handle_continue tuple. This tuple contains enough information to figure out what stream the data belongs to and what should be done next. Gun will therefore route the data to the correct layer and continue sending it.

This solution is also used for receiving data, except in the reverse order.

Routing to the right stream

In order to know where to route the data, the stream_ref had to be modified to contain all the references to the individual streams. So if the tunnel is identified by StreamA and a request on this tunnel is identified by StreamB, then the stream is known as [StreamA, StreamB] to the user. Gun then routes first to StreamA, a tunnel, which continues routing to StreamB.

A problem comes up if an intermediary is a SOCKS server, for example in the following scenario:

HTTP/2 proxy <-> SOCKS proxy <-> HTTP/1.1 origin

The SOCKS protocol doesn’t have a concept of stream, therefore when we refer to request to the origin server they are [StreamA, StreamB], not [StreamA, StreamB, StreamC]. This is a problem for routing encoded/decoded TLS data to the SOCKS layer: we don’t have a built-in way of referring to the SOCKS layer.

The solution is to have a separate handle_continue_stream_ref value that assigns a reference to the SOCKS layers. Gun will then be able to forward handle_continue message, and only them, to the appropriate layer.

Gun therefore has two different routing avenues, but the mechanism remains the same otherwise.

gun_tunnel

In order to simplify the routing, the gun_tunnel module was introduced. For each intermediary (including the original CONNECT stream at the HTTP/2 layer) there is a gun_tunnel module.

Going back to the example above:

HTTP/2 proxy <-> SOCKS proxy <-> HTTP/1.1 origin

In this case the modules involved to handle the request will be as follow:

gun_http2 <-> gun_tunnel <-> gun_tunnel <-> gun_http

The gun_http2 module doesn’t do any routing, it just passes everything to the gun_tunnel module. The gun_tunnel module will then do the routing, which involves removing StreamA from [StreamA, StreamB] where appropriate.

The gun_tunnel module also receives the TLS encoded/decoded data and forwards it appropriately. When it comes to sending data, it will return a send command that allows the previous module to continue sending the data. The gun_http2 module will ultimately wrap the data to be sent in a DATA frame and send it to the "inner socket".

Additional information

Appendix A: Migrating from Gun 2.1 to 2.2

Gun 2.2 contains many features and fixes, including an experimental HTTP/3 implementation.

Gun 2.2 requires Erlang/OTP 24.0 or greater.

Features added

Gun will now do wildcard certificate matching by default, as required by the HTTP protocol, by setting the customize_hostname_check ssl option.
User pings are now supported for HTTP/2. User pings are PING frames sent by the user, with the corresponding PING_ACK resulting in a gun_notify message being sent to the user process. They can be used to measure latency or do other checks.
The reply_to request option has been extended to accept a fun or an MFA that will be called when a reply must be sent to the user process.
The state_name, event_handler and event_handler_state fields were added to gun:info/1.
Update Cowlib to 2.15.0.

Experimental features added

Experimental support for HTTP/3 has been added. Websocket over HTTP/3 is not currently implemented. HTTP/3 support is disabled by default; to enable, the environment variable GUN_QUICER must be set at compile-time.

Bugs fixed

TLS 1.3 performs certificate validation independently from the handshake, which means that certificate errors can arrive after the handshake has completed. This is in part to support post-handshake authentication. Gun will now replace {error, closed} and similar socket errors with the TLS alert when possible, including for TLS over TLS and TLS over HTTP/2 scenarios.
HTTP/2 tunneling has been improved: WINDOW_UPDATE frames are now properly sent, and flow control is handled. In addition closing the stream is better handled. This impacts both proxying scenarios as well as Websocket over HTTP/2.
HTTP/2 will now properly send a NO_ERROR "error" code when initiating graceful shutdown.
HTTP/2 will now properly track the number of streams currently running and immediately fail when the user tries to open a stream too many compared to what the server allows.
Sometimes stray HTTP/2 timeout messages could be received, producing a log message. This was because they were not cleaned up on disconnect. Now they are.
Socket errors were not properly handled when sending Websocket PONG frames. This has been corrected.
Trying to send a Websocket frame over an HTTP connection (before the Websocket upgrade) will now result in an error sent to the user process.
When connecting to HTTP via a Unix domain socket, Gun will now set the host header to "localhost" by default. Previously an invalid host header was sent.
Data could be lost when switching to the raw protocol. This has been corrected.
Documentation has been updated to describe the notify_settings_changed option; as well as to recommend disabling automatic reconnect when Websocket is used; and to provide context around potential security risks due to HTTP/2 compressed headers.

Appendix B: Migrating from Gun 2.0 to 2.1

Gun 2.1 contains a small security improvement for the HTTP/2 protocol, as well as includes a small number of fixes and improvements.

Gun 2.1 requires Erlang/OTP 22.0 or greater.

Features added

A new HTTP/2 option max_fragmented_header_block_size has been added to limit the size of header blocks that are sent over multiple HEADERS and CONTINUATION frames.
Update Cowlib to 2.13.0.

Bugs fixed

Gun will no longer configure the NPN TLS extension, which has long been replaced by ALPN. NPN is not compatible with TLS 1.3.
Gun will no longer crash when TLS connections close very early in the connection’s life time.

Appendix C: Migrating from Gun 1.3 to 2.0

Gun 2.0 includes state of the art tunnel support. With Gun 2.0 it is possible to make requests or data go through any number of proxy endpoints using any combination of TCP or TLS transports and HTTP/1.1, HTTP/2 or SOCKS5 protocols. All combinations of the scenario Proxy1 → Proxy2 → Origin are tested and known to work.

Gun 2.0 adds many more features such as Websocket over HTTP/2, a built-in cookie store, graceful shutdown, flow control for data messages, event handlers and more.

Gun 2.0 also introduces an experimental pool module that automatically maintains connections and routes requests to the right process, in a similar way as browsers do.

Gun 2.0 greatly improves the HTTP/2 performance when it comes to receiving large response bodies; and when receiving response bodies from many separate requests concurrently.

Gun now shares much of its HTTP/2 code with Cowboy, including the HTTP/2 state machine. Numerous issues were fixed as a result because the Cowboy implementation was much more advanced.

The Gun connection process is now implemented using gen_statem.

Gun 2.0 requires Erlang/OTP 22.0 or greater.

Features added

Cookie store support has been added. The cookie_store option allows configuring the cookie store backend. The gun_cookies module provides functions to help implementing such a backend. Gun comes with the backend gun_cookies_list which provides a per-connection, non-persistent cookie store. The cookie store engine implements the entire RFC6265bis draft algorithms except the parts about non-HTTP cookies as no such interface is provided; and the parts about SameSite as Gun has no concept of "browsing context".
Graceful shutdown has been implemented. Graceful shutdown can be initiated on the client side by calling the new function gun:shutdown/1 or when the owner process goes away; or on the peer side via the connection: close HTTP/1.1 header, the HTTP/2 GOAWAY frame or the Websocket close frame. Gun will try to complete existing streams when possible; other streams get canceled immediately. The closing_timeout option controls how long we are willing to wait at most before closing the connection.
Gun will better detect connection failures by checking the return value from sending data to the socket. This applies to all supported protocols. In addition, Gun now enables send_timeout_close with a send_timeout value defaulting to 15s.
Flow control has been added. It allows limiting the number of data/Websocket messages Gun sends to the calling process. Gun will stop reading from the socket or stop updating the protocol’s flow control window when applicable as well, to apply some backpressure to the remote endpoint(s). It is disabled by default and can be applied on a per-request basis if necessary.
An event handler interface has been added providing access to many internal Gun events. This can be used for a variety of purposes including logging, tracing or otherwise instrumenting a Gun connection.
In order to get separate events when connecting, the domain lookup, connection and TLS handshakes are now performed separately by Gun. As a result, there exists three separate timeout options for each of the steps, and the transport options had to be split into tcp_opts and tls_opts.
Gun now supports connecting through SOCKS proxies, including secure SOCKS proxies. Both unauthenticated and username/password authentication are supported.
Gun can connect through any number of HTTP, HTTPS, SOCKS or secure SOCKS proxies, including SOCKS proxies located after HTTP(S) proxies. The ultimate endpoint may be using any protocol, including plain TCP, TLS, HTTP/1.1 or HTTP/2.
When specifying which protocols to use, options can now be provided specific to those protocols. It is now possible to have separate HTTP options for an HTTP proxy and the origin HTTP server, for example. See the new gun:protocols() type for details.
Gun can now be used to send and receive raw data, as if it was just a normal socket. This can be useful when needing to connect through a number of HTTP/Socks proxies, allowing the use of Gun’s great proxying capabilities (including TLS over TLS) for any sort of protocols. This can also be useful when performing HTTP/1.1 Upgrade to custom protocols.
Headers can now be provided as a map.
Header names may now be provided as binary, string or atom.
Gun now automatically lowercases provided header names.
Many HTTP/2 options have been added, allowing great control over how Gun and the remote endpoint are using the HTTP/2 connection. They can be used to improve performance or lower the memory usage, for example.
A new keepalive_tolerance option for HTTP/2 enables closing the connection automatically when ping acks are not received in a timely manner. It nicely complements the keepalive option that makes Gun send pings.
Gun now supports Websocket subprotocol negotiation and the feature is fully documented and tested. This can be used to create handlers that will implement a protocol from within the Gun process itself. The negotiation is enabled by setting the protocols setting. The default_protocol and user_opts settings are also useful.
It is now possible to send many Websocket frames in a single gun:ws_send/3 call.
Gun may now send Websocket ping frames automatically at intervals determined by the keepalive option. It is disabled by default.
A new silence_pings option can be set to false to receive all ping and pong frames when using Websocket. They are typically not needed and therefore silent by default.
The reply_to option has been added to gun:ws_upgrade/4. The option applies to both the response and subsequent Websocket frames.
The reply_to option is also propagated to messages following a CONNECT request when the protocol requested is not HTTP.
A new option retry_fun can be used to implement different backoff strategies when reconnecting.
A new option supervise can be used to start a Gun connection without using Gun’s supervisor. It defaults to true.
Many improvements have been done to postpone or reject requests and other operations while in the wrong state (for example during state transitions when switching protocols or connecting to proxies).
Update Cowlib to 2.12.0.

Experimental features added

The gun_pool module was introduced. Its interface is very similar to the gun module, but as it is an experimental feature, it has not been documented yet. The intent is to obtain feedback and document it in an upcoming minor release. Pools are created for each authority (host/port) and scope (user-defined value) pairs and are resolved accordingly using the information provided in the request and request options. Connections may concurrently handle multiple requests/responses from as many different processes as required.

Features removed

Gun used to reject operations by processes that were not the owner of the connection. This behavior has been removed. In general the caller of a request or other operation will receive the relevant messages unless the reply_to option is used.
The connect_destination() option protocol has been removed. It was previously deprecated in favor of protocols.
The keepalive timeout is now disabled by default for HTTP/1.1 and HTTP/2. To be perfectly clear, this is unrelated to the HTTP/1.1 keep-alive mechanism.

Functions added

The function gun:set_owner/2 has been added. It allows changing the owner of a connection process. Only the current owner can do this operation.
The function gun:shutdown/1 has been added. It initiates the graceful shutdown of the connection, followed by the termination of the Gun process.
The function gun:stream_info/2 has been added. It provides information about a specific HTTP stream.

Functions modified

The function gun:info/1 now returns the owner of the connection as well as the cookie store.
The functions gun:await/2,3,4, gun:await_body/2,3,4 and gun:await_up/1,2,3 now distinguish the error types. They can be a timeout, a connection error, a stream error or a down error (when the Gun process exited while waiting).
The functions gun:await/2,3,4 will now receive upgrades, tunnel up and Websocket messages and return them.
Requests may now include the tunnel option to send the request on a specific tunnel.
The functions gun:request/4,5,6 have been replaced with gun:headers/4,5 and gun:request/5,6. This provides a cleaner separation between requests that are followed by a body and those that aren’t.
The function gun:ws_send/2 has been replaced with the function gun:ws_send/3. The stream reference for the corresponding Websocket upgrade request must now be given.

Messages added

The gun_tunnel_up message has been added.

Messages modified

The gun_down message no longer has its final element documented as UnprocessedStreams. It never worked and was always an empty list.

Bugs fixed

POTENTIAL SECURITY VULNERABILITY: Fix transfer-encoding precedence over content-length in responses. This bug may contribute to a response smuggling security vulnerability when Gun is used inside a proxy.
Gun will now better detect connection closes in some cases.
Gun will no longer send duplicate connection-wide gun_error messages to the same process.
Gun no longer crashes when trying to upgrade to Websocket over a connection restricted to HTTP/1.0.
The default value for the preferred protocols when using CONNECT over TLS has been corrected. It was mistakenly not enabling HTTP/2.
Protocol options provided for a tunnel destination were sometimes ignored. This should no longer be the case.
Gun will no longer send an empty HTTP/2 DATA frame when there is no request body. It was not necessary.
Gun will no longer error out when the owner process exits. The error reason will now be a shutdown tuple instead.
The host header was set incorrectly during Websocket upgrades when the host was configured with an IP address, resulting in a crash. This has been corrected.
A completed stream could be found in the gun_down message when the response contained a connection: close header. This is no longer the case.
Hostnames can now be provided as atom as stated by the documentation.
Gun will no longer attempt to send empty data chunks. When using HTTP/1.1 chunked transfer-encoding this caused the request body to end, even when nofin was given.
Gun now always retries connecting immediately when the connection goes down.
The default port number for the HTTP and HTTPS schemes is no longer sent in the host header.
An invalid stream reference was sent on failed Websocket upgrade responses. This has been corrected.
HTTP/2 connection preface errors are now properly detected and propagated in the gun_down message to the connection owner as well as the exit reason of the Gun process.
HTTP/2 connection preface errors now provide a different human readable error when the data received looks like an HTTP/1.x response.
HTTP/2 connection errors were missing the human readable reason in the gun_error message. This has been corrected.
Fix the host and :authority (pseudo-)headers when connecting to an IPv6 address given as a tuple. They were lacking the surrounding brackets.
Fix a crash in gun:info/1 when the socket was closed before we call Transport:sockname/1.
Fix flushing by stream reference. When the gun_inform message was flushed the function would switch to flushing all messages from the pid instead of only messages from the given stream.
Allow setting a custom SNI value.
Fix double sending of last chunk in HTTP/1.1 when Gun is asked to send empty data before closing the stream.
Gun will now properly ignore parameters when the media type is text/event-stream.
Avoid noisy crashes in the TLS over TLS code.
Gun will now include the StreamRef of Websocket streams when sending gun_down messages.
Gun will no longer reject HTTP proxies that use HTTP/1.0 for the version in their response.

Appendix D: Migrating from Gun 1.2 to 1.3

Gun 1.3 improves the support for CONNECT requests introduced in the previous version and documents Websocket protocol negotiation.

Features added

The protocols CONNECT destination option has been added as a replacement for the now deprecated protocol option.
Add built-in support for Websocket protocol negotiation through the Websocket option protocols. The interface of the handler module currently remains undocumented and must be set to gun_ws_h.
Add the h2specd HTTP/2 test suite from the h2spec project.

Bugs fixed

Fix connecting to HTTP/2 over TLS origin servers via HTTP/1.1 CONNECT proxies.
Do not send the HTTP/1.1 keepalive while waiting for a response to a CONNECT request.
Do not crash on HTTP/2 HEADERS frames with the PRIORITY flag set.
Do not crash on HTTP/2 HEADERS frames when the END_HEADERS flag is not set.
Do not crash on unknown HTTP/2 frame types.
Reject HTTP/2 WINDOW_UPDATE frames when they would cause the window to overflow.
Send a GOAWAY frame on closing the HTTP/2 connection.

Appendix E: Migrating from Gun 1.1 to 1.2

Gun 1.2 adds support for the CONNECT request over HTTP/1.1 connections.

Features added

CONNECT requests can now be issued on HTTP/1.1 connections. The tunneled connection can use any of the protocols Gun supports: HTTP/1.1, HTTP/2 and Websocket over both TCP and TLS transports. Note that Gun currently does not support tunneling a TLS connection over a TLS connection due to limitations in Erlang/OTP.
Gun supports sending multiple CONNECT requests, allowing the tunnel to the origin server to go through multiple proxies.
Gun supports sending CONNECT requests with authorization credentials using the Basic authentication mechanism.
Update Cowlib to 2.6.0

Functions added

The functions gun:connect/2,3,4 have been added. They can be used to initiate CONNECT requests on HTTP/1.1 connections.

Appendix F: Migrating from Gun 1.0 to 1.1

Gun 1.1 updates the Cowlib dependency to 2.5.1 and fixes a few problems with experimental features.

Features added

Update Cowlib to 2.5.1

Bugs fixed

A bug in the experimental gun_sse_h where lone id lines were not propagated has been fixed by updating the Cowlib dependency.
The status code was incorrectly given to the experimental content handlers as a binary. It has been fixed an an integer is now given as was intended.
A number of Dialyzer warnings have been fixed.