Asynchronous Messaging Protocol

AMP  is an RPC protocol for sending multiple asynchronous request/response pairs over the same connection. Requests and responses are both collections of key/value pairs. Keys are limited to 255 bytes in length, and values to 65,535 bytes (Why?). Any number of key/value pairs may be encoded in a single AMP  packet.

AMP  enables a rich set of Internet applications, ranging from "traditional" client-server APIs, to highly custom and efficient RPC protocols, to distributed peer-to-peer messaging topologies - or you may implement more advanced patterns such as a "spoke" topology where a central hub/broker receives and routes messages to and from connected peers.

Once connected, the AMP  protocol is symmetric, allowing either client or server to initiate an asynchronous request, provided that the other side of the connection implements a handler for it.

For values, AMP  defines a number of Standard Data Types, while keys are always strings.

AMP  was originally developed by Glyph Lefkowitz and appeared in Twisted, a Python networking framework - however AMP  is not Python or Twisted-specific.

Implementation are available for a wide variety of languages including Python 2.x/3.x, C, C#, Java, PHP and many more! See the Implementations Page for details.

Wire-Level Format

AMP  uses a very simple and logical wire-level format, making it easy to implement in your language of choice.

Lets take a hypothetical Sum command which accepts two arguments, a and b, and returns a+b as the response argument named total.

Sample key/values of a request to the Sum command:

_ask

42

_command

Sum

a

13

b

81

Key/values of the response:

_answer

42

total

94

That's all there is to it for a successful invocation of the Sum command!
(We'll talk about error handling later.)

The special _ask key is a unique-per-connection identifier, which identifies individual requests on a particular connection (e.g. a TCP socket).

Its value is used with the special _answer key (in response packets) to indicate the original request that is being answered.

And obviously the special _command key indicates which command-handler should be invoked to process the request.

AMP  is asynchronous, meaning that requests and responses may be interleaved on the wire at will. You may happily fire off several requests before receiving any responses. And responses may be returned in any order. This is often quite useful, though you may wish to enforce certain ordering for your AMP  calls depending on the requirements of your application; AMP  does not preclude you from doing this. Many other RPC protocols enforce certain ordering, and do not fully support asynchronous interleaving of requests and responses on the wire.

AMP  is full-duplex (or symmetric), meaning that requests may be initiated from either side of the connection - provided that the peer implements a handler for it. Each side of the connection may choose to (and often will) implement a different set of Command handlers, or none at all.

Sample Code

Example "Sum" command is here. You will need Python (2.x or 3.x) and Twisted.

Or check out a C# example.

The Actual Bytes

Keys and values in AMP  are length-prefixed. Keys and values both use a 16-bit (2 byte) prefix value, however, key length is restricted to 8 bits (1 byte). This means keys have a maximum length of 255 bytes, and values a maximum length of 65,535 bytes. (Explain?).

It also means that an AMP  conversation always begins with a NULL byte. This has significant advantages, as it means that you can, for instance, operate an HTTP server and an AMP  server on the exact same port 80. Since an HTTP client always begins the conversation with a non-NULL byte, it means you may easily distinguish between an HTTP client and an AMP  client.

Prefix values are in Network Byte Order (big-endian).

So, the request above (a "packet" or "box", in AMP  terms) looks like this on the wire:

0x00 0x04

A key name of length 4 follows...

0x5F 0x61 0x73 0x6B

The utf-8 string "_ask"

0x00 0x02

A value of length 2 follows...

0x32 0x33

The utf-8 string "23"

0x00 0x08

A key name of length 8 follows...

0x5F 0x63 0x6F 0x6D 0x6D 0x61 0x6E 0x64

The utf-8 string "_command"

0x00 0x03

A value of length 3 follows...

0x53 0x75 0x6D

The utf-8 string "Sum"

0x00 0x01

A key name of length 1 follows...

0x61

The utf-8 string "a"

0x00 0x02

A value of length 2 follows...

0x31 0x33

The utf-8 string "13"

0x00 0x01

A key name of length 1 follows...

0x62

The utf-8 string "b"

0x00 0x02

A value of length 2 follows...

0x38 0x31

The utf-8 string "81"

0x00 0x00

A key of length 0. THIS MARKS THE END OF THE BOX/PACKET.

Simple stuff, eh? Key names are UTF-8 encoded strings. Technically you may use any arbitrary byte sequence as a key name, but there is no practical reason for ever doing this - stick to UTF8-decodable strings. Also, some implementations of AMP  may not work correctly with key names that are not UTF8.

The integer arguments, a and b, are also encoded as UTF-8 strings. Many common AMP  types are encoded as the Python repr() of the object being encoded. This is handy for ease-of-implemetation and debugging your implementations with a packet-capture tool; many other languages have compatible "to string" functions/methods making AMP  a good choice for implementing in a new language.

The response is encoded like so:

0x00 0x07

A key name of length 7 follows...

0x5F 0x61 0x6E 0x73 0x77 0x65 0x72

The utf-8 string "_answer"

0x00 0x02

A value of length 2 follows...

0x32 0x33

The utf-8 string "23"

0x00 0x05

A key name of length 5 follows...

0x74 0x6F 0x74 0x61 0x6C

The utf-8 string "total"

0x00 0x02

A value of length 2 follows...

0x39 0x34

The utf-8 string "94"

0x00 0x00

A key of length 0. THIS MARKS THE END OF THE BOX/PACKET.

Fire and Forget Commands

With AMP  it is possible to fire off a request for which no result is generated, or for which you simply don't care what the result will be.

You do this by omitting the _ask key. The only request key that is absolutely required by the wire-protocol is _command.

Error Handling

When something goes wrong making an AMP  call the response will be quite different than the response seen above. Instead of the _answer key, there is an _error key instead.

For example, the first thing you can do wrong is to make a request for an AMP  command which doesn't exist:

_ask

1

_command

GetSecretFile

path

/etc/shadow

Presumably the other side won't know anything about a GetSecretFile command, so you get this:

_error

1

_error_code

UNHANDLED

_error_description

Unhandled Command: 'GetSecretFile'

The string "UNHANDLED" is a special code that all compliant AMP  implementations use.

Custom Errors

Let's imagine a Divide command which takes two integers, numerator and denominator, and returns a floating-point value result (the result of numerator / denominator).

On most platforms, division will fail if you divide by zero, so if you need special handling for this error condition it is necessary to define what the _error_code will be.

Using the Twisted implementation it would done something like this:

class Divide(amp.Command):
        arguments = [('numerator', amp.Integer()),
                     ('denominator', amp.Integer())]
        response = [('result', amp.Float())]
        errors = {ZeroDivisionError: 'ZERO_DIVISION'}
    

Then, if the responder for the Divide command raises a ZeroDivisionError exception, the response will look something like this:

_error

1

_error_code

ZERO_DIVISION

_error_description

float division

AMP  implementations should then raise this same exception on the client (calling) side. In languages without exceptions the raw error code should be made available so that special handling may be implemented.

Unknown Errors

So, what if something happens that wasn't planned for? AMP  doesn't just encode information about arbitrary exceptions, because this could represent a potentially harmful information leak. And because the authors' consider it better design to make error-handling explicit rather than implicit.

Lets say the responder for a given command raises a UniverseImplodedException for which you have not defined an _error_code. (What would be the point, after all?)

You will get something like this:

_error

1

_error_code

UNKNOWN

_error_description

Unknown Error

The string "UNKNOWN" is a special code that all compliant AMP  implementations use.

That's all there is to error-handling in AMP !

Conclusion

As you've seen, AMP  is simple, efficient and highly flexible. You can build virtually any kind of protocol, suitable for your application, on top of AMP.

Security features are achieved by layering AMP  on top of TLS. AMP  runs just as well over ordinary TCP sockets as it does over SSL/TLS sockets. Or you could run AMP  over stdin/stdout between multiple processes. It is entirely transport-agnostic.

A standard set of rich data types are shipped with the reference implementation.

Other implementations  may, or may not, provide the full set of types out of the box. However, adding custom data types with your own serialization format is generally quite easy.

--

Happy AMP'ing!