Edit

10. Vanes III: Eyre, Iris

This lesson covers Arvo's HTTP server vane (Eyre) and its HTTP client vane, Iris. We discuss how Arvo interacts with clients over HTTP.

Servers

One of Urbit's primary use cases is to act as a “personal server”. To examine this statement, we need to consider what a server does. Etymologically, a server serves a service. Generally speaking, it is the locus of a computation and coordination process. A server program is a system daemon—and since Gall agents are essentially daemons in many respects, Urbit's execution model fulfills this niche nicely.

Some servers are physical or logical devices which talk to other devices as clients. Internet webpage and application servers typically follow this model. Other servers are software processes that run on the same hardware or local network as the client process, e.g. mail servers, print servers, and file servers.

The two major operational models for servers are the request–response model and the publish–subscribe (pub-sub) model. The request–response pattern corresponds to pokes and gifts in Arvo terms, while the pub-sub pattern is supplied by subscriptions and updates.

  • A client originates and submits requests, and receives responses.

  • A server accepts requests and replies with responses.

A client and a server need to agree on a communications protocol. There are many of these, but the basis for the World Wide Web is the HyperText Transfer Protocol (HTTP).

Serving a Web Page

Two of the simplest actions one can take with a basic web server are to simply post a web page to any clients and to respond to interactions with that web page. Some interactions take place purely in the client session (form entry in the browser before submission), but then are propagated to the server.

Requests

HTTP requests are like Gall agent pokes: they are messages to trigger some action on the server. A method is specified (like GET, PUT, or POST) and the associated service-specific data follow.

It consists of a block of request headers, a block of general headers, and a block of representation headers. These may by followed by the body.

  • GET means a read-only request for information (like an Urbit scry but without the bound namespace).

  • PUT requests a state creation or update.

  • POST asks for the server to process data. (Both PUT and POST are analogous to Urbit pokes.)

Responses

A server response to a web page looks like this:

The response code is normatively 200 OK for a successful page access, but 404 Not Found and other errors and special messages also occur frequently. (It would be very interesting if Urbit would implement 402 Payment requested.)

The actual mechanics of communicating both kinds of communications are wrapped by /lib/server and Eyre. Generally speaking, an agent will receive HTTP requests in +on-poke, and commonly includes a +handle-http arm to deal with inbound-request:eyre values. /lib/server has request header parsers and response handlers which make it easy to respond appropriately (e.g. (send:server ~ [%login-redirect './apps/my-agent'])).

Eyre

Eyre is an HTTP server, which receives HTTP messages from Unix and produces HTTP messages in reply. Your agent can register endpoints which a browser or other tool can interact with. Eyre can be instrumented to work with threads and generators.

HTTP requests include a method tag. While other methods exist, we are primarily interested in POST, PUT, and GET requests. Since we don't want to deal with client-side code yet, we're going to use curl to send requests here.

  • POST is only used with Eyre to obtain a cookie.

This cookie should be included in subsequent requests.

  • PUT requests are used to send actions to Eyre: pokes, subscriptions, acks, unsubscribe requests, and channel deletions.

  • GET requests are used to connect to a channel and receive any pending events. (Remember how Urbit prefers a dataflow computing model?)

/sys/lull Definition

Eyre is responsible for a few subsystems that facilitate userspace applications (unlike, say, Behn or Ames, most of what Eyre does is to support userspace).

  • Authentication

  • Channels

  • Threads and generators

  • HTTP request handling

  • Scry interface

Several of these are handled by the $action dispatch system, invoked when a binding matches a known path.

A $binding is a system unique mapping for a path to match. A $binding must be system unique because we don't want two handlers for a path; what happens if there are two different actions for [~ /]?

/sys/vane/eyre is relatively more straightforward than (say) Ames. There is only one interface (search for ~% %http-server).

Authentication

Client sessions typically require a login. (This is not true for materials served to the clearweb, e.g. via %blog.) A cookie is generated for each session in response to a login using +code.

(Visits are part of the EAuth system, q.v.)

Authentication is handled by +authentication.

  • See +authentication in /sys/vane/eyre. Locate where the session cookie is created and logged.

Authentication is enforced by +request-is-logged-in and +request-is-authenticated.

  • See +request-is-logged-in in /sys/vane/eyre.

Channels

Channels are the main method where a webpage communicates with Gall apps. Subscriptions and pokes are issues with PUT requests on a path, while GET requests on that same path open a persistent EventSource channel. The EventSource API is a sequence number based API that browser provide which allow the server to push individual events to the browser over a connection held open. In case of reconnection, the browser will send a 'Last-Event-Id' header to the server; the server then resends all events since then.

An EventSource interface is a way to track server-sent events for a client session. The JS on the browser/client-side receives text/event-stream formatted events. So a channel is a given connection to a browser including the EventSource connection.

Conventional channels communicate in JSON. Values passed into Urbit can be sent through a mark file to be transformed into a %noun or other type automatically. On the way out, a similar transformation can take the values back into MIME types.

  • Trace %poke and %poke-json in /sys/vane/eyre.

  • Examine /lib/schooner and the /mar files in the %yard desk. How does it handle JSON transformations? What about binary types like audio/mpeg (MP3)?

Noun channels make it possible for external applications to speak Urbit nouns. This means that you can communicate with an Urbit ship in a way other than using a JSON payload. The content-type is marked as application/x-urb-jam. Nouns are +jammed when sent into Eyre.

  • Locate where x-urb-jam is processed in /sys/vane/eyre and in /mar/noun.

Thanks to the mark system and +find-channel-mode, it is straightforward on Urbit's side to implement noun channels. However, on the other side you need something that speaks nouns, such as noun.py.

HTTP request handling

All else set aside, the real purpose of Eyre is to act as the HTTP server for an Urbit ship. Eyre maintains a server configuration. There is an $inbound-request type to receive an HTTP request, but the main HTTP types are actually in another arm, +http.

A raw HTTP request handle happens like this:

  • Eyre subscribes to an app at /http-response/[eyre-id].

  • Eyre pokes the app with %handle-http-request and the ID.

  • The app produces %facts of ?(%http-response-header %http-response-data %http-response-cancel).

  • Examine +http. Find the request and response handlers. In particular, see $simple-payload.

  • Examine /lib/server, which contains wrapper arms for mere mortals.

In some ways, although this is the meat-and-potatoes of Eyre, it's all rather straightforward.

To actually get a value into userspace, Eyre sends the response to Gall:

  • +request-to-app to dispatch an %app $action to Gall.

  • +deal-as to %pass to Gall.

Threads and generators

We can treat a (local) ship as a “serverless” function call for a client.

A generator is a standalone computation based on arguments. Eyre supports generators explicitly:

and runs them in +request, branch %gen.

  • How does the generator run? Note the $roof and the +mock call.

Having bound a generator /gen/eyre-gen

to an endpoint /mygen,

a client may initiate a generator call by posting a PUT request thus:

A thread is a transient standalone computation similar in some regards to a generator. Spider provides thread support using an Eyre binding.

EAuth

Eyre's EAuth system “a mechanism by which HTTP clients may authenticate themselves as a specific urbit on HTTP endpoints served by any other urbit.” In other words, you can provide a comet-like client to an arbitrary client.

  • How robust to collision is random EAuth assignment? ($\frac{1}{2^{128}-2^{64}} \approx 3^{-39}$, or one in 100 undecillion)

  • If you are interested in investigating EAuth in detail, see $visitor, $logbook, $eauth-plea, $eauth-boon as well as the source description at ~palfun-foslup, “mirage (eauth)” and the app ~paldev, %chat-stream.

SSL

If you are working locally, you typically just have HTTP set up instead of secure HTTPS. SSL is a transport layer protocol formerly used for client–server encrypted channels, but now HTTPS actually uses TLS.

The %acme agent configures a certificate if you have a domain set up to use with Urbit.

  • Read the +install arm in /app/acme.

Scry interface

Eyre exposes some information about bindings and connections, such as the sessions and cookies:

Vere I/O Driver: vere/io/http.c

The runtime counterpart to Eyre is vere/io/http.c, which is the HTTP server.

Vere's http.c uses libh2o as its HTTP server:

H2O is a new generation HTTP server that provides quicker response to users with less CPU utilization when compared to older generation of web servers. Designed from ground-up, the server takes full advantage of HTTP/2 features including prioritized content serving and server push

Take especial note of the following functions:

  • u3_http_io_init() to start the HTTP server manager.

  • _http_serv_listen_cb(), the callback for receiving a value

  • _http_serv_accept()

  • _http_seq_accept() to process a new HTTP request

  • _http_hgen_send() to send an HTTP response

Iris

Iris is an HTTP client. It is not currently widely used since Urbit ships do not often serve as HTTP clients (rather as peers).

/sys/lull Definition

Structure

/sys/vane/iris is quite short and legible. Most of the vane is either tracking connection state as a client or sending updates for data transmission progress.

  • Examine the “Iris Guide: Example” thread.

  • Use this as a springboard for examining how the response header and body are constructed in Urbit.

Iris has relatively little information to expose at any given time, and has an extremely minimal scry interface:

  • %i %x %whey, show memory usage.

Vere I/O Driver: vere/io/cttp.c

The runtime counterpart to Iris is vere/io/cttp.c, which is the HTTP client.

  • Read u3_cttp_io_init(), which initializes the client manager state.

Like Eyre, Vere's Iris uses libh2o as its HTTP server/client library.

  • Read _cttp_creq_on_body(), the callback upon receiving a response body

  • _cttp_creq_respond()

  • _cttp_http_client_receive()

Exercise

  • Produce an app which allows a clearweb login. This can be done using EAuth but it would be interesting to implement standard username/password login as well.

Previous9. Vanes II: AmesNext11. Vanes IV: Clay

Last updated