8. Vanes I: Behn, Dill, Kahn, Lick
In this lesson we'll look at Arvo's timer vane, Behn, and Dill, the terminal driver vane. This includes discussion of the Dojo terminal, and the userspace/kernelspace interfaces %helm and %hood.
A vane is an Arvo kernel module that performs essential system operations. The vanes are:
Ames, the peer-to-peer networking vane.
Behn, the timer vane.
Clay, the filesystem, revision-control and build system vane.
Dill, the terminal-driver vane.
Eyre, the HTTP vane.
Gall, the application vane.
Iris, the server HTTP vane.
Jael, the security vane.
Khan, the control vane.
Lick, the interprocess communication (IPC) vane.
As described above, we use Arvo proper to route and control the flow of
moves. However, Arvo proper is rarely directly responsible for processing the event data that directly causes the desired outcome of amove. This event data is contained within acard. Instead, Arvo proper passes thecardoff to one of its vanes, which each present an interface to clients for a particular well-defined, stable, and general-purpose piece of functionality.
Vane Interface
Arvo is a message dispatcher, which doesn't really know about the vanes except via their existence in a van=(map term vane) in Arvo's +$soul.
Formally, a vane must be a “vane-shaped noun”—an interface presenting the arms:
|%
:: +call: handle a +task request
++ call
:: +load: migrate an old state to a new vane version
++ load
:: +scry: view vane state at a particular /path
++ scry
:: +stay: extract state before reload
++ stay
:: +take: handle $response sign
++ take
--++loadand++stayare necessary to update the vane.++callis used to pass a request in (“advance to target”).++scryexposes the read-only scry namespace of the vane.++takereceives a response from another vane (“retreat along call stack”).
(Now the formerly-elided distinctions between sign, gift, task, and note start to matter.)
A
noteis sent by a vane to the Arvo kernel's++callarm.Arvo dispatches a
taskto a vane's++callarm.The vane performs the work.
If a result needs to be passed back, it is emitted as a
giftalong theductback to Arvo's++takearm.Arvo dispatches a
signto the original caller's++takearm.
The actual mechanics of this are that the moves are placed into the appropriate duct, which is a (list wire), simply an ordered collection of moves representing the causal history.
/sys/arvo tracks what little it knows about vanes at a few points, e.g.:
:: van: vanes while we desire it (in larval stage)
van=(map term (trap vase))
::
++ grow
|= way=term
?+ way way
%a %ames
%b %behn
%c %clay
%d %dill
%e %eyre
%g %gall
%i %iris
%j %jael
%k %khan
%l %lick
==
::
+$ vane [=vase =worm]vaseis of course a generic vase, but specifically it expects a noun with the correct+$type.wormis the worm cache as discussed in Arvo II: The Boot Sequence.
Arvo interacts with vanes in vase mode; for instance, a scry takes place via a call to the ++scry arm via a ++slap against the %limb named %scry: (~(slap wa sac) rig [%limb %scry]). As usual, working in vase mode permits dynamic updates to the source.
Vanes have as their subject:
/sys/hoonfor language definitions./sys/arvofor message dispatch./sys/lullfor a shared interface definition./sys/zusefor various stdlib utilities.
In particular, /sys/lull acts as a header so that vanes can “see” each other's interface.
Updates
As with other parts of the system, vanes are rebuilt if the inner core on which they rely has been updated or if the vane itself has changed.
An update to a vane is triggered by ++mod:what:pith in the ++le event-loop engine. (Recall from ca05 that ++what is involved in a system upgrade.) While there are some unfamiliar types here, note particularly the %= centis clause building each vane.
++ mod
|= [del=news all=?]
^+ ..pith
=^ job=oped fat.mod.sol (~(adorn adapt fat.mod.sol) del all)
=? lul.mod.sol ?=(^ lul.job)
(smit:va "lull" pit /sys/lull/hoon u.lul.job)
=? zus.mod.sol ?=(^ zus.job)
(smit:va "zuse" lul.mod.sol /sys/zuse/hoon u.zus.job)
%- %+ need:wyrd kel.ver.zen
:~ lull/;;(@ud q:(slap lul.mod.sol limb/%lull))
zuse/;;(@ud q:(slap zus.mod.sol limb/%zuse))
==
%= ..pith
van.mod
%+ roll van.job
|= [[nam=term txt=cord] van=_van.mod.sol]
^+ van
=/ nex (create:va our zus.mod.sol nam /sys/vane/[nam]/hoon txt)
=/ nav (~(get by van) nam)
=? nex ?=(^ nav) (update:va vase.u.nav nex)
(~(put by van) nam (settle:va nex))
==The recompilation against %zuse takes place in ++adorn:adapt:part. (Arvo Pärt)
:: kernel modules
::
:: %zuse is the subject of the vanes; force all if we have a new %zuse
::
=. all |(all ?=(^ zus))
=| nav=(map term cord)
=? nav all
%- ~(gas by nav)
%+ turn
~(tap by dir:(~(dip of fat) /sys/vane))
|=([name=@ta _fat] [`@tas`name (sole (need fil))])Behn
Behn is a timer/wake-up call system. Since it's a simple vane, let's approach it obliquely, by looking at a generator that calls it.
Open
/base/gen/timers/hoonand examine the code.
.^((list [date=@da =duct]) %bx (en-beam [our %$ [%da now]] /debug/timers))/sys/lull Definition
/sys/lull DefinitionThe interface to Behn is defined in /sys/lull:
:: ::::
:::: ++behn :: (1b) timekeeping
:: ::::
++ behn ^?
|%
+$ gift :: out result <-$
$% [%doze p=(unit @da)] :: next alarm
[%wake error=(unit tang)] :: wakeup or failed
[%meta p=vase]
[%heck syn=sign-arvo] :: response to %huck
==
+$ task :: in request ->$
$~ [%vega ~] ::
$% $>(%born vane-task) :: new unix process
[%rest p=@da] :: cancel alarm
[%drip p=vase] :: give in next event
[%huck syn=sign-arvo] :: give back
$>(%trim vane-task) :: trim state
$>(%vega vane-task) :: report upgrade
[%wait p=@da] :: set alarm
[%wake ~] :: timer activate
==
-- ::behn(A %vega task informs the vane that the kernel has been upgraded.)
Structure
+$ behn-state
$: %2
timers=(tree [key=@da val=(qeu duct)])
unix-duct=duct
next-wake=(unit @da)
drips=drip-manager
==How does Behn think of timers?
/sys/behn presents three primary cores:
A type definition core.
A helper core.
The primary vane interface.
Behn only has two kinds of moves: %wait tasks and %wake gifts. How are these processed and what do they result in? (See the +$timer-map structure too.)
What is a drip? How is it used?
Say an app (the Target) is subscribed to updates from Clay (the Client). If Clay
%gives updates to the app directly and the app crashes, this may cause Clay to crash as well. If instead Clay%passes Behn a%driptaskwrapping the updategift, Behn will set a timer fornowthat, when fired, will cause the updategiftto be given. If it causes a crash then it will have been in response to the%dripmove, thereby isolating Clay from the crash. Thus%dripacts as a sort of buffer against cascading sequences of crashes.
The Nested Core Pattern
(~rabsef-bicrym calls this the “++abet engine” and while it's not a popular term inside of the core development team, I like the pithiness of it. “Engine” as a term of art is frequently used in the kernel to refer to ++le two-letter doors so this usage is not completely inconsistent, just more specialized in application.)
The basic concept of the nested core pattern is to have an outer core which builds a list of cards and state changes, then produces the queued changes all at once. (I always think about this as being one of those wind-up cars that you crank and then set down to whir away.)
Behn's nested core pattern is pretty simple: it has an alias to this, one ++emit arm to prepend a move to a list of moves, and an ++abet arm to yield the [moves state]. The ++per-event core is used to script the neighboring ++scry and ++call arms for the vane without leaking state invariants. Behn's instantiation of the ++abet pattern centralizes the helper outer core as a centralized state machine.
These are some common ++abet pattern arms. These are not all unique, and many cores will omit all or most of these.
++abed—initialize. Set up the state of the inner core.++yoke—initialize. Start from a particular value.++abet—finalize. Exit from an inner core to an outer core, taking changes. Commonly, take a modified valued and overwrite it into the final state with a++put:by.++abut—finalize. Alternative exit from++abetfor a deletion.++move—send a move. Generalization for++pass/++give.++pass—request an action. Prepend a%passmove to the current list of moves.++give—return a result. Prepend a standard%giveto the current list of moves.++emit—submit a card. Prepend a card to the current list of cards.++emil—submit cards. Prepend a list of cards to the current list of cards.
If some state needs to be maintained, this can be built in a door, but Behn's particular example is even more basic. In files with associated doors or with multiple nested core instances, it is common to prepend a two-letter identifier to disambiguate which outer core is being scripted at any given time, such as ++mo-abet or ++ap-emil.
Vere I/O Driver: vere/io/behn.c
vere/io/behn.cArvo acquires its timer updates from Unix via vere/io/behn.c. This file presents its primary interface at u3_behn_io_init() to initialize a timer. This simply retrieves the current Unix time for a starting point and sets up the interface.
Each communicating vane is linked various I/O drivers in vere/auto.c. (These do not correspond one-to-one with vanes.) These are registered into car_u, a global _u3_auto used for I/O driver invocations and callbacks.
For Behn, the timer is set in _behn_ef_doze() using uv_timer_start(). The corresponding wakeup timer is emitted in _behn_time_cb() when the libuv main event loop handler.
Finally, we can examine how the injection comes back into Arvo in _behn_time_cb(). An ovum is produced by u3_ovum_init(), manually injected using u3_auto_plan(), and subscribed to with u3_auto_peer(). /sys/behn then processes this wakeup event as a %wake via its ++call arm.
Dill
Dill is Urbit's terminal driver.
|pass [%d %text "hello world"]Dill as a vane is mostly responsible for actually constructing terminal sessions and coordinating input and output. Thus most of the terminal stack actually lives in userspace (instrumented by Gall) rather than in /sys/dill.
What do we mean when we talk about a terminal? Originally, of course, computer were directly programmed by moving wires between vacuum tubes or chips; later, this evolved to the ability to read and output cards. Computer terminals with CRT-based character displays began to be used in the late 1950s and gradually became more common. In fact, the original plasma display screens were used with PLATO in the 1970s–1990s.
When we refer to the terminal today, we typically mean a modern terminal emulator, which presents a terminal-like text user interface (TUI) for software to treat as if it were an actual character display. Terminal emulators need to track information like dimensions ($x$, $y$), content layout, active sessions or connexions, and cursor position. They provide affordances like a color space, escape codes, layout libraries, and scrollable sessions.
Dill is responsible for interfacing with keystrokes and with the terminal emulator session. Since Urbit can be run in a daemon mode, it's not necessary for Dill to actually have a terminal session for Urbit to run.
Due to terminal emulator limitations, Dill sessions are only properly supported today in the %webterm app.
/sys/lull Definition
/sys/lull DefinitionThe /sys/lull interface specification for Dill is more complicated than that of Behn. Unlike Behn, a number of supporting types are necessary to produce the basic pair of gift/task for Dill.
:: ::::
:::: ++dill :: (1d) console
:: ::::
++ dill ^?
|%
+$ gift :: out result <-$
$% [%blit p=(list blit)] :: terminal output
[%logo ~] :: logout
[%meld ~] :: unify memory
[%pack ~] :: compact memory
[%trim p=@ud] :: trim kernel state
[%logs =told] :: system output
== ::
+$ task :: in request ->$
$~ [%vega ~] ::
$% [%boot lit=? p=*] :: weird %dill boot
[%crop p=@ud] :: trim kernel state
[%flog p=flog] :: wrapped error
[%heft ~] :: memory report
$>(%init vane-task) :: after gall ready
[%logs p=(unit ~)] :: watch system output
[%meld ~] :: unify memory
[%pack ~] :: compact memory
[%seat =desk] :: install desk
[%shot ses=@tas task=session-task] :: task for session
$>(%trim vane-task) :: trim state
$>(%vega vane-task) :: report upgrade
[%verb ~] :: verbose mode
[%knob tag=term level=?(%hush %soft %loud)] :: deprecated removeme
session-task :: for default session
told :: system output
== ::
:: ::
+$ session-task :: session request
$% [%belt p=belt] :: terminal input
[%blew p=blew] :: terminal config
[%flee ~] :: unwatch session
[%hail ~] :: terminal refresh
[%open p=dude:gall q=(list gill:gall)] :: setup session
[%shut ~] :: close session
[%view ~] :: watch session blits
== ::
:: ::
+$ told :: system output
$% [%crud p=@tas q=tang] :: error
[%talk p=(list tank)] :: tanks (in order)
[%text p=tape] :: tape
== ::
::
:::: :: (1d2)
::
+$ blew [p=@ud q=@ud] :: columns rows
+$ belt :: client input
$? bolt :: simple input
[%mod mod=?(%ctl %met %hyp) key=bolt] :: w/ modifier
[%txt p=(list @c)] :: utf32 text
== ::
+$ bolt :: simple input
$@ @c :: simple keystroke
$% [%aro p=?(%d %l %r %u)] :: arrow key
[%bac ~] :: true backspace
[%del ~] :: true delete
[%hit x=@ud y=@ud] :: mouse click
[%ret ~] :: return
== ::
+$ blit :: client output
$% [%bel ~] :: make a noise
[%clr ~] :: clear the screen
[%hop p=$@(@ud [x=@ud y=@ud])] :: set cursor col/pos
[%klr p=stub] :: put styled
[%mor p=(list blit)] :: multiple blits
[%nel ~] :: newline
[%put p=(list @c)] :: put text at cursor
[%sag p=path q=*] :: save to jamfile
[%sav p=path q=@] :: save to file
[%url p=@t] :: activate url
[%wyp ~] :: wipe cursor line
== ::
+$ dill-belt :: arvo input
$% belt :: client input
[%cru p=@tas q=(list tank)] :: errmsg (deprecated)
[%hey ~] :: refresh
[%rez p=@ud q=@ud] :: resize, cols, rows
[%yow p=gill:gall] :: connect to app
== ::
+$ dill-blit :: arvo output
$% blit :: client output
[%qit ~] :: close console
== ::
+$ flog :: sent to %dill
$% [%crop p=@ud] :: trim kernel state
$>(%crud told) ::
[%heft ~] ::
[%meld ~] :: unify memory
[%pack ~] :: compact memory
$>(%text told) ::
[%verb ~] :: verbose mode
== ::
:: ::
+$ poke :: dill to userspace
$: ses=@tas :: target session
dill-belt :: input
== ::
-- ::dillThe main concepts to keep in mind:
Dill receives
%belttasks and sends%blitgifts.%belttasks result from keystrokes, terminal resizing,%blitgifts result from output events: putting a character, clearing the screen, placing the cursor.
Structure
Dill's primary state is its +$axle with a logging level:
|% :: console protocol
+$ axle ::
$: %7 ::
hey=(unit duct) :: default duct
dug=(map @tas axon) :: conversations
eye=(jug @tas duct) :: outside observers
ear=(set duct) :: syslog listeners
lit=? :: boot in lite mode
egg=_| :: see +take, removeme
== ::
+$ axon :: dill session
$: ram=term :: console program
tem=(unit (list dill-belt)) :: pending, reverse
wid=_80 :: terminal width
== ::
+$ log-level ?(%hush %soft %loud) :: none, line, full
--As with /sys/behn, the primary cores of Dill include:
Type definitions (two cores).
Helper core (
++asper-cause engine).The primary vane interface.
A good way to familiarize yourself with Dill operations is to follow the full lifecycle of input and output in the next section.
Dill is the first vane in the boot sequence, and is used to boot Jael. (Compare %aqua, which does not need to start Dill and can initialize Jael directly.)
Vere I/O Driver: vere/io/term.c, ptty.c
vere/io/term.c, ptty.cThe main entrypoint for the terminal is u3_term_io_init(), which simply sets up the interface and callbacks.
As we noted before, each communicating vane is linked various I/O drivers in vere/auto.c. Here the global car_u has term.c connected for invocations and callbacks.
vere/vere.h:
// u3_term_start_spinner(): prepare spinner state. RETAIN.
void u3_term_start_spinner(u3_noun say, c3_o del_o);
// u3_term_stop_spinner(): reset spinner state and restore input line.
void u3_term_stop_spinner(void);
// u3_term_get_blew(): return window size [columns rows].
u3_noun u3_term_get_blew(c3_l tid_l);
// u3_term_ef_winc(): window change.
void u3_term_ef_winc(void);
// u3_term_ef_ctlc(): send ^C.
void u3_term_ef_ctlc(void);
// u3_term_io_init(): initialize terminal I/O.
u3_auto* u3_term_io_init(u3_pier* pir_u);
// u3_term_io_hija(): hijack console for cooked print.
FILE* u3_term_io_hija(void);
// u3_term_it_log(): writes a log message
void u3_term_io_log(c3_c* line);
// u3_term_io_loja(): release console from cooked print.
void u3_term_io_loja(int x, FILE* f);
// u3_term_log_init(): initialize terminal for logging
void u3_term_log_init(void);
// u3_term_log_exit(): clean up terminal.
void u3_term_log_exit(void);
// u3_ptty_init(): initialize platform-specific tty.
u3_utty* u3_ptty_init(uv_loop_t* lup_u, const c3_c** err_c);For instance, a keystroke is processed in the following way:
uv_read_start()is thelibuvevent loop injector._term_read_cb()is the character keystroke callback._term_suck()processes input._term_io_suck_char()decides if it's anxtermterminal emulator issue or something for Arvo to know about._term_io_spit()inputs the buffer and belt._term_io_belt()actually sends a value along the belt (as anovum).
Now in Arvo, the keystroke is routed to Dill:
++calltakes thetaskfrom Arvo.++call:asreceives the input and dispatches on%belt.++send:assends the action to the proper session.++deal:assignals to pass the keystroke to Gall.++pass:asexecutes the pass to Gall.
Output can happen in three ways:
Some output traverses a path from (say) Gall outbound. These are conventionally known as
%slogs.Somehow a noun is marked for output using a
%sloghint:~&sigpam does this directly.~>siggar can do this with a%sloghint and a priority value.++slogwraps this as a function.
Back in the runtime:
Once we have a
%sloghint for the runtime, it can be emitted from the Nock processor via Nock Eleven.noun/nock.c:_n_bint()dispatches this viaSLOGand thencedo_slogin the bytecode processor.noun/trace.c:u3t_slog()prints a value directly through theu3C.slog_fprint handler, which is_cw_serf_send_slog().vere/main.c::_cw_serf_send_slog()sends the hint output to the serf.vere/main.c:_cw_serf_send()is a plea handler to send pleas to the daemon.vere/newt.c:u3_newt_send()sends a++jammed noun of the output (u3s_jam_xeno()) as a buffer to a stream. In this case, the value ends up at thelibuvbuffer usinguv_write().
In this case, output results in the fact of the layout of the terminal handler rather than being explicitly known about by Dill. As a consequence, if you are building a TUI application and you don't want to have misaligned output, you need to build it directly using
%blits and suppress%slogs within your app. (Seetui-toysfor an example in%snek.)Other output goes via Dill because the terminal vane explicitly needs to know about its position. This are
%blits.A program (like
%snek) can specify to manually output a%blitlike%klr(styled text) or%put(plain text) at the cursor./lib/etuioffers some interesting demonstrations in this vein; see the++zoengine.
vere/io/term.c:_term_io_kick()applies effects sent toterm.c, including blits. It was registerd as the effect handler when the I/O drivers were registered.vere/io/term.c:_term_ef_blit()switches on the type of blit (notice it can also track a cursor position).vere/io/term.c:_term_it_show_tour()emits UTF-32 to the cursor location.Finally,
vere/io/term.c:_term_it_show_line()prints at the actual cursor position.
The runtime terminal manager automatically handles aspects of layout such as maintaining the input line at the bottom of the screen. See
vere/io/term.c:_term_it_restore_line()for details. (CSI = Control Sequence Introducer sequence) A different terminal handler (like%webterm) may handle these decisions differently.
Other output may follow yet a different path; for instance, u3l_log() directly prints using vsnprintf()to stderr. Some output goes via the libuv event loop, such as u3_ptty_init()..
Examine
u3_term_log_init()invere/io/term.c.Examine
_term_it_show_line()in the same file.
Dojo, %hood, %helm, %drum
%hood, %helm, %drumDojo is Urbit's primary CLI interface, and while it is too complicated to delve deeply into here, the major parts to consider include:
Hoon parser. Real-time parsing of input, which evaluates Hoon code for syntactical correctness. (This is the reason that typing at the Dojo prompt is frequently slower than typing at other CLIs.)
Specialized syntax. Besides Hoon input, generators and pokes can be set up and invoked directly from the Dojo prompt.
%sayand%askgenerators are head-tagged pairs with gates following that returnsole-results.+generatorprefixes cause Dojo to look in/genfor a particular generator file.|generatoris a Hood generator, on which more in a moment.+desk!generatorinvokes a generator on a particular desk.:agent|generatortakes the output from a generator (at/gen/agent/generator) and feeds it as a noun to the agent's++on-pokearm.-threadinvokes a thread, with a similar desk-specific prefix as above.Hood. Most of the interesting Urbit instrumentation is provided to Dojo by the Hood/Helm agent pipeline.
%hoodis the overarching system app, to which Dojo redirects generator invocations prefixed with|bar such as|passand|install. (Thus,|passis in fact shorthand for:hood|pass.)%helmprovides the interface for kernel and system functionality, such as|verb,|moon, etc. Hood calls into Helm.%drummanages the active CLI apps (|dojo/link,Ctrl+x).%kilninstruments filesystem operations using Clay.
The overall call web of these is surprisingly tangled; as an example, let's trace |dojo/link, which tells Dojo to register a CLI interface to a Gall agent.
/gen/hood/link/app/hood/lib/drum→++pokew/%drum-link/lib/drum→++poke-link/lib/drum→++se-linkeel(what is this?)
Exercises
Trace the entire lifecycle of
|pass [%d %text "Hello Mars!"]. Include a function-by-function annotation and commentary.Write a basic vane,
/sys/vane/.~&on receiving atask.
Last updated
