19 The host interface

(1) The host interface is one C header, pinax.h. It uses only <stddef.h> and <stdint.h> and reads as C99, C23 and C++. This clause defines what its functions do; the comments in the header give more detail for each.

(2) One state is used by one thread at a time; states share nothing. A pinax_str a function returns is valid until the next call on that state unless stated otherwise.

19.1 Error codes

(1) Functions return a pinax_err; PINAX_OK is 0. Three outcomes are not failures: PINAX_SUSPENDED (the budget ran out, 19.5), PINAX_WAITING (every task waits for the host, 19.8) and PINAX_PENDING (a host word finishes later). From 0x100 come the faults and failures of scripts (the codes of 11.1); from 0x200 the failures of the interface itself: ARGUMENT, NO_ROOM (the block is too small), SYNTAX, TYPE (load refusals), NOT_FOUND, SIGNATURE, FORMAT, STATE (a call not allowed now).

(1a) From 0x400 come the failure codes a program declares (8.14), numbered in the order the state declares them. A word that fails with one gives that number; pinax_error_text gives its name. A host reads the name: the number of a code depends on what was loaded before it.

(2) pinax_error_text gives the last failure as text; for a fault, in the form module:line: name.

(3) A failure of the interface means the same in every function.

CodeMeaning
PINAX_ERR_ARGUMENTan argument is wrong: a null pointer, a word number that is none, a size other than the declaration’s
PINAX_ERR_NO_ROOMthe block has no room; pinax_needed says at least how big it had to be (19.3)
PINAX_ERR_SYNTAX PINAX_ERR_TYPEthe source was refused; the place and the reason are in diag and pinax_error_text
PINAX_ERR_NOT_FOUNDno word, global, module or host word of that name; also a host word not yet given again to a state opened from an image
PINAX_ERR_SIGNATUREthe signature differs: of the word looked for, of the host word registered, of the global shared
PINAX_ERR_FORMATa container, image or share file is not of this processor’s form, or failed its checks
PINAX_ERR_STATEa call not possible now: a run is suspended, a host word is running, no code is loaded, the state is frozen

19.2 Opening and closing

(1) pinax_open(block, config, &state) opens a state inside block. Fields of config that are 0 take the processor’s defaults: the slots of the banks, the size of the main run’s region, the number of handles alive at once, the instruction budget of a run (0 for none), the function that receives load diagnostics. When the block is too small the answer is PINAX_ERR_NO_ROOM, and pinax_needed tells at least the size that would have been enough: the size up to what was being taken when the block ran out. A load may take more after that, so when the block opened again at that size runs out too, read it again.

(2) config.hash_key is the 16-byte key of the hash indexes use (8.10). A host that indexes text from outside gives a random key. All zeros means a fixed key.

(3) pinax_close closes the handles still open (calling their types’ close functions). It does not free the block; the block is the host’s.

(4) The block serves three uses. Opening takes the fixed parts of the state: the banks, the call frames, the main run’s region, the tables of host words and handles. With the defaults that is about 150 KB, most of it the region (64 KB) and the integer bank (32 KB), which config can shrink. The loaded program is stacked after it and stays. While a load runs, the unused tail of the block is the compiler’s working memory: about 20 KB for a small source, about 1 KB more per word, and 64 KB more when the source has a constant pattern (16.7). The working memory is free again when the load ends. A body uses working memory for the blocks it actually nests, not for the deepest nesting allowed.

19.3 Host words

(1) pinax_register(state, module, defs, count) registers host functions as the words of one module. Each entry of defs has a name, a signature text (in the form of 9.2, such as "(s str gives n int or fail)"), the function, a user pointer and flags. Signatures are checked once, here. Register words before the sources that use them.

(2) A host function receives (call, user, args). Arguments and results are in arrays by type: integers (and bool, characters, handles and declared types over int) in args->i[] and args->ri[], strings in args->s[] and args->rs[], in the order of the signature, counted from 0 within each type. The integer pointers point straight at the state’s slots. A narrow integer of the signature (7.6) travels as one 64-bit slot, and after the function returns the processor checks the range of each result: outside it is the fault overflow (the result of a waiting word gets the same check in pinax_complete and ends with that code). An argument of pinax_run outside its narrow type is PINAX_ERR_ARGUMENT, and nothing runs.

(3) When the function returns something other than PINAX_OK, a word whose signature has or fail fails with that code (11.2), and a word without it ends the run with a fault.

(4) A string result is copied into the caller’s region unless the word has the flag PINAX_WORD_KEEP_STR. With the flag it is not copied but pointed at; those bytes shall outlive the state and not change (static text, the program’s arguments and the like), and there is no limit to the number of such results. Room for new bytes comes from pinax_call_alloc, in the caller’s region. A parameter of type buffer in the signature is a buffer of the script: pinax_call_buffer gives its free room and is told how many bytes were filled.

19-host-words.c

#include "pinax.h"
#include <stdio.h>
#include <string.h>

/* (s str gives r str): the result is written into the caller's region. */
static pinax_err shout(pinax_call *call, void *user, const pinax_args *args) {
    (void)user;
    pinax_mem room = pinax_call_alloc(call, args->s[0].len + 1);
    if (!room.ptr)
        return PINAX_ERR_FULL;                       /* no `or fail`: this is a fault */
    unsigned char *out = room.ptr;
    for (size_t k = 0; k < args->s[0].len; ++k) {
        unsigned char c = args->s[0].ptr[k];
        out[k] = c >= 'a' && c <= 'z' ? (unsigned char)(c - 32) : c;
    }
    out[args->s[0].len] = '!';
    args->rs[0] = (pinax_str){out, args->s[0].len + 1};
    return PINAX_OK;
}

/* (s str gives n int or fail): an outcome other than PINAX_OK is the word's failure. */
static pinax_err digits(pinax_call *call, void *user, const pinax_args *args) {
    (void)call; (void)user;
    int64_t n = 0;
    if (args->s[0].len == 0)
        return PINAX_ERR_FORMAT;
    for (size_t k = 0; k < args->s[0].len; ++k) {
        unsigned char c = args->s[0].ptr[k];
        if (c < '0' || c > '9')
            return PINAX_ERR_FORMAT;
        n = n * 10 + (c - '0');
    }
    args->ri[0] = n;
    return PINAX_OK;
}

/* (b buffer or fail): writes straight into a buffer the script declared. */
static pinax_err stamp(pinax_call *call, void *user, const pinax_args *args) {
    (void)user;
    pinax_mem room = pinax_call_buffer(call, args->i[0], 0);
    if (room.len < 5)
        return PINAX_ERR_FULL;
    memcpy(room.ptr, "[ok] ", 5);
    pinax_call_buffer(call, args->i[0], 5);
    return PINAX_OK;
}

int main(void) {
    static unsigned char block[1 << 20];
    pinax_config config = {.struct_size = sizeof config};
    pinax_state *st;
    if (pinax_open((pinax_mem){block, sizeof block}, &config, &st) != PINAX_OK)
        return 1;
    static const pinax_word_def words[] = {
        {"shout", "(s str gives r str)", shout, NULL, 0},
        {"digits", "(s str gives n int or fail)", digits, NULL, 0},
        {"stamp", "(b buffer or fail)", stamp, NULL, 0},
    };
    pinax_register(st, "app", words, 3);
    static const char source[] =
        "use app\n"
        "buffer line 32\n"
        "word demo (gives s str)\n"
        "  line clear\n"
        "  line app.stamp or_else (e) end\n"
        "  \"12x\" app.digits or_else (e) -1 end to n\n"
        "  cat (\"hello\" app.shout, \" \", \"42\" app.digits or_else (e) -1 end, \" \", n) line append or_else (e) end\n"
        "  line contents\n"
        "end\n";
    if (pinax_load(st, "main", (pinax_str){(const unsigned char *)source, sizeof source - 1}, PINAX_LOAD_CODE) != PINAX_OK)
        return 1;
    pinax_word w;
    pinax_str out;
    if (pinax_find(st, "demo", "(gives s str)", 0, &w) != PINAX_OK)
        return 1;
    pinax_run(st, w, &(pinax_args){.rs = &out});
    printf("%.*s\n", (int)out.len, (const char *)out.ptr);
    pinax_close(st);
    return 0;
}

Output

$ ./19-host-words
[ok] HELLO! 42 -1

19.4 Loading and lines

(1) pinax_load(state, name, source, mode) loads a source (5.2). name is the module’s name; the modes are PINAX_LOAD_CODE and PINAX_LOAD_DATA (13.4). A refusal is PINAX_ERR_SYNTAX or PINAX_ERR_TYPE, and the diagnostic arrives at config.diag as (file, line, column, text).

(1a) pinax_load_bytecode(state, bytes) loads a bytecode container made by the processor into an empty state. The shape of a container is the processor’s (1 (3)). The processor does not trust a container; it checks it, and refuses with PINAX_ERR_FORMAT one that could break 4 (2a). Among what this implementation refuses: registers outside the window, jumps out of the word, a register read before it is written, a jump back or a call that does not charge the budget, a tail call to a word with other results, an operation that can fail without a failure word after it or with nothing after that, and an instruction that changes a string register and names it twice.

(1b) A load has no budget. So the processor loads a source or a container in time proportional to its length: no pass of it, finding names, deciding which words yield or checking the flow of the registers, grows with the square of the length. A host that takes untrusted sources therefore limits only their length, without fear that one load never ends.

(1c) A load into a state that already holds a program appends what the new source adds to the program and checks the new words only, so the bound of (1b) holds for the lines of a session and for modules loaded one by one. The declaration tables are kept the same way: a load reads what earlier loads declared where it lies and appends only its own declarations. The pass that decides which words yield covers the words the load brings, and the whole program only when the load redefines a word. The room the program takes in the block grows with what was declared: when a part has no room left to append to, the processor moves that part to a larger place, and when the load has ended it takes back the room it left; while a task (clause 14) is alive it does not, and the next load after the task has ended does. redefine appends the new body and leaves the old code where it was. The room used is read from block_bytes_used of pinax_stats. A session line that only runs code takes no room.

(1d) pinax_capacity(state, name, value) gives the number for a capacity written host (8 (4a)), before the load that has it. name is the declaration’s name: the plain name for the first source loaded, and module.name for any source. Giving a name again replaces its number. The state keeps up to 16 names of up to 47 bytes; a value is 1 to 231 − 1. A capacity may become a constant of the code (capacity, a check the loader proves against it), so exported code (19.11) made with one number may not be the code of a load with another; such a load runs without it.

19-capacity.c

#include "pinax.h"
#include <stdio.h>

/* The host knows at start how many requests it serves at once; the script leaves the number to it. */
int main(void) {
    static unsigned char block[1 << 20];
    pinax_config config = {.struct_size = sizeof config};
    pinax_state *st;
    if (pinax_open((pinax_mem){block, sizeof block}, &config, &st) != PINAX_OK)
        return 1;
    static const char source[] =
        "tasks handlers host 32 8192 apart\n"
        "list seen int host\n"
        "word room (gives n int) seen capacity end\n";
    const uint32_t at_once = 48;
    pinax_capacity(st, "handlers", at_once + 1); /* one more for the task that takes the requests */
    pinax_capacity(st, "seen", at_once);
    if (pinax_load(st, "main", (pinax_str){(const unsigned char *)source, sizeof source - 1}, PINAX_LOAD_CODE) != PINAX_OK) {
        pinax_str why = pinax_error_text(st);
        fprintf(stderr, "%.*s\n", (int)why.len, (const char *)why.ptr);
        return 1;
    }
    pinax_word w;
    int64_t n = 0;
    pinax_find(st, "room", "(gives n int)", 0, &w);
    pinax_run(st, w, &(pinax_args){.ri = &n});
    printf("seen holds %lld\n", (long long)n);
    pinax_close(st);
    return 0;
}

Output

$ ./19-capacity
seen holds 48

(2) pinax_eval(state, text) runs one line as a session does. All lines form one module (the session). What a line leaves is read with pinax_eval_count and pinax_eval_value.

(2a) The session has a stack that stays from line to line. A line starts with what the lines before it left and is checked as a word whose inputs are those values and whose results are what it leaves. The session stack holds at most 16 values: an int or a type over it, a bool, an err, a str of at most 255 bytes, a handle that is not owned. A line that would leave more, or a collection, a stem or an owned value, is refused; one that leaves a longer str ends with PINAX_ERR_NO_ROOM after it ran. A line that is refused or ends with a fault leaves the stack as it was before it; so does one that ran out of budget and is abandoned, while one resumed to its end leaves what it left. A str of the session stack is copied into the state and stays valid until the next pinax_eval or pinax_eval_clear. pinax_eval_clear(state) empties the stack, as does a line that begins with the tool word clear. A load or a run leaves it as it is; an image and a clone do not carry it.

(3) pinax_on_barrier(state, function, user) gives the function a dot calls (10.8 (3)): at each dot the code reaches, and at the end of the code of a load or of a session line, also one that ends with a fault, the processor calls function(user). In it the host writes what its host words hold back and waits for what they started in the background; it does not call the state. NULL, the default, finishes nothing. A clone keeps the function; an image does not. The host library’s pinax_host_register_all gives one that writes the output io holds back.

19.5 Finding and running

(1) pinax_find(state, name, signature, flags, &word) finds a word and checks that its signature is the one given; another is PINAX_ERR_SIGNATURE. A bare name is a word of the first source loaded; module.word a word of that module (exported or not).

(2) pinax_run(state, word, args) runs a word. When the budget runs out it returns PINAX_SUSPENDED, and pinax_resume goes on where it stopped; the budget does not change results. A fault returns its code, and the state takes the next run.

(2a) The budget counts the cost of each check at a back branch or a call, the bytes the string words scan, and each turn that run_all gives a task (14.1 (2)), which costs one. So every run of a container the processor accepts meets the budget, even one whose tasks spawn and end without a loop of their own.

(3) While a run is stopped (PINAX_SUSPENDED, PINAX_WAITING), a new pinax_run is PINAX_ERR_STATE. The host either continues it to its end or gives it up with pinax_abort.

(3a) pinax_abort(state) ends a stopped run as a fault would: the handles the run opened are closed, its tasks are freed and its pending tokens are forgotten (pinax_complete with one of them is PINAX_ERR_NOT_FOUND). What it already stored in globals stays. When what stopped is the code of a load, the load is taken back as a failed one is. The state then takes the next run. With nothing stopped it is PINAX_ERR_STATE. A host that must take the globals back too saves an image (19.9) before the run.

19-run.c

#include "pinax.h"
#include <stdio.h>
#include <string.h>

static void load(pinax_state *st, const char *text) {
    if (pinax_load(st, "main", (pinax_str){(const unsigned char *)text, strlen(text)}, PINAX_LOAD_CODE) != PINAX_OK) {
        pinax_str why = pinax_error_text(st);
        printf("refused: %.*s\n", (int)why.len, (const char *)why.ptr);
    }
}

int main(void) {
    static unsigned char block[1 << 20];
    pinax_config config = {.struct_size = sizeof config, .budget = 1000};   /* instructions per run or resume */
    pinax_state *st;
    if (pinax_open((pinax_mem){block, sizeof block}, &config, &st) != PINAX_OK)
        return 1;
    load(st,
         "word total (n int gives r int) 0 to s  1 n 1 add range do (i int) s i add to s end each  s end\n"
         "word ratio (a int, b int gives r int) a 100 mul b div end\n");

    pinax_word w;
    /* the signature asked for must be the word's */
    printf("find with another signature: %s\n",
           pinax_find(st, "total", "(gives r int)", 0, &w) == PINAX_ERR_SIGNATURE ? "PINAX_ERR_SIGNATURE" : "?");
    pinax_find(st, "total", "(n int gives r int)", 0, &w);
    int64_t n = 100000, r = 0;
    pinax_args args = {.i = &n, .ri = &r};
    int resumes = 0;
    pinax_err err = pinax_run(st, w, &args);
    while (err == PINAX_SUSPENDED) {                 /* the budget ran out: go on where it stopped */
        ++resumes;
        err = pinax_resume(st, &args);
    }
    printf("total(100000) = %lld after %s resumes\n", (long long)r, resumes > 10 ? "many" : "few");

    /* a fault comes back as the error code, with its place in the error text */
    pinax_find(st, "ratio", "(a int, b int gives r int)", 0, &w);
    int64_t ab[2] = {3, 0};
    err = pinax_run(st, w, &(pinax_args){.i = ab, .ri = &r});
    pinax_str why = pinax_error_text(st);
    printf("ratio(3, 0): %s, \"%.*s\"\n", err == PINAX_ERR_DIV_ZERO ? "PINAX_ERR_DIV_ZERO" : "?", (int)why.len, (const char *)why.ptr);

    /* the state is still usable */
    ab[1] = 4;
    err = pinax_run(st, w, &(pinax_args){.i = ab, .ri = &r});
    printf("ratio(3, 4) = %lld\n", (long long)r);

    /* one line at a time, as a session does */
    const char *line = "6 7 mul \"done\"";
    if (pinax_eval(st, (pinax_str){(const unsigned char *)line, strlen(line)}) == PINAX_OK) {
        for (uint32_t k = 0; k < pinax_eval_count(st); ++k) {
            pinax_value v;
            pinax_eval_value(st, k, &v);
            if (v.kind == PINAX_VALUE_INT)
                printf("value %u: %lld (%.*s)\n", k, (long long)v.i, (int)v.type.len, (const char *)v.type.ptr);
            else
                printf("value %u: %.*s (%.*s)\n", k, (int)v.s.len, (const char *)v.s.ptr, (int)v.type.len, (const char *)v.type.ptr);
        }
    }
    pinax_close(st);
    return 0;
}

Output

$ ./19-run
find with another signature: PINAX_ERR_SIGNATURE
total(100000) = 5000050000 after many resumes
ratio(3, 0): PINAX_ERR_DIV_ZERO, "main:2: div_zero"
ratio(3, 4) = 75
value 0: 42 (int)
value 1: done (str)

(4) pinax_run may be called from inside a host word too. There it runs only words found with the flag PINAX_FIND_NESTED, and words that may yield (14.2) cannot be found with that flag.

(5) pinax_word_effects(state, word, &effects) tells what a word may do beyond its stack and locals, worked out from the loaded code of the word and of every word it may call. effects.effects holds PINAX_EFFECT_READS_GLOBALS and PINAX_EFFECT_WRITES_GLOBALS (a list’s length, a queue and a task set count as globals), PINAX_EFFECT_READS_OUTSIDE and PINAX_EFFECT_WRITES_OUTSIDE (through host words), PINAX_EFFECT_YIELDS, PINAX_EFFECT_FAILS and PINAX_EFFECT_ANYTHING (a C function of extern); module and module_effects name the host modules it reaches (the first 16 of n_modules). A host word tells what it touches with the flags PINAX_WORD_READS and PINAX_WORD_WRITES of its registration; one with neither counts as both, and one with PINAX_WORD_PURE as neither. A call through a word value counts every word of the program with that signature, a spawned task every word without parameters and results. A host may refuse a program whose words reach what they should not.

19.6 Handles

(1) pinax_handle_type makes a handle type with a name and a close function; scripts use the name as a type in signatures. pinax_handle_new makes a handle of a C pointer, pinax_handle_get gives a handle’s pointer (NULL for a handle given back), and pinax_handle_release gives a handle back.

(2) A handle is a number and a generation (as in 8.9). A script cannot get an address from a handle. The handles a run that ends with a fault opened are closed by the state, calling their close functions.

19-handles.c

#include "pinax.h"
#include <stdio.h>

typedef struct { int value; int open; } counter;
static counter pool_[4];
static uint32_t counter_type;

static void close_counter(void *object, void *user) {   /* called for handles the script never closed */
    (void)user;
    ((counter *)object)->open = 0;
    printf("closed by the state: counter at %d\n", ((counter *)object)->value);
}
static pinax_err c_new(pinax_call *call, void *user, const pinax_args *args) {
    (void)user;
    for (int k = 0; k < 4; ++k)
        if (!pool_[k].open) {
            pool_[k] = (counter){0, 1};
            pinax_handle h;
            pinax_err err = pinax_handle_new(pinax_call_state(call), counter_type, &pool_[k], &h);
            args->ri[0] = (int64_t)h;
            return err;
        }
    return PINAX_ERR_FULL;
}
static pinax_err c_bump(pinax_call *call, void *user, const pinax_args *args) {
    (void)user;
    counter *c = pinax_handle_get(pinax_call_state(call), counter_type, (pinax_handle)args->i[0]);
    if (!c)
        return PINAX_ERR_STALE;
    args->ri[0] = ++c->value;
    return PINAX_OK;
}
static pinax_err c_free(pinax_call *call, void *user, const pinax_args *args) {
    (void)user;
    counter *c = pinax_handle_get(pinax_call_state(call), counter_type, (pinax_handle)args->i[0]);
    if (c) {
        printf("freed by the script: counter at %d\n", c->value);
        c->open = 0;
    }
    return pinax_handle_release(pinax_call_state(call), counter_type, (pinax_handle)args->i[0]);
}

int main(void) {
    static unsigned char block[1 << 20];
    pinax_config config = {.struct_size = sizeof config};
    pinax_state *st;
    if (pinax_open((pinax_mem){block, sizeof block}, &config, &st) != PINAX_OK)
        return 1;
    pinax_handle_type(st, "counter", close_counter, NULL, &counter_type);
    static const pinax_word_def words[] = {
        {"new", "(gives owned counter or fail)", c_new, NULL, 0},
        {"bump", "(c counter gives n int)", c_bump, NULL, 0},
        {"free", "(c owned counter)", c_free, NULL, 0},
    };
    pinax_register(st, "ctr", words, 3);
    static const char source[] =
        "use ctr\n"
        "word twice (gives n int or fail)\n"
        "  ctr.new or_fail to c\n"
        "  c ctr.bump drop  c ctr.bump to n\n"
        "  c ctr.free\n"
        "  n\n"
        "end\n"
        "word broken (gives n int or fail)\n"
        "  ctr.new or_fail to c\n"
        "  c ctr.bump 0 div to n\n"            /* a fault while the counter is open */
        "  c ctr.free\n"
        "  n\n"
        "end\n";
    if (pinax_load(st, "main", (pinax_str){(const unsigned char *)source, sizeof source - 1}, PINAX_LOAD_CODE) != PINAX_OK)
        return 1;
    pinax_word w;
    int64_t n = 0;
    pinax_find(st, "twice", "(gives n int or fail)", 0, &w);
    pinax_run(st, w, &(pinax_args){.ri = &n});
    printf("twice = %lld\n", (long long)n);
    pinax_find(st, "broken", "(gives n int or fail)", 0, &w);
    pinax_err err = pinax_run(st, w, &(pinax_args){.ri = &n});
    printf("broken: %s\n", err == PINAX_ERR_DIV_ZERO ? "PINAX_ERR_DIV_ZERO" : "?");
    pinax_close(st);
    return 0;
}

Output

$ ./19-handles
freed by the script: counter at 2
twice = 2
closed by the state: counter at 1
broken: PINAX_ERR_DIV_ZERO

19.7 Binding

(1) pinax_bind(state, global, memory, bytes) changes the storage of a global declared bound (8.12) to host memory; the size shall match the declaration: for an array, the cells times the bytes of a cell (the width of a narrow integer, 8 for any other value; 8.4). Script and host read and write the same bytes without copying. pinax_global_find tells where a declared global is and how large it is.

19-bind.c

#include "pinax.h"
#include <stdio.h>

int main(void) {
    static unsigned char block[1 << 20];
    pinax_config config = {.struct_size = sizeof config};
    pinax_state *st;
    if (pinax_open((pinax_mem){block, sizeof block}, &config, &st) != PINAX_OK)
        return 1;
    static const char source[] =
        "bound array samples int 5\n"
        "bound global scale int\n"
        "word scaled_sum (gives n int) samples sum scale mul end\n"
        "word clear_first () 0 0 set samples end\n";
    if (pinax_load(st, "main", (pinax_str){(const unsigned char *)source, sizeof source - 1}, PINAX_LOAD_CODE) != PINAX_OK)
        return 1;
    static int64_t samples[5] = {1, 2, 3, 4, 5};    /* host memory the script reads and writes in place */
    static int64_t scale = 10;
    pinax_bind(st, "samples", samples, sizeof samples);
    pinax_bind(st, "scale", &scale, sizeof scale);
    pinax_word w;
    int64_t n = 0;
    pinax_find(st, "scaled_sum", "(gives n int)", 0, &w);
    pinax_run(st, w, &(pinax_args){.ri = &n});
    printf("scaled_sum = %lld\n", (long long)n);
    samples[4] = 105;                                /* the host changes it; no copy to refresh */
    pinax_run(st, w, &(pinax_args){.ri = &n});
    printf("after the host wrote samples[4]: %lld\n", (long long)n);
    pinax_find(st, "clear_first", "()", 0, &w);
    pinax_run(st, w, NULL);
    printf("after the script wrote cell 0: samples[0] = %lld\n", (long long)samples[0]);
    pinax_close(st);
    return 0;
}

Output

$ ./19-bind
scaled_sum = 150
after the host wrote samples[4]: 1150
after the script wrote cell 0: samples[0] = 0

(1a) pinax_share(state, global, source, from) gives a global that state declares shared (8.13) the storage of the global from of source, without copying. Kind, type and capacity must be the same, else PINAX_ERR_SIGNATURE. An index also needs the same hash key (19.2), and its table given first from the same source. A list keeps the length it has then. When a table is given again, its index is given again too.

(1b) A source that lent storage is frozen: pinax_run, pinax_load, pinax_eval and pinax_bind on it are PINAX_ERR_STATE from then on. So its bytes do not change, and states on several threads may read them at once. source must stay open while the states that borrow from it use it. Until every shared global is given, runs are refused with PINAX_ERR_NOT_FOUND. After an image is opened or a state cloned, give them again (19.9).

19-share.c

#include "pinax.h"
#include <stdio.h>
#include <string.h>

/* One state holds the data; two others read it in place. Nothing is copied. */
static const char data[] =
    "record city\n  name str(12)\n  people int\nend\n"
    "list cities city 8\n"
    "index by_name cities city.name\n"
    "row cities (\"Seoul\", 9411)\n"
    "row cities (\"Busan\", 3317)\n"
    "row cities (\"Incheon\", 3001)\n";
static const char reader[] =
    "record city\n  name str(12)\n  people int\nend\n"
    "shared list cities city 8\n"
    "shared index by_name cities city.name\n"
    "word people_of (name str gives n int or fail) name by_name lookup or_fail city.people end\n"
    "word total (gives n int)\n"
    "  0 to t  0 to i\n"
    "  loop i cities len lt while  t i cities idx city.people add to t  i 1 add to i end\n"
    "  t\n"
    "end\n";

static pinax_state *open_in(unsigned char *block, size_t bytes) {
    pinax_config config = {.struct_size = sizeof config};
    pinax_state *st;
    return pinax_open((pinax_mem){block, bytes}, &config, &st) == PINAX_OK ? st : NULL;
}

static int64_t call(pinax_state *st, const char *word, const char *signature, const char *name, pinax_err *err) {
    pinax_word w;
    int64_t out = 0;
    pinax_str in = {(const unsigned char *)name, name ? strlen(name) : 0};
    pinax_args args = {.s = &in, .ri = &out};
    *err = pinax_find(st, word, signature, 0, &w);
    if (*err == PINAX_OK)
        *err = pinax_run(st, w, &args);
    return out;
}

int main(void) {
    static unsigned char blocks[3][1024 * 1024];
    pinax_state *source = open_in(blocks[0], sizeof blocks[0]);
    pinax_state *a = open_in(blocks[1], sizeof blocks[1]), *b = open_in(blocks[2], sizeof blocks[2]);
    if (!source || !a || !b || pinax_load(source, "data", (pinax_str){(const unsigned char *)data, strlen(data)},
                                          PINAX_LOAD_DATA) != PINAX_OK)
        return 1;
    pinax_err err;
    pinax_state *readers[2] = {a, b};
    for (int k = 0; k < 2; ++k) {
        if (pinax_load(readers[k], "main", (pinax_str){(const unsigned char *)reader, strlen(reader)},
                       PINAX_LOAD_CODE) != PINAX_OK)
            return 1;
    }
    call(a, "total", "(gives n int)", NULL, &err);
    printf("before sharing: %#x (not found)\n", (unsigned)err);

    for (int k = 0; k < 2; ++k) /* the table first, then its index */
        if (pinax_share(readers[k], "cities", source, "cities") != PINAX_OK ||
            pinax_share(readers[k], "by_name", source, "by_name") != PINAX_OK)
            return 1;
    printf("a: Busan has %lld\n", (long long)call(a, "people_of", "(name str gives n int or fail)", "Busan", &err));
    printf("b: total %lld\n", (long long)call(b, "total", "(gives n int)", NULL, &err));

    pinax_global mine, theirs;
    pinax_global_find(source, "cities", &theirs);
    pinax_global_find(b, "cities", &mine);
    printf("same bytes: %s\n", mine.ptr == theirs.ptr ? "yes" : "no");
    err = pinax_load(source, "more", (pinax_str){(const unsigned char *)"global x int\n", 13}, PINAX_LOAD_CODE);
    printf("the source loads again: %#x (state)\n", (unsigned)err);
    pinax_close(a);
    pinax_close(b);
    pinax_close(source);
    return 0;
}

Output

$ ./19-share
before sharing: 0x204 (not found)
a: Busan has 3317
b: total 15729
same bytes: yes
the source loads again: 0x207 (state)

(1c) Several processes share through a file. pinax_share_save(source, out, &written) writes the globals of source that can be shared (scalars, arrays, lists, buffers and indexes; not those it declares shared itself) into one file without addresses; pinax_share_size is its length. Saving copies, so source is not frozen. The host library’s pinax_map_file(path, &bytes, &map) maps the file read-only, so that the processes that map the same file use the same pages. pinax_share_file(state, global, bytes, from) gives the file’s global from, with the checks of (1a); an index needs the hash key of the file and its table given first from the same file. The file’s head, the size and length of each global and the slots of an index are checked, else PINAX_ERR_FORMAT; its other bytes are trusted as an image’s are (19.9), except that no more than N bytes are read from a str(N) cell whose length byte says more. bytes starts on a 16-byte boundary and outlives the states that read it: call pinax_unmap_file after they are closed.

19-share-file.c

#include "pinax.h"
#include "pinax_host.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

/* One program writes the data as a share file; any number of processes map it and read it in
   place. Here the same program maps it twice, as two processes would. */
static const char data[] =
    "record city\n  name str(12)\n  people int\nend\n"
    "list cities city 8\n"
    "index by_name cities city.name\n"
    "row cities (\"Seoul\", 9411)\n"
    "row cities (\"Busan\", 3317)\n";
static const char reader[] =
    "record city\n  name str(12)\n  people int\nend\n"
    "shared list cities city 8\n"
    "shared index by_name cities city.name\n"
    "word people_of (name str gives n int or fail) name by_name lookup or_fail city.people end\n";

static unsigned char blocks[3][1024 * 1024];

static int write_file(const char *path) {
    pinax_config config = {.struct_size = sizeof config};
    pinax_state *source;
    if (pinax_open((pinax_mem){blocks[0], sizeof blocks[0]}, &config, &source) != PINAX_OK ||
        pinax_load(source, "data", (pinax_str){(const unsigned char *)data, strlen(data)}, PINAX_LOAD_DATA) != PINAX_OK)
        return 1;
    size_t size = pinax_share_size(source), written = 0;
    unsigned char *bytes = malloc(size);
    if (!bytes || pinax_share_save(source, (pinax_mem){bytes, size}, &written) != PINAX_OK)
        return 1;
    FILE *f = fopen(path, "wb");
    int bad = !f || fwrite(bytes, 1, written, f) != written;
    bad |= f && fclose(f) != 0;
    printf("wrote %zu bytes\n", written);
    free(bytes);
    pinax_close(source); /* the file needs nothing of it */
    return bad;
}

static void read_file(const char *path, unsigned char *block, size_t bytes, const char *name) {
    pinax_config config = {.struct_size = sizeof config};
    pinax_state *st;
    pinax_str file;
    pinax_mapping *map;
    pinax_word w;
    int64_t out = 0;
    pinax_str in = {(const unsigned char *)name, strlen(name)};
    pinax_args args = {.s = &in, .ri = &out};
    if (pinax_map_file(path, &file, &map) != PINAX_OK)
        return;
    if (pinax_open((pinax_mem){block, bytes}, &config, &st) == PINAX_OK &&
        pinax_load(st, "main", (pinax_str){(const unsigned char *)reader, strlen(reader)}, PINAX_LOAD_CODE) == PINAX_OK &&
        pinax_share_file(st, "cities", file, "cities") == PINAX_OK &&      /* the table first */
        pinax_share_file(st, "by_name", file, "by_name") == PINAX_OK &&    /* then its index */
        pinax_find(st, "people_of", "(name str gives n int or fail)", 0, &w) == PINAX_OK &&
        pinax_run(st, w, &args) == PINAX_OK)
        printf("%s has %lld\n", name, (long long)out);
    pinax_close(st);
    pinax_unmap_file(map); /* after the state that read it */
}

int main(void) {
    const char *path = "cities.pnxs";
    if (write_file(path) != 0)
        return 1;
    read_file(path, blocks[1], sizeof blocks[1], "Seoul");
    read_file(path, blocks[2], sizeof blocks[2], "Busan");
    remove(path);
    return 0;
}

Output

$ ./19-share-file
wrote 528 bytes
Seoul has 9411
Busan has 3317

(1d) While a run is not finished — suspended by the budget, waiting for a host operation, or inside a host word — pinax_bind, pinax_share and pinax_share_file are PINAX_ERR_STATE: a view the run holds may point into the storage they would replace.

(2) The number that pinax_global_find gives is what a word with a collection parameter (9.9) takes as that argument, an integer in i. A number of a collection of another kind or another size of element, or of no global, is the fault PINAX_ERR_STATE where the word uses it.

19.8 Host words that finish later

(1) A host word registered with PINAX_WORD_AWAITABLE may start some work, take a token with pinax_call_pending, and return PINAX_PENDING. The task that called it waits and others run. When nothing else can run, pinax_run or pinax_resume returns PINAX_WAITING. The host finishes the work with pinax_complete(state,
token, outcome, values) and calls pinax_resume.

19-await.c

#include "pinax.h"
#include <stdio.h>

/* A host word that finishes later: it starts the work, keeps a token and returns PINAX_PENDING. */
static pinax_token waiting[4];
static int64_t asked[4];
static int n_waiting;
static pinax_err fetch(pinax_call *call, void *user, const pinax_args *args) {
    (void)user;
    asked[n_waiting] = args->i[0];
    waiting[n_waiting++] = pinax_call_pending(call);
    return PINAX_PENDING;
}

int main(void) {
    static unsigned char block[1 << 20];
    pinax_config config = {.struct_size = sizeof config};
    pinax_state *st;
    if (pinax_open((pinax_mem){block, sizeof block}, &config, &st) != PINAX_OK)
        return 1;
    static const pinax_word_def words[] = {{"fetch", "(n int gives r int or fail)", fetch, NULL, PINAX_WORD_AWAITABLE}};
    pinax_register(st, "net", words, 1);
    static const char source[] =
        "use net\n"
        "tasks workers 2\n"
        "list got int 4\n"
        "word both (gives r int)\n"
        "  do 1 net.fetch or_else (e) -1 end got push or_else (e) end end workers spawn or_else (e) 0 return end drop\n"
        "  do 2 net.fetch or_else (e) -1 end got push or_else (e) end end workers spawn or_else (e) 0 return end drop\n"
        "  workers run_all\n"
        "  got sum\n"
        "end\n";
    if (pinax_load(st, "main", (pinax_str){(const unsigned char *)source, sizeof source - 1}, PINAX_LOAD_CODE) != PINAX_OK)
        return 1;
    pinax_word w;
    int64_t r = 0;
    pinax_args args = {.ri = &r};
    pinax_find(st, "both", "(gives r int)", 0, &w);
    pinax_err err = pinax_run(st, w, &args);
    printf("run: %s, %d operations started\n", err == PINAX_WAITING ? "PINAX_WAITING" : "?", n_waiting);
    /* the host finishes them in its own time, here the second first */
    for (int k = n_waiting - 1; k >= 0; --k) {
        int64_t result = asked[k] * 100;
        pinax_complete(st, waiting[k], PINAX_OK, &(pinax_args){.ri = &result});
    }
    err = pinax_resume(st, &args);
    printf("resume: %s, both = %lld\n", err == PINAX_OK ? "PINAX_OK" : "?", (long long)r);
    pinax_close(st);
    return 0;
}

Output

$ ./19-await
run: PINAX_WAITING, 2 operations started
resume: PINAX_OK, both = 300

(2) In a state the host loop (17.4) runs, a host word takes its token with pinax_loop_pending in place of pinax_call_pending and finishes the work with pinax_loop_complete, from any thread. The loop calls pinax_complete on the state’s thread and continues the run.

19.9 Images and clones

(1) An image is a whole state: code, declarations, globals, slots, regions, stopped runs and tasks. pinax_image_save writes it and pinax_image_open opens it. An image that lies at the start of the block is opened in place, without copying. An image of another format, or of another build’s memory layout, is PINAX_ERR_FORMAT.

(1a) An image is data the host vouches for (4 (2a)): the processor checks its head, format and layout, and that the addresses inside it lie in the block it was saved from, but not every byte. Open only images this host or one it trusts saved. Strings that pointed outside the block (a host’s text) and the last pattern match do not survive an image.

(2) On an opened state the host gives host words, handle types and extern functions again, by name. Until all are given, runs are refused with PINAX_ERR_NOT_FOUND. Handles come back closed (without their close functions), and bound globals back on their own storage. pinax_clone saves and opens in another block in one call.

(2a) A clone opens in a block of pinax_image_size(source) bytes or more; a smaller block is PINAX_ERR_NO_ROOM.

(2b) pinax_clone_with(source, block, flags, out) is pinax_clone with flags. With PINAX_CLONE_KEEP_HOSTS the clone keeps what the source was given in this process, with the same user pointers: its host words, handle types, extern functions and resolver, its exported code (19.11) and its task fault function (19.10 (3a)). So it runs at once. Bound globals are bound again and handles do not pass, as for any clone. An unknown flag is PINAX_ERR_ARGUMENT.

19-image.c

#include "pinax.h"
#include <stdio.h>
#include <stdlib.h>

static pinax_err twice(pinax_call *call, void *user, const pinax_args *args) {
    (void)call; (void)user;
    if (args->i[0] > INT64_MAX / 2 || args->i[0] < INT64_MIN / 2)
        return PINAX_ERR_OVERFLOW;
    args->ri[0] = args->i[0] * 2;
    return PINAX_OK;
}
static const pinax_word_def words[] = {{"twice", "(n int gives r int)", twice, NULL, 0}};

static int64_t call_next(pinax_state *st) {
    pinax_word w;
    int64_t r = 0;
    if (pinax_find(st, "next", "(gives r int)", 0, &w) != PINAX_OK)
        return -1;
    pinax_err err = pinax_run(st, w, &(pinax_args){.ri = &r});
    return err == PINAX_OK ? r : -(int64_t)err;
}

int main(void) {
    static _Alignas(16) unsigned char a[1 << 20], b[1 << 20], c[1 << 20];
    pinax_config config = {.struct_size = sizeof config};
    pinax_state *st;
    if (pinax_open((pinax_mem){a, sizeof a}, &config, &st) != PINAX_OK)
        return 1;
    pinax_register(st, "host", words, 1);
    static const char source[] =
        "use host\n"
        "global calls int\n"
        "word next (gives r int) calls 1 add set calls  calls host.twice end\n";
    if (pinax_load(st, "main", (pinax_str){(const unsigned char *)source, sizeof source - 1}, PINAX_LOAD_CODE) != PINAX_OK)
        return 1;
    call_next(st);
    printf("in the first state: %lld\n", (long long)call_next(st));

    /* save the whole state: code, globals, everything */
    size_t size = pinax_image_size(st), written = 0;
    unsigned char *image = malloc(size);
    pinax_image_save(st, (pinax_mem){image, size}, &written);

    /* open it in another block; host words are given again by name */
    pinax_state *copy;
    if (pinax_image_open((pinax_mem){b, sizeof b}, (pinax_str){image, written}, &config, &copy) != PINAX_OK)
        return 1;
    printf("before the host word is given again: %s\n", call_next(copy) == -PINAX_ERR_NOT_FOUND ? "PINAX_ERR_NOT_FOUND" : "?");
    pinax_register(copy, "host", words, 1);
    printf("the copy goes on: %lld\n", (long long)call_next(copy));
    printf("the first state is its own: %lld\n", (long long)call_next(st));

    /* a clone is the same in one call */
    pinax_state *twin;
    if (pinax_clone(copy, (pinax_mem){c, sizeof c}, &twin) != PINAX_OK)
        return 1;
    pinax_register(twin, "host", words, 1);
    printf("the clone of the copy: %lld\n", (long long)call_next(twin));
    free(image);
    pinax_close(twin);
    pinax_close(copy);
    pinax_close(st);
    return 0;
}

Output

$ ./19-image
in the first state: 4
before the host word is given again: PINAX_ERR_NOT_FOUND
the copy goes on: 6
the first state is its own: 6
the clone of the copy: 8

19.10 Measuring, and the locals of a fault

(1) pinax_stats_get gives the highest use of slots, region, handles and call depth, and the number of instructions run. It is for choosing the block and the capacities.

(2) pinax_fault_local gives, after a run that ended with a fault, the locals of the word the fault happened in with their values at that time (up to 16; text cut to its first 32 bytes, and shown only when its bytes lie in memory the state can vouch for). They are what pnx shows below a fault (11.3).

(3) pinax_task_faults gives how many tasks of apart groups (14.4 (3a)) ended with a fault since the state opened, and where the last one happened and why, in the form of a fault’s place (module:line: error). After the run that had it, pinax_fault_local reads that task’s locals, until the next run ends.

(3a) pinax_on_task_fault(state, fn, user) has the state call fn(user, state, fault) once for each such fault, at the fault, before the task’s slot is used again. fault gives the fault, the task’s tag (14.5) and its place in the same form. Several tasks that fault in one run are told one by one. Inside fn, pinax_fault_local reads the locals of that task. The run is in the middle, so fn may call only the functions that read the state: pinax_fault_local, pinax_task_faults, pinax_error_text. NULL takes the function away. An image and a clone forget it, as they forget host words (19.9), unless the clone keeps them (19.9 (2b)).

19.11 Exported code

(1) pnx export-c (20 (3)) writes a program’s code as C. The host compiles that file with pinax_native.h of the same build and gets one pinax_native table. It covers the program as it was exported: its code and its words, counted in code_words and words. pinax_native_attach(state, table) makes the state run that code through the table from the first run at which the state’s program begins with the one exported: the same code, the same start of every word, the same build. Given before the loads, it serves a load’s own code too. NULL takes it back.

(1a) What is loaded after it (the lines of a session, a module loaded later) is interpreted, and the table stays in use: a call, a return or a task switch that goes into the code it covers runs there through the table again. A redefinition (20.4 (2a)) of one of its words with the same signature changes the program it covers: from then on the table is not used. One with another signature leaves the code it covers as it was, and the table stays in use.

(2) A program and a host see the same as interpreted: results, faults with their places and locals, failures, the budget (a run stops at the same CHECKs), waiting and tasks. Exported code does not make the checks the loader proved cannot fire (20 (1b)); a build with PINAX_CHECK_PROOFS makes them and stops the program when one fires.

(3) An image does not carry the table: attach it again. Nor does a clone, unless it is made with PINAX_CLONE_KEEP_HOSTS (19.9 (2b)). Attaching while a run is suspended or waits is PINAX_ERR_STATE. A table whose struct_size is another, or with one of code_words and words 0 and the other not, is PINAX_ERR_ARGUMENT; both 0 is a table of the whole program, whatever its length.

(4) The table is data the host vouches for, as an image is (4 (2a)): only its hash is checked, not its machine code. For exported code the guarantee of 4 (2a) holds as far as the loader’s proofs are right.