8 Declarations

8.1 General

Syntax

declaration = [ "export" ] ( [ "bound" ] bindable | "shared" shareable | decl ) ;
bindable    = "global" NAME type
            | "array"  NAME type capacity
            | "buffer" NAME capacity ;
shareable   = bindable
            | "list"   NAME type capacity
            | "index"  NAME NAME PATH ;
decl        = "const"  NAME literal
            | "err"    NAME
            | "list"   NAME type capacity
            | "queue"  NAME type capacity
            | "pool"   NAME NAME capacity
            | "record" NAME { field | "big_endian" } "end"
            | "type"   NAME base
            | "index"  NAME NAME PATH
            | "tasks"  NAME capacity [ size [ size [ size ] ] ] [ "apart" ]
            | "extern" c-name "(" effect ")" "from" STRING [ "as" NAME ]
            | "test"   NAME { body } "end" ;
capacity    = size | "host" ;
size        = INT | NAME ;                         (* NAME: a const *)

(1) A declaration makes a name and what it stands for (a value, storage, a type). Declarations are written only outside definitions, at the outermost level of a source.

(2) A processor reads a source once, from top to bottom. A name shall be declared before the place where it is used. The exceptions are a word that calls itself, and forward (8.11).

(3) Within one module a name cannot be made twice, not even for different kinds of things. The names of core words and types cannot be made again either.

(4) A capacity (size) is a positive integer literal or a const of one. Capacities are fixed at load and do not change while running. All storage is placed inside the state’s block at load.

(4a) The capacity of an array, a buffer, a list, a queue, a pool or a task group may be written host. Then it is the number the host gave for that declaration before the load (19.4 (1d)), and the load treats it as that number written there. A load that meets host for a declaration the host gave no number is refused, and the message names the declaration. host is a word of the declaration only in that place; a constant of that name is not looked up there.

tasks handlers host 32 8192 apart     rem as many as the host serves at once
list seen int host
word count (gives n int) seen len end

$ pnx 08-host-capacity.pnx
08-host-capacity.pnx:1:16: the host gave no capacity for handlers (pinax_capacity)

(5) Globals, cells of arrays, slots of lists, queues and pools, buffers and fields of records start at zero (7 (3)).

Constraints

(6) Making a name that exists already is refused.

global total int
const total 5

$ pnx 08-name-twice.pnx
08-name-twice.pnx:2:7: this name is already defined

(7) Using a name before its declaration is refused.

use str
word show (gives s str) total str.to_str end
global total int

$ pnx 08-use-before.pnx
08-use-before.pnx:2:25: this name is not declared before this place

8.2 Constants

(1) const NAME literal makes a constant. Its type is the literal’s. A constant has no storage: its value goes where it is used. Constants can be capacities, case values and row values.

Constraints

(2) Nothing can be stored into a constant. Inside a definition v to CONST would make a new local of the same name and is refused by the rule against hiding names (9.5); outside a definition to makes no local, so it is refused. set writes globals only (9.4).

const limit 3
word raise () 4 to limit end

$ pnx 08-const-store.pnx
08-const-store.pnx:2:20: this name is already visible here; a new name may not hide it

8.3 Globals

(1) global NAME type makes a global that holds one value. The type is int, bool, err, dec(N), str(N), a declared type, a row type (the name of a table) or a handle type; not str (7.3).

(2) A global is read by writing its name and stored with v set NAME (9.4). Its value stays as long as the state does — after a word’s run ends, and across later loads.

use io
use str
const limit 3
const greeting "hello"
global hits int
global last str(16)
global seen bool
hits str.to_str io.println           rem a global starts at zero
limit set hits
greeting set last
true set seen
cat (last, " ", hits, " ", seen) io.println

$ pnx 08-const-global.pnx
0
hello 3 true

8.4 Arrays

Syntax

assign-item = "assign" NAME "(" value { "," value } ")" ;

(1) array NAME type capacity makes an array with that many cells. An array is always full: len is its capacity. The element type is int, bool, err, dec(N), str(N), a narrow integer (7.6), a declared type, the name of a record, or the name of a table, whose rows the cells then hold (8.8 (5a)). A cell of a narrow integer takes the bytes of its width, a cell of any other value 8 bytes; the cells of lists and queues are the same.

(2) i ARRAY idx reads cell i; v i set ARRAY stores into cell i; a cell of a collection parameter (9.9) is stored the same way. Numbers count from 0. For an array of records idx gives a row (8.8), not a value.

(3) assign ARRAY (value, …) fills an array from cell 0, outside definitions. With fewer values than cells the rest stay as they are. Assigning the same array again starts from cell 0 again. It may be used in a data load (13.4).

(4) The name of an array used as a stem yields its cells from 0 in order (10.6).

use io
use str
array primes int 6
assign primes (2, 3, 5, 7)                rem cells 4 and 5 stay 0
11 4 set primes                            rem v i to array: store into cell i
primes do (p int) p str.to_str io.print " " io.print end each
"" io.println
cat ("len ", primes len, ", cell 2 is ", 2 primes idx) io.println

$ pnx 08-array.pnx
2 3 5 7 11 0 
len 6, cell 2 is 5

Constraints

(5) More values in assign than the array has cells is refused.

array small int 2
assign small (1, 2, 3)

$ pnx 08-assign-many.pnx
08-assign-many.pnx:2:21: assign has more values than the array has cells

Faults

(6) An index below 0, or equal to or above the capacity, is the fault range.

use io
array primes int 4
"reading cell 4 of 4" io.println
4 primes idx drop

$ pnx 08-array-range.pnx
reading cell 4 of 4
08-array-range.pnx:4: index out of range

8.5 Lists

(1) list NAME type capacity makes a list whose length grows from 0 up to its capacity. Its cells are read and stored like an array’s, but only indexes below the length may be used.

(2) v LIST push adds one value at the end; LIST pop takes the last one off and gives it. Both can fail (or fail): a push into a full list fails with err.full, a pop from an empty list with err.range; a push of text too long for a list of str(N) fails with err.range (7.3 (4a)). A list of records gets an empty row at the end with LIST add_row, which gives that row. LIST clear sets the length to 0.

use io
use str
list stack int 3
word fill (or fail)
  1 stack push or_fail
  2 stack push or_fail
end
fill or_else (e) "fill failed" io.println end
cat ("len ", stack len, " of ", stack capacity) io.println
stack pop or_else (e) -1 end str.to_str io.println
cat ("len ", stack len) io.println

$ pnx 08-list.pnx
len 2 of 3
2
len 1

use io
list two int 2
1 two push or_else (e) end
2 two push or_else (e) end
3 two push or_else (e) cat ("third push failed: ", e err_text) io.println end

$ pnx 08-list-full.pnx
third push failed: full

(2a) n LIST truncate cuts the length to n, for lists of integers, of text and of records alike. It takes back a row added with add_row whose values turn out to be wrong.

use io
record city
  name str(16)
  people int
end
list cities city 4
rem Add a row, and take it back when its values turn out wrong.
word add_city (name str, people int or fail)
  cities len to before
  cities add_row or_fail to c
  name c to city.name
  people 0 lt if before cities truncate err.format fail end
  people c to city.people
end
"Seoul" 9411000 add_city or_else (e) end
"Nowhere" -1 add_city or_else (e) cat ("refused: ", e err_text) io.println end
cat (cities len, " rows") io.println

$ pnx 08-list-truncate.pnx
refused: format
1 rows

Constraints

(2b) A list of records whose rows are kept elsewhere cannot be truncated. It is refused when the same source has a field, a global or a collection that holds its rows, a row of it named with row, or an index of it (8.10): once a new row takes the place cut off, they would point at the wrong row. A row held in a local is checked when it is used (fault range).

record city
  name str(16)
end
list cities city 4
index by_name cities city.name
word forget () 0 cities truncate end

$ pnx 08-list-truncate-kept.pnx
08-list-truncate-kept.pnx:6:25: this list's rows are kept by a field, a global, a collection, a named row or an index; it cannot be truncated

Faults

(3) An index past the length is the fault range, even when it is below the capacity. So is a number for truncate larger than the length, or below zero.

list items int 8
1 items push or_else (e) end
1 items idx drop                          rem the list holds one value: index 1 is past it

$ pnx 08-list-past-len.pnx
08-list-past-len.pnx:3: index out of range

Diagnostics

8.6 Queues

(1) queue NAME type capacity makes a queue: what goes in first comes out first. The element type is int, bool, err or dec(N). v QUEUE put puts a value at the back, QUEUE take takes the front one off and gives it, and QUEUE peek gives the front one without taking it. All three can fail: putting into a full queue is err.full, taking from an empty one err.range.

use io
use str
queue jobs int 4
word show (gives s str or fail) jobs take or_fail str.to_str end
10 jobs put or_else (e) end
20 jobs put or_else (e) end
jobs peek or_else (e) 0 end str.to_str io.println
show or_else (e) "?" end io.println
show or_else (e) "?" end io.println
show or_else (e) cat ("empty: ", e err_text) end io.println

$ pnx 08-queue.pnx
10
10
20
empty: range

8.7 Buffers

(1) buffer NAME capacity makes a buffer of that many bytes: a sequence of bytes that can be changed, for building text piece by piece.

(2) s BUF append, n BUF append_int (decimal; n an int or a narrow integer) and b BUF append_byte (b a u8) add at the end; when there is not enough room they fail with err.full and add nothing. BUF contents gives a str view of what it holds now. n BUF truncate cuts the length to n. BUF clear sets the length to 0.

use io
buffer out 32
word build (or fail)
  "x=" out append or_fail
  42 out append_int or_fail
  '!' out append_byte or_fail
end
build or_else (e) "full" io.println end
out contents io.println
2 out truncate
out contents io.println

$ pnx 08-buffer.pnx
x=42!
x=

(3) The view contents gives points at the buffer’s bytes: when the buffer changes, so do the bytes the view sees.

8.8 Records and rows

Syntax

record   = "record" NAME { field | "big_endian" } "end" ;
field    = NAME width ;
width    = "u8" | "u16" | "u32" | "u64" | "i8" | "i16" | "i32" | "i64"
         | "int" | "bool" | "varint" | "dec" "(" INT ")" | "str" "(" size ")"
         | NAME ;                                         (* NAME: a table *)
row-item = [ "export" ] "row" NAME [ NAME ] "(" value { "," value } ")" ;
value    = literal | NAME ;                         (* NAME: a const, a row name, none *)

(1) record NAME … end makes a record type, a sequence of fields, each with a width. An int or i64 field reads as int, a u8 … i32 field as that narrow integer (7.6), a bool field as bool, a dec(N) field as dec(N) (8 bytes), a str(N) field as a str view. A field takes a value of its type only. A field whose type is the name of a table holds a row of that table.

(2) Records are not handled as values. One declares an array, list or pool of records (a table) and handles its rows. A row’s type is named after its table. A row holds a number within the table, and every use checks its range (and, for a pool, its generation, 8.9).

(3) Fields are written as the path RECORD.FIELD: row RECORD.FIELD reads, v row to RECORD.FIELD stores.

(4) row TABLE (value, …) adds one row to a list, outside definitions. Values go in field order; trailing fields may be left out (they are zero). row TABLE NAME (…) names the row; the name stands for the row in later row values and in code.

use io
record user
  name str(16)
  age u8
  admin bool
end
list users user 8
row users ("kim", 30, true)
row users lee ("lee", 25)                  rem a named row; the fields left out stay zero
word show (u users) cat (u user.name, " ", u user.age, " ", u user.admin) io.println end
users do (u users) u show end each
26 u8 lee to user.age                         rem v row to record.field
lee show

$ pnx 08-record.pnx
kim 30 true
lee 25 false
lee 26 false

(5) A field or global that holds a row starts as “none”. row is_none asks whether it is none, TABLE none gives the none row of that table, and row row_index gives the row’s number.

(5a) An array or a list whose element type is the name of a table holds rows of that table, in cells of 8 bytes: the row’s number plus one, with a pool row’s generation in the upper 32 bits; 0 is none, and every cell starts as none. Its cells are read, stored, pushed, popped and walked as any cells are, and a value going in shall be a row of that table. A row read from a cell is checked where it is used, as any row is. A queue holds no rows. A row that alloc has just given (8.9) is stored into a cell with set, which takes it over as a field does; push refuses it, since a push that fails would lose it.

use io
record city
  name str(12)
  people int
end
list cities city 4
list big cities 4                          rem rows of cities
row cities ("Oslo", 709)
row cities ("Bergen", 291)
row cities ("Tromso", 78)
cities do (c cities gives bool) c city.people 100 gt end big filter_into or_else (e) end
big cities sort_by city.name
big do (c cities) c city.name io.println end each

$ pnx 08-row-cells.pnx
Bergen
Oslo

(6) big_endian sets only the byte order that bytes.pack and bytes.unpack of 16.5 use. It does not change the layout in memory.

Constraints

(7) The values of row shall match the fields in order and type.

record user
  name str(16)
  age u8
end
list users user 8
row users (30, "kim")

$ pnx 08-row-types.pnx
08-row-types.pnx:6:12: a value has another type than its field

Faults

(8) An int stored into a narrow integer field is converted first (n u8); a value that does not fit the width makes that conversion the fault overflow (7.6). Nothing is cut: to store the low bits take a word such as low_u8. Reading or storing a field of the none row is the fault range.

record pixel
  level u8
end
array image pixel 4
word set_level (n int) n u8 0 image idx to pixel.level end     rem the conversion checks the range
300 set_level                                                   rem 300 does not fit in u8

$ pnx 08-record-narrow.pnx
08-record-narrow.pnx:5: integer overflow
  n int = 300

8.9 Pools

(1) pool NAME RECORD capacity makes a pool of rows that are taken and given back. POOL alloc takes a free row (or fail, err.full when none is left); row POOL free gives it back; row POOL is_alive asks whether the row is still alive.

(2) A row of a pool holds a generation besides its number. Giving a row back changes the generation of its slot, so a row used after it was given back is noticed — even when another row has taken the slot since.

(3) The row alloc gives is an owned value (12.4). On every path it shall be stored exactly once into a field or global, or given to free. Once stored, the generation check guards it.

use io
record node
  value int
end
pool nodes node 2
global kept nodes
word demo (or fail)
  nodes alloc or_fail to a                 rem an owned row: it must be stored or freed
  7 a to node.value
  a set kept                                rem stored: the global holds the row now
  cat ("value ", kept node.value, ", alive: ", kept nodes is_alive) io.println
  kept nodes free
  cat ("alive after free: ", kept nodes is_alive) io.println
end
demo or_else (e) "no room" io.println end

$ pnx 08-pool.pnx
value 7, alive: true
alive after free: false

(3a) POOL do (r POOL) … end each visits the rows of the pool that are alive, in the order of their slots, not in the order they were taken. A row given back in the block before the walk reaches it is not visited; a row taken in the block is visited when its slot comes later. each is the only combining word (10.6) that walks a pool.

Constraints

(4) A path on which a taken row is neither stored nor given back is refused.

record node
  value int
end
pool nodes node 2
word leak (or fail)
  nodes alloc or_fail to a
  7 a to node.value
end

$ pnx 08-pool-leak.pnx
08-pool-leak.pnx:8:1: an owned value in a local is not given on every path (close, free or store it)

(4a) A combining word other than each on a pool is refused.

record node
  value int
end
pool nodes node 4
word live (gives n int)
  nodes do (r nodes) true end count
end

$ pnx 08-pool-walk.pnx
08-pool-walk.pnx:6:22: a pool is walked by each only

Faults

(5) Using a row that was given back is the fault stale. pnx shows the locals of the word below the fault.

record node
  value int
end
pool nodes node 2
global keep nodes
word demo (or fail)
  nodes alloc or_fail to a
  a set keep                                 rem stored: the row is handed over
  keep nodes free
  keep node.value drop                      rem the row was given back: stale
end
demo or_else (e) end

$ pnx 08-pool-stale.pnx
08-pool-stale.pnx:10: a row or handle was used after it was given back
  a nodes = 4294967297

Diagnostics

8.10 Indexes

(1) index NAME TABLE RECORD.FIELD makes an index that finds rows of a table of records (an array, a list or a pool) by one field. The key field is int or str(N). An index is a hash table of fixed size, matched to the table’s capacity, that holds only row numbers; the data stay in the table.

(2) key INDEX lookup gives the row with that key (or fail, err.not_found when there is none). row INDEX insert puts a row in (or fail; it fails when the key is there already) and row INDEX remove takes it out. To change a key field, remove the row, change it and insert it again.

(2a) key INDEX has gives true when a row with that key is in the index, else false. It does not fail.

(3) Rows written with row at load are in the index. Rows added while running with add_row are found only after insert.

use io
use str
record user
  name str(16)
  age u8
end
list users user 8
index by_name users user.name
row users ("kim", 30)
row users ("lee", 25)
word find_age (name str gives age int or fail) name by_name lookup or_fail user.age raw end
"lee" find_age or_else (e) -1 end str.to_str io.println
"park" find_age or_else (e) -1 end str.to_str io.println

$ pnx 08-index.pnx
25
-1

use io
use str
record user
  name str(16)
  age int
end
list users user 4
index by_name users user.name
word add_user (name str, age int, indexed bool or fail)
  users add_row or_fail to r
  name r to user.name
  age r to user.age
  indexed if r by_name insert or_fail end  rem a row added while running is indexed by insert
end
word age_of (name str gives int or fail) name by_name lookup or_fail user.age end
"kim" 30 true add_user or_else (e) end
"lee" 25 false add_user or_else (e) end
"kim" age_of or_else (e) -1 end str.to_str io.println
"lee" age_of or_else (e) -1 end str.to_str io.println     rem not inserted: not found

$ pnx 08-index-insert.pnx
30
-1

(3a) The table of an index may be a pool (8.9). Then lookup gives the row as it is now, with its generation; a row given to insert or remove after it was given back is the fault stale. free takes the row out of every index over its pool first. A row alloc has just given is handed over to insert (12.4): when insert fails, the row goes back to the pool before the failure passes on.

use io
record user
  name str(16)
  age int
end
pool users user 4
index by_name users user.name
word add_user (name str, age int or fail)
  users alloc or_fail to r
  name r to user.name
  age r to user.age
  r by_name insert or_fail                rem hands the row over to the index
end
word forget (name str)
  name by_name lookup or_else (e) return end to r
  r users free                            rem out of by_name too
end
"kim" 30 add_user or_else (e) end
"lee" 25 add_user or_else (e) end
"kim" 41 add_user or_else (e) cat ("kim again: ", e err_text) io.println end
cat ("kim: ", "kim" by_name has, ", park: ", "park" by_name has) io.println
"kim" forget
cat ("kim after free: ", "kim" by_name has) io.println
users do (r users) cat (r user.name, " ", r user.age) io.println end each

$ pnx 08-index-pool.pnx
kim again: failed
kim: true, park: false
kim after free: false
lee 25

(4) The hash is keyed (SipHash-2-4); the host gives the key (19.2). A host that indexes text from outside should give a random key: whoever knows the key can pile up collisions and make lookups slow.

Constraints

(5) An index over a pool is not shared (8.13), as a pool is not.

record user
  age int
end
pool users user 4
shared index by_age users user.age

$ pnx 08-index-pool-shared.pnx
08-index-pool-shared.pnx:5:21: an index over a pool is not shared, as a pool is not

Diagnostics

8.11 Forward declarations

Syntax

forward = "forward" ( "array" | "list" | "pool" ) NAME
        | "forward" "word" NAME "(" effect ")" ;

(1) forward list NAME announces a table declared further on. It is for a record that holds rows of the table that holds it. The name of an announced table may be used only where a type goes. forward word is in 9.7.

use io
use str
forward list nodes                         rem the record below points into this list
record node
  value int
  next nodes
end
list nodes node 4
row nodes b (2)
row nodes a (1, b)
a node.next node.value str.to_str io.println
a node.next node.next is_none if "the chain ends at b" io.println end

$ pnx 08-forward.pnx
2
the chain ends at b

Constraints

(2) An announced table shall be declared in the same module with the same kind.

forward list nodes
record node
  value int
end

$ pnx 08-forward-missing.pnx
08-forward-missing.pnx:5:1: a table announced by forward is never declared

8.12 Bound storage

(1) global, array and buffer with bound in front get their storage from the host. Scripts use them like other globals. Until the host binds them they use room inside the state. 19.7 says how to bind.

8.13 Shared storage

(1) global, array, list, buffer and index with shared in front only read another state’s storage. The declaring state has no storage for them; the host gives it the same declaration of another state with pinax_share (19.7). Many states may read the same data at once, on many threads, with no copy and no lock.

Constraints

(2) Nothing changes a shared global: storing, set, push, pop, clear, truncate, add_row, storing into a field, insert, remove, the words that append to a buffer, and a map_into or filter_into destination are all refused. The processor also refuses, at load, bytecode that changes a shared global, and ends with the fault state bytecode that would change a shared global it was given as a parameter.

record city
  name str(12)
  people int
end
shared list cities city 1000
word grow (r cities) r city.people 1 add r to city.people end

$ pnx 08-shared-write.pnx
08-shared-write.pnx:6:47: a shared global is read-only: nothing here may change it

(3) A shared collection may be passed as a collection parameter (9.9) of a word that only reads that parameter. A word changes its parameter when it uses an instruction that changes it, or passes it to a word that changes that parameter or to a host word’s buffer parameter. The processor decides once the whole source is read, so the same holds for a word announced with forward and for a word of an earlier load.

use io
use str
shared list scores int 100
word total (l list int gives int) l sum end       rem reads its parameter only
word everyone (gives int) scores total end        rem so a shared list may be given to it
scores total str.to_str io.println                rem not given yet: empty

$ pnx 08-shared-param.pnx
0

shared list scores int 100
word reset (l list int) l clear end
word keep (l list int) l reset end                rem gives it to a word that changes it
word start () scores keep end

$ pnx 08-shared-change.pnx
08-shared-change.pnx:4:22: a shared collection is read-only: the word it is given to changes this parameter

(3a) A shared collection is not passed as a host word’s buffer parameter or as a parameter of the word a block value names (call), since what the receiver does is not known at load. redefine cannot make a word change a parameter it only read.

(4) shared does not stand before queue, pool, tasks, const or record.

shared queue jobs int 8

$ pnx 08-shared-queue.pnx
08-shared-queue.pnx:1:8: shared stands before global, array, list, buffer or index

Semantics

(5) Until the host gives it, it is empty: lists and arrays have length 0, an index finds nothing, a scalar is 0 and a buffer has no bytes. The code of a load runs as usual. Runs of the host are refused until every shared global is given (19.7).

8.14 Failure codes

(1) err NAME makes a failure code: a new error code (11.1), of type err. Like a constant (8.2) it has no storage, and its name is a name of its module: another module writes module.NAME, when it is exported. It is used where any code is: with fail, with or_else (e) and eq, as a case value, in an err global or collection. err_text gives its name.

(2) The failure codes a state declares are numbered from 0x400 in the order they are declared (19.1). A program does not depend on the number; a host reads the name (pinax_error_text).

use io
use str
err key_too_long
err value_too_long
word check (k str, v str or fail)
  k str.len 8 gt if key_too_long fail end
  v str.len 16 gt if value_too_long fail end
end
word try (k str, v str)
  k v check or_else (e)
    e case
      key_too_long of "the key is too long" end
      value_too_long of "the value is too long" end
      otherwise e err_text end
    end io.println
    return
  end
  "stored" io.println
end
"name" "value" try
"a long key name" "value" try
"name" "a value of thirty bytes, more" try
key_too_long err_text io.println

$ pnx 08-err.pnx
stored
the key is too long
the value is too long
key_too_long

Constraints

(3) err is followed by a name, of at most 31 bytes. The name follows 8.1 (3) and (6).

err "too long"

$ pnx 08-err-name.pnx
08-err-name.pnx:1:5: err is followed by the name of a new failure code

Diagnostics