90 Strings and text

90.1 Two types, one rule

The string vocabulary is only two.

• proven_u8str_t — an owning string. It has a buffer, a length and a capacity.
• proven_u8str_view_t — a borrowed string. Only a pointer and a length.

If the name has view in it, it is borrowed, and what is borrowed is not destroyed. The rule set up in chapter 89 applied to strings as it is. u8 means UTF-8 — that encoding seen in chapter 9 is this library’s default text representation.

Open them up and why they were divided in two becomes clear.

/* the borrowed string — the same shape as chapter 88's mem_view_t */
typedef struct {
    const proven_byte_t *ptr;
    proven_size_t        size;
} proven_u8str_view_t;

/* the owning string — one buffer and an "is it borrowed" flag */
typedef struct {
    proven_buf_t internal;   /* { ptr, len, cap } */
    bool         borrowed;
} proven_u8str_t;

The three slots of proven_buf_t state the string’s character as they are. len is the number of bytes of content held now, cap is the whole buffer’s size, and the difference between them is the spare room including the NUL’s place. So create(alloc, 64) really takes 65 bytes — 64 is the limit of the content and one byte is the NUL’s share. Thanks to that one byte proven_u8str_as_cstr can hand out a C string with neither copying nor allocation.

The borrowed flag marks a string made with _borrow. When this flag is on, _destroy releases nothing and merely empties the struct — somebody else’s memory cannot be given back.

There are three roads to obtaining a string.

Only beware that _borrow’s cap is the whole capacity including the NUL — give it buf[64] and the content goes to 63 bytes.

Making a view from a literal is done with one macro.

PROVEN_LIT("hello")        /* becomes { ptr, 5 } at compile time — no strlen */
proven_u8str_view_from_cstr(p)   /* counts the length at run time */

PROVEN_LIT draws the length from sizeof("...") - 1, so its run-time cost is zero. Using it for string literals is the practice, and _from_cstr for a C string whose length is unknown.

#include <proven.h>
#include <stdio.h>

static void show(const char *label, proven_u8str_view_t v)
{
    printf("%-12s [%.*s] (%zu bytes)\n", label, (int)v.size, (const char *)v.ptr, v.size);
}

int main(void)
{
    /* (1) a string without allocation: a stack buffer is borrowed.
          A signature with no allocator = it does not allocate */
    proven_byte_t buf[16];
    proven_u8str_t s = proven_u8str_borrow(buf, sizeof buf);

    proven_err_t e = proven_u8str_append(&s, proven_u8str_view_from_cstr("hello"));
    printf("append 'hello' : %s\n", proven_is_ok(e) ? "ok" : "refused");
    show("content", proven_u8str_as_view(&s));

    /* (2) if there is not enough room it refuses — it does not truncate */
    e = proven_u8str_append(&s, proven_u8str_view_from_cstr(" world, and more"));
    printf("append long    : %s (err=%d)\n",
           proven_is_ok(e) ? "ok" : "refused", (int)e);
    show("unchanged", proven_u8str_as_view(&s));   /* failure atomicity */

    /* (3) finding and slicing — all on views, with no copying */
    proven_u8str_view_t csv = proven_u8str_view_from_cstr("name,age,city");
    proven_u8str_view_t comma = proven_u8str_view_from_cstr(",");

    proven_size_t at = proven_u8str_view_find(csv, 0, comma);
    printf("first comma at : %zu\n", at);
    show("field 1", proven_u8str_view_slice(csv, 0, at));

    /* (4) sweeping by a separator: when none is found the sentinel (PROVEN_INDEX_NOT_FOUND) comes back */
    proven_size_t start = 0;
    int n = 0;
    for (;;) {
        proven_size_t hit = proven_u8str_view_find(csv, start, comma);
        proven_size_t end = (hit == PROVEN_INDEX_NOT_FOUND) ? csv.size : hit;
        char label[24];
        snprintf(label, sizeof label, "field %d", ++n % 100);
        show(label, proven_u8str_view_slice(csv, start, end - start));
        if (hit == PROVEN_INDEX_NOT_FOUND) break;
        start = hit + 1;
    }

    /* (5) comparison and a prefix test */
    printf("eq 'name,age,city' : %d\n",
           proven_u8str_view_eq(csv, proven_u8str_view_from_cstr("name,age,city")));
    printf("starts with 'name' : %d\n",
           proven_u8str_view_starts_with(csv, proven_u8str_view_from_cstr("name")));
    return 0;
}

append 'hello' : ok
content      [hello] (5 bytes)
append long    : refused (err=2)
unchanged    [hello] (5 bytes)
first comma at : 4
field 1      [name] (4 bytes)
field 1      [name] (4 bytes)
field 2      [age] (3 bytes)
field 3      [city] (4 bytes)
eq 'name,age,city' : 1
starts with 'name' : 1

The first line shows chapter 89′s rule again. proven_u8str_borrow has no allocator — therefore it does not allocate. It is doing string operations on a 16-byte array taken on the stack, and this is exactly how strings are handled in embedded work without a heap.

90.2 Refuse, rather than truncate

The second append is the most important single line of this part. On trying to attach " world, and more" after "hello" in a 16-byte vessel, the library wrote nothing and returned PROVEN_ERR_OUT_OF_BOUNDS. And as the next line shows, the original content hello stands as it was — the failure atomicity learned in chapter 87.

It is the exact opposite of snprintf’s choice seen in chapter 85. Why not truncate?

Because a truncated value is not a short value but a different value. A truncated path points at a different file, a truncated command is a different command, a truncated user name is a different person. Truncate and declare success and the caller cannot know whether what was received is what was requested. So this library gives the decision back to the caller — “there is not enough room. What shall we do?”

proven_u8str_t s = proven_u8str_borrow(buf, sizeof buf);
proven_err_t e = proven_u8str_append_grow(alloc, &s, view);  /* dangerous */

90.3 Three kinds of writing — refuse, truncate, grow

This library’s string operations are made so that the kind can be known from the name alone. That kind is “what happens when there is not enough room”.

All three kinds are needed because the right answer differs by place. Where a file path is being built truncation must not happen, so refusing is right; where a log line is fitted to the screen truncating is right; where the length is unknown growing is right. That the default (the short name) is refusal shows this design’s attitude.

90.4 The operations that mend a string

Appending alone is not enough. There are operations that mend the middle, delete, and rewrite.

/* The operations that change a string — inserting, removing, replacing and
   rewriting. Every one of them "does it if it fits, and if it does not, refuses
   and leaves the original as it was". */
#include <proven.h>

static void show(const char *label, const proven_u8str_t *s)
{
    proven_u8str_view_t v = proven_u8str_as_view(s);
    proven_println("{:<16} \"{}\" (len {})",
                   PROVEN_ARG(label), PROVEN_ARG(v), PROVEN_ARG(v.size));
}

int main(void)
{
    proven_allocator_t alloc = proven_heap_allocator();

    /* ── (1) made straight from a view ───────────────────────────── */
    proven_result_u8str_t r =
        proven_u8str_create_from_view(alloc, PROVEN_LIT("hello world"));
    if (!proven_is_ok(r.err)) return 1;
    proven_u8str_t s = r.value;
    show("just made", &s);

    /* ── (2) inserted in the middle ──────────────────────────────── */
    proven_err_t e = proven_u8str_insert_grow(alloc, &s, 5, PROVEN_LIT(","));
    show("insert(5, \",\")", &s);

    /* ── (3) a range replaced — the lengths need not match ───────── */
    e = proven_u8str_replace_at_grow(alloc, &s, 7, 5, PROVEN_LIT("proven C"));
    show("replace_at", &s);

    /* ── (4) the first occurrence replaced ───────────────────────── */
    e = proven_u8str_replace_first(&s, 0, PROVEN_LIT("hello"), PROVEN_LIT("HELLO"));
    show("replace_first", &s);

    /* asked to replace something that is not there, it does nothing and succeeds */
    e = proven_u8str_replace_first(&s, 0, PROVEN_LIT("zzz"), PROVEN_LIT("!"));
    proven_println("replacing what is absent -> err={} (left as it was, and success)",
                   PROVEN_ARG((int)e));

    /* ── (5) a range removed ─────────────────────────────────────── */
    e = proven_u8str_remove(&s, 0, 6);
    show("remove(0,6)", &s);

    /* ── (6) rewriting — the buffer stays, only the content is emptied ─ */
    e = proven_u8str_reset(&s);
    show("after reset", &s);

    e = proven_u8str_append(&s, PROVEN_LIT("written again"));
    show("after refilling", &s);
    proven_println("(reset is for rewriting without reallocating — for code that rebuilds every frame)");

    /* ── (7) reserved in advance ─────────────────────────────────── */
    e = proven_u8str_reserve(alloc, &s, 256);
    proven_println("reserve(256)      -> err={} (worth most on an arena)",
                   PROVEN_ARG((int)e));

    /* ── (8) the query operations ────────────────────────────────── */
    proven_u8str_view_t v = proven_u8str_as_view(&s);
    proven_println("starts_with(\"written\")={} ends_with(\"again\")={} find(\"ten\")={}",
                   PROVEN_ARG((bool)proven_u8str_view_starts_with(v, PROVEN_LIT("written"))),
                   PROVEN_ARG((bool)proven_u8str_view_ends_with(v, PROVEN_LIT("again"))),
                   PROVEN_ARG(proven_u8str_view_find(v, 0, PROVEN_LIT("ten"))));

    proven_size_t nf = proven_u8str_view_find(v, 0, PROVEN_LIT("absent"));
    proven_println("when it is not found: {} (PROVEN_INDEX_NOT_FOUND)",
                   PROVEN_ARG((bool)(nf == PROVEN_INDEX_NOT_FOUND)));

    proven_u8str_destroy(alloc, &s);
    return 0;
}

just made        "hello world" (len 11)
insert(5, ",")   "hello, world" (len 12)
replace_at       "hello, proven C" (len 15)
replace_first    "HELLO, proven C" (len 15)
replacing what is absent -> err=0 (left as it was, and success)
remove(0,6)      " proven C" (len 9)
after reset      "" (len 0)
after refilling  "written again" (len 13)
(reset is for rewriting without reallocating — for code that rebuilds every frame)
reserve(256)      -> err=0 (worth most on an arena)
starts_with("written")=true ends_with("again")=true find("ten")=4
when it is not found: true (PROVEN_INDEX_NOT_FOUND)

The starred contract of _replace_first needs care. Not finding it is not an error but a success — to distinguish “there was nothing to replace” from “it was replaced” you must first check with proven_u8str_view_find. That the example’s “replacing what is not there” comes back as err=0 is that confirmation.

_reset and _reserve are a pair for performance. In code that builds a string afresh every frame or per request, reusing the buffer instead of throwing it away is _reset, and taking it in advance when you roughly know how large it will grow is _reserve. As seen in chapter 89, _reserve pays especially on an arena — take it large before another allocation intervenes and growth in place becomes possible.

90.5 Finding and cutting — text handling without copying

The example’s ③ and ④ are a view’s real usefulness. Obtaining a substring needs no copying — it merely calculates a new pointer into the original and a new length. While one CSV line was divided into three fields, not one allocation happened.

This pattern is especially powerful in parsers. When parsing a line with sscanf in chapter 25 a buffer had to be prepared in advance to hold the result, whereas cutting with views leaves only marks upon the original. In exchange one more thing must be kept — as seen in chapter 88, it is valid only while the original is alive.

There is a convention too in the value find returns when it does not find. It is not 0 or a negative number but a sentinel with a name, PROVEN_INDEX_NOT_FOUND. Chapter 85 spoke of the danger of sentinel values, and what differs here is that it has a name and is documented — a nameless magic number and a named contract are different things.

90.6 The boundary of two worlds — NUL termination and UTF-16

Conversion is needed at every place where the outside world is met.

• proven_u8str_as_cstr — obtains a NUL-terminated pointer from an owning string. A NUL being kept internally, there is neither copying nor allocation. It is used when passing to printf("%s") or to a file API.
• proven_u8str_view_to_cstr — makes a NUL-terminated string from a view. This one takes an allocator — because a view may point into the middle of the original and a NUL cannot be written in that place. The signature states the fact once again.
• proven_u16str_t / proven_u16str_view_t — the bridge to the UTF-16 world. The Windows API uses this encoding so conversion is needed, and conversion can fail (unpaired surrogates and the like). So the result comes as a bundle.

We can now hold and cut strings safely. But the work of making them remains — turning numbers and values into letters, and turning letters back into numbers. It is the place where chapter 85′s third bug waits, and the place where this library’s most conspicuously different syntax appears.