Lossy decoding and diagnostics

Unknown widget types and malformed payloads are dropped with a key path. The rest of the page still renders. That sentence is the production difference between a server driven UI stack you can leave in an old binary and a decoder that treats one bad promo as a failed session.

Synthesized Decodable arrays are all or nothing. One broken element throws, the section disappears, and if that throw escapes to PageStore the user sees the failed state instead of the home page. Flow-UI does not decode widget lists that way. LossyArray keeps what it can, records what it cannot, and lets the page continue. DecodeDiagnostics is the log of those decisions.

The concept page is Resilience. This article is the four Kind values, where they fire, and how a host should test them. Schema versioning and unknown policy defaults are the next post. Identity collisions are the post after that. Here we stay on decode.

LossyArray is the list primitive#

LossyArray<Element> walks an unkeyed container. Each decode(Element.self) either appends or records droppedElement and consumes the failed fragment as JSONValue so the index advances. If even JSONValue fails, the loop stops rather than spinning.

while !container.isAtEnd {
    do {
        result.append(try container.decode(Element.self))
    } catch {
        diagnostics?.record(
            .droppedElement,
            widgetType: nil,
            codingPath: container.codingPath,
            message: describeDecodingError(error)
        )
        if (try? container.decode(JSONValue.self)) == nil {
            break
        }
    }
}

Diagnostics come from decoder.userInfo[.flowDiagnostics]. If no collector is attached, drops are still drops. You just will not see them in the overlay. PageStore always attaches a fresh DecodeDiagnostics per page decode.

let diagnostics = DecodeDiagnostics()

LossyArray is not only for widgets. Sections on a page, buttons in a button_row, tags in a tag_rail, mutations in an api response: all of those lists use the same type. One malformed button does not empty the row. One unknown mutation kind throws inside MutationInstruction, the lossy list swallows it, and the rest of the mutations apply.

What still throws#

A page document that is not JSON, or that cannot decode the envelope keys the decoder needs to continue, can still fail the request. Lossy behavior is for elements and for widget content problems, not for “this is not a page.” PageStore maps a thrown DecodingError to a failed state with describeDecodingError. That is the right outcome for garbage bytes. It is the wrong outcome for a single unknown type in an otherwise fine home envelope, and that case does not throw.

Four diagnostic kinds#

DecodeDiagnostics.Kind is a small enum. Treat it as the taxonomy your telemetry should keep.

Each Entry carries an id (a UUID for Identifiable in the overlay), kind, optional widgetType, a dotted codingPath, and a readable message. describeDecodingError turns keyNotFound, typeMismatch, valueNotFound, and dataCorrupted into one-line strings with a path.

unknownType#

AnyWidget asks WidgetDecoding.decodeContent. nil means nothing is registered. Diagnostics record unknownType with the type string and path. Content becomes UnknownWidgetContent, which preserves the raw data fragment for tooling. Rendering is a policy decision on the registry, not a throw.

malformedPayload#

The type is known but V.Content threw. Diagnostics record malformedPayload with describeDecodingError. Content becomes MalformedWidgetContent with that message. Debug builds surface it in red. The page does not fail.

droppedElement#

This is the lossy array path: a child that never became a value. A typeless object in a widget list, a structurally invalid fragment, an unknown mutation kind. widgetType on the entry may be nil because there was no widget to name.

duplicateID#

Not a decode failure of JSON shape. After the page exists, makeWidgetIdentifiersUnique() claims ids. A collision records duplicateID and rewrites the later id. Rendering stays correct. Mutations targeting the original id are now ambiguous. Send unique ids from the backend.

How hosts should attach and read diagnostics#

FlowDecoder.make(widgetDecoding:diagnostics:) writes both into JSONDecoder.userInfo. PageStore.decodePage creates a fresh collector per fetch so a refresh does not append to last week’s noise. The debug overlay in FlowRender reads store.diagnostics.entries.

Do not only look at the overlay. In production, copy entries into your logger after a successful load: kind, type, path, message. That is how you learn the backend shipped hologram_projector to a binary that does not have it, or that title went missing on a campaign row.

Reset is explicit (diagnostics.reset()). The store replaces the object instead on each page decode. Tests can allocate a collector, decode a fixture with FlowDecoder.make, and assert on entries.

Testing lossy behavior#

A useful fixture has five widgets: two healthy, one unknown type, one missing required key, one missing type entirely. Expect two rendered widgets, two containment entries (unknownType and malformedPayload), and one droppedElement if the typeless object never became AnyWidget. The playground Resilience preset is this idea. Host tests should own a copy so CI fails when someone “simplifies” decode back to a throwing array.

Assert that a malformed button_row button does not empty buttons. Assert that a bad section does not empty page.sections. Those are LossyArray guarantees at different layers.

What lossy does not mean#

It does not mean “parse any JSON into a UI.” The envelope still has a shape. Widgets still have WidgetContent models. Unknown keys on a struct are ignored by Decodable; unknown types are contained; unknown required keys are malformed.

It does not mean mutations are best-effort in a fuzzy way. An api response still decodes as ActionResponse. Each mutation instruction is lossy. Applying a well-formed replace_widget is exact.

It does not mean you can skip id hygiene. Duplicate ids are recovered, not blessed. The diagnostic exists because recovery is a compromise.

It does not mean Flow-UI will fetch a better payload. There is no networking in the package. If the document is stale or truncated, that is PageLoader and the backend.

Overlay versus production logs#

The debug overlay in FlowRender is for a developer holding a phone. Production needs the same four kinds in your logger: unknownType, malformedPayload, droppedElement, duplicateID, plus widgetType, coding path, and message. A Friday campaign that omits title should show up as a count of malformedPayload on image_text_card, not as a spike in failed page loads.

Do not sample diagnostics down to nothing. One unknown type per session is cheap. A thousand droppedElement rows on a single page is a backend array that is not JSON. Alert on volume, not on the mere existence of an entry. Unknown types are normal when you ship a new widget to old binaries. Duplicate ids are never normal.

Where this sits#

Resilience is a feature, not a fallback. Old binaries will see new type strings. Campaign JSON will omit a field on Friday. How to implement SDUI on iOS listed containment as a required checklist item for that reason. When not to use server driven UI still applies: lossy decode will not make a map canvas a good JSON problem. It will keep a merchandising page alive when one promo is wrong.

Log the four kinds. Keep lists lossy. Never fail the page for a single widget.

Read next#

• Resilience
• How to implement SDUI on iOS
• Starter widgets as worked examples
• When not to use server driven UI

Decode what you can. Record the rest with a path. Render the page.