DataFrameVisualizer.kt

src/jvmMain/kotlin/afield/ui/visualization/DataFrameVisualizer.kt · 190 lines · cited by DataFrames in a Box

package afield.ui.visualization

import afield.app.diagnostics.named
import afield.data.table.div
import afield.data.table.get
import afield.data.table.minus
import afield.data.table.plus
import afield.data.table.rem
import afield.data.table.times
import afield.document.graph.box
import afield.document.graph.nested
import afield.runtime.execution.plus
import afield.runtime.lifecycle.frames
import afield.runtime.lifecycle.plus
import afield.runtime.lifecycle.rem
import afield.runtime.lifecycle.then
import afield.ui.notation.with

import afield.app.protocol.put
import afield.audio.synth.*
import afield.graphics.vulkan.*
import afield.media.stream.*
import afield.media.video.*
import afield.runtime.lifecycle.map
import afield.ui.output.column
import org.jetbrains.kotlinx.dataframe.DataColumn
import org.jetbrains.kotlinx.dataframe.DataFrame
import org.jetbrains.kotlinx.dataframe.DataRow
import org.jetbrains.kotlinx.dataframe.api.GroupBy

Makes Kotlin DataFrame values render as Field values (.h, .h2, display and box return values) without DataFrame having to know Field exists.

DataFrame is a third-party interface, so it cannot implement ToHTML the way Table and Img do. Both registries exist for exactly this: ValuePrinterRegistry for the compact .h form, and H2ValueRegistry for .h2. Everything here is a thin adapter onto primitives that already exist, so a frame gets the same styling, the same expander and the same paging as every other value.

The compact .h form

table already renders a column-oriented table in Field's output style, so a frame's compact form is that table over the first HEAD_ROWS rows, plus a line giving the shape. Only the head is touched: each column is indexed for its first HEAD_ROWS cells rather than the frame being materialised, so printing a million-row frame costs about what printing ten rows costs.

The .h2 scroller

H2ValueSnapshot adapts one value into another for the lazy renderer rather than producing HTML. So .h2 on a frame adapts it into a private list-of-rows view that reads a row only when asked, and the h2 explorer (h2explorer.kt, reached as an iframe onto /h2-explorer?root=<token> and fed over the /ws __h2_explorer__ channel) streams windows of it by index. The server holds the frame rather than a rendering of it behind an .h2 token, and each scroll materialises only the window being looked at, which is what makes a large frame a scroller instead of a large HTML dump. ValueVisualizer's /lazy-value-fragment renderer is a separate path, for expanding a value in place rather than scrolling one.

Going through the registries rather than a bespoke widget also means a frame nested inside another value, a list of frames or a data class with a frame field, expands with the same machinery.

Where the adapters get registered

install is idempotent and runs during server start-up. A bare jvm-main harness does not run that, so tests call it explicitly; DataFrameVisualizerSmokeTest is the worked example.

object DataFrameVisualizer

The registration point, and the four renderers behind it: a frame, a single column, a single row and a grouped frame, each as one HTML fragment.

object DataFrameVisualizer {

const val HEAD_ROWS: Int = 10

Rows shown in the compact .h form. .h2 streams the whole frame instead.

    @Volatile
    private var installed = false

fun install()

Register every DataFrame renderer. Idempotent, and safe to call from any thread.

    @Synchronized
    fun install() {
        if (installed) return
        installed = true

        ValuePrinterRegistry.register(DataFrame::class.java) { frame -> frameHead(frame) }
        ValuePrinterRegistry.register(DataColumn::class.java) { column -> columnHead(column) }
        ValuePrinterRegistry.register(DataRow::class.java) { row -> rowFacts(row) }
        ValuePrinterRegistry.register(GroupBy::class.java) { grouped -> groupBySummary(grouped) }

        // `.h2` on a frame streams every row; the other three are small enough that their compact
        // form is already the whole story, so they fall through to the default lazy rendering.
        H2ValueRegistry.register(DataFrame::class.java) { frame ->
            H2ValueSnapshot(
                value = LazyRows(frame),
                summary = shape(frame),
                title = "DataFrame"
            )
        }

        // …and say that out loud, because since `H2Reflect` the sentence above stopped being true on
        // its own. The reflective default claims any type it can read two properties from, and
        // `DataColumn`/`GroupBy` are neither platform types nor adapter-served, so `.h2` on a column
        // showed three reflected properties instead of `columnHead`. Measured on
        // `ValueColumnImpl`/`GroupByImpl`: both `isExplorableObject` before this call and not after.
        // `DataRow` needs no entry — reflection finds zero readable properties on `DataRowImpl`, so
        // it fails the two-property floor by itself — but it is named here anyway, because relying on
        // a third-party implementation detail to enforce our own decision is how this broke once.
        listOf(DataColumn::class.java, DataRow::class.java, GroupBy::class.java)
            .forEach { H2Reflect.refuseReflection(it) }
    }

12 rows × 3 columns, the line worth having above any table.

    private fun shape(frame: DataFrame<*>): String =
        "${frame.rowsCount()} rows × ${frame.columnsCount()} columns"

The head of a frame as a Table, plus its shape.

take(HEAD_ROWS) per column rather than materialising the frame: this is the form that shows up in a box output on every execution, so it has to stay cheap for a frame far larger than the screen.

    private fun frameHead(frame: DataFrame<*>): String {
        val columns = frame.columns()
        if (columns.isEmpty()) return "<span style='color: $TABLE_MUTED;'>empty DataFrame</span>"

        val head: List<Pair<String, List<Any?>>> =
            columns.map { column -> column.name() to headCells(column) }
        val rendered = table(*head.toTypedArray()).toHTML()
        val hidden = frame.rowsCount() - HEAD_ROWS
        val note = if (hidden > 0) " — first $HEAD_ROWS; use <code>.h2</code> to scroll all of them" else ""
        return rendered + "<div style='color: $TABLE_MUTED; font-size: 90%;'>${shape(frame)}$note</div>"
    }

The first HEAD_ROWS cells of a column, by index.

Indexing rather than take: DataFrame declares its own take on a column and Kotlin declares one on Iterable, so the call is ambiguous — and indexing is what actually guarantees only the head is touched, which is the point for a frame far larger than the screen.

    private fun headCells(column: DataColumn<*>): List<Any?> {
        val n = minOf(HEAD_ROWS, column.size())
        return (0 until n).map { column[it] }
    }

A single column: its name and element type, then its head.

    private fun columnHead(column: DataColumn<*>): String {
        val cells: Pair<String, List<Any?>> = column.name() to headCells(column)
        val rendered = table(cells).toHTML()
        val hidden = column.size() - HEAD_ROWS
        val note = if (hidden > 0) " — first $HEAD_ROWS of ${column.size()}" else ""
        return rendered +
            "<div style='color: $TABLE_MUTED; font-size: 90%;'>${column.type()}$note</div>"
    }

A row reads as label/value pairs, which is what facts is for.

    private fun rowFacts(row: DataRow<*>): String =
        facts(row.df().columns().map { column -> column.name() to row[column.name()] }).toHTML()

A grouped frame: how many groups, keyed by what.

    private fun groupBySummary(grouped: GroupBy<*, *>): String {
        val keys = grouped.keys
        return facts(
            "groups" to keys.rowsCount(),
            "keyed by" to keys.columnNames().joinToString(", "),
        ).toHTML() + keys.let { frameHead(it) }
    }

A frame seen as its rows, materialised one at a time.

This is what turns .h2 into a scroller rather than a dump. The lazy visualizer pages a list by index, so holding the frame and answering get on demand means only the rows actually being looked at are ever built — the whole point when the frame is larger than memory would like.

    private class LazyRows(private val frame: DataFrame<*>) : AbstractList<Map<String, Any?>>() {
        private val names = frame.columns().map { it.name() }

        override val size: Int get() = frame.rowsCount()

        override fun get(index: Int): Map<String, Any?> {
            val row = frame[index]
            return LinkedHashMap<String, Any?>(names.size).apply {
                names.forEach { name -> put(name, row[name]) }
            }
        }
    }
}