Imaging

BitmapBrowserSurface.kt

A browser-frame receiver that uploads CPU pixels to the GPU, as a fallback where Chromium's shared-texture export delivers an empty frame.

src/jvmMain/kotlin/afield/media/browser/BitmapBrowserSurface.kt — read the source

BitmapBrowserSurface

class BitmapBrowserSurface(renderer: Renderer, spec: ExternalBrowserSurfaceSpec, releaseListener: BrowserFrameReleaseListener = BrowserFrameReleaseListener { }) : CopiedExternalBrowserSurface(renderer, spec, releaseListener)

The non-zero-copy browser receiver: frames arrive as CPU pixels and are uploaded, rather than arriving as a shared GPU allocation and being imported.

Declaration	Summary
override fun importFrame(frame: ExternalBrowserFrame): ImportedImage?
override fun recordPreCopy(command: CommandBuffer, source: ImportedImage)
override fun cleanupImports()

CopiedExternalBrowserSurface.kt

The shared base of the native browser-frame receivers, which copies each imported frame into one stable texture for rendering.

src/jvmMain/kotlin/afield/media/browser/CopiedExternalBrowserSurface.kt — read the source

CopiedExternalBrowserSurface

sealed class CopiedExternalBrowserSurface(protected val renderer: Renderer, final override val spec: ExternalBrowserSurfaceSpec, releaseListener: BrowserFrameReleaseListener) : ExternalBrowserSurface

The platform-independent half of all native browser receivers. A backend only imports a frame's native image; this class owns conflation, the stable resident texture, copies, fence-delayed release acknowledgements, and close ordering.

Declaration	Summary
final override val spec: ExternalBrowserSurfaceSpec
final override val imageView: ImageView
final override val imageViewArray: List<HasImageView>
final override val presentedFrame: ExternalBrowserFrame?
final override fun offerFrame(frame: ExternalBrowserFrame): BrowserFrameOfferResult
final override fun prepareSampledImagesFor(command: CommandBuffer)
final override fun close()

ElectronBrowserTextureSession.kt

The Electron process that paints a web page for Field, and the ways a frame of it crosses into this process.

ElectronBrowserTextureSession starts electron/browser-texture-producer.js under the Electron that ./gradlew electronInstall put in the checkout, passes it the page, the surface size and the rest as FIELD_BROWSER_TEXTURE_* environment variables, and talks to it in JSON lines over a loopback socket the session opens for the purpose. All of that is platform-neutral; only the GPU handle transfer is not, and each BrowserHandleTransport here is one way to do it: a Mach send right on macOS, a dma-buf descriptor over SCM_RIGHTS on Linux, and an opt-in fallback that carries the pixels themselves over a second socket for when no handle can cross. The receiver side of the contract is ExternalBrowserSurface, described in dev_docs/external_browser_surfaces.md.

src/jvmMain/kotlin/afield/media/browser/ElectronBrowserTextureSession.kt — read the source

ExternalBrowserSurface.kt

The resident texture that browser paints arrive in: a fixed-size image Field samples like any other, refreshed from frames a producer offers between draws.

ExternalBrowserSurfaceSpec fixes size, pixel format and slot count; a producer calls ExternalBrowserSurface.offerFrame with an ExternalBrowserFrame, and the platform import or copy is recorded in prepareSampledImagesFor before the image is bound. ExternalBrowserSurfaces.create picks the macOS, Windows or Linux backend. Frame hand-off is covered by ExternalBrowserFrameMailboxTest.

src/jvmMain/kotlin/afield/media/browser/ExternalBrowserSurface.kt — read the source

ExternalBrowserSurfaceSpec

data class ExternalBrowserSurfaceSpec(val width: Int, val height: Int, val pixelFormat: BrowserSurfacePixelFormat = BrowserSurfacePixelFormat.BGRA8_SRGB, val slotCount: Int = 3, val deviceScaleFactor: Double = 1.0)

Fixed properties of the resident texture exposed to Field.

Declaration	Summary
val width: Int
val height: Int
val pixelFormat: BrowserSurfacePixelFormat
val slotCount: Int
val deviceScaleFactor: Double

BrowserSurfacePixelFormat

enum class BrowserSurfacePixelFormat

Pixel layouts and transfer functions accepted from accelerated browser producers.

Constants: BGRA8_SRGB, RGBA8_SRGB, BGRA8_UNORM, RGBA8_UNORM, RGBA16_FLOAT.

BrowserSurfaceColorSpace

data class BrowserSurfaceColorSpace(val primaries: String, val transfer: String, val matrix: String, val range: String)

Electron's four color-space fields, preserved without baking its evolving string vocabulary in.

Declaration	Summary
val primaries: String
val transfer: String
val matrix: String
val range: String

BrowserSurfaceRect

data class BrowserSurfaceRect(val x: Int, val y: Int, val width: Int, val height: Int)

ExternalBrowserImageHandle

sealed interface ExternalBrowserImageHandle : AutoCloseable

A receiver-side platform handle. IPC is responsible for turning Electron's process-local handle into one of these values. Each value owns the receiver-local handle(s) until a platform importer either consumes them or closes them. Closing a rejected or superseded frame is therefore enough to keep transport resources bounded.

Declaration	Summary
override fun close()	Releases any live receiver-side transport resource. Implementations must be idempotent.
class MacIOSurfaceMachPort(val value: Int) : ExternalBrowserImageHandle	A Mach port name in Field's process after the producer transferred an IOSurface send right. This is the normal Electron path: Electron's IOSurfaceRef itself is process-local.
data class MacIOSurfaceId(val value: Long) : ExternalBrowserImageHandle	Same-process or explicitly global IOSurfaces only; not the normal Electron transport.
class WindowsD3D11NtHandle(val value: Long) : ExternalBrowserImageHandle
class CpuBitmap(val pixels: java.nio.ByteBuffer, val stride: Int) : ExternalBrowserImageHandle	Pixels that have already been read back to the CPU, for the non-zero-copy fallback. Not a native handle at all: nothing is shared, and the bytes are owned by this value until close frees them.
class LinuxDmaBuf(val planes: List<Plane>, val modifier: Long) : ExternalBrowserImageHandle

ExternalBrowserFrame

data class ExternalBrowserFrame(val streamId: Long, val frameId: Long, val releaseToken: Long = frameId, val slot: Int, val width: Int, val height: Int, val pixelFormat: BrowserSurfacePixelFormat, val handle: ExternalBrowserImageHandle? = null, val colorSpace: BrowserSurfaceColorSpace? = null, val visibleRect: BrowserSurfaceRect? = null, val contentRect: BrowserSurfaceRect? = null, val timestampMicros: Long? = null)

One accelerated browser paint. Stream ids increase whenever the producer is replaced. handle is sticky within one streamId/slot: it may be omitted after the first announcement of that slot, but a new stream must announce its handles again. releaseToken is echoed when the GPU has finished copying the frame, allowing Electron's main process to call OffscreenSharedTexture.release() safely.

Declaration	Summary
val streamId: Long
val frameId: Long
val releaseToken: Long
val slot: Int
val width: Int
val height: Int
val pixelFormat: BrowserSurfacePixelFormat
val handle: ExternalBrowserImageHandle?
val colorSpace: BrowserSurfaceColorSpace?
val visibleRect: BrowserSurfaceRect?
val contentRect: BrowserSurfaceRect?
val timestampMicros: Long?

BrowserFrameReleaseReason

enum class BrowserFrameReleaseReason

Constants: PRESENTED, SUPERSEDED, STALE, INVALID, IMPORT_FAILED, CLOSED.

BrowserFrameRelease

data class BrowserFrameRelease(val streamId: Long, val frameId: Long, val releaseToken: Long, val slot: Int, val reason: BrowserFrameReleaseReason)

The receiving surface was closed or its producer stream was replaced.

Declaration	Summary
val streamId: Long
val frameId: Long
val releaseToken: Long
val slot: Int
val reason: BrowserFrameReleaseReason

BrowserFrameReleaseListener

fun interface BrowserFrameReleaseListener

Declaration	Summary
fun released(release: BrowserFrameRelease)

BrowserFrameOfferResult

enum class BrowserFrameOfferResult

Constants: ACCEPTED, ACCEPTED_REPLACING_PENDING, REJECTED_STALE, REJECTED_INVALID, REJECTED_CLOSED.

ExternalBrowserSurface

interface ExternalBrowserSurface

A browser surface is a stable Field texture: platform textures are copied on-GPU before their producer lease is released, so Chromium may recycle them without changing or corrupting what Field samples between browser paints.

Declaration	Summary
val spec: ExternalBrowserSurfaceSpec
val presentedFrame: ExternalBrowserFrame?
fun offerFrame(frame: ExternalBrowserFrame): BrowserFrameOfferResult	Thread-safe; intended to be called by the IPC reader.
override fun prepareSampledImagesFor(command: CommandBuffer)	Records any pending platform import/copy before Field binds imageView.

ExternalBrowserSurfaces

object ExternalBrowserSurfaces

The platform-neutral construction point; Windows and Linux backends can fill in behind it.

Declaration	Summary
fun create(renderer: afield.graphics.vulkan.Renderer, spec: ExternalBrowserSurfaceSpec, releaseListener: BrowserFrameReleaseListener = BrowserFrameReleaseListener { }): ExternalBrowserSurface

ExternalBrowserVulkanImports.kt

Turning a browser's own GPU allocation into a Vulkan image this process can read: the Windows D3D11 shared-handle import and the Linux dma-buf import.

Both create a VkImage that declares its external handle type, allocate memory dedicated to that image which imports the handle, and bind the two. The image is only ever a transfer source and a sampled texture. The Linux side asks the driver whether it can import that format, DRM modifier and usage before building anything from it, passes each plane's stride and offset as an explicit layout, and requires the planes to share one allocation, which is what the first Linux backend supports. macOS does not come through this file: an IOSurface is imported by MacExternalBrowserSurface. What the receivers do with the result, and how far each platform has got, is in dev_docs/external_browser_surfaces.md.

src/jvmMain/kotlin/afield/media/browser/ExternalBrowserVulkanImports.kt — read the source

FieldUiBrowserTexture.kt

A live web page as a Field texture: a hidden Electron window renders offscreen and its paints become an ExternalBrowserSurface a shader can sample.

FieldUiBrowserTexture is the full PuppetBrowser, so the same object takes navigation, input and JavaScript. On macOS the frames move GPU-side as IOSurfaces, with no bitmap readback.

src/jvmMain/kotlin/afield/media/browser/FieldUiBrowserTexture.kt — read the source

FieldUiBrowserTexture

class FieldUiBrowserTexture(private val renderer: Renderer, val url: String, width: Int = 1280, height: Int = 720, slots: Int = 3, readyBoxUid: String? = null, private val cpuBitmapFallback: Boolean = false) : PuppetBrowser

A live offscreen Chromium view exposed as an ordinary stable Field texture.

Declaration	Summary
val url: String
override val spec: ExternalBrowserSurfaceSpec
override val presentedFrame: ExternalBrowserFrame?
override val state: PuppetBrowserState
val pageReady: Boolean
override val imageView: ImageView
override fun offerFrame(frame: ExternalBrowserFrame): BrowserFrameOfferResult
override fun prepareSampledImagesFor(command: CommandBuffer)
override fun addEventListener(listener: (PuppetBrowserEvent) -> Unit): () -> Unit
override fun navigate(url: String)
override fun goBack()
override fun goForward()
override fun reload(ignoreCache: Boolean)
override fun stop()
override fun focus()
override fun blur()
override fun pointer(event: BrowserPointerEvent)
override fun key(event: BrowserKeyEvent)
override fun insertText(text: String)
override fun setComposition(text: String, selectionStart: Int, selectionEnd: Int, replacementStart: Int?, replacementEnd: Int?)
override fun touch(type: BrowserTouchEventType, points: List<BrowserTouchPoint>, modifiers: Set<BrowserModifier>)
override fun flushInput(): CompletableFuture<Unit>
override fun evaluate(javascript: String, world: BrowserJavascriptWorld, returnMode: BrowserJavascriptReturnMode, userGesture: Boolean, timeoutMillis: Long): CompletableFuture<BrowserJavascriptResult>
override fun releaseJavascriptObject(objectId: String): CompletableFuture<Unit>
override fun cdp(method: String, params: JsonObject, sessionId: String?): CompletableFuture<JsonObject>
override fun forwardCdpEvents(enabled: Boolean): CompletableFuture<Unit>
override fun onPageRequest(name: String, handler: (JsonElement) -> CompletableFuture<JsonElement>): () -> Unit
override fun close()

FieldVrKeyboard.kt

src/jvmMain/kotlin/afield/media/browser/FieldVrKeyboard.kt — read the source

FieldVrKeyboard

class FieldVrKeyboard(private val browser: PuppetBrowserControl, private val recenter: () -> Unit, private val focusKeyboard: () -> Unit = {}) : AutoCloseable

A focus-preserving keyboard inside the offscreen IDE; keys go through Chromium's input path.

Declaration	Summary
fun install(): CompletableFuture<BrowserJavascriptResult>	Idempotent; also called after navigation to a different document.
fun reportInputStatus(status: String): CompletableFuture<BrowserJavascriptResult>
fun press(key: String, shift: Boolean = false, control: Boolean = false, alt: Boolean = false)
override fun close()

FieldVrWorkspace.kt

src/jvmMain/kotlin/afield/media/browser/FieldVrWorkspace.kt — read the source

FieldVrWorkspace

class FieldVrWorkspace(val renderer: Renderer, val url: String, width: Int = 2560, height: Int = 1600, readyBoxUid: String? = null, val widthMeters: Float = 2.0f, val distanceMeters: Float = 1.35f, cpuBitmapFallback: Boolean = FieldUiBrowserTexture.cpuBitmapFallbackRequired, headsetInput: Boolean = defaultHeadsetInput(renderer.openXRHmdTarget != null), pairedInput: Boolean = defaultPairedInput()) : AutoCloseable

A live Field document in a world-locked Chromium panel. Construct after the renderer's initial frame, then drive the renderer normally. The desktop preview uses the same page and input path. The supplied URL should come from FieldUiBrowserTexture.fieldDocumentUrl.

Declaration	Summary
val renderer: Renderer
val url: String
val widthMeters: Float
val distanceMeters: Float
var headsetDevices: WiVRnBrowserInput?
var keyboardReady
var keyboardError: String?
val browser
val stereo: MultiviewStereoTarget
val panel: PuppetBrowserVrPanel
val input: PuppetBrowserVrInput
val grab: PuppetBrowserVrGrab
val keyboard
val companion: PuppetBrowserWindowPanel
fun recenter()	Keep the panel in the room; call this to bring it in front of the current head position.
override fun close()

HeadlessPuppetBrowser.kt

A Chromium tab with the full browser control plane and no picture, for driving or scraping a page from a box without a renderer.

HeadlessPuppetBrowser uses the same Electron producer and protocol as FieldUiBrowserTexture, started with frames dropped at the source. HeadlessPuppetBrowserTest exercises it.

src/jvmMain/kotlin/afield/media/browser/HeadlessPuppetBrowser.kt — read the source

HeadlessPuppetBrowser

class HeadlessPuppetBrowser(val url: String = "about:blank", width: Int = 1280, height: Int = 720, deviceScaleFactor: Double = 1.0, val framesPerSecond: Int = 8, projectRoot: File = locateFieldProjectRoot()) : PuppetBrowserControl

A Chromium tab with no picture: the whole PuppetBrowserControl plane — navigation, pointer, keyboard, IME, touch, promise-aware JavaScript, the isolated world, arbitrary CDP and the allowlisted page→Field request bridge — with no afield.graphics.vulkan.Renderer, no Vulkan surface, no native handle bridge and no readback socket.

Declaration	Summary
val url: String	Loaded as the producer starts; loaded completes when it has finished.
val framesPerSecond: Int	Offscreen frames per second. Nothing consumes them, but Chromium only acknowledges a dispatched mouseMoved on the renderer's next frame, so this is the pointer's latency floor: at 1 a mouse move costs a second. Raise it if you are driving a page, drop it to 1 if you are only reading one.
val viewport	Viewport size in CSS pixels. Nothing samples it; scripts and media queries still see it.
override val state: PuppetBrowserState
val loaded: CompletableFuture<PuppetBrowserState>	Completes when the tab first reaches did-finish-load. navigate is the barrier for every later page; this one exists because the constructor's url is loaded before you can call anything.
override fun addEventListener(listener: (PuppetBrowserEvent) -> Unit): () -> Unit
override fun navigate(url: String)
override fun goBack()
override fun goForward()
override fun reload(ignoreCache: Boolean)
override fun stop()
override fun focus()
override fun blur()
override fun pointer(event: BrowserPointerEvent)
override fun key(event: BrowserKeyEvent)
override fun insertText(text: String)
override fun setComposition(text: String, selectionStart: Int, selectionEnd: Int, replacementStart: Int?, replacementEnd: Int?)
override fun touch(type: BrowserTouchEventType, points: List<BrowserTouchPoint>, modifiers: Set<BrowserModifier>)
override fun flushInput(): CompletableFuture<Unit>
override fun evaluate(javascript: String, world: BrowserJavascriptWorld, returnMode: BrowserJavascriptReturnMode, userGesture: Boolean, timeoutMillis: Long): CompletableFuture<BrowserJavascriptResult>
override fun releaseJavascriptObject(objectId: String): CompletableFuture<Unit>
override fun cdp(method: String, params: JsonObject, sessionId: String?): CompletableFuture<JsonObject>
override fun forwardCdpEvents(enabled: Boolean): CompletableFuture<Unit>
override fun onPageRequest(name: String, handler: (JsonElement) -> CompletableFuture<JsonElement>): () -> Unit
override fun close()

LinuxExternalBrowserSurface.kt

The Linux browser-frame receiver, which imports Electron's dma-buf frames into Vulkan.

src/jvmMain/kotlin/afield/media/browser/LinuxExternalBrowserSurface.kt — read the source

LinuxExternalBrowserSurface

class LinuxExternalBrowserSurface(renderer: Renderer, spec: ExternalBrowserSurfaceSpec, releaseListener: BrowserFrameReleaseListener = BrowserFrameReleaseListener { }) : CopiedExternalBrowserSurface(renderer, spec, releaseListener)

Linux receiver for Electron nativePixmap dma-bufs. The producer must transfer every fd with SCM_RIGHTS; copying the integer values from Electron's process is not a transfer.

Declaration	Summary
override fun importFrame(frame: ExternalBrowserFrame): ImportedImage?
override fun cleanupImports()

MacExternalBrowserSurface.kt

The macOS browser-frame receiver, which imports Electron's IOSurfaces into Vulkan.

src/jvmMain/kotlin/afield/media/browser/MacExternalBrowserSurface.kt — read the source

MacExternalBrowserSurface

class MacExternalBrowserSurface(renderer: Renderer, spec: ExternalBrowserSurfaceSpec, releaseListener: BrowserFrameReleaseListener = BrowserFrameReleaseListener { }) : CopiedExternalBrowserSurface(renderer, spec, releaseListener)

macOS implementation of ExternalBrowserSurface. A native producer transfers a Mach send right for Electron's process-local IOSurface; this receiver resolves and imports it, while CopiedExternalBrowserSurface copies into a stable image and acknowledges only after the fence.

Declaration	Summary
override fun importFrame(frame: ExternalBrowserFrame): ImportedImage?
override fun recordPreCopy(command: CommandBuffer, source: ImportedImage)
override fun cleanupImports()

PuppetBrowser.kt

The control vocabulary for an offscreen Chromium tab driven from Field: its state, events, and the pointer, key, touch and JavaScript requests a box sends it.

PuppetBrowserControl is the command plane (navigation, input, JavaScript in the main or isolated world, raw CDP); PuppetBrowser adds an ExternalBrowserSurface, so the same tab can be sampled as a texture. HeadlessPuppetBrowser implements only the control plane and FieldUiBrowserTexture implements both. The wire encoding is covered by PuppetBrowserProtocolTest.

src/jvmMain/kotlin/afield/media/browser/PuppetBrowser.kt — read the source

Used in: Chromium as a native Field surface

Top-level functions

Declaration	Summary
fun PuppetBrowserControl.addEventListener(name: String, listener: (PuppetBrowserEvent) -> Unit): () -> Unit	Replace the event listener under name on this browser. Stop or rerun of the registering box removes it; callbacks retain that box's context. The returned function removes this listener only.

PuppetBrowserState

data class PuppetBrowserState(val url: String = "about:blank", val title: String = "", val loading: Boolean = false, val canGoBack: Boolean = false, val canGoForward: Boolean = false, val audible: Boolean = false, val focused: Boolean = false, val unresponsive: Boolean = false, val crashed: Boolean = false, val cursor: String = "default", val navigationGeneration: Long = 0)

Current observable state of an offscreen Chromium tab.

Declaration	Summary
val url: String
val title: String
val loading: Boolean
val canGoBack: Boolean
val canGoForward: Boolean
val audible: Boolean
val focused: Boolean
val unresponsive: Boolean
val crashed: Boolean
val cursor: String
val navigationGeneration: Long

PuppetBrowserEvent

data class PuppetBrowserEvent(val sequence: Long, val name: String, val payload: JsonObject)

One browser or protocol event; its payload carries Electron's or CDP's own fields verbatim, unrenamed.

Declaration	Summary
val sequence: Long
val name: String
val payload: JsonObject

BrowserJavascriptWorld

enum class BrowserJavascriptWorld

Constants: MAIN, ISOLATED.

BrowserJavascriptReturnMode

enum class BrowserJavascriptReturnMode

Constants: VALUE, HANDLE.

BrowserJavascriptResult

data class BrowserJavascriptResult(val type: String, val value: JsonElement = JsonNull, val description: String? = null, val unserializableValue: String? = null, val objectId: String? = null)

A JavaScript result, either copied by value or retained in Chromium as objectId.

Declaration	Summary
val type: String
val value: JsonElement
val description: String?
val unserializableValue: String?
val objectId: String?

PuppetBrowserCommandException

class PuppetBrowserCommandException(message: String, val command: String? = null, val details: JsonObject? = null) : RuntimeException(message)

Declaration	Summary
val command: String?
val details: JsonObject?

BrowserModifier

enum class BrowserModifier(val cdpBit: Int)

Constants: ALT, CONTROL, META, SHIFT.

BrowserMouseButton

enum class BrowserMouseButton(val cdpName: String, val cdpBit: Int)

Constants: NONE, LEFT, RIGHT, MIDDLE, BACK, FORWARD.

BrowserPointerType

enum class BrowserPointerType(val cdpName: String)

Constants: MOUSE, PEN.

BrowserPointerEventType

enum class BrowserPointerEventType

Constants: MOVE, DOWN, UP, WHEEL.

BrowserPointerEvent

data class BrowserPointerEvent(val type: BrowserPointerEventType, val x: Double, val y: Double, val button: BrowserMouseButton = BrowserMouseButton.NONE, val buttons: Int = 0, val modifiers: Set<BrowserModifier> = emptySet(), val clickCount: Int = 0, val deltaX: Double = 0.0, val deltaY: Double = 0.0, val pointerType: BrowserPointerType = BrowserPointerType.MOUSE, val pressure: Double = 0.0, val tiltX: Double = 0.0, val tiltY: Double = 0.0, val twist: Int = 0)

Browser pointer coordinates are CSS pixels, origin at the viewport's top-left.

Declaration	Summary
val type: BrowserPointerEventType
val x: Double
val y: Double
val button: BrowserMouseButton
val buttons: Int
val modifiers: Set<BrowserModifier>
val clickCount: Int
val deltaX: Double
val deltaY: Double
val pointerType: BrowserPointerType
val pressure: Double
val tiltX: Double
val tiltY: Double
val twist: Int

BrowserKeyEventType

enum class BrowserKeyEventType(val cdpName: String)

Constants: RAW_KEY_DOWN, KEY_DOWN, KEY_UP, CHAR.

BrowserKeyEvent

data class BrowserKeyEvent(val type: BrowserKeyEventType, val key: String, val code: String = key, val modifiers: Set<BrowserModifier> = emptySet(), val windowsVirtualKeyCode: Int = 0, val nativeVirtualKeyCode: Int = windowsVirtualKeyCode, val autoRepeat: Boolean = false, val keypad: Boolean = false, val text: String? = null, val unmodifiedText: String? = text)

Declaration	Summary
val type: BrowserKeyEventType
val key: String
val code: String
val modifiers: Set<BrowserModifier>
val windowsVirtualKeyCode: Int
val nativeVirtualKeyCode: Int
val autoRepeat: Boolean
val keypad: Boolean
val text: String?
val unmodifiedText: String?

BrowserTouchEventType

enum class BrowserTouchEventType(val cdpName: String)

Constants: START, MOVE, END, CANCEL.

BrowserTouchPoint

data class BrowserTouchPoint(val id: Int, val x: Double, val y: Double, val radiusX: Double = 1.0, val radiusY: Double = 1.0, val rotationAngle: Double = 0.0, val force: Double = 1.0)

Declaration	Summary
val id: Int
val x: Double
val y: Double
val radiusX: Double
val radiusY: Double
val rotationAngle: Double
val force: Double

PuppetBrowserControl

interface PuppetBrowserControl : AutoCloseable

High-level control surface for a Chromium tab: navigation, input, JavaScript, the Chrome DevTools Protocol and the page→Field request bridge. Fire-and-forget input methods retain TCP order; commands return futures completed by Electron.

Declaration	Summary
val state: PuppetBrowserState
fun addEventListener(listener: (PuppetBrowserEvent) -> Unit): () -> Unit
fun navigate(url: String): CompletableFuture<PuppetBrowserState>
fun goBack(): CompletableFuture<PuppetBrowserState>
fun goForward(): CompletableFuture<PuppetBrowserState>
fun reload(ignoreCache: Boolean = false): CompletableFuture<PuppetBrowserState>
fun stop(): CompletableFuture<PuppetBrowserState>
fun focus(): CompletableFuture<PuppetBrowserState>
fun blur(): CompletableFuture<PuppetBrowserState>
fun pointer(event: BrowserPointerEvent)
fun key(event: BrowserKeyEvent)
fun insertText(text: String)
fun setComposition(text: String, selectionStart: Int = text.length, selectionEnd: Int = selectionStart, replacementStart: Int? = null, replacementEnd: Int? = null)
fun touch(type: BrowserTouchEventType, points: List<BrowserTouchPoint>, modifiers: Set<BrowserModifier> = emptySet())
fun flushInput(): CompletableFuture<Unit>	Completes after every previously posted pointer, keyboard, IME, and touch event was dispatched.
fun evaluate(javascript: String, world: BrowserJavascriptWorld = BrowserJavascriptWorld.MAIN, returnMode: BrowserJavascriptReturnMode = BrowserJavascriptReturnMode.VALUE, userGesture: Boolean = false, timeoutMillis: Long = 10_000): CompletableFuture<BrowserJavascriptResult>
fun releaseJavascriptObject(objectId: String): CompletableFuture<Unit>
fun cdp(method: String, params: JsonObject = JsonObject(emptyMap()), sessionId: String? = null): CompletableFuture<JsonObject>
fun forwardCdpEvents(enabled: Boolean): CompletableFuture<Unit>
fun onPageRequest(name: String, handler: (JsonElement) -> CompletableFuture<JsonElement>): () -> Unit	Register an allowlisted request callable as window.fieldPuppet.request(name, value).

PuppetBrowser

interface PuppetBrowser : PuppetBrowserControl, ExternalBrowserSurface

A PuppetBrowserControl whose pixels are also a stable Field texture — the same Chromium tab, seen from the render loop as well as from the keyboard.

PuppetBrowserInteraction.kt

Input adapters that route mouse, keyboard and controller rays into a PuppetBrowser.

PuppetBrowserUvInput turns panel UVs into CSS-pixel pointer events, with click counting; PuppetBrowserWindowInput forwards one Field window's pointer and key stream; PuppetBrowserPanel places the browser as a world-space rectangle and hit-tests rays against it (orientation is covered by PuppetBrowserPanelOrientationTest).

src/jvmMain/kotlin/afield/media/browser/PuppetBrowserInteraction.kt — read the source

PuppetBrowserUvInput

class PuppetBrowserUvInput(val browser: PuppetBrowserControl, val viewport: ExternalBrowserSurfaceSpec)

Maps normalized panel UVs into the browser's CSS-pixel input coordinate system.

Declaration	Summary
val cssWidth: Double
val cssHeight: Double
val position: Pair<Double, Double>
var modifiers: Set<BrowserModifier>
var buttons: Int
fun move(u: Double, v: Double, pointerType: BrowserPointerType = BrowserPointerType.MOUSE)
fun down(button: BrowserMouseButton = BrowserMouseButton.LEFT, u: Double = lastU, v: Double = lastV, pointerType: BrowserPointerType = BrowserPointerType.MOUSE, pressure: Double = if (pointerType == BrowserPointerType.PEN) 1.0 else 0.0)
fun up(button: BrowserMouseButton = BrowserMouseButton.LEFT, u: Double = lastU, v: Double = lastV, pointerType: BrowserPointerType = BrowserPointerType.MOUSE)
fun wheel(deltaX: Double, deltaY: Double, u: Double = lastU, v: Double = lastV)
fun releaseAll()

PuppetBrowserWindowInput

class PuppetBrowserWindowInput(val browser: PuppetBrowser, val window: Window, private val hitTest: (x: Double, y: Double) -> Pair<Double, Double>? = { x, y -> val size = window.size() if (x < 0 || y < 0 || x >= size.w || y >= size.h) null else x / size.w to y / size.h }) : AutoCloseable

Sends one Field Window's pointer and keyboard stream to a PuppetBrowser. hitTest converts window-local logical coordinates to browser UVs; return null when the browser was not hit.

Declaration	Summary
val browser: PuppetBrowser
val window: Window
val uv
var active: Boolean
var focusFollowsWindow	A companion window can regain keyboard input through normal desktop focus.
var onPointerActivity: (() -> Unit)?
val lastPointerUv: Pair<Double, Double>?
fun takeFocus()
fun releaseFocus()
fun releasePointer()
override fun close()

PuppetBrowserPanel

class PuppetBrowserPanel(val browser: PuppetBrowser, var halfWidth: Float, var halfHeight: Float)

A world-space rectangular browser panel with pointer capture for controller or mouse rays.

Declaration	Summary
val browser: PuppetBrowser
var halfWidth: Float
var halfHeight: Float
val model
val uv by lazy
data class Hit(val worldPoint: Vector3f, val u: Double, val v: Double, val onPanel: Boolean)
fun releasePointer()	Cancel capture when a device disappears or another input source takes ownership.
fun panelMvp(view: Matrix4f, projection: Matrix4f, destination: Matrix4f = Matrix4f()): Matrix4f	Compose this panel's world transform with one eye's view/projection matrices.
fun place(position: Vector3f, right: Vector3f, up: Vector3f): Matrix4f
fun faceToward(position: Vector3f, eye: Vector3f, worldUp: Vector3f = Vector3f(0f, 1f, 0f)): Matrix4f
fun raycast(origin: Vector3f, direction: Vector3f): Hit?
fun interact(origin: Vector3f, direction: Vector3f, triggerPressed: Boolean, scrollY: Double = 0.0, pointerType: BrowserPointerType = BrowserPointerType.MOUSE, scrollX: Double = 0.0): Hit?
fun interact(state: ControllerState, scrollY: Double = 0.0, localForward: Vector3f = Vector3f(0f, 0f, 1f)): Hit?	Feed an OpenXR controller directly into this panel.

PuppetBrowserKaleidoscope.kt

A Vulkan effect that draws a live puppet browser page through a grid of moving, mirrored triangular shards.

src/jvmMain/kotlin/afield/media/browser/PuppetBrowserKaleidoscope.kt — read the source

PuppetBrowserKaleidoscope

class PuppetBrowserKaleidoscope(val renderer: Renderer, val browser: PuppetBrowser, val name: String = "puppet-browser-kaleidoscope", val columns: Int = 14, val rows: Int = 8) : AutoCloseable

Draws a live PuppetBrowser through an animated, independently moving field of triangular shards.

Declaration	Summary
val renderer: Renderer
val browser: PuppetBrowser
val name: String
val columns: Int
val rows: Int
var motion: Float	Vertex animation amplitude. Zero leaves a cracked but stationary mosaic.
var facets: Float	Base angular repetition used by the fragment-stage kaleidoscope.
var chroma: Float	Size of the RGB sampling separation, in texture-coordinate scale.
val triangleCount: Int
fun build()
override fun close()

PuppetBrowserPage.kt

Page queries on a puppet browser: an element's text or attribute, a count, a table as rows, and a wait for a condition.

They are thin — each is one PuppetBrowserControl.evaluate — and they are here rather than in a box because the escaping and the null handling are the parts that are easy to get subtly wrong, and because a shared spelling is what makes two boxes agree about what text means.

They run in an isolated world by default. An isolated world sees the same DOM and none of the page's globals, so a page that has redefined Array.prototype.map, shadowed document.querySelectorAll or installed its own Element.prototype.getAttribute cannot change what Field reads out of it; ordinary dashboards ship polyfills that would.

Every one of them works against a HeadlessPuppetBrowser with no GPU, and equally against a FieldUiBrowserTexture you are already rendering — the receiver is the control plane, not the texture.

src/jvmMain/kotlin/afield/media/browser/PuppetBrowserPage.kt — read the source

Used in: Chromium as a native Field surface

Top-level functions

Declaration	Summary
fun PuppetBrowserControl.text(selector: String, world: BrowserJavascriptWorld = BrowserJavascriptWorld.ISOLATED, timeoutMillis: Long = 10_000): CompletableFuture<String?>	The single element matching selector, as trimmed text, or null if there is no such element.
fun PuppetBrowserControl.texts(selector: String, world: BrowserJavascriptWorld = BrowserJavascriptWorld.ISOLATED, timeoutMillis: Long = 10_000): CompletableFuture<List<String>>	Every element matching selector, as trimmed text, in document order.
fun PuppetBrowserControl.attribute(selector: String, name: String, world: BrowserJavascriptWorld = BrowserJavascriptWorld.ISOLATED, timeoutMillis: Long = 10_000): CompletableFuture<String?>	One attribute of the single element matching selector; null when either the element or the attribute is absent. This reads the attribute rather than the DOM property, so href comes back as the markup wrote it and not as the absolute URL Chromium resolved it to.
fun PuppetBrowserControl.count(selector: String, world: BrowserJavascriptWorld = BrowserJavascriptWorld.ISOLATED, timeoutMillis: Long = 10_000): CompletableFuture<Int>	How many elements match selector.
fun PuppetBrowserControl.table(selector: String, world: BrowserJavascriptWorld = BrowserJavascriptWorld.ISOLATED, timeoutMillis: Long = 10_000): CompletableFuture<PuppetPageTable>	The table matching selector, as a header and rows of trimmed cell text.
fun PuppetBrowserControl.waitFor(javascriptCondition: String, timeoutMillis: Long = 10_000, pollMillis: Long = 25, world: BrowserJavascriptWorld = BrowserJavascriptWorld.ISOLATED): CompletableFuture<Unit>	Waits, in the page, until javascriptCondition evaluates truthy, and fails if it has not within timeoutMillis.

PuppetPageTable

data class PuppetPageTable(val header: List<String>, val rows: List<List<String>>)

A table's header cells and body rows, exactly as they were laid out.

Declaration	Summary
fun numbers(index: Int): List<Double?>	Column index of every row as a number; a cell that is not a number becomes null rather than zero, so a missing reading stays distinguishable from a reading of zero. Commas are stripped before parsing, so 1,024 reads as 1024.
fun column(name: String): List<String>	A column by header name rather than by index.
fun numbers(name: String): List<Double?>	A column by header name, as numbers.

PuppetBrowserVrInput.kt

src/jvmMain/kotlin/afield/media/browser/PuppetBrowserVrInput.kt — read the source

BrowserVrPointer

data class BrowserVrPointer(val id: String, val origin: Vector3f, val direction: Vector3f, val pressed: Boolean, val scrollY: Double = 0.0, val gripPressed: Boolean = false, val orientation: Quaternionf = Quaternionf(), val scrollX: Double = 0.0)

One already tracked ray. OpenXR aim poses point along local -Z.

Declaration	Summary
val id: String
val origin: Vector3f
val direction: Vector3f
val pressed: Boolean
val scrollY: Double
val gripPressed: Boolean
val orientation: Quaternionf
val scrollX: Double

PuppetBrowserVrInput

class PuppetBrowserVrInput(val panel: PuppetBrowserPanel)

Arbitrates two hands against Chromium's single mouse. A press owns the pointer until release, including outside the panel; tracking loss cancels capture. Hover alone cannot steal a drag.

Declaration	Summary
var activePointer: String?
fun update(pointers: List<BrowserVrPointer>, enabled: Boolean = true): PuppetBrowserPanel.Hit?
fun cancel()
fun controller(id: String, state: ControllerState, elapsedSeconds: Double): BrowserVrPointer?	Prefer the runtime's aim pose. Invalid poses never fall back to an untracked mouse.
fun hand(id: String, state: HandTrackingState?): BrowserVrPointer?	Joint-based fallback when the runtime supplies hands without a controller aim action.

PuppetBrowserVrGrab

class PuppetBrowserVrGrab(private val panel: PuppetBrowserPanel, private val minimumWidthMeters: Float = 0.4f, private val maximumWidthMeters: Float = 8f)

One grip moves the panel; two grips resize around its centre and translate with the hands.

Declaration	Summary
var owner: String?
fun update(pointers: List<BrowserVrPointer>, enabled: Boolean = true): Boolean
fun cancel()

PuppetBrowserVrPanel.kt

Shows a puppet browser page as a clickable, scrollable quad in VR or in the desktop stereo stand-in.

src/jvmMain/kotlin/afield/media/browser/PuppetBrowserVrPanel.kt — read the source

PuppetBrowserVrPanel

class PuppetBrowserVrPanel(val renderer: Renderer, val stereo: MultiviewStereoTarget, val browser: PuppetBrowser, val widthMeters: Float = 1.6f, val distanceMeters: Float = 1.7f, val name: String = "puppet-browser-panel", val automaticInteraction: Boolean = true)

Presents a PuppetBrowser as an aspect-correct, interactive world-space quad.

Declaration	Summary
val renderer: Renderer
val stereo: MultiviewStereoTarget
val browser: PuppetBrowser
val widthMeters: Float
val distanceMeters: Float
val name: String
val automaticInteraction: Boolean
val aspect: Float
val panel
var lastHit: PuppetBrowserPanel.Hit?
fun build()
fun enableDesktopInteraction()	Install the desktop preview's mouse ray. A headset controller automatically supersedes it.
fun showPointer(hit: PuppetBrowserPanel.Hit?, pressed: Boolean)	Publish a cursor driven by an external input arbiter, shared by both eye passes.
fun cleanup()

PuppetBrowserWindowPanel.kt

Shows a puppet browser page letterboxed inside a Field render window, with mouse hit-testing back into page coordinates.

src/jvmMain/kotlin/afield/media/browser/PuppetBrowserWindowPanel.kt — read the source

PuppetBrowserWindowViewport

data class PuppetBrowserWindowViewport(val x: Double, val y: Double, val width: Double, val height: Double)

An aspect-correct rectangle in one Field afield.graphics.vulkan.util.Window's logical coordinates.

Declaration	Summary
val x: Double
val y: Double
val width: Double
val height: Double
fun hitTest(px: Double, py: Double): Pair<Double, Double>?

PuppetBrowserWindowPanel

class PuppetBrowserWindowPanel(val renderer: Renderer, val browser: PuppetBrowser, val name: String = "puppet-browser-window-panel") : AutoCloseable

Draws one PuppetBrowser into a native Field window and routes that window's real mouse, keyboard, text-input and wheel stream back into Chromium.

Declaration	Summary
val renderer: Renderer
val browser: PuppetBrowser
val name: String
var viewport: PuppetBrowserWindowViewport
val input
fun build(takeFocus: Boolean = true)
override fun close()

WiVRnBrowserInput.kt

src/jvmMain/kotlin/afield/media/browser/WiVRnBrowserInput.kt — read the source

WiVRnBrowserInput

class WiVRnBrowserInput internal constructor(private val browser: PuppetBrowserControl, viewport: ExternalBrowserSurfaceSpec, private val source: WiVRnEventSource, layout: String, variant: String, options: String, private val pointerActivity: () -> Unit, private val otherModifiers: () -> Set<BrowserModifier>) : AutoCloseable

Exclusive, focus-independent Linux input from WiVRn's virtual keyboard and mouse (or, through the internal source constructor, from a paired headset app; see PairedInputSource). The mouse lives in browser coordinates, so moving a VR panel never moves its cursor onto another app. Call poll once per graphics frame; close releases keys, buttons and evdev grabs. See tools/setup_wivrn_input.py for the narrowly scoped device permissions.

Declaration	Summary
val uv
val modifiers: Set<BrowserModifier>
val capturedDevices: Set<String>
val status: String
fun poll(deliver: Boolean = true)	Poll even during page loads: input stays captured, but is discarded until the page is ready.
fun placeCursor(u: Double, v: Double)	Adopt the controller's last hit when handing the relative mouse the shared cursor.
fun releasePointer()	Another explicit pointer press may take over; held buttons then require a fresh press.
override fun close()

WindowsExternalBrowserSurface.kt

Windows import path for Electron browser frames: wraps a D3D11 shared texture, passed as an NT handle, in a Vulkan image that Field can copy from.

WindowsExternalBrowserSurface imports each frame that carries a handle through VK_KHR_external_memory_win32, closes the NT handle after the import, replaces that slot's earlier image, and retires every slot when the stream id changes. It requires Windows and a device that supports that extension. It has been compiled but not yet run on Windows.

src/jvmMain/kotlin/afield/media/browser/WindowsExternalBrowserSurface.kt — read the source

WindowsExternalBrowserSurface

class WindowsExternalBrowserSurface(renderer: Renderer, spec: ExternalBrowserSurfaceSpec, releaseListener: BrowserFrameReleaseListener = BrowserFrameReleaseListener { }) : CopiedExternalBrowserSurface(renderer, spec, releaseListener)

Windows receiver for Electron D3D11 shared textures. The producer must duplicate Electron's process-local NT handle into Field before constructing ExternalBrowserImageHandle.WindowsD3D11NtHandle.

Declaration	Summary
override fun importFrame(frame: ExternalBrowserFrame): ImportedImage?
override fun cleanupImports()

NativeCaptureTarget.kt

src/jvmMain/kotlin/afield/media/capture/NativeCaptureTarget.kt — read the source

imaging.kt

CPU image and computer-vision family for the Field stdlib: the raster counterpart to the time-domain timeseries.kt and frequency-domain spectral.kt families.

This file, with imaging_filters.kt, imaging_features.kt, imaging_geometry.kt and imaging_field.kt, is the stdlib's image layer: convolution and blurs, gradients, pyramids, corner and edge features, and warps, over a dependency-free core.

The whole family is built on one value type, Img:

val photo  = imgFromFile("cat.png")        // 3-channel sRGB-ish floats in 0..1
val gray   = photo.luma()                  // 1 channel
val edges  = gray.sobelMagnitude()         // 1 channel
val canny  = gray.canny(0.08, 0.20)        // 1 channel binary mask
canny.toFLines()                           // .. straight into Field's drawing layer

What the family commits to:

• Float channels, not bytes. Every operation is float math in 0..1 (or unbounded, for gradients and Laplacians, which legitimately go negative). Nothing quantises mid-pipeline, so a 12-stage chain does not accumulate rounding steps. Clamping happens once, on output.
• Immutable-by-convention value type. Ops return a new Img; the backing FloatArray is public (data) so a hot inner loop in a box can write it directly if it wants to. Interleaved channel-major-last layout (data[(y*width + x)*channels + c]), the same layout BufferedImage/GL/Vulkan uploads want.
• 1, 3 or 4 channels. 1 = scalar/gray/mask, 3 = RGB, 4 = RGBA. Nothing else; a 12-channel feature tensor is a different problem, and not one this family takes on.
• No external dependency. Pure Kotlin plus javax.imageio for load/save and afield.media.image.util.PngEncoder for the AWT-free path used by box output.
• Live-codeable. Img implements ToHTML, so evaluating an Img in a box (or img.h) shows the actual picture as an inline thumbnail, not Img@3f1a2b.

Out of scope: GPU work (that is Field 4 / StackSL / RectSL), video decoding, and anything needing a model file.

src/jvmMain/kotlin/afield/media/image/imaging.kt — read the source

Used in: Imaging and computer vision, Visualizers, Displays, and Little Instruments, Wave Function Collapse examples

Top-level functions

Declaration	Summary
fun BufferedImage.toImg(channels: Int? = null): Img	Convert a BufferedImage to an Img with samples in 0..1.
fun Img.toBufferedImage(): BufferedImage	Convert to a BufferedImage for AWT drawing / ImageIO. Samples are clamped to 0..1 and quantised to 8 bits at this point (and only at this point). A 1-channel image becomes gray RGB.
fun Img.toPngBytes(): ByteArray	Encode straight to PNG bytes with no AWT (via PngEncoder) — the path a box uses to put a picture in its own output, and the path Img.toHTML uses for its thumbnail.
fun Img.savePng(path: String): File	Write this image to path as a PNG (creating parent directories). Returns the file.
fun imgFromFile(path: String, channels: Int? = null): Img	Loads an image from a file path, straight through ImageIO — the path is used verbatim and does not go through FileManager, so a workspace-relative spec will not resolve here.
fun imgFromBytes(bytes: ByteArray, channels: Int? = null): Img	Decode an image from in-memory encoded bytes (PNG/JPEG/… — whatever ImageIO handles).
fun floatsToImg(values: FloatArray, width: Int, height: Int): Img	Wrap a plain FloatArray of scalars as a 1-channel image — the bridge from the rest of the stdlib (a heightfield, a spectrogram frame stack, a DoubleArray reduction) into imaging.
fun doublesToImg(values: DoubleArray, width: Int, height: Int): Img	floatsToImg for doubles.
fun Img.sampleNearest(x: Double, y: Double, c: Int = 0, border: Border = Border.CLAMP): Float	Nearest-neighbour sample at continuous coordinates.
fun Img.sampleBilinear(x: Double, y: Double, c: Int = 0, border: Border = Border.CLAMP): Float	Bilinear sample at continuous coordinates — pixel centres sit at integer coordinates, so sampleBilinear(3.0, 4.0) == at(3, 4) exactly.
fun Img.sampleBilinearAll(x: Double, y: Double, border: Border = Border.CLAMP): FloatArray	All channels bilinearly sampled at once.
fun Img.crop(x: Int, y: Int, w: Int, h: Int): Img	Rectangular sub-image. Coordinates are clamped into range; the result is at least 1x1.
fun Img.pad(pad: Int, border: Border = Border.CLAMP): Img	Grow the image by pad pixels on every side, filling from border.
fun Img.resize(newWidth: Int, newHeight: Int, areaAverage: Boolean = true): Img	Resize to newWidth x newHeight.
fun Img.upscaleNearest(factor: Int): Img	Enlarge by an integer factor with no interpolation — every pixel becomes a crisp factor x factor block.
fun Img.scaleBy(factor: Double, areaAverage: Boolean = true): Img	Resize by a scale factor (both axes). See resize.
fun Img.flipX(): Img	Mirror left-right.
fun Img.flipY(): Img	Mirror top-bottom.
fun Img.rotate90(): Img	Rotate 90 degrees clockwise (width and height swap).
fun Img.transpose(): Img	Transpose (x and y swap).
fun Img.channel(c: Int): Img	Extract channel c as a 1-channel image.
fun Img.split(): List<Img>	Split into one 1-channel image per channel.
fun mergeChannels(vararg parts: Img): Img	Merge 1-channel images into a multi-channel image. All inputs must share a shape and the count must be 1, 3 or 4.
fun Img.luma(): Img	Perceptual grayscale (BT.709 luma) as a 1-channel image. A 1-channel input is returned as a copy; an RGBA input ignores alpha.
fun Img.toRGB(): Img	Replicate a 1-channel image into 3 channels (the inverse direction of luma).
fun Img.toRGBA(alpha: Float = 1f): Img	Add an opaque (or alpha-valued) alpha channel.
fun linearToSrgb(v: Float): Float	sRGB transfer function applied to a linear-light value.
fun srgbToLinear(v: Float): Float	Inverse sRGB transfer function: encoded value → linear light.
fun Img.toLinear(): Img	Decode sRGB-encoded samples to linear light. Blurring, downsampling and averaging are only correct in linear light; img.toLinear().gaussianBlur(2.0).toSrgb() is the physically right blur, and visibly different from blurring the encoded values.
fun Img.toSrgb(): Img	Encode linear-light samples back to sRGB. Inverse of toLinear.
fun Img.rgbToHsv(): Img	RGB → HSV, in place of the channels: h in 0..1 (hue angle / 360), s in 0..1, v in 0..1. Requires a 3- or 4-channel image; alpha is dropped.
fun Img.hsvToRgb(): Img	HSV → RGB, the inverse of rgbToHsv.
fun Img.normalized(lo: Float = 0f, hi: Float = 1f): Img	Linearly rescale so the sample range becomes [lo, hi] (default 0..1). A constant image maps to the midpoint rather than dividing by zero.
fun Img.normalizedPercentile(pLow: Double = 0.01, pHigh: Double = 0.99): Img	Robust normalisation: map the pLow/pHigh percentiles of the sample distribution to 0..1 and clamp. Far more useful than normalized on noisy gradient/response images, where one hot pixel otherwise squashes everything else to black.
fun Img.histogram(bins: Int = 256, lo: Float = 0f, hi: Float = 1f): IntArray	Histogram of a 1-channel image over [lo, hi] with bins equal-width buckets. Values outside the range land in the first/last bucket.
fun Img.equalize(bins: Int = 256): Img	Histogram equalisation of a 1-channel image: remap intensities through the cumulative histogram so the output distribution is (approximately) flat. Classic contrast enhancement.
fun Img.otsuThreshold(bins: Int = 256): Float	Otsu's method: the intensity threshold in [lo, hi] that maximises between-class variance — i.e. the best two-way split of a bimodal histogram, computed rather than guessed.
fun Img.threshold(threshold: Float): Img	Binarise: samples > threshold become 1, everything else 0. Result is 1-channel.
fun Img.thresholdOtsu(bins: Int = 256): Img	Binarise at the otsuThreshold.
fun Img.thresholdAdaptive(radius: Int = 8, bias: Float = 0.02f): Img	Adaptive (local-mean) threshold: each pixel is compared to the mean of a (2*radius+1) square neighbourhood, minus bias. Handles uneven illumination that defeats a global threshold.
fun Img.meanAbsDiff(other: Img): Double	Mean absolute difference per sample against other — a blunt but unbiased similarity measure.
fun Img.psnr(other: Img): Double	Peak signal-to-noise ratio in dB against other, assuming a 0..1 signal range.

Top-level values

Declaration	Summary
val RGB_TO_LUMA: FloatArray	ITU-R BT.709 luma weights — the perceptual RGB→gray mix used throughout the family.

Img

class Img(val width: Int, val height: Int, val channels: Int, val data: FloatArray) : ToHTML

A CPU raster image: width * height * channels floats, interleaved, row-major, origin top-left (the image-processing convention, matching BufferedImage and PNG).

Declaration	Summary
val width: Int	Pixels across; at least 1.
val height: Int	Pixels down; at least 1.
val channels: Int	1 (scalar/gray), 3 (RGB) or 4 (RGBA); the constructor rejects anything else.
val data: FloatArray	Interleaved samples, data[(y * width + x) * channels + c].
val pixelCount: Int	Number of pixels (not samples).
val isScalar: Boolean	true when this is a single-channel scalar image (gray / mask / gradient / distance).
fun index(x: Int, y: Int, c: Int = 0): Int	Flat index of channel c of pixel (x, y). No bounds checking; see at.
fun at(x: Int, y: Int, c: Int = 0): Float	Sample at (x, y), channel c; throws if out of bounds.
fun set(x: Int, y: Int, c: Int, v: Float)	Write v at (x, y), channel c; throws if out of bounds. Mutates in place.
fun get(x: Int, y: Int, c: Int = 0, border: Border = Border.CLAMP): Float	Sample with an out-of-range policy — the workhorse every filter uses at the image border.
fun copy(): Img	An independent copy.
fun zerosLike(channels: Int = this.channels): Img	A same-shape image of zeros (optionally a different channel count).
fun sameShapeAs(other: Img): Boolean	true if other has the same width, height and channel count.
inline fun map(f: (Float) -> Float): Img	Apply f to every sample (all channels), returning a new image.
inline fun mapIndexed(f: (x: Int, y: Int, c: Int, v: Float) -> Float): Img	Apply f to every pixel given (x, y, channel, value), returning a new image.
inline fun zip(other: Img, f: (Float, Float) -> Float): Img	Element-wise combination of two same-shape images.
operator fun plus(other: Img): Img
operator fun minus(other: Img): Img
operator fun times(other: Img): Img
operator fun times(k: Number): Img
operator fun plus(k: Number): Img
operator fun unaryMinus(): Img
fun abs(): Img	Absolute value of every sample.
fun clamp(lo: Float = 0f, hi: Float = 1f): Img	Clamp every sample into [lo, hi] (default the display range 0..1).
fun min(): Float	Smallest sample over all channels.
fun max(): Float	Largest sample over all channels.
fun mean(): Float	Arithmetic mean of every sample.
fun sum(): Double	Sum of every sample — the "DC" of the image; convolution with a normalised kernel preserves it.
fun stdDev(): Float	Population standard deviation of every sample.
override fun toString(): String
override fun toHTML(): String	Inline picture for the value visualizer — the actual pixels, aspect-correct, PNG-encoded and inlined as a data URI, plus a readout and a disclosure holding the per-channel strip, the histogram and the statistics.

Border

enum class Border

How a filter resolves coordinates that fall outside the image.

Constants: ZERO, CLAMP, REFLECT, WRAP.

imaging_features.kt

Edges, corners, lines, blobs and morphology for Img — the "find things in the picture" half of the stdlib imaging family (see imaging.kt for the value type and imaging_filters.kt for convolution and gradients).

Everything here reduces pixels to structure — a point set, a label map, a list of lines — and that structure is what imaging_field.kt turns into FLines so it lands in Field's drawing layer rather than staying trapped in a raster.

Every detector here is the textbook algorithm, implemented plainly and documented with the property a test can check:

function	what it finds	test invariant
canny	thin, connected edges	a synthetic square yields a 1-pixel-wide closed outline
harrisCorners / shiTomasiCorners	corners via the structure tensor	4 responses at a square's 4 corners
fastCorners	corners via a brightness circle test	same, and much faster
houghLines	straight lines in polar form	a drawn line is recovered at its (rho, theta)
connectedComponents	blobs in a binary mask	N drawn discs give N labels with the right areas
distanceTransform	distance to the nearest zero	a disc's centre has distance = its radius

src/jvmMain/kotlin/afield/media/image/imaging_features.kt — read the source

Used in: Imaging and computer vision, Visualizers, Displays, and Little Instruments, Agent transcript gallery

Top-level functions

Declaration	Summary
fun Img.canny(low: Double = 0.08, high: Double = 0.20, sigma: Double = 1.0): Img	Canny edge detection.
fun Img.cannyAuto(highPercentile: Double = 0.90, lowRatio: Double = 0.4, sigma: Double = 1.0): Img	canny with the thresholds chosen from the image instead of guessed.
fun nonMaximumSuppression(magnitude: Img, gradient: Gradient): Img	Thin a gradient-magnitude image to ridge crests: a pixel survives only if its magnitude is >= both neighbours sampled along the gradient direction (quantised to the nearest of the four 45-degree axes). Exposed because it is useful on its own, on any gradient field.
fun hysteresis(thinned: Img, low: Float, high: Float): Img	Double-threshold + connectivity: pixels above high are edges; pixels above low become edges only if 8-connected (transitively) to one. Returns a binary mask.
fun Img.structureTensor(sigma: Double = 1.5, gradientSigma: Double = 1.0): Triple<Img, Img, Img>	The per-pixel structure tensor (second-moment matrix) of a scalar image, Gaussian-weighted over a neighbourhood — the shared engine behind Harris, Shi–Tomasi and optical flow.
fun Img.harrisResponse(k: Double = 0.04, sigma: Double = 1.5, gradientSigma: Double = 1.0): Img	Harris corner response image, det(M) - k * trace(M)² over the structureTensor M. Large and positive at a corner, negative along an edge, near zero on flat ground.
fun Img.shiTomasiResponse(sigma: Double = 1.5, gradientSigma: Double = 1.0): Img	Shi–Tomasi ("good features to track") response: the smaller eigenvalue of the structure tensor. Unlike Harris it needs no magic k, and the response is directly interpretable — it is the weakest intensity variation over any direction, so a large value means the point is localisable in both axes. This is the response optical-flow trackers want.
fun peakCorners(response: Img, threshold: Double = 0.01, minDistance: Int = 5, maxCount: Int = 200, subPixel: Boolean = true): List<Corner>	Peak extraction shared by the corner detectors: keep local maxima of response that clear threshold * max(response), sort by strength, and greedily drop any peak within minDistance pixels of a stronger one.
fun Img.harrisCorners(k: Double = 0.04, threshold: Double = 0.01, minDistance: Int = 5, maxCount: Int = 200, sigma: Double = 1.5): List<Corner>	Harris corners: harrisResponse followed by peakCorners.
fun Img.shiTomasiCorners(threshold: Double = 0.01, minDistance: Int = 5, maxCount: Int = 200, sigma: Double = 1.5): List<Corner>	Shi–Tomasi corners: shiTomasiResponse followed by peakCorners. The default tracker seed.
fun Img.goodFeaturesToTrack(maxCount: Int = 200, threshold: Double = 0.01, minDistance: Int = 5): List<Corner>	Alias for shiTomasiCorners under the name the OpenCV-literate will reach for first.
fun Img.fastCorners(threshold: Double = 0.08, contiguous: Int = 9, minDistance: Int = 5, maxCount: Int = 300): List<Corner>	FAST corner detection (Rosten & Drummond): a pixel is a corner if at least contiguous of the 16 pixels on a radius-3 circle around it are all brighter than centre + t or all darker than centre - t. No gradients, no eigenvalues — just a wrap-around run-length test, which is why it is the fastest usable corner detector.
fun houghLines(edges: Img, thetaSteps: Int = 360, rhoStep: Double = 1.0, threshold: Int = 0, maxLines: Int = 20, minSeparationTheta: Double = 0.15, minSeparationRho: Double = 12.0): List<HoughLine>	Standard Hough line transform over a binary edge mask.
fun houghCircles(edges: Img, minRadius: Int = 8, maxRadius: Int = 40, radiusStep: Int = 2, threshold: Double = 0.5, maxCircles: Int = 10, minSeparation: Double = 10.0, angleStepDeg: Double = 3.0): List<HoughCircle>	Hough circle transform over a binary edge mask, brute force over the radius range. Each edge pixel votes for the centres of all circles of each candidate radius passing through it.
fun Img.erode(radius: Int = 1, shape: StructuringElement = StructuringElement.SQUARE): Img	Grayscale erosion: each pixel becomes the minimum over the structuring element. On a binary mask this shrinks bright regions and deletes anything thinner than the element.
fun Img.dilate(radius: Int = 1, shape: StructuringElement = StructuringElement.SQUARE): Img	Grayscale dilation: each pixel becomes the maximum over the structuring element. On a binary mask this grows bright regions and bridges gaps narrower than the element.
fun Img.open(radius: Int = 1, shape: StructuringElement = StructuringElement.SQUARE): Img	Erode then dilate — removes specks and thin protrusions, keeps the overall size of what remains.
fun Img.close(radius: Int = 1, shape: StructuringElement = StructuringElement.SQUARE): Img	Dilate then erode — fills pinholes and hairline cracks, keeps the overall size.
fun Img.morphologicalGradient(radius: Int = 1, shape: StructuringElement = StructuringElement.SQUARE): Img	dilate - erode: a one-pixel-ish outline of every bright region.
fun Img.topHat(radius: Int = 3, shape: StructuringElement = StructuringElement.SQUARE): Img	image - open(image): bright detail smaller than the element (specks, text on a gradient).
fun Img.blackHat(radius: Int = 3, shape: StructuringElement = StructuringElement.SQUARE): Img	close(image) - image: dark detail smaller than the element.
fun Img.connectedComponents(connectivity: Int = 8): ComponentLabelling	Two-pass connected-component labelling of a binary mask (samples > 0.5 are foreground), with per-blob area, bounding box and centroid.
fun Img.distanceTransform(): Img	Euclidean distance transform of a binary mask: each foreground pixel's value becomes its distance (in pixels) to the nearest background pixel.
fun Img.traceContours(minPerimeter: Int = 8): List<List<Vec2>>	Trace the outline of every blob in a binary mask as a closed pixel-boundary polygon (Moore-neighbour tracing). Complements connectedComponents' statistics with actual geometry — feed the result to contoursToFLines in imaging_field.kt to draw it.

Corner

data class Corner(val x: Double, val y: Double, val response: Double)

A detected interest point.

Declaration	Summary
val x: Double	Column, refined to sub-pixel when the detector was asked for it.
val y: Double	Row, refined to sub-pixel when the detector was asked for it.
val response: Double	Detector strength: Harris response, Shi–Tomasi min-eigenvalue or FAST score.
fun toVec2(): Vec2	As a Vec2 so it drops straight into polylineFrom / the rest of the stdlib geometry.

HoughLine

data class HoughLine(val rho: Double, val theta: Double, val votes: Int)

A line in Hesse normal form: x*cos(theta) + y*sin(theta) = rho.

Declaration	Summary
val rho: Double	Signed distance from the image origin (top-left) to the line, in pixels.
val theta: Double	Line normal angle in radians, in [0, π).
val votes: Int	Accumulator strength: how many edge pixels voted for this line.
fun clipTo(w: Int, h: Int): Pair<Vec2, Vec2>?	Clip the (infinite) line to the w x h image rectangle; null if it misses entirely.

HoughCircle

data class HoughCircle(val x: Double, val y: Double, val radius: Double, val votes: Int)

A circle found by houghCircles.

Declaration	Summary
val x: Double	Centre column, on the whole-pixel accumulator grid: these are not refined to sub-pixel.
val y: Double	Centre row, on the whole-pixel accumulator grid.
val radius: Double	Radius in pixels, one of the radii that were tested — minRadius + k * radiusStep.
val votes: Int	Accumulator strength at the centre for this radius.

StructuringElement

enum class StructuringElement

Neighbourhood shape for the morphological operators.

Constants: SQUARE, DISC, CROSS.

Component

data class Component(val label: Int, val area: Int, val minX: Int, val minY: Int, val maxX: Int, val maxY: Int, val centroidX: Double, val centroidY: Double)

One labelled blob from connectedComponents.

Declaration	Summary
val label: Int	The value this blob carries in the label map; 1-based, since 0 is background.
val area: Int	Pixel count.
val minX: Int	Leftmost column of the bounding box, inclusive.
val minY: Int	Topmost row of the bounding box, inclusive.
val maxX: Int	Rightmost column of the bounding box, inclusive.
val maxY: Int	Bottom row of the bounding box, inclusive.
val centroidX: Double	Mean column over the blob's pixels.
val centroidY: Double	Mean row over the blob's pixels.
val boundingWidth: Int	Bounding-box width in pixels; inclusive, so a single-pixel blob is 1 wide.
val boundingHeight: Int	Bounding-box height in pixels; inclusive, so a single-pixel blob is 1 high.
val fillRatio: Double	Area over bounding-box area — 1.0 for a filled rectangle, ~π/4 for a disc.
fun centroid(): Vec2	Centroid as a Vec2.

ComponentLabelling

class ComponentLabelling(val labels: Img, val components: List<Component>)

Result of a connected-component labelling.

Declaration	Summary
val labels: Img	1-channel image whose samples are the integer label of each pixel; 0 is background.
val components: List<Component>	Per-blob statistics, in label order.
val count: Int	How many blobs were found (excluding background).
fun mask(label: Int): Img	A binary mask selecting only the pixels of label.
fun largest(): Component?	The largest blob by area, or null if there are none.
fun filterByArea(minArea: Int, maxArea: Int = Int.MAX_VALUE): List<Component>	Blobs with at least minArea pixels.
override fun toString(): String

imaging_field.kt

What makes the imaging family Field's rather than a generic CV library: the bridges out of the raster and into the things Field already draws, colours and displays.

Three bridges:

1. Structure → FLine. Every detector in imaging_features.kt produces geometry, and geometry in Field is an FLine. Img.toContourFLines, cornersToFLines, linesToFLines, FlowField.toFLines and Img.isoContourFLines hand you drawable lines, so a Canny result goes straight into the same drawing/animation/export machinery as anything hand-drawn.
2. Scalar → colour. Img.colormap runs a 1-channel response image through the same Colormaps tables RectSL's heat/bars/scatter use, so a Harris response, a distance transform and a RectSL chart are coloured by one source of truth.
3. Value → screen. Img is a afield.ui.visualization.ToHTML, so evaluating one in a box shows the picture; Box.showImg pushes a full-size version to an output channel, and Box.showImgGrid lays several out side by side with captions — the fastest way to see a whole pipeline at once.

The scalar-field bridge runs the other way too: Img.asScalarField2D adapts an Img to the ScalarField2D type contours.kt marches, so Field's existing marching-squares iso-contour extractor works on image data with no glue.

src/jvmMain/kotlin/afield/media/image/imaging_field.kt — read the source

Used in: Imaging and computer vision, Wave Function Collapse examples, Spatial artificial life

Top-level functions

Declaration	Summary
fun contourToFLine(points: List<Vec2>, closed: Boolean = true, z: Number = 0): FLine	Convert a point list in image coordinates to an FLine polyline.
fun contoursToFLines(contours: List<List<Vec2>>, closed: Boolean = true, z: Number = 0): List<FLine>	Convert many point lists (e.g. from Img.traceContours) to one FLine each.
fun Img.toContourFLines(minPerimeter: Int = 8, simplifyTolerance: Double = 1.0, z: Number = 0): List<FLine>	Trace this binary mask's blob outlines and hand them back as FLines — the one-call path from "I thresholded an image" to "I have drawable geometry".
fun simplifyPolyline(points: List<Vec2>, tolerance: Double): List<Vec2>	Ramer–Douglas–Peucker polyline simplification: recursively keep the point furthest from the chord until every dropped point is within tolerance of the retained line. Exposed on its own because pixel-traced contours are unusable without it.
fun cornersToFLines(corners: List<Corner>, size: Double = 4.0, box: Boolean = false, z: Number = 0): List<FLine>	Draw each corner as a small cross (or box) FLine at its location, sized by size.
fun HoughLine.toFLine(w: Int, h: Int, z: Number = 0): FLine?	Clip this HoughLine to a w x h image rectangle and return the visible segment as an FLine, or null if the line misses the rectangle entirely.
fun linesToFLines(lines: List<HoughLine>, w: Int, h: Int, z: Number = 0): List<FLine>	Clip each HoughLine to a w x h image rectangle and return the visible segments as FLines.
fun circlesToFLines(circles: List<HoughCircle>, segments: Int = 48, z: Number = 0): List<FLine>	Draw each HoughCircle as a closed polygon of segments sides.
fun FlowField.toFLines(scale: Double = 1.0, minMagnitude: Double = 0.25, headSize: Double = 3.0, z: Number = 0): List<FLine>	Draw a sparse optical-flow field as one arrow FLine per tracked point — shaft plus a two-stroke arrowhead, so direction is visible and not just magnitude.
fun Img.asScalarField2D(border: Border = Border.CLAMP): ScalarField2D	Adapt this 1-channel image to the ScalarField2D that contours.kt marches — the image becomes a bilinearly-interpolated continuous field over [0, width-1] x [0, height-1].
fun Img.isoContourFLines(iso: Double, cells: Int = max(width, height), z: Number = 0): List<FLine>	Extract smooth iso-contours at iso through Field's existing marching-squares extractor and return them as FLines. Unlike toContourFLines (which traces the pixel boundary of a binary mask) this interpolates between samples, so the contour is sub-pixel smooth — the right tool for a distance transform, a blurred response or a height field.
fun Img.colormap(name: String = "viridis", lo: Float? = null, hi: Float? = null): Img	Colour-map a 1-channel image through one of Field's Colormaps ramps (viridis, magma, inferno, plasma, turbo, coolwarm, grayscale), returning a 3-channel RGB image.
fun Img.colormapDiverging(name: String = "coolwarm"): Img	Colour-map with a symmetric range around zero — the correct default for a signed image (a Laplacian, a difference of Gaussians, a gradient component), where the meaningful thing is which side of zero a pixel is on. Pair with a diverging map like coolwarm.
fun Img.maskToRGBA(color: Color4 = Color4(1.0, 0.35, 0.15), alpha: Double = 1.0): Img	Tint a binary mask a single colour over a transparent background — for overlaying a detection on the original picture with overlay.
fun Img.overlay(top: Img, opacity: Double = 1.0): Img	Alpha-composite top (RGBA, or RGB treated as opaque) over this image, returning RGB. The standard way to show a mask/overlay on top of the source picture.
fun hstack(vararg images: Img, gap: Int = 8, background: Float = 0.08f): Img	Lay images out left-to-right into one strip; heights are matched by scaling to the tallest.
fun vstack(vararg images: Img, gap: Int = 8, background: Float = 0.08f): Img	Lay images out top-to-bottom into one column; widths are matched by scaling to the widest.
fun Box.showImg(img: Img, channel: String = "main", maxSide: Int = 900, caption: String? = null)	Push an Img to one of this box's output channels as a real PNG, at up to maxSide pixels on the long edge. Signed / out-of-range images are normalised for display, and when a caption is asked for the caption says so.
fun montage(panels: List<Pair<String, Img>>, columns: Int = 2, cellSide: Int = 320, gap: Int = 10, background: Float = 0.08f, displayRange: ClosedFloatingPointRange<Float>? = null, textScale: Int = 2): Img	Lay several captioned images out on a uniform grid and return the montage as one Img.
fun Img.withLabel(text: String, x: Int = 4, y: Int = 4, scale: Int = 2, color: Color4 = Color4(0.85, 0.87, 0.92)): Img	Draw a caption into an Img at (x, y), anti-aliased and without AWT — useful on its own for labelling a frame before you write it out.
fun Box.showImgGrid(vararg panels: Pair<String, Img>, channel: String = "main", columns: Int = 2, cellSide: Int = 320, displayRange: ClosedFloatingPointRange<Float>? = null)	Show several images side by side with their captions burned in — the fastest way to see a whole pipeline (source / blurred / gradient / edges) at a glance.

imaging_filters.kt

Convolution, smoothing, gradients and multiscale pyramids for Img — the filter half of the stdlib imaging family (see imaging.kt for the value type, imaging_features.kt for edges, corners and blobs).

Everything here is per-channel: a 3-channel image blurs each of R, G, B independently. Filters that only make sense on a scalar field (sobel, laplacian, …) take the Img.luma of a colour input automatically rather than erroring, because that is what you meant.

Two performance-relevant choices:

• Separable when possible. Gaussian and box blurs run as two 1-D passes (O(k) per pixel, not O(k²)). convolveSeparable is public so your own separable kernels get the same treatment, and convolve2D exists both as a general fallback and as the naive reference the unit tests check the separable path against.
• Box blur is a running sum. boxBlur is O(1) per pixel regardless of radius, so a radius-40 mean is as cheap as radius-1. Three box passes approximate a Gaussian closely (see boxBlurApproxGaussian) if you need a very large sigma cheaply.

src/jvmMain/kotlin/afield/media/image/imaging_filters.kt — read the source

Used in: Imaging and computer vision, Visualizers, Displays, and Little Instruments, Agent transcript gallery

Top-level functions

Declaration	Summary
fun gaussianKernel1D(sigma: Double, truncate: Double = 4.0): Kernel1D	A normalised 1-D Gaussian kernel with standard deviation sigma, truncated at truncate sigmas (the SciPy default of 4 gives < 1e-4 of the mass outside).
fun boxKernel1D(radius: Int): Kernel1D	A normalised box (mean) kernel of half-width radius.
fun Img.convolveSeparable(kx: Kernel1D, ky: Kernel1D? = kx, border: Border = Border.CLAMP): Img	Separable convolution: kx along x then ky along y (or kx both ways if ky is null). O(kx + ky) per pixel per channel.
fun Img.convolve2D(k: Kernel2D, border: Border = Border.CLAMP): Img	Dense 2-D convolution with k. True convolution (the kernel is flipped); pass an already-flipped kernel or a symmetric one if you want correlation. O(kw * kh) per pixel per channel — this is the general fallback, and the reference the separable path is tested against.
fun Img.correlate2D(k: Kernel2D, border: Border = Border.CLAMP): Img	Cross-correlation with k (kernel not flipped). This is what most "apply this stencil" intuitions actually mean; identical to convolve2D for symmetric kernels.
fun Img.gaussianBlur(sigma: Double, truncate: Double = 4.0, border: Border = Border.CLAMP): Img	Gaussian blur with standard deviation sigma, run as two separable 1-D passes.
fun Img.boxBlur(radius: Int, border: Border = Border.CLAMP): Img	Box (mean) blur over a (2*radius+1)² window, computed with running sums so the cost is independent of radius.
fun Img.boxBlurApproxGaussian(sigma: Double, border: Border = Border.CLAMP): Img	Three successive boxBlur passes, which by the central limit theorem approximate a Gaussian of standard deviation sigma — O(1) per pixel at any sigma. Use when sigma is large and you care more about speed than about the exact kernel shape.
fun Img.median(radius: Int = 1, border: Border = Border.CLAMP): Img	Median filter over a (2*radius+1)² window — the standard salt-and-pepper de-noiser, and unlike a mean it does not smear edges.
fun Img.bilateral(sigmaSpace: Double = 3.0, sigmaRange: Double = 0.1, border: Border = Border.CLAMP): Img	Bilateral filter: a Gaussian blur whose weights are additionally attenuated by intensity distance, so it smooths flat regions while leaving edges sharp.
fun Img.unsharpMask(sigma: Double = 1.5, amount: Double = 1.0, threshold: Float = 0f): Img	Unsharp mask: image + amount * (image - gaussianBlur(image, sigma)). The oldest sharpening trick there is; threshold suppresses amplification of low-contrast noise.
fun Img.sobel(border: Border = Border.CLAMP): Gradient	Sobel gradients. The 3x3 Sobel operator factors into a [-1, 0, 1] derivative across the gradient axis and a [1, 2, 1]/4 smoothing along it, so both components run separably.
fun Img.sobelMagnitude(border: Border = Border.CLAMP): Img	Shorthand for sobel().magnitude().
fun Img.scharr(border: Border = Border.CLAMP): Gradient	Scharr gradients — same structure as sobel but with the [3, 10, 3]/16 smoothing factor, which has markedly better rotational symmetry (the direction estimate is more accurate on diagonal edges).
fun Img.gaussianGradient(sigma: Double, truncate: Double = 4.0, border: Border = Border.CLAMP): Gradient	Gradients of a Gaussian-smoothed image at scale sigma, computed as the analytic derivative of the Gaussian (not a blur followed by a difference), so the scale is exact.
fun Img.laplacian(eightNeighbour: Boolean = false, border: Border = Border.CLAMP): Img	Discrete Laplacian (second derivative). eightNeighbour picks LAPLACIAN_8 over LAPLACIAN_4.
fun Img.laplacianOfGaussian(sigma: Double = 1.5, border: Border = Border.CLAMP): Img	Laplacian of Gaussian at scale sigma — computed as the difference of the image and its blur's Laplacian, i.e. laplacian(gaussianBlur(img, sigma)). Zero crossings are edges at that scale.
fun Img.differenceOfGaussians(sigma1: Double = 1.0, sigma2: Double = 2.0, border: Border = Border.CLAMP): Img	Difference of Gaussians, blur(sigma1) - blur(sigma2) with sigma1 < sigma2 — the cheap band-pass approximation to laplacianOfGaussian and the basis of blob detection.
fun Img.pyrDown(sigma: Double = 1.0): Img	One-octave downsample: blur by sigma then take every other pixel. The blur is what keeps this from aliasing — a bare decimation of a fine texture produces moiré, not a small picture.
fun Img.pyrUp(targetWidth: Int = width * 2, targetHeight: Int = height * 2, sigma: Double = 1.0): Img	One-octave upsample to exactly targetWidth x targetHeight (default 2x), bilinear then smoothed by sigma to suppress the interpolation grid. The approximate inverse of pyrDown.
fun Img.gaussianPyramid(levels: Int = 4, sigma: Double = 1.0, minSize: Int = 8): GaussianPyramid	Build a GaussianPyramid with at most levels octaves, stopping at minSize pixels.
fun Img.laplacianPyramid(levels: Int = 4, sigma: Double = 1.0, minSize: Int = 8): LaplacianPyramid	Build a LaplacianPyramid: at each octave, band = level - pyrUp(pyrDown(level)). The reconstruction is exact up to the upsampling filter, so it is a genuine invertible band-splitting — the basis of multiband blending, texture synthesis and detail transfer.

Top-level values

Declaration	Summary
val SOBEL_DERIV: Kernel1D	The Sobel 3x3 derivative factor, scaled to true units-per-pixel: [-1, 0, 1]/2, i.e. the central difference (f(x+1) - f(x-1)) / 2. The classic unscaled [-1, 0, 1] reports twice the actual slope; dividing by 2 here means sobel().dx on a ramp of slope s really is s, and a unit step edge really does integrate to a total gradient of 1.
val SOBEL_SMOOTH: Kernel1D	The Sobel smoothing factor, normalised.
val SCHARR_SMOOTH: Kernel1D	The Scharr smoothing factor ([3, 10, 3]/16) — better rotational symmetry than Sobel's.
val LAPLACIAN_4: Kernel2D	The 4-neighbour discrete Laplacian.
val LAPLACIAN_8: Kernel2D	The 8-neighbour discrete Laplacian.

Kernel1D

class Kernel1D(val taps: FloatArray)

A 1-D convolution kernel: taps centred on the middle element (so the length is normally odd).

Declaration	Summary
val taps: FloatArray	The weights; index radius is the centre tap.
val radius: Int	Half-width; index radius in taps is the centre tap.
fun sum(): Float	Sum of the taps — 1 for a normalised smoothing kernel, 0 for a derivative kernel.
fun normalized(): Kernel1D	A copy scaled so the taps sum to 1. Throws for a zero-sum (derivative) kernel.
fun outer(other: Kernel1D): Kernel2D	The outer product this ⊗ other as a full 2-D kernel — the separable→dense expansion.
override fun toString(): String

Kernel2D

class Kernel2D(val width: Int, val height: Int, val weights: FloatArray)

A dense 2-D convolution kernel, width x height, centred (odd dimensions expected).

Declaration	Summary
val width: Int	Taps across; at least 1, and odd if the kernel is to have a true centre.
val height: Int	Taps down; at least 1, and odd if the kernel is to have a true centre.
val weights: FloatArray	The weights, row-major: weights[y * width + x].
val radiusX: Int	Half-width; column radiusX holds the centre tap.
val radiusY: Int	Half-height; row radiusY holds the centre tap.
fun sum(): Float	Sum of the weights — 1 for a normalised smoothing kernel, 0 for a derivative kernel.
fun flipped(): Kernel2D	180-degree rotation — the difference between convolution and correlation.

Gradient

class Gradient(val dx: Img, val dy: Img)

A gradient field: per-pixel partial derivatives of a scalar image.

Declaration	Summary
val dx: Img	Derivative along +x (rightwards), 1-channel.
val dy: Img	Derivative along +y (downwards, matching image coordinates), 1-channel.
val width: Int	Width of either component; the constructor requires them to agree.
val height: Int	Height of either component; the constructor requires them to agree.
fun magnitude(): Img	Per-pixel sqrt(dx² + dy²).
fun magnitudeL1(): Img	Per-pixel \|dx\| + \|dy\| — the cheap L1 stand-in for magnitude.
fun orientation(): Img	Per-pixel gradient direction in radians, atan2(dy, dx) in (-π, π]. Note dy points down, so a bright-below-dark horizontal edge has orientation +π/2.
fun orientationUnsigned(): Img	Orientation folded into [0, π) — direction without sign, the form Hough and HOG want.
fun toColorImg(): Img	Colour visualisation: hue = direction, value = magnitude (normalised).
override fun toString(): String

GaussianPyramid

class GaussianPyramid(val levels: List<Img>)

A Gaussian pyramid: levels[0] is the original, each subsequent level is pyrDown of the last. Stops early if a level would fall below minSize on either axis.

Declaration	Summary
val levels: List<Img>	Coarsening octaves, finest first: levels[0] is the source image. At least one.
val size: Int	Number of octaves, including the base.
val base: Img	The full-resolution base.
operator fun get(i: Int): Img	Octave i, counting from 0 at full resolution.
override fun toString(): String

LaplacianPyramid

class LaplacianPyramid(val bands: List<Img>, val residual: Img)

A Laplacian pyramid: bands hold the band-pass detail at each octave and residual is the coarsest low-pass. reconstruct inverts the decomposition (to within the blur's precision), which is exactly the invariant the unit test checks.

Declaration	Summary
val bands: List<Img>	Band-pass detail, finest first: bands[0] is at the source resolution. Signed.
val residual: Img	The coarsest low-pass, left over once every band has been subtracted.
val size: Int	Number of detail bands.
fun reconstruct(sigma: Double = 1.0): Img	Collapse back to a single image: start from residual, upsample to each band's size and add the band back on. Round-trips the source image to within a small numerical error.
override fun toString(): String

imaging_geometry.kt

Geometry and motion for Img: affine and perspective warps, homography estimation from point correspondences (DLT + RANSAC), and pyramidal Lucas–Kanade optical flow.

This is the part of the imaging family that relates two pictures rather than transforming one. The through-line is:

val pts   = a.goodFeaturesToTrack(120)                 // where is there anything to track?
val flow  = opticalFlowLK(a, b, pts)                   // where did each of those go?
val fit   = findHomographyRansac(flow.sourcePoints(), flow.targetPoints())
val warped = b.warpPerspective(fit.homography.inverse()) // undo the motion

The linear algebra is self-contained (a 9x9 Jacobi eigen-solver and a 6x6 Gaussian elimination), because a 3x3 homography does not justify a dependency and the whole point of this family is that it works headless in a box with nothing installed.

src/jvmMain/kotlin/afield/media/image/imaging_geometry.kt — read the source

Used in: Imaging and computer vision, Visualizers, Displays, and Little Instruments

Top-level functions

Declaration	Summary
fun solveDenseSystem(A: Array<DoubleArray>, b: DoubleArray): DoubleArray	Solve A x = b for a small dense square system by Gaussian elimination with partial pivoting.
fun smallestEigenvector(symmetric: Array<DoubleArray>, sweeps: Int = 64): DoubleArray	Eigenvector of the smallest eigenvalue of a small symmetric matrix, by the cyclic Jacobi rotation method. Used to find the null space of AᵀA in the DLT, which is what makes homography estimation possible without an SVD library.
fun findHomography(src: List<Vec2>, dst: List<Vec2>): Homography	Direct Linear Transform: the exact homography through 4 correspondences, or the least-squares homography through more.
fun findHomographyRansac(src: List<Vec2>, dst: List<Vec2>, threshold: Double = 3.0, iterations: Int = 500, seed: Long = 0L): RansacFit	RANSAC homography estimation: repeatedly fit a homography to a random minimal sample of 4 correspondences, score it by how many of the rest it maps to within threshold pixels, and keep the best consensus set — then refit on that set with a full least-squares findHomography.
fun Img.warpAffine(t: Affine2D, outWidth: Int = width, outHeight: Int = height, bilinear: Boolean = true, border: Border = Border.ZERO): Img	Resample through an affine transform. The transform maps source → destination; the implementation inverts it and does the correct backwards mapping (for every output pixel, find where it came from), so there are no holes.
fun Img.warpPerspective(h: Homography, outWidth: Int = width, outHeight: Int = height, bilinear: Boolean = true, border: Border = Border.ZERO): Img	Resample through a projective transform, mapping source → destination (inverted internally, see warpAffine). Pixels whose source falls outside the image are resolved by border.
fun Img.rotate(radians: Double, bilinear: Boolean = true, border: Border = Border.ZERO): Img	Rotate about the image centre by radians, keeping the same canvas size. Positive angles rotate clockwise on screen (y points down).
fun Img.warpQuadToRect(quad: List<Vec2>, outWidth: Int, outHeight: Int, bilinear: Boolean = true): Img	Rectify a quadrilateral in this image to a outWidth x outHeight rectangle — the "flatten the photographed page" operation, and the most immediately convincing use of a homography.
fun opticalFlowLK(a: Img, b: Img, points: List<Vec2>, windowRadius: Int = 7, levels: Int = 3, iterations: Int = 20, minEigenvalue: Double = 1e-4, epsilon: Double = 0.01): FlowField	Pyramidal Lucas–Kanade sparse optical flow.
fun opticalFlowBlockMatch(a: Img, b: Img, step: Int = 16, blockRadius: Int = 8, maxShift: Int = 12): FlowField	Dense block-matching flow on a regular step-pixel grid: for each grid cell, search [-maxShift, maxShift]² for the integer displacement minimising the sum of absolute differences over a (2*blockRadius+1)² block. Brute force, but robust to the large displacements that break Lucas–Kanade's linearisation, and it needs no gradient.
fun frameDifference(a: Img, b: Img, threshold: Double = 0.0, blurSigma: Double = 0.0): Img	Absolute frame difference, optionally thresholded to a binary motion mask. The crudest and most reliable motion cue there is; pair with Img.open to drop sensor noise and Img.connectedComponents to count what moved.
fun updateBackground(background: Img, frame: Img, alpha: Double = 0.05): Img	Running background model: background = (1 - alpha) * background + alpha * frame. Returns the updated background; difference the next frame against it for a cheap foreground mask that adapts to slow lighting changes.

Affine2D

data class Affine2D(val a: Double, val b: Double, val c: Double, val d: Double, val e: Double, val f: Double)

A 2-D affine transform, [a c e; b d f; 0 0 1] — the same six-number layout as java.awt.geom.AffineTransform and SVG's matrix(a, b, c, d, e, f).

Declaration	Summary
val a: Double	Output x per unit of input x: the horizontal scale, where there is no rotation or shear.
val b: Double	Output y per unit of input x: the vertical shear.
val c: Double	Output x per unit of input y: the horizontal shear.
val d: Double	Output y per unit of input y: the vertical scale, where there is no rotation or shear.
val e: Double	Translation along x, added after the linear part.
val f: Double	Translation along y, added after the linear part.
fun apply(x: Double, y: Double): Pair<Double, Double>	Apply to a point.
fun apply(v: Vec2): Vec2	Apply to a Vec2.
fun determinant(): Double	Determinant of the linear part — the area scale factor.
fun inverse(): Affine2D	Inverse transform. Throws if the linear part is singular.
fun compose(other: Affine2D): Affine2D	this applied after other: this ∘ other.
fun toHomography(): Homography	Promote to a Homography (last row 0 0 1).

Homography

class Homography(val m: DoubleArray)

A 2-D projective transform (homography), a 3x3 matrix in row-major order, mapping (x, y) -> ((m0 x + m1 y + m2)/(m6 x + m7 y + m8), (m3 x + m4 y + m5)/(m6 x + m7 y + m8)).

Declaration	Summary
val m: DoubleArray	The nine coefficients, row-major; exactly nine, or the constructor throws.
fun apply(x: Double, y: Double): Pair<Double, Double>	Map a point; returns (x', y') after the perspective divide.
fun apply(v: Vec2): Vec2	Map a Vec2.
fun compose(other: Homography): Homography	Matrix product this * other (apply other first).
fun inverse(): Homography	Adjugate-based inverse (equivalent up to scale, which is all a homography has).
fun normalized(): Homography	Rescale so m8 == 1 where possible, otherwise to unit Frobenius norm.
fun maxCoefficientDifference(other: Homography): Double	Largest absolute coefficient difference against other, both normalized.
fun isAffine(tol: Double = 1e-9): Boolean	true if the last row is (0, 0, k) — i.e. this is really an affine transform.
override fun toString(): String

RansacFit

class RansacFit(val homography: Homography, val inliers: List<Int>, val iterations: Int)

Result of findHomographyRansac.

Declaration	Summary
val homography: Homography	The model refit by least squares on the inlier set, not the minimal sample that found it.
val inliers: List<Int>	Indices, into the input correspondence lists, of the pairs that agree with homography.
val iterations: Int	How many random minimal samples were actually drawn.
val inlierCount: Int	Size of the consensus set.
fun inlierRatio(total: Int): Double	Inlier fraction of the supplied correspondences.
override fun toString(): String

FlowVector

data class FlowVector(val x0: Double, val y0: Double, val x1: Double, val y1: Double, val ok: Boolean, val error: Double)

One tracked point's motion, as produced by opticalFlowLK.

Declaration	Summary
val x0: Double	Column in the first frame, as supplied to the tracker.
val y0: Double	Row in the first frame, as supplied to the tracker.
val x1: Double	Column in the second frame, sub-pixel.
val y1: Double	Row in the second frame, sub-pixel.
val ok: Boolean	false if the solve was ill-conditioned or the point left the image. Check it before reading x1 and y1: a failed track still reports a position.
val error: Double	Mean absolute residual over the tracking window after convergence.
val dx: Double	Horizontal displacement in pixels, positive rightwards.
val dy: Double	Vertical displacement in pixels, positive downwards.
val magnitude: Double	Displacement magnitude in pixels.
fun from(): Vec2	Start point as a Vec2.
fun to(): Vec2	End point as a Vec2.

FlowField

class FlowField(val vectors: List<FlowVector>) : Iterable<FlowVector>

The result of a sparse optical-flow run.

Declaration	Summary
val vectors: List<FlowVector>	Every vector the producer emitted, failed tracks included: one per input point for opticalFlowLK, one per grid cell for opticalFlowBlockMatch.
override fun iterator(): Iterator<FlowVector>	Every vector, converged or not; tracked is the filtered view.
fun tracked(): List<FlowVector>	Only the vectors whose track converged.
fun sourcePoints(): List<Vec2>	Source points of the converged tracks (pairs with targetPoints for homography fitting).
fun targetPoints(): List<Vec2>	Target points of the converged tracks.
fun medianMagnitude(): Double	Median displacement magnitude over the converged tracks — a robust "how much moved".
fun medianDisplacement(): Pair<Double, Double>	Component-wise median displacement — a robust global translation estimate.
override fun toString(): String

imaging_overlay.kt

Drawing FLines into an Img, in software and without AWT.

Img.drawFLines is the whole surface. It resamples the image up by a scale factor and blends each line over it, reading color, thickness, pointed, filled and the rest from the line's own properties the way Field draws it on the canvas; round-capped segments and even-odd fills are rasterised here, each with a pixel of antialiasing. What comes back is another Img, so an overlay costs no GPU, no window and no AWT font or toolkit, and showImgGrid can show it wherever a box runs.

src/jvmMain/kotlin/afield/media/image/imaging_overlay.kt — read the source

Top-level functions

Declaration	Summary
fun Img.drawFLines(lines: List<FLine>, scale: Double = 1.0, color: Vec4 = Vec4(1f, 0.3f, 0.1f, 1f), thickness: Double = 1.5): Img	This image as RGB with lines drawn over it, antialiased and without AWT. Line coordinates are in this image's pixels; the result is scale times larger, so thin curves over a small image stay crisp.

imaging_simd.kt

The SIMD span kernels the imaging filters are built out of.

Why this file exists

The scalar filter layer reads every sample through Img.get, which per multiply-add costs a range check, a channel clamp and a when over Border. That is correct and readable, and it is also unvectorizable: C2 cannot hoist a branch whose outcome depends on the loop index out of the inner loop, so a 3-tap Sobel pays three bounds-checked calls per sample.

The fix is not to make the accessor faster — it is to stop calling it in the interior. Because Img stores samples interleaved (data[(y * width + x) * channels + c]), both passes of a separable convolution address contiguous memory:

• Vertical. For a fixed output row, tap t reads the whole of source row y + t - r. The border decision is therefore per row, not per pixel: resolve the row index once and the entire row is a straight span. Every row takes the fast path.
• Horizontal. Adding (t - r) * channels to a flat index moves to the same channel of the pixel t - r columns over. So for a fixed tap the whole interior of a row is, again, a straight span — only the r columns at each end need the general accessor.

The one primitive that matters: stencil

Both observations reduce a convolution — separable or dense — to the same shape: a destination span is a weighted sum of k source spans at fixed element offsets. That is stencil, and it keeps the running sum in a vector register across all taps, so k taps cost k reads and one write.

The alternative — accumulate in memory, one axpy per tap — costs k reads and k read-modify-writes. For the sigma-2 Gaussian used in the benchmark (radius 8, 17 taps) that is 17 write passes over the image where one would do, and it is why stencil rather than axpy is the inner loop of everything here. axpy survives only for the handful of border rows where each tap resolves to a different, border-dependent source row and there is no single offset to fold.

Why the main loop keeps four accumulators

stencil is a chain of k dependent FMAs into one accumulator. On Apple silicon an FMA has ~4 cycles of latency and can issue ~4 per cycle, so a single-accumulator 17-tap chain runs at latency, not throughput — about 16x off peak. The main loop therefore keeps four independent accumulators (four output chunks in flight), which converts the chain into something the machine can pipeline, then falls back to one chunk at a time and finally to scalar for the tail. This is the difference between "uses the Vector API" and "is fast".

Decimation, without a gather

A pass that throws half its output away should not compute it. Halving rows is free (just skip them), but halving columns makes the reads stride-2, which on NEON means a gather. splitByPhase avoids it: output x reads source pixel 2x + s, whose parity is fixed by the parity of s. So even-offset taps only ever read even-indexed pixels and odd-offset taps only odd ones. Split the row into its even and odd pixels once — into one scratch buffer, laid out even-half then odd-half, so a single stencil call can address both — and every tap is unit-stride again. This is the polyphase decomposition, and it is what makes Img.pyrDown a half-resolution filter rather than a full-resolution one with a discard at the end.

On FMA

These kernels use fused multiply-add: one rounding per tap instead of two. Results therefore differ from the scalar reference in the last ulp — more accurate, not less, but not bit-identical. ImagingSimdTest asserts agreement to 2e-6f, which is four orders of magnitude above this.

src/jvmMain/kotlin/afield/media/image/imaging_simd.kt — read the source

NodeTool.kt

Locates a node binary new enough to run the JavaScript this repository generates, for the tests and tools that shell out to Node.

Generated Kotlin/JS uses globalThis (Node 12+) and optional chaining (Node 14+). A shell that inherits several nvm bin directories can put an old Node first on PATH, and taking the first match then fails inside a temporary .js file with a SyntaxError that says nothing about the binary. NodeTool.resolveNode therefore scans PATH for a candidate that meets the version floor rather than the first one that exists.

Resolution order, first match wins:

1. AFIELD_NODE / NODE, if set and executable. An explicit override is honoured even below the floor, so asking for a binary by path gets that binary, and a clear failure if it is too old.
2. Every PATH entry, in order, whose node reports a major version at or above the floor (NodeTool.DEFAULT_MIN_MAJOR unless the caller passes one).
3. The repository-local Node that Gradle installs under .gradle/nodejs, so JS-backed tests work under a minimal PATH.

Nothing qualifying gives null. Call sites decide whether that is a skip or a failure, and NodeTool.unavailableMessage renders the reason either way.

src/jvmMain/kotlin/afield/media/image/util/NodeTool.kt — read the source

NodeTool

object NodeTool

Resolves and caches a node binary that meets a major-version floor; see the file overview for the resolution order.

Declaration	Summary
const val DEFAULT_MIN_MAJOR: Int	Node major version required to run this repository's generated JavaScript.
data class Candidate(val path: String, val major: Int?, val version: String?)	A node candidate found on disk, with whatever --version reported.
fun resolveNode(minMajor: Int = DEFAULT_MIN_MAJOR): String?	Absolute path to a usable node, or null if none meets minMajor.
fun unavailableMessage(minMajor: Int = DEFAULT_MIN_MAJOR): String	Why resolveNode came back null, phrased for a test's skip/failure message: what was found, what version it reported, and what was needed.

PngEncoder.kt

Encodes ARGB pixels, or an RGBA ByteBuffer read back from the GPU, into PNG bytes without AWT or javax.imageio.

Output is 8-bit RGBA (PNG colour type 6), one unfiltered scanline per row, compressed with java.util.zip.Deflater, with an sRGB chunk. PngEncoder.encode takes one ARGB Int per pixel; PngEncoder.encodeFromBuffer takes a readback buffer and can flip it vertically and force alpha to opaque.

src/jvmMain/kotlin/afield/media/image/util/PngEncoder.kt — read the source

PngEncoder

object PngEncoder

Pure-Kotlin PNG writer used for screenshot capture and image export.

Declaration	Summary
fun encode(pixels: IntArray, width: Int, height: Int): ByteArray	Encode ARGB pixel data to PNG format.
fun encodeFromBuffer(buffer: ByteBuffer, width: Int, height: Int, flipY: Boolean = false, opaque: Boolean = false): ByteArray	Convert a ByteBuffer containing RGBA image data to PNG bytes.

DlssReconstruction.kt

DLSS super resolution, DLAA and Ray Reconstruction through NVIDIA NGX, as a pass recorded into a Vulkan command buffer.

DlssReconstruction takes render-resolution color, depth and motion (ReconstructionInputs), plus RayReconstructionGuides when ray reconstruction is on, and writes an output-resolution texture. Each frame is described by a ReconstructionFrame: its index, the ReconstructionJitter the scene was sampled with (ReconstructionJitter.halton gives a sequence sized to the upscale ratio), frame time and pre-exposure. Call reset on camera cuts and live edits so history is discarded.

src/jvmMain/kotlin/afield/media/reconstruction/DlssReconstruction.kt — read the source

Used in: From modules to reconstruction, Neural reconstruction, Ray Reconstruction

ReconstructionInputs

data class ReconstructionInputs(val color: HasImageView,val depth: HasImageView,val motion: HasImageView, val motionCurrentMinusPrevious: Boolean=false,val motionInPixels: Boolean=true)

Motion defaults to input pixels: origin top left, X right, Y down, current-to-previous, without jitter. For Field direct-lighting motion, set motionCurrentMinusPrevious true and motionInPixels false; the adapter supplies the corresponding sign and scale to NGX without an intermediate image pass. Color is opaque linear HDR RGB, pre-exposure explicitly supplied; coverage/latent channels stay outside reconstruction. Depth is standard Vulkan [0,1], background=1. All inputs are render-resolution, sampled images, mip 0 / layer 0; producer and consumer must use the renderer's graphics queue.

Declaration	Summary
val motionCurrentMinusPrevious: Boolean

RayReconstructionGuides

data class RayReconstructionGuides(val diffuseAlbedo: HasImageView,val specularAlbedo: HasImageView, val normalRoughness: HasImageView,val worldToView: afield.math.linalg.Matrix4f,val viewToClip: afield.math.linalg.Matrix4f,val specularHitDistance: HasImageView?=null)

RR guides at input resolution. Diffuse/specular albedo are the actual BRDF response; normals.xyz are world-space unit normals and .w is linear roughness. Matrices are unjittered column-major right-multiply (equivalent to NGX's row-major left-multiply convention). Optional specular hit distance is world-space distance from the primary surface; sky=65504.

Declaration	Summary
val normalRoughness: HasImageView,val worldToView: afield.math.linalg.Matrix4f,val viewToClip: afield.math.linalg.Matrix4f,val specularHitDistance: HasImageView?

ReconstructionFrame

data class ReconstructionFrame(val index: Long,val jitter: ReconstructionJitter,val milliseconds: Float=16.6667f, val preExposure: Float=1f,val reset: Boolean=false,val revision: Long=0)

Declaration	Summary
val preExposure: Float

ReconstructionJitter

data class ReconstructionJitter(val x: Float,val y: Float)

Projection displacement in input pixels, positive right/down. Sample the scene at pixel-jitter.

DlssReconstruction

class DlssReconstruction(val renderer: Renderer,val inputSize: ReconstructionSize,val outputSize: ReconstructionSize, val mode: ReconstructionMode,val rayReconstruction: Boolean=false) : AutoCloseable

One mono view. Construct/edit/evaluate/close on the render thread, outside a render pass. Query recommend before allocating the scene targets. Recreate on resolution/mode/format change; the old instance retires behind Field's GPU completion queue. Failure leaves caller's source intact. SR is independent of ray tracing. UI and unsupported animated layers belong after this pass.

Declaration	Summary
val mode: ReconstructionMode,val rayReconstruction: Boolean
val recommendation
val output: RenderToTexture
var resetCount
var resetReason
fun reset(reason: String="camera cut or live edit")
fun evaluate(command: CommandBuffer,index: Int,source: RenderToTexture,inputs: ReconstructionInputs, frame: ReconstructionFrame,guides: RayReconstructionGuides?=null)	Record source, then reconstruct its completed attachments in the same command buffer. This owns the source's scheduling; registering it separately in preTargets is rejected, including registrations added after this view was created.
fun evaluate(command: CommandBuffer,inputs: ReconstructionInputs,frame: ReconstructionFrame,guides: RayReconstructionGuides?=null)
override fun close()

NeuralToolsFallbackSmoke.kt

src/jvmMain/kotlin/afield/media/reconstruction/NeuralToolsFallbackSmoke.kt — read the source

NeuralToolsFallbackSmoke

object NeuralToolsFallbackSmoke

Exercises the same renderer with every newly optional native/hardware feature disabled.

Declaration	Summary
fun main(args: Array<String>)

NgxSupport.kt

Detection of NVIDIA NGX support and the sizes and modes DLSS reconstruction accepts.

NgxSupport reports whether DLAA, DLSS super resolution and Ray Reconstruction are available on a device; ordinary rendering never requires NGX. ReconstructionMode lists the quality modes, and ReconstructionRecommendation holds the optimal, minimum and maximum input sizes for an output.

src/jvmMain/kotlin/afield/media/reconstruction/NgxSupport.kt — read the source

Used in: Neural reconstruction, From modules to reconstruction

NgxSupport

object NgxSupport

Feature-specific SR/DLAA discovery; ordinary rendering never requires NGX.

Declaration	Summary
const val SDK_REVISION
const val SDK_VERSION
fun availability(device: Device): Result<String>
fun rayReconstructionAvailability(device: Device): Result<String>

ReconstructionMode

enum class ReconstructionMode(internal val sdkValue: Int)

Constants: DLAA, QUALITY, BALANCED, PERFORMANCE, ULTRA_PERFORMANCE.

ReconstructionSize

data class ReconstructionSize(val width: Int,val height: Int)

ReconstructionRecommendation

data class ReconstructionRecommendation(val optimal: ReconstructionSize,val minimum: ReconstructionSize,val maximum: ReconstructionSize)

Declaration	Summary
fun smallestFor(output: ReconstructionSize): ReconstructionSize	Smallest legal input at the output's aspect ratio, rounded up to whole pixels.

PathTracedReconstruction.kt

Runs a path tracer, converts its output into ray reconstruction guides, and denoises it with DLSS Ray Reconstruction, in one ordered pass.

PathTracedReconstruction takes an uninstalled PathTracingRenderer, owns its dispatch and closes it. view selects the raw, progressive or reconstructed image, and gpuMilliseconds reports the GPU time of tracing, guide conversion and NGX together.

src/jvmMain/kotlin/afield/media/reconstruction/PathTracedReconstruction.kt — read the source

PathTracedReconstruction

class PathTracedReconstruction(val tracer: PathTracingRenderer, val outputSize: ReconstructionSize=ReconstructionSize(tracer.width,tracer.height), val mode: ReconstructionMode=ReconstructionMode.DLAA,val name: String="path-reconstruction") : Target,AutoCloseable

One ordered path-trace -> raw guides -> NGX pass. This owner records the tracer exactly once, and closes it; install takes a tracer from pathTracingHere(...) off its standalone passes. The raw and progressive sources remain inspectable at all times. Camera motion reprojects; call reset on cuts and appearance changes. Tracer property and shader revisions reset RR automatically. Recreate this owner to change input/output sizes.

Declaration	Summary
val outputSize: ReconstructionSize
val mode: ReconstructionMode
enum class View
val source
val reconstruction
var view
fun install(): PathTracedReconstruction
fun reset(reason: String="camera cut or appearance edit")
override fun render(command: CommandBuffer,index: Int)
override fun close()

PathTracedReconstructionSmoke.kt

src/jvmMain/kotlin/afield/media/reconstruction/PathTracedReconstructionSmoke.kt — read the source

PathTracedReconstructionSmoke

object PathTracedReconstructionSmoke

Real GPU contract, quality, motion and lifetime check. Readback is diagnostic only.

Declaration	Summary
fun main(args: Array<String>)

RayReconstructionDemo.kt

A direct-lighting scene lit by RTXDI whose output can be switched between raw, spatially denoised and DLSS Ray Reconstruction paths.

RayReconstructionDemo reuses DirectLightingDemo for the G-buffer and lighting, builds the guide images ray reconstruction needs, and runs two DlssReconstruction instances at native resolution. Setting path resets history and changes which image is shown.

src/jvmMain/kotlin/afield/media/reconstruction/RayReconstructionDemo.kt — read the source

RayReconstructionDemo

class RayReconstructionDemo(val renderer: Renderer,val size: ReconstructionSize=ReconstructionSize(1920,1080), val name: String="ray-reconstruction") : Target,AutoCloseable

Existing Fabricate G-buffer and RTXDI, with interchangeable reconstruction at native resolution. The direct-lighting bridge is Lambertian: specular albedo=0 and roughness=1 describe that BRDF. Do not reuse these guides for a glossy/path-traced signal. All editing happens between frames.

Declaration	Summary
val name: String
enum class Path
val scene
val settings
val rr
val sr
var path
var animate
var cameraMotion
var time
fun reset(reason: String="camera cut or edit")
override fun render(command: CommandBuffer,index: Int)
override fun close()

RayReconstructionSmoke.kt

src/jvmMain/kotlin/afield/media/reconstruction/RayReconstructionSmoke.kt — read the source

RayReconstructionSmoke

object RayReconstructionSmoke

Declaration	Summary
fun main(args: Array<String>)

ReconstructionDemo.kt

A small editable raster scene for trying DLAA and DLSS super resolution without ray tracing.

ReconstructionDemo renders color, depth and motion at the input resolution recommended for its ReconstructionMode and hands the three to DlssReconstruction. Leave mode null to see the unjittered source at output resolution. reload swaps in new vertex and fragment shaders, and reset discards reconstruction history after a scene edit.

src/jvmMain/kotlin/afield/media/reconstruction/ReconstructionDemo.kt — read the source

ReconstructionDemo

class ReconstructionDemo(val renderer: Renderer, val mode: ReconstructionMode? = ReconstructionMode.DLAA, val outputSize: ReconstructionSize = ReconstructionSize(1920, 1080), val name: String = "neural-aa", attach: Boolean = true, inputSize: ReconstructionSize? = null) : Target, AutoCloseable

Editable mono raster experiment; no ray tracing is needed. Replace at a frame boundary when changing mode/resolution. Leave mode null for the unjittered source at native output resolution.

Declaration	Summary
val renderer: Renderer
val mode: ReconstructionMode?
val outputSize: ReconstructionSize
val name: String
val inputSize
val color
val depth
val motion
val reconstruction
val output
var time
var abstractScene
var cameraMotion
var animate
var horizontalShiftPixels
fun attach(): ReconstructionDemo
fun reset(reason: String = "scene edit")
fun reload(vertex: String = shader("scene.vert"), fragment: String = shader("scene.frag")): Result<Unit>	Compile and reflect both shaders, reject changed layouts, then commit at a frame boundary.
override fun render(command: CommandBuffer, index: Int)
override fun close()

ReconstructionMotionGpuSmoke.kt

src/jvmMain/kotlin/afield/media/reconstruction/ReconstructionMotionGpuSmoke.kt — read the source

ReconstructionMotionGpuSmoke

object ReconstructionMotionGpuSmoke

Compare real NGX history fed pixel motion with the same motion encoded as Field UV velocity. An untagged UV control must disagree, so equality cannot pass by ignoring motion on both paths. Also exercises rejection of a source independently registered after view construction. Run explicitly through run-jvm-main; this requires the optional Linux/NVIDIA NGX adapter.

Declaration	Summary
fun main(args: Array<String>)

ReconstructionSmoke.kt

src/jvmMain/kotlin/afield/media/reconstruction/ReconstructionSmoke.kt — read the source

ReconstructionSmoke

object ReconstructionSmoke

Actual NGX evaluation with RT disabled. --clip records consecutive scene frames, not wall time. --fallback verifies ordinary rendering with FIELD_NGX_DISABLE=1, without initializing NGX.

Declaration	Summary
fun main(args: Array<String>)

WaylandReconstructionSmoke.kt

src/jvmMain/kotlin/afield/media/reconstruction/WaylandReconstructionSmoke.kt — read the source

WaylandReconstructionSmoke

object WaylandReconstructionSmoke

Driven by tools/lighting/check_wayland.py: the compositor, not GLFW, confirms minimization.

Declaration	Summary
fun main(args: Array<String>)

ElectronRTTView.kt

Shows a Field 4 render target live in a box's output area in the Field Electron app, sharing GPU surfaces with Electron instead of reading pixels back.

ElectronRTTView is the entry point: render into its ElectronRTTView.rtt and display its ElectronRTTView.html. Works on macOS and on Linux under native Wayland; outside Electron the widget says it needs the app.

src/jvmMain/kotlin/afield/media/stream/ElectronRTTView.kt — read the source

ElectronRTTView

class ElectronRTTView(val width: Int, val height: Int, private val ringSize: Int = 3, format: Int = VK_FORMAT_B8G8R8A8_SRGB, private val renderer: Renderer = fromContext(), val name: String = "electron-rtt-${counter.incrementAndGet()}") : Target, HasRenderToTexture, AutoCloseable

A Field 4 render target drawn straight into a box's output area in the Field Electron app.

Declaration	Summary
val width: Int
val height: Int
val name: String
override val rtt	Render your content into this; it is drawn to the canvas as is. +view and view += reach it too.
val viewId
val stats
fun html(): String	The output-area widget: a canvas the preload draws into, and the script that wires it up.
override fun render(command: CommandBuffer, index: Int)
override fun close()
data class Sample(val toMain: Double, val toRenderer: Double, val toDrawn: Double)	One frame's timeline, in ms after its fence completed.
class Stats

FieldEncodeProcess.kt

The JVM side of the field-encode helper, which encodes Field render targets (VideoToolbox on macOS, the field-encode-linux build on Linux) and streams them to the visionOS app.

FieldEncodeProcess starts the helper and exchanges newline-delimited JSON with it; image data never passes through this pipe. FieldRTTStreamer is the usual way to drive it from a box.

src/jvmMain/kotlin/afield/media/stream/FieldEncodeProcess.kt — read the source

FieldEncodeProcess

class FieldEncodeProcess(private val binaryPath: String = defaultBinaryPath(), private val workingDirectory: String? = null)

Launches and drives the field-encode Swift co-process (native/field-encode) — the macOS VideoToolbox sender for the visionOS stereo/flat docks (see dev_docs/vision_frontend.md -> "The sender seam" and native/field-encode/WIRE_FORMAT.md).

Declaration	Summary
var running: Boolean
var lastEvent: JSONObject?
var onEvent: ((JSONObject) -> Unit)?	Invoked for every JSON event line on the helper's stdout (opened/encoded/error/...).
var onExit: ((Int) -> Unit)?	Invoked once when the process exits, with its exit code.
fun start(): Boolean
fun alloc(width: Int, height: Int, count: Int)	Allocate a ring of count IOSurfaces in the helper and get their global IDs back (via an {"ev":"allocated","ids":[…]} event on onEvent). Field imports these by ID as its RTT backing — they carry a real pixelFormat, unlike MoltenVK's export. Precedes open.
fun open(width: Int, height: Int, fps: Int, bitrateKbps: Int, dest: String, codec: String = "hevc", idrIntervalMs: Int = 1000, surfaces: List<Long> = emptyList(), fdSurfaces: List<afield.graphics.texture.VulkanExternalMemory.FdHandle> = emptyList(), fdSocket: String? = null, stereo: Boolean = false)	Set up the encoder + UDP stream to dest ("host:port"). On macOS, pass surfaces (global IOSurfaceIDs) to look them up in the helper (legacy); omit to use the ring created by a prior alloc. On Linux, pass fdSurfaces + fdSocket — the helper has no alloc (it can't allocate a Vulkan image itself), and a raw fd number can't travel over JSON (it's per-process, not a system-wide handle like an IOSurfaceID): the real fds cross over an SCM_RIGHTS handoff on fdSocket instead (see afield.graphics.texture.UnixFdPass — caller sends them right after this command), and only size/rowPitch travel on the wire, in the same order as the fds sent over the socket.
fun frame(surface: Int, seq: Long, ptsMicros: Long, forceIdr: Boolean = false, poseFrameId: Long? = null, poseDisplayTimeNanos: Long? = null)	Signal that ring surface surface is rendered and ready to encode now.
fun bitrate(kbps: Int)
fun selftest(dest: String, frames: Int = 120, width: Int = 1280, height: Int = 720, fps: Int = 60, bitrateKbps: Int = 20000, codec: String = "hevc")	Fire the helper's self-contained smoke test (synthetic frames, no RTT needed).
fun send(cmd: JSONObject)
fun close()	Graceful: send close, then close stdin (helper exits its readLine loop), then reap.
fun stop()

FieldRTTStreamer.kt

Streams a Field 4 render target to the visionOS app as video (HEVC by default), through the field-encode helper.

FieldRTTStreamer owns a ring of shared surfaces that a box renders into; each frame is handed to the encoder once the GPU has finished it. Works on macOS (VideoToolbox) and Linux (NVENC).

src/jvmMain/kotlin/afield/media/stream/FieldRTTStreamer.kt — read the source

Used in: Field on Apple Vision Pro

FieldRTTStreamer

class FieldRTTStreamer(private val renderer: Renderer, private val dest: String = "127.0.0.1:9000", private val width: Int = 1280, private val height: Int = 720, private val ringSize: Int = 1, private val fps: Int = 60, private val bitrateKbps: Int = 20_000, private val codec: String = "hevc", private val idrIntervalMs: Int = 1000, private val stereo: Boolean = false, private val colorFormat: Int = VK_FORMAT_B8G8R8A8_UNORM, private val name: String = "field-stream", private val encoder: FieldEncodeProcess = FieldEncodeProcess()) : Target

Streams an IOSurface-backed RTT off the GPU to the visionOS receiver via the field-encode co-process (the producer seam from dev_docs/vision_frontend.md).

Declaration	Summary
val surfaces	The ring of RTTs to render into, created on start. Empty until then.
val ioSurfaceIds: List<Long>	macOS: the global IOSurfaceIDs of the helper's ring, in slot order. Empty until start.
var framesEncoded: Long	Frames the helper reported encoded.
var streaming: Boolean
var poseTag: (() -> StreamPose?)?	The head pose the frame about to be submitted was rendered from, or null when there isn't one.
fun tagPosesFrom(target: afield.graphics.vulkan.targets.SideBySideStereoTarget)	Tag each submitted frame with the pose target last latched.
val activeSlot: Int	The ring slot the next rendered frame will be encoded from (render into this one).
fun start(): List<RenderToTexture>	Launch the helper, allocate + import the surface ring, open the stream, and register the per-frame tick. Returns the ring RTTs to render into (empty on failure). Call from the render thread (a box). Blocks briefly for the alloc round-trip.
fun stop()	Stop ticking, unregister from the render list, and tear down the helper.
override fun render(command: CommandBuffer, index: Int)	Called during graphics recording each frame (this is a render-list Target). Defers the encoder's per-frame ready tick until this frame's fence signals — the correctness gate.

StreamPose

data class StreamPose(val frameId: Long, val displayTimeNanos: Long)

The head pose a streamed frame was rendered from, as the media wire carries it.

LinuxElectronImportProbe.kt

A standalone Linux probe that hands Electron one finished Vulkan image and reads back its verdict on the pixels.

It clears a 64×64 dma-buf-exportable RenderToTexture to a known colour, copies that back through a staging buffer to confirm the Field side is not blank, releases the image to VK_QUEUE_FAMILY_EXTERNAL behind a waited fence, and sends the descriptor over UnixFdPass with the plane's stride and offset and the allocation's DRM modifier written beside it as texture.json. Then it waits up to a minute for the Node side's result.json. Start tools/electron_dmabuf_probe/main.js first; the single argument is the directory the two sides exchange evidence in, and native/field-fdpass has to be built.

src/jvmMain/kotlin/afield/media/stream/LinuxElectronImportProbe.kt — read the source

LinuxElectronImportProbe

object LinuxElectronImportProbe

One immutable, fence-completed Vulkan image for the Electron nativePixmap import.

Declaration	Summary
fun main(args: Array<String>)

HardwareOpticalFlow.kt

The hardware optical-flow backend, which drives the VK_NV_optical_flow engine on NVIDIA Ada and later GPUs.

HardwareOpticalFlow is what OpticalFlowBackend.AUTO picks when the device has that extension enabled.

src/jvmMain/kotlin/afield/media/vision/HardwareOpticalFlow.kt — read the source

HardwareOpticalFlow

class HardwareOpticalFlow(override val renderer: Renderer,override val first: HasImageView,override val second: HasImageView, override val grid: Int=4,val quality: Quality=Quality.SLOW) : OpticalFlowOperation

Dedicated hardware image correspondence. Call evaluate between Field frames, after producers have submitted. Images stay on the GPU. This first implementation waits for one final fence; it does not overlap pairs, and includes conversion/synchronization in milliseconds.

Declaration	Summary
override val grid: Int
enum class Quality(internal val vk: Int)
override val width
override val height
override val flowWidth
override val flowHeight
val opticalTimestampBits
override val output	Sampled RGBA32F texture: XY displacement in input pixels, Z matching cost, W = 1. Dimensions are ceil(input width/grid) by ceil(input height/grid), owned by this operation.
override val milliseconds
override fun evaluate(reset: Boolean)	Reset temporal hints for a cut, image-order change or nonconsecutive pair.
override fun close()

OpticalFlow

typealias OpticalFlow = HardwareOpticalFlow

The old name of HardwareOpticalFlow, which read as the general entry point; opticalFlow(...) is that.

OpticalFlowDemo.kt

A self-contained optical-flow demonstration that renders two warped test images, measures the motion between them, and drives a feedback image with it.

OpticalFlowDemo builds its own inputs, so it runs without a camera or video source; it is a starting point for wiring opticalFlow output into shaders.

src/jvmMain/kotlin/afield/media/vision/OpticalFlowDemo.kt — read the source

OpticalFlowDemo

class OpticalFlowDemo(val renderer: Renderer, val width: Int = 960, val height: Int = 540, val name: String = "optical-flow", backend: OpticalFlowBackend = OpticalFlowBackend.AUTO) : AutoCloseable

Two generated images, their motion/cost, and a feedback image driven by that motion. backend defaults to OpticalFlowBackend.AUTO: the NV engine where it exists, PyramidOpticalFlow elsewhere.

Declaration	Summary
val renderer: Renderer
val width: Int
val height: Int
val name: String
val pair
val flow: OpticalFlowOperation
val output
var dx
var dy
var gain
var damping
val costDisplayRange	Cost at which the COST strip saturates. The backends' costs are on different scales (see OpticalFlowOperation): the hardware engine's raw cost, or PyramidOpticalFlow's mean window residual in grey levels.
var costThreshold	Advection keeps the previous image only where cost is below this; defaults are per backend, see costDisplayRange.
var deformation
var pairStep	Separation of the two source frames, in seconds. Advance update(time) by this amount.
var displayRange	Only affects arrows/color intensity, never the displacement delivered to another shader.
fun update(time: Float = 0f, reset: Boolean = false)
override fun close()
fun expectedDisplacement(x: Float, y: Float, time: Float): afield.math.linalg.Vector2f	Invert the known source warp: a feature at (x,y) in the first image moves to this offset. Diagnostic only; the runtime motion field comes exclusively from flow.

OpticalFlowOperation.kt

The shared contract for dense optical flow in Field: the OpticalFlowOperation interface, the OpticalFlowBackend choice, and the opticalFlow factory that picks a backend for a renderer.

Both backends, HardwareOpticalFlow and PyramidOpticalFlow, take two images of one renderer and write the same RGBA32F flow texture, so code that samples OpticalFlowOperation.output does not depend on which one ran.

src/jvmMain/kotlin/afield/media/vision/OpticalFlowOperation.kt — read the source

Used in: Optical flow

Top-level functions

Declaration	Summary
fun opticalFlow(renderer: Renderer,first: HasImageView,second: HasImageView,grid: Int=4, backend: OpticalFlowBackend=OpticalFlowBackend.AUTO): OpticalFlowOperation	Dense optical flow from first to second on whichever backend backend resolves to on this device.

OpticalFlowBackend

enum class OpticalFlowBackend

Which implementation computes a OpticalFlowOperation.

Constants: AUTO, HARDWARE, PYRAMID.

Declaration	Summary
fun resolve(renderer: Renderer): OpticalFlowBackend	The backend AUTO would pick on this renderer; other values are returned unchanged.

OpticalFlowOperation

interface OpticalFlowOperation : AutoCloseable

A dense motion field between two same-sized images of one renderer, kept on the GPU.

Declaration	Summary
val renderer: Renderer
val first: HasImageView
val second: HasImageView
val grid: Int
val width: Int
val height: Int
val flowWidth: Int
val flowHeight: Int
val backend: OpticalFlowBackend	The backend that actually runs, never OpticalFlowBackend.AUTO.
val output: RenderToTexture	Sampled RGBA32F motion field and cost, owned by this operation.
val milliseconds: ArrayDeque<Double>	Wall time of recent evaluate calls, including submission and the final fence.
val gpuTimings: Map<String,List<Double>>	Named GPU timestamp samples (milliseconds) for this backend's stages; empty lists without timestamps.
val allocatedBytes: Long	Bytes of the images and buffers this operation allocated (driver-private memory excluded).
fun evaluate(reset: Boolean=false)	Compute output from the current contents of first and second. reset marks a cut: no temporal hints.

OpticalFlowSmoke.kt

src/jvmMain/kotlin/afield/media/vision/OpticalFlowSmoke.kt — read the source

OpticalFlowSmoke

object OpticalFlowSmoke

Declaration	Summary
fun main(args: Array<String>)

PortableOpticalFlowSmoke.kt

src/jvmMain/kotlin/afield/media/vision/PortableOpticalFlowSmoke.kt — read the source

PortableOpticalFlowSmoke

object PortableOpticalFlowSmoke

Holds PyramidOpticalFlow to known synthetic motion, through the production consumer OpticalFlowDemo.

Declaration	Summary
fun main(args: Array<String>)

PyramidOpticalFlow.kt

The portable optical-flow backend: coarse-to-fine Lucas–Kanade built from ordinary fragment passes, for devices without a hardware motion engine.

PyramidOpticalFlow is what OpticalFlowBackend.AUTO picks on any device without VK_NV_optical_flow enabled, MoltenVK included.

src/jvmMain/kotlin/afield/media/vision/PyramidOpticalFlow.kt — read the source

PyramidOpticalFlow

class PyramidOpticalFlow(override val renderer: Renderer,override val first: HasImageView,override val second: HasImageView, override val grid: Int=4,val levels: Int=5,val radius: Int=4,val iterations: Int=8,val finestIterations: Int=2,val search: Int=4, val consistency: Boolean=false) : OpticalFlowOperation

Portable dense optical flow: coarse-to-fine Lucas–Kanade in ordinary fragment passes.

Declaration	Summary
override val grid: Int
val consistency: Boolean
override val width
override val height
override val flowWidth
override val flowHeight
override val output	The median of the finest level is output; every level has a raw and a median-filtered field.
val consistencyMap: RenderToTexture?	Forward–backward check, same dimensions as output; null unless consistency. See the class comment.
override val gpuTimings: Map<String,List<Double>>	"pyramid" covers conversion, downsampling and gradients; "levelN" covers level N's search, track and median.
override val milliseconds
override val allocatedBytes: Long
override fun evaluate(reset: Boolean)
override fun close()

WebRTC.kt

WebRTC video for Field 4 graphics: publish a render target to a classroom, and sample anyone's published stream as a texture.

webrtcSink publishes what you render into its rtt; webrtcSource is a texture showing one publisher; webrtcPublishers is the live list of who is publishing. They join aaaframe's ShaderTrinket network (set WebRTCNetwork.default, or AFIELD_WEBRTC_SERVER), where browser ShaderTrinkets see a Field sink as one more publisher, or Field's own relay for a LAN of Field instances. Chromium does the WebRTC; Field moves the frames on the GPU and relays signalling. How and why: dev_docs/WEBRTC_SOURCES_SINKS.md.

src/jvmMain/kotlin/afield/media/webrtc/WebRTC.kt — read the source

Used in: WebRTC Streams

Top-level functions

Declaration	Summary
fun webrtcSink(name: String, width: Int = 640, height: Int = 360, network: WebRTCNetwork = WebRTCNetwork.default, renderer: Renderer = fromContext()): WebRTCSink	Publishes a new width×height render target as login/name; render into the sink's rtt and display its html(). See WebRTCSink.
fun webrtcSink(view: ElectronRTTView, name: String, network: WebRTCNetwork = WebRTCNetwork.default): WebRTCSink	Publishes an existing ElectronRTTView as login/name.
fun webrtcSource(target: String, width: Int = 640, height: Int = 360, network: WebRTCNetwork = WebRTCNetwork.default, fit: String = "contain", key: String = target, measure: Boolean = false, renderer: Renderer = fromContext()): WebRTCSource	target's published stream as a width×height texture: textures("remote" to webrtcSource("alice")). fit is the video's CSS object-fit (contain, cover or fill). key tells several sources in one box apart, so a re-run reuses each one's page; it defaults to the target. With measure, a sink's time-code band is decoded into WebRTCSource.latency. See WebRTCSource.
fun webrtcPublishers(network: WebRTCNetwork = WebRTCNetwork.default): WebRTCPublishers	The live publisher list on network; see WebRTCPublishers. Closed with the box's run.

WebRTCSink

class WebRTCSink(val view: ElectronRTTView, val name: String, val network: WebRTCNetwork = WebRTCNetwork.default, val framesPerSecond: Int = 30) : HasRenderToTexture

A render target published as a WebRTC stream, named login/name on its network.

Declaration	Summary
val view: ElectronRTTView	The output-area view the stream is captured from.
val name: String	The publish suffix: the stream is login/name, or just login when empty.
val network: WebRTCNetwork
val framesPerSecond: Int	The frame rate asked of canvas.captureStream; Chromium encodes at most this many a second.
var publishedAs: String?	The name viewers watch, once the network has confirmed it (login/name); null until then.
var status: String	The page's last status line: "publishing @…", "publishing @… to 2 viewer(s)", or an error.
var viewers: Int	How many peers are receiving the stream now.
fun html(): String	The output-area widget: ElectronRTTView.html, plus the script that publishes its canvas and a status line naming the stream.

WebRTCSource

class WebRTCSource internal constructor(private val renderer: Renderer, target: String, val width: Int, val height: Int, val network: WebRTCNetwork, private val fit: String, measure: Boolean) : HasImageView, PreparesSampledImages, AutoCloseable

One publisher's WebRTC stream as a Field texture: textures("remote" to source).

Declaration	Summary
val width: Int
val height: Int
val network: WebRTCNetwork
var target: String	The publisher shown, as normalised by the network (alice/left).
var status: String	The page's last status: "requesting @…", "waiting for @… to publish", "receiving @…", or an error.
val browser	The hidden page's browser, for anything else you want to ask it (it is a full PuppetBrowser).
val latency	Latency samples decoded from a sink's time-code band (only with measure = true).
override val imageView: ImageView
override fun prepareSampledImagesFor(command: CommandBuffer)
val pageReady: Boolean	True once the page has painted a frame after loading (the stream itself may still be connecting).
fun watch(name: String)	Show name's stream instead (alice, @alice/left).
override fun close()
class LatencyStats	Percentiles of decoded time codes: sender's render to this page presenting the frame.

WebRTCPublishers

class WebRTCPublishers internal constructor(val network: WebRTCNetwork)

Who is publishing on a network, kept live: names now, onChange as it changes, and html as a panel with a button per publisher that calls onPick.

Declaration	Summary
val names: List<String>	The publisher names now, sorted.
val error: String?	Why the list is empty, if it is because the network refused or dropped the connection.
fun onChange(listener: (List<String>) -> Unit): () -> Unit	Calls listener with the names now and on every change; returns the remover.
fun onPick(listener: (String) -> Unit)	Calls listener with the name a button in html was clicked for.
fun html(highlight: () -> String? = { null }): String	A panel listing the publishers, live; highlight marks the one you are showing.
fun close()	Stops listening; the panel stops updating.

WebRTCPublish.kt

Publishing what Field renders as a WebRTC stream, and the colour conversion that makes the stream look like the window.

webrtcPublish is the one call a box makes: it creates a sink of the size you ask for and adds a copy pass that samples your source into it every frame. The copy is a full-screen triangle whose fragment shader is written for the source and target formats it finds, so an 8-bit UNORM source holding display values and a float or _SRGB one holding linear light both come out looking the same. WebRTCPublishTest pins which of the four pairings converts and which passes the colour through, and compiles every variant. The sink itself, the sources and the networks are in WebRTC.kt.

src/jvmMain/kotlin/afield/media/webrtc/WebRTCPublish.kt — read the source

Top-level functions

Declaration	Summary
fun Box.webrtcPublish(source: HasImageView, name: String, width: Int = sampleableSize(source, 640 to 360).first, height: Int = sampleableSize(source, 640 to 360).second, network: WebRTCNetwork = WebRTCNetwork.default, show: String? = "main", renderer: Renderer = fromContext()): WebRTCSink	Publish anything Field can sample — a RenderToTexture, a shadertoy canvas's rtt, a camera, a received stream — as the WebRTC stream login/name, and show its preview and status in this box's show output channel (null to show nothing).

WebRTCRoutes.kt

The signalling a Field WebRTC sink or source gets from its own server: the /ws/webrtc-signaling?network=<id> WebSocket its page exchanges signalling over, and /api/webrtc-config?network=<id>, which answers with that network's login and ICE servers.

Installed beside Field's other WebSockets. Both answer only for networks box code has registered (WebRTCNetwork.aaaframe/join, or the local relay), so a client cannot make Field connect anywhere else; the guard decisions are in dev_docs/server_surface_inventory.md. A local network is relayed inside this process and every other one is forwarded upstream to aaaframe; WebRTCSignalingTest drives the socket both ways.

src/jvmMain/kotlin/afield/media/webrtc/WebRTCRoutes.kt — read the source

Top-level functions

Declaration	Summary
fun Route.installWebRTCSignalingRoutes()	Adds both signalling endpoints to a Ktor route.

WebRTCSignaling.kt

WebRTC signalling for Field's sources and sinks: the networks a page can join, the relay on Field's own server, and the bridge to an aaaframe classroom.

Pages never connect to a signalling server themselves. They open /ws/webrtc-signaling?network=<id> on Field, and Field either relays the messages itself (WebRTCNetwork.local) or copies them to and from aaaframe's /ws/shadertrinket-signaling (WebRTCNetwork.aaaframe), adding the student's session cookie. The messages are aaaframe's in both cases, so the page code is the same. The design is in dev_docs/WEBRTC_SOURCES_SINKS.md.

src/jvmMain/kotlin/afield/media/webrtc/WebRTCSignaling.kt — read the source

WebRTCNetwork

class WebRTCNetwork private constructor(val id: String, val baseUrl: String?, session: String?)

A signalling network: Field's own relay, or an aaaframe server's ShaderTrinket network.

Declaration	Summary
val id: String
val baseUrl: String?	The aaaframe base URL (https://host or http://host:port), or null for the local relay.
val isLocal: Boolean
val hasSession: Boolean	True when there is a session to present: from aaaframe's argument, the environment, or a sign-in.
override fun toString(): String