Encoding (TextEncoder / TextDecoder)

Elide implements the WHATWG Encoding Standard globals TextEncoder and TextDecoder, available on globalThis and mirrored at require('node:util').TextEncoder / require('node:util').TextDecoder.

`TextEncoder`

new TextEncoder()

Member	Type	Notes
`encoding`	`string`	Readonly; always `"utf-8"`.
`encode(input = "")`	`Uint8Array`	Returns a fresh `Uint8Array` containing the UTF-8 bytes.
`encodeInto(source, destination)`	`{ read, written }`	Writes into `destination`; never writes a partial code point.

Unpaired surrogate code units in the input are replaced with U+FFFD, producing EF BF BD in the output.

`TextDecoder`

new TextDecoder(label = "utf-8", options = {})
// options: { fatal: boolean = false, ignoreBOM: boolean = false }

Member	Type	Notes
`encoding`	`string`	Readonly; canonical lowercase encoding name (e.g. `"utf-8"`).
`fatal`	`boolean`	Readonly; `true` → malformed input throws `TypeError`.
`ignoreBOM`	`boolean`	Readonly; when `false` and encoding is `utf-8`/`utf-16le`/`utf-16be`, a leading BOM is stripped.
`decode(input?, options?)`	`string`	`options.stream: boolean = false`. Partial trailing bytes are retained for the next call only when `stream` is true.

Supported labels

Every label defined by the WHATWG Encoding names-and-labels table. Single-byte encodings are decoded directly from the standard’s index tables (§5) rather than a JVM Charset, so encodings the JVM lacks (iso-8859-10, iso-8859-14, iso-8859-8-i, macintosh, x-mac-cyrillic) are fully supported with spec-correct error semantics. This covers:

Unicode: utf-8 (and aliases utf8, unicode-1-1-utf-8, unicode11utf8, unicode20utf8, x-unicode20utf8), utf-16le (aliases utf-16, unicode, ucs-2, etc.), utf-16be.
Western: windows-1252 (default for ascii, iso-8859-1, latin1), iso-8859-2 … iso-8859-16, windows-1250 … windows-1258, macintosh, x-mac-cyrillic.
Cyrillic: ibm866, iso-8859-5, koi8-r, koi8-u, windows-1251.
Greek / Arabic / Hebrew / Thai: iso-8859-6/7/8/8-i, windows-874.
CJK: gbk, gb18030, big5, euc-jp, iso-2022-jp, shift_jis, euc-kr — decoded by Elide’s own Encoding-Standard §11-§13 state machines (see below), including gb18030’s 4-byte ranges, Big5’s astral and two-code-point pointers, and the stateful ISO-2022-JP escape modes.
Other: x-user-defined (algorithmic: 0x80–0xFF → U+F780 + byte − 0x80). The replacement encoding and its labels (iso-2022-cn, iso-2022-kr, hz-gb-2312, csiso2022kr) are recognized but rejected at construction with RangeError, per the standard’s security-motivated handling.

Label normalization: ASCII whitespace (tab, LF, FF, CR, space) is stripped from each end and the remainder is lowercased. Unknown labels throw RangeError. For single-byte encodings, a byte with no mapping produces U+FFFD (or, when fatal is set, throws TypeError).

Call vs `new`

Both constructors must be invoked with new; calling TextEncoder(...) or TextDecoder(...) as plain functions throws TypeError.

Streaming: `TextEncoderStream` / `TextDecoderStream`

TextEncoderStream and TextDecoderStream are installed on globalThis as TransformStreams built over TextEncoder / TextDecoder. TextEncoderStream turns a stream of strings into UTF-8 Uint8Array chunks; TextDecoderStream turns a stream of byte chunks into strings, carrying partial multi-byte sequences across chunk boundaries.

new TextDecoderStream([label][, options]) — options.fatal and options.ignoreBOM are honoured; instances expose encoding, fatal, ignoreBOM, and the readable / writable pair.
new TextEncoderStream() — always UTF-8; exposes encoding and the readable / writable pair.

const lines = byteSource.pipeThrough(new TextDecoderStream());
for await (const chunk of lines) {
  // chunk: decoded string
}

Example

const enc = new TextEncoder();
const bytes = enc.encode("Hello, мир! 🌍");
// bytes = Uint8Array([72,101,108,108,111,44,32,208,188,208,184,209,128,33,32,240,159,140,141])

const dec = new TextDecoder("utf-8");
dec.decode(bytes); // → "Hello, мир! 🌍"

// Streaming across partial multi-byte sequences:
const streamer = new TextDecoder();
streamer.decode(new Uint8Array([0xc3]), { stream: true }); // → ""
streamer.decode(new Uint8Array([0xa9]));                    // → "é"

// fatal mode raises TypeError on invalid input instead of replacing with U+FFFD:
const strict = new TextDecoder("utf-8", { fatal: true });
strict.decode(new Uint8Array([0xff])); // throws TypeError

Notes on the implementation

Single-byte encodings (and x-user-defined) are decoded by Elide’s own Encoding-Standard §9.1 decoder in JSTextDecoder.java, keyed off vendored index tables in SingleByteIndexes.java (data from the standard’s indexes.json). This gives one code path with correct fatal/replacement semantics on every host and does not depend on the JVM shipping the charset.
The legacy multi-byte encodings are likewise decoded directly: MultiByteDecoders.java holds the six §11-§13 decoder state machines (gb18030/gbk share one; big5; euc-jp; shift_jis; euc-kr; the stateful iso-2022-jp), driven over a byte queue that supports the standard’s byte-restore error recovery and retains decoder state across streaming chunks. Their index tables are vendored in MultiByteIndexes.java (chunked interned strings inflated to char[] on first use; Big5’s astral code points in a sparse side table; gb18030’s 4-byte ranges as small int[]s). Only UTF-8 and UTF-16 still delegate to a JVM CharsetDecoder.
The fix that routes plain-call TextEncoder(...) / TextDecoder(...) to TypeError lives in the generated JSTextEncoderClassBase.java and JSTextDecoderClassBase.java under packages/generated/. Upstream elide-dev/bindgen has the corresponding template (JSConstructorCodegen.kt); the next regenerate will emit the same branch automatically.
TextDecoder instance creation explicitly reinstalls the intrinsic prototype via JSObjectUtil.setPrototypeImpl after JSObjectFactory.initProto — in single-context mode the factory’s initProto only swaps the prototype when it differs from the cached default, and our subclass-with-extra-fields path doesn’t always hit the shape-embedded prototype correctly. The explicit call guarantees instance instanceof TextDecoder holds.

TextEncoder

TextDecoder