Editor’s Note: As we mark the celebration of the Three.js Conference, Edoardo Lunardi introduces an intricate examination of ASCII rendering. He moves past conventional luminance ramps to demonstrate how GPU-based shape-aware sampling can revolutionize 3D to ASCII transformations.

Common methods of ASCII rendering typically rely on color gradients that translate brightness into characters, but there's a significant shortfall when it comes to detailing edges. When a solid form has a complex edge, such as a diagonal line, the result can appear grainy due to the limited character resolution, generating artifacts instead of the crisp transitions you’d expect at the edges.

My own work with ASCII expands into the domains of dithering and nostalgia for vintage technology aesthetics. Each of these projects runs into the same limitation: the finite character grid combined with minimal representation options. The challenge, therefore, is recognizing the best character for each specific locale on that grid.

In my latest iteration, I’ve taken the Codrops logo and crafted it as a draggable shape rendered entirely in ASCII via GPU processes. Every character cell in this rendering method samples multiple points within its boundaries alongside neighboring cells, constructs a shape vector, then cross-references that against 95 potential glyphs to find the most fitting representation. Every frame, this entire process occurs dynamically.

Efficient Rendering Through Multiple Passes

The efficiency in this new ASCII rendering approach lies in executing three distinct rendering passes per frame, which considerably reduces the computational load of the glyph search.

    renderer.setRenderTarget(this.#sceneTarget);
    renderer.render(this.#scene, this.#camera);

    this.#quad.material = this.#cellMaterial;
    renderer.setRenderTarget(this.#cellTarget);
    renderer.render(this.#frame, this.#frameCamera);

    this.#quad.material = this.#postMaterial;
    renderer.setRenderTarget(null);
    renderer.render(this.#frame, this.#frameCamera);
    

The initial scene pass captures the geometry and lighting of the shape drawn to an offscreen buffer, while the cell pass resolves individual glyph selections per character unit. The final rendering layer compiles across those selections and applies the appropriate glyph from a defined atlas, thereby producing the finished ASCII output.

One of the notable efficiencies comes from sampling a defined cell size of 6 by 10 pixels in CSS, which results in a significant reduction of searches — only requiring a comparison once per every 60 pixels rather than across the entire pixel canvas. This structuring keeps processing costs manageable even as display resolutions increase.

Construction of the Mark: Geometry Matters

The construction process of the mark diverges from traditional extrusion methods, opting instead for a lens-like structure designed to enhance ASCII quality by implementing curvature that exposes distinct topology.

Utilizing two curved faces, the geometry captures varying tones based on lighting, ultimately impacting how well the ASCII representation retains its shape in printed form.

This entire method emphasizes the role of strategic geometry in ensuring that edges are clearly defined, thereby preventing ambiguity that typically arises in standard rasterization.

Creating a Dynamic Glyph Atlas

The glyph atlas is another pivotal component in this rendering method, generated directly in the browser during runtime. It hosts 95 of the most common glyphs, all meticulously spaced to avoid overlap — achieving optimal clarity with no false edges.

    ctx.fillStyle = "#ffffff";
    ctx.textAlign = "center";
    ctx.textBaseline = "middle";
    ctx.font = `${weight} ${Math.floor(Math.min(cellH * 0.92, cellW / 0.58))}px ${font}`;

    for (let glyph = 0; glyph < GLYPHS.length; glyph++) {
        ctx.fillText(GLYPHS[glyph], (glyph % cols) * padW + padW / 2, Math.floor(glyph / cols) * padH + padH / 2);
    }
    

The atlas ensures each glyph’s nuances are faithfully represented by capturing ink positions in various localized samples. This approach is particularly essential to preserve the character's integrity during the sampling process.

Multiple Sample Points Enhance Definition

The innovative aspect of this method includes giving each glyph a six-value vector that distinctly represents its alpha channel, measured at six precise locations within the sample cell. This is a game-changing shift from traditional one-point average methods.

    const INNER_SAMPLES = [
    [0.28, 0.26],
    [0.72, 0.14],
    [0.28, 0.56],
    [0.72, 0.44],
    [0.28, 0.86],
    [0.72, 0.74],
    ];
    

The asymmetrical distribution of values allows for discerning contrasts within glyphs even when their coverage appears similar. If we compare two identical coverage arrangements, our approach ensures they yield different vectors — avoiding the compression common in conventional methods which often reduce detailed representations to a single glyph.

Seeking Distinction Without Compromise

Normalizing sampled values individually across the six vectors preserves the richness of character diversity. If we were to aggregate against a single peak, it could risk eclipsing subtler distinctions, drowning out unique glyphs under a homogenized output.

    for (let sample = 0; sample < INNER_SAMPLES.length; sample++) {
        let peak = 0;

        for (let glyph = 0; glyph < count; glyph++) {
            peak = Math.max(peak, vectors[glyph * INNER_SAMPLES.length + sample]);
        }

        if (peak > 0) {
            for (let glyph = 0; glyph < count; glyph++) {
                vectors[glyph * INNER_SAMPLES.length + sample] /= peak;
            }
        }
    }
    

This method ensures that vowels filled by lighter glyphs maintain their representation without compromising their individuality, while also allowing for complex constructs like shadows or volumetric shapes to find their place in ASCII art.

The Complexity of Cell Pass Operations

The cell pass is where a significant chunk of processing time is dedicated, as the various sample positions become critical. Each cell's assessments involve a composite of readings that exceed simple one-to-one mappings.

    vec4 sampleCircle(vec2 c) {
        vec2 middle = cellBase + vec2(c.x, 1.0 - c.y) * uCellPx;
        float r = uCellPx.y * 0.161;
        vec4 acc = fetchTap(middle);

        for (int k = 0; k < 6; k++) {
            acc += fetchTap(middle + RING[k] * r);
        }

        return acc / 7.0;
    }
    

By sampling a circle instead of a single point, this approach greatly enhances the fidelity of rendered edges. This is especially vital for defining boundaries that would otherwise appear blurred in traditional techniques.

The renderer extracts luminance by unpremultiplying alpha values, providing a clearer context for the character's visual qualities against its background:

    float circleLum(vec4 acc) {
        vec3 straight = acc.rgb / max(acc.a, 1e-4);
        return clamp(dot(straight, vec3(0.2126, 0.7152, 0.0722)), 0.0, 1.0) * acc.a;
    }
    

This encapsulates essential shading details while also allowing the renderer to recognize composite nuances in the final ASCII output.

Nurturing Edge Intelligence

The introduction of neighboring taps in this cell pass fortifies the renderer's ability to discern edges distinctly. Each character cell positions itself according to its relative brightness to neighboring cells, intensifying the definition of those edges:

    float dirContrast(float value, float ext) {
        float peak = max(value, ext);
        if (peak < 1e-4) {
            return value;
        }
        return pow(value / peak, EDGE_CONTRAST) * peak;
    }
    
    

Through these adjustments, the renderer ensures that more three-dimensional edges materialize clearly, avoiding the confusion of flat representations.

Ultimately, this leads to an efficient linear search operation for the best glyph match, executing with optimized precision:

    int best = 0;
    float bestD = 1e9;
    for (int g = 0; g < uGlyphCount; g++) {
        float d = 0.0;
        for (int i = 0; i < 6; i++) {
            float diff = v[i] - texelFetch(tShapes, ivec2(i, g), 0).r;
            d += diff * diff;
        }
        if (d < bestD) {
            bestD = d;
            best = g;
        }
    }
    outColor = vec4(colAcc / max(alphaAcc, 1e-4), float(best) / 255.0);
    

Each cell undergoes significant computational intensity with nearly 570 operations ensuring optimal character selections for pixel representation. This method allows for efficient management of resources, making it feasible for higher-resolution outputs.

Compositing Techniques Without Edge Artifacts

One challenge that emerges in this approach lies in the compositing phase—specifically, the handling of texture atlas UVs. If not managed correctly, discontinuities can lead to visible seams between cells:

The trickiest aspect occurs naturally at cell boundaries, where transitions from one vertex to the next can yield unintended minification due to abrupt changes in UV space. Such disparities can create blend issues that waver across resolutions, often passing unnoticed unless meticulously scrutinized.