Accessibility in diagrams (GoJS)

[Dropek]

Problem

The GoJS library offers some solutions to enhance accessibility, but they are limited and fairly straightforward. In Synergy, we have implemented additional solutions that can further improve accessibility.

One of the missing aspects is accessibility support for the palette component. The default GoJS palette does not have built-in keyboard support which means we neither can navigate through palette elements only with arrow keys nor place them on the diagram without using mouse. These are very important aspects of web accessibility since there are users that rely only on the keyboards since they cannot use mouse because of different disabilities.

Thanks to the solution below, we will be able to add nodes using only the keyboard. Additionally, we will leverage an external library to create the palette using HTML, which will enhance both the UI and UX of the palette items. The HTML palette has many advantages over the canvas one. Not only from accessibility aspects but it gives much more customizations options.

This example uses dnd-kit, a React library that helps handle drag-and-drop functionality and adds keyboard support by default. Therefore, the example is written in React. However, most frontend frameworks/libraries have their own drag-and-drop solutions that can be used in a similar way.

Solution

Introduction

Below, I will present only the most essential parts required for implementing the drag-and-drop functionality. Some parts of the solution might be omitted in this explanation, but the complete implementation will be linked at the end via StackBlitz, where you can see how it works.

We will use the following libraries:

• React
• GoJS
• @dnd-kit/core

Implementation

First, we need to wrap our Diagram and Palette components with the DndContext from @dnd-kit/core:

// App.tsx

import { DndContext } from '@dnd-kit/core';

...

<DndContext onDragStart={handleOnDragStart} onDragEnd={handleOnDragEnd}>
  <div className="wrapper">
    <Diagram />
    <Palette activeItemId={activeItemId} />
  </div>
</DndContext>

In the example above, we render the Diagram and Palette components within the DndContext. We pass activeItemId to the palette, representing the ID of the currently dragged item. This will allow us to show the dragged element moving with the cursor.

Below are the implementations of handleOnDragStart and handleOnDragEnd:

// App.tsx

import { DragEndEvent, DragStartEvent } from "@dnd-kit/core";

...

// Set activeItemId on drag start
const handleOnDragStart = (event: DragStartEvent) =>
  setActiveItemId(event.active.id as string);

const handleOnDragEnd = ({
  over,
  active,
  activatorEvent,
  delta,
}: DragEndEvent) => {
  setActiveItemId(null);
  if (over?.id !== "diagram" || !diagram?.div || !active.data) {
    return;
  }
  let startX = 0;
  let startY = 0;
  // The data below is passed by the palette item, providing the starting position when dragging begins.
  const data = active.data.current as {
    x: number;
    y: number;
    width: number;
    height: number;
  };
  // Determine startX and startY depending on whether dragging was initiated via mouse or keyboard.
  if (
    activatorEvent instanceof PointerEvent ||
    activatorEvent instanceof MouseEvent
  ) {
    startX = activatorEvent.clientX;
    startY = activatorEvent.clientY;
  } else {
    startX = data.x;
    startY = data.y;
  }
  const diagramRect = diagram.div.getBoundingClientRect();
  const x = startX + delta.x - diagramRect.x - (data.width * diagram.scale) / 2;
  const y =
    startY + delta.y - diagramRect.y - (data.height * diagram.scale) / 2;

  diagram.commit(() => {
    // Add node with calculated position
    diagram.model.addNodeData({
      location: go.Point.stringify(
        diagram.transformViewToDoc(new go.Point(x, y))
      ),
      text: `Node ${activeItemId}`,
    });
  });
};

Next, we need to make the Diagram a droppable area so that users can drop nodes onto it:

// Diagram.tsx

import { useDroppable } from '@dnd-kit/core';

...

const diagramRef = useRef<HTMLDivElement>(null);
const { setNodeRef: setDroppableRef } = useDroppable({
  id: "diagram",
});

...

setDroppableRef(diagramRef.current);

...

<div className="diagram" ref={diagramRef} />

The final step is to prepare the Palette and PaletteItem components to enable dragging from the palette:

// PaletteItem.tsx

import { useEffect, useRef } from "react";
import { useDraggable } from "@dnd-kit/core";

export function PaletteItem({ id }: { id: string }) {
  const ref = useRef<HTMLDivElement | null>(null);
  const rect = ref.current?.getBoundingClientRect();
  const { attributes, listeners, setNodeRef } = useDraggable({
    id,
    data: rect
      ? {
          x: rect.x + 40,
          y: rect.y + 20,
          width: rect.width,
          height: rect.height,
        }
      : undefined,
  });

  useEffect(() => {
    if (ref.current) {
      setNodeRef(ref.current);
    }
  }, [ref, setNodeRef]);

  return (
    <div ref={ref} {...attributes} {...listeners}>
      Node {id}
    </div>
  );
}

The code above is a complete example of the PaletteItem component, which renders a simple palette item. We call useDraggable to make the element draggable, passing its position and size as data. The x and y values represent the center point, since GoJS nodes have their locationSpot set to the center. This data is used inside the handleOnDragStart function in App.tsx.

// Palette.tsx
import { DragOverlay } from '@dnd-kit/core';

...

<DragOverlay dropAnimation={null}>
    {activeItemId ? <PaletteItem id={activeItemId} /> : null}
</DragOverlay>

Inside the Palette component, we render a PaletteItem for each item. Additionally, we implement the DragOverlay using the same PaletteItem component. This ensures that the dragged element looks exactly like the one inside the palette.

Summary

The above code sections are sufficient to enable keyboard support for the palette, which is provided by default by @dnd-kit/core. The key is to wrap everything properly so that it integrates well with GoJS.

The complete solution can be found here: StackBlitz

[Dropek]

Navigating a Diagram Using the Keyboard ️

In the previous post, we were able to add nodes to the diagram using the keyboard. Now, we will focus on navigating the diagram using the keyboard to move elements and switch between them efficiently.

Built-in Solution

GoJS provides a built-in solution for navigating between diagram elements, including a tree graph approach where you can move between parents, siblings, and children: AriaCommandHandler.

This is an extension of CommandHandler that captures keyboard events and modifies selection accordingly.

Custom Solution and Explanation

Introduction

For one of my applications, the built-in solution was not sufficient or perfectly suited, so I decided to write my own "AriaCommandHandler" with custom navigation logic, taking inspiration from GoJS's approach (another reason was educational—I wanted to understand how it works ). I will explain what happens and how, so you can also tailor navigation to your needs.

Implementation

First, we need to determine which keys will be used for navigating between nodes and which for moving elements.

Let's establish that selection changes will occur using the WSAD keys, and movement will require holding the alt key.

type Direction = 0 | 90 | 180 | 270;

export class AriaCommandHandler extends CommandHandler {
  // Define keys with direction which will be used later
  private keyDirectionMap: Record<string, Direction> = {
    D: 0,
    S: 90,
    A: 180,
    W: 270,
  };

  doKeyDown(): void {
    const e: go.InputEvent = this.diagram.lastInput;
    // If one of WSAD clicked, go to custom logic.
    if (e.key in this.keyDirectionMap) {
      // If alt key is held, move element
      if (e.alt) {
        this._moveSelection(e.key as keyof typeof this.keyDirectionMap);
      } else {
        // Else, change selection
        this._changeFocus(e.key as keyof typeof this.keyDirectionMap);
      }
    } else {
      // Otherwise, fall back to base implementation
      super.doKeyDown();
    }
  }
}

In the above implementation, the appropriate keys are already handled, so now we just need to write the logic.
Let's start with moving elements, as this part is simpler.

_moveSelection(key: keyof typeof this.keyDirectionMap) {
  const direction = this.keyDirectionMap[key];
  const originalPart = this.diagram.selection.first();
  if (!originalPart) {
    return;
  }
  const { x, y } = originalPart.location;
  const { width, height } = this.diagram.grid.gridCellSize;
  // Depending on direction, move element by grid cell size
  if (direction === 0) {
    originalPart.moveTo(x + width, y);
  } else if (direction === 90) {
    originalPart.moveTo(x, y + height);
  } else if (direction === 180) {
    originalPart.moveTo(x - width, y);
  } else if (direction === 270) {
    originalPart.moveTo(x, y - height);
  }
}

The above logic simply moves the node by the appropriate distance in the correct direction. Additionally, it uses a grid, so the movement aligns with the grid cell size.

Now, let's move on to changing the selection.

_changeFocus(key: keyof typeof this.keyDirectionMap) {
  const direction = this.keyDirectionMap[key];
  const closestNode = this._findClosestNode(direction);
  closestNode && this.diagram.select(closestNode);
}

The _changeFocus method retrieves the direction and searches for the nearest node. If it finds one, it selects it using this.diagram.select. Now, let's look at the _findClosestNode method, which is key to determining which node to select next.

_findClosestNode(direction: Direction) {
  const originalPart = this.diagram.selection.first();
  // If no nodes are selected, skip searching
  if (originalPart === null) {
    return null;
  }

  const allParts: go.Part[] = [];
  // Extract all visible nodes and convert them to an array
  this.diagram.nodes.each((x) => x.isVisible() && allParts.push(x));

  // The point from which we search for the nearest node is the center of the current node
  const originalPoint = originalPart.actualBounds.center;
  let closestDistance = Infinity;
  let closest = originalPart;

  // Iterate through all nodes
  for (const nextPart of allParts) {
    // Skip the currently selected node and those that cannot be selected
    if (nextPart === originalPart || !nextPart.canSelect()) {
      continue;
    }

    // Use the center point of other nodes to determine distances
    const nextPoint = nextPart.actualBounds.center;
    const distance = this._calculateDistance(originalPoint, nextPoint, direction);

    // If the distance is smaller than the last smallest, define a new target node
    if (distance < closestDistance) {
      closestDistance = distance;
      closest = nextPart;
    }
  }
  // If the closest is still the current node, return null
  return closest === originalPart ? null : closest;
}

This logic is fairly straightforward. However, the key part is the _calculateDistance method.
At first glance, calculating the distance between two points may seem simple, but in this case, it's more complex. If there are three nodes in a line (First, Second, and Third), where First is far to the left of Second (which is selected), and Third is far to the right, pressing A should select First, even though it may not be the closest node. This is because we are considering direction.

To handle this, we use Direction, which we compare with the angles between nodes.
We use directionPoint to get the angle between two points on a coordinate plane.

_calculateDistance(originalPoint: go.Point, nextPoint: go.Point, direction: Direction) {
  const angleDiff = this._angleCloseness(
    originalPoint.directionPoint(nextPoint),
    direction
  );
  ...
}

_angleCloseness(angle: number, direction: number) {
  // Return the actual difference between angles
  return Math.min(
    Math.abs(direction - angle),
    Math.min(
      Math.abs(direction + 360 - angle),
      Math.abs(direction - 360 - angle)
    )
  );
}

Using _angleCloseness, we get the difference in angles between two points.
Now, we need to calculate the actual distance and combine it with angle difference information.
We will use distanceSquaredPoint, which returns the squared distance between two points.
To determine the next node, we apply a penalty based on the angle difference—the greater the difference, the greater the penalty.
We also set a maximum angle of 60 degrees. If the angle difference exceeds this, the penalty is Infinity, and the node is skipped.

_calculateDistance(originalPoint: go.Point, nextPoint: go.Point, direction: Direction) {
  const angleDiff = this._angleCloseness(originalPoint.directionPoint(nextPoint), direction);
  const penalty = angleDiff > 60 ? Infinity : angleDiff / 8;

  return (
    originalPoint.distanceSquaredPoint(nextPoint) *
    (1 + Math.sin((angleDiff * Math.PI) / 180) * penalty)
  );
}

Thanks to this logic, we can accurately determine which node the user wants to navigate to using WSAD. Some values result from experimenting with different options.

Summary

This solution is just one of many ways to implement keyboard navigation for a diagram. The best approach depends on project needs and diagram structure. If nodes vary in size, choosing their center points may not be ideal. Additionally, selecting links may be a factor.

A useful addition would be Tab and Shift+Tab navigation to cycle through nodes, as WSAD navigation might sometimes be blocked by other nodes.