Cypress Testing Drag & Drop Three.js Canvas

I'm currently trying to get some drag and drop cypress tests working for a react three fibre project. I'm stuggling a little to test this however as I don't have selectable ID's for the 3D elements to be used in cypress as all of the three.js elements are wrapped up in the HTML canvas.

My tests start as follows:

cy.viewport(1920, 1080);
cy.get('[data-testid="r3f"] canvas').as('canvas');
cy.get('@canvas').click({ x: 1390, y: 320 }); <--- Pos of object to drag & drop

With the above snippet I'm able to select object in the scene and enable the transform controls as follows:

I then want to select the yellow drag square and move the objects position for my test.

I've tried both using cypress trigger & cypress-real-events & a combination of the two with no sucsess

  cy.get('@canvas')
    .realMouseMove(1390, 340)
    .realMouseDown(1390, 340) <--- Mousedown not able to take take X/Y as params
    .realMouseMove(1100, 300)
    .realMouseUp({ force: true });

Using the above cypress-real-events seems to have got me closest to what I need, in the test replay realMouseMove is placing the mouse over the drag box, as follows:

However .realMouseDown(1390, 340) doesn't have the option of passing X&Y instead it defaults to the topLeft the selected element, so in my case the canvas @ 0,0. This then results in the orbit controls of the scene being triggered on the second realMouseMove rotating the scene and no drag and drop being perfomed. realMouseEnd working as expected

  cy.get('@canvas')
    .trigger('mousemove', { clientX: 1394, clientY: 320, force: true })
    .trigger('mousedown', { clientX: 1394, clientY: 320, force: true, button: 0 })
    .trigger('mousemove', { clientX: 1100, clientY: 300, force: true })
    .trigger('mouseup');

Using the above cypress trigger method seems to be working less. mousemove doesn't show the active hover state over the transform controls as with realEvents.

Any help with this would be much appreciated. Feel like I'm getting nowhere with this

The cypress-real-events documentation needs some updating.

If you take a look here mouseDown.ts you can see that the coordinates are passed in as an object like this: { x:123, y:456 } (relative to the <canvas> not the screen).

Type definition for mouseDown options

export interface realMouseDownOptions {
  ...
  cy.get("canvas").realMouseDown({ x: 100, y: 115 })

Implementation

const position =
    options.x && options.y ? { x: options.x, y: options.y } : options.position;

const { x, y } = getCypressElementCoordinates(
  subject,
  position,
  options.scrollBehavior,
);
...
await fireCdpCommand("Input.dispatchMouseEvent", {
  type: "mousePressed",
  x,
  y,