"Host key verification failed" error when running GitHub Actions on self-hosted runner (Windows 10)

I'm trying to run a simple GitHub Action on my self-hosted runner (Windows 10), but I'm getting the error Host key verification failed. [error]fatal: Could not read from remote repository. Here's the code for the GitHub Action:

name: GitHub Actions Demo
on:
  push:
    branches: ["feature"]
jobs:
  build:
    runs-on: self-hosted
    steps:
      - name: Check out repository code
        uses: actions/checkout@v3

I've verified that the self-hosted runner is properly configured and connected to the repository, and I can manually clone and fetch the repository on the same machine without any issues. I've also tried running the ssh-keyscan command and adding the resulting host key to the known_hosts file, but that doesn't solve the problem.

Instead of running directly the checkout action, try first running a

    steps:
      - name: Test SSH access
        run: ssh -Tv git@github.com

The is to see which key is communicated, and it the account used is the same as the one you are with, when you do your manual test (when the clone/fetch is working).

The OP ysief-001 then sees (in the comments)

> After 1h30m I cancelled the workflow.
The last two lines are
>
> Found key in C:\Users\ysief/.ssh/known_hosts:4
> read_passphrase: can't open /dev/tty: No such file or directory

That simply means a passphrase-protect (IE: encrypted) private key is not supported.
You need one without passphrase. (Or you can remove the passphrase from your existing key)