How can I upload files to the replicate API with Node JS

I'm trying to use the sadTalker AI model through replicate.

The point of sadtalker is that you upload an image of a person and an audio file of someone talking, and it gives you back a deepfake video of someone talking.

I've setup a basic environment to do this. All I have is a .env file storing my API key, two files called deepfakeAudio.mp3 and deepfakeImage.jpg.

My code is as follows:

import Replicate from "replicate";

import dotenv from "dotenv";
import fs from "fs";

dotenv.config();

fs.readFile("./deepfakeImage.jpg", (error, inputImage) => {
  fs.readFile("./deepfakeAudio.mp3", (error, inputAudio) => {
    getVideo();
  });
});

async function getVideo(inputImage, inputAudio) {
  const replicate = new Replicate({
    auth: process.env.REPLICATE_API_TOKEN,
  });
  const output = await replicate.run(
    "cjwbw/sadtalker:3aa3dac9353cc4d6bd62a8f95957bd844003b401ca4e4a9b33baa574c549d376",
    {
      input: {
        source_image: inputImage,
        driven_audio: inputAudio
      }
    }
  );
  console.log(output);
}

When I try to run this, I get the error:

throw new Error(`API request failed: ${response.statusText}`);
            ^

Error: API request failed: Unprocessable Entity

The problem is that I don't know how I'm supposed to load the image and audio into the API.

To load the image and audio, I've tried 2 things:

The first time, I tried to just input the path of the audio and image. This gave me an error.

The second time, as shown in the code, I tried to load the audio and image using fs, I still get an error.

Has anyone experienced this issue before?

This is very confusing because I have no idea of what I'm actually supposed to do.

The replicate docs mention that replicate expects file input as base64 encoded data URI.

So your modified code would look something like this :

import Replicate from "replicate";

import dotenv from "dotenv";
import { promises as fs } from "fs";

dotenv.config();

const data = await fs.readFile("./deepfakeImage.jpg", "utf-8");
const base64 = data.toString("base64");
const mimeType = "image/jpg";
const dataURI = `data:${mimeType};base64,${base64}`;


async function getVideo(inputImage, inputAudio) {
  const replicate = new Replicate({
    auth: process.env.REPLICATE_API_TOKEN,
  });
  const output = await replicate.run(
    "cjwbw/sadtalker:3aa3dac9353cc4d6bd62a8f95957bd844003b401ca4e4a9b33baa574c549d376",
    {
      input: {
        source_image: dataURI,
        driven_audio: inputAudio
      }
    }
  );
  console.log(output);
}