subprocess.run command with non-utf-8 characters (UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbb)

Encoding honestly continues to confuse me, so hopefully this isn't a totally daft question.

I have a python script that calls metaflac to compare the flac fingerprints in a file to the flac fingerprints of a file. Recently I came across files with » (https://bytetool.web.app/en/ascii/code/0xbb/) in the file name. This failed with how I was dealing with the file name strings, so I'm trying to work around that. My first thought was that I needed to deal with this as bytes objects. But when I do that and then call subprocess.run, I get a UnicodeDecodeError

Here's the snippet of code that is give me errors:

def test():
    directory = b'<redacted>'
    ffp_open = open(directory + b'<redacted>.ffp','rb')
    ffp_lines = ffp_open.readlines()
    print(ffp_lines)
    for line in ffp_lines:
        if not line.startswith(b';') and b':' in line:
            txt = line.split(b':')

            ffp_cmd = b'/usr/bin/metaflac --show-md5sum \'' + directory + b'/' + txt[0]+ b'\''
            print(ffp_cmd)
            get_ffp_process = subprocess.run(ffp_cmd, stdout=PIPE, stderr=PIPE, universal_newlines=True,shell=True)

With that, I get the following output (shortened to make more sense):

[b'01 - Intro.flac:eee7ca01db887168ce8312e7a3bdf8d6\r\n', b'04 - Song title \xbb Other Song \xbb.flac:98d2d03f47790d234052c6c9a2ca5cfd\r\n']
b"/usr/bin/metaflac --show-md5sum '<redacted>/01 - Intro.flac'"
b"/usr/bin/metaflac --show-md5sum '<redacted>/04 - Song title \xbb Other Song \xbb.flac'"

    get_ffp_process = subprocess.run(ffp_cmd, stdout=PIPE, stderr=PIPE, universal_newlines=True,shell=True)
  File "<redacted>/python/lib/python3.9/subprocess.py", line 507, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "<redacted>/python/lib/python3.9/subprocess.py", line 1134, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "<redacted>/python/lib/python3.9/subprocess.py", line 2021, in _communicate
    stderr = self._translate_newlines(stderr,
  File "<redacted>/python/lib/python3.9/subprocess.py", line 1011, in _translate_newlines
    data = data.decode(encoding, errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbb in position 85: invalid start byte

If I run this directly on the command line it works just fine (using tabs to fill in the file name):

metaflac --show-md5sum 04\ -\ Song\ title\ »\ Other Song\ ».flac 
98d2d03f47790d234052c6c9a2ca5cfd

The FFP file, through nano, looks like this:

01 - Intro.flac:eee7ca01db887168ce8312e7a3bdf8d6
04 - Song title � Other Song �.flac:98d2d03f47790d234052c6c9a2ca5cfd

I have no control over the file names, so I'm trying to be as flexible as possible to handle them, which is why I thought a bytes object would be best. I'd appreciate any direction. Thanks!

I believe coding of "latin1" or "cp1252" will do decode that successfully. Also, it is easier to deal with strings than with bytes, so here is my suggestion:

import pathlib
import subprocess

directory = pathlib.Path("/tmp")

with open(directory / "data.ffp", "r", encoding="latin1") as stream:
    for line in stream:
        if line.startswith(";"):
            continue
        if ":" not in line:
            continue

        file_name, expected_md5sum = line.strip().split(":")
        print(f"{name=}")
        print(f"{expected_md5sum=}")
        command = [
            "/usr/bin/metaflac",
            "--show-md5sum",
            str(directory / file_name)
        ]
        print(f"{command=}")

        # Now you can run the command. I assume that the command will return a MD5 sum back.
        completed_process = subprocess.run(
            command,
            encoding="latin1",
            capture_output=True,
        )

        # Now, completed_process.stdout will hold the output
        # as a string, not bytes.

Here is a sample output:

name='04 - Song title » Other Song ».flac'
expected_md5sum='eee7ca01db887168ce8312e7a3bdf8d6\n'
command=['/usr/bin/metaflac', '--show-md5sum', '/tmp/01 - Intro.flac']
name='04 - Song title » Other Song ».flac'
expected_md5sum='98d2d03f47790d234052c6c9a2ca5cfd\n'
command=['/usr/bin/metaflac', '--show-md5sum', '/tmp/04 - Song title » Other Song ».flac']

Since my system does not have the metaflac command, I cannot test it. Please forgive any error that come up. If an error found, please post in the comment and I will try to fix it.