Entity Framework Core define relationship without reciprocal property

I have an entity that has a one-to-many relationship with another entity utilizing a Principle Key. However, I wonder if I can create this relationship without explicitly declaring the "many" side?

public class CarModel{
    public Guid Id { get; set; }
    public long AltId { get; set; }
    public OwnerModel Owner { get; set; }
    public ColorModel Color { get; set; }
}
public class ColorModel{
    public Guid Id { get; set; }
    public long AltId { get; set; }
}
public class OwnerModel{
    public Guid Id { get; set; }
    public long AltId { get; set; }
    public List<CarModel> Cars { get; set; }
}

Caveat, also, the Primary Key is the long AltId property, for reasons far too complex to explain here. The FK that connects these however is the Guid Id property.

I had no issue configuring the Cars => Owners relationship:

builder.Entity<CarModel>()
    .HasKey(x => x.AltId);
builder.Entity<CarModel>()
    .HasOne(x => x.Owner)
    .WithMany(x => x.Cars)
    .HasPrincipalKey(x => x.Id)
    .HasForeignKey(x => x.Id);
builder.Entity<OwnerModel>()
    .HasKey(x => x.AltId);

However, I'd like to avoid putting a Cars property in the ColorModel. Maybe I'm overthinking it, but I don't want navigation from color -> car, I just want to know what color a particular car is. (Actual use case is slightly more complex than a car color, this is all just for simplicity)

This doesn't work, because the definitions require reciprocity. You can't do a HasPrincipalKey without defining properties on both ends, unless I'm missing something?

builder.Entity<CarModel>()
    .HasOne(x => x.Color)
    .HasPrincipalKey(x => x.Id)
    .HasForeignKey(x => x.Id);

Another Caveat, it has to be done here in the fluent API, I can't do it with data annotations. The entity models are used in two different contexts - connected to two different databases - so defining them in the model causes conflicts.

After writing out this entire question, it occurred to me to try the WithMany without defining a property within it, something I don't recall seeing in any example from the EF Core documentation.

builder.Entity<CarModel>()
    .HasOne(x => x.Color)
    .WithMany()
    .HasPrincipalKey(x => x.Id)
    .HasForeignKey(x => x.Id);