why does JcaPEMKeyConverter have a setProvider method?

I get why java.security.Security has an addProvider method - because Java has multiple providers that can provide stuff like javax.crypto.Cipher.getInstance() can use (sun.security.provider.Sun, org.bouncycastle.jce.provider.BouncyCastleProvider, etc).

But why does org.bouncycastle.openssl.jcajce.JcaPEMKeyConverter have an addProvider method? Given the namespacing being used I find it hard to imagine that any "provider" other than BouncyCastle would provide a drop in replacement...

I don't know what 'namespacing' you think is being used. JcaPEMKeyConverter uses JCA to implement the crypto operations it needs, and it can use any JCA provider that provides the needed operations; nearly the whole point of JCA is that providers use the same API (or technically SPI, Service Provider Interface) so that you can selectively use different providers for the same operation(s).

Some, perhaps most, of the operations in bcpkix bcpg and bcmail libraries can use either JCA API (using any suitable JCA provider(s)) or Bouncycastle's own private API (using only Bouncy code), e.g. org.bouncycastle.pkcs.PKCS12MacCalculator[Builder] are interfaces with interchangeable implementations org.bouncycastle.pkcs.bc.BcPKCS12MacCalculator[Builder] and org.bouncycastle.pkcs.jcajace.JcePKCS12MacCalcuator[Builder]. (Bouncy isn't as careful as one might wish about distinguishing JCA from JCE in names consistently.) However, JcaPEMKeyConverter comes only in the JCA form.

It is true someone who has Bouncy add-on libraries will often have and be able to use the Bouncy provider as well, but not always. For example, US federal government systems are required to use certain cryptographic functions (mostly primitives) that are validated under FIPS140 (currently at revision -2, soon -3), and while Bouncy does have a FIPS140 implementation, using it commercially requires payment, while if you are using e.g. IBM Java on certain IBM systems it has providers (different from the common Sun/Oracle/OpenJDK ones) that are FIPS140 validated at no additional charge.

It's correct that the API is not designed on the assumption that the user has to use BouncyCastle for the provision of cryptographic services - and a ready example of this is people using a FIPS provider, such as BCFIPS.

One small correction though to the preceding answer, there is no license fee or similar for the BCFIPS Java provider and as one of the members of the project I'm more than a little disturbed to see someone stating that. I would be curious to know where that was heard from.

We do have a support program for FIPS and non-FIPS users and it does provide early access to the in-progress FIPS releases as part of it and that does cost (that's how we fund everything). However, the actual released FIPS jars, once published on bouncycastle.org are licensed under the BC license, which is basically the MIT X11 license, like all our other published work. I hope this clears up the confusion.