Is it possible to verify a SHA256withRSA signature with a SHA256 hash of the original data?

For a long time I've been working with signatures using X509 certificates.

PrivateKey privateKey = ...
String document = ...

Signature signature = Signature.getInstance("SHA256withRSA");
signature.initSign(privateKey);
signature.update(document.getBytes());
return signature.sign();

and verifying...

PublicKey publicKey = ...
String document = ...
byte[] signature = ...

Signature signature = Signature.getInstance("SHA256withRSA");
signature.initVerify(publicKey);
signature.update(document.getBytes());
return signature.verify(signature);

Pretty simple.

But recently I heard that it is possible to verify the signature with only the SHA256 hash of the document, instead of the whole document...

PublicKey publicKey = ...
byte[] documentHash = MessageDigest.getInstance("SHA-256").digest(document.getBytes());
byte[] signature = ...

Signature signature = Signature.getInstance("SHA256withRSA");
signature.initVerify(publicKey);
// ???
return signature.verify(signature);

Is it possible? How would it be in Java?

I heard it from another company, so I don't have access to the source code... =(

The short answer is YES.

The long answer has to do with an encoding that wraps the signature together with the algorithm Identifier and to archive the "verify on hash only" functionality you have to do 2 changes.

First - prepend the algorithm identifier to the signature:
As @President James K. Polk wrote you have to add some extra byte to get a correct encoding of the input to verification function. As you need
the "EMSA-PKCS1-v1_5"-Padding (described here: https://www.rfc-editor.org/rfc/rfc3447#page-41) you have to prepend some bytes that represent the
algorithm that was used to calculate the hash.

I'm a bit lazy and prepending the necessary bytes as hard coded byte array and so this version does work only on SHA-256 algorithm - if
you ever use a different hashing algorithm you need to change the prepended bytes:

String prependSha256String = "3031300D060960864801650304020105000420";
byte[] prependSha256 = hexStringToByteArray(prependSha256String);
int combinedLength = prependSha256.length + documentHash.length;
byte[] documentHashFull = new byte[combinedLength];
System.arraycopy(prependSha256, 0, documentHashFull, 0, prependSha256.length);
System.arraycopy(documentHash, 0, documentHashFull, prependSha256.length, documentHash.length);

Second - use another RSA signature scheme:
As we have done the SHA-256 part already we need a "naked" RSA-scheme called "NonewithRSA", so you need to change the instantiation like:

Signature signatureVerifyHash = Signature.getInstance("NonewithRSA");

These are the results of the two RSA signature verifications (old and "new" one):

verify the signature with the full document
sigVerified: true
verify the signature with the SHA256 of the document only
sigVerifiedHash: true

Here is the full working code:

import java.security.*;
public class MainSo2 {
public static void main(String[] args) throws NoSuchAlgorithmException, InvalidKeyException, SignatureException {
System.out.println("Is it possible to verify a SHA256withRSA signature with a SHA256 hash of the original data?");
// create a rsa keypair of 2048 bit keylength
KeyPairGenerator rsaGenerator = KeyPairGenerator.getInstance("RSA");
SecureRandom random = new SecureRandom();
rsaGenerator.initialize(2048, random);
KeyPair rsaKeyPair = rsaGenerator.generateKeyPair();
PublicKey publicKey = rsaKeyPair.getPublic();
PrivateKey privateKey = rsaKeyPair.getPrivate();
String document = "The quick brown fox jumps over the lazy dog";
// sign
Signature signature = Signature.getInstance("SHA256withRSA");
signature.initSign(privateKey);
signature.update(document.getBytes());
byte[] sig = signature.sign();
// verify with full message
System.out.println("\nverify the signature with the full document");
Signature signatureVerify = Signature.getInstance("SHA256withRSA");
signatureVerify.initVerify(publicKey);
signatureVerify.update(document.getBytes());
boolean sigVerified =  signatureVerify.verify(sig);
System.out.println("sigVerified: " + sigVerified);
// verify just the sha256 hash of the document
System.out.println("\nverify the signature with the SHA256 of the document only");
byte[] documentHash = MessageDigest.getInstance("SHA-256").digest(document.getBytes());
// you need to prepend some bytes: 30 31 30 0D 06 09 60 86 48 01 65 03 04 02 01 05 00 04 20
// see https://www.rfc-editor.org/rfc/rfc3447#page-41
// warning: this string is only for SHA-256 algorithm !!
String prependSha256String = "3031300D060960864801650304020105000420";
byte[] prependSha256 = hexStringToByteArray(prependSha256String);
int combinedLength = prependSha256.length + documentHash.length;
byte[] documentHashFull = new byte[combinedLength];
System.arraycopy(prependSha256, 0, documentHashFull, 0, prependSha256.length);
System.arraycopy(documentHash, 0, documentHashFull, prependSha256.length, documentHash.length);
// lets verify
Signature signatureVerifyHash = Signature.getInstance("NonewithRSA");
signatureVerifyHash.initVerify(publicKey);
// signatureVerifyHash.update(document.getBytes());
signatureVerifyHash.update(documentHashFull);
boolean sigVerifiedHash =  signatureVerifyHash.verify(sig);
System.out.println("sigVerifiedHash: " + sigVerifiedHash);
}
public static byte[] hexStringToByteArray(String s) {
int len = s.length();
byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
+ Character.digit(s.charAt(i + 1), 16));
}
return data;
}
}