Seeking explanation Long -> Byte Array -> String -> Byte Array -> Long

I'm seeking an explanation for some oddity I've seen in someone elses code, they were retrieving an "int64" value from a third party library reading from an LDAP attribute, this library returned a byte array. To get the value they were trying something like

String s = new String(bytesFrom3rdParty);
BigInteger i = new BigInteger(s.getBytes());
System.out.println(i.toString());

With some long values this gave incorrect output that wasn't expected. To me there were two things that stood out

1. Why go from byte array -> String -> Bytes -> BigInteger
2. Why use a BigInteger for a 64 bit numeric value.

Anyway I did a little experiment

private static byte[] longToByteArray(Long l) {
	return ByteBuffer.allocate(Long.SIZE / Byte.SIZE).putLong(l).array();
}

private static Long byteArrayToLong(byte[] bytes) {
	return ByteBuffer.wrap(bytes).getLong();
}

public static void main(String[] args) {
	
	for (long l = 0L; l < 1000; l++) {
		byte[] origBytes = longToByteArray(l);
		String s = new String(origBytes);
		byte[] stringBytes = s.getBytes();
		Long origL = byteArrayToLong(origBytes);
		Long stringL = byteArrayToLong(stringBytes);
		System.out.println(origL.toString() + " " + stringL.toString());
	}
	
}

As I suspected skipping the conversion to string then back to a byte array fixed the issue, the output from the above is something like

124 124
125 125
126 126
127 127
128 239
129 239
130 239
131 239
132 239

And then the right hand value corrects itself again when it hits 256

254 239
255 239
256 256
257 257
258 258
259 259
260 260
261 261
262 262
263 263
264 264

So a couple of questions from me

1. Why is the right hand value wrong? I assume it's something to do with conversion between a 64 bit long value to a 32 bit String value?
2. Why doesn't the incorrect value change until the value of l gets to 256?

byte[] can be different things, for example:

• serialized String value (UTF-8 encoding for example) "123" -> bytes representing string, which actually encodes every character with 2 bytes
• serialized Long value in binary 123 -> 8 bytes representing one number

So when it makes sense to convert byte[] to String is when you are actually getting String in byte[], and after that you are parsing the String into number (in your case BigInteger). Going back to bytes doesn't make much sense to me.

String s = new String(bytesFrom3rdParty); // binary from UTF-8 string
BigInteger i = new BigInteger(s); // parse String "123" to BigInteger
System.out.println(i.toString()); // now i will be 123 in BigInteger

This will work too:

String s = new String(bytesFrom3rdParty); // binary from UTF-8 string
Long i = Long.parseLong(s); // parse String "123" to Long
System.out.println(i.toString()); // now i will be 123 in Long

What you are doing in your example is second case, you are serializing Long in binary form to byte[] (not UTF-8 string). Then you are making a string of that binary data and getting bytes. What happens is due to conversion to Java's backing implementation of Charset - it expects it to be valid Charset encoding it changes your binary representation to something that fits Charset encoding.

When you try to retrive it back and build Long from it breaks, why 128. Probably up to 127 (old ASCII standard had this many characters) your binary representation somehow fits Java's charset encoding, but after it breaks.

• serialized String value should be parsed Long.parseFrom(String) or new BigInteger(String)
• binary Serialized number should be binary read ByteBuffer.getLong()

Lets make it a little simpler, byte[] -> String -> byte[] is performing an encoding and a decode. When you use new String(byte[] b) it will:

>Constructs a new String by decoding the specified array of bytes using the platform's default charset.

What happens if the character is not in your platform default character set?

>The behavior of this constructor when the given bytes are not valid in the default charset is unspecified.

So, in your situation, when an invalid byte is passed it converts the character to 65533 the java replacement character.

byte[] b = {-1};
System.out.println( Arrays.toString( new String(b).getBytes() ) );

> [-17, -65, -67]

That is why the value doesn't change, they're all mapped to the replacement character.

You might use BigInteger simple access to a constructor that takes byte[] to create a long.