Mongodb migration causes duplicate key

In an old project we are trying to migrate from basic spring to spring boot 2.3.1. To do so and because we have a mongo database, we have to migrate from spring-data-mongodb:1.10.18 where this code was written :

DBCollection contextCollection = this.mongoTemplate.getCollection("productStock");
BulkWriteOperation builder = contextCollection.initializeUnorderedBulkOperation();
StockType stockItem = stockMessage.getStockItem();
final BasicDBObject id = new BasicDBObject("storeID", stockItem.getStoreID()).append("productID", stockItem.getProductID());
BulkWriteRequestBuilder bulkWriteRequestBuilder = builder.find(new BasicDBObject("_id", id));
HashMap<String, Object> stock = new HashMap<>();
Date currentDate = Calendar.getInstance().getTime();
stock.put("value", stockItem.getValue());
if (stockItem.getAssociateDate() != null) {
stock.put("associateDate", stockItem.getAssociateDate());
}
if (stockItem.getLastAccessDateSource() != null) {
stock.put("lastAccessDateSource", stockItem.getLastAccessDateSource());
// check
BasicDBObject ltLast = new BasicDBObject("$lt", stockItem.getLastAccessDateSource());
BasicDBList dbList = new BasicDBList();
dbList.add(new BasicDBObject(stockItem.getStockCategory() + ".lastAccessDateSource", ltLast));
dbList.add(new BasicDBObject(stockItem.getStockCategory() + ".lastAccessDateSource", null));
bulkWriteRequestBuilder = builder.find(new BasicDBObject("_id", id).append("$or", dbList));
} else {
stock.put("lastAccessDateSource", currentDate);
}
stock.put("lastUpdateDate", currentDate);
BasicDBObject set = new BasicDBObject(stockItem.getStockCategory(), new Document(stock));
bulkWriteRequestBuilder.upsert().updateOne(new BasicDBObject("$set", set));
builder.execute();

to spring-data-mongodb:3.0.1.RELEASE with this updated code

Map<String, List<StockType>> mapMultiUpdate = new HashMap<>();
StockType stockItem = stockMessage.getStockItem();
final Document id = new Document("storeID", stockItem.getStoreID()).append("productID", stockItem.getProductID());
HashMap<String, Object> stock = new HashMap<>();
Date currentDate = Calendar.getInstance().getTime();
Document searchQuery = new Document("_id", id).append("$or", dbList);
stock.put("value", stockItem.getValue());
if (stockItem.getAssociateDate() != null) {
stock.put("associateDate", stockItem.getAssociateDate());
}
if (stockItem.getLastAccessDateSource() != null) {
stock.put("lastAccessDateSource", stockItem.getLastAccessDateSource());
// check
Document ltLast = new Document("$lt", stockItem.getLastAccessDateSource());
List<Document> dbList = Lists.newArrayList();
dbList.add(new Document(stockItem.getStockCategory() + ".lastAccessDateSource", ltLast));
dbList.add(new Document(stockItem.getStockCategory() + ".lastAccessDateSource", null));
} else {
stock.put("lastAccessDateSource", currentDate);
}
//Bulk write options
BulkWriteOptions bulkWriteOptions = new BulkWriteOptions();
bulkWriteOptions.ordered(false);
bulkWriteOptions.bypassDocumentValidation(true);
MongoCollection<Document> mongoCollection = this.mongoTemplate.getCollection("productStoreStock");
mongoCollection.bulkWrite(updateDocuments, bulkWriteOptions);

But when the new code is executed on an already existing object we get a duplicated key error

com.mongodb.MongoBulkWriteException: Bulk write operation error on server localhost:27017. Write errors: [BulkWriteError{index=0, code=11000, message='E11000 duplicate key error collection: test.productStoreStock index: _id_ dup key: { : { storeID: 400, productID: 100000 } }', details={}}]. 

We have also switched from mongo-java-driver:3.6.4 to mongodb-driver-sync:4.0.4

EDIT :

This error is thrown during the 3rd step of a test phase, on an empty database / collection. the steps :

• initiating the collection with one stock on a product at a specific date
• checking the value of the base
• modifying the value of the stock in java but not the date and trying to update it
• checking the value is still the first one because of the LT filter on the mongo query

We never reach the checking value and before the migration, everything was good on this test

test.productStoreStock index: _id_ dup key: { : { storeID: 400, productID: 100000 } }', details={}}

Looks like you are replacing the internal id field with your own 'constructed' id. This is not a good thing. Let mongo create its own _id, these are guranteed unique across usage.. will never duplicate.
Nothing wrong with having your own companyId as an additional field, but its dangerous to replace mongodbs auto generated _id field... as you have found out.

Mongo is not updating it is upserting ( insert a new row ) when no match is found.