loading data to Elasticsearch v7.3 using Bulk API

I need to load the data to the elasticsearch index. I am using BULK API of elasticsearch to load the JSONs to index.

private String FOLDER_PATH = "src/main/resources/allJsons";
    private String index = "test1";
    private static final String TYPE = "test_type";

 @Autowired
    private RestHighLevelClient restHighLevelClient;

 public String loadBulkData() throws IOException {

        BulkRequest bulkRequest = new BulkRequest();
        AtomicInteger counter = new AtomicInteger();
        try (Stream<Path> filePathStream = Files.walk(Paths.get(FOLDER_PATH))) {
            filePathStream.forEach(filePath -> {
                if (Files.isRegularFile(filePath)) {
                    counter.getAndIncrement();
                    try {
                        String content = Files.readString(filePath);
                        JSONObject jsonObject1 = new JSONObject(content);
                        HashMap yourHashMap1 = new Gson().fromJson(jsonObject1.toString(), HashMap.class);
                        IndexRequest indexRequest = new IndexRequest(index, TYPE).source(yourHashMap1);
                        bulkRequest.add(indexRequest);

                    } catch (IOException e) {
                        e.printStackTrace();
                    }


                }
            });
        }
        try {
            restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return "Bulk data loaded to index " + index + "";
    }
}

I have multiple JSONs based on the following format

[
 {
  "Nutrient" : "Calories",
  "Amount" : " 289.00",
  "Unit" : " kcal"
}, {
  "Nutrient" : "Fat",
  "Amount" : " 17.35",
  "Unit" : " g"
}
]

While running the code it gives me error ,
>org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.json.JSONException: A JSONObject text must begin with '{' at 1 [character 2 line 1]

I think the data is in JSONArray and for the code, we need JSONObject. Anyone could please guide to how to do this

You can do bulk insertion by passing hashmap of your json objects to Elasticsearch Bulk API.
You can create Hashmap by parsing your JSON file through JSONParser.

Here is the code for the same :

Code :

          Integer id= 1;

          //You need to call this method for inserting bulk documents which 
          // internally calls `createBulkRequest` and `parseObjectList` methods.
          //This method uses JSONParser to parse your file and convert into JSONArray.
           public String insertBulkDocuments() throws Exception {
        		Object obj = new JSONParser().parse(new FileReader(<path-of-file>)); 
                JSONArray objList= (JSONArray) obj;       
                BulkRequest request = createBulkRequest(objList);
                BulkResponse bulkresp=restHighLevelClient.bulk(request, RequestOptions.DEFAULT);
                return bulkresp.status().toString();
            }
        	
        	// Each JSONArray element that was obtained through first method 
           //is parsed individually through Gson and converted into you defined Object. 
           //This object is then converted to Map and passed to IndexRequest object.
        	private BulkRequest createBulkRequest(JSONArray objList) {
        		BulkRequest request = new BulkRequest();
        		objList.forEach( obj -> parseObjectList((JSONObject) obj, request,id++));
        		return request;
        	}
        	
        	private void parseObjectList(JSONObject obj, BulkRequest request, int id) {
        		Gson gson = new GsonBuilder().create();
        		NutrientDocument doc = gson.fromJson(obj.toJSONString(), NutrientDocument .class);
                
                Map<String, Object> documentMapper = objectMapper.convertValue(doc, Map.class);
        
                IndexRequest indexRequest = new IndexRequest(<your-index-name>).id(Integer.toString(id)).source(documentMapper);
                request.add(indexRequest);
        	}

You need to create Custom object which has same feilds as your json . I have created NutrientDocument for testing which has same fields as your JSON and this I am using in parseObjectList method.

public class NutrientDocument {
	private String Nutrient;
	private Float Amount;
	private String Unit;
	public String getNutrient() {
		return Nutrient;
	}
	public void setNutrient(String nutrient) {
		Nutrient = nutrient;
	}
	public Float getAmount() {
		return Amount;
	}
	public void setAmount(Float amount) {
		Amount = amount;
	}
	public String getUnit() {
		return Unit;
	}
	public void setUnit(String unit) {
		Unit = unit;
	}
	
	
	
}

NOTE :

For each document elasticserach generates unique id .

For creating our own id value instead of Elasticsearch autogenerated value, we are using id variable. But, if you want to go with Elasticsearch autogenerated number , you can create IndexRequest object as below in parseObjectList method and remove id variable wherever we are passing.

IndexRequest indexRequest = new IndexRequest(<your-index-name>).source(documentMapper);