Saving sucessfully copied Files from A to B in Java when Programm crashes?

i have to copy lots of files from A to B in Java, problem is that the Programm might crash.
Already sucessfully copied Files cannot be copied again, when the programm restarts. How can I save in a textFile already copied Files?

My solution below is not 100% sucessfull, sometimes it doesnt save all the successfully copied files.
I wonder if theres a better approach to this issue.Thanks a lot in advance.

 package com.mycompany;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.PrintWriter;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
public class CopyingData {
public static void saveFilesAlreadyCopied(ArrayList<String> filesToSave, String fileName) {
PrintWriter pw = null;
try {
pw = new PrintWriter(new FileOutputStream(fileName));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
for (String file : filesToSave) {
pw.println(file);
}
pw.close();
}
public static void main(String[] args) {
//Variables:
File sourceDir = new File("sourceDir");
File destinationDir = new File("destDir");
ArrayList<String> listOfFiles = new ArrayList<>();
ArrayList<String> filesAlreadyCopied = new ArrayList<>();
//Create dirs:
sourceDir.mkdirs();
destinationDir.mkdirs();
//Generate 1000 Files for Testing:
for(int i = 0;i<1000;i++) {
File textFile = new File("sourceDir/textfile"+i+".txt");
try {
textFile.createNewFile();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
listOfFiles.add(textFile.getAbsolutePath());
}
//Copy files from sourceDir to destinationDir:
for(String file : listOfFiles) {
Path sourcePath = Paths.get(file);
Path destPath = Paths.get(destinationDir.getAbsolutePath() + File.separator + new File(file).getName());
try {
Files.copy(sourcePath, destPath, StandardCopyOption.REPLACE_EXISTING);
filesAlreadyCopied.add(file);
saveFilesAlreadyCopied(filesAlreadyCopied,sourceDir.getAbsolutePath() + File.separator +"backupCopiedFiles.txt");
} catch (IOException e) {
e.printStackTrace();
}
}				
}
}

One possible problem you might have is running out of disk space, and I can see how after copying millions of files, terabytes of data, & then crashing on the last file, you would want to just free up some disk space & continue with the last file.

But consider this: the algorithm as written is basically as follows (assuming a checkpoint file "chkpt" contains list of files successfully copied):

1. for each file f,
2. copy file f,
3. update checkpoint file chkpt

So, if step #2 fails to write to the file system, then you can resume based on the contents of chkpt. But what happens if step #3 fails to write to the file system? (Do you create a checkpoint file for the checkpoint file?) The odds of this happening increase with the size of chkpt, which you currently have as containing the entire list of files to copy (could be a problem for a large list of files).

I might recommend the following (as food for thought) to decrease the odds of being unable to recover:

1. if you want to persist a list of files to copy, then create that first on disk, just once; e.g., filelist.txt, containing file1.txt, file2.txt, etc
2. before starting the copy, create a file called e.g. copying containing the "next file to copy", e.g., if you've already copied files file1.txt through file31.txt, it would contain file32.txt
3. start to copy the file (eg file32.txt)
4. if the copy fails, and the program crashes, the existence of a file called copying indicates where things stopped -- restart copy of file32.txt
5. if the copy succeeded, do a "move" (file rename) of copying to completed (so, file completed would contain filename file32.txt ...also, obviously, call these files whatever you want). A file move/rename won't usually fail, nor should it become corrupted as often as writing a large file.
6. continue with the next file: create a new file copying with the next filename in the list (don't modify the file list)

Does that make sense? (Thoughts?) The level of effort above what you currently have isn't much more, but it might reduce the chance of running into an "unrecoverable" state (i.e., corrupted checkpoint file)..

I have made some changes to your code, to handle fail scenarios, please try below code and make happy your boss:

import java.io.*;
import java.nio.file.*;
import java.util.ArrayList;
import java.util.stream.Collectors;
public class CopyingData {
public static void saveFilesAlreadyCopied(ArrayList<String> filesToSave, String fileName) {
PrintWriter pw = null;
try {
pw = new PrintWriter(new FileOutputStream(fileName));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
for (String file : filesToSave) {
pw.println(file);
}
pw.close();
}
public static void main(String[] args) throws IOException {
String operationFolder = "C:/Downloads/code_play-folder/";
//Variables:
File sourceDir = new File(operationFolder+"sourceDir");
File destinationDir = new File(operationFolder+"destDir");
String progressFilePath = operationFolder+"sourceDir" +File.separator + "backupCopiedFiles.txt";
File progressReportFile = new File(progressFilePath);
ArrayList<String> listOfFiles = new ArrayList<>();
ArrayList<String> filesAlreadyCopied = new ArrayList<>();
if (progressReportFile.exists()){
filesAlreadyCopied.addAll(Files.readAllLines(Paths.get(progressFilePath)).stream().filter(data-> !data.trim().equals("")).collect(Collectors.toList()));
}
//Create dirs:
sourceDir.mkdirs();
destinationDir.mkdirs();
//Generate 1000 Files for Testing:
for(int i = 0;i<1000;i++) {
String fileName = operationFolder+"sourceDir/textfile" + i + ".txt";
File textFile = new File(fileName);
try {
if (!textFile.exists()) {
textFile.createNewFile();
Files.write(
Paths.get(fileName),
("contentToAppend"+fileName).getBytes(),
StandardOpenOption.APPEND);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
listOfFiles.add(textFile.getAbsolutePath());
}
listOfFiles.removeAll(filesAlreadyCopied);
//Copy files from sourceDir to destinationDir:
for(String file : listOfFiles) {
Path sourcePath = Paths.get(file);
Path destPath = Paths.get(destinationDir.getAbsolutePath() + File.separator + new File(file).getName());
try {
Files.copy(sourcePath, destPath, StandardCopyOption.REPLACE_EXISTING);
filesAlreadyCopied.add(file);
saveFilesAlreadyCopied(filesAlreadyCopied,sourceDir.getAbsolutePath() + File.separator +"backupCopiedFiles.txt");
} catch (IOException e) {
e.printStackTrace();
}
}
}
}