How can I give different names to files in a directory with a for loop in a bash script?

I'm expecting to get 17 different paired-end fastq files (34 in total), so I want to make a bash script to just run my code through all the fastq files in a directory at once. How can I change the name of the input and output files each time the script runs through each file? So when it moves to the file_002, all names have file_002 at the beginning instead of file_001, and so on. And also, when merging the R1 and R2 reads how can I make that it only merges the correspondant files with a loop? for examples merging only file_001_R1 with file_001_R2, file_002_R1 with file_002_R2, file_003_R1 with file_003_R2, and so on.

for file in directory_name
do
pear -f file_001_R1.fastq.gz -r file_001_R2.fastq.gz -o file_001.fastq
cutadapt -g TGATAACAATTGGAGCAGCCTC...GGATCGACCAAGAACCAGCA -o file_001_barcode.fastq file_001.fastq
cutadapt -g GTGTACAAATAATTGTCAAC...CTGTCTCTTATACACATCTC -o file_001_UMI.fastq file_001.fastq
seqkit concat file_001_barcode.fastq file_001_UMI.fastq > file_001_concatenation.fastq
seqkit rmdup -s file_001_concatenation.fastq -o file_001_unique_pairs.fastq
seqkit subseq -r file_001_unique_pairs.fastq > file_001_unique_barcodes.fasta
bowtie -q --suppress 1,2,4,6,7,8 -x ref_index file_001_unique_barcodes.fasta > file_001_barcodes_allignment.bowtie
sort file_001_barcodes_allignment.bowtie | uniq -c > file_001_barcode_counts.txt
awk 'BEGIN{print "Barcode,TF_variant,Code"}{print $3","$2","$1}' file_001_barcode_counts.txt > file_001_barcode_counts.csv
done

You can use bash parameter expansion to capture the "file_001" part of the filename, e.g.

cd directory_name
for file in ./*_R1.fastq.gz
do
    pear -f "$file" -r "${file%_*}_R2.fastq.gz" -o "${file%_*}.fastq"
    cutadapt -g TGATAACAATTGGAGCAGCCTC...GGATCGACCAAGAACCAGCA -o "${file%_*}_barcode.fastq" "${file%_*}.fastq"
    cutadapt -g GTGTACAAATAATTGTCAAC...CTGTCTCTTATACACATCTC -o "${file%_*}_UMI.fastq" "${file%_*}.fastq"
    seqkit concat "${file%_*}_barcode.fastq" "${file%_*}_UMI.fastq" > "${file%_*}_concatenation.fastq"
    seqkit rmdup -s "${file%_*}_concatenation.fastq" -o "${file%_*}_unique_pairs.fastq"
    seqkit subseq -r "${file%_*}_unique_pairs.fastq" > "${file%_*}_unique_barcodes.fasta"
    bowtie -q --suppress 1,2,4,6,7,8 -x ref_index "${file%_*}_unique_barcodes.fasta" > "${file%_*}_barcodes_allignment.bowtie"
    sort "${file%_*}_barcodes_allignment.bowtie" | uniq -c > "${file%_*}_barcode_counts.txt"
    awk 'BEGIN{print "Barcode,TF_variant,Code"} {print $3 "," $2 "," $1}' "${file%_*}_barcode_counts.txt" > "${file%_*}_barcode_counts.csv"
done

Not sure whether your pipeline is appropriate/optimal though; you might want to ask for advice from the experts over at https://bioinformatics.stackexchange.com

I am not sure entirely if I understand your question but you can use a bash script to do something like this: loop through files and extract the file names without the extensions and merge only the corresponding R1 R2 files.

# Set the directory where your files are located
directory="/path/to/directory"

# Loop through the files in the directory
for file in "$directory"/*_R1.fastq; do
    # Extract the file name without the extension and suffix
    filename=$(basename "$file" | sed 's/_R1.fastq//')

# Set the R1 and R2 file names
r1_file="${filename}_R1.fastq"
r2_file="${filename}_R2.fastq"

# Set the output file name
output_file="${filename}_merged.fastq"

# Perform the merge operation using the corresponding R1 and R2 files
# Replace this line with whatever code you want to run

done