Extract pattern from several hundreds files

I have several hundreds of *.changelog files named generated by rpm -q --changelog named package01.changelog, package02.changelog, package03.changelog, and so on...

I want to extract the CVE (Common Vulnerabilities and Exposures) numbers from each of them and create two separate files:

1. One file that lists all CVEs for each *.changelog file, for example:

CVE-2014-3513 | CVE-2014-3567 | CVE-2014-3568

2. A second file that concatenates all CVE's from all *.changelog files, for example:

CVE-2014-3513:package01 | CVE-2014-3567:package02 | CVE-2014-3568:package03

The CVEs listed in the the *.changelog files are not placed in a specific order or have a specific structure, for example:

  references (add CVE-2022-28356 bsc#1197391).
  (CVE-2022-1016 bsc#1197227).
  in error path (CVE-2022-28389 bsc#1198033).
- xprtrdma: fix incorrect header size calculations (CVE-2022-0812
  bsc#1196761 CVE-2022-0850).


(add CVE-2022-28356 bsc#1197391)
(CVE-2022-1016 bsc#1197227).
(CVE-2022-0812
CVE-2022-0850).
  (bsc#1196836 CVE-2022-26966).

I am using a loop to go through each *.changelog file, extract the CVEs and write them to the two output files mentioned above, for example:

for chlg in /path/to/*.changelog
do
  pkg_name="$(echo $(basename $chlg) | sed 's/.changelog//')"
  cve_file="$(echo $(basename $chlg) | sed 's/.changelog/.cve.html/')"
 
  grep -o 'CVE-[[:digit:]]*-[[:digit:]]*' $chlg | sort -f | uniq | awk -v pkg_name="$pkg_name" -v cve_file="/path/to/${cve_file}" -v all_cve_file="/path/to/cve.html" '
        {
          print "<a href=\"https://cve.mitre.org/cgi-bin/cvename.cgi?name="$1"\" target=\"_blank\" title=\""$1"\">"$1"</a> | " > cve_file
          print "<a href=\"https://cve.mitre.org/cgi-bin/cvename.cgi?name="$1"\" target=\"_blank\" title=\""$1"\">"$1"</a>:<span style=\"color:red; font-family:consolas;\">"pkg_name"</span> | " >> all_cve_file
        }'
done

I get what the results I want, even though it's taking more than an hour to finish considering that I'm processing ~ 49000 files. Another issue (that's a discussion for a different topic) is that the resulting cve.html file is too big to be loaded by a browser which eventually crashes.

My question is if the approach I'm using to extract the CVE numbers from these ~49000 files is good enough or if there is another more "elegant" and efficient way.

You might implement getting unique values in GNU AWK itself and then sort -f them which might shorten time if number of CVEs total is much greater that number of unique CVEs, this can be done following way, let file.txt content be

  references (add CVE-2022-28356 bsc#1197391).
  (CVE-2022-1016 bsc#1197227).
  in error path (CVE-2022-28389 bsc#1198033).
- xprtrdma: fix incorrect header size calculations (CVE-2022-0812
  bsc#1196761 CVE-2022-0850).


(add CVE-2022-28356 bsc#1197391)
(CVE-2022-1016 bsc#1197227).
(CVE-2022-0812
CVE-2022-0850).
  (bsc#1196836 CVE-2022-26966).

then

awk 'BEGIN{FPAT="CVE-[[:digit:]]*-[[:digit:]]*"}$1&&!arr[$1]++{print $1}' file.txt | sort -f

gives output

CVE-2022-0812
CVE-2022-0850
CVE-2022-1016
CVE-2022-26966
CVE-2022-28356
CVE-2022-28389

Explanation: I inform GNU AWK that field is CVE- followed by zero-or-more (*) digits followed by - followed by zero-or-more digits. Then I print 1st field if there is 1st field and it was not seen earlier which fact is stored in array arr.

(tested in GNU Awk 5.0.1)