SAS macro variable distinct

I want to please delete all the duplicates inserted in the KI table, knowing that I have the tables from 2007 to 2022

%do I=2007 %to 2022 ;
proc sql noprint;
select nliteral(name) into :varlist separated by ','
   from dictionary.columns
   where libname='SI' and memname="SI&I."
;
insert into LIB1.KI (&varlist) select &varlist
from SI.SIQ&I.;
quit;
%end;
%mend;
%KI;

Here is one way

options nodsnferr;
data lib1.ki;
  set lib1.si2007-lib1.si2022;
run;
options dsnferr;

proc sort data=lib1.ki nodupkey;
  by _all_;
run;

With out suppositions of sameness or enforced validations the problem is fraught with issues such as:

• variable type discrepancies
• variable length discrepancies
• value fuzz discrepancies
• variable names (as a set) discrepancies

The initial naive is

create table newnique as 
select * from data
where catx('~', &varlist) not in (select catx('~', &varlist) from base)

insert into base
select * from newnique;

If you only want to insert distinct observations from SI.SIQ&I in the current INSERT statement add the DISTINCT keyword to the SELECT expression.

insert into LIB1.KI (&varlist)
 select distinct &varlist
 from SI.SIQ&I.
;

If you need to not insert observations that already exist then you will need to work harder to remove the observations that exist in LIB1.KI from the observations in SI.SIQ&I. I suspect you will have to make a temporary dataset to do that since you would need to test for exclusion by reading from the same LIB1.KI dataset that you are inserting into.

You might do something like this to create such a dataset .

 create table to_add as 
    select &varlist from SI.SIQ&i. 
    except
    select &varlist from LIB1.KI
 ;

Then use that in the INSERT statement instead of SI.SIQ&i.