How to Join tables flat with fixed params with Spark SQL

I have datesets in spark with certain columns. I want to join those dataset flat by keeping some columns as reference(id, date, name).

DATASET 1: (d1)

DATASET 1: (d2)

Output

I will make the id, data, name columns fixed, If all three are same then I will do flat join, else create new row with the values being 0 incase of null

Please help me with appropriate sql or dataset logic in spark to get the desired output.

use full outer join based on Id,date,name to Result all data of two data sets if matching Records are there it will join based on Id,date,name. use coalesce to avoid null's

	declare @Datset1 table (
	Id int,
	date varchar(120),
	name varchar(10),
	val1 int
	);

	insert into @Datset1
	values(1,'15-06-2022','n1',10),
	(2,'15-06-2022','n1',12),
	(3,'15-06-2022','n1',24),
	(4,'16-09-2021','n2',27);


	declare @Datset2 table (
	Id int,
	date varchar(120),
	name varchar(10),
	val2 int
	);
	insert into @Datset2
	values(1	,'15-06-2022','n1',	15),
	(1	,'12-06-2021','n1',	123),
	(2	,'15-06-2022','n2',	56),
	(15	,'21-03-2022','n1',	43),
	(4	,'16-09-2021','n2',	65);

	select coalesce(dt1.Id,dt2.Id) as Id,
	coalesce(dt1.date,dt2.date) as date,
	coalesce(dt1.name,dt2.name) as name,
	coalesce(dt1.val1,0) as val1,
	coalesce(dt2.val2,0) as val2
	from @Datset1 dt1 full outer join @Datset2 dt2 on dt1.Id=dt2.Id and dt1.date=dt2.date and dt1.name=dt2.name;

Output will be as follows