efficient way to iterate over large data sets in go & mongodb

More of a general question than anything, but here is my problem. We have 2 databases, both are mongo database. We are in the process of migrating data from one mongo instance to another, difference between v1 and v2 of our app meant a refactoring of said data. Problem here in lies. one particular collection holds roughly 840 thousand records. Getting those 840 records is not the problem. I need to convert the data structure to the new database schema. Part of that refactoring was embedding the newly generated objectId's of 3 other collections in this particular part of the migration.

Example would be
patients collection,
caregivers collection,
agencies collection,
these are imported without ids and are generated by mongo

they are all referenced in the 840k records in visits collection by there newly generated objectID's

my problem lies when I range over all 4 collections to line up the data this process is taking I don't even know how long I left it running for 4 hours before I killed it.

	for _, a := range agencies {
		for _, c := range caregivers {
			for _, p := range patients {
				for _, m := range model {
					if a.Name == *m.AgencyName && m.CaregiverID == &c.CareGiverID && m.PatientID == &p.PatientID {
						visit := &models.Visit{
							CreatedAt:  time.Now(),
							UpdatedAt:  time.Now(),
							AgencyID:   a.ID.Hex(),
							ScheduleID: *m.ScheduleID,
							Status:     *m.Status,
							Start:      *m.Start,
							End:        *m.End,
							PatientID:  p.ID.Hex(),
							UserID:     c.ID.Hex(),
						}
						updateModel = append(updateModel, mongo.NewUpdateOneModel().SetFilter(
							bson.D{
								{Key: "agencyId", Value: a.ID.Hex()},
								{Key: "userId", Value: c.ID.Hex()},
								{Key: "patientId", Value: p.ID.Hex()},
								{Key: "scheduleId", Value: m.ScheduleID},
								{Key: "start", Value: m.Start},
								{Key: "end", Value: m.End},
							},
						).SetUpdate(
							bson.D{{Key: "$set", Value: visit}},
						).SetUpsert(true))
					}
				}
			}
		}
	}

Now that code works for the smaller collections when I was doing them but this one is taking longer than we have time to wait around for. My question is, what is the least time consuming way of lining up 840k records to 32 agencies, 8000 patients, 6000 caregivers. I did some math and it's ranging more times than my calculator could spit out a number for. FWI I don't care about resource consumption as we can spin up a cloud machine of any size for the time it takes to run the app. Just an efficient way of getting those object id's from the 3 other collections for each of the 840k visits. I did have a though of importing those collections to the old database and lookup the id's during the aggregate for the input, but that is not very programatic.

Modified range with cached objectIds

	for _, m := range model {
		agency, agencyFound := c.Get(*m.AgencyName)
		uid, uidFound := c.Get(fmt.Sprintf("caregiver-%v-%d", agency, *m.CaregiverID))
		patientId, patientFound := c.Get(fmt.Sprintf("patient-%v-%d", agency, *m.PatientID))
		if agencyFound && uidFound && patientFound {
			visit := &models.Visit{
				CreatedAt:  time.Now(),
				UpdatedAt:  time.Now(),
				AgencyID:   fmt.Sprintf("%v", agency),
				ScheduleID: *m.ScheduleID,
				Status:     *m.Status,
				Start:      *m.Start,
				End:        *m.End,
				PatientID:  fmt.Sprintf("%v", patientId),
				UserID:     fmt.Sprintf("%v", uid),
			}
			updateModel = append(updateModel, mongo.NewUpdateOneModel().SetFilter(
				bson.D{
					{Key: "agencyId", Value: fmt.Sprintf("%v", agency)},
					{Key: "userId", Value: fmt.Sprintf("%v", uid)},
					{Key: "patientId", Value: fmt.Sprintf("%v", patientId)},
					{Key: "scheduleId", Value: m.ScheduleID},
					{Key: "start", Value: m.Start},
					{Key: "end", Value: m.End},
				},
			).SetUpdate(
				bson.D{{Key: "$set", Value: visit}},
			).SetUpsert(true))
		}
	}

Cartesian product of agencies, caregivers, patients and visits is 32x8000x6000x840000 is about 10^15.

Let's suppose the inner if takes one CPU cycle to be computed (it takes much more) with a 4 gigahertz CPU it will take more than 89 hours to complete.

Generating all possible combinations of (agencies, caregivers, patients) for each visit is a massive waste of resources.

Reverse approach: iterate over all visits and fetch the agency, caregiver and patient for that particular document. To save time accessing db it is feasible to store a mapping in memory with the IDs needed.