Way to check for duplicates before writing into a file?

So I wrote a small script that takes text files as an input, reads every line and tries to validate it as an email. If it passes, it writes the line into a new ('clean') file, if it doesn't pass, it strips it of spaces and tries to validate it again. Now, if it passes this time, it writes the line into a new file and if it fails, it ignores the line.

Thing is, such as it is, my script may write duplicate emails into the output files. How should I go around that and check for duplicates present in the output file before writing?

Here's the relevant code:

<!-- language: lang-golang -->

// create reading and writing buffers
	scanner := bufio.NewScanner(r)
	writer := bufio.NewWriter(w)

	for scanner.Scan() {
		email := scanner.Text()

		// validate each email
		if !correctEmail.MatchString(email) {
			// if validation didn't pass, strip and lowercase the email and store it
			email = strings.Replace(email, " ", "", -1)
			// validate the email again after cleaning
			if !correctEmail.MatchString(email) {
				// if validation didn't pass, ignore this email
				continue
			} else {
				// if validation passed, write clean email into file
				_, err = writer.WriteString(email + "\r\n")
				if err != nil {
					return err
				}
			}

		} else {
			// if validation passed, write the email into file
			_, err = writer.WriteString(email + "\r\n")
			if err != nil {
				return err
			}
		}

	}

	err = writer.Flush()
	if err != nil {
		return err
	}

Create a type that implements writer then create a custom WriteString

Inside WriteString open the file where you store your emails, iterate over each email and save the new emails.

You may use a Go built-in map as a set like this:

<!-- language: lang-golang -->

package main

import (
	"fmt"
)

var emailSet map[string]bool = make(map[string]bool)

func emailExists(email string) bool {
	_, ok := emailSet[email]
	return ok
}

func addEmail(email string) {
	emailSet[email] = true
}

func main() {
	emails := []string{
		"duplicated@golang.org",
		"abc@golang.org",
		"stackoverflow@golang.org",
		"duplicated@golang.org", // <- Duplicated!
	}
	for _, email := range emails {
		if !emailExists(email) {
			fmt.Println(email)
			addEmail(email)
		}
	}
}

Here is the output:

duplicated@golang.org
abc@golang.org
stackoverflow@golang.org

You may try the same code at <kbd>The Go Playground</kbd>.