DeleteRegKey works only on second installation

I'm new to NSIS.
I created an installer with NSIS, it works fine with no issue.

Then I tried to install it at a different location before uninstall the former one. Then when I try to uninstall it from control panel, the entry doens't got deleted after uninstaller finish the task. I invisted further and it seems to me that for some reason DeleteRegKey stops working.

Then I notice, when I install twice like what I did, and go to control panel to uninstall it, that is the only case DeleteRegKey works and will remove the entry from control panel.

Below is my code for writing register key and deleting register key.

Section "-hidden app"
    SectionIn RO
    SetOutPath "$INSTDIR"
    File /r "client\*.*" 
    WriteRegStr HKCU "Software${NAME}" "" "$INSTDIR"
    SetRebootFlag True
    WriteUninstaller "$INSTDIR\Uninstall.exe"
    SetRegView 64
    WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall${NAME}" "DisplayName" "${NAME} stupid thing"
    WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall${NAME}" "DisplayIcon" '"$INSTDIR\app.ico"'
    WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall${NAME}" "UninstallString" '"$INSTDIR\Uninstall.exe"'
  SectionEnd

;--------------------------------
; Remove empty parent directories

  Function un.RMDirUP
    !define RMDirUP '!insertmacro RMDirUPCall'

    !macro RMDirUPCall _PATH
          push '${_PATH}'
          Call un.RMDirUP
    !macroend

    ; $0 - current folder
    ClearErrors

    Exch $0
    ;DetailPrint "ASDF - $0\.."
    RMDir "$0\.."

    IfErrors Skip
    ${RMDirUP} "$0\.."
    Skip:

    Pop $0

  FunctionEnd



Section "Uninstall"

  ;Delete Shortcut
  Delete "$DESKTOP${NAME}.lnk"

  ;Delete Uninstall
  Delete "$INSTDIR\Uninstall.exe"

  ;Delete Folder
  RMDir /r "$INSTDIR"
  ${RMDirUP} "$INSTDIR"

  DeleteRegKey HKCU "Software${NAME}"
  SetRegView 64
  DeleteRegKey HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall${NAME}"

SectionEnd

I tried to add a Iferrors block after DeleteRegKey section.
I expect it may catch some error, but no errors were catched.

After more trials, I notice the issue was not from DeleteRegKey. It's from function RMDirUP.

One simple workaround is to adjust the uninstall section from:

;Delete Folder
RMDir /r "$INSTDIR"
${RMDirUP} "$INSTDIR"

SetRegView 64

DeleteRegKey HKCU "Software${NAME}"
DeleteRegKey HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall${NAME}"

to:

SetRegView 64

DeleteRegKey HKCU "Software${NAME}"
DeleteRegKey HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall${NAME}"

;Delete Folder
RMDir /r "$INSTDIR"
${RMDirUP} "$INSTDIR"