I think you could create the new project in any accessible folder you want.
To delete the existing folder (usually located in “C:\Users<USER>\source\repos” ), you might need to close Visual Studio and other windows or applications that could prevent deletion.
But if it is a Git Repository, then the approach is probably different.