It’s been a while since I had to make copies of large numbers of files in complex directory structures from the Windows command prompt but, faced with the need to take a backup within a command line environment (the WinRE Command Prompt), I needed to refresh my Windows command line skills.  There’s loads of advice out there on the Internet (most of it subjective) but the general consensus seems to be that the Extended Copy command (xcopy.exe) is deprecated and has been replaced in recent versions of Windows by the Robust File Copy command (robocopy.exe). Of course, there are many alternatives but they are not natively provided in WinRE!

(Some of the more useful articles I found are Nicholas Tyler’s reply on Stack Overflow, Oliver Muchai’s reply on Super User and Scott Hanselman’s blog post from 2007.)

Robocopy has loads of options but the ones I selected in the end were:

robocopy sourcefolder targetfolder /MIR /ZB /XJ /R:3 /W:1 /log:filename.txt

to make a mirror copy of the data, in restartable mode (to survive network glitches), using backup mode in the case of an access denied error, to exclude Junction Points, to retry 3 times on failure, waiting 1 second each time (compared with the defaults of 1 million and 30 seconds respectively) and to log to the chosen file.

The /XJ switch was added as a late addition after some abortive attempts ended up with recursive Application Data folders. Some people have erroneously referred to this as a bug in Robocopy – actually it’s caused by Windows’ attempts to prevent application developers writing to system locations (and forcing them to write to the user profile instead, as described by “DaddyMan” on a Microsoft Forum post:

“The Application Data folder is actually a junction, which points back to its parent folder.
[%username]\AppData\Local\Application Data\
points to
[%username]\AppData\Local\”

and by Shawn Keene (@LtCmdrKeene) in another, similar, post:

“[Any time] an application tries to save a file to a naughty location (such as C:\Windows or C:\Program Files), Windows will force the actual save to end up at a place inside your user folder instead (C:\Users\Username\LocalSettings\VirtualStore\Program Files).  It tricks the program into thinking that the file really did go to the Program Files folder, but in reality it’s somewhere inside your user folder.

This [virtualisation] (tricking the program) is required so that badly-created apps that save to naughty locations will still work.  The alternative is that the program tries to save and then crashes when it can’t access the Program Files folder.  If Windows didn’t do this, the program would require administrator access every time it runs — which is very insecure, plus would make the program impossible to use in corporate environments where users aren’t allowed to be administrators.

Rest assured that the multiple layers you are seeing are a result of folder redirection and [virtualisation] (also known as junction points).  There’s no need to clean these up or correct it, and you are well advised to avoid exploring those files.”

Finally, I needed to remove the folders that I had accidentally created with recursive Application Data folders inside (I counted 25 in one case!). Neither Windows nor the Windows Command Prompt (del and rmdir commands) could do this, resulting in “too long” errors but Super User Aaron has the answer (which is a variation on the method Bob Coss commented on one of my own old blog posts):

“Create an empty directory with mkdir empty, then use robocopy empty\ "Application Data\" /mir" which will remove the whole directory tree. Then issue a rmdir empty and rmdir "Application Data to clean up and you’re done.”