Command Line Moving of Data. Copying and Such.

So in the beginning there was…. Copy and Paste. This was mostly a Command line function. Then the perverted GUI got in the way. And all you had to do was right click and copy and paste. Or the Keyboard shortcuts like:

(Windows)

Ctrl + C = Copy

Ctrl + V = Paste

Ctrl + X = Cut

(I would add the Linux and Unix commands but those who read my blog are already yawning.)

Ok so this is a post based on easier movement of files from one place to the other. Those who know me know I love Xcopy. I was Very upset that Xcopy was second best in our now unloved and much laughed at Windows Vista. It used RoboCopy.

However I must digress. The more I use Robocopy the more I have fell in love with some of the great features. Like Resume and MultiThread.

So lets just list some of the fun to be had when you want to really copy files (a large amount that is.) Lets start with my old favorite XCOPY.

XCOPY:

Syntax of Xcopy command

Xcopy has a large number of possible switches, which gives the command a great deal of flexibility. The syntax for the command is given by:

XCOPY source [destination] [/A | /M] [/D[:date]] [/P] [/S [/E]] [/V] [/W] [/C] [/I] [/Q] [/F] [/L] [/G] [/H] [/R] [/T] [/U] [/K] [/N] [/O] [/X] [/Y] [/-Y] [/Z] [/EXCLUDE:file1[+file2][+file3]...]

Upper case letters have been used above but the command is case-insensitive. A description of the various switches is given in Table I. Note that Windows Vista has an additional switch "/B". The function of this switch is to copy a Symbolic Link itself instead of the target of the link.

Table I. Description of switches for the command Xcopy

Switch
Description

/A
Copies only files with the archive attribute set, doesn’t change the attribute.

/M
Copies only files with the archive attribute set, turns off the archive attribute. Useful in backup.

/D:m-d-y
Copies files changed on or after the specified date. If no date is given, copies only those files whose source time is newer than the destination time. Useful in backup.

/P
Prompts you before creating each destination file.

/S
Copies directories and subdirectories except empty ones.

/E
Copies directories and subdirectories, including empty ones. Same as /S /E. May be used to modify /T.

/V
Verifies each new file. Not used by Windows XP.

/W
Prompts you to press a key before copying.

/C
Continues copying even if errors occur.

/I
If destination does not exist and copying more than one file, assumes that destination must be a directory.

/Q
Does not display file names while copying.

/F
Displays full source and destination file names while copying.

/L
Displays files that would be copied.

/G
Allows the copying of encrypted files to destination that does not support encryption.

/H
Copies hidden and system files also.

/R
Overwrites read-only files.

/T
Creates directory structure, but does not copy files. Does not include empty directories or subdirectories.

/U
Copies only files that already exist in destination.

/K
Copies attributes. Normal Xcopy will reset read-only attributes.

/N
Copies using the generated "short" names. May be necessary when copying from NTFS to FAT16.

/O
Copies file ownership and ACL information.

/X
Copies file audit settings (implies /O).

/Y
Suppresses prompting to confirm that you want to overwrite an existing destination file. May be preset in the COPYCMD environment variable

/-Y
Prompts to confirm you want to overwrite an existing destination file.

/Z
Copies over a network in restartable mode.

/EXCLUDE:file1[+file2][+file3]…
Specifies a list of files containing strings to be excluded. Tricky to use. See below.

Applications of Xcopy

Xcopy provides an excellent tool for backing up selected folders. With appropriate switches, a variety of backup scenarios can be created. One possible backup configuration would be to copy only those files that have been changed. Here is an example command:xcopy C:\somefolder E:\backupfolder /D /E /C /R /H /I /K /YThis command will copy all files, including those in sub-folders, that are newer in the source folder. It will copy hidden as well as read-only files and will create the destination folder and/or sub-folders if they do not already exist.

The next example shows the use of a wildcard; it collects all files of a given type and copies them into one place. The commandxcopy C:\*.mp3 E:\mp3folder /S /I /C > E:\mp3List.txtwill collect all MP3 files on the C: drive and copy them to a folder on the E: drive. It also creates a list of the files copied and places the list on the E: drive. This simple command preserves the sub-folder structure. If desired, a more advanced script could be written that places all MP3 files in one folder with no sub-folders.

Excluding files and folders

A useful feature of Xcopy is the ability to exclude certain files and/or folders from being copied by means of the "/Exclude" switch. All names containing a given string of characters can be excluded. Unfortunately, Microsoft’s description of the switch is neither a model of clarity nor accurate. In fact, postings on the Internet indicate considerable confusion exists about how to implement this handy feature. Perhaps the list below will help clarify how to use it.

The strings contained in the names of the files and/or folders to be excluded are not entered directly in the command switch itself.
The entry in the command switch is one or more text files that list the strings to be excluded.
The format of the switch is/Exclude:{path}list_of_exclusions.txtNote the colon between Exclude and the name of the file that lists the exclusions. Although more than one exclusion file can be used, stick with one if you can.
The name of the file that lists exclusions cannot have spaces. Quoting does not help.
Be careful with pathnames. Remember that operations are relative to the working directory for the command prompt. I suggest placing the exclusion list in the top directory of the directories being copied and opening the command prompt in the directory just above the directory and sub-directories to be copied. (Go here if you don’t know how to open the command prompt wherever you want.) The various relative paths will then be simple. If you open the command prompt in the default location, things can get complicated.
The structure of the exclusions list is one exclusion string per line. Wildcards are not used and do not work. If you want to exclude all files whose names contain ABC, simply place ABC on one line of the exclusion file. If it is being used together with the switch "/S", the exclusion file will apply to sub-directories as well as the main directory.
To exclude a particular sub-directory, place its name in the exclusion file with backward slashes before and after its name, viz.,\excluded_directory\
Use a text editor to create exclusion files. Do not use Microsoft Word or other word processor unless you are careful to create text files. Use ANSI encoding. When I tried UTF-8 encoding, the file did not work.

Exit codes for Xcopy

Many commands issue integer exit codes to indicate the status of the command after it is run. The exit codes for Xcopy are given in Table II. When placed in "If" statements, these error codes can be useful in batch files .

Exit codes for Xcopy

Exit code | Description

0. Files were copied without error.

1. No files were found to copy.

2. The user pressed Ctrl+C to terminate xcopy.

4. Various errors including insufficient memory or disk space, an invalid drive name, or invalid syntax.

5. Disk write error occurred.

RoboCopy:

Robocopy (Robust File Copy) is a command-line file copying tool in Windows Vista. Although Robocopy is available for free with the download Windows Resource Kit since Windows NT 4.0, it has never been an official feature of the operating system until the arrival of Vista. Unlike other built-in file copying commands such as Copy and XCopy, Robocopy is designed for reliable copy or mirroring of entire folders of any size, and in the copying process, ensure that all NTFS ACLS, attributes, owner information, alternate data streams, auditing information, timestamps and properties are copied except security information unless explicitly requested with /COPYALL switch. And best of all, Robocopy works over network connections that are subject to disruption or outages with resume copying feature, and has progress indicator on the command line that is useful when copying large files.
Robocopy Syntax

ROBOCOPY source destination [file [file]…] [options]

where source is Source Directory (drive:\path or \\server\share\path), destination is Destination Directory (drive:\path or \\server\share\path) and file is File(s) to copy where names or wildcards can be specified and default is “*.*” (all files).

Robocopy Options and Switches

Copy options :

/S :: copy Subdirectories, but not empty ones.
/E :: copy subdirectories, including Empty ones.
/LEV:n :: only copy the top n LEVels of the source directory tree.

/Z :: copy files in restartable mode.
/B :: copy files in Backup mode.
/ZB :: use restartable mode; if access denied use Backup mode.
/EFSRAW :: copy all encrypted files in EFS RAW mode.

/COPY:copyflag[s] :: what to COPY for files (default is /COPY:DAT).
(copyflags : D=Data, A=Attributes, T=Timestamps).
(S=Security=NTFS ACLs, O=Owner info, U=aUditing info).

/DCOPY:T :: COPY Directory Timestamps.

/SEC :: copy files with SECurity (equivalent to /COPY:DATS).
/COPYALL :: COPY ALL file info (equivalent to /COPY:DATSOU).
/NOCOPY :: COPY NO file info (useful with /PURGE).

/SECFIX :: FIX file SECurity on all files, even skipped files.
/TIMFIX :: FIX file TIMes on all files, even skipped files.

/PURGE :: delete dest files/dirs that no longer exist in source.
/MIR :: MIRror a directory tree (equivalent to /E plus /PURGE).

/MOV :: MOVe files (delete from source after copying).
/MOVE :: MOVE files AND dirs (delete from source after copying).

/A+:[RASHCNET] :: add the given Attributes to copied files.
/A-:[RASHCNET] :: remove the given Attributes from copied files.

/CREATE :: CREATE directory tree and zero-length files only.
/FAT :: create destination files using 8.3 FAT file names only.
/256 :: turn off very long path (> 256 characters) support.

/MON:n :: MONitor source; run again when more than n changes seen.
/MOT:m :: MOnitor source; run again in m minutes Time, if changed.

/RH:hhmm-hhmm :: Run Hours – times when new copies may be started.
/PF :: check run hours on a Per File (not per pass) basis.

/IPG:n :: Inter-Packet Gap (ms), to free bandwidth on slow lines.

File Selection Options :

/A :: copy only files with the Archive attribute set.
/M :: copy only files with the Archive attribute and reset it.
/IA:[RASHCNETO] :: Include only files with any of the given Attributes set.
/XA:[RASHCNETO] :: eXclude files with any of the given Attributes set.

/XF file [file]… :: eXclude Files matching given names/paths/wildcards.
/XD dirs [dirs]… :: eXclude Directories matching given names/paths.

/XC :: eXclude Changed files.
/XN :: eXclude Newer files.
/XO :: eXclude Older files.
/XX :: eXclude eXtra files and directories.
/XL :: eXclude Lonely files and directories.
/IS :: Include Same files.
/IT :: Include Tweaked files.

/MAX:n :: MAXimum file size – exclude files bigger than n bytes.
/MIN:n :: MINimum file size – exclude files smaller than n bytes.

/MAXAGE:n :: MAXimum file AGE – exclude files older than n days/date.
/MINAGE:n :: MINimum file AGE – exclude files newer than n days/date.
/MAXLAD:n :: MAXimum Last Access Date – exclude files unused since n.
/MINLAD:n :: MINimum Last Access Date – exclude files used since n.
(If n < 1900 then n = n days, else n = YYYYMMDD date).

/XJ :: eXclude Junction points. (normally included by default).

/FFT :: assume FAT File Times (2-second granularity).
/DST :: compensate for one-hour DST time differences.

/XJD :: eXclude Junction points for Directories.
/XJF :: eXclude Junction points for Files.

Retry Options :

/R:n :: number of Retries on failed copies: default 1 million.
/W:n :: Wait time between retries: default is 30 seconds.

/REG :: Save /R:n and /W:n in the Registry as default settings.

/TBD :: wait for sharenames To Be Defined (retry error 67).

Logging Options :

/L :: List only – don’t copy, timestamp or delete any files.
/X :: report all eXtra files, not just those selected.
/V :: produce Verbose output, showing skipped files.
/TS :: include source file Time Stamps in the output.
/FP :: include Full Pathname of files in the output.
/BYTES :: Print sizes as bytes.

/NS :: No Size – don’t log file sizes.
/NC :: No Class – don’t log file classes.
/NFL :: No File List – don’t log file names.
/NDL :: No Directory List – don’t log directory names.

/NP :: No Progress – don’t display % copied.
/ETA :: show Estimated Time of Arrival of copied files.

/LOG:file :: output status to LOG file (overwrite existing log).
/LOG+:file :: output status to LOG file (append to existing log).

/UNILOG:file :: output status to LOG file as UNICODE (overwrite existing log).
/UNILOG+:file :: output status to LOG file as UNICODE (append to existing log).

/TEE :: output to console window, as well as the log file.

/NJH :: No Job Header.
/NJS :: No Job Summary.

/UNICODE :: output status as UNICODE.

Job Options :

/JOB:jobname :: take parameters from the named JOB file.
/SAVE:jobname :: SAVE parameters to the named job file
/QUIT :: QUIT after processing command line (to view parameters).

/NOSD :: NO Source Directory is specified.
/NODD :: NO Destination Directory is specified.
/IF :: Include the following Files.

Robocopy Examples

To use Robocopy is simple, just like how you would use Copy and Xcopy commands. For example, to copy entire folder of C:\Users to C:\UserBackup, simply type:

Robocopy C:\Users C:\UserBackup

Some extra’s:

Microsoft Robocopy GUI: Download

MultiThread:

As Robocopy is generally a command-line only utility (although a GUI add-on is available for Robocopy), the new multi-threaded operation capability has to be called via a new switch supported by Robocopy. The new multi-threaded copy feature can be enabled and turned on with the following parameter:

/MT[:n]

Where n will instruct Robocopy to do multi-threaded copies with n threads (default 8). The value of n must be at least 1 and not greater than 128 (between 1 to 128), with 1 as single thread. In fact, Robocopy will copy files and folders in multi-threaded operation by default, with 8 threads in one go. Note that /MT[:n] switch is not compatible with the /IPG and /EFSRAW operations.

For example,

robocopy C:\Backup I:\MoreBackup /MT:88

Yes this is a compilation of a few articles and some old notes of mine and friends.