Using Windows PowerShell to Create BITS Transfer Jobs

You can use PowerShell cmdlets to create synchronous and asynchronous Background Intelligent Transfer Service (BITS) transfer jobs.

All of the examples in this topic use the Start-BitsTransfer cmdlet. To use the cmdlet, be sure to import the module first. To install the module, run the following command: Import-Module BitsTransfer. For more information, type Get-Help Start-BitsTransfer at the PowerShell prompt.

When you use *-BitsTransfer cmdlets from within a process that runs in a noninteractive context, such as a Windows service, you may not be able to add files to BITS jobs, which can result in a suspended state. For the job to proceed, the identity that was used to create a transfer job must be logged on. For example, when creating a BITS job in a PowerShell script that was executed as a Task Scheduler job, the BITS transfer will never complete unless the Task Scheduler’s task setting «Run only when user is logged on» is enabled.

To create a synchronous BITS transfer job

The grave-accent character (`) is used to indicate a line break.

In the preceding example, the local and remote names of the file are specified in the Source and Destination parameters, respectively. The command prompt returns when the file transfer is complete or when it enters an error state.

The default transfer type is Download. When you upload files to an HTTP location, the TransferType parameter must be set to Upload.

Because parameter position is enforced for the Start-BitsTransfer cmdlet, the parameter names do not need to be specified for the Source and Destination parameters. Therefore, this command can be simplified as follows.

To create a synchronous BITS transfer job with multiple files

In the preceding example, the Start-BitsTransfer command creates a new BITS transfer job. All of the files are added to this job and transferred sequentially to the client.

The destination path cannot use wildcard characters. The destination path supports relative directories, root paths, or implicit directories (that is, the current directory). Destination files cannot be renamed by using a wildcard character. Additionally, HTTP and HTTPS URLs do not work with wildcards. Wildcards are only valid for UNC paths and local directories.

To create a synchronous BITS transfer job and specify credentials for a remote server

In the preceding example, a user creates a BITS transfer job to download a file from a server that requires authentication. The user is prompted for credentials, and the Credential parameter passes a credential object to the Start-BitsTransfer cmdlet. The user sets an explicit proxy, and the BITS transfer job uses only the proxies that are defined by the ProxyList parameter. The DisplayName parameter gives the BITS transfer job a unique display name.

To create a synchronous BITS transfer job from a CSV file

The «|» is the pipe character.

In the preceding example, a user creates a BITS transfer job that uploads multiple files from a client. The Import-CSV cmdlet imports the source and destination file locations and pipes them to the Start-BitsTransfer command. The Start-BitsTransfer command creates a new BITS transfer job for each file, adds the files to the job, and then transfers them sequentially to the server.

The contents of the Filelist.txt file should be in the following format:

To create an asynchronous BITS transfer job

In the preceding example, the BITS transfer job was assigned to the $Job variable. The files are downloaded sequentially. After the transfer job is complete, the files are immediately available. If $Job.JobState returns «Transferred», the $Job object is sent to the Complete-BitsTransfer cmdlet.

If $Job.JobState returns «Error», the $Job object is sent to the Format-List cmdlet to list the errors.

To manage PowerShell Remote sessions

Starting with WindowsВ 10, version 1607, you can run PowerShell Cmdlets, BITSAdmin, or other applications that use the BITSВ interfaces from a PowerShell Remote command line connected to another machine (physical or virtual). This capability is not available when using a PowerShell Direct command line to a virtual machine on the same physical machine, and it is not available when using WinRM cmdlets.

A BITS Job created from a Remote PowerShell session runs under that session’s user account context and will only make progress when there is at least one active local logon session or Remote PowerShell session associated with that user account. You can use PowerShell’s persistent PSSessions to run remote commands without the need to keep a PowerShell window open for each job to continue making progress, as described in PowerShell Basics: Remote Management.

• New-PSSession creates a persistent Remote PowerShell session. Once created, the PSSession objects persist in the remote machine until explicitly deleted. Any BITS jobs initiated in an active session will make progress transferring data, even after the client has disconnected from the session.
• Disconnect-PSSession disconnects the client machine from a Remote PowerShell session and the session’s state continues to be maintained by the remote machine. Most importantly, the remote session’s processes will continue executing, and BITS jobs will continue to make progress. The client machine can even reboot and/or turn off after calling Disconnect-PSSession.
• Connect-PSSession re-connects the client machine to an active Remote PowerShell session.
• Remove-PSSession tears down a Remote PowerShell session.

The example below shows how to use PowerShell Remote to work with asynchronous BITS transfer jobs in a way that allows the job to continue to make progress even when you are not actively connected to the remote session.

Job Objects

A job object allows groups of processes to be managed as a unit. Job objects are namable, securable, sharable objects that control attributes of the processes associated with them. Operations performed on a job object affect all processes associated with the job object. Examples include enforcing limits such as working set size and process priority or terminating all processes associated with a job.

Creating Jobs

To create a job object, use the CreateJobObject function. When the job is created, no processes are associated with the job.

To associate a process with a job, use the AssignProcessToJobObject function. After a process is associated with a job, the association cannot be broken. A process can be associated with more than one job in a hierarchy of nested jobs. For more information, see Nested Jobs.

WindowsВ 7, Windows ServerВ 2008В R2, WindowsВ XP with SP3, Windows ServerВ 2008, WindowsВ Vista and Windows ServerВ 2003: A process can be associated with only one job. Jobs cannot be nested. The ability to nest jobs was added in WindowsВ 8 and Windows ServerВ 2012.

You can specify a security descriptor for a job object when you call the CreateJobObject function. For more information, see Job Object Security and Access Rights.

Managing Processes in Jobs

After a process is associated with a job, by default any child processes it creates using CreateProcess are also associated with the job. (Child processes created using Win32_Process.Create are not associated with the job.) This default behavior can be changed by setting the extended limit JOB_OBJECT_LIMIT_BREAKAWAY_OK or JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK for the job.

• If the job has the extended limit JOB_OBJECT_LIMIT_BREAKAWAY_OK and the parent process was created with the CREATE_BREAKAWAY_FROM_JOB flag, then child processes of the parent process are not associated with the job.
• If the job has the extended limit JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK, then child processes of any parent process associated with the job are not associated with the job. It is not necessary for parent processes to be created with the CREATE_BREAKAWAY_FROM_JOB flag.

If the job is nested, the breakaway settings of parent jobs in the hierarchy affect whether child processes are associated with another job in the hierarchy. For more information, see Nested Jobs.

To determine if a process is running in a job, use the IsProcessInJob function.

To terminate all processes currently associated with a job object, use the TerminateJobObject function.

Job Limits and Notifications

A job can enforce limits such as working set size, process priority, and end-of-job time limit on each process that is associated with the job. If a process associated with a job attempts to increase its working set size or process priority from the limit established by the job, the function calls succeed but are silently ignored. A job can also set limits that trigger a notification when they are exceeded but allow the job to continue to run.

To set limits for a job, use the SetInformationJobObject function. For a list of possible limits that can be set for a job, see the following topics:

Security limits must be set individually for each process associated with a job object. For more information, see Process Security and Access Rights.

WindowsВ XP with SP3 and Windows ServerВ 2003: The SetInformationJobObject function can be used to set security limitations for all processes associated with a job object. Starting with WindowsВ Vista, security limits must be set individually for each process associated with a job object.

If the job is nested, parent jobs in the hierarchy influence the limit that is enforced for the job. For more information, see Nested Jobs.

If the job has an associated I/O completion port, it can receive notifications when certain job limits are exceeded. The system sends messages to the completion port when a limit is exceeded or certain other events occur. To associate a completion port with a job, use the SetInformationJobObject function with the job object information class JobObjectAssociateCompletionPortInformation and a pointer to a JOBOBJECT_ASSOCIATE_COMPLETION_PORT structure. It is best to do this when the job is inactive, to reduce the chance of missing notifications for processes whose states change during the association of the completion port.

All messages are sent directly from the job as if the job had called the PostQueuedCompletionStatus function. A thread must monitor the completion port using the GetQueuedCompletionStatus function to pick up the messages. Note that, with the exception of limits set with the JobObjectNotificationLimitInformation information class, delivery of messages to the completion port is not guaranteed; failure of a message to arrive does not necessarily mean that the event did not occur. Notifications for limits set with JobObjectNotificationLimitInformation are guaranteed to arrive at the completion port. For a list of possible messages, see JOBOBJECT_ASSOCIATE_COMPLETION_PORT.

Resource Accounting for Jobs

The job object records basic accounting information for all its associated processes, including those that have terminated. To retrieve this accounting information, use the QueryInformationJobObject function. For a list of the accounting information that is maintained for a job, see the following topics:

If the job object is nested, accounting information for each child job is aggregated in its parent job. For more information, see Nested Jobs.

Managing Job Objects

The state of a job object is set to signaled when all of its processes are terminated because the specified end-of-job time limit has been exceeded. Use WaitForSingleObject or WaitForSingleObjectEx to monitor the job object for this event.

To obtain a handle for an existing job object, use the OpenJobObject function and specify the name given to the object when it was created. Only named job objects can be opened.

To close a job object handle, use the CloseHandle function. The job is destroyed when its last handle has been closed and all associated processes have been terminated. However, if the job has the JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE flag specified, closing the last job object handle terminates all associated processes and then destroys the job object itself. If a nested job has the JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE flag specified, closing the last job object handle terminates all processes associated with the job and its child jobs in the hierarchy.

Managing a Process Tree that Uses Job Objects

Starting with WindowsВ 8 and Windows ServerВ 2012, an application can use nested jobs to manage a process tree that uses more than one job object. However, an application that must run on WindowsВ 7, Windows ServerВ 2008В R2, or earlier versions of Windows that do not support nested jobs must manage the process tree in other ways.

If a tool must manage a process tree that uses job objects and it is not possible to use nested jobs, both the tool and the members of the process tree must cooperate. Use one of the following options:

Use the JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK limit. If the tool uses this limit, it cannot monitor an entire process tree. The tool can monitor only the processes it adds to the job. If these processes create child processes, they are not associated with the job. In this option, child processes can be associated with other job objects.

Use the JOB_OBJECT_LIMIT_BREAKAWAY_OK limit. If the tool uses this limit, it can monitor the entire process tree, except for those processes that any member of the tree explicitly breaks away from the tree. A member of the tree can create a child process in a new job object by calling the CreateProcess function with the CREATE_BREAKAWAY_FROM_JOB flag, then calling the AssignProcessToJobObject function. Otherwise, the member must handle cases in which AssignProcessToJobObject fails.

The CREATE_BREAKAWAY_FROM_JOB flag has no effect if the tree is not being monitored by the tool. Therefore, this is the preferred option, but it requires advance knowledge of the processes being monitored.

Prevent breakaways of any kind by setting neither the JOB_OBJECT_LIMIT_BREAKAWAY_OK nor the JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK limit. In this option, the tool can monitor the entire process tree. However, if a child process attempts to associate itself or another child process with a job by calling AssignProcessToJobObject, the call will fail. If the process was designed to be associated with a specific job, this failure may prevent the process from working properly.