Skip to content

Allow for nested ThreadJobs by resetting the pool object #59

@codykonior

Description

@codykonior

Summary of the new feature / enhancement

ThreadJobs become a problem if you want to use them throughout a module; you have to be careful that you never create a ThreadJob within something else running within a ThreadJob, because it has a global queue.

If you are executing any number of tasks over your throttle limit, and then start a new job within one of those, they can deadlock because the pool is full.

So something as simple as this will never return:

$jobs = 1..5 | %{ 
        Start-ThreadJob { 
                Start-Sleep -Seconds 1
                Start-ThreadJob { 
                        Start-Sleep -Seconds 1
                } | Wait-Job | Receive-Job
        } -Throttle 5
}
$jobs | Wait-Job

Ideally you should have the option to avoid this.

Proposed technical implementation details (optional)

If you add a Restart-ThreadJob cmdlet that will reset the pool, you can now control when you want to create a fresh pool. This guards against that kind of deadlock.

$jobs = 1..5 | %{ 
        Start-ThreadJob { 
                Start-Sleep -Seconds 1
                Restart-ThreadJob
                Start-ThreadJob { 
                        Start-Sleep -Seconds 1
                } | Wait-Job | Receive-Job
        } -Throttle 5
}
$jobs | Wait-Job

Works. Of course, you now have 1 pool of 5 and another 5 pools of 5 (the default), but it doesn't hang. You get to control it.

diff --git a/src/Microsoft.PowerShell.ThreadJob.psd1 b/src/Microsoft.PowerShell.ThreadJob.psd1
index 7b127d4..342009b 100644
--- a/src/Microsoft.PowerShell.ThreadJob.psd1
+++ b/src/Microsoft.PowerShell.ThreadJob.psd1
@@ -44,7 +44,7 @@ number of jobs drops below the throttle limit.
 PowerShellVersion = '5.1'
 
 # Cmdlets to export from this module
-CmdletsToExport = 'Start-ThreadJob'
+CmdletsToExport = 'Start-ThreadJob', 'Restart-ThreadJob'
 
 # Private data to pass to the module specified in RootModule/ModuleToProcess. This may also contain a PSData hashtable with additional module metadata used by PowerShell.
 PrivateData = @{
diff --git a/src/code/Microsoft.PowerShell.ThreadJob.cs b/src/code/Microsoft.PowerShell.ThreadJob.cs
index c967e9d..ff45cf9 100644
--- a/src/code/Microsoft.PowerShell.ThreadJob.cs
+++ b/src/code/Microsoft.PowerShell.ThreadJob.cs
@@ -142,6 +142,31 @@ namespace Microsoft.PowerShell.ThreadJob
         #endregion
     }
 
+    [Cmdlet(VerbsLifecycle.Restart, "ThreadJob")]
+    public sealed class RestartThreadJobCommand : PSCmdlet
+    {
+        #region Overrides
+
+        protected override void BeginProcessing()
+        {
+            base.BeginProcessing();
+
+            ThreadJob.Reset();
+        }
+
+        protected override void ProcessRecord()
+        {
+            base.ProcessRecord();
+        }
+
+        protected override void EndProcessing()
+        {
+            base.EndProcessing();
+        }
+
+        #endregion
+    }
+
     public sealed class ThreadJobSourceAdapter : JobSourceAdapter
     {
         #region Members
@@ -491,6 +516,8 @@ namespace Microsoft.PowerShell.ThreadJob
 
         private static ThreadJobQueue s_JobQueue;
 
+        private static object _syncObject = new object();
+
         #endregion
 
         #region Properties
@@ -508,10 +535,18 @@ namespace Microsoft.PowerShell.ThreadJob
 
         #region Constructors
 
+        public static void Reset()
+        {
+            lock (_syncObject)
+            {
+                s_JobQueue = new ThreadJobQueue(5);
+            }
+        }
+
         // Constructors
         static ThreadJob()
         {
-            s_JobQueue = new ThreadJobQueue(5);
+            Reset();
             if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
             {
                 Assembly assembly = typeof(PSObject).Assembly;
@@ -806,7 +841,10 @@ namespace Microsoft.PowerShell.ThreadJob
         /// <param name="throttleLimit"></param>
         public static void StartJob(ThreadJob job, int throttleLimit)
         {
-            s_JobQueue.EnqueueJob(job, throttleLimit);
+            lock (_syncObject)
+            {
+                s_JobQueue.EnqueueJob(job, throttleLimit);
+            }
         }
 
         /// <summary>

If you want to implement it in some better way, go ahead. This is just something.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions