Changeset 456f19 for src/Fragmentation

Timestamp:

Apr 5, 2018, 9:03:48 PM (7 years ago)

Author:

Frederik Heber <frederik.heber@…>

Branches:

AutomationFragmentation_failures, Candidate_v1.6.1, ChemicalSpaceEvaluator, Enhanced_StructuralOptimization_continued, Exclude_Hydrogens_annealWithBondGraph, ForceAnnealing_with_BondGraph, ForceAnnealing_with_BondGraph_contraction-expansion, Gui_displays_atomic_force_velocity, JobMarket_RobustOnKillsSegFaults, JobMarket_StableWorkerPool, PythonUI_with_named_parameters, StoppableMakroAction, TremoloParser_IncreasedPrecision

Children:

1e58bb

Parents:

0542e2

git-author:

Frederik Heber <frederik.heber@…> (04/05/18 12:36:19)

git-committer:

Frederik Heber <frederik.heber@…> (04/05/18 21:03:48)

Message:

SpecificFragmentController checks whether there are still workers left alive.

• this avoids all workers having crashed over a job but the server is still present and thus the action hangs indefinitely.

File:

• src/Fragmentation/Automation/SpecificFragmentController_ReceiveResultContainer_impl.hpp (modified) (2 diffs)

src/Fragmentation/Automation/SpecificFragmentController_ReceiveResultContainer_impl.hpp

@@ -20 +20 @@
 #include "CodePatterns/toString.hpp"
 
+#include <numeric>
 
 template <typename T>
@@ -72 +73 @@
       LOG(1, "INFO: #" << JobStatus.first << " are waiting in the queue and #" << NoReceivedResults << " of " << NoExpectedResults << " jobs are calculated so far.");
     }
-  }
+
+    // if not all jobs are done yet, check number of present workers
+    if (NoReceivedResults != NoExpectedResults) {
+      callback.checkEnrolledWorkers(callback.host, callback.port);
+      callback.RunService("Checking on number of workers");
+      if (callback.getExitflag() != 0)
+        break;
+      const std::vector<size_t>& NumberWorkers = callback.getNumberOfWorkers();
+      const int TotalNumberWorkers = std::accumulate(
+          NumberWorkers.begin(), NumberWorkers.end(), 0);
+      if (TotalNumberWorkers == 0) {
+        ELOG(0, "Not all jobs are finished, but no workers are present working on it?!");
+        callback.setExitflag(ExitflagContainer::ErrorFlag);
+        break;
+      } else {
+        LOG(4, "There are " << NumberWorkers << " workers enrolled.");
+      }
+    }
+}
 }