Ignore:
Timestamp:
Jun 27, 2012, 4:07:18 PM (13 years ago)
Author:
Frederik Heber <heber@…>
Branches:
Action_Thermostats, Add_AtomRandomPerturbation, Add_FitFragmentPartialChargesAction, Add_RotateAroundBondAction, Add_SelectAtomByNameAction, Added_ParseSaveFragmentResults, AddingActions_SaveParseParticleParameters, Adding_Graph_to_ChangeBondActions, Adding_MD_integration_tests, Adding_ParticleName_to_Atom, Adding_StructOpt_integration_tests, AtomFragments, Automaking_mpqc_open, AutomationFragmentation_failures, Candidate_v1.5.4, Candidate_v1.6.0, Candidate_v1.6.1, ChangeBugEmailaddress, ChangingTestPorts, ChemicalSpaceEvaluator, CombiningParticlePotentialParsing, Combining_Subpackages, Debian_Package_split, Debian_package_split_molecuildergui_only, Disabling_MemDebug, Docu_Python_wait, EmpiricalPotential_contain_HomologyGraph, EmpiricalPotential_contain_HomologyGraph_documentation, Enable_parallel_make_install, Enhance_userguide, Enhanced_StructuralOptimization, Enhanced_StructuralOptimization_continued, Example_ManyWaysToTranslateAtom, Exclude_Hydrogens_annealWithBondGraph, FitPartialCharges_GlobalError, Fix_BoundInBox_CenterInBox_MoleculeActions, Fix_ChargeSampling_PBC, Fix_ChronosMutex, Fix_FitPartialCharges, Fix_FitPotential_needs_atomicnumbers, Fix_ForceAnnealing, Fix_IndependentFragmentGrids, Fix_ParseParticles, Fix_ParseParticles_split_forward_backward_Actions, Fix_PopActions, Fix_QtFragmentList_sorted_selection, Fix_Restrictedkeyset_FragmentMolecule, Fix_StatusMsg, Fix_StepWorldTime_single_argument, Fix_Verbose_Codepatterns, Fix_fitting_potentials, Fixes, ForceAnnealing_goodresults, ForceAnnealing_oldresults, ForceAnnealing_tocheck, ForceAnnealing_with_BondGraph, ForceAnnealing_with_BondGraph_continued, ForceAnnealing_with_BondGraph_continued_betteresults, ForceAnnealing_with_BondGraph_contraction-expansion, FragmentAction_writes_AtomFragments, FragmentMolecule_checks_bonddegrees, GeometryObjects, Gui_Fixes, Gui_displays_atomic_force_velocity, ImplicitCharges, IndependentFragmentGrids, IndependentFragmentGrids_IndividualZeroInstances, IndependentFragmentGrids_IntegrationTest, IndependentFragmentGrids_Sole_NN_Calculation, JobMarket_RobustOnKillsSegFaults, JobMarket_StableWorkerPool, JobMarket_unresolvable_hostname_fix, MoreRobust_FragmentAutomation, ODR_violation_mpqc_open, PartialCharges_OrthogonalSummation, PdbParser_setsAtomName, PythonUI_with_named_parameters, QtGui_reactivate_TimeChanged_changes, Recreated_GuiChecks, Rewrite_FitPartialCharges, RotateToPrincipalAxisSystem_UndoRedo, SaturateAtoms_findBestMatching, SaturateAtoms_singleDegree, StoppableMakroAction, Subpackage_CodePatterns, Subpackage_JobMarket, Subpackage_LinearAlgebra, Subpackage_levmar, Subpackage_mpqc_open, Subpackage_vmg, Switchable_LogView, ThirdParty_MPQC_rebuilt_buildsystem, TrajectoryDependenant_MaxOrder, TremoloParser_IncreasedPrecision, TremoloParser_MultipleTimesteps, TremoloParser_setsAtomName, Ubuntu_1604_changes, stable
Children:
cc276e
Parents:
7670865
git-author:
Frederik Heber <heber@…> (02/06/12 09:57:56)
git-committer:
Frederik Heber <heber@…> (06/27/12 16:07:18)
Message:

Abstracted FragmentJob and introduced SystemCommandJob which has its former functionality.

  • FragmentJob now has just virtual function Work(), not implemented.
  • former command and outputfile is taken over by SystemCommandJob.
  • a FragmentJob or derived class is default cstor'ed always with JobId::IllegalJob.
  • so far SystemCommandJob captures command's stdout and parses it back from file. We use mkstemps() for creating a random file and popen() to pipe stdout directly. For this we require boost::iostreams and ::filesystem.
  • SystemCommandJob has virtual method extractString() to be adaptable to the output of various system commands.
  • changes almost everywhere due to SystemCommandJob taking place of FragmentJob and FragmentJob::ptr and FragmentResult::ptr requirements.
  • added FragmentWorker::dummyInit() to enforce binding of the executable to all possible derived FragmentJob's.
  • added unit test for SystemCommandJob.
  • added FragmentJobStub that implements a no-brainer job for testing.
  • NOTE: boost::iostreams is incompatible with MemDebug due to placement new.
Location:
src/Fragmentation/Automation
Files:
6 added
11 edited

Legend:

Unmodified
Added
Removed

  • src/Fragmentation/Automation/Controller/Commands/ReceiveJobsOperation.cpp

    r7670865 rd920b9  
    120120    jobs.push_back(job);
    121121  }
     122  LOG(1, "INFO: ReceiveJobsOperation now contains " << jobs.size() << " job(s).");
    122123}
    123124

  • src/Fragmentation/Automation/FragmentScheduler.cpp

    r7670865 rd920b9  
    3434#include "CodePatterns/Info.hpp"
    3535#include "CodePatterns/Log.hpp"
    36 #include "Jobs/FragmentJob.hpp"
     36#include "Jobs/SystemCommandJob.hpp"
    3737#include "JobId.hpp"
    3838
    3939#include "FragmentScheduler.hpp"
    4040
    41 FragmentJob::ptr FragmentScheduler::NoJob(new FragmentJob(std::string("/bin/false"), std::string("NoJob"), JobId::NoJob));
     41FragmentJob::ptr FragmentScheduler::NoJob(new SystemCommandJob(std::string("/bin/false"), std::string("NoJob"), JobId::NoJob));
    4242
    4343/** Constructor of class FragmentScheduler.

  • src/Fragmentation/Automation/FragmentWorker.cpp

    r7670865 rd920b9  
    3434#include "CodePatterns/Log.hpp"
    3535#include "Jobs/FragmentJob.hpp"
     36#include "Jobs/SystemCommandJob.hpp"
    3637#include "Results/FragmentResult.hpp"
    3738#include "FragmentWorker.hpp"
    3839
    3940FragmentResult::ptr FragmentWorker::EmptyResult( new FragmentResult(JobId::NoJob, std::string("EmptyResult")) );
     41
     42/** Helper function to enforce binding of FragmentWorker to possible derived
     43 * FragmentJob classes.
     44 */
     45void dummyInit() {
     46  SystemCommandJob("/bin/false", "something", JobId::IllegalJob);
     47}
    4048
    4149/// Constructor starts the asynchronous connect operation.
    … …  
    102110    if (job->getId() != JobId::NoJob) {
    103111      // Print out the data that was received.
    104       std::cout << "Job output: " << job->outputfile << "\n";
    105112      std::cout << "Job id: " << job->getId() << "\n";
    106113
    … …  
    143150  LOG(1, "INFO: Job #" << job->getId() << " calculated and sent.");
    144151  // erase job
    145   job = FragmentJob::ptr(new FragmentJob());
     152  job.reset();
    146153}
    147154

  • src/Fragmentation/Automation/Jobs/FragmentJob.cpp

    r7670865 rd920b9  
    2020#include "CodePatterns/MemDebug.hpp"
    2121
    22 #include <cstdlib>
    23 #include <fstream>
    24 #include <string>
    25 
    26 #include "CodePatterns/Info.hpp"
    27 #include "CodePatterns/Log.hpp"
    28 #include "CodePatterns/toString.hpp"
    2922#include "FragmentJob.hpp"
    3023#include "Results/FragmentResult.hpp"
    … …  
    3427 */
    3528FragmentJob::FragmentJob() :
    36   JobId(-1),
    37   outputfile("")
     29  JobId(JobId::IllegalJob)
    3830{}
    3931
    4032/** Constructor for class FragmentJob.
    4133 *
    42  * \param _command command to execute
    43  * \param _outputfile configuration file for solver
    4434 * \param _JobId unique id of this job
    4535 */
    46 FragmentJob::FragmentJob(const std::string &_command, const std::string &_outputfile, const JobId_t _JobId) :
    47   JobId(_JobId),
    48   command(_command),
    49   outputfile(_outputfile)
     36FragmentJob::FragmentJob(const JobId_t _JobId) :
     37  JobId(_JobId)
    5038{}
    5139
    … …  
    5644{}
    5745
    58 /** Work routine of this FragmentJob.
    59  *
    60  * This function encapsulates all the work that has to be done to generate
    61  * a FragmentResult. Hence, the FragmentWorker does not need to know anything
    62  * about the operation: it just receives it and executes this function.
    63  *
    64  * \return result of this job
    65  */
    66 FragmentResult::ptr FragmentJob::Work()
    67 {
    68   Info info((std::string(__FUNCTION__)+std::string(", id #")+toString(getId())).c_str());
    69   FragmentResult::ptr s( new FragmentResult(getId()) );
    70 
    71   // write outputfile to file
    72   std::string inputfilename =
    73       std::string("/tmp/dummy_")
    74       +toString(getId());
    75   std::string outputfilename = inputfilename+std::string(".out");
    76   inputfilename += std::string(".dat");
    77   std::ofstream output(outputfilename.c_str());
    78   output << outputfile << std::endl;
    79   output.close();
    80 
    81   // fork into subprocess and launch command
    82   s->exitflag = std::system((command+std::string(" ")+inputfilename
    83                         +std::string(" >")+outputfilename).c_str());
    84   std::ifstream resultfile;
    85   resultfile.open(outputfilename.c_str(), ifstream::in);
    86   char line[1024];
    87   while (resultfile.good()) {
    88     resultfile.getline(line, 1023);
    89     s->result += line;
    90   }
    91   resultfile.close();
    92 
    93   return s;
    94 }
    95 
    9646/** Comparator for class FragmentJob.
    9747 * \param other instance to compare to
    … …  
    10050bool FragmentJob::operator==(const FragmentJob &other) const
    10151{
    102   return (command == other.command)
    103       && (outputfile == other.outputfile)
    104       && (getId() == other.getId());
     52  return (getId() == other.getId());
    10553}

  • src/Fragmentation/Automation/Jobs/FragmentJob.hpp

    r7670865 rd920b9  
    4343  typedef boost::shared_ptr<FragmentJob> ptr;
    4444
    45   FragmentJob();
    46   FragmentJob(const std::string &_command, const std::string &_outputfile, const JobId_t _JobId);
     45  FragmentJob(const JobId_t _JobId);
    4746  ~FragmentJob();
    4847
    49   FragmentResult::ptr Work();
     48  virtual FragmentResult::ptr Work() = 0;
    5049
    5150  bool operator==(const FragmentJob &other) const;
    … …  
    5554  }
    5655
    57   //!> string containing the command to launch the solver
    58   std::string command;
     56private:
     57  /** private default cstor for serialization only
     58   *
     59   * Use normal cstor with JobId::IllegalJob if you want to instantiate
     60   * a job without id.
     61   */
     62  FragmentJob();
    5963
    60   //!> string containing the configuration file for the solver
    61   std::string outputfile;
    62 
    63 private:
    6464  friend class boost::serialization::access;
    6565  // serialization
    … …  
    6868  {
    6969    ar & boost::serialization::base_object<JobId>(*this);
    70     ar & command;
    71     ar & outputfile;
    7270  }
    7371};

  • src/Fragmentation/Automation/Makefile.am

    r7670865 rd920b9  
    1616  JobId.cpp \
    1717  Jobs/FragmentJob.cpp \
     18  Jobs/SystemCommandJob.cpp \
    1819  Results/FragmentResult.cpp
    1920         
    … …  
    2223  JobId.hpp \
    2324  Jobs/FragmentJob.hpp \
     25  Jobs/SystemCommandJob.hpp \
    2426  Results/FragmentResult.hpp
    2527
    … …  
    2830nobase_libMolecuilderFragmentJobs_la_include_HEADERS = $(FRAGMENTJOBSHEADER)
    2931libMolecuilderFragmentJobs_la_SOURCES = $(FRAGMENTJOBSSOURCE)
     32libMolecuilderFragmentJobs_la_LIBADD = \
     33  $(BOOST_FILESYSTEM_LDFLAGS) $(BOOST_FILESYSTEM_LIBS) \
     34  $(BOOST_IOSTREAMS_LDFLAGS) $(BOOST_IOSTREAMS_LIBS)
    3035
    3136FRAGMENTATIONAUTOMATIONSOURCE = \

  • src/Fragmentation/Automation/Worker.cpp

    r7670865 rd920b9  
    4141  atexit(cleanUp);
    4242
    43   setVerbosity(3);
     43  setVerbosity(5);
    4444
    4545  size_t Exitflag = 0;

  • src/Fragmentation/Automation/controller.cpp

    r7670865 rd920b9  
    3939#include "Controller/Commands/SendResultsOperation.hpp"
    4040#include "Controller/Commands/ShutdownOperation.hpp"
    41 #include "Jobs/FragmentJob.hpp"
     41#include "Jobs/SystemCommandJob.hpp"
    4242#include "Results/FragmentResult.hpp"
    4343
    … …  
    5252void createjobs(std::vector<FragmentJob::ptr> &jobs)
    5353{
    54   FragmentJob::ptr testJob( new FragmentJob( std::string("cat"), std::string("Nothing"), 1) );
    55   FragmentJob::ptr othertestJob( new FragmentJob( std::string("cat"), std::string("Nothing"), 2) );
     54  FragmentJob::ptr testJob( new SystemCommandJob( std::string("cat"), std::string("Nothing"), 1) );
     55  FragmentJob::ptr othertestJob( new SystemCommandJob( std::string("cat"), std::string("Nothing"), 2) );
    5656  jobs.push_back(testJob);
    5757  jobs.push_back(othertestJob);

  • src/Fragmentation/Automation/unittests/FragmentJobUnitTest.cpp

    r7670865 rd920b9  
    4040/********************************************** Test classes **************************************/
    4141
     42#include "stubs/FragmentJobStub.hpp"
     43
    4244// Registers the fixture into the 'registry'
    4345CPPUNIT_TEST_SUITE_REGISTRATION( FragmentJobTest );
    … …  
    4850  // Throw assertions
    4951  ASSERT_DO(Assert::Throw);
    50 
    51   command = std::string("/bin/true");
    52   outputfile = std::string("do something");
    5352}
    5453
    … …  
    6261void FragmentJobTest::equalityTest()
    6362{
    64   FragmentJob::ptr testJob(new FragmentJob(command, outputfile, 1));
    65   FragmentJob::ptr sameIdJob(new FragmentJob(command, outputfile, 1));
    66   FragmentJob::ptr otherIdJob(new FragmentJob(command, outputfile, 2));
    67   FragmentJob::ptr copiedJob(new FragmentJob(*testJob));
    68   FragmentJob::ptr copiedchangedJob(new FragmentJob(*testJob));
    69   copiedchangedJob->setId(3);
     63  FragmentJob::ptr testJob( new FragmentJobStub(1) );
     64  FragmentJob::ptr sameIdJob( new FragmentJobStub(1) );
     65  FragmentJob::ptr otherIdJob( new FragmentJobStub(2) );
    7066
    7167  CPPUNIT_ASSERT( *testJob == *sameIdJob );
    7268  CPPUNIT_ASSERT( *testJob != *otherIdJob );
    73   CPPUNIT_ASSERT( *testJob == *copiedJob );
    74   CPPUNIT_ASSERT( *testJob != *copiedchangedJob );
    7569}
    7670
    … …  
    7973void FragmentJobTest::WorkTest()
    8074{
    81   FragmentJob::ptr testJob(new FragmentJob(command, outputfile, 1));
    82   FragmentJob::ptr othertestJob(new FragmentJob(command, outputfile, 2));
     75  FragmentJob::ptr testJob( new FragmentJobStub(1) );
     76  FragmentJob::ptr othertestJob( new FragmentJobStub(2) );
    8377
    8478  FragmentResult::ptr testResult(testJob->Work());
    … …  
    9387void FragmentJobTest::serializationTest()
    9488{
    95   FragmentJob::ptr testJob(new FragmentJob(command, outputfile, 1));
     89  FragmentJob::ptr testJob( new FragmentJobStub(1) );
    9690  // write element to stream
    9791  std::stringstream stream;
    … …  
    108102  ia >> otherJob;
    109103
     104  // FragmentJobStub is same as FragmentJob contentwise, hence no casting required
    110105  CPPUNIT_ASSERT (*testJob == *otherJob);
    111106}

  • src/Fragmentation/Automation/unittests/FragmentQueueUnitTest.cpp

    r7670865 rd920b9  
    3636/********************************************** Test classes **************************************/
    3737
     38#include "stubs/FragmentJobStub.hpp"
     39
    3840// Registers the fixture into the 'registry'
    3941CPPUNIT_TEST_SUITE_REGISTRATION( FragmentQueueTest );
    … …  
    5860void FragmentQueueTest::JobTest()
    5961{
    60   FragmentJob::ptr testJob(new FragmentJob);
     62  FragmentJob::ptr testJob(new FragmentJobStub(JobId::IllegalJob));
    6163  /// check for illegal id
    6264#ifndef NDEBUG
    … …  
    6668  // set to valid id
    6769  testJob->setId(1);
    68   testJob->outputfile = std::string("do something");
    6970
    7071  CPPUNIT_ASSERT_EQUAL(false, queue->isJobPresent() );
    … …  
    122123{
    123124  // prepare some jobs
    124   FragmentJob::ptr testJob(new FragmentJob);
     125  FragmentJob::ptr testJob(new FragmentJobStub(JobId::IllegalJob));
    125126  testJob->setId((JobId_t)1);
    126   testJob->outputfile = std::string("do something");
    127   FragmentJob::ptr anothertestJob(new FragmentJob);
     127  FragmentJob::ptr anothertestJob(new FragmentJobStub(JobId::IllegalJob));
    128128  anothertestJob->setId((JobId_t)2);
    129   anothertestJob->outputfile = std::string("do something else");
    130129
    131130  // prepare a vector of them
    … …  
    173172{
    174173  // prepare a job
    175   FragmentJob::ptr testJob(new FragmentJob);
     174  FragmentJob::ptr testJob(new FragmentJobStub(JobId::IllegalJob));
    176175  testJob->setId(1);
    177   testJob->outputfile = std::string("do something");
    178176#ifndef NDEBUG
    179177  CPPUNIT_ASSERT_NO_THROW( queue->pushJob(testJob) );
    … …  
    240238{
    241239  // prepare a job
    242   FragmentJob::ptr testJob( new FragmentJob );
    243   testJob->setId(1);
    244   testJob->outputfile = std::string("do something");
    245   FragmentJob::ptr anothertestJob( new FragmentJob );
    246   anothertestJob->setId(2);
    247   anothertestJob->outputfile = std::string("do something else");
     240  FragmentJob::ptr testJob( new FragmentJobStub(JobId::IllegalJob) );
     241  testJob->setId((JobId_t)1);
     242  FragmentJob::ptr anothertestJob( new FragmentJobStub(JobId::IllegalJob) );
     243  anothertestJob->setId((JobId_t)2);
    248244
    249245#ifndef NDEBUG

  • src/Fragmentation/Automation/unittests/Makefile.am

    r7670865 rd920b9  
    55  unittests/FragmentJobUnitTest.cpp \
    66  unittests/FragmentResultUnitTest.cpp \
    7   unittests/FragmentQueueUnitTest.cpp
     7  unittests/FragmentQueueUnitTest.cpp \
     8  unittests/SystemCommandJobUnitTest.cpp
    89
    910FRAGMENTATIONAUTOMATIONTESTSHEADERS = \
    1011  unittests/FragmentJobUnitTest.hpp \
    1112  unittests/FragmentResultUnitTest.hpp \
    12   unittests/FragmentQueueUnitTest.hpp
     13  unittests/FragmentQueueUnitTest.hpp \
     14  unittests/SystemCommandJobUnitTest.hpp
    1315
    1416FRAGMENTATIONAUTOMATIONTESTS = \
    1517  FragmentJobUnitTest \
    1618  FragmentResultUnitTest \
    17   FragmentQueueUnitTest
     19  FragmentQueueUnitTest \
     20  SystemCommandJobUnitTest
    1821
    1922
    … …  
    3538  unittests/FragmentJobUnitTest.hpp \
    3639  unittests/stubs/FragmentResultStub.cpp \
     40  unittests/stubs/FragmentJobStub.cpp \
     41  unittests/stubs/FragmentJobStub.hpp \
    3742  JobId.cpp \
    3843  JobId.hpp \
    … …  
    4954FragmentQueueUnitTest_SOURCES = $(top_srcdir)/src/unittests/UnitTestMain.cpp \
    5055  unittests/FragmentQueueUnitTest.cpp \
    51   unittests/FragmentQueueUnitTest.hpp
     56  unittests/FragmentQueueUnitTest.hpp \
     57  unittests/stubs/FragmentJobStub.cpp \
     58  unittests/stubs/FragmentJobStub.hpp
    5259FragmentQueueUnitTest_LDADD = \
    5360  libMolecuilderFragmentJobs.la \
    … …  
    6774  ${CodePatterns_LIBS}
    6875
     76SystemCommandJobUnitTest_LDFLAGS = ${CodePatterns_LIBS} $(CPPUNIT_LIBS) -ldl
     77SystemCommandJobUnitTest_CPPFLAGS = ${BOOST_CPPFLAGS} $(CPPUNIT_CFLAGS) ${CodePatterns_CFLAGS}
     78SystemCommandJobUnitTest_SOURCES = $(top_srcdir)/src/unittests/UnitTestMain.cpp \
     79  unittests/SystemCommandJobUnitTest.cpp \
     80  unittests/SystemCommandJobUnitTest.hpp \
     81  Results/FragmentResult.cpp \
     82  Results/FragmentResult.hpp \
     83  Jobs/FragmentJob.cpp \
     84  Jobs/FragmentJob.hpp \
     85  Jobs/SystemCommandJob.cpp \
     86  Jobs/SystemCommandJob.hpp \
     87  JobId.cpp \
     88  JobId.hpp
     89SystemCommandJobUnitTest_LDADD = \
     90  $(BOOST_FILESYSTEM_LDFLAGS) $(BOOST_FILESYSTEM_LIBS) \
     91  $(BOOST_IOSTREAMS_LDFLAGS) $(BOOST_IOSTREAMS_LIBS) \
     92  $(BOOST_SERIALIZATION_LDFLAGS) $(BOOST_SERIALIZATION_LIBS) \
     93  $(BOOST_SYSTEM_LDFLAGS) $(BOOST_SYSTEM_LIBS) \
     94  ${CodePatterns_LIBS}
    6995
    7096#AUTOMAKE_OPTIONS = parallel-tests
Note: See TracChangeset for help on using the changeset viewer.