Generic Scheduler

From Daxtoolkit
Jump to: navigation, search
I apparently forgot I had written this design document and ended up creating another one named Simplified Scheduler Objects. The design is similar but this latter one is based on some more recently added features of Dax. --Kenneth Moreland 11:28, 22 October 2013 (EDT)

The most recent changes to the control environment scheduler classes (SHA 8965a31b8d5e25008caa389c03d9e20779fcc7c2 currently on the scheduler_infastructure branch but soon to be merged into master) make the scheduler code much easier to read, understand, and extend. Of course, I'm a creep who takes every cool new thing and then asks for something way above and beyond that. In this case, I'm asking for an even easier mechanism to implement new schedulers.

Current Implementation and Motivation

The current implementation has multiple Scheduler* classes in dax::cont::scheduling, each designed to schedule worklets in a different way (and generally tied to one or more base worklet classes) and each is basically independent of the others. The expected interface for one of these Scheduler* classes is simply a single templated function called Invoke.

Of course, "simply" is probably the wrong word here. The Invoke method is required to be a templated method that takes a variable number of template arguments. If you happen to have a C++11 compliant compiler, and it is currently too early to assume that, then the implementation is not too bad although not many developers (including myself) are familiar with this new syntax. Otherwise, you have to use this recursive include that is wholly confusing even when using Boost to manage it.

So, my desire is to encapsulate the methods requiring variable length template arguments in such a way as to allow new Schedulers to be created using only standard template parameters.

Proposed Solution

A word of warning before we go on, my proposed solution involves C++ constructs that I have little to no experience in, so it is quite conceivable I will propose things that are not practical. In that case please add discussion comments to this page. --Kenneth Moreland 11:54, 7 December 2012 (EST)

Base Scheduler Class

The proposed solution starts with defining a base class for all schedulers, called SchedulerBase or SchedulerGeneric or something like that. I'll call it SchedulerBase for now but feel free to global find/replace later.

SchedulerBase follows the curiously recurring template pattern. It takes three template arguments: the derived class, the number of arguments the derived class expects, and the device adapter tag. As we shall see in a moment, the number of arguments is used by the derived class to accept certain arguments of known types that are needed to set up the schedule (e.g. the input and output grids for the schedule generate topology).

The SchedulerBase code would look more or less like this. For simplicity, I will use (and abuse) the C++11 variable template argument syntax and leave the actual implementation up to the reader.

template<class DerivedScheduler, int NumSchedulerArguments, class DeviceAdapterTag>
class SchedulerBase
{
public:
  template<class WorkletType, typename...T>
  DAX_CONT_EXPORT void Invoke(const WorkletType &worklet, T...arguments) const
  {
    // Compile check to make sure that the length of arguments matches the ControlSignature.

    // Compile check to make sure sizeof...(T) is at least as large as NumSchedulerArguments.

    // Pack the arguments into something like a boost tuple.  The actual type does not matter
    // as other classes shouldn't care about the type.
    boost::tuple<T...> packedArguments(arguments...);

    // Invoke the derived class with the requested number of arguments.  Not entirely sure
    // how to do this (or even if it is feasible).  It would probably actually require calling
    // some other recursively called function.
    static_cast<DerivedScheduler*>(this)->InvokeImpl(argument:1,
                                                     argument:2,
                                                     ..
                                                     argument:NumSchedulerArguments,
                                                     packedArguments);
  }
protected:
  template <class WorkletType, class ArgumentsType>
  DAX_CONT_EXPORT void BasicInvoke(const WorkletType &w, const ArgumentsType &arguments) const
  {
    // Basically the same implementation that is currently in the default scheduler's Invoke.
  }
};

Concrete Scheduler Derived Classes

The concrete schedulers derive from SchedulerBase. They implement an InvokeImpl method that takes the requested number of arguments, do whatever pre and post processing is necessary, and call BasicInvoke to do the actual worklet call in the execution environment.

To start, the implementation of SchedulerDefault is quite simple.

template <class DeviceAdapterTag>
class Scheduler<DeviceAdapterTag,dax::cont::scheduling::ScheduleDefaultTag>
  : public dax::cont::scheduling::SchedulerBase<
      Scheduler<DeviceAdapterTag,dax::cont::scheduling::ScheduleDefaultTag>, 0, DeviceAdapterTag>
{
public:
  template <class WorkletType, class ArgumentsType>
  DAX_CONT_EXPORT void Invoke(WorkletType worklet, const ArgumentsType &arguments) const
  {
    this->BasicInvoke(worklet, arguments);
  }
};

The implementation for SchedulerGenerateTopology is a bit more interesting. It looks at the first two arguments and does interesting stuff with them before the actual invoke.

template <class DeviceAdapterTag>
class Scheduler<DeviceAdapterTag,dax::cont::scheduling::ScheduleGenerateTopologyTag>
  : public dax::cont::scheduling::SchedulerBase<
      Scheduler<DeviceAdapterTag,dax::cont::scheduling::ScheduleGenerateTopologyTag>,
      2,
      DeviceAdapterTag>
{
public:
  template <class WorkletType, class InputGrid, class OutputGrid, class ArgumentsType>
  DAX_CONT_EXPORT void InvokeImpl(WorkletType worklet,
                                  const InputGrid &inputGrid,
                                  OutputGrid &outputGrid,
                                  const ArgumentsType &arguments) const
  {
  // Do the same processing as in the current SchedulerGenerateTopology.h to create
  // validCellRange and visitIndex.

  this->BasicInvoke(...);

  // Likewise, do the appropriate remove duplicate points code.
  }
};

Alternate Solution: Argument List to Argument Container

As I was writing the previous sections, I started thinking of problems with the approach. The first is that I am unsure how easy or possible it is to extract some number of arguments and then call a function with those arguments. More importantly, however, I am concerned that this approach is going to constrain the derived schedulers such that, for example, the generate topology scheduler can no longer operate.

The problem I see is that the generate topology schedule has to modify the parameter arguments before calling the basic invoke. It has to establish a mapping for the cells of the input grid and it has to add a visitIndex array. It is not clear how to do this, especially the adding of a visitIndex, if the arguments are wrapped in an opaque type.

So, in response to myself, I propose an alternate solution that should both simplify the implementation of SchedulerBase and expand the capabilities of derived classes. The basic idea is to make a less opaque container for arguments.

Argument Container

The argument container is a class that has identical functionality to the Boost tuple class. In fact, the implementation can be Boost tuple, but the name has to be different because, first, we hide any references to Boost in our API and, second, the name is easily confused with the dax::Tuple.

From here on I will refer to this class as dax::cont::scheduling::Arguments. However, I am not married to this name or namespace. This could be a generally useful class and might be put elsewhere if we deem it useful.

Base Scheduler Class

The base scheduler class is the same idea as before with the exception that we do not attempt to extract some arguments and package others. Instead, we simply package all the arguments and forward them on to the derived class's InvokeImpl. Even the template parameter arguments become simpler because we don't need to know the number of arguments the derived class is going to look at.

template<class DerivedScheduler, int NumSchedulerArguments, class DeviceAdapterTag>
class SchedulerBase
{
public:
  template<class WorkletType, typename...T>
  DAX_CONT_EXPORT void Invoke(const WorkletType &worklet, T...arguments) const
  {
    // Compile check to make sure that the length of arguments matches the ControlSignature.
 
    // Compile check to make sure sizeof...(T) is at least as large as NumSchedulerArguments.
 
    // Pack the arguments.
    dax::cont::scheduling::Arguments<T...> packedArguments(arguments...);
 
    // Invoke the derived class.
    static_cast<DerivedScheduler*>(this)->InvokeImpl(worklet, packedArguments);
  }
protected:
  template <class WorkletType, typename...T>
  DAX_CONT_EXPORT void BasicInvoke(const WorkletType &worklet,
                                   const dax::cont::scheduling::Arguments<T...> &arguments) const
  {
    // Basically the same implementation that is currently in the default scheduler's Invoke.
  }
};

Concrete Scheduler Derived Classes

The derived class is much the same as the previous design except that now we have more control over the arguments now that they are no longer in an opaque type. The AddVisitIndexArg class can help modify the argument list to add the visitIndex.

(Note, this requires manipulations of arguments list type to support adding and changing types in the "tuple." There may need to be some helper classes to do this as well as some special structure of the dax::cont::scheduling::Arguments class; perhaps more like a Boost cons.)

template <class DeviceAdapterTag>
class Scheduler<DeviceAdapterTag,dax::cont::scheduling::ScheduleGenerateTopologyTag>
  : public dax::cont::scheduling::SchedulerBase<
      Scheduler<DeviceAdapterTag,dax::cont::scheduling::ScheduleGenerateTopologyTag>,
      DeviceAdapterTag>
{
public:
  template <class WorkletType, class ArgumentsType>
  DAX_CONT_EXPORT void InvokeImpl(WorkletType worklet,
                                  const ArgumentsType &arguments) const
  {
    this->GenerateNewTopology(worklet, arguments.Get<0>(), arguments.Get<1>(), arguments);
  }

private:
  template <class WorkletType, class InputGrid, class OutputGrid, class ArgumentsType>
  DAX_CONT_EXPORT void InvokeImpl(
    dax::cont::ScheduleGenerateTopology<WorkletType,DeviceAdapterTag> worklet,
    const InputGrid &inputGrid,
    OutputGrid &outputGrid,
    const ArgumentsType &arguments) const
  {
  // Do the same processing as in the current SchedulerGenerateTopology.h to create validCellRange.

  // Same code for adding visit index except it also modifies our arguments.
  typedef dax::cont::scheduling::AddVisitIndexArg<WorkletType,
    Algorithm,IdArrayHandleType,ArgumentsType> AddVisitIndexFunctor;
  typedef typename AddVisitIndexFunctor::VisitIndexArgType IndexArgType;
  typedef typename AddVisitIndexFunctor::DerivedWorkletType DerivedWorkletType;
  typedef typename AddVisitIndexFunctor::DerivedArgumentsType ArgumentsWithVisit;

  ArgumentsWithVisit argumentsVisit;
  AddVisitIndexFunctor createVisitIndex;
  createVisitIndex(this->DefaultScheduler,validCellRange,arguments,argumentsVisit);

  DerivedWorkletType derivedWorklet(newTopo.GetWorklet());

  // We also need to do some argument mangling to replace the first argument (which is inputGrid)
  // with whatever class represents the mapped input cells.  We'll just call that
  // argumentsVisitMapped.
 
  this->BasicInvoke(derivedWorklet, argumentsVisitMapped);
 
  // Do the appropriate remove duplicate points code.
  }
};

Acknowledgements

Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-AC04-94AL85000.

SandiaLogo.png DOELogo.png