DeviceAdapter Dependency Issues

From Daxtoolkit
Jump to: navigation, search

The current version of the DeviceAdapter has a few odd but intentional quirks that are causing problems with header file dependencies. The first of these is the way in which the default DeviceAdapter is specified. The current setup is that when a specific device adapter is included (like DeviceAdapterOpenMP.h), that header file sets a macro to make it the default. However, this means that the device adapter header must be included before any other header to make sure that the correct default is specified before dependent code is declared. But then this can cause problems if secondary device adapters are to be declared but not made the default.

Another problem is with the interdependency of ArrayHandle and a DeviceAdapter. An ArrayHandle needs facilities in the DeviceAdapter to manage data between the control and execution environments. But many of the DeviceAdapter algorithms operate on ArrayHandle objects. Apart from just being icky (technical term), this is requiring some preprocessor gymnastics to make things declared in the correct order.

Defining the Default DeviceAdapter

The current method of defining the default device adapter, by including its header file, is problematic as described above. Instead, I propose we move to a simpler method of setting a macro specifying the default device adapter in the same way Thrust uses a macro to specify the default device/back end.

#define DAX_DEVICE_ADAPTER_ERROR     -1
#define DAX_DEVICE_ADAPTER_UNDEFINED  0
#define DAX_DEVICE_ADAPTER_SERIAL     1
#define DAX_DEVICE_ADAPTER_CUDA       2
#define DAX_DEVICE_ADAPTER_OPENMP     3

#ifndef DAX_DEVICE_ADAPTER
#ifdef DAX_CUDA
#define DAX_DEVICE_ADAPTER DAX_DEVICE_ADAPTER_CUDA
#elif defined(DAX_OPENMP) // !DAX_CUDA
#define DAX_DEVICE_ADAPTER DAX_DEVICE_ADAPTER_OPENMP
#else // !DAX_CUDA && !DAX_OPENMP
#define DAX_DEVICE_ADAPTER DAX_DEVICE_ADAPTER_SERIAL
#endif // DAX_DEVICE_ADAPTER

Once the DAX_DEVICE_ADAPTER macro is defined, we can use trivial preprocessing to define and load the appropriate default interface. The foremost feature we need is a tag object to use as the default template parameter. We can do this by setting another macro.

Thrust uses a typedef for the tag instead. Should we adopt that? The advantages are slightly earlier compiler typechecking (meh) and less potential confusion between macros DAX_DEVICE_ADAPTER and DAX_DEFAULT_DEVICE_ADAPTER_TAG. The advantage of the macro is a few preprocessor tricks. --Kenneth Moreland 17:35, 18 July 2012 (EDT)
#if DAX_DEVICE_ADAPTER == DAX_DEVICE_ADAPTER_SERIAL

#include <dax/cont/internal/DeviceAdapterTagSerial.h>
#define DAX_DEFAULT_DEVICE_ADAPTER_TAG ::dax::cont::DeviceAdapterTagSerial

#elif DAX_DEVICE_ADAPTER == DAX_DEVICE_ADAPTER_CUDA

#include <dax/cuda/cont/internal/DeviceAdapterTagCuda.h>
#define DAX_DEFAULT_DEVICE_ADAPTER_TAG ::dax::cuda::cont::DeviceAdapterTagCuda

#elif DAX_DEVICE_ADAPTER == DAX_DEVICE_ADAPTER_OPENMP

#include <dax/openmp/cont/internal/DeviceAdapterTagOpenMP.h>
#define DAX_DEFAULT_DEVICE_ADAPTER_TAG ::dax::openmp::cont::DeviceAdapterTagOpenMP

#elif DAX_DEVICE_ADAPTER == DAX_DEVICE_ADAPTER_ERROR

#include <dax/cont/internal/DeviceAdapterTagError.h>
#define DAX_DEFAULT_DEVICE_ADAPTER_TAG ::dax::cont::internal::DeviceAdapterTagError

#elif (DAX_DEVICE_ADAPTER == DAX_DEVICE_ADAPTER_UNDEFINED) || !defined(DAX_DEVICE_ADAPTER)

#ifndef DAX_DEFAULT_DEVICE_ADAPTER_TAG
#warning If device adapter is undefined, DAX_DEFAULT_DEVICE_ADAPTER_TAG must be defined.

#endif

The includes are described in the following section.

The ArrayContainerControl classes should also be moved to a similar default mechanism.

DeviceAdapter and ArrayHandle Interdependency

As mentioned previously, there is a chicken-or-egg problem between the DeviceAdapter and ArrayHandle which mutually depend on each other. However, thanks to the design with Flattening Template Parameters with Tags, the DeviceAdapter is no longer a monolithic class. Instead, it is a collection of classes and functions that are templated or overloaded by a tag. Thus, we can think of the DeviceAdapter as at least three distinct pieces: a tag, an ArrayManagerExecution object, and a collection of base algorithms. The interdependency with these, the ArrayHandle, the default device, and the ArrayContainerControl (which here is treated as a single unit) is as follows.

This is a graph with borders and nodes. Maybe there is an Imagemap used so the nodes may be linking to some Pages.

In this graph the dependency of ArrayHandle on an ArrayManagerExecution is illustrated as a dotted line because technically the ArrayHandle does not need any concrete implementation of ArrayManagerExecution. However, logically this dependency exists and it can cause headaches if this connection created a circular dependency (which it does not).

Now that we know the dependencies, we can design a set of header files with non-circular dependencies. First, we define a set of three header files in the internal directory: DeviceAdapterTag.h, ArrayManagerExecution.h, and DeviceAdapterAlgorithm.h. Every implementation of a DeviceAdapter implements its own version of each of these headers to define the specific object of that type. The generic versions of each of these include the default implementation. Finally, there remains the DeviceAdapter.h in the cont directory along with any implementation versions, but now they just include the appropriate headers in the internal directory.

This is a graph with borders and nodes. Maybe there is an Imagemap used so the nodes may be linking to some Pages.

Although this graph looks confusing, the connections are actually fairly regular. Notice that the connections generally go from left to right and then top to bottom. The one exception is the dependency of DeviceAdapterAlgorithmX back to ArrayHandle. However, this back connection does not cause a cycle because ArrayHandle makes sure it does not depend on any algorithms. Careful examination reveals that no cycles exist.

As I implement this, I realize that the ArrayManagerExecution.h and ArrayManagerExecutionX.h need to include each other (and for completeness the tag and algorithm headers do the same). Although this is itself a dependency loop, it is easily managed and does not introduce any further loops. It itself could be broken by replicating the ArrayManagerExecution template prototype in all the implementations, but I find the repeated prototypes just as bad as the circular dependency. --Kenneth Moreland 18:54, 8 August 2012 (EDT)

Acknowledgements

Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-AC04-94AL85000.

SandiaLogo.png DOELogo.png

SAND 2012-5991P