Dax TCon 2012-12-06
The meeting started with a discussion between Ken and Robert about Robert's scheduler_infrastructure branch. The intention of the changes is to make the code easier to change (as we need to add more features in the future) and to make the code easier to understand. On review, Ken wholeheartedly agreed with the later point. The code is much easier to understand than what is currently in master, and Ken suggested that, although later discussion proposed some changes, the current code is a big enough improvement to warrant being merged into master now.
One issue that Ken brought up (or rather, one perpetual issue that has been discussed multiple times) is the fact that for some basic worklet types, like field map and cell map, you just feed the instantiation of the worklet to Scheduler::Invoke whereas for other worklet types, like generate topology, you have to wrap the worklet instantiation in another control environment class before feeding to Invoke. As discussed previously, this is a necessary evil because scheduling topology generation involves some control environment manipulation for scheduling parameters and post-execution state and operations. Rather than shove all this into the execution environment or create lots of independent schedulers, we use this convention of the control-side wrapper object.
None of this is new, but Ken observed that it was easy (too easy) to couple the worklet with the wrong control-side wrapper object (or non at all when it needed one). When this happens, the scheduler happily invokes the worklet with the wrong scheduling convention, which may lead to bizarre compile or runtime errors and most certainly will produce the wrong results. Because there is no foreseeable use case for using a worklet with any scheduler mechanism other than what was intended for its base class, Ken suggested that there be a compile time check that issues an error if there is a mismatch between the worklet base type and the appropriate control Schedule* wrapper. The idea was that the SchedulerTags should be moved to the execution environment (or perhaps the shared environment if that makes more sense) and each base worklet class would contain a typedef on which scheduler is expected. That would significantly clean up the DetermineScheduler code and also make it straightforward to check the type of the class passed to Scheduler::Invoke. When a mismatch is found, the error should be a clear message explaining the problem and giving clear instructions on how to fix it (e.g. pass a ScheduleGenerateTopology containing your worklet to Invoke).
Ken also brought up that SchedulerGenerateTopology expects the first two arguments to the worklet to be the input grid and the output grid, respectively, but there is no check for that. Specifically, there is no check to make sure that the ControlSignature has Topology and Topology(Out) as the first two arguments. There should be a check somewhere to ensure that the first two arguments are of the correct type and to provide a descriptive error if they are not.
Finally, Ken suggested that it would be nice if it was easier to create new Schedule classes (for example, a new schedule class that reduces some data on a key or vertex). He postulated on a way to implement a Generic Scheduler so that new schedulers can be created without the variable template arguments.
This week Ken (almost) completed work on the CellTag branch implemented Breaking Up the Cell Class. Most of the discussion was on the changes to classes in dax/exec/arg, which Ken is not that familiar with. To implement these changes, Ken resurrected the behavior of the BindCellPointIds with its Topology::PointIds modifier for ExecutionSignature arguments. A problem he noticed was that with this syntax it is not possible to declare a worklet operator to return the cell vertices because Topology::PointIds(*) cannot be used in the return type of the ExecutionSignature. Robert suggested a way around it might be to make the declaration for a template (i.e. Topology::PointIds<*>), but it could be a lot of work and probably not worth it.
A bigger problem with the CellTag branch is that topology generation does not work. The issue is that in a topology generation schedule, there is a map from output cell index to input cell index which is handled through a FieldMap wrapper. The FieldMap wrapper expects all invocations to be through operator(), but for the case of point indices it is not. The appropriate way to solve this problem is probably to use the permutation array portal that Nikhil is working on to implicitly map the indices when they are loaded. Then all the FieldMap classes can just go away. Robert said he would take a look at the CellTag branch.
Finally, Bob talked about a project he just started on to apply Dax to an application problem. The task is a feature tracking technique that, starting with 2D images but later moving to 3d uniform grids, would perform on operation on individual cells to classify what kind of group the flow through that cell is. In a secondary step a fuzzy search is performed to find the features. The first step is implemented with a Dax map field. The second step should be able to be implemented with a reduce on point or reduce on key feature which Dax currently does not have but needs anyway. The specific application domain Bob is currently working with is a combustion, but the technique should work reasonably well for any vector field.