Adding a Task

First you will need to understand the dependencies between tasks using the file dependency_graph.dot generated by swift at the beginning of any simulation and then decide where it will fit (see Task System).

For the next paragraphs, let’s assume that we want to implement the existing task cooling.

Adding it to the Task List

First you will need to add it to the task list situated in task.h and task.c.

In task.h, you need to provide an additional entry to the enum task_types (e.g. task_type_cooling). The last entry task_type_count should always stay at the end as it is a counter of the number of elements. For example:

enum task_types {
  task_type_none = 0,
  task_type_sort,
  task_type_self,
  task_type_pair,
  task_type_sub_self,
  task_type_sub_pair,
  task_type_ghost_in,
  task_type_ghost,
  task_type_ghost_out,
  task_type_extra_ghost,
  task_type_drift_part,
  task_type_end_force,
  task_type_kick1,
  task_type_kick2,
  task_type_timestep,
  task_type_send,
  task_type_recv,
  task_type_cooling,
  task_type_count
} __attribute__((packed));

In task.c, you will find an array containing the name of each task and need to add your own (e.g. cooling). Be careful with the order that should be the same than in the previous list. For example:

/* Task type names. */
const char *taskID_names[task_type_count] = {
  "none",          "sort",       "self",        "pair",      "sub_self",
  "sub_pair",      "ghost_in",   "ghost",     "ghost_out",
  "extra_ghost",   "drift_part", "end_force", "kick1",
  "kick2",         "timestep",   "send",        "recv",
  "cooling"};

Adding it to the Cells

Each cell contains a list to its tasks and therefore you need to provide a link for it.

In cell.h, add a pointer to a task in the structure. In order to stay clean, please put the new task in the same group than the other tasks. For example:

struct cell {
  /* Lot of stuff before. */

  /*! Task for the cooling */
  struct task *cooling;

  /*! The second kick task */
  struct task *kick2;

  /* Lot of stuff after */
}

Adding a new Timer

As SWIFT is HPC oriented, any new task need to be optimized. It cannot be done without timing the function.

In timers.h, you will find an enum that contains all the tasks. You will need to add yours inside it. For example:

enum {
  timer_none = 0,
  timer_prepare,
  timer_init,
  timer_drift_part,
  timer_drift_gpart,
  timer_kick1,
  timer_kick2,
  timer_timestep,
  timer_endforce,
  timer_dosort,
  timer_doself_density,
  timer_doself_gradient,
  timer_doself_force,
  timer_dopair_density,
  timer_dopair_gradient,
  timer_dopair_force,
  timer_dosub_self_density,
  timer_dosub_self_gradient,
  timer_dosub_self_force,
  timer_dosub_pair_density,
  timer_dosub_pair_gradient,
  timer_dosub_pair_force,
  timer_doself_subset,
  timer_dopair_subset,
  timer_dopair_subset_naive,
  timer_dosub_subset,
  timer_do_ghost,
  timer_do_extra_ghost,
  timer_dorecv_part,
  timer_do_cooling,
  timer_gettask,
  timer_qget,
  timer_qsteal,
  timer_locktree,
  timer_runners,
  timer_step,
  timer_cooling,
  timer_count,
};

As for task.h, you will need to give a name to your timer in timers.c:

const char* timers_names[timer_count] = {
  "none",
  "prepare",
  "init",
  "drift_part",
  "kick1",
  "kick2",
  "timestep",
  "endforce",
  "dosort",
  "doself_density",
  "doself_gradient",
  "doself_force",
  "dopair_density",
  "dopair_gradient",
  "dopair_force",
  "dosub_self_density",
  "dosub_self_gradient",
  "dosub_self_force",
  "dosub_pair_density",
  "dosub_pair_gradient",
  "dosub_pair_force",
  "doself_subset",
  "dopair_subset",
  "dopair_subset_naive",
  "dosub_subset",
  "do_ghost",
  "do_extra_ghost",
  "dorecv_part",
  "gettask",
  "qget",
  "qsteal",
  "locktree",
  "runners",
  "step",
  "cooling",
};

You can now easily time your functions by using:

TIMER_TIC;
/* Your complicated functions */
if (timer) TIMER_TOC(timer_cooling);

Adding your Task to the System

Now the tricky part happens. SWIFT is able to deal automatically with the conflicts between tasks, but unfortunately cannot understand the dependencies.

To implement your new task in the task system, you will need to modify a few functions in engine_maketasks.c.

First, you will need to add mainly two functions: scheduler_addtask and scheduler_addunlocks in the engine_make_hierarchical_tasks_* functions (depending on the type of task you implement, you will need to write it to a different function).

In engine_make_hierarchical_tasks_hydro, we add the task through the following call:

/* Add the cooling task. */
c->cooling =
scheduler_addtask(s, task_type_cooling, task_subtype_none, 0,
                  0, c, NULL);

As the cooling cannot be done before the end of the force computation and the second kick cannot be done before the cooling:

scheduler_addunlock(s, c->super->end_force, c->cooling);
scheduler_addunlock(s, c->cooling, c->super->kick2);

The next step is to activate your task in engine_marktasks_mapper in engine_marktasks.c:

else if (t->type == task_type_cooling || t->type == task_type_sourceterms) {
  if (cell_is_active_hydro(t->ci, e)) scheduler_activate(s, t);
}

Then you will need to update the estimate for the number of tasks in engine_estimate_nr_tasks in engine.c by modifying n1 or n2, and give the task an estimate of the computational cost that it will have in scheduler_reweight in scheduler.c:

case task_type_cooling:
  cost = wscale * count_i;
  break;

Initially, the engine will need to skip the task that updates the particles. It is the case for the cooling, therefore you will need to add it in engine_skip_force_and_kick.

Finally, you also need to initialize your new variables and pointers in space_rebuild_recycle_mapper in space.c.

Implementing your Task

The last part is situated in runner_main.c.

You will need to implement a function runner_do_cooling (do not forget to time it):

void runner_do_cooling(struct runner *r, struct cell *c, int timer) {

  TIMER_TIC;

  /* Now you can check if something is required at this time step.
   * You may want to use a different cell_is_active function depending
   * on your task
   */
  if (!cell_is_active_hydro(c, e)) return;

  /* Recurse? */
  if (c->split) {
    for (int k = 0; k < 8; k++)
      if (c->progeny[k] != NULL) runner_do_cooling(r, c->progeny[k], 0);
  } else {
    /* Implement your cooling here */
  }

  if (timer) TIMER_TOC(timer_do_cooling);
}

and add a call to this function in runner_main in the switch:

case task_type_cooling:
  runner_do_cooling(r, t->ci, 1);
  break;

Adding your task to the analysis tools

To produce the task graphs, the analysis scripts need to know about the new task. You will need to edit the python script in the tools/task_plots directory. At the start of each of analyse_tasks.py, plot_tasks.py and iplot_tasks.py you will find a long list of task names. You will need to add the name of the new task to that list. The order of this list needs to be the same as the enum type in the task.h file!

Finalizing your Task

Now that you have done the easiest part, you can start debugging by implementing a test and/or an example. Before creating your merge request with your new task, do not forget the most funny part that consists in writing a nice and beautiful documentation ;)