<?xml version='1.0' encoding='UTF-8' standalone='no'?>
<doxygen xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="compound.xsd" version="1.9.6" xml:lang="en-US">
  <compounddef id="DependentAsyncTasking" kind="page">
    <compoundname>DependentAsyncTasking</compoundname>
    <title>Asynchronous Tasking with Dependencies</title>
    <tableofcontents>
      <tocsect>
        <name>Create a Dynamic Task Graph</name>
        <reference>DependentAsyncTasking_1CreateADynamicTaskGraph</reference>
    </tocsect>
      <tocsect>
        <name>Specify a Range of Dependent Tasks</name>
        <reference>DependentAsyncTasking_1SpecifyARagneOfDependentTasks</reference>
    </tocsect>
      <tocsect>
        <name>Understand the Lifetime of a Dependent Async Task</name>
        <reference>DependentAsyncTasking_1UnderstandTheLifeTimeOfADependentAsyncTask</reference>
    </tocsect>
      <tocsect>
        <name>Create a Dynamic Task Graph by Multiple Threads</name>
        <reference>DependentAsyncTasking_1CreateADynamicTaskGraphByMultipleThreads</reference>
    </tocsect>
    </tableofcontents>
    <briefdescription>
    </briefdescription>
    <detaileddescription>
<para>This chapters discusses how to create a task graph dynamically using asynchronous tasks, which is extremely beneficial for workloads that want to (1) explore task graph parallelism out of dynamic control flow or (2) overlap task graph creation time with individual task execution time. We recommend that you first read <ref refid="AsyncTasking" kindref="compound">Asynchronous Tasking</ref> before digesting this chapter.</para>
<sect1 id="DependentAsyncTasking_1CreateADynamicTaskGraph">
<title>Create a Dynamic Task Graph</title>
<para>When the construct-and-run model of a task graph is not possible in your application, you can use <ref refid="classtf_1_1Executor_1aee02b63d3a91ad5ca5a1c0e71f3e128f" kindref="member">tf::Executor::dependent_async</ref> and <ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">tf::Executor::silent_dependent_async</ref> to create a task graph dynamically. This type of parallelism is also known as <emphasis>on-the-fly</emphasis> task graph parallelism, which offers great flexibility for expressing dynamic task graph parallelism. The example below dynamically creates a task graph of four dependent async tasks, <computeroutput>A</computeroutput>, <computeroutput>B</computeroutput>, <computeroutput>C</computeroutput>, and <computeroutput>D</computeroutput>, where <computeroutput>A</computeroutput> runs before <computeroutput>B</computeroutput> and <computeroutput>C</computeroutput> and <computeroutput>D</computeroutput> runs after <computeroutput>B</computeroutput> and <computeroutput>C:</computeroutput> </para>
<para><dotfile name="simple.dot"></dotfile>
</para>
<para><programlisting filename=".cpp"><codeline><highlight class="normal"><ref refid="classtf_1_1Executor" kindref="compound">tf::Executor</ref><sp/>executor;</highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>A<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){<sp/>printf(</highlight><highlight class="stringliteral">&quot;A\n&quot;</highlight><highlight class="normal">);<sp/>});</highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>B<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){<sp/>printf(</highlight><highlight class="stringliteral">&quot;B\n&quot;</highlight><highlight class="normal">);<sp/>},<sp/>A);</highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>C<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){<sp/><ref refid="cpp/io/c/fprintf" kindref="compound" external="/Users/twhuang/Code/taskflow/doxygen/cppreference-doxygen-web.tag.xml">printf</ref>(</highlight><highlight class="stringliteral">&quot;C\n&quot;</highlight><highlight class="normal">);<sp/>},<sp/>A);</highlight></codeline>
<codeline><highlight class="normal"></highlight><highlight class="keyword">auto</highlight><highlight class="normal"><sp/>[D,<sp/>fuD]<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1aee02b63d3a91ad5ca5a1c0e71f3e128f" kindref="member">dependent_async</ref>([](){<sp/><ref refid="cpp/io/c/fprintf" kindref="compound" external="/Users/twhuang/Code/taskflow/doxygen/cppreference-doxygen-web.tag.xml">printf</ref>(</highlight><highlight class="stringliteral">&quot;D\n&quot;</highlight><highlight class="normal">);<sp/>},<sp/>B,<sp/>C);</highlight></codeline>
<codeline><highlight class="normal">D.get();<sp/><sp/></highlight><highlight class="comment">//<sp/>wait<sp/>for<sp/>D<sp/>to<sp/>finish,<sp/>which<sp/>in<sp/>turns<sp/>means<sp/>A,<sp/>B,<sp/>C<sp/>finish</highlight></codeline>
</programlisting></para>
<para>Both <ref refid="classtf_1_1Executor_1aee02b63d3a91ad5ca5a1c0e71f3e128f" kindref="member">tf::Executor::dependent_async</ref> and <ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">tf::Executor::silent_dependent_async</ref> create a task of type <ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref> to run the given function asynchronously. Additionally, <ref refid="classtf_1_1Executor_1aee02b63d3a91ad5ca5a1c0e71f3e128f" kindref="member">tf::Executor::dependent_async</ref> returns a <ulink url="https://en.cppreference.com/w/cpp/thread/future">std::future</ulink> that eventually holds the result of the execution. When returning from both calls, the executor has scheduled a worker to run the task whenever its dependencies are met. That is, task execution happens <emphasis>simultaneously</emphasis> with the creation of the task graph, which is different from constructing a Taskflow and running it from an executor, illustrated in the figure below:</para>
<para><image type="html" name="dependent_async_execution_diagram.png"></image>
</para>
<para>Since this model only allows relating a dependency from the current task to a previously created task, you need a correct topological order of graph expression. In our example, there are only two possible topological orderings, either <computeroutput>ABCD</computeroutput> or <computeroutput>ACBD</computeroutput>. The code below shows another feasible order of expressing this dynamic task graph parallelism:</para>
<para><programlisting filename=".cpp"><codeline><highlight class="normal"><ref refid="classtf_1_1Executor" kindref="compound">tf::Executor</ref><sp/>executor;</highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>A<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){<sp/>printf(</highlight><highlight class="stringliteral">&quot;A\n&quot;</highlight><highlight class="normal">);<sp/>});</highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>C<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){<sp/>printf(</highlight><highlight class="stringliteral">&quot;C\n&quot;</highlight><highlight class="normal">);<sp/>},<sp/>A);</highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>B<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){<sp/><ref refid="cpp/io/c/fprintf" kindref="compound" external="/Users/twhuang/Code/taskflow/doxygen/cppreference-doxygen-web.tag.xml">printf</ref>(</highlight><highlight class="stringliteral">&quot;B\n&quot;</highlight><highlight class="normal">);<sp/>},<sp/>A);</highlight></codeline>
<codeline><highlight class="normal"></highlight><highlight class="keyword">auto</highlight><highlight class="normal"><sp/>[D,<sp/>fuD]<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1aee02b63d3a91ad5ca5a1c0e71f3e128f" kindref="member">dependent_async</ref>([](){<sp/><ref refid="cpp/io/c/fprintf" kindref="compound" external="/Users/twhuang/Code/taskflow/doxygen/cppreference-doxygen-web.tag.xml">printf</ref>(</highlight><highlight class="stringliteral">&quot;D\n&quot;</highlight><highlight class="normal">);<sp/>},<sp/>B,<sp/>C);</highlight></codeline>
<codeline><highlight class="normal">D.get();<sp/><sp/></highlight><highlight class="comment">//<sp/>wait<sp/>for<sp/>D<sp/>to<sp/>finish,<sp/>which<sp/>in<sp/>turns<sp/>means<sp/>A,<sp/>B,<sp/>C<sp/>finish</highlight></codeline>
</programlisting></para>
<para>In addition to using <ulink url="https://en.cppreference.com/w/cpp/thread/future">std::future</ulink> to synchronize the execution, you can use <ref refid="classtf_1_1Executor_1ab9aa252f70e9a40020a1e5a89d485b85" kindref="member">tf::Executor::wait_for_all</ref> to wait for all scheduled tasks to finish:</para>
<para><programlisting filename=".cpp"><codeline><highlight class="normal"><ref refid="classtf_1_1Executor" kindref="compound">tf::Executor</ref><sp/>executor;</highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>A<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){<sp/>printf(</highlight><highlight class="stringliteral">&quot;A\n&quot;</highlight><highlight class="normal">);<sp/>});</highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>B<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){<sp/>printf(</highlight><highlight class="stringliteral">&quot;B\n&quot;</highlight><highlight class="normal">);<sp/>},<sp/>A);</highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>C<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){<sp/><ref refid="cpp/io/c/fprintf" kindref="compound" external="/Users/twhuang/Code/taskflow/doxygen/cppreference-doxygen-web.tag.xml">printf</ref>(</highlight><highlight class="stringliteral">&quot;C\n&quot;</highlight><highlight class="normal">);<sp/>},<sp/>A);</highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>D<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){<sp/><ref refid="cpp/io/c/fprintf" kindref="compound" external="/Users/twhuang/Code/taskflow/doxygen/cppreference-doxygen-web.tag.xml">printf</ref>(</highlight><highlight class="stringliteral">&quot;D\n&quot;</highlight><highlight class="normal">);<sp/>},<sp/>B,<sp/>C);</highlight></codeline>
<codeline><highlight class="normal">executor.<ref refid="classtf_1_1Executor_1ab9aa252f70e9a40020a1e5a89d485b85" kindref="member">wait_for_all</ref>();</highlight></codeline>
</programlisting></para>
</sect1>
<sect1 id="DependentAsyncTasking_1SpecifyARagneOfDependentTasks">
<title>Specify a Range of Dependent Tasks</title>
<para>Both <ref refid="classtf_1_1Executor_1aee02b63d3a91ad5ca5a1c0e71f3e128f" kindref="member">tf::Executor::dependent_async(F&amp;&amp; func, Tasks&amp;&amp;... tasks)</ref> and <ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">tf::Executor::silent_dependent_async(F&amp;&amp; func, Tasks&amp;&amp;... tasks)</ref> accept an arbitrary number of tasks in the dependency list. If the number of dependent tasks is unknown at programming time, such as those relying on runtime variables, you can use the following two overloads to specify dependent tasks in an iterable range <computeroutput>[first, last)</computeroutput>:</para>
<para><itemizedlist>
<listitem><para><ref refid="classtf_1_1Executor_1a01e51e564f5def845506bcf6b4bb1664" kindref="member">tf::Executor::dependent_async(F&amp;&amp; func, I first, I last)</ref></para>
</listitem><listitem><para><ref refid="classtf_1_1Executor_1aa9b08e47e68ae1e568f18aa7104cb9b1" kindref="member">tf::Executor::silent_dependent_async(F&amp;&amp; func, I first, I last)</ref></para>
</listitem></itemizedlist>
</para>
<para>The code below creates an asynchronous task that depends on <computeroutput>N</computeroutput> previously created asynchronous tasks stored in a vector, where <computeroutput>N</computeroutput> is a runtime variable:</para>
<para><programlisting filename=".cpp"><codeline><highlight class="normal"><ref refid="classtf_1_1Executor" kindref="compound">tf::Executor</ref><sp/>executor;</highlight></codeline>
<codeline><highlight class="normal"><ref refid="cpp/container/vector" kindref="compound" external="/Users/twhuang/Code/taskflow/doxygen/cppreference-doxygen-web.tag.xml">std::vector&lt;tf::AsyncTask&gt;</ref><sp/>dependents;</highlight></codeline>
<codeline><highlight class="normal"></highlight><highlight class="keywordflow">for</highlight><highlight class="normal">(</highlight><highlight class="keywordtype">size_t</highlight><highlight class="normal"><sp/>i=0;<sp/>i&lt;N;<sp/>i++)<sp/>{<sp/><sp/></highlight><highlight class="comment">//<sp/>N<sp/>is<sp/>a<sp/>runtime<sp/>variable</highlight><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"><sp/><sp/>dependents.push_back(executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){}));</highlight></codeline>
<codeline><highlight class="normal">}</highlight></codeline>
<codeline><highlight class="normal">executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){},<sp/>dependents.begin(),<sp/>dependents.end());</highlight></codeline>
<codeline><highlight class="normal">executor.<ref refid="classtf_1_1Executor_1ab9aa252f70e9a40020a1e5a89d485b85" kindref="member">wait_for_all</ref>();</highlight></codeline>
</programlisting></para>
</sect1>
<sect1 id="DependentAsyncTasking_1UnderstandTheLifeTimeOfADependentAsyncTask">
<title>Understand the Lifetime of a Dependent Async Task</title>
<para>A <ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref> is a lightweight handle that retains <emphasis>shared</emphasis> ownership of a dependent async task created by an executor. This shared ownership ensures that the async task remains alive when adding it to the dependency list of another async task, thus avoiding the classical <ulink url="https://en.wikipedia.org/wiki/ABA_problem">ABA problem</ulink>.</para>
<para><programlisting filename=".cpp"><codeline><highlight class="comment">//<sp/>main<sp/>thread<sp/>retains<sp/>shared<sp/>ownership<sp/>of<sp/>async<sp/>task<sp/>A</highlight><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>A<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){});</highlight></codeline>
<codeline><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"></highlight><highlight class="comment">//<sp/>task<sp/>A<sp/>remains<sp/>alive<sp/>(i.e.,<sp/>at<sp/>least<sp/>one<sp/>ref<sp/>count<sp/>by<sp/>the<sp/>main<sp/>thread)<sp/></highlight><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"></highlight><highlight class="comment">//<sp/>when<sp/>being<sp/>added<sp/>to<sp/>the<sp/>dependency<sp/>list<sp/>of<sp/>async<sp/>task<sp/>B</highlight><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>B<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){},<sp/>A);</highlight></codeline>
</programlisting></para>
<para>Currently, <ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref> is implemented based on C++ smart pointer <ref refid="cpp/memory/shared_ptr" kindref="compound" external="/Users/twhuang/Code/taskflow/doxygen/cppreference-doxygen-web.tag.xml">std::shared_ptr</ref> and is considered cheap to copy or move as long as only a handful of objects own it. When a worker completes an async task, it will remove the task from the executor, decrementing the number of shared owners by one. If that counter reaches zero, the task is destroyed.</para>
</sect1>
<sect1 id="DependentAsyncTasking_1CreateADynamicTaskGraphByMultipleThreads">
<title>Create a Dynamic Task Graph by Multiple Threads</title>
<para>You can use multiple threads to create a dynamic task graph as long as the order of simultaneously creating tasks is topologically correct. The example below uses creates a dynamic task graph using three threads (including the main thread), where task <computeroutput>A</computeroutput> runs before task <computeroutput>B</computeroutput> and task <computeroutput>C:</computeroutput> </para>
<para><programlisting filename=".cpp"><codeline><highlight class="normal"><ref refid="classtf_1_1Executor" kindref="compound">tf::Executor</ref><sp/>executor;</highlight></codeline>
<codeline><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"></highlight><highlight class="comment">//<sp/>main<sp/>thread<sp/>creates<sp/>a<sp/>dependent<sp/>async<sp/>task<sp/>A</highlight><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>A<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){});</highlight></codeline>
<codeline><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"></highlight><highlight class="comment">//<sp/>spawn<sp/>a<sp/>new<sp/>thread<sp/>to<sp/>create<sp/>an<sp/>async<sp/>task<sp/>B<sp/>that<sp/>runs<sp/>after<sp/>A</highlight><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"><ref refid="cpp/thread/thread" kindref="compound" external="/Users/twhuang/Code/taskflow/doxygen/cppreference-doxygen-web.tag.xml">std::thread</ref><sp/>t1([&amp;](){</highlight></codeline>
<codeline><highlight class="normal"><sp/><sp/><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>B<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){},<sp/>A);</highlight></codeline>
<codeline><highlight class="normal">});</highlight></codeline>
<codeline><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"></highlight><highlight class="comment">//<sp/>spawn<sp/>a<sp/>new<sp/>thread<sp/>to<sp/>create<sp/>an<sp/>async<sp/>task<sp/>C<sp/>that<sp/>runs<sp/>after<sp/>A</highlight><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal"><ref refid="cpp/thread/thread" kindref="compound" external="/Users/twhuang/Code/taskflow/doxygen/cppreference-doxygen-web.tag.xml">std::thread</ref><sp/>t2([&amp;](){</highlight></codeline>
<codeline><highlight class="normal"><sp/><sp/><ref refid="classtf_1_1AsyncTask" kindref="compound">tf::AsyncTask</ref><sp/>C<sp/>=<sp/>executor.<ref refid="classtf_1_1Executor_1a0e2d792f28136b8227b413d0c27d5c7f" kindref="member">silent_dependent_async</ref>([](){},<sp/>A);</highlight></codeline>
<codeline><highlight class="normal">});</highlight></codeline>
<codeline><highlight class="normal"></highlight></codeline>
<codeline><highlight class="normal">executor.<ref refid="classtf_1_1Executor_1ab9aa252f70e9a40020a1e5a89d485b85" kindref="member">wait_for_all</ref>();</highlight></codeline>
<codeline><highlight class="normal">t1.join();</highlight></codeline>
<codeline><highlight class="normal">t2.join();</highlight></codeline>
</programlisting></para>
<para>Regardless of <computeroutput>t1</computeroutput> runs before or after <computeroutput>t2</computeroutput>, the resulting topological order is always correct with the graph definition, either <computeroutput>ABC</computeroutput> or <computeroutput>ACB</computeroutput>. </para>
</sect1>
    </detaileddescription>
    <location file="cookbook/dependent_async_tasking.dox"/>
  </compounddef>
</doxygen>
