short ref
Automatic Differentiation by OverLoading in C++ The Basic Idea ⇓ Operator overloading (C++) VectorfunctioninC/C++: F:IRn→IRm:x→y=F(x) Internal representation of F (≡tape) Interpretation ⇓ Reverse mode yj =yj(x0,x1,…,xj) ⇓ ∂yj = ∂yj−i ∂xi ∂x0 = Aj−i (x0,x1,…,xj−i) =⇒ Gradients (adjoints) Forward mode x(t) = xjtj y(t)=yjtj +Otd+1 =⇒ Directional derivatives ∂y0 = ∂y1 = ∂y2 ∂x0 ∂x1 …