AD Myths Debunked: AD Will Infect All My Other Libraries

This post is part of the AD Myths Debunked series.

You’ve heard implementing Automatic Differentiation into your project will get you first and second-order sensitivities faster. Much faster. But you’ve also heard you will need to modify your whole code base, including every single library, which means other internal users of said libraries will need to understand AD as well, right? That sounds too intrusive, so you don’t pursue the idea much further.  

This is another story we have heard many times over the years. So, the questions are… Can you get a proof of concept (POC)? Can you apply AD to a library that contains code parts you would like to leave untouched? Can you implement AD in your library without affecting other users? The answer to all of these questions is ‘Yes’.

Let’s look at two very commonly found situations:

1. A proof of concept, or ‘PoC’, can be a great way of understanding quickly the benefits of AD, and the ease with which it can be implemented. When working on a PoC, we inherently use a small but reasonable subset of the client’s library.  Manual and tool-based dependency analysis can help extract the relevant subset of the code to do this. Of course, having the PoC up and running quickly is important. Our experience helps us to efficiently implement AD to ensure a rapid, informed, and expert assessment of the impact of AD. Additionally, our AD tools have APIs to easily transform between active and passive data, which makes interfaces between AD code and non AD code possible. Together with smart code design decisions and modern C++ features (like templates and lambdas), this is a powerful technique to avoid unnecessary expansion of AD into other libraries.  
2. If the library you’re working on is embedded by internal users into other libraries that are unaware of AD, clean data encapsulation is possible to hide AD completely, while still exploiting its benefits in your library. With our support, internal users will only see the runtime and accuracy benefits, staying clear of any AD-related business.
    

nAG’s AD toolset has been developed and enhanced over the last 12 years and it builds upon a further 10 years of AD R&D experience in C++ (even longer in Fortran!). We know that details matter. Applying AD to only a subset of the code base is part of our day-to-day work and we’ve not come across a single case we couldn’t handle.  

Myths are narratives, which might sound like truths, but by talking through these in some detail and sharing our experiences, we hope to help businesses navigate these issues. Results matter, myths should not.