QuantLib Python - debugging C++ side with Visual Studio and PyCharm

QuantLib Python - a port of C++ library to Python via SWIG - provides a lot of advantages for a practical usage. In particular, it gives a great flexibility due to interactive python console and allows a seamless integration with the AI libraries like Keras and Tensorflow. However, it seems to be challenging to debug the C++ code, called from Python side. So far we found out a quick but dirty solution.

Prolog
Several years ago I commenced to write the Notes on Getting started with QuantLib, primarily intended for the junior quants but also for the seniors with not so much programming skills. I did not expect that this book will make me rich, however, I pursued some monetary reward. But [not] surprisingly, there was (and still is) little [pay-willing] interest, as the following survey on QuantNet shows.

Thus I decided to drop writing and to make already written chapters freely available, encouraging to make a small donation, if found the stuff useful. In return I promised to provide a debugger-friendly version of the QuantLib::Date class; however, it currently got obsolete both in the sense of QuantLib as intraday calculations were introduced, and in the sense of Visual Studio as autoexp.dat was depreciated. Additionally, QuantLibPython already provides a nicely looking representation of the type Date.

Anyway, I did not care much since until recently no single donation was made (well, I wonder, how many persons besides myself bought Luigi's book). However, the as the first donation come, I felt obliged in some way. Although purely technical stuff is not the area of my specialization (I am particularly strong at the junction of tech and domain knowledge), I wrote this post as my thankyou for the donation.

So as I already said, in practice an interpreted language (and Python in particular) provides a lot of advantages over a pure C++ solution. As long as you just implement a quantitative model, you'd better do it in C++ (or, sometimes, even in CUDA). But as you come to a day-to-day usage, you need to glue your program and integrate it with the data sources. Additionally, you may need to provide a [web] front-end. Finally, it is much easier to interactively write some lines of a python code than to recompile a C++ project.

However, you need to be able to occasionally debug the C++ side; otherwise the development will not be rapid (rather it will be a hell job). But it seems to be not so easy if you use PyCharm for Python and Visual Studio 2017 for C++. A common approach to attach python.exe to the Visual Studio debugger does not work.

Indeed you encounter a bunch of problems: not only it is implied that you need to compile QuantLib library in Release configuration (in which the debugging symbols are lost) but also PyCharm and Visual Studio python debuggers conflict with each other. I tried to proceed like this, setting Debugger to Native Only, but it did not work for me.

So after some trial and error I did the following:

1) do a complete Python installation, including Debugging symbols and debug binaries.
Note that they say require VS 2015 or later but for me it does not work with VS 2017 (as I attach python_d.exe to the debugger in VS2017, it exits with a non-verbose error code).

2) Build QuantLib in Visual Studio 2015 in Debug / x64 configuration (x64 is for 64x Python, as in my case). Run VS x64 native command prompt (note that Tools -> Visual Studio Command Prompt is wrong)
and run the following commands:
set INCLUDE=%INCLUDE%C:\boost\boost_1_65_1 set QL_DIR=D:\QuantLib\QuantLib-1.13 cd D:\QuantLib\QuantLib-SWIG-1.13\Python python_d setup.py build --debug python_d setup.py test python_d setup.py install
Note that you need python_d and a proper set of INCLUDE (in your case it may also be %INCLUDE%;C:\boost\boost_1_65_1). If you just set set INCLUDE=C:\boost\boost_1_65_1 (as suggested on QuantLib official site), the Visual Studio compiler might be missing some headers.