GPUs in AI – are they always cool? No, sometimes they are hot!

Finally I harnessed Tesla K80 for my AI modeling but unfortunately the overheat brings much overheads and not by all models the GPU is superior over CPU. Continue reading "GPUs in AI – are they always cool? No, sometimes they are hot!"

Affordable Hardware for Stockpicking AI – BeerWulf’s eBay Adventure

In my previous post I reported howto build and install TensorFlow and horovod from sources and howto setup a BeerWulf (BeoWulf) cluster. Building this BeerWulf cluster is though a good exercise to make a (resilient) system of a commodity hardware, however, it is not the most efficient way for a practical purpose (in my case: for creating an AI model, which helps me to pick up stocks). In this post I consider the hardware alternatives in the sense of making them both as efficient and as cheap as possible. Continue reading "Affordable Hardware for Stockpicking AI – BeerWulf’s eBay Adventure"

Building TensorFlow 2.5 (CPU only) and Horovod from source in Ubuntu 20.04.2 LTS

Short summary:

sudo swapoff /swapfile
sudo dd if=/dev/zero of=/swapfile bs=1M count=65536 oflag=append conv=notrunc
sudo mkswap /swapfile
sudo swapon /swapfile

sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 1
sudo apt update
sudo apt install python3-dev python3-pip
sudo apt install python3-testresources
pip install -U --user pip numpy==1.19.5 wheel
pip install -U --user keras_preprocessing --no-deps
sudo apt install git
git clone https://github.com/tensorflow/tensorflow.git
cd tensorflow
git checkout r2.5
sudo apt install npm
sudo npm install -g @bazel/bazelisk
./configure

bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip install /tmp/tensorflow_pkg/tensorflow-2.5.0-cp38-cp38-linux_x86_64.whl

sudo apt-get install openssh-server
sudo systemctl enable ssh
sudo systemctl start ssh
ssh-keygen
ssh-copy-id vasily@SERVER2
ssh SERVER2
ssh-keygen
ssh-copy-id vasily@SERVER1

sudo snap install cmake --classic
sudo apt install openmpi-bin
mpirun -H SERVER1:1,SERVER2:1 hostname

git clone --recursive https://github.com/uber/horovod.git
cd horovod
python setup.py clean
python setup.py bdist_wheel
HOROVOD_WITH_TENSORFLOW=1 pip install ./dist/horovod-0.22.1-cp38-cp38-linux_x86_64.whl[tensorflow,keras]
mpirun -H SERVER1:3,SERVER2:3 python3 /home/vasily/horovod/examples/tensorflow2/tensorflow2_keras_mnist.py
Continue reading "Building TensorFlow 2.5 (CPU only) and Horovod from source in Ubuntu 20.04.2 LTS"

Howto Install Tensorflow-GPU with Keras in R – A manual that worked on 2021.02.20 (and likely will work in future)

A brief instruction:
0. Update your Nvidia graphic card driver (just driver; you need NOT install/update CUDA but make sure that your card has cuda compute capability >= 3.5)
1. install Anaconda (release Anaconda3-2020.11 from anaconda.org)
2. open anaconda prompt and run
>conda create -n tfgpu210p37 python==3.7
>conda activate tfgpu210p37
>conda install cudatoolkit=10.1 cudnn=7.6 -c=conda-forge
>conda install -c anaconda tensorflow-gpu
3. in R run
>install.packages("keras")
>reticulate::use_condaenv("tfgpu210p37", required = TRUE)
>library(keras)
4. If you wanna understand what is going on under the hood, read further
Continue reading "Howto Install Tensorflow-GPU with Keras in R – A manual that worked on 2021.02.20 (and likely will work in future)"

Interbreeding R and Python to Give Birth to an Optimal German Mortgage

In many countries a mortgage shall be completely redeemed to the end of its duration and may be refinanced anytime. In USA one additionally can foreclose his mortgage by mailing his house-keys to a bank. In Germany is different.
First of all, one may completely refinance a mortgage only after 10 years (otherwise one have to pay Vorfälligkeitsentschädigung - a compensation to a bank for missed interest). But taking a long-term mortgage (20 or 30 years) implies a higher mortgage rate, thus it is not uncommon to take several consequent 10- (or sometimes even 5-) year mortgages. In order to optimize mortgage costs / rate change risk relation one can take several submortgages with different durations. Last but not least there is no requirement to amortize a mortgage completely: normally one refinances the residual debt (Restschuld) with a follow-up mortgage (Anschlussfinanzierung).
In this post we discuss the challenges of mortgage optimization and show how two popular programming languages - R and Python - can help us together
Continue reading "Interbreeding R and Python to Give Birth to an Optimal German Mortgage"

Building QuantLib 1.14 with Visual Studio 2019 preview

QuantLib development often lags the version of Visual Studio. Thus you need to do some manual tuning to build QL v1.14 in VS 2019. Generally, if you can, you shall so far stay by VS2017. Continue reading "Building QuantLib 1.14 with Visual Studio 2019 preview"

Salaries in German IT Branch – a Case Study of Critical Statistics Review

Recently I have read the results of salary survey among Russian-speaking  software developers in Germany, published on dou.ua. I was skeptical about the validity of conclusions and expressed my critics (a bit less polite than I should have done it). But the survey author reasonably pointed out that he did his best in his free time and did provide the raw survey data. Recalling a popular motto in Soviet Union: if you are disagree then criticize but if you criticize then do it better I try to interpret the survey results more correctly. Continue reading "Salaries in German IT Branch – a Case Study of Critical Statistics Review"

PCA, Autoencoders and the Feasibility of Stockpicking

The idea that the stock picking makes less and less sense since the markets are more and more driven by the macroeconomic factors is quite popular.
Especially right now, as the markets are falling (like on todays FEd decision to increase the rates), this idea may seem to be plausible. However, we show that in the long term the stocks do show enough of individuality.
Continue reading "PCA, Autoencoders and the Feasibility of Stockpicking"

letYourMoneyGrow.com Serves You on Ubuntu 18.04.1 LTS Linux

We have successfully migrated from Ubuntu 16 to Ubuntu 18, assuring stable and secure functioning of letYourMoneyGrow.com for the next several years. We had to disable social login but the users that registered with their social network accounts can use our services as before (they merely need to reset their passwords). Continue reading "letYourMoneyGrow.com Serves You on Ubuntu 18.04.1 LTS Linux"

Yahoo Finance Live Feeds in Excel after their API Discontinuation in November 2017

On the fateful Wednesday of November 1st, 2017 Yahoo decided to stop their – until then – free service of delivering real time market data as a text stream through a special URL. For hundreds of businesses and individuals who had relied for years on Yahoo's benevolent free service, this single action meant only one thing: Instant death! Continue reading "Yahoo Finance Live Feeds in Excel after their API Discontinuation in November 2017"