Tag: neironka

In my previous post I reported howto build and install TensorFlow and horovod from sources and howto setup a BeerWulf (BeoWulf) cluster. Building this BeerWulf cluster is though a good exercise to make a (resilient) system of a commodity hardware, however, it is not the most efficient way for a practical purpose (in my case: for creating an AI model, which helps me to pick up stocks). In this post I consider the hardware alternatives in the sense of making them both as efficient and as cheap as possible. Continue reading "Affordable Hardware for Stockpicking AI – BeerWulf’s eBay Adventure"

Short summary:

sudo swapoff /swapfile
sudo dd if=/dev/zero of=/swapfile bs=1M count=65536 oflag=append conv=notrunc
sudo mkswap /swapfile
sudo swapon /swapfile

sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 1
sudo apt update
sudo apt install python3-dev python3-pip
sudo apt install python3-testresources
pip install -U --user pip numpy==1.19.5 wheel
pip install -U --user keras_preprocessing --no-deps
sudo apt install git
git clone https://github.com/tensorflow/tensorflow.git
cd tensorflow
git checkout r2.5
sudo apt install npm
sudo npm install -g @bazel/bazelisk
./configure

bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip install /tmp/tensorflow_pkg/tensorflow-2.5.0-cp38-cp38-linux_x86_64.whl

sudo apt-get install openssh-server
sudo systemctl enable ssh
sudo systemctl start ssh
ssh-keygen
ssh-copy-id vasily@SERVER2
ssh SERVER2
ssh-keygen
ssh-copy-id vasily@SERVER1

sudo snap install cmake --classic
sudo apt install openmpi-bin
mpirun -H SERVER1:1,SERVER2:1 hostname

git clone --recursive https://github.com/uber/horovod.git
cd horovod
python setup.py clean
python setup.py bdist_wheel
HOROVOD_WITH_TENSORFLOW=1 pip install ./dist/horovod-0.22.1-cp38-cp38-linux_x86_64.whl[tensorflow,keras]
mpirun -H SERVER1:3,SERVER2:3 python3 /home/vasily/horovod/examples/tensorflow2/tensorflow2_keras_mnist.py
Continue reading "Building TensorFlow 2.5 (CPU only) and Horovod from source in Ubuntu 20.04.2 LTS"