AI Stock Picking Dashboard via mwShiny in Docker behind Apache ReverseProxy

In this post I demonstrate [the performance of] a multi-window interactive graphical dashboard, which visualizes the stock-picking signals from an ensemble of deep neural networks.
Further I describe the online deployment of this dashboard by means of Docker and Apache ReverseProxy.
Everybody, who invests quantitatively and significantly contributed to the engaged software: in partucular R, [mw]Shiny, LAMP-stack, Docker (and of course TF/keras) are encouraged to claim their free access to this dashboard (others are also encouraged to request a paid subscription :)).

Updated on 2023.04.22 - added proper configuration of WebSocket. Interestingly, the previous version of the dashboard did also work without websockets but as I added yet another window it stopped working (although no principally new Shiny functionality was engaged).

If you follow my blog you probably remember a series of posts about hard- and software for deep learning, which I used to create an ensemble of deep neural networks for the stock picking. The artificial intelligence is though smart but it still knows nothing about Covid-19, nor about war in Ukraine. Thus I, myself, consider it not as a substitute but rather as a complement to [my own] natural intelligence. In other words the neural networks help me quickly identify the most promising from the thousands of stocks. But in order to interpret the AI-signals really quickly I need a good visualization of them: finally our right brain semisphere (which processes images) is about 100 times faster than the left one (which process symbols).

So the multiscreen dashboard looks like this (I hope you do not try to eye this screenshot on a smartphone :)).

In the middle there are recommendation snapshots for the current day for the complete universe of stocks in question. Triangles mark especially highly recommended stocks. If you click on a stock in any chart (each of 6 charts represents an element of the neural ensemble) it will be highlighted in all charts. Additionally, the time series of the neural signals will be opened on the right-hand side and historical price chart with volatility insights will be opened on the left-hand side. I decide whether to buy a stock only after a critical evaluation of these views.

You probably may mind: yes, looks pretty, but does it really bring a dollar value? Oh, yes! Have a look at the performance of the respective test wikifolio.

BTW, I do not publish this wikifolio yet since all committed trades for all published wikifolios are disclosed in real-time, which is of course not what I want. I am going to publish this wikifolio as a proof of track record in a couple of years (but those who are seriously interested may of course have a look at it earlier). And for those who are still bemused by a mantra "the market is efficient, every outperformance is just a luck" I have only the one sentence: large number of trades + sufficiently long testing period = the law of large numbers.

Now let us come to the technical part (as you likely noticed, I am not much more verbose than Jim Simons when I speak about the architecture and calibration of my neural networks but I conceal no info on technical infrastructure).

So in order to setup a Shiny Server it is very convenient to use Docker. The build file is as follows

FROM rocker/shiny:latest

USER shiny
EXPOSE <port>
CMD ["/usr/bin/shiny-server"]


Build it by docker build . -t lymg/shinyserver:1.0 and run by docker run -d --restart unless-stopped -p <port>:<port> -v <local/path/to/appsRootFolder>:/srv/shiny-server/apps -v <local/path/to/logs>:/var/log/shiny-server/ lymg/shinyserver:1.0


If everything goes good, you will be able to run your shiny app at http://<yourserver>:<port>/mwShinyApp.
However, there are essentially two problems: first for the safety reasons we rather need https and secondly, only Pro version of the shiny server allows a password protected access (remark: there are alternative access management solutions for shiny but not yet for mwShiny).

So we need to setup a Reverse Proxy (I did it with Apache Server, since I had it already configured, incl. the SSL and .htpasswd). I must admit I mostly did it by trial and error (do not forget to backup your server before brave trials or your risk to lose it due to a fatal error). What I describe worked for me but may not work for you.
Anyway, first you need to enable the respective apache modules

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod proxy_balancer
sudo a2enmod lbmethod_byrequests
sudo a2enmod proxy_wstunnel

sudo systemctl restart apache2


Then run sudo nano /etc/apache2/sites-available/000-default-le-ssl.conf

and directly after <VirtualHost *:443> put the following


RewriteEngine on
RewriteCond %{HTTP:Upgrade} =websocket
RewriteRule ^/your/mwShinyApp/?(.*) ws://127.0.0.1:<port>/mwShinyApp/$1 [P,L]

<Location /your/mwShinyApp/>
AuthType Basic
AuthName "Enter your login name and password"
AuthUserFile </path/to/>.htpasswd
Require valid-user
</Location>


Then sudo systemctl restart apache2 and you can securely access your mwShiny app at https://yourserver/your/mwShinyApp/ (not that the end-slash in mwShinyApp/ is necessary).

Last but not least you should likely disable the access to your app via insecure http://<yourserver>:<port>/mwShinyApp.
However, there is another (pretty unexpected) problem: Docker and UFW (the most popular Linux firewall) turn out not to be friends!

Fortunately in my case there was an easy solution: DigitalOcean (my hosting provider) offers firewalls out of box.

DigitalOcean also offers a series of excellent tutorials, so if you wanna learn how to setup Docker or Apache with SSL and .htpasswd (which we assumed as already done), have a look at DO website.