Howto Setup SoS Jupyter in Conda – R4 and Python38 with Tensorflow and Spyder

Anaconda is a very practical tool to manage the virtual environments... when it properly works. In theory, a desired configuration shall be setup seamlessly. In practice Conda may need hours to resolve the configuration and finally do it wrong. In this manual I explain how to setup a SoS Jupyter Notebook with R4 and Python38 with a TensorFlow and Spyder. This worked on 01.01.2024

Instead of participating in holy wars meaningless discussions whether R or Python is better for data scientists I prefer to use them both.
It can be done either via R library reticulate or by means of SoS (Script-of-Scripts) Jupyter Notebook. An official SoS installation guide is available here. It worked seamlessly for Python3.7 in the past but currently works neither for Python3.8 nor even for Python3.7 (because some builds for 3.7 are not anymore available on conda servers).

After some iterations I found out how to do the setup for Python3.8. It requires an explicit specification of package versions and sometimes even build versions.
So:


conda create -n jupysos38 python=3.8.18
conda activate jupysos38
conda install tensorflow=2.10.0
conda install -c conda-forge r-base=4.1.3
conda install -c conda-forge r-devtools=2.4.5=r41hc72bb7e_0
conda install -c conda-forge r-reticulate=1.34.0=r41ha856d6a_0
conda install jupyter=1.0.0=py38_7

(Note that in the last line we have also explicitly specified the build).

Then call
RGui and in R Gui run

>install.packages('IRkernel')
>install.packages('arrow') #official SoS guide says you need it
>IRkernel::installspec()


Further
conda install -c conda-forge jsonschema-with-format-nongpl=4.20.0

Optionally install Spyder (a simple but powerful Python IDE)
conda install Spyder=5.4.3

Then run

conda install -c conda-forge sos-rmarkdown=0.1.3

This will also install sos=0.24.3, sos-notebook=0.20.1 (but we need>=0.24), sos-papermill=0.2.1 and sos-r=0.20.1

If you like matplotlib

conda install matplotlib=3.7.3 -c conda-forge


Finally run jupyter notebook and check that your test document looks like this

Remark: A very helpful conda command to guess a proper package/build version is conda search, try e.g. conda search tensorflow. Usually (but not always) the package file names are self-explanatory.

P.S.
There is no guarantee that one will keep the libraries for older Python versions on Conda servers.
So if you need a long-term permanence and portability, you shall setup a docker container (as long as the above-written manual works, you may even use conda in docker).
I, for one, do have projects that work on "legacy" software for years (yes, top-notch soft is technically cool but for the dollar value it is sometimes better not to touch what works).
I will describe how to setup a docker container in next posts, stay tuned.