When I decided to make my foray into the Pythonic World, I stumbled upon the sorcery between system level Python2.7 and Python3. What are pip and pip3? I used to install some Python packages using pip and if not worked, used pip3. Either of them always worked 🙅♂️. I did not know what I was doing apart from just getting the system up and running until I determined to see how deep the rabbit hole goes. But the rabbit hole was not that deep, it was my confused mind that made it deep until now…
pip vs pip3
As you have already guessed that Python3 is a predecessor of Python2. In order to maintain the backward compatibility of the package manager pip for Python2, Python3 came up with its own package manager under the name of pip3. However, we can point python and pip commands directly to python3 and pip3 executables respectively (which we will see in the later sections), so that we do not have to deal with python3 or pip3 commands while running a python script or installing any python package.
The upshot is that pip by default points to the system level Python 2.7 and pip3 points to whatever version we have for Python3.
⇒ pip --version
pip 19.0 from (python 2.7)
⇒ pip3 --version
pip 18.1 from (python 3.7)
To naively create an alias for python and pip commands to point to python3 and pip3, we can add following in our bash/zsh file and reload the shell to take its effect.
alias python=python3
alias pip=pip3
# BEFORE
⇒ python --version
Python 2.7.15
⇒ pip --version
pip 19.0 from (python 2.7)
# AFTER
source ~/.zshrc
⇒ python --version
Python 3.7.1
⇒ pip --version
pip 18.1 from (python 3.7)
This approach works, however, we constantly have to edit the bash/zsh file to switch between two or more versions of Python. Clearly, we can do better.
Introducing pyenv
Pyenv allows us to install and switch between multiple versions of Python.
pyenv versions
We can check what versions of Python are installed on our system with the following command. The * in the beginning represents the current Python version (System Python 2.7 in this case) the python and pip commands point to.
⇒ pyenv versions
* system
3.6.3
pyenv install <version>
We can install any version of Python using the following install command. Note that the installation does not switch to the installed python version implicitly.
⇒ pyenv install 3.7.2
⇒ pyenv versions
* system
3.6.3
3.7.2
pyenv shell <version>
To manually switch to any Python version (only in the current shell), we can use this particular command. That means, killing the shell window would restore the Python version to the system level one. Here we have switched to Python 3.7.2 in the current shell.
⇒ pyenv shell 3.7.2
⇒ pyenv versions
system
3.6.3
* 3.7.2
Introducing pyenv-virtualenv
Now that we have fixed the problem of maintaining different versions of Python to be used in various Python Projects. The different but somewhat similar problem persists for Python packages too.
For example, imagine we have two Python projects running on top of Python 3.7.2 but using different versions of Django, 2.1.5 (latest) and 1.9. So installing both one after the other using pip install Django==2.1.5 and pip install Django==1.9 commands would override the 2.1.5 version with the 1.9 one. Hence, both projects inadvertently would end up using the same Django version which we do not want. That’s where Python Virtual Environments help.
There are many Python packages out there to manage our virtual environments and some of them are virtualenv, virtualenvwrapper, etc. Although, either is better or worse than others in some way. However, we are going to use pyenv-virtualenv which is a pyenv plugin using virtualenv under the hood.
pyenv virtualenvs
Similar to pyenv versions, this command shows us a list of virtual environments we have on our system. Below I have one virtualenv venv already created for Python 3.6.3.
⇒ pyenv virtualenvs
3.6.3/envs/venv
venv
pyenv virtualenv <environment-name>
Let’s create a virtual environment for Python 3.7.2. Now we can see the two virtual environments created but none of them are activated yet.
⇒ pyenv virtualenv venv-3.7.2
⇒ pyenv virtualenvs
3.6.3/envs/venv
3.7.2/envs/venv-3.7.2
venv
venv-3.7.2
pyenv activate <environment-name>
Let’s activate the virtual environment venv-3.7.2. The * in the beginning represents the activated virtual environment where Django will be installed.
⇒ pyenv activate venv-3.7.2
⇒ pyenv virtualenvs
3.6.3/envs/venv
3.7.2/envs/venv-3.7.2
venv
* venv-3.7.2
First, we can confirm if Django is installed in the activated virtual environment. If not, we will install Django 1.9.
# BEFORE
⇒ pip list --format=columns
Package Version
---------- -------
pip 19.0.1
setuptools 28.8.0
# AFTER
⇒ pip install Django==1.9
⇒ pip list --format=columns
Package Version
---------- -------
Django 1.9
pip 19.0.1
setuptools 28.8.0
So far so good. Now we must verify whether we got the isolation for packages using pyenv-virtualenv that we wanted.
pyenv deactivate
To check that we can deactivate the current virtual environment. This command will restore Python to system level one. And pip list will now show all the global Python packages installed on our system. Notice that Django is not installed anymore since we got out of the venv-3.7.2 virtual environment.
⇒ pyenv deactivate
⇒ pyenv virtualenvs
3.6.3/envs/venv
3.7.2/envs/venv-3.7.2
venv
venv-3.7.2
⇒ pip list --format=columns
Package Version
---------- -------
pip 9.0.1
setuptools 28.8.0
airbrake 2.1.0
aniso8601 4.1.0
arrow 0.10.0
asn1crypto 0.24.0
attrs 18.2.0
bcrypt 3.1.6
bitarray 0.8.3
boto 2.49.0
boto3 1.9.83
.
.
.
Wrap up
As of now, pyenv and pyenv-virtualenv are serving me well. I hope that things will be stable going forward too. 🤟
Its always better to hack around than... 