Fantastic Beasts: The Crimes of Pythonworld

When I decided to make my foray into the Pythonic World, I stumbled upon the sorcery between system level Python2.7 and Python3. What are pip and pip3? I used to install some Python packages using pip and if not worked, used pip3. Either of them always worked 🙅‍♂️. I did not know what I was doing apart from just getting the system up and running until I determined to see how deep the rabbit hole goes. But the rabbit hole was not that deep, it was my confused mind that made it deep until now…

pip vs pip3

As you have already guessed that Python3 is a predecessor of Python2. In order to maintain the backward compatibility of the package manager pip for Python2, Python3 came up with its own package manager under the name of pip3. However, we can point python and pip commands directly to python3 and pip3 executables respectively (which we will see in the later sections), so that we do not have to deal with python3 or pip3 commands while running a python script or installing any python package.

The upshot is that pip by default points to the system level Python 2.7 and pip3 points to whatever version we have for Python3.

⇒  pip --version
pip 19.0 from (python 2.7)
⇒  pip3 --version
pip 18.1 from (python 3.7)

To naively create an alias for python and pip commands to point to python3 and pip3, we can add following in our bash/zsh file and reload the shell to take its effect.

alias python=python3
alias pip=pip3

# BEFORE
⇒  python --version
Python 2.7.15
⇒  pip --version
pip 19.0 from (python 2.7)

# AFTER
source ~/.zshrc
⇒  python --version
Python 3.7.1
⇒  pip --version
pip 18.1 from (python 3.7)

This approach works, however, we constantly have to edit the bash/zsh file to switch between two or more versions of Python. Clearly, we can do better.

Introducing pyenv

Pyenv allows us to install and switch between multiple versions of Python.

pyenv versions

We can check what versions of Python are installed on our system with the following command. The * in the beginning represents the current Python version (System Python 2.7 in this case) the python and pip commands point to.

⇒  pyenv versions
* system
  3.6.3

pyenv install <version>

We can install any version of Python using the following install command. Note that the installation does not switch to the installed python version implicitly.

⇒  pyenv install 3.7.2
⇒  pyenv versions
* system
  3.6.3
  3.7.2

pyenv shell <version>

To manually switch to any Python version (only in the current shell), we can use this particular command. That means, killing the shell window would restore the Python version to the system level one. Here we have switched to Python 3.7.2 in the current shell.

⇒  pyenv shell 3.7.2
⇒  pyenv versions
  system
  3.6.3
* 3.7.2

Introducing pyenv-virtualenv

Now that we have fixed the problem of maintaining different versions of Python to be used in various Python Projects. The different but somewhat similar problem persists for Python packages too.

For example, imagine we have two Python projects running on top of Python 3.7.2 but using different versions of Django, 2.1.5 (latest) and 1.9. So installing both one after the other using pip install Django==2.1.5 and pip install Django==1.9 commands would override the 2.1.5 version with the 1.9 one. Hence, both projects inadvertently would end up using the same Django version which we do not want. That’s where Python Virtual Environments help.

There are many Python packages out there to manage our virtual environments and some of them are virtualenv, virtualenvwrapper, etc. Although, either is better or worse than others in some way. However, we are going to use pyenv-virtualenv which is a pyenv plugin using virtualenv under the hood.

pyenv virtualenvs

Similar to pyenv versions, this command shows us a list of virtual environments we have on our system. Below I have one virtualenv venv already created for Python 3.6.3.

⇒  pyenv virtualenvs
  3.6.3/envs/venv
  venv

pyenv virtualenv <environment-name>

Let’s create a virtual environment for Python 3.7.2. Now we can see the two virtual environments created but none of them are activated yet.

⇒  pyenv virtualenv venv-3.7.2
⇒  pyenv virtualenvs
  3.6.3/envs/venv
  3.7.2/envs/venv-3.7.2
  venv 
  venv-3.7.2 

pyenv activate <environment-name>

Let’s activate the virtual environment venv-3.7.2. The * in the beginning represents the activated virtual environment where Django will be installed.

⇒  pyenv activate venv-3.7.2
⇒  pyenv virtualenvs
  3.6.3/envs/venv 
  3.7.2/envs/venv-3.7.2 
  venv 
* venv-3.7.2 

First, we can confirm if Django is installed in the activated virtual environment. If not, we will install Django 1.9.

# BEFORE
⇒  pip list --format=columns
Package    Version
---------- -------
pip        19.0.1
setuptools 28.8.0

# AFTER
⇒  pip install Django==1.9
⇒  pip list --format=columns
Package    Version
---------- -------
Django     1.9
pip        19.0.1
setuptools 28.8.0

So far so good. Now we must verify whether we got the isolation for packages using pyenv-virtualenv that we wanted.

pyenv deactivate

To check that we can deactivate the current virtual environment. This command will restore Python to system level one. And pip list will now show all the global Python packages installed on our system. Notice that Django is not installed anymore since we got out of the venv-3.7.2 virtual environment.

⇒  pyenv deactivate
⇒  pyenv virtualenvs
  3.6.3/envs/venv 
  3.7.2/envs/venv-3.7.2 
  venv 
  venv-3.7.2 
⇒  pip list --format=columns
Package    Version
---------- -------
pip        9.0.1
setuptools 28.8.0
airbrake               2.1.0
aniso8601              4.1.0
arrow                  0.10.0
asn1crypto             0.24.0
attrs                  18.2.0
bcrypt                 3.1.6
bitarray               0.8.3
boto                   2.49.0
boto3                  1.9.83
.
.
.

Wrap up

As of now, pyenv and pyenv-virtualenv are serving me well. I hope that things will be stable going forward too. 🤟

If you found this article useful in anyway, feel free to donate me and receive my dilettante painting as a token of appreciation for your donation.

Advertisement

Kung Fu “Pandas”

I have planned to write an article per week on Data Engineering from the perspective of a beginner – the first one was Python for Frontend Engineers. Do not expect a proven pathway to become a Data Engineer someday because I do not have any strategy at the moment. I am just following my gut feeling to become proficient sooner than later. So consider these blog posts as my personal notes which may or may not be helpful to others.

So, Pandas is a data processing tool which helps in data analysis – meaning it provides various functions/methods to manipulate large datasets efficiently.

I am still learning pandas and will continue to explore its new features. The Pandas documentation is pretty self-explanatory so I will just give a glimpse of its powers in this article just like the trailer.

This is how you can import pandas and start using it right away.

Importing Pandas

import pandas as pd

pd.* # where * denotes all the supported methods

Pandas supports two data structures at the moment.

Series

The Series data structure represents a one-dimensional array with labels i.e. Python Dictionary. However, the data in the dictionary can be of any primitive types supported by Python.

Creating Series

sons_of_pandu = {
  'son1': 'Yudhishthira',
  'son2': 'Bhima',
  'son3': 'Arjuna',
  'son4': 'Nakula',
  'son5': 'Sahadeva'
}
pandavas_series = pd.Series(sons_of_pandu)
print(pandavas_series)
# Prints following in Jupyter Notebook
# son1   Yudhishthira
# son2   Bhima
# son3   Arjuna
# son4   Nakula
# son5   Sahadeva
# dtype: object

Changing Indices of Series

Sometimes we prefer to change the indexing for brevity. So here we can change the index of the series to Pandavas’ progenitors.

pandavas_series.index = ["Yama", "Vayu", "Indra", "Ashwini Kumara Nasatya", "Ashwini Kumara Darsa"] # Prints following in Jupyter Notebook
# Yama                   Yudhishthira
# Vayu                   Bhima
# Indra                  Arjuna
# Ashwini Kumara Nasatya Nakula
# Ashwini Kumara Darsa   Sahadeva
# dtype: object

Slicing Series

Slicing is really handy when glancing at a large dataset. We can also slice the series for an exploratory view as follows.

pandavas_series[0:2] # Prints first and second rows excluding the third
pandavas_series[1:]  # Prints all rows except the first one
pandavas_series[-2:] # Prints the last two rows only

Appending Series

It is very common to deal with different data sets in Pandas and the append method is just a compliment you can not ignore.

kauravas = ["Duryodhan", "Dushasana", "Vikarna", "Yuyutsu", "Jalsandh", "Sam", "Sudushil", "Bheembal", "Subahu", "Sahishnu", "Yekkundi", "Durdhar", "Durmukh", "Bindoo", "Krup", "Chitra", "Durmad", "Dushchar", "Sattva", "Chitraksha", "Urnanabhi", "Chitrabahoo", "Sulochan", "Sushabh", "Chitravarma", "Asasen", "Mahabahu", "Samdukkha", "Mochan", "Sumami", "Vibasu", "Vikar", "Chitrasharasan", "Pramah", "Somvar", "Man", "Satyasandh", "Vivas", "Upchitra", "Chitrakuntal", "Bheembahu", "Sund", "Valaki", "Upyoddha", "Balavardha", "Durvighna", "Bheemkarmi", "Upanand", "Anasindhu", "Somkirti", "Kudpad", "Ashtabahu", "Ghor", "Roudrakarma", "Veerbahoo", "Kananaa", "Kudasi", "Deerghbahu", "Adityaketoo", "Pratham", "Prayaami", "Veeryanad", "Deerghtaal", "Vikatbahoo", "Drudhrath", "Durmashan", "Ugrashrava", "Ugra", "Amay", "Kudbheree", "Bheemrathee", "Avataap", "Nandak", "Upanandak", "Chalsandhi", "Broohak", "Suvaat", "Nagdit", "Vind", "Anuvind", "Arajeev", "Budhkshetra", "Droodhhasta", "Ugraheet", "Kavachee", "Kathkoond", "Aniket", "Kundi", "Durodhar", "Shathasta", "Shubhkarma", "Saprapta", "Dupranit", "Bahudhami", "Yuyutsoo", "Dhanurdhar", "Senanee", "Veer", "Pramathee", "Droodhsandhee", "Dushala"]
kauravas_series = pd.Series(kauravas)
pandavas_series.append(kauravas_series) # Prints following in Jupyter Notebook
# Yama                   Yudhishthira
# Vayu                   Bhima
# Indra                  Arjuna
# Ashwini Kumara Nasatya Nakula
# Ashwini Kumara Darsa   Sahadeva
# 0                      Duryodhan
# 1                      Dushasana
.
.
.
# Length: 106, dtype: object

Dropping from Series

Pass the index to drop any row from the series.

pandavas_series.drop('Yama') # Prints following in Jupyter Notebook
# Vayu                   Bhima
# Indra                  Arjuna
# Ashwini Kumara Nasatya Nakula
# Ashwini Kumara Darsa   Sahadeva
# 0                      Duryodhan
# 1                      Dushasana
.
.
.
# Length: 105, dtype: object

Dataframes

The Dataframe data structure represents a two-dimensional list with labels i.e. Python List.

Creating Dataframe

sons_of_pandu = [{
  'name': 'Yudhishthira',
  'progenitor': "Yama"
}, {
  'name': 'Bhima',
  'progenitor': "Vayu"
}, {
  'name': 'Arjuna',
  'progenitor': "Indra"
}, {
  'name': 'Nakula',
  'progenitor': "Ashwini Kumara Nasatya"
}, {
  'name': 'Sahadeva',
  'progenitor': "Ashwini Kumara Darsa"
}]
df_pandavas = pd.DataFrame(sons_of_pandu)

Head’ing DataFrame

df_pandavas.head()  # returns first 5 rows
df_pandavas.head(3) # returns first 3 rows

Tail’ing DataFrame

df_pandavas.tail()  # returns last 5 rows
df_pandavas.tail(3) # returns last 3 rows

Sorting DataFrame

df_pandavas.sort_values(by="name")

Slicing DataFrame

df_pandavas[0:2] # Prints first and second rows excluding the third
df_pandavas[1:]  # Prints all rows except the first one
df_pandavas[-2:] # Prints the last two rows only
df_pandavas[["name"]] # Prints all rows with "name" column only

Copying DataFrame

df_pandavas_in_alternate_dimension = df_pandavas.copy()

Wrap up

That’s it. There are more to Pandas than mere slicing/merging/copying/sorting. You can easily read/write CSV/XL files in Pandas like never before. Head over to Pandas Documentation for more information.

If you found this article useful in anyway, feel free to donate me and receive my dilettante painting as a token of appreciation for your donation.

Python for Frontend Engineers

Py thee on.

— William Shakespeare (meaning Python code turns you on) 😅

I tried to teach Python to myself for a week and now I’m able to read and understand Python code comfortably. This blog summarises my thought process while learning Python and comparing its syntax with Javascript at every step. I’ll keep updating it as and when I encounter new Pythonic stuff.

First and foremost, wear a gauntlet and snap a finger for semicolons, curly braces, and camelCase names to bite the dust.

console.log vs print

Javascript	Python
console.log(“Hello World!”);	print(“Hello World!”)
console.log(“Hello ” + “World!”);	print(“Hello ” + “World!”)
var greet = “Hello”; var who = “World!”; console.log(`${greet} ${who}`);	greet = “Hello” who = “World!” print(“{0} {1}”.format(greet, who))

Primitive Types

Javascript	Python
var name = “John Doe”;	name = “John Doe”
var age = 100;	age = 100
var isMale = true;	is_male = True

Utility Methods

Javascript	Python
“amit”.toUppercase();	“amit”.upper() # “amit”.upper().isupper()
“amit”.length;	len(“amit”)
“amit”.indexOf(“a”);	“amit”.index(“a”) // throws error if not found 😦
String(100);	str(100)
parseInt(100.300, 10);	int(100.300)
parseFloat(100.300, 10);	float(100.300)
Math.abs(-100);	abs(-100)
Math.pow(3, 2);	pow(3, 2)
Math.round(3.2);	round(3.2)
Math.floor(3.2);	floor(3.2)
Math.ceil(3.2);	ceil(3.2)
Math.sqrt(25);	sqrt(25)
NA (use Lodash)	max(4, 2)
NA (use Lodash)	min(4, 2)

User Input

Javascript	Python
var ans = prompt(“What’s up?”);	ans = input(“What’s up?”)

Arrays vs Lists

Javascript	Python
var arrStr = [“a”, “b”, “c”];	lst_str = [“a”, “b”, “c”]
arrStr[0]	lst_str[0]
arrStr[-1] // returns undefined	lst_str[-1] # returns “c”
NA	lst_str[1:] # returns [“b”, “c”]
NA	lst_str[1:2] # returns [“b”] ignores 3rd
NA	lst_str[:2] # returns [“a”, “b”] ignores 3rd
arrStr[1] = “d”;	lst_str[1] = “d”
arrStr.push(“d”);	lst_str.append(“d”)
arrStr.splice(1, 0, “f”);	lst_str.insert(1, “f”)
arrStr.indexOf(“b”);	lst_str.index(“b”)
NA	lst_str.count(“d”) # returns 2 occurrences
arrStr = arrStr.sort();	lst_str.sort()
arrStr.reverse();	lst_str.reverse()
var arrNum = [1, 2, 3];	lst_num = [1, 2, 3]
arrStr = arrStr.concat(arrNum);	lst_str.extend(lst_num)
var arrStr2 = arrStr.slice();	lst_str2 = lst_str.copy()

Tuple in Typescript vs Tuple in Python

Javascript	Python
var point = [“x”, “y”];	point = (“x”, “y”)
point[1];	point[1]
var arrTuple = [[“x”, “y”], [“w”, “z”]];	list_tuple = [(“x”, “y”), (“w”, “z”)]

function vs def

Javascript	Python
function greetMe(name){ console.log( "Hi! " + name ); }	def greet_me(name): print("Hi! " + name)
function greetMe(name){ return "Hi! " + name; }	def greet_me(name): return "Hi! " + name

If Statements

Javascript	Python
if ((1&&1)||(2 === 2)) { console.log("124"); } else if (1 !== 2) { console.log("421"); } else { console.log("Lakhan!"); }	if (1 and 1) or (2 == 2): print("124") elif (1 != 2): print("421") else: print("Lakhan!")

Objects vs Dictionaries

Javascript	Python
var objRomans = { "five": "V", "ten":"X" }	dict_romans = { "five": "V", "ten":"X" }
// Returns undefined // if key not exists objRomans["five"]	# throws an error # if key not exists dict_romans["five"]

Loops

Javascript	Python
var i = 0; var crossedLimit = false; while (i !== 10) { i += 1; crossedLimit = i==10; if (!crossedLimit) { console.log(i); } }	i = 0 crossed_limit=False while i != 10: i += 1 crossed_limit = i == 10 if not(crossed_limit): print(i)
var name = "Amit"; var l = name.length; for (var i = 0;i<l;i++) { console.log( name.charAt(i) ); }	name = "Amit" for char in name: print(char)
var names = ["A", "M"]; for (let name in names) { console.log( names[name] ); }	names = ["A", "M"] for name in names: print(name)

Comments

Javascript	Python
// single line comment	# single line comment
/* multi-line comment can be very long */	""" multi-line comment can be very long """

Imports

Javascript	Python
// imports everything // from cookies.js import "./cookies"; // imports `a` // from cookies.js import a from "./cookies";	# imports everything # from cookies.py import cookies # imports `a` # from cookies.py from cookies import a

Try Catch vs Try Except

Javascript	Python
try { let obj = JSON.parse("") } catch(e) { console.log(e.message); }	import json try: dict = json.loads("") except Exception as e: print(str(e))

Classes

Javascript	Python
class Bike { isBikeToo() { return true; } } class Car extends Bike { constructor( tire ) { super(); this.tire=tire; } whoAmI() { if(this.tire==3){ return "Auto"; } else { return "Bike"; } } } var c=new Car(3); console.log( c.tire, c.whoAmI(), c.isBikeToo() );	class Bike: def is_bike_too(self): return True class Car(Bike): def __init__(self, tire): self.tire = tire def who_am_i(self): if self.tire == 3: return "Auto" else: return "Bike" c = Car(3) print("{0} {1} {2}".format( str(c.tire), c.who_am_i(), str(c.is_bike_too()) ))

If you found this article useful in anyway, feel free to donate me and receive my dilettante painting as a token of appreciation for your donation.