RTFC


I really could not think of a better title for this post because it is not just about using an @Input property setter instead of the life-cycle hook ngAfterViewInit. Hence the title is pretty much inspired from RTFM where “Manual” is replaced by “Code”.

It’s about how important it is to read the code.

Just read the code..!

Last month I had published the Angular blog post on NgConf Medium in which I had proposed various ways to use jQuery plugins in Angular. If you have not read it yet, do read it here and comment if any. Unfortunately, I did not get lucky enough to be ngChampions (Kudos to those who become) and hence I have decided to publish the sequel here on my personal blog.

So after publishing the first post, I went on reading the source code for Material Badge component, just casually.

And to my surprise, I noticed 3 profound things:

Structural Directive over Component

It depends on the functionality you want to build into the component. If all you want to do is alter a single DOM then always go for a custom structural directive instead of writing a custom component. Because the custom component mostly introduces its own APIs unnecessarily.

For example, take a look at the app-toolbar-legends component from the last article. Remember, I’m not contradicting myself in this article, however, for this particular jQuery plugin in Angular, we could safely create an Angular Directive rather than having the Angular Component with its own API in terms of class and icon attributes below.

<app-toolbar-legends class="btn-toolbar-success" icon="fa-bitcoin" [toolbarConfig]="{position: 'right'}">
  <div class="toolbar-icons hidden">
    <a href="#"><i class="fa fa-bitcoin"></i></a>
    <a href="#"><i class="fa fa-eur"></i></a>
    <a href="#"><i class="fa fa-cny"></i></a>
  </div>
</app-toolbar-legends>
<app-toolbar-legends class="btn-toolbar-dark" icon="fa-apple" [toolbarConfig]="{position: 'right', style: 'primary', animation: 'flip'}">
  <div class="toolbar-icons hidden">
    <a href="#"><i class="fa fa-android"></i></a>
    <a href="#"><i class="fa fa-apple"></i></a>
    <a href="#"><i class="fa fa-twitter"></i></a>
  </div>
</app-toolbar-legends>

That means we can simplify the usage of the jQuery plugin in Angular by slapping the Angular Directive on the existing markup as follows. There is no need for an extraneous understanding of where class or icon values go in the component template, it’s pretty clear and concise in here. Easy, just slap a directive appToolbarLegends along with the jQuery plugin configurations.

<div class="btn-toolbar btn-toolbar-success" [appToolbarLegends]="{position: 'right'}" >
  <i class="fa fa-bitcoin"></i>
</div>
<div class="toolbar-icons hidden">
  <a href="#"><i class="fa fa-bitcoin"></i></a>
  <a href="#"><i class="fa fa-eur"></i></a>
  <a href="#"><i class="fa fa-cny"></i></a>
</div>


<div class="btn-toolbar btn-toolbar-dark" [appToolbarLegends]="{position: 'right', style: 'primary', animation: 'flip'}">
  <i class="fa fa-apple"></i>
</div>
<div class="toolbar-icons hidden">
  <a href="#"><i class="fa fa-android"></i></a>
  <a href="#"><i class="fa fa-apple"></i></a>
  <a href="#"><i class="fa fa-twitter"></i></a>
</div>

Generate Unique Id for the DOM

I wanted a unique id attribute for each instance of the toolbar in order to map them to their respective toolbar buttons. I’m still laughing at myself for going above and beyond just to generate a unique ID with 0 dependencies. Finally, StackOverflow came to the rescue πŸ˜…

Math.random().toString(36).substr(2, 9)

But while reading the source code for Material Badge component, I found an elegant approach that I wish to frame on the wall someday πŸ˜‚. This will generate a unique _contentId for each instance of the directive without much fuss.

import { Directive } from '@angular/core';
let nextId = 0;
@Directive({
  selector: '[appToolbarLegends]'
})
export class LegendsDirective {
  private _contentId: string = `toolbar_${nextId++}`;
}

@Input property setter vs ngAfterViewInit

Before we get into the getter/setter, let’s understand when and why to use ngAfterViewInit. It’s fairly easy to understandβ€Šβ€”β€Šit is a life cycle hook that triggers when the View of the component or the directive attached to is initialized after all of its bindings are evaluated. That means if you are not concerned with querying the DOM or DOM attributes which have interpolation bindings on them, you can simply use Class Setter method as a substitute.

import { Directive, Input } from '@angular/core';
let nextId = 0;
@Directive({
  selector: '[appToolbarLegends]'
})
export class LegendsDirective {
  private _contentId: string = `toolbar_${nextId++}`;
  @Input('appToolbarLegends')
  set config(toolbarConfig: object) {
    console.log(toolbarConfig); // logs {position: "right"} object
  }
}

The Class Setters are called way before ngAfterViewInit or ngOnInit and hence they speed up the directive instantiation, slightly. Also, unlike ngAfterViewInit or ngOnInit , the Class Setters are called every time the new value is about to be set, giving us the benefit of destroying/recreating the plugin with new configurations.

Demo Day

Thanks for coming this far. So the moral of the story is to do read code written by others, does not matter which open source project it is.

Just read the code..!

https://stackblitz.com/edit/angular-zgi4er?embed=1&file=src/app/app.component.ts
If you found this article useful in anyway, feel free to donate me and receive my dilettante painting as a token of appreciation for your donation.
Advertisements

TypeScript: Generocks


It’s been a while that this blog post about Typescript Generics was in my drafts (not sure why) and finally it is out. 🀞But before we get into Generics, we should be aware of what makes Types in Typescript exciting.

Typescript allows us to type check the code so that errors can be caught during compile time, instead of run time. For example, the following code in Javascript may look correct at compile time but will throw an error in a browser or NodeJS environment.

const life = 42;
life = 24;  // OK at compile time

In this case, Typescript may infer the type of the variable life based on its value 42 and notify about the error in the editor/terminal. Additionally, you can specify the correct type explicitly if needed:

const life: number = 42;
life = 24; // Throws an error at compile time

Named Types

So there are few primitive types in Typescript such as number, string, object, array, boolean, any, and void (apart from undefined, null, never, and recently unknown). However, these are not enough, especially when we use them together in a large project and we might need a sort of an umbrella type or a custom type for them to hold together and be reusable. Such aliases are called named types which can be used to create custom types. They are classes, interfaces, enums, and type aliases.

For example, we can create a custom type, MyType comprising a few primitive types as follows.

interface MyType {
  foo: string;
  bar: number;
  baz: boolean;
}

But what if we want foo to be either string or object or array!? One way is to copy the existing interface to MyType2 (and so on).

interface MyType2 {
  foo: Array<string>;
  bar: number;
  baz: boolean;
}

The way we pass any random value to a function with the help of a function parameter to make the function reusable, what if we allowed to do the same for MyType as well. With this approach, the duplication of code will not be needed while handling the same set but different types of data.  But before we dive into it, let us first understand the problem with more clarity. And to understand it, we can write a cache function to cache some random string values.

function cache(key: string, value: string): string {
  (<any>window).cacheList = (<any>window).cacheList || {};
  (<any>window).cacheList[key] = value;
  return value;
}

Because of the strict type checking, we are forcing the parameter value to be of type String. But what if someone wants to use a numeric value? I’ll give you a hint: what operator in Javascript do we use for a fallback value if the expected value of a variable is Falsy? Correct! we use || a.k.a. logical OR operator. Now imagine for a second that you are the creator of Typescript Language (Sorry, Anders Hejlsberg) and willing to resolve this issue for all developers. So you might go for a similar solution to have a fallback type and after countless hours of brainstorming, end up using a bitwise operator i.e. | in this case (FYI, that thing is called union types in alternate dimensions where Anders Hejlsberg is still the creator of Typescript).

function cache(key: string, value: string | number): string | number {
  (<any>window).cacheList = (<any>window).cacheList || {};
  (<any>window).cacheList[key] = value;
  return value;
}

Is not that amazing? Wait, but what if someone wants to cache boolean values or arrays/objects of custom types!? Since the list is never ending, looks like our current solution is not scalable at all. Would not it be great to control these types from outside!? I mean, how about we allow to define placeholder types inside the above implementation and provide real types from the call site instead.

Generic Function

Let us use ValueType (or use any other placeholder or simply T to suit your needs) as a placeholder wherever needed.

function cache<ValueType>(key: string, value: ValueType): ValueType {
  (<any>window).cacheList = (<any>window).cacheList || {};
  (<any>window).cacheList[key] = value;
  return value;
}

We can even pass the custom type parameter MyType to the cache method in order to type check the value for correctness (try changing bar‘s value to be non-numeric and see for yourself).

cache<MyType>("bar", { foo: "foo", bar: 42, baz: true });

This mechanism of parameterizing types is called Generics. This, in fact, is a generic function.

Generic Classes

Similar to the generic function, we can also create generic classes using the same syntax. Here we have created a wrapper class CacheManager to hold the previously defined cache method and the global (<any>window).cacheList variable as a private property cacheList.

class CacheManager<ValueType> {
  private cacheList: { [key: string]: ValueType } = {};
  cache(key: string, value: ValueType): ValueType {
    this.cacheList[key] = value;
    return value;
  }
}
new CacheManager<MyType>().cache("bar", { foo: "bar", bar: 42, baz: true });

Even though the above code is perfect to encourage reusability of CacheManager while caching all types of values, but someday in future, there will be a need to provide varying types of data in MyType‘s properties as well. That exposed us to the original problem of MyType vs MyType2 (from the Named Types section above). To prevent us from duplicating the custom type MyType to accommodate varying types of properties, Typescript allows using generic types, even with interfaces, which makes them Generic Interfaces. In fact, we are not restricted to use only one parameter type below, use as many as needed. Additionally, we can have union types as a fallback to the provided parameter types. This permits us to pass an empty {} object while using the generic interface wherever we desire the default types of values.

interface MyType<FooType, BarType, BazType> {
  foo: FooType | string;
  bar: BarType | number;
  baz: BazType | boolean;
}
new CacheManager<MyType<{},{},{}>>().cache("bar", { foo: "bar", bar: 42, baz: true });
new CacheManager<MyType<number, string, Array<number>>>().cache("bar", { foo: 42, bar: "bar", baz: [0, 1] })

I know that it looks a bit odd to pass {} wherever the default parameter types are used. However, this has been resolved in Typescript 2.3 by allowing us to provide default type arguments so that passing {} in the parameter types will be optional.

Generic Constraints

When we implicitly or explicitly use a type for anything such as the key parameter of the cache method above, we are purposefully constraining the type of key to be of type String. But we are inadvertently allowing any sort of string here. What if we want to constraint the type of key to be the type of one of the interface properties of MyType only!? One naive way to do so is to use String Literal Types for key below and in return get [ts] Argument of type '"qux"' is not assignable to parameter of type '"foo" | "bar" | "baz"'error when the key type qux does not belong to MyType. This works correctly, however, because of the hard-coded string literal types, we can not replace MyType with YourType interface since it has different properties associated with it as follows.

interface YourType {
  qux: boolean;
}
class CacheManager<ValueType> {
  private cacheList: { [key: string]: ValueType } = {};
  cache(key: "foo" | "bar" | "baz", value: ValueType): ValueType {
    this.cacheList[key] = value;
    return value;
  }
}
new CacheManager<MyType<{},{},{}>>().cache("foo", { foo: "bar", bar: 42, baz: true }); // works
new CacheManager<MyType<{},{},{}>>().cache("qux", { foo: "bar", bar: 42, baz: true }); // !works
new CacheManager<YourType>().cache("qux", { qux: true }); // !works but should have worked

To make it work, we’ve to manually update the string literal types used for key each time in the class implementation. However, similar to Classes, Typescript also allows us to extend Generic Types. So that we can get away with the string literal types "foo" | "bar" | "baz" of key with the generic constraint i.e. KeyType extends keyof ValueType as follows. Here, we are forcing the type-checker to validate the type of key to be one of the interface properties provided in CacheManager. This way we can even serve previously mentioned YourType without any error.

class CacheManager<ValueType> {
  private cacheList: { [key: string]: ValueType } = {};
  cache<KeyType extends keyof ValueType>(key: KeyType, value: ValueType): ValueType {
    this.cacheList[key] = value;
    return value;
  }
}
new CacheManager<MyType<{},{},{}>>().cache("foo", { foo: "bar", bar: 42, baz: true }); // works
new CacheManager<MyType<{},{},{}>>().cache("qux", { foo: "bar", bar: 42, baz: true }); // !works
new CacheManager<YourType>().cache("qux", { qux: true }); // works

Alright, that brings us to the climax of Generics. I hope \0/ Generocks  \m/ for you.

If you found this article useful in anyway, feel free to donate me and receive my dilettante painting as a token of appreciation for your donation.

Enable tree-shaking in Rails/Webpacker: A Sequel


A month ago, I wrote a blog post explaining a hacky way to enable tree-shaking in Rails/Webpacker project at Simpl. I would definitely recommend skimming through the previous post if you have not already.

In this post, we will directly jump into a more robust and stable solution. But before that, I want to resurrect my old memories for you that haunted me for months wherein a broken manifest.json was generated during webpack compilation at a random place. This time, after upgrading @rails/webpacker and related webpack plugins, the problem has been escalated beyond repair wherein an incomplete but valid manifest.json was generated randomly having fewer pack entries than expected. So even the generated manifest.json has little chance of succor by the hacky NodeJS fix_manifest.js script I had written to fix the broken JSON last time.

After a bit of googling my way out, I learned that webpack, with multi-compiler configurations, compiles each webpack configuration asynchronously and disorderly. Which is why I was getting an invalid manifest.json earlier.

Imagine two webpack compilations running simultaneously and writing to the same manifest.json at the same time:

{
  "b.js": "/packs/b-b8a5b1d3c0c842052d48.js",
  "b.js.map": "/packs/b-b8a5b1d3c0c842052d48.js.map"
}  "a.js": "/packs/a-a3ea1bc1eb2b3544520a.js",
  "a.js.map": "/packs/a-a3ea1bc1eb2b3544520a.js.map"
}

Using different manifest file for each pack

Yes, this is the robust and stable solution I came up with. First, you have to override Manifest fileName in every webpack configuration in order to generate a separate Manifest file for each pack such as manifest-0.json, manifest-1.json, and so on. Then, use the same NodeJS script fix_manifest.js with a slight modification to concatenate all the generated files into a final manifest.json which will be accurate (having all the desired entries) and valid (JSON).

For that, we have to modify the existing generateMultiWebpackConfig method (in ./config/webpack/environment.js) in order to remove the existing clutter of disabling/enabling writeToEmit flag in Manifest which we no longer need. Instead, we will create a deep copy of the original webpack configuration and override the Manifest plugin opts for each entry. The deep copying is mandatory so that a unique Manifest fileName can endure for each pack file.

const { environment } = require('@rails/webpacker')
const cloneDeep = require('lodash.clonedeep')

environment.generateMultiWebpackConfig = function(env) {
  let webpackConfig = env.toWebpackConfig()
  // extract entries to map later in order to generate separate 
  // webpack configuration for each entry.
  // P.S. extremely important step for tree-shaking
  let entries = Object.keys(webpackConfig.entry)

  // Finally, map over extracted entries to generate a deep copy of
  // Webpack configuration for each entry to override Manifest fileName
  return entries.map((entryName, i) => {
    let deepClonedConfig = cloneDeep(webpackConfig)
    deepClonedConfig.plugins.forEach((plugin, j) => {
      // A check for Manifest Plugin
      if (plugin.opts && plugin.opts.fileName) {
        deepClonedConfig.plugins[j].opts.fileName = `manifest-${i}.json`
      }
    })
    return Object.assign(
      {},
      deepClonedConfig,
      { entry: { [entryName] : webpackConfig.entry[entryName] } }
    )
  })
}

Finally, we will update the ./config/webpack/fix_manifest.js NodeJS script to concatenate all the generated Manifest files into a single manifest.json file.

const fs = require('fs')

let manifestJSON = {}
fs.readdirSync('./public/packs/')
  .filter((fileName) => fileName.indexOf('manifest-') === 0)
  .forEach(fileName => {
    manifestJSON = Object.assign(
      manifestJSON,
      JSON.parse(fs.readFileSync(`./public/packs/${fileName}`, 'utf8'))
    )
})

fs.writeFileSync('./public/packs/manifest.json', JSON.stringify(manifestJSON))

Wrap up

Please note that the compilation of a huge number of JS/TS entries takes a lot of time and CPU, hence it is recommended to use this approach only in a Production environment. Additionally, set max_old_space_size to handle the out-of-memory issue for production compilation as per your need – using 8000MB i.e. 8GB in here.

$ node --max_old_space_size=8000 node_modules/.bin/webpack --config config/webpack/production.js
$ node config/webpack/fix_manifest.js

Always run those commands one after the other to generate fit and fine manifest.json πŸ˜™

If you found this article useful in anyway, feel free to donate me and receive my dilettante painting as a token of appreciation for your donation.

Fantastic Beasts: The Crimes of Pythonworld


When I decided to make my foray into the Pythonic World, I stumbled upon the sorcery between system level Python2.7 and Python3. What are pip and pip3? I used to install some Python packages using pip and if not worked, used pip3. Either of them always worked πŸ™…β€β™‚οΈ. I did not know what I was doing apart from just getting the system up and running until I determined to see how deep the rabbit hole goes. But the rabbit hole was not that deep, it was my confused mind that made it deep until now…

pip vs pip3

As you have already guessed that Python3 is a predecessor of Python2. In order to maintain the backward compatibility of the package manager pip for Python2, Python3 came up with its own package manager under the name of pip3. However, we can point python and pip commands directly to python3 and pip3 executables respectively (which we will see in the later sections), so that we do not have to deal with python3 or pip3 commands while running a python script or installing any python package.

The upshot is that pip by default points to the system level Python 2.7 and pip3 points to whatever version we have for Python3.

β‡’  pip --version
pip 19.0 from (python 2.7)
β‡’  pip3 --version
pip 18.1 from (python 3.7)

To naively create an alias for python and pip commands to point to python3 and pip3, we can add following in our bash/zsh file and reload the shell to take its effect.

alias python=python3
alias pip=pip3
# BEFORE
β‡’  python --version
Python 2.7.15
β‡’  pip --version
pip 19.0 from (python 2.7)

# AFTER
source ~/.zshrc
β‡’  python --version
Python 3.7.1
β‡’  pip --version
pip 18.1 from (python 3.7)

This approach works, however, we constantly have to edit the bash/zsh file to switch between two or more versions of Python. Clearly, we can do better.

Introducing pyenv

Pyenv allows us to install and switch between multiple versions of Python.

pyenv versions

We can check what versions of Python are installed on our system with the following command. The * in the beginning represents the current Python version (System Python 2.7 in this case) the python and pip commands point to.

β‡’  pyenv versions
* system
  3.6.3

pyenv install <version>

We can install any version of Python using the following install command. Note that the installation does not switch to the installed python version implicitly.

β‡’  pyenv install 3.7.2
β‡’  pyenv versions
* system
  3.6.3
  3.7.2

pyenv shell <version>

To manually switch to any Python version (only in the current shell), we can use this particular command. That means, killing the shell window would restore the Python version to the system level one. Here we have switched to Python 3.7.2 in the current shell.

β‡’  pyenv shell 3.7.2
β‡’  pyenv versions
  system
  3.6.3
* 3.7.2

Introducing pyenv-virtualenv

Now that we have fixed the problem of maintaining different versions of Python to be used in various Python Projects. The different but somewhat similar problem persists for Python packages too.

For example, imagine we have two Python projects running on top of Python 3.7.2 but using different versions of Django, 2.1.5 (latest) and 1.9. So installing both one after the other using pip install Django==2.1.5 and pip install Django==1.9 commands would override the 2.1.5 version with the 1.9 one. Hence, both projects inadvertently would end up using the same Django version which we do not want. That’s where Python Virtual Environments help.

There are many Python packages out there to manage our virtual environments and some of them are virtualenv, virtualenvwrapper, etc. Although, either is better or worse than others in some way. However, we are going to use pyenv-virtualenv which is a pyenv plugin using virtualenv under the hood.

pyenv virtualenvs

Similar to pyenv versions, this command shows us a list of virtual environments we have on our system. Below I have one virtualenv venv already created for Python 3.6.3.

β‡’  pyenv virtualenvs
  3.6.3/envs/venv
  venv

pyenv virtualenv <environment-name>

Let’s create a virtual environment for Python 3.7.2. Now we can see the two virtual environments created but none of them are activated yet.

β‡’  pyenv virtualenv venv-3.7.2
β‡’  pyenv virtualenvs
  3.6.3/envs/venv
  3.7.2/envs/venv-3.7.2
  venv 
  venv-3.7.2 

pyenv activate <environment-name>

Let’s activate the virtual environment venv-3.7.2. The * in the beginning represents the activated virtual environment where Django will be installed.

β‡’  pyenv activate venv-3.7.2
β‡’  pyenv virtualenvs
  3.6.3/envs/venv 
  3.7.2/envs/venv-3.7.2 
  venv 
* venv-3.7.2 

First, we can confirm if Django is installed in the activated virtual environment. If not, we will install Django 1.9.

# BEFORE
β‡’  pip list --format=columns
Package    Version
---------- -------
pip        19.0.1
setuptools 28.8.0

# AFTER
β‡’  pip install Django==1.9
β‡’  pip list --format=columns
Package    Version
---------- -------
Django     1.9
pip        19.0.1
setuptools 28.8.0

So far so good. Now we must verify whether we got the isolation for packages using pyenv-virtualenv that we wanted.

pyenv deactivate

To check that we can deactivate the current virtual environment. This command will restore Python to system level one. And pip list will now show all the global Python packages installed on our system. Notice that Django is not installed anymore since we got out of the venv-3.7.2 virtual environment.

β‡’  pyenv deactivate
β‡’  pyenv virtualenvs
  3.6.3/envs/venv 
  3.7.2/envs/venv-3.7.2 
  venv 
  venv-3.7.2 
β‡’  pip list --format=columns
Package    Version
---------- -------
pip        9.0.1
setuptools 28.8.0
airbrake               2.1.0
aniso8601              4.1.0
arrow                  0.10.0
asn1crypto             0.24.0
attrs                  18.2.0
bcrypt                 3.1.6
bitarray               0.8.3
boto                   2.49.0
boto3                  1.9.83
.
.
.

Wrap up

As of now, pyenv and pyenv-virtualenv are serving me well. I hope that things will be stable going forward too. 🀟

If you found this article useful in anyway, feel free to donate me and receive my dilettante painting as a token of appreciation for your donation.

Managing CRON with Ansible


Setting up a CRON job manually is a child’s play so why am I writing about it? The reason I’m writing about it because of two main reasons:

  1. My experience of setting it up with Ansible
  2. Common mistakes I made which others can avoid

CRON

CRON daemon is a long-running process that executes commands at specific dates and times. This makes it easy to schedule activities like sending bulk emails. The CRON relies on a crontab file that has a list of scheduled commands to run. So we can manually add/edit/remove scheduled commands directly in the crontab file but this may induce bugs, especially when the list is too long. Ansible helps in deploying such CRON jobs effortlessly.

Ansible

Ansible is an IT automation and orchestration engine. It uses YAML file syntax for us to write such automation called Plays and the file itself is referred to as a Playbook. Playbooks contain Plays. Plays contain Tasks. And Tasks run pre-installed modules sequentially and trigger optional handlers.

Similarly, I have used a CRON module in Ansible to set up a task which configures a CRON job as follows:

- name: Run CRON job every day at 11.05 PM UTC
  become: yes
  become_method: sudo
  cron:
    name: "apache_restart"
    user: "root"
    hour: 23
    minute: 5
    job: "/bin/sh /usr/sbin/apache_restart.sh"
    state: present

Imagine there is a fictional CRON job to run Apache2 at the specified time every day – god knows why? But I made unfair mistakes initially while setting it up. Let us go step by step into each of those mistakes:

become and become_method

These flags are only necessary while running the job with sudo or any other privilege escalation method. In this case, I wanted to run /bin/sh /usr/sbin/apache_restart.sh command with sudo and I wished not to expect a password prompt that we usually get while running such commands manually. So the become flag prevents the password prompt.

In the beginning, I had forgotten to add these flags preventing the CRON job from executing the bash apache_restart.sh file as expected.

cron module

Ansible lets us use the pre-installed CRON module so that it will be far easy to setup CRON jobs. Although, by mistake, I had made CRON module an Ansible task as mentioned below.

- cron:
    name: "apache_restart"
    user: "root"
    hour: 23
    minute: 5
    job: "/bin/sh /usr/sbin/apache_restart.sh"
    state: present

As we learned before, only Tasks can run pre-installed modules. So Ansible instantly threw an error while deploying and I managed to save my face πŸ€¦β€β™‚οΈ

cron name

I thought that since I had already named the task, naming the CRON module will not be necessary. But I was embarrassed more than wrong. Because each time you deploy any changes to Ansible, without a CRON name, it will set up a new CRON job leaving the previous one as is. So I was literally restarting Apache2 thrice at a time. Remember, the CRON name works as a unique key to identify if any CRON job is already set up with the same name. If not, it will set up a brand new CRON job in the crontab file. Otherwise, override the existing one with new configurations.

state

The default state of the CRON job is present. Although to disable a particular CRON job, you change the state to absent and redeploy it via Ansible. I was using the state present without the CRON name that was creating multiple crontab entries on each deployment.

job

The job key takes the actual command that you want to run at a specific time/date. But make to use absolute command paths for brevity.

Wrap up

I also use tail -f /var/log/syslog and grep CRON /var/log/syslog commands to check the logs to make sure that CRON actually runs the bash file I specified.

If you found this article useful in anyway, feel free to donate me and receive my dilettante painting as a token of appreciation for your donation.

Kung Fu “Pandas”


I have planned to write an article per week on Data Engineering from the perspective of a beginner – the first one was Python for Frontend Engineers. Do not expect a proven pathway to become a Data Engineer someday because I do not have any strategy at the moment. I am just following my gut feeling to become proficient sooner than later. So consider these blog posts as my personal notes which may or may not be helpful to others.

So, Pandas is a data processing tool which helps in data analysis – meaning it provides various functions/methods to manipulate large datasets efficiently.

I am still learning pandas and will continue to explore its new features. The Pandas documentation is pretty self-explanatory so I will just give a glimpse of its powers in this article just like the trailer.

This is how you can import pandas and start using it right away.

Importing Pandas

import pandas as pd

pd.* # where * denotes all the supported methods

Pandas supports two data structures at the moment.

Series

The Series data structure represents a one-dimensional array with labels i.e. Python Dictionary. However, the data in the dictionary can be of any primitive types supported by Python.

Creating Series

sons_of_pandu = {
  'son1': 'Yudhishthira',
  'son2': 'Bhima',
  'son3': 'Arjuna',
  'son4': 'Nakula',
  'son5': 'Sahadeva'
}
pandavas_series = pd.Series(sons_of_pandu)
print(pandavas_series)
# Prints following in Jupyter Notebook
# son1   Yudhishthira
# son2   Bhima
# son3   Arjuna
# son4   Nakula
# son5   Sahadeva
# dtype: object

Changing Indices of Series

Sometimes we prefer to change the indexing for brevity. So here we can change the index of the series to Pandavas’ progenitors.

pandavas_series.index = ["Yama", "Vayu", "Indra", "Ashwini Kumara Nasatya", "Ashwini Kumara Darsa"] # Prints following in Jupyter Notebook
# Yama                   Yudhishthira
# Vayu                   Bhima
# Indra                  Arjuna
# Ashwini Kumara Nasatya Nakula
# Ashwini Kumara Darsa   Sahadeva
# dtype: object

Slicing Series

Slicing is really handy when glancing at a large dataset. We can also slice the series for an exploratory view as follows.

pandavas_series[0:2] # Prints first and second rows excluding the third
pandavas_series[1:]  # Prints all rows except the first one
pandavas_series[-2:] # Prints the last two rows only

Appending Series

It is very common to deal with different data sets in Pandas and the append method is just a compliment you can not ignore.

kauravas = ["Duryodhan", "Dushasana", "Vikarna", "Yuyutsu", "Jalsandh", "Sam", "Sudushil", "Bheembal", "Subahu", "Sahishnu", "Yekkundi", "Durdhar", "Durmukh", "Bindoo", "Krup", "Chitra", "Durmad", "Dushchar", "Sattva", "Chitraksha", "Urnanabhi", "Chitrabahoo", "Sulochan", "Sushabh", "Chitravarma", "Asasen", "Mahabahu", "Samdukkha", "Mochan", "Sumami", "Vibasu", "Vikar", "Chitrasharasan", "Pramah", "Somvar", "Man", "Satyasandh", "Vivas", "Upchitra", "Chitrakuntal", "Bheembahu", "Sund", "Valaki", "Upyoddha", "Balavardha", "Durvighna", "Bheemkarmi", "Upanand", "Anasindhu", "Somkirti", "Kudpad", "Ashtabahu", "Ghor", "Roudrakarma", "Veerbahoo", "Kananaa", "Kudasi", "Deerghbahu", "Adityaketoo", "Pratham", "Prayaami", "Veeryanad", "Deerghtaal", "Vikatbahoo", "Drudhrath", "Durmashan", "Ugrashrava", "Ugra", "Amay", "Kudbheree", "Bheemrathee", "Avataap", "Nandak", "Upanandak", "Chalsandhi", "Broohak", "Suvaat", "Nagdit", "Vind", "Anuvind", "Arajeev", "Budhkshetra", "Droodhhasta", "Ugraheet", "Kavachee", "Kathkoond", "Aniket", "Kundi", "Durodhar", "Shathasta", "Shubhkarma", "Saprapta", "Dupranit", "Bahudhami", "Yuyutsoo", "Dhanurdhar", "Senanee", "Veer", "Pramathee", "Droodhsandhee", "Dushala"]
kauravas_series = pd.Series(kauravas)
pandavas_series.append(kauravas_series) # Prints following in Jupyter Notebook
# Yama                   Yudhishthira
# Vayu                   Bhima
# Indra                  Arjuna
# Ashwini Kumara Nasatya Nakula
# Ashwini Kumara Darsa   Sahadeva
# 0                      Duryodhan
# 1                      Dushasana
.
.
.
# Length: 106, dtype: object

Dropping from Series

Pass the index to drop any row from the series.

pandavas_series.drop('Yama') # Prints following in Jupyter Notebook
# Vayu                   Bhima
# Indra                  Arjuna
# Ashwini Kumara Nasatya Nakula
# Ashwini Kumara Darsa   Sahadeva
# 0                      Duryodhan
# 1                      Dushasana
.
.
.
# Length: 105, dtype: object

Dataframes

The Dataframe data structure represents a two-dimensional list with labels i.e. Python List.

Creating Dataframe

sons_of_pandu = [{
  'name': 'Yudhishthira',
  'progenitor': "Yama"
}, {
  'name': 'Bhima',
  'progenitor': "Vayu"
}, {
  'name': 'Arjuna',
  'progenitor': "Indra"
}, {
  'name': 'Nakula',
  'progenitor': "Ashwini Kumara Nasatya"
}, {
  'name': 'Sahadeva',
  'progenitor': "Ashwini Kumara Darsa"
}]
df_pandavas = pd.DataFrame(sons_of_pandu)

Head’ing DataFrame

df_pandavas.head()  # returns first 5 rows
df_pandavas.head(3) # returns first 3 rows

Tail’ing DataFrame

df_pandavas.tail()  # returns last 5 rows
df_pandavas.tail(3) # returns last 3 rows

Sorting DataFrame

df_pandavas.sort_values(by="name")

Slicing DataFrame

df_pandavas[0:2] # Prints first and second rows excluding the third
df_pandavas[1:]  # Prints all rows except the first one
df_pandavas[-2:] # Prints the last two rows only
df_pandavas[["name"]] # Prints all rows with "name" column only

Copying DataFrame

df_pandavas_in_alternate_dimension = df_pandavas.copy()

Wrap up

That’s it. There are more to Pandas than mere slicing/merging/copying/sorting. You can easily read/write CSV/XL files in Pandas like never before. Head over to Pandas Documentation for more information.

If you found this article useful in anyway, feel free to donate me and receive my dilettante painting as a token of appreciation for your donation.

Python for Frontend Engineers


Py thee on.

— William Shakespeare (meaning Python code turns you on) πŸ˜…

I tried to teach Python to myself for a week and now I’m able to read and understand Python code comfortably. This blog summarises my thought process while learning Python and comparing its syntax with Javascript at every step. I’ll keep updating it as and when I encounter new Pythonic stuff.

First and foremost, wear a gauntlet and snap a finger for semicolons, curly braces, and camelCase names to bite the dust.

console.log vs print

JavascriptPython
console.log(“Hello World!”);print(“Hello World!”)
console.log(“Hello ” + “World!”);print(“Hello ” + “World!”)
var greet = “Hello”;
var who = “World!”;
console.log(`${greet} ${who}`);
greet = “Hello”
who = “World!”
print(“{0} {1}”.format(greet, who))

Primitive Types

JavascriptPython
var name = “John Doe”;name = “John Doe”
var age = 100;age = 100
var isMale = true;is_male = True

Utility Methods

JavascriptPython
“amit”.toUppercase(); “amit”.upper() # “amit”.upper().isupper()
“amit”.length;len(“amit”)
“amit”.indexOf(“a”);“amit”.index(“a”) // throws error if not found 😦
String(100);str(100)
parseInt(100.300, 10);int(100.300)
parseFloat(100.300, 10);float(100.300)
Math.abs(-100);abs(-100)
Math.pow(3, 2);pow(3, 2)
Math.round(3.2);round(3.2)
Math.floor(3.2);floor(3.2)
Math.ceil(3.2);ceil(3.2)
Math.sqrt(25);sqrt(25)
NA (use Lodash)max(4, 2)
NA (use Lodash)min(4, 2)

User Input

JavascriptPython
var ans = prompt(“What’s up?”);ans = input(“What’s up?”)

Arrays vs Lists

JavascriptPython
var arrStr = [“a”, “b”, “c”];lst_str = [“a”, “b”, “c”]
arrStr[0]lst_str[0]
arrStr[-1] // returns undefinedlst_str[-1] # returns “c”
NAlst_str[1:] # returns [“b”, “c”]
NAlst_str[1:2] # returns [“b”] ignores 3rd
NAlst_str[:2] # returns [“a”, “b”] ignores 3rd
arrStr[1] = “d”;lst_str[1] = “d”
arrStr.push(“d”); lst_str.append(“d”)
arrStr.splice(1, 0, “f”);lst_str.insert(1, “f”)
arrStr.indexOf(“b”);lst_str.index(“b”)
NAlst_str.count(“d”) # returns 2 occurrences
arrStr = arrStr.sort();lst_str.sort()
arrStr.reverse();lst_str.reverse()
var arrNum = [1, 2, 3];lst_num = [1, 2, 3]
arrStr = arrStr.concat(arrNum);lst_str.extend(lst_num)
var arrStr2 = arrStr.slice();lst_str2 = lst_str.copy()

Tuple in Typescript vs Tuple in Python

JavascriptPython
var point = [“x”, “y”];point = (“x”, “y”)
point[1];point[1]
var arrTuple = [[“x”, “y”], [“w”, “z”]];list_tuple = [(“x”, “y”), (“w”, “z”)]

function vs def

JavascriptPython
function greetMe(name){
    console.log(
       "Hi! " + name
    );
}
def greet_me(name):
    print("Hi! " + name)
 
 
 


function greetMe(name){
    return "Hi! " + name;
}
def greet_me(name):
    return "Hi! " + name

 

If Statements

JavascriptPython
if ((1&&1)||(2 === 2)) {
  console.log("124");
} else if (1 !== 2) {
  console.log("421");
} else {
  console.log("Lakhan!");
}
if (1 and 1) or (2 == 2):
   print("124")
elif (1 != 2):
   print("421")
else:
  print("Lakhan!")
 

Objects vs Dictionaries

JavascriptPython
var objRomans = {
  "five": "V",
  "ten":"X"
}
dict_romans = {
  "five": "V",
  "ten":"X"
}
// Returns undefined
// if key not exists
objRomans["five"]
# throws an error
# if key not exists
dict_romans["five"]

Loops

JavascriptPython
var i = 0;
var crossedLimit = false;
while (i !== 10) {
  i += 1;
  crossedLimit = i==10;
  if (!crossedLimit) {
    console.log(i);
  }
}
i = 0
crossed_limit=False
while i != 10:
  i += 1
  crossed_limit = i == 10
  if not(crossed_limit):
    print(i)
 
 
var name = "Amit";
var l = name.length;
for (var i = 0;i<l;i++) {
  console.log(
    name.charAt(i)
  );
}
name = "Amit"
 
for char in name:
  print(char)
 
 
 
var names = ["A", "M"];
for (let name in names) {
  console.log(
    names[name]
  );
}
names = ["A", "M"]
for name in names:
  print(name)
 
 

Comments

JavascriptPython
// single line comment
# single line comment
/*
  multi-line comment
  can be very long
*/
"""
  multi-line comment
  can be very long
"""

Imports

JavascriptPython
// imports everything
// from cookies.js
import "./cookies";

// imports `a`
// from cookies.js
import a from "./cookies";
# imports everything
# from cookies.py
import cookies

# imports `a`
# from cookies.py
from cookies import a

Try Catch vs Try Except

JavascriptPython
try {
  let obj = JSON.parse("")
} catch(e) {
  console.log(e.message);
}
import json
try:
  dict = json.loads("")
except Exception as e:
  print(str(e))

Classes

JavascriptPython
class Bike {
  isBikeToo() {
    return true;
  }
}

class Car
extends Bike {
  constructor(
    tire
  ) {
    super();
    this.tire=tire;
  }

  whoAmI() {
    if(this.tire==3){
      return "Auto";
    } else {
      return "Bike";
    }
  }
}

var c=new Car(3);
console.log(
  c.tire,
  c.whoAmI(),
  c.isBikeToo()
);
class Bike:
  def is_bike_too(self):
    return True
 
 

class Car(Bike):
   
  def __init__(self, tire):
     
    
    
    self.tire = tire


  def who_am_i(self):
    if self.tire == 3:
      return "Auto"
    else:
      return "Bike"
 
 
 
 
c = Car(3)
print("{0} {1} {2}".format(
  str(c.tire),
  c.who_am_i(),
  str(c.is_bike_too())
))
If you found this article useful in anyway, feel free to donate me and receive my dilettante painting as a token of appreciation for your donation.