Ease and Impact matrix

I've been reading up strategies for successful project management for a while now. Going through some material on what are the ideal things to do during the project initialization step, I came across another useful tool, Ease and impact matrix. A while back, I had covered Responsibiltiy Assignment Matrix, which is a relevant tool for managing different responsibilities every stakeholder role may have, and associated communication.

Coming back to Ease and impact matrix, when a new project is beginning, a lot of details are still unknown - for example, who are the stakeholders, what is the success criteria, who is responsible for delivering the project, who takes accountability of the developments and so on. At this stage, defining the scope of the project is one of the most critical pieces, as it affects all other timelines for every other part. The basic idea then, is to brainstorm amongst various stakeholders to generate the list of ideas, rank each of these ideas on the basis of ease of doing and impact they create for the organization, and have a chart visualization for the prioritization.

Now those of you who would have worked with bug tracking tools (like bugzilla, JIRA, github issues etc) would be familiar with two fields that most tickets carry - ticket type (bug, task, epic etc) and priority (low to high to blocker). These tickets help in the management of all development during the lifecycle of the project, and if one were to think of it, during any sprint, teams usually pick a mix of tickets (tech debt, critical hotfixes, regular development, maintainence, and so on) to satisfy all their stakeholders. Similarly, organizations regularly use the BCG matrix to evaluate how to invest within their portfolios, based on a matrix of market share of the products and the market growth rate. The common theme with any such matrix solution remains picking up ideas that fall within the best quadrants.

In the case of Ease and impact matrix, this usually follows the pattern: 1) High impact, high ease of doing 2) High Impact, low ease of doing 3) low impact, high ease of doing and 4) low impact, low ease of doing. Depending on the strategic objectives and stakeholder needs, a bunch of ideas can thus be picked, making sure that the goals associated with the project can be SMART (Specific, measurable, achievable, relevant, and time bound).

To build your Ease and impact matrix, you would need to:

1. Brainstorm with your stakeholders and sponsors to arrive at a list of ideas
2. Identify the goal of the strategy - make sure that you what is one most important measure that would indicate failure
3. Rank each of these ideas in terms of impact achieved on the goal - You can use a 1-10 scale to do this, or could even do estimates using t-shirt sizing of the buckets - plot this data along Y axis
4. Further rank these ideas based on the ease of doing them, and make sure you are using the same scale here (1-10 numbers vs t-shirt sizing) as for impact - plot this data along X axis
5. Make sure every idea/activity is mapped on the matrix
6. Priortise within each qudrant, on basis of impact, and then ease of doing:
   1. High impact, High ease of doing: Prioritise these activities as the first to do
   2. High impact, Low ease of doing: Spend more time in planning these activities, as these may otherwise result in poor outcome because of the complexity.
   3. Low impact, High ease of doing: Do these tasks if they are a must for a higher impact task
   4. Low impact, Low ease of doing: Depriortize these activites brutally, and don’t attempt them unless they are necessary for a high impact task.
7. In general, activities that are hard to do should be the first to be rejected by wavering support from project sponsors.
8. Depending on the scope of the project, you can even try planning activities in 90 day blocks, while planning the remaining years at the high level only.

And thus, you would have a very relevant framework/tool, which would help you remain focussed and achive due success.

Book review: As A Man Thinketh, The Very Best of Common Man

As a Man Thinketh is a short essay, written by James Allen, and published in 1903 as a small book. The subject of the book is the power and right application of thought.

Its title is influenced by the biblical verse: "As a man thinketh in his heart, so is he". I picked this book up for reading on a whim - I was looking for some motivational quotes for a side project, when I came across a suggestion for reading this book on one of the blogs. Googling for the title brought forth stellar reviews on Amazon, and I ordered it from Amazon. The good guys at Amazon delivered it the next evening (perks of a Prime account), and I kept it in stash over the week to be read on weekend.

The book is a quick read, with 7 chapters in total. Working on a few personal projects these days, I found the overall language and content of the book inspirational - like how doubts and fears kill our chances of success, and how dreaming is important to achieve success in life. James Allen has some mesmerising observations, and I've listed 4 that I think are really profound:

1. Thoughts of doubts and fear never accomplish anything, and never can.
2. Men are anxious to improve their circumstances, but are unwilling to improve themselves, they therefore remain bound.
3. Circumstance does not make the man, it reveals him to himself.
4. Men do not attract what they want, but what they are.

Standing at 80 pages, this book is small enough to be read on a lazy Sunday evening, and yet set the right tone for the week(s) to come. Recommended for everyone.

---

The Very Best of The Common Man is a compilation of drawings by legendary cartoonist R. K. Laxman. The Common Man is a caricature that depicts various social and political situations from the point of view of the layman.

I got this book due to its sentimental value - while growing up, the You Said It column in Times of India was a great source of humour on the uncertainties in political and economic environment of the day.

The book has some of the better drawings by Laxman, where he has taken potshots at the political habits of politicians, and explores the pains that a common man faces in his day to day life when interacting with the government.

The book is a good collectible item for anyone who loves cartoons drawn by Laxman, but other than that, don't expect any more from it. 

HPL/SQL: Procedural SQL on Hadoop, NoSQL and RDBMS

I admit it - I've a lot to learn when it comes to the internals of big data technologies. Even though I've worked on many data science projects at Goibibo, my knowledge of the principles involved is very limited. So when this project happened, where we needed to build some data pipelines for business needs, I found the perfect opportunity to explore the Hadoop and Hive ecosystem. This blog post is about a new tool that I came across, HPL/SQL.


Years back, the database world was ruled by relational databases, chief amongst them being Oracle. And it had this great language for writing queries - PL/SQL. From the Oracle website,

PL/SQL is a procedural language designed specifically to embrace SQL statements within its syntax. PL/SQL program units are compiled by the Oracle Database server and stored inside the database. And at run-time, both PL/SQL and SQL run within the same server process, bringing optimal efficiency. PL/SQL automatically inherits the robustness, security, and portability of the Oracle Database.

However, the explosion in information technology requirements led to a humongous amount of data being available for analysis. This gave rise to two other classes of databases:

Big data systems

Big data is data sets that are so voluminous and complex that traditional data processing application software are inadequate to deal with them.

NOSQL database

NoSQL (originally referring to "non SQL") database provide a mechanism for storage and retrieval of data that is modeled in means other than the tabular relations used in relational databases.

So depending on the problem at hand, engineers now had a host of RDBMS (Oracle, MySQL, Aurora etc), NOSQL (Cassandra, MongoDB, Redis etc), and Big Data systems (Hadoop) to choose from. 

This gave rise to a complex challenge - most data analysis earlier could be powered by the same RDBMS alone - example, businesses could build intelligence using stored procedures and triggers in PL/SQL on Oracle. In the world of multiple databases, engineers now have to learn multiple tools for data collection, analysis and so on - example Spark, Mapreduce, Hive and so on.

Different ecosystems and data analysis tools started coming out with their approaches to solve this problem. For example, connecting to multiple databases, custom SQL that is compatible with a covered set of databases and so on. However, the coverage has not been big so far.

To solve this complex problem, Hive comes with a language/tool called as HPL/SQL. The promise of HPL/SQL is simple - it is a hybrid and heterogeneous language that understands syntax of most  procedural SQL dialect, and can be used with any databases, running existing Oracle PL/SQL code on Apache Hive and Microsoft SQL Server, or running Transact-SQL on Oracle, Cloudera Impala or Amazon Redshift. So, you can use it as a language to write new procedural SQL with Hive, and you can use it as a tool to execute existing PL/SQL and logic that you have written for other databases.

Compared to writing code in Python, Java or Scala, HPL/SQL is much more friendly for the BI/SQL expert to run, and allows them to hit the ground running faster. It offers features like

• Control Flow Statements
• Built-in Functions
• Stored Procedures, Functions and Packages
• Exception and Condition Handling
• On-the-fly SQL Conversion

So, if you are exploring an efficient way to implement ETL processes in Hadoop, give HPL/SQL a try. Since Hive 2.0, HPL/SQL comes built in with Hive. All you need to do is, 

When executing commands directly:

hplsql -e "SELECT * FROM src LIMIT 1" 

When running from a file:

hplsql -f script.sql

You can go through the documentation to get more details if required.

The morning verses

I've been trying to code a website for last couple of days. Being a side project, I set this up as 2017 goal for me, and targeted developing it before 1st January.

Having been working exclusively on backend for past 4 years now, the HTML, CSS, JS of the day are much different than what the libraries 4 years ago looked like. I overshot the timeline, and at around 6 am in the morning, when exhausted after triaging one of the last few UI bugs, I needed a diversion. Thankfully, two insects and the Sun provided the inspiration.

This is what I came up with - formatting it and posting now. Happy new year 2018, and hope you enjoy this short poem :)

The night it seemed would never end
as the code it seemed would never compile,
All my brain in thinking spent
while all my effort seemed futile.

Hours it took to dawn upon me
what silly errors had I made,
The classes somehow manipulated me
for all the errors could now fade.

For this respite I was thanking god
when two insects flew into my space,
Death with a slipper was their reward 
for flying and hitting upon my face. 

And then the birds started chirping out my door
while the sun began to rise in the sky,
The dead were dancing on the floor
as I started having a morning high. 

And thus I made peace with my brain
while the peace of my mind returned again,
For the morning air started refreshing me
while the sunshine took away my pain.