Custom hardware/software and scale

Though this is a start, I think at scale, the answer to all the problems is that you can not solve high scale issues with consumer level solutions.  So ironically a thing that started with using off the shelf consumer grade hardware and software as a base and built custom solutions on top of it, again needs specialization. Or the other point is that if there is one flaw in this suggested approach, it is the fact that this approach is not universal.

http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html?utm_source=feedly&utm_medium=feed&utm_campaign=Feed%3A+HighScalability+(High+Scalability)

Most of the technologies in this area of high scalability do use standard software and hard ware at its base and build on top of it.

Every thing including big table,big data and other associated technologies do use the off-the-shelf file system and off-the-shelf operating system on individual nodes. The speed and scalability is achieved by software running on these machines by custom software which most of the times runs in user space as well as is written in high level languages (mostly JAVA).

So even when the suggested approach works, it is definitely needed to first work for the individual machines for existing software stack in consumer space before it gets really prevalent usage.

Hello world!!

My third day at EMC and it has already been a roller coaster experience. There have been a lot of things that are unexpected and hence even more pleasant. Some of the things are shocking and even when such things should be expected are irritating for the short term.
But I am here and I am loving every moment of it. right now I am sitting in a cubicle and this is a change from my earlier job. The buzz is really heartening. Occasional snippet of Kannada tell me how much I liked hearing it in my earlier stint in BLR. The Bangalore variey of the language is really endearing. This time I want to really learn so that I can understand and enjoy it even more .

This is my first post from my laptop. Something which I am able to do after three days of joining this company. I have always heard about companies where people change code,check in and roll out to production on their first day.Such companies exist. But because of the nature of the domain we exist in, as well as size of the org, this definitely does not happen here. Since joining I filled forms, submitted documents, resubmitted them, showed people signatures on different forms from HR and got different things as gifts.  A bunch of people joined with me and the joining process, though tedious, looks pretty stream lined. Nearly hundred people joined in last two weeks, everyone more or less started working almost on their second day. But you just fall out of step with process and then beware of the effects. As it happens with a stream lined process, any thing that does not fit the process takes extra ordinary amount of effort and time. And I have natural talent that such things happen with me.

But first the thing I liked the most: I liked the large number of empty spaces as well as I really dig the facade of the building.

"A very bad photograph of the gorgeous facade of the building"

 Here are some more from a search engine :) : 

"A lot of empty spaces with breath taking views"

Quality of the workspace is better than my earlier org which has really been a pleasant surprise. The coffee is better, cafeteria is better stuffed. 

But as I am lucky enough to draw all the cosmic rays of distrubance. It is a talent I have from a long time and I keep exceeding myself every where I go. The same thing happened this time also. One of the most important person you'd meet is the photo ID printer person. Also known as the key-maker.
(and as importantly hidden as the keymaker from matrix - behind the reception in the basement)
"I am still preparing your ID card"

As the disturbance in the field caused by my presence, he happened to throw away my form without actually printing it. Another interesting thing I found here is the interesting checks and balances. I have to give "form X - 100" signed by "P Q and R" to pick up a thing "Y".
After a lot of concentration I get all the pre-reqs to collect the card just to be confidently questioned on my existence in the company. Finally this got solved as the gentleman found his mistake and retrieved my crumpled form quickly from debris.

This cool, confident and kind smile has been the major companion of my interaction with the non tech staff till now.

Even the emperor at IT,  placed appropriately in the farthest corner on the top floor showed that. With the same kind time he told me "you will get your machine in one month". I was shocked. I ventured: " Did you say one day" - *smile* "One month". "Did you mean a week?" - "One month".

"When I say one month, it is one Month".

But then I talked to the HR and admin and things worked out pretty well after that. Now I have a machine and a team and looking forward to my first interactions with my team and especially looking at the code :).

So with some interesting twists, my rendezvous with EMC is well and truely on its way :)

The problem with activism/criticism in general

As passionately as this is put and the lady also has some very interesting points, it is really interesting to study this from meta point of  view.

It more or less looks like the lady has issues with every thing. Nothing that exists is right. Things have been going wrong for centuries together,
and they are still degrading. Generally this is not some thing you would want to pursue. Even when it is not optimum, human beings have been surviving and majority of it is flourishing.
Especially it is post-reform India. She some how comes across as trying to make a point that green revolution was not a success. This in itself is a point of debate. I know that it definitely did have its own disadvantages and side effects. But that has to be a separate piece of activism with good time spent in elaborating the stand and nuances. By not focusing only on seed freedom, she is deteriorating her other wise rather strong position.
This is some thing atheists also fail at. Actually in my opinion, their point should actually just stop at questioning an omnipotent deity/afterlife. They are the m,most successful when they raise doubts and prove religion wrong. Giving an alternative belief system/lack of it and the framework for it is really the next steps.
By just concentrating on the attack on theism, the individual is freed and then starts thinking on his own, which is a great win. But by just trying to focus on even bigger wins, in my opinion even this advantage is lost.

        Then there comes a time when the activists' win their point and the opinion is with them. Now is   the time for the next generation where the solutions and alternate structures are proposed. Most of the times it is done by the next generations or is done by the activist in the next phase.

So from my pulpit, the sermon for the day is: "you should fight your limited battles with full strength, rather than cover the whole spectrum and give complete alternate solution".

Only the liberals love their country ...

Some of the things that are mentioned here are universally relevant.
Actually short cuts and corruption seems like a human tendency. This is basically an agency problem.
This basically means that when ever you delegate some thing to another person or the group of people, they would always have their interests first.
If this involves subverting the original goal, then they would do that as quickly as possible. Running a democracy for the people is one such thing.
Here is one such try to alleviate the agency problem:

This is really interesting to compare this situation with my country, India.
Here things are at the same time better or worse from different angles. One thing is that more and more the separation between the funder and fundee is going away.
This means that the politicians themselves are wealthy enough that they need not think about life after their time in the legislator. In fact politics is a life time and most of the times a family business.
The impact of big businesses and corporations is not as much as it is present in US. Here the politician is inherently strong and in most of the states, both opposition and the ruling party keep switching places. This means that it is a fight between the equals and people get a say at least a few times or just by chance.
But a true republic is still an utopia. It is not achievable so the problem that is mentioned in the above talk is not solvable.
We can only go as near to the ideal state as possible. As this is a continuous process and just a state of mind, only way it can be achieved is by appealing to the emotions.

I completely agree with the goal though.

Country is the best way to get people together and work for their benefit, democracy is the best way they can be governed, a true democracy is always a republic.
And following transparent and small funding is a very important (though a very small) step towards this state !!

(Exeunt stage left, end act3 scene1 ...)

Fri 5 April is going to be my last day at Microsoft. I did have a blast in last 7 years and enjoyed every moment of it.
From this point on, this gets horribly cheesy and sentimental. I do not have any advices to give as well as any confessions to make or even axes to grind.

So the sane would stop here and make and keep peace ;).


Ok, so you were warned:)
So long and thanks for all the fish!!
The time I spent with MS has been one of the most productive part of my career and life thus far. I have worked with incredibly smart people, picked up large number of technical and soft skills..
Spread myself across different teams with small hacks of their own in the culture. Saw release cycles shrink and methodologies change. For example, my first  release was SFU in Vista which was an OS release maybe after a 6 years or more gap and now we are talking about quarterly release of features. Did stay back for nights together, pulled all nighters with "this is the end"(doors) playing in the back ground as well as as well as spent time mindlessly watching youtube videos of best stand up comics to while away time.
It was not all at my desk either, I reached my peaks as a volley ball player :), changed teams, had regular fights disguised as design discussions and always made a point to ensure that every one knows my mental age - all of five years.

Talking about work life balance and astounding feats of efficiency:  I also added ten plus KGs, bought an apartment,my two kids were born during this time.

But through all this, I always knew that this will end at some point in time.but I knew that my stint was MS was always supposed to end. The days flew  by and the time to make a decision has come. The reason more or less being this: :)

Some lines from one of my most fav Hindi film also explains:
http://www.youtube.com/watch?v=tCWonHf6SvI&feature=youtu.be&t=45s

But I never thought it will be this sudden and will get over so soon. It looks just yesterday when i got down at Hyderabad airport and the heat burnt me n the face (BTW this is some thing that I won't be missing. Do enjoy your time in the sun ppl ...). But the call of the wild beckons, my clock has run its time and it is time to force reboot my machine again as it is running slower and slower.  The building and furniture being younger than you is a definite sign and I am responding to it :)
Though this looks relatively simple thing to do, this is dicey and destabilizing enough move for me to hear the ringing in the ears on and off, and I need all the good wishes as I can get.
So fair maidens and stout hearted codermen of MS, fare thee well ;) See you again in the after life. It will be an honor to cross paths with you worthy folks again. It has indeed been a privilege. 

Agile design and development in large orgs - some practical fallacies

Traditionally the agile story says that the pigs make all the decision (http://en.wikipedia.org/wiki/The_Chicken_and_the_Pig). But this step is really hard, especially for the organizations where hierarchy exists.

Apart from this fallacy, one more thing that is left unsaid is the thing where we decide when to stop or when to make decisions. It is really an art to close on some thing or let some thing remain open. I think the addition to the eight fold path(http://en.wikipedia.org/wiki/Noble_Eightfold_Path) has to be right timing.

The real deal is when the decisions are made by committee and by hierarchy. but the worst of it are decisions made by hierarchy and enforced by committee.

This is the place where sunshine never reaches and middle management thrives. The places where decisions needs to be made are pushed to the top. Which gives the top execs some sense of ownership and control. Also such decisions are easy to force on to the teams which are actually executing it. where as the control remains with the middle management. Who are most of the times just chicken (all the puns intended )

I think the  problem is systemic and all middle to large orgs can not run a truly agile project. This is because the definition of agile and large are almost opposite to each other.

Are there any stories which prove me wrong? The caveat is that being agile is not equivalent to being successful, neither is true the opposite way. So is the case with being quick. you can be quick and still not be actually agile - follow all the points mentioned here : http://en.wikipedia.org/wiki/Agile_Manifesto#Agile_Manifesto