Tuesday, April 5, 2011

Rescue of data miner


Recently I was called to assist one of the junior data miners at one telco company.  Junior was buried under an avalanche of data. Breathing heavily, she seemed like planet was resting on her. Model that she tried to build had thousands of variables and millions of rows.  Her problem had a more potential solution than the Milky Way has planets.
Smoke was coming out of her quad processing computer, and I was fearing the worst.
There was no time to be wasted. It was only one solution, and that was to go back in time and start fresh.
So, we unplugged computer and left a room. She desperately needed some electrolytes in her system.  
Once, she was back to okay, we started our trip from beginning with slow steps.
Just two of us, with the pen and paper.
Soon, there were more people… and more information.
This time we understood better where we were, and where we wanted to go.  And we knew why are we going on that trip, and what will await us when we get there.
Soon, we also found out that there are many things that we could carry on that trip. Too many!
Then we call it a day.
The next day, we met again.
And it told her, that we need to make our trip lighter and more mobile. More flexible.
So, we decided that we will drop a few things, combine the rest, so that our step as light as the butterfly wings.
This time we wanted no avalanches.  
So, when we started the engine, plugging back her computer – she was very surprised to realize that we were already half way to our destination.
The rest of the trip was most enjoyable, and we managed to get to our endpoint sooner than we thought.
What was the difference this time?
Starting point was the same. The endpoint was the same.  The reason for the trip was the same. Means of travel was the same.  It was only one difference!
The route we took.  

No comments:

Post a Comment