Today while being on twitter saw a few tweets from some journos (@seemagoswami, @virsanghvi) about Pak not having civil society. It was some satirical tweet that u can find on their profile. Something that would surely spread the not so good message about our neighbor. My problem with that was simple… Why do you make such exaggerated statements about the country about which you really don’t know much? We have such elements (the mujahiddins, the hindu extremists, the senas) then are we to be branded the same way by Pak media?

The even bigger problem is that these stupid journos go on to make big statements for their own telecast TRP ratings. But they forget that they have some social responsibility of reporting without their own personal bias. They make the story sound so horrific that the people who consume the news without analyzing it then go to form opinions based on those blown up reports. They form a misinformed decision. I am sure their is an argument that the twitter messages are personal views. But to that I would say, it is a social media consumed by many people who trust these journos for correct information. If they are fed in with biased news then I think this is completely WRONG!! These journos leverage their celebrity status to gather their followers and then feed them their personal views without even mentioning it might be biased.

Another problem I have with this kind of reporting is, if can not bring peace to people’s life then at least don’t feed them with hate and anger or with information which will raise these two emotions. We must understand it is because of these emotions that people take to extremism and violence. Can you imagine 1 billion insane and angry people fed completely on irrelevant and biased information? I don’t want to… there is not enough arsenal with any army to eliminate those many humans. And well killing one angers 100s and going by this proportion we are looking at something TERRIBLE in the coming time. It is not all lost as yet, we can change this. We should demand for journalism and not biased, TRP motivated information for all of us. WE ARE A DEMOCRACY!!! Lets act like one.

Please consider this for all journos not just the two mentioned. They are mere examples…

Today is 31st of Dec 2009. We all are eager to welcome the new year 2010. I am very excited about the new year, as I hope it will bring me a good and interesting job.

But today I am writing this entry to review the past year 2009. 2009 started off with a party at Radhika Tai’s place in Portland and academically with the last semester in sight leading to completion of my degree. My last semester at the University was most fulfilling and enriching for me from the educational perspective. I loved the course Machine Learning and my research work turned out fine. I have been since hooked on with various machine learning algorithms and their applications to the real world problems. Personally, not everything went as planned but I would not talk too much about it here. Job hunt in the past year has been a rough one, my work as a RA at the Univ has been fun. I had to take a break in last few weeks due to an important function at home and return to India.

I hope next year will bring some zing in the markets and will bring cheer to the faces of people around the world. May the next year bring peace and harmony to the man kind. May we fight less and live more! Lets hope next year will be healthier for all of us…

May next year  be happy and prosperous one for everyone.

(WEIRD THOUGHTS ALERT: Some of these thoughts may offend you, hurt you or leave you with no reason to follow certain sects… reader discretion advised.)

This all started as a discussion with Archis Gore a friend of mine, we discussing some issues about human beings and how we follow religion and stuff. It got me started with the thought process to think and understand why did all of this come up? What must have been a purpose of such set of rules that we follow today? Rules that lead to the violence that might lead to the elimination of the entire human race?

Lets think about the time when man did not have any of these rules? Did we even have one such time? Probably we did. This all did not come form the blue, at least it does not look like. If we try to reason every rule that exits we can find at least one reason that must have been good enough to put the rule in place at that point in time. So, what was the purpose of this much hyped about species called HUMAN BEINGS or HOMOSPIENS? What did we do then? An answer that comes to mind looking at today’s world is probably reproduce and reproduce and reproduce even more…

I know it is kind of gross to think of our ancestors without brains, knowledge and act like animals. But well I am sure there must have been times like those, when we did not fight each other based on religion or may be then we did not fight at all. I think humans went through a phase on this planet when there was a tranquil and peace and then they started loosing their edge for survival, the sharpness of body and brain. It was at this point to save the human race from extinction and to improve the chances of survival we learned from the beings around us. We learned two most important things to humans, fear and anger. The two contrasting but potentially very basic instincts for survival. Then with these two came the feeling of intimidation. To award the intimidator for the being one he was awarded with some reward, something that he / she wanted. The award must have been the most vaunted thing at time. (Make a guess for yourself…) Now we come to a very interesting phase, the phase of competition amongst humans. If one thinks carefully even today this is the most important aspect of our race that keeps us way ahead of other species (at least to the best of our knowledge). That keeps us competing with each other for superiority, for the reward. The reward changed with time, I guess but aim to keep our species competitive remains still intact.

Now come to think of religion, why did we need to believe in something that is beyond humans? One reason could be we had not yet found the answer to the most mysterious question or death. Then, also to keep the ever growing power of these intimidators in check we had to believe in some super power that was beyond the comprehension of the common people. Now to avoid people claiming the closeness to the almighty and to give an identity to the community to distinguish them from the others outside we needed certain rules to be followed by the members of the community. Basically to bring order to otherwise orderless bunch of competing humans these rules were set. It was an attempt to make the race to the top of intimidators “interesting” and most coveted,  these rules were devised. The groups became jingoistic about their own rules and then started competing with others who followed other rules or did not follow any. As the number grew the stature of the almighty grew and those intimidators who revered them also grew in stature. I think these must have been the basics of our civil society as we see it today.

Now lets mix the above two things staying competitive and then following a set of rules. What we see is the so called unruly set of humans who have more rules to follow than their own populations. Who live their entire life for the simple purpose of keeping human race competitive by “improving their skills”, reproducing, teaching their off springs the same line of thoughts that they learned from their parents  and following those set of rules that they think will help them ahead of the rest of the creatures on this planet.

We are walking the path that our ancestors laid down for us as per the best of their knowledge at that time. You don’t believe me? Think about it with a cool head. Half the wars in this world are because we wanted to be superior than others, rest are due to the set rules that we follow and want others to agree that they are the best and follow them(again competition) and where are we headed? Towards a war where we will destroy each other because we want to prove to each other that we are the best. Oh! and we forgot it was for the survival of the human race. This surely is a contradiction and as would be in mathematics we would say our assumption was wrong. Can we prove something here? Yes the purpose of of human race as of now seems to be to destroy itself and they are heading very quickly towards it.

This must have been really absurd or totally crap or absolute truth for you… in any case I would like to hear back from you. Share your thoughts…

PS: These are my thoughts and I am solely responsible for them.

I have lived in the US for the past two years and prior to that have worked at various companies all of which were in the software development bussiness, mostly outsourced software development or off shore development as one might call it. Basically the term I prefer for this is cheap labor. Any company, you name it and I can bet the reason for an expanded Indian office catering clients far away, is cheap labor. At nearly 1/10th the original cost and for doing the same work. I know none of this is a news to most of you readers. Then what am I writing about?

It is a question, a question that I have asked people a lot times and answers have been varied. My own answer to this is not very pleasing to many around me. So, today I try to pose it to you all, my readers (in whatever short numbers you are, you are valuable).

It is about the outsourcing and relates the notion of pricing the product to the current market standards. If the demand is bigger than supply then price is up if the opposite then it is low. So, in this context how would you put the equation of outsourcing? Not a brain teaser, I suppose. India and China have large number of human resources, for one they don’t have one thing in scarcity and that is humans. In case of India engineers and scientist. There is a flood of them, thousands and millions graduating every 5 years. They are being added to the pool with similar skill sets, similar aspirations,similar desire to earn big bucks in small time. Now when you have this heavy flow of “Talent” (?) and that too at the 1/10 of the price, WHY WILL SOMEONE PAY THE SAME 10 TIMES HIGH PRICE?? Now come the grand question.

“IS THE JOB REALLY WORTH THE ORIGINAL PRICE WHEN THE SAME JOB CAN BE DONE IN LOWER 1/10th PRICE?”

Will the workers in the US settle for 1/10 of their salary because their job can be done at a much lower price. In fact let me put this with an example, will you buy a “shampoo” (please replace it with any product you like) from a store that employs expensive sales workers for $10 more than the lower price at Walmart or Target for the same brand and product? Whatever is your answer I would like to know.

I have the Indian side of out sourcing too, I will write about it soon. It is an employee’s experience. Till then keep your views on this one coming.

One of my friends Anand Kishore along with some of his friends at Yahoo built a nice Text summarising app “Dygest” using the Search Monkey and some other SDKs. Their nice achievement is they say they were able to write a statistical text summarising algorithm. Now I don’t have the details of the algorithm with me, but what I could see from the summary was that it was HELPFUL. So first up Kudos to you guys, Good Job. Next I was kind of intrigued by the thought how they must have done it, and that set my brain rolling and I started reading what they must have done.

Few Observations I made about their results,

1. Very well presented. , I am not good at web design and stuff so I really admire that.

2. Their results (sentences) were just too well formed to be machine generated. So, I was like where did they come from?

3. First hunch was, may be they just put out most interesting sentences.Which turned out to be partly true, their sentences are infact “picked” as is from the source text. But they put entire sentence so that it reads well, and also they probably put more than one sentence in order to make it sound coherent. They might also be doing some grammar analysis before putting the sentences together.

So these things were going in my head and I was like what would it take to pick up meaningful sentence form the text, the simplest thing. Then I set out to write my own code to do the same. I pulled from net a python script that could extract text from a url (it is not so powerful, but works). Then I took two web pages returned by Dygest for search term “Stimulus” and converted it to text using this python script. I wrote a small perl script to clean the text and build a matrix of the form “paragraph X words” and then scored them based on the number of meaningful words contributed by the paragraph. Here meaningful words are the words that are left after striping out the stop words (put together by searching some stop list online). The paragraph with maximum score is selected as the representative for the article.

I know this is really naive method of doing things, but I wanted to validate the thought I had in mind about the ability of this idea. And it turns out it stands validated. I don’t have results with me on this machine right now to put up here but will do that once I go home. Also, I was surprised as it did select some of the really good paragraphs as a answer.

There are a few variations I would have liked to try but did not, here are those.

1. Use a better importance measure.

2. Add more granularity to the text selected. I could easily go to sentence level and then show the top 3 sentences as answers.

3. Use second order context similarity for the search term and the paragraph selected. This would be really interesting but is a lot more involved and I did not have enough time.

All in all, it was fun app that it turned out. I will be uploading it soon here so keep watching this post or email me if you are in a real hurry. But again I don’t claim that the code is a top class code and is the best way to go. It is one of the way to go though. I also want to thank Andy (Anand Kishore) who’s post (Dygest) I mentioned before, inspired this act of mine.

I have been away from home for almost two years now. I talk and chat frequently with my mom and dad back home. My mom is more comfortable using my mother tounge Marathi to communicate with me and has to fight with the Roman Script keyboard that comes fitted to the computer back home. What is more annoying is despite of the fact that I am a computer science Masters Stuudent she still has to struggle, can’t I just write a small program that will encode the Roman key sequence and change it to the resective Devnagari Character? Well yes I can, but does she use the same key sequence every time to mean the same letter? Is she consistant with her spelling of certain phonemes? No she is not. Now should I make her mug up (Indan term for memorize) all the key sequences just coz she has to write to me in Devanagari script? Is it really worth the pain? (Well for those emotional types it might be, but my mom would say look kid, you want me to read this and mug it up? forget it! I will call you).  To all those of you who are by now eager to point me to Google’s Indic Transliteration web interface, my point is my mom is not great with computer and although she would be keen on doing it, it would be a bigger pain for me to explain her how to really do it. So forget it.

What do I do? Leave her to call me and not pick the call coz I am either in school or playing Cricket or cooking or out with friends? No I chose to write the code which will adapt to her key sequences probabilistically and then suggest some output of the word to select. I know this is really a long term project and needs a lot of dedication. Below is my plan of action.

1. Identify the phonemes that are formed with the script consonants + vowels and other combinations.

2. Build a probabilistic model for phoneme preceedence using the two Marathi corpus available form CFILT-IIT-B.

3. draw association between Roman key strokes and Devenagari phonemes.

4. Write up a predction algorithm that that based on the previous key strokes and the probabilistic models predict the nextt phoneme in the word.

I am through with step one. It has a small python file as code. I would love to write a C++ code but I am trying to write like a small app that can be used in a web browser hence I am keeping it in something like Python. But again I don’t know if Python will be useful. So for now I am designing and developing the algo and a sample aap as my weekend project. Lets see where we go. Feedback welcome!

This is one video form Google Round Table video series and you guys know what? They have shown a iPhone if I am not wrong in this video used for one of their maps products.

Did Google not have Android based phones then? Or they just goofed up?

Good question…

I happen to Read this article on Indiatimes.com about the Pune traffic condition and it made to the front page international edition for something quite interesting. This techie apparently beat up a cop, and now instead of putting him through the court case and stuff the commissioner of police has made him work with the same cop on the job that the cop was doing, MAN A TRAFFIC SIGNAL IN PUNE. Guys believe me it is as tough as watching the Indian borders for infiltrators from across the border. It is in the hope that the techie will realize how difficult it is to manage the traffic and then also may be have some sympathy towards the police men who work extremely hard towards managing the ever unmanageable Pune traffic.

Kudos to the cops for finding this innovative way of teaching lesson, it is the fact that lack of understanding and apppreciation of eachothers job is something that lands most of us in quarrels and arguments with other, which at times can be avoided by having this simple knowledge. BUT, Will the police send their men to a techie’s cube to sit with him and see what kind of pressure he handles, if in some (most?) cases cop is wrong? We shall see… for now it it is a good initiative.

Better late than never… thats how i first react to this new thing that comes up. Well can we change the mindset of people and make them quit it? time shall tell all that…

For now it is a good step.

Today it is not just one or few cities in India whose residents wake up worried will they see their family again? It is the whole country! Why? Did you just come form Jupiter? or You live in America? If so I would not say anything… than just read a few news papers. I do not want to drool over how ignorant American society is of the rest of world, it is a matter for some other day.

Today Indians are bleeding because the problems which we always thought were caused form outside are now being spread at will by the insiders. Yes lets accept it that, it is not any foreign country thats behind this kind of an attack but rather they are some radical sects of our own society that are doing this In-Humane act of killing innocents and trying to make their presence felt right across the country be it Eastern region of Tripura or Western city of Gujrat or the Southern City of Banglore or the Capitol Delhi. They are just planting them like crackers and exploding them at will. While I am writing it, I do not know who of those jerks is making a bomb. But more importantly what are we going to do about this situation?Are we going to look at a particular community as the people who did this? Is it really fare? Did the good neighbour that you have do it? Probably No, it is some jerks who have sought the wrong path for themselves who need to be brought on the right track. They probably have no religion, because religion is for human development and not destruction. Anyway, the point is we as a society need to rip such problems that have sneaked in to our society. To do this, we need to stand together in Unision.

I know people will say “sitting in the US it is easy to talk like this”. My only point is this did not start after I came back to USA it was even when I was in India. So the point is these Indian Mujahiddins need to understand that they are doing no good to themselves by terrorizing people, the anger of people might get them some time and that time no law will be enough to save them. Dont let the movie A Wednesday come in to a reality! The day that happens things will be quite different for the mujahiddins, it will be a no point of return.