On Idling Around

Why, oh why is it that I'm wasting the present moment? From where has this nature to idle around come from?

Although to be fair, I've been idling along a lot since childhood. And I know I'm being really fair since I'm writing this from a geographic coordinate and altitude that's exactly where I idled around a lot in my early childhood (subtracting any tectonic plate shifts of course). I'm talking about my native home in Kerala, in my bedroom, from where the sights of coconut, banana and papaya trees full of papayas hasn't changed one bit. What has however changed is the fact that I now have access to a fast internet connection.

What's also changed is that I'm now educated enough to use this internet to gather data, research papers and potentially* do world-changing work. Except that my mind has not warped itself to this new reality and instead prefers to take comfort in the soft warm bed and blankets, watch the monsoon drops hit the mud-tiled roof as the monsoon clouds make it dark, turning the green surroundings into a shade of dark green.

The heavy rains also flood into me countless childhood memories – that of cricket with cousins, collecting mangoes from the nearby tree, make a hole in them as we'd squeeze the pulpy and sweet juice into our mouths, as our cuffs become stained in yellow. How exactly am I to break away from this vivid, colorful image and study probability instead?

For in it there's a beauty more profound, but harder to obtain. She expects you to pursue her relentlessly while herself showing not the slightest interest, hard to flatter and turns down all attempts at gaining her hand. Although internally, she enjoys these nerdy suitors spending hours to please her and can't help herself from giving a faint smile, a slight blush now and then. This sometimes catches the eye of the suitor who's overjoyed, starts pursuing with the fullest might and not before long, inexplicable beauty is within reach!

What's absolutely essential to appreciate this subtle beauty is to turn one's head away from the mundane to the magical. To turn away from the various BuzzFeeds of the internet and instead surround oneself with activities involving mental effort, so that some learning happens in the perceptrons within! In essence, it's time I realize that unlike in my childhood, I stand to lose something if I idle around – the more long-lasting joy of cracking problems! This childhood artefact will have to be cleared out to have any hope of gaining her hand!

*the word potentially is among the most abused words in English

Life, Perception and Exponential Functions

Motivation to work is important. As we move forward, the sources from which we derive motivation changes. In childhood, to do well in a set of exams is all that's required and motivations were simple - the rod in early childhood, some delight in learning and competition in later stages. However, in college, there's usually no fixed goal to pursue. One tries a bunch of things, realizes the sub-field closest to one's calling and proceeds. Even so, in undergraduate college, things like maintaining a good GPA provide some form of a concrete goal.

But the cocoon is broken as one leaves for the next venture, be it a job in a company, a startup or graduate studies. One enters a stage where the goals to aspire for and the means to achieve these are no longer well-defined. Here we typically have to work with the perceived rewards in mind. And so I make the argument that our ability to perceive these rewards is detrimental to success. It is here that I believe some vestiges from evolution lie in our way.

Isn't it surprising that sounds are reported to us as Decibels instead of in a linear scale? It turns out that we, in fact, perceive sounds in a logarithmic and not linear scale. This phenomenon is not restricted to just sounds. It carries forward to all forms of sensory perception, memory, and our spatial sense. Young children, when made to place numbers from 1 to 10 on a scale, place 3 at right about the middle. Since log 3 ~ (log 10)/2, when seen from the logarithmic light, it's no longer as surprising.

In a recent work, Lav R. Varshney and John Z. Sun, using some ideas from information theory, attempt to explain this. Their work shows that logarithm as the quantization function for sensory inputs produces the least distortion if we assume sensory inputs to follow a power law distribution. In practice, many natural inputs follow power laws such as frequencies of the English language. In plain language, logarithm allows us to best store and process information. To put it less technically but at all loss of rigor, our ancestors benefitted more from knowing if they're facing one or two lions as opposed to knowing whether there are 99 or 100 lions.

Now when it comes to long-term rewards, I claim that we perceive them to be at best linear in time. A simple thought experiment proves this. Suppose we were to pose one the question, "What technological advances do you predict by the end of 2100?". The intuitive way of answering this would be to provide a figure by accounting the progress that's happened in the previous 100 years. However, a more critical analysis reveals that since innovations happen on top of existing innovations, it's more likely an exponential curve. Indeed, history suggests the same. Take a man from 1000 B.C to 500 A.D. and he might see what he expects. But on taking him from 500 A.D. to 2000 A.D., he's bound to be held speechless for a few days at the least and permanently at the most.

Also, why else do we express such surprise in rags to riches stories or glance at that super successful multi-billionaire start-up kid with smoldering jealousy? The famous 10,000 hours rule, that one can become a world-class expert in a reasonable topic of choice, by putting in 10,000 hours of effort might, in fact, have a lot of truth to it. Progress always happens in baby steps. In steps of 1,2,4,8,16,32 and before you know it, the emperor runs out of money.

In a similar sense, seemingly meaningless investments suddenly start to give results. In a venture like a PhD, where one's success depends very much on the extent of knowledge that one accumulates, this becomes crucial. Since knowledge is built upon previous knowledge, it's reasonable to assume that the utilities of studying a certain topic also grows exponentially.

After reasonable digression, getting back to the topic at hand - motivation, one strong motivation for writing this post is that it, in some ways allows me (and the readers) to realize the value in seemingly inconsequential knowledge quests. What we have is an exponential reward function f(t), which we see as a log-linear g(t). Exponentially large rewards remain within our reach if we only alter our perception!

But Really, Why UC San Diego?

Choosing a grad university and advisor should be given about one-fourths* as much importance as choosing one's life partner. It's a significant decision since it involves investments of thousands of hours in time, the study happens in one of the most exciting periods of one's life, it's also the last and arguably the most significant period of focused study.

With this in mind, perhaps the last post ended too quickly. Now that enough time has passed, I can, from my current vantage point, present the big picture of my decision making. Hopefully, a picture that's big (or small?) enough to be interesting and yet covering all the essential details. Very often, coming up with this 'right' level of abstraction is quite the challenge.

This is a goal that's sought after in all scientific disciplines, quoting Einstein "Everything must be made as simple as possible, but not one bit simpler". I've always had an interest in such simple, elegant and fundamental ideas, with repercussions that are wide and closely coupled to our everyday world. While in high school, this made me develop a special interest in physics. Concurrently, I also developed a certain disinterest in mathematics since all it seemed to bring to the table was a set of tools, but with no direct repercussions.

However, these impressions were reduced to 'bits' as I came across Shannon's information theory in college. The theory, which quantified the all-pervasive notion of information into bits (0s and 1s) and which helps understand the fundamental limits of compression and reliable transmission of information, was simply marvelous. As I explored further, in certain problems, I also came up with strategies and algorithms which attain the holy grail - the information theoretic limits. I found this to be an exercise like no other and thus, through information theory, I discovered just how closely coupled to everyday life fundamental notions derived purely from mathematics can be! I also realized that my real interest is in cracking questions which are more fundamental than applied.

Now, over the course of my studies, it's the boundary between CS and EE that I've found to be the most interesting. Algorithms, learning theory, compression, coding theory and estimation (aside from information theory) are topics that I liked the most. Of these, learning theory is of special interest to many lately. With the large quantity of data and computational power at disposal, it's of interest to a large number of people about what can be inferred from this data. Many researchers from traditional disciplines have been shifting to this area, carried away by this wave of funding interest.

And since it's always a good idea to follow the trending areas of funding interest, when I identified a certain group which works on fundamental problems in statistics and learning theory and who have been very successful in their research, with the advisor having special interest in my most favorite part of information theory - compression, and with an excellent track record of graduating students in the group, no further questions remained! Needless to say, I'm very excited about what lies ahead!

*Might correct this a decade later; will be interesting to know if I was right about this one.