It’s the end of the Winter 07 quarter at Stanford, but it’s also a bigger milestone for myself. This was my last day of school. Forever. (Well, there may be a change of plans in the future, but for now, it is.)
The title probably sounds corny, like I’m some middle schooler looking forward to the first day of summer vacation, but in truth, it couldn’t be any farther from what I’m feeling right now. It’s a sort of relief that now I am finally done with school after 17+ years worth of education (1-12 grade school, 13-16 college, and 16-17 masters), but it is also a jarring change – I’m now, officially, a full-time contributing member of society. Fortunately, this is what those 17+ years have been design to accomplish.
Of course, the more depressing point of view that I keep revisiting is that now I’m not the one absorbing knowledge – I’m going to have to watch it drain away as time goes by, regardless of how much I try to retain it and learn more.
As a wise man once told me, “the advancement of computer science can be measured in levels of abstraction.” Okay, I lied, no wise man ever told me anything like this, although it’s not far-fetched and not too hard to believe.
Now that I am done fibbing, I’ll jump straight to my beef. The problem with abstractions is that they naturally take a multitude of forms, because not everyone can agree on how to abstract away simple mechanisms into higher level concepts. What usually ends up happening is as the levels of abstraction stack higher, the path to a simple task becomes simpler, but we lose sight of the overall picture.
If I take the first level of abstraction as assembly code/machine code, which abstracts away physical CPU instructions into some binary representation, I can argue that this first level of abstraction is as simple as anyone will ever get. It’s just add, subtract, multiple, and maybe a divide here and there. Add in some reads and writes and you’ve got yourself the basic LEGOs of every modern day system.
Our problems start here. The next logical progression is to complain that we need higher order constructs to be more expressive and more elegant in our approach to problems. This gives rise to higher level languages like C. Our complaints continue until we finally reach the pinnacle of general expressiveness, a strongly-typed, polymorphic, object-oriented, virtualized language (like Java) having lost all direct connection to those loveable adds, subtracts, multiplies, and divides.
Next we grow unhappy with how the general approach has become so … general… so we develop languages made for specific tasks. Out pops the glut of scripting languages, PHP, PERL, Ruby, JavaScript, VBScript, Python, and [insert your language du jour here]. We live with this lack of generality because we say things like, “Oh, but PHP makes it easier for you to do [Task A] and, oh, PERL is so much easier for [Task B],” all while sounding totally knowledgeable.
Don’t get me wrong. I like choices, but sometimes I think there is just too much emphasis placed on being “politically correct” in choosing the weapons of war.
Exhibit A. I’m building a web site to track financial portfolios in real time. This project entails (from back to front) interacting with several web services exposing interfaces via WSDL/SOAP then wrapping that functionality along with some peripheral necessities into another WSDL web service that manages entire portfolios made up of sets of stock symbols. Next this web service must have a client that can translate and expose these features as an HTTP service. The HTTP/HTML documents generated are then rendered in the end-user’s browser where we would like to have all the snazzy web 2.0 goodies to provide a GUI. That’s the at-a-glance high-bit description of the problem.
Now let’s count the alphabets in our soup (again from back to front). We’ve got WSDL, so I’ll choose to use a specific implementation of XQuery (mxquery by Donald Kossman et al.) to query the web services of interest, process the XML, and spit out more functionality in the web service extensions provided by mxquery. So far I feel good because I’ve tackled a 2 front problem using 1 (non-standard) technology. Next I need a SOAP client, since browsers don’t have direct SOAP functionality. I’ll choose PHP to do this for me since, again, I tackle two fronts with one language – SOAPClient on one end of PHP and bad-ass HTTP functionality on the other end. Afterall, PHP is the Hypertext Preprocessor. Next we need to 2.0 goodness so we’ve got to have AJAX running, making HTTP GET/POST requests to my PHP. Lastly, we’ll need CSS to make sure this whole ordeal doesn’t look like something from circa 1990.
Depending on how you count, you’ll get a total of 4 to 6 technologies/concepts in that quick little sketch. Although it might seem like it while reading through that grocery list of implementation decisions, the choices for what technologies and languages to utilize at what step are not easy decisions. Remember, you want to be able to say, “X makes Y easy to do.”
The problem? That was great and all for an individual project at such a small scale, but imagine having to build companies on these evolving technologies. You want to stay nibble, so you want to utilize all the newest technologies. “X makes Y easy to do.” But, you also want to have a go-to move that you can hang your hat on, because, frankly, the only way you’ll ever be flexible enough to utilize the perfect technology X to solve Y in the real world is if you churn through employees at a high rate. This doesn’t sound like a good solution.
Speaking of XQuery, it officially became a W3C recommendation less than 3 weeks ago. I don’t want to pick on XQuery, because I honestly love some of the decisions they made with the specification – it’s almost as flawless as a commitee decision can be, but it is fresh on my mind, and it was what brought up my angst. XQuery, like all computer science, is an abstraction. It’s an abstraction built to handle the very specific task of wrangling the wild world of XML and taming it in elegance. PHP was an abstraction, an abstraction to wrestle and tackle the evolving problems in an evolving world of HTTP. JavaScript was an abstraction, likewise, to provide the richness to the browser. See where I am going?
No? Okay. Well, I never was a convincing person. The problem here is that these technologies are here to provide the layer of abstraction that is the essence of progress in this field. However, in doing so at such specific means, they diffract our efforts to generalize lower level problems into higher level solutions. Because as we tackle the problems we want to solve, we inevitably take on a whole host of specific solutions and are forced to piece them together. As I said before, this is essentially correct and great. Who doesn’t like to have rectangular LEGOs to make walls and square LEGOs to make floors? The problem arises when you’ve got a work force thats used to using square LEGOs all their lives and are now looking at rectangular LEGOs in that odd manner. Don’t believe me? Try talking to SQL folks about XQuery.
My B-Day is coming up woohoo! Already got NBA2K7 for XBox 360 and a new Dell Desktop. What else is in store for me?
This is a website built with a strict discipline of adhering to web standards and guidelines. That means everything you see here is XHTML Strict styled in CSS. Currently only screen media is supported.
If you take a look around you, you will probably find all means of technological devices that were not available 20 years ago. For example, the cell phone in your pocket, the laptop sitting somewhere near you, or that sleek LCD screen you’ve got sitting on your desk. Even more mundane, what about 1000 songs in your pocket? Was that possible 20 years ago? What about the plasma TV you’ve either got sitting in your living room right now or sitting somewhere at Best Buy’s just waiting for you to buy it?
Technology moves fast. We are finding more and better ways of staying digital, staying connected. But is that really innovation? How do you define “innovation”, and do you find it to be inherantly different from “technology”? In my opinion, innovation is something that is decoupled from technology and, relatively speaking, innovation moves slower than its more tangible counterpart.
Better. Thinner. Faster. Prettier. These are characteristics of technology. Technology makes available all sorts of possibilities to our world. As a result, we have so many more ways of doing the same thing. We’re able to make our computers run faster, make our monitors sleeker, make our batteries last longer, and make our world spin just a bit more hectically. We can now carry gigabytes in small capsules rather than megabytes. We’ve got GPUs outdoing one another almost monthly. When it comes to technology, there is no such thing as pace. Humankind is on a tear and we cannot stop ourselves from outdoing our current status with bigger numbers, smaller delays, and better devices.
But that does not mean we are innovating. All the characteristics of technology are orthogonal to the characteristic of innovation: Difference. Surprisingly though, as the pace of technology picks up, the pace of innovation is more or less unaffected. I like to attribute this to simple reasons, the most obvious of which is that cultures simply cannot adjust to a fast rate of innovation turnover. Instead, we as humans are more comfortable with incremental changes that, in the overall picture of things, lead to an annuity of innovative income. Fortunately, we are usually open to innovation as we are generally happy to embrace change when it is beneficial. However, the way I see it, innovation is not at a premium in our society. Sometimes we’re just stuck in a rut trying to make it faster, thinner, or smaller that we forget to make it different.
It’s hard to qualify in words exactly what we look for when we search for innovation – all we know is that when we see it, we say, “Wow, that will change the way I live.” If not to that extreme, innovation will general open some channels in your mind when you encounter it. With the pace of technology setting such a open canvas, here’s to hoping we’ll begin to see the artists of innovation pick up a brush.
Unless you’ve been hibernating under a rock for the past month or so, you’ve probably heard about the utterly insane performance by Kobe Bryant against the Toronto Rapters on Jan 22 – almost exactly one month ago. Scoring 81 points and leading the Lakers back from a third quarter deficit of 18 points, Kobe launched himself into the annals of sports history overnight. Soon enough, sports fans around the country (and perhaps the world) began debating whether Kobe’s 81 points at 6′6″ in the modern era was a comparably more impressible performance than the NBA 100 point record still held by Wilt Chamberlain (7′1″) from the 1960’s.
However, even more impressive than Kobe’s 81 is the fact that less than 48 hours later, the performance in its entirety was available for download from the Google Video Store (http://video.google.com) for $3.99. Take a moment to wrap your head around that thought. What if such a service existed when Wilt dropped his 100? What if such a service were provided back in 1998 when Jordan hit “the shot” against Utah in one of basketball’s most memorable moments? This technological step brings validity to a point I made in one of my October entries talking about the “long tail” of media. To sum up the ideas from that entry, our culture, encompassing literature, music, and multimedia of all sorts, is growing at such an incredible rate that cataloguing all of the data is a very real and prevalent challenge for us. It was reassuring to see that Kobe’s performance, along with almost every NBA game since the start of the service, is available on-demand for the world to enjoy, albeit at a slight cost. This is one small step in accomplishing the holy grail of human culture: a library of media comprehensive enough to store the ever evolving story of the human race.
While Chamberlain’s historic night in Hershey, Pennsylvania is relegated to forever live in the fogs of myth, survived only by the famous picture of Wilt holding a makeshift score card reading “100″, Kobe’s night is blessed by our technological advancements as to be forever encased in 1’s and 0’s, perfectly preserved for all those who wish to access it. Most importantly, the footage, the data itself, is not bound to some unknown archive; rather, it is widely available to anyone who wishes to get their hands on it. The next time someone comes along and shatters records, we won’t have to wonder about how it compares to the mysterious past – it’s there anytime you wish to recall it. January 22, 2006 won’t have to go into history as yet another foggy memory.
Believe it or not, there is a still a large percentage of people out there who are computer illiterate in the United States. Of course, nothing needs to be said about the world in general — we are far away from a completely digital world. However, I believe the problem of “computer illiteracy” is really a misconception and there are alternative angles to view the issue. The problem really is: Why should it be something so hard to learn in the first place?
Why should learning to use a computer be a challenge? Why is there even a question of illiteracy? Computers are not math equations, they’re not languages — they’re tools, and as such, shouldn’t even require the label of “literacy.” They shouldn’t require some specific intelligent insight to operate. From the invention of the mouse to the abstraction of files and folders to dawn of the web, there had always been a general trend of turning that PhD made microprocessor into a device suitable for your everyday American. Maybe its just my impatience, but I feel that this trend has faltered and stumbled a bit in recent years.
This is where I feel Apple has really stepped up the game lately. From the use of huge visual icons to smooth interface features coordinated into the “aqua” interface, the Mac OS clearly outshines Microsoft’s OS in the “newbie” factor. It’s just so easy to start clicking away with that mono-button mouse that the otherwise computer illiterate population is beginning to understand: I shouldn’t have to bend over backwards to use a computer, it should beg to be used by me. Intuitive and smart, the entire Apple package of hardware and software appeals to the average person. This begs the question: Are we witnessing the start of a new age in computing?
Pundits have given this user-friendly approach to selling chips trend names like “Home Media Computing” inferring the computer’s rightful place as a media device alongside your TV (or HDTV if you’re cool), your VCR (or DVR if you’re cool), your PlayStation (or your XBox 360 if you’re, once again, cool), and your Audio System (read: iPod). Basically, from this point of view, your computer should really be no harder to figure out than your remote control. To me, there is really no fault in that idea. If there are still people out there who can’t figure out what a scroll bar does, why should we burden the general public with atrociously complex interfaces?
This blog entry has so far been littered with rhetorical questions, but really, these are fair questions to ask. I guess I’ve taken a long path to get to the point: Apple’s hardware will continue to fly off the shelves as long as they are as simply to use as they are now. Because of how they are able to integrate powerful functionality with a fresh-n-clean look, they will be able to get the market share that corresponds to the segment of America that doesn’t believe in needing “literacy” to operate a mouse. Also, Apple’s new partnership with Intel, a household brand name if there ever was one in the high tech industry, will certainly bolster sales and allow Macs to slowly creep up in market share.
I’ve mentioned Microsoft’s OS of course, and I suppose this is where I really want to end up. My concern is whether or not Vista can one-up Macintosh. Will we see an OS that is powerful under the hood with a polished clear-coat finish? Or will this just be another complicated bore along the veins of Win95, 98, and XP? The good news is: from what I’ve seen of Office 12, every indication tells me that Microsoft is aware of the need to make power as transparent as possible. I suppose that’s good news considering the first leap a “computer illiterate” person makes into the world of computers is usually with Windows.
My blog has been quiet for awhile. I guess that’s the benefit of having a blog where there isn’t much traffic. It’s just me and my notes and there is no real pressure for updates unless I’m feeling like formally expressing some thoughts. Anyway, it’s finals time here at Stanford, and that means it’s time to draw conclusions from a quarter of experiences. (Yeah, I know temporal comments and statements make a page age faster than usual.)
All the science and mathematics that gets the attention in formal computer science sometimes just doesn’t do justice to particular area of computer science/engineer – the UI. Of course, there is the HCI (Human Computer Interaction) with their ideas and reasonings, but none of that seems to be concrete. Maybe that’s how it has got to be. The user interface, being a topic where metrics can only be subjective, should perhaps stay intangable, abstract, and, in a way, volatile.
In the end, the biggest indication of success for any project is the reaction of the audience. Whether the benefits of a happy audience mean monetary gain for commercial projects, or whatever the underlying cause, there is no doubt that no project should get that seal of completeness until it has been through extensive end-user testing and feedback. In a project happens to find itself in the public’s good graces in the factor of usability, that project is bound to find successes (financial or otherwise).
Take any number of current digital successes and it’s immediately apparent that the prior statement holds by empirical observation. In fact, it can be said that the most intuitive, most fluid, most customizable, and most understandable user interfaces will draw success given that all other factors remain constant or equal. Take the iPod and compare it with other DAPs in the market and you have one example of how UI can be the sole factor for success in special circumstances. (Of course, the other big factor is price, but let’s leave that for the economists.)
However, as the measure of a good UI is such a subjective thing, how does a programmer or developer – one who necessarily thinks in 1’s and 0’s translate logic and computation into art – into something that is beautiful in design, something that transpires engineering in numbers? Conversely, how does an artist or designer – one who necessarily posits art in some non-engineering medium help translate design to concrete development? How does a team know when they have something that is canonical in its interfacing – when they’ve created something that handles the user’s every whim like a telepath? Those who can answer these questions are certainly those creating all the magic. Those who cannot are doomed to one dimensional development. Approaches to bridge the gap between artistic design and system design certainly vary from site to site, project to project; but, we should recognize that such an approach must be explicit for any project to be successful. One thing is certain: when the perfect design is achieved for a given project, its success is self-evident. As if some magic phenomenon of optimization has been reached, the end-product begs to be used, and we will all beg to use it.
The boom around search has come in large part from the financial success of search pioneers like Google. In celebrating this newfound interest in the web, it’s always interesting to step back and look at the nascent industry and reflect on how it got to where it is now and perhaps draw inferences to where it might be going.
It all began with the first “spiders” or “crawlers” that automated the process of exploring the unknown of the web. The first crawler based engines used classical information retrieval techniques to identify documents with search phrases. These techniques used some components of the infamous “tf.idf” measure (as it is referred to in IR circles) to score documents based on a frequency of a term and the rarity of the term. These scores, being cached for each document/term pair provided a basis for the earliest web ranking techniques, but were too generous to overly “optimized” pages. Being a classical technique, standalone tf.idf was suited for older applications, where there wasn’t necessarily as dynamic an environment as the web. But because the web offered such an opportunity for anyone looking to publish content, the search engines began waging what would be an uphill battle against the spamming (aka optimization) community intent on making a buck via the gaming of search engine results.
Enter “Backrub”, now known as Google. To fight off the gross irrelevance being returned by the search engines of that time, the founders of Google looked at how the link structure of the web could be exploited to produce more relevant results. The intuition was that pages with a high count of backlinks are pages that are generally considered more popular (also known as the random walker argument), and thus, more likely to be authoritative on any given subject, provided that the page was relevant with the subject in the first place. Combined with the classical techniques of identifying key terms in a “secret sauce”, this new PageRank system proved to provide astoundingly accurate results for most queries and skyrocketed Google to what it is today. Of course, the generalized idea here is that this success was largely in part because of the way the PageRank system mitigated the local optimization efforts of spammers on their webpages.
Thus, if you look at the success of PageRank, you’ll see that really, it was an arms escalation on the part of the search industry to fight off the spammers. This has essentially bought some time for the industry, but it is very apparent that the spammers have caught up. Using a combination of tricks to farm PageRank for pages, “search engine optimization” efforts have, to a large degree, closed the gap on Google. The battle has yet to be decided, and it is hotly debated in the IR circles whether or not there is even a true solution, short of human intelligence, that can deal with the rising sophistication of gaming techniques. As one would expect, new methods are being research and developed in both the industry and academia for fighting off search spam.
But what really needs more reflection is whether or not PageRank is still as relevant to the success of a search engine as it was when it was first invented. Of course there will always be something to be said of the democratic backlink model because its just intuitive that a page is about x if everyone says its about x. But the converse is not true. That is, it can’t necessarily be said that a page is not about x, just because no one recognizes it as x. The idea can basically be summed up in the fact that because PageRank so heavily weights the importance of backlinks, it might be ignoring what can be referred to as the long tail of search. In the IR industry, this of course known as sacrificing recall for precision.
If PageRank was effective in fighting off spam in 2000, it can not be recognized as being just as effective today. In fact, it is quite clear, even to an outsider, that the issue of spam has to be handled in an entirely different manner for Google today. That said, it begs the question whether or not new models of search can be built that diminish the role backlinks play and instead focus on a “smarter” way of categorizing pages, and in one fell swoop, solve the problems of topical ranking and spam. From what I here, (and I could be totally wrong) http://www.kosmix.com/ is one such project that is focused on high recall by not necessarily letting backlink pageranking be the major ingredient of the formula.
To sum up, what is evident to me is that search has to gradually change as consumer expectation changes. And as time goes on, it is worth reflecting and re-evaluating ubiquitos techniques of the day to see if they would still be relevant tomorrow. However, I trust the guys at Google are well aware that resting on their laurels, especially in a dynamic and nascent field, is not an option.
If computing were a snake, it would have two heads. On the one side would be the business world, driving, pushing the snake toward the automation model. On the other would be the academic, focused on the core definition of computing and its mathematical heritage. As a computer scientist, computer engineer, software developer, or what have you, I often find myself conflicted in finding a unifying theme, a universal purpose for what computing really is.
Ask the economist and he/she will tell you that the two forces of our world are gravity and currency. The latter force is the one that drives our innovation, because, as fate would have it, capitalist societies encourage those who give back value in the form of money. This basic driving force for innovation is key to the technological golden age we are in today. Here, for the reason of financial gain, is where we find the birthplace of a lot of software computing ideas. CRM, ERP, Help Desk, and generalized financial software are all forms of software innovation that had risen to face the challenges of a world ever increasing in its demand for automation. Software products succeed and fail on the central criterion of: “Does it solve my problem, and can I afford it?” This is perhaps why, if you read material intended for the industry, you’ll often find it 90% business and, if your lucky, 10% technical. For example, you can ask, in a typical company, how much of the CIO’s roles are business related? How many are conversely technical? What backgrounds and experiences are important for a CIO/CTO?
On the flip side, you have academic computer science. Being research based, the primary actors might or might not be motivated by immediate financial gain. Most of the innovation here comes from a deep down passion for the subject and for a will to make a change to the technological landscape, however big or small. Unfortunately, the lack of monetary stimuli leaves much of it to the personal desires of the individuals involved. As a result, you will typically have few students of computer science interested in the theories and the fundamentals. Having much experience in this matter myself, I can tell you not many people go into the computer science degree expecting to learn what it is they learn. Where they expected a practical curriculum mirroring the demands of our external society and industry (like implementing solutions in a multitude of languages) they instead find deep mathematical theory involved in no one concrete, intended application. Even those who check “Computer Engineer” instead of “Computer Science” on their applications are typically in for a surprise. Look at each school’s degree conferal summary and you will find something like: “A degree in Computer Science from (insert school here) is a guarantee by the school that you are well prepared in the breadth of knowledge to tackle problems faced in a computer science occupation.” Is that so? Many understandably doubt it, and as a result, many people simply do not see the point and drop out.
Computing has come down to two threads of existences. One survives to automate – to solve practical problems in the business world and find financial reward. The other survives to truly further our world through innovation. While the former certainly has played a big role in our world (just think of all the big software businesses out there), the latter thread is the one that, at the end of the day, creates the great leaps we enjoy as a society today and in the years to come. Just think of the internet and its roots as a military/academic project to provide a way to decentralize computing. With it came email, and then the web. These are the truly essential technologies that even the automation world depends upon.
We have been given the gift of technology – but this gift is no promise. When it comes down to it, it’s up to the many brilliant minds out there toiling away in the land of academia to create new ways we can change computing. Perhaps an interesting question is then: can we find a blend of computing that is both viable as an academic pursuit and as an industrial success? To that, I’d have to point out that the Google boys did. They are one example of how an academic innovation can bear both intellectual gain (in the form of benefitting our society in knowledge) and financial gain. In the end, I believe that perhaps the reality is that there will always be financial success guaranteed in a truly brilliant leap in technology, but conversely, there is not always a leap in technology guaranteed for every financial success.
- LW
