I often ask myself what technology means to our lives at a fundamental level. Is the human thirst for ease, innovation, and knowledge a never-ending quest? Is computer technology, especially software technology, a means to some fundamental end necessary for the growth of humankind?
Our current line of progress seems to indicate that our global society is obsessed with making data available faster, more transparently, and in readily available forms. To understand our problem, let’s take a look of the root of knowledge collection and our advances in this area. Take the digital camera – a device with which our world can now capture visual data and translate it into persistent binary – and compare it with traditional cameras where the output is a physical object with all the negative properties associated with it, like the properties of decay. The gain we’ve had in the “language of binary” is that data forms are readily transferable and forever preserved. These advantages and the impacts that these advantages afforded by digital media have on our society are probably not immediately apparent to most people. With time, I think people will realize that the amazing rate at which digital media aggregates in our modern civilization yields output to rival our physical ability to archive this everlasting data form.
I’m confident that no official estimate on the volume of data that people put into our networks has been measured. In fact, such a measurement might not be possible. With the increasing popularity of data collection devices like digital recorders, digital cameras, camera phones, PDAs, smart phones, even sensor nodes, etc. it’s easy to imagine our collection of data growing exponentially as time goes on. Even the minute fraction of digital media that gets published publicly is a great deal of data moving into our human collective. As this data is pushed into the web (humankind’s ad hoc Library of Alexandria), our technology must scale to sort, index, analyze, categorize, and archive the never-ending stream of input.
“But isn’t the situation controllable for now?” some may ask. After all, according to studies, the rate at which web pages are appearing is slowing down almost to a point of manageable convergence. Well, the truth is, while textual HTML pages may be growing at a manageable rate, other richer media forms are emerging as the mainstream content of the web and the net. As these media forms, like music and movies are immortalized in the prism of binary, our society is burdened (or blessed) with the duty of maintaining the work of all of human history. For now, this duty seems to have been picked up by the oh-so-popular search engine market. Companies like Google have understood the fundamental role that software technology plays in humankind’s everlasting quest for organizing knowledge.
And yet, to think of this possible future granted to us, it’s almost regretful that we’ve already lost so much. Think of all the knowledge that humankind has lost up to this date for the trivial problems of physical space and physical delicacy. Think of the books or articles that have been unpublished up to this due to economic reasons. Think of all the art that has been lost due to lack of appreciation or lack of interest. Then think of how our digital network affords authors and artists of today the many ways to preserve their works, from the insignificant to the needle in the haystack.
Fifty years from now, due to our recent well-treatment of digital media, I may just be able to look up “Friends” season 5 episode 4 on some media network and be offered the chance to buy it a la cart. But in order for that to happen, our society has to realize the importance of this data and its challenges in space and time. Fortunately, it seems that this is just what is happening on the hardware (space) front, as hard disk technology advances to the point where we can stuff gigabytes of data into a few square millimeters. Even as I write, scientists work diligently reducing the space/size requirement evermore. Going back to the digital camera case, can consumers continue to be offered free space for their ever growing collection of photos? The size at which mailboxes grow for Gmail accounts gives great hope – it seems hard disk space technology is outpacing our ability to produce media for now. But on the software (time) front, our media indexing technologies are lacking. There is no standard for media indexing to speak of, other than what search companies deem to be a reasonable web indexing schemes. These textual schemes like the popular “inverted indexes” were not designed to handle richer media forms, nor are the many algorithms that crawl for data. Instead, we have to turn to companies like Apple and products like iTunes who have a vested interest in only providing spaces for the most popular tunes.
The “Long Tail” of media grows ever longer as we speak, and it is doubtful that these companies have the spirit to keep up. So really, the question and challenge remains: as our space and media grow, can our software keep up? Google’s PR rhetoric seems to be optimistic in this regard, but I fear the perfect solution has yet to be discovered. - LW