I ran across this University of California, Berkeley study while doing market research. According to the study, mankind produced approximately 1.5 billion gigabytes of unique information in 2000, or roughly 250 megabytes for every man, woman, and child worldwide. Put into perspective, this represents roughly 150,000 times the amount of information housed at the Library of Congress, the largest library in the world. Furthermore, the study projects that the rate of information creation will continue to balloon, doubling every year for the foreseeable future.

The study also illustrates a trend the authors call the “democratization of data”–a vast amount of information is being created and stored by individuals. According to the study, approximately 55% of all hard disks are installed in desktop computers, and the sum total of information being generated by individuals is 2,600 times larger than that published in print or digital form. In other words, not only does the average person have access to unprecedented amounts of information, but the average person has the capacity to create unprecedented amounts of information themselves. Furthermore, with the advent of the web, the average person now has the capability to publish that information to the world.

Unfortunately, the tools we use to manage all that information are hopelessly outdated. The concepts that underlie the Personal Information Management tools that are ubiquitous today are products of the 1980s, when personal computers didn’t have hard disks, there was no web, and the total amount of personal information to be managed counted in the few hundred kilobytes.

It’s no surprise, then, that it’s virtually impossible to find the information we need when we need it. Ideally, it would be possible to browse through all of the information in our lives within a common framework, to visualize how the various pieces relate to each other, and to search on a topic of interest and receive all relevant results, not just the subset that was created by a particular program or that is stored in a particular place. With current tools, it is becoming increasingly difficult to look at the collective whole and see the big picture. Given the sea of information overload, it is easy to understand why we so often ask ourselves, “Now, where did I see that?”

Today, with gigabytes of information on hard disks and terabytes more information on the web, our antiquated information management tools are hopelessly insufficient. Although the amount of information in our lives has increased many orders of magnitude over the past twenty years, the tools we use to manage that information have barely changed. New tools to manage the information overload are badly needed. Yes, there’s Google, but that’s only a partial solution. Indeed, it’s often much easier to find some obscure fact that it is to find that email I know I received last week. The notion of “personal information management” needs to evolve; we need personal knowledge management solutions that allow us to quickly and easily capture, organize, recall, and analyze information, and ultimately, transform that information into knowledge.

P.S. – Yes, I know about Chandler. Their vision is on the money, and I’m following the project with great interest.