Web 3.0 : Under Construction

What will Web 3.0 have in store for us? The most educated guesses will be short-sighted, but there are a few tantalizing hints from the bleeding edge of Web 2.0 development.

Think of the web as an ecosystem, a constantly evolving, organic and volatile entity. Just as real world ecosystems have eras, the web too goes through phases. The first phase was the e-era or the dot com era. It became commercially viable to offer content and services online, and a big bunch of companies that were known primarily by their internet addresses came up. This stage of the web was also known as the “bubble”. Web 2.0, came into existence roughly in the second decade of the internet, and this was the “cloud” stage. User interaction, social networking, and user generated content are some of the hallmarks of Web 2.0. Right now, we are on the verge of another era, moving beyond what Web 2.0 has to offer. Tentatively called Web 3.0, exciting times lie ahead, where the web will get more firmly ingrained as a part of our lives. Most of the technology used in Web 2.0, the wikis, the widgets, and the APIs, were around on little used sites, and on design boards long before the onset of Web 2.0. In fact, some of these ideas were around since before the internet itself came into being, let alone the web. To get a clear idea of where the web is headed, it is necessary to have a deeper understanding of the technology that drives it. On the way to Web 3.0, users have conquered massive digital divides, survived a massive bubble burst, and even endured a full fledged war.

Laying the foundations

The web is made up of many layers of standards and protocols. Assume ten computers are connected to each other, and needed to communicate with each other. The simplest solution was in the wiring itself. Ask for something in a remote location, and you got it as if you were navigating to another location in your own memory. If you have ten clusters of ten computers each, depending on the wires was not feasible. The answer to this was to make each computer a host, identify it with a number, and allow for navigation using these numbers. This was a protocol that laid the foundations to the internet, and was called TCP/IP. These protocols themselves have versions numbers, and currently version 4 is being used. The next generation will be version 6 that will allow for more computers to be interconnected. The number of digits available in the original standards have been exceeded by the growth of the internet. Now you have hundreds of computers, connected to each other, each with a uniquely identifiable number on the network. What then?

Each computer, or host, on the network, had something to offer, usually in the form of data. One solution was to give access to documents in whatever format they were created in. This would, however, cause a lot of problems. For one, not everyone could read all the formats of all the documents. This would mean that every user would have to install a large number of document readers, which was not practical at all. Moreover, this would heavily eat up the limited bandwidth available. The other solution was to represent all the data in a common and simple language, as close to plain text as possible, and have client software interpret the data to display on screen. This was the approach taken for the web, the client software was a web browser, and the data was represented then in HTML. Hyper Text Markup Language was an innovative answer. Tags owe their existence to these early days of the web. Each line in a plain text document was tagged in a way that allowed a browser to interpret the tag, and display the text attached to it, on screen. This was a simple, and robust enough system. Anyone could set up a host, write in a few lines of code, and offer their content and information online for the world to see. There were however, a few challenges at this juncture of the internet.

Bridging the great divide

The first challenge was to get people online at all. The network of computers had to spread slowly from continent to continent and country to country. Phone lines were initially used for this, soon to be replaced by a massive grid of fiber-optic lines. The next challenge was to standardize the HTML language. This lead to the browser wars, where Netscape, and IE would render the tags slightly differently, and sometimes support tags that other browsers did not. So if someone who created a web site wanted text to scroll across the screen in a marquee, one browser allowed users to view it as such, but the other did not. The browser wars have toned down a bit now, but are still an ongoing source of irritation for the users of the web. Another important breakthrough necessary was something to make sense of, and consolidate all the information on the web. Imagine going online, without Google, if you can. That is how the web was for quite some time. Without a search engine to direct you to the information that you wanted, the web was a meaningless heap of content that could not be navigated. The only way to get anywhere before search engines was through portals, that heavily selected the direction of your surfing. However, crawlers that jumped from blue lines to blue lines, and indexed all the information on the web came into being, allowing for a window into the wide and complex web.

The web back then was basically for information. You wanted to know about something, you went online, searched for it, browsed through it, and logged off. The users were spectators, without any real active participation. People who generated content did so on their own home pages. Email forwards had their day, one of the few ways that people could actually reach out to there people and say “hey! check this out! this is interesting”. We could have actually used 1GB mailboxes back then. Success stories such as Usenet and Napster were few and far between, and soon died out for various reasons. To post images online, a few tech-savvy users relied on software that churned out HTML web galleries. A few services like Geocities and Angelfire allowed users to create and upload their own content, but this too required an understanding of HTML, and how hosts work. For a good ten years, this was the web, but dramatic changes were in the offing behind the scenes.

Ward Cunningham was working on a system that allowed users to edit databases, and quickly change and generate content. A Harvard kid with a laptop called Mark Zuckerberg was contemplating comparing his friends to farm animals, and have his buddies vote between the two. A company called Ludicorp was thinking of ways in which users could chat and pool photos at the same time, while playing their online multiplayer game. Pyra labs was working on a setup that allowed users to easily create and generate content in the form of online journals. These ideas were dormant for some time, and needed some kind of a catalyst. Something that fundamentally shifted the way the web was used. That came along, as a major reality check for the web in the form of the dot com bubble burst.

Handyman services

Web sites could no longer set up interesting portals, or put up content and expect to earn money. The focus shifted to getting users actively involved in the process of content generation. Wikis, so far used for side projects and coding communities, emerged as a massive political statement in the face of online reference web sites such as Encarta and gave them a run for their money. A simple method for editing, and creating content ballooned out of proportion, and became the de facto authority for gaining information online. Yahoo! bought off Flickr from Ludicorp, as a platform for sharing and pooling photos, and Google came up with it’s own Picasa Web as an image hosting solution. Stumble Upon and Digg, both small services, occupied the same space as the search giant Google for being windows to the web. Facebook spread from Ivy League collages in the US to the general population as a powerful platform for social interaction. Blogger was bought out by Google, and became a platform for everything from meaningless meandering to citizen journalism, an independent source of news and ideas that soon became mainstream. YouTube became the web’s equivalent of television, offering almost any kind of video content on demand. Recently, Twitter emerged as another powerful platform that has managed to achieve so much by encouraging users to post so little.

Can you spot something fundamentally common to all these services? Not just the fact that all of these heavily depend on the users to create content - user generated material on Facebook, for example, can hardly be called content. These are not just web sites, these are platforms, and can be consumed in more ways than simply navigating to a particular URL. The key words here are tags, feeds, APIs, widgets. You can write applications that interface with the wealth of data in Facebook, Twitter or Flickr, you can navigate through the content on Stumble Upon and Blogger through a feed reader on your desktop, you can post videos from YouTube or songs from Grooveshark in your blog’s sidebar by copying some lines of code.

Many Twitter users have never seen the Fail Whale graphic of Twitter, and have never even gone to Twitter.com. To them, Twitter exists only on their mobile phones. The possibilities of how to use these platforms are not limited to what the creators of these web sites could think of. They are open to the world, to do what they want to do with them. The technology driving the web is no longer restricted to browsers, or even web pages. HTML already feels like stone age technology, because of how much the fabric of the web has evolved within two decades of the web’s existence.

Rewiring the web

JavaScript, AJAX, LAMP, PHP, Flash, Silverlight and Air are the most common technologies used in Web 2.0. These are not merely improved ways of allowing web browsers to display more eye candy on screen, although this has definitely happened. Instead, they are ways and means by which users can interact with the web better. Most of these services form another sphere, a content rich network in its own right, another layer above the web itself. The Blogosphere, the network on Facebook, and the Tweetosphere are not named because of some kind of fad or to better promote these services. These are real worlds, changing the social, political and economic landscapes. The web is no longer made up of static pages of information. Instead, the web is now made up of dynamic platforms and applications, which are two different things. There are many, invisible ways in which the web has changed. Creating, designing and maintaining web sites takes much more time, effort and money now, than it used to. The process has become much more complicated instead of simpler, and the end user is often oblivious of these shifts in the way sites and services on the web are conceived.

Think of an application that you run on your computer. Every time you click on something, the computer checks the memory for further instructions, and changes the screen according to your input. On the web, this was a different story. Your input data would be compressed, transferred to a server where it would be processed, and thrown back at you. Web 2.0 uses a bunch of technologies to reduce this difference between applications run on the desktop, and applications run through a browser. A portion of the processing is done on your machine, while another chunk is done at the server. However, users no longer have to wait while a set of instructions are sent to the server, and can continue to use the service. Instead of the entire page refreshing, portions of the same page change now, once a web site is loaded. Online image editing applications such as Pixlr, or services such as Google Maps are examples of this shift. You might have noticed how search suggestions on Google change in real time while you search. This was something undreamed of in the original implementation of the web. Web 2.0 is about now - what is trending right now on Twitter, what is happening right now in your friend circle on Facebook, what is the buzz in the blogosphere right now. Things have started “happening” online. Another major introduction is the use of third-party applications with a platform. It’s fairly simple, for example to make a slideshow screensaver of all the images on Flickr with a particular tag, and scroll text of Twitter or Blogger entries of the same tag over these images. Open APIs are not only a way to encourage more users to get personal with the web service or application, they are a way to use the pool of content in new and interesting ways. Another shift is in how much Web 2.0 allows people to collaborate on projects. Web sites such as Jamendo, Sourceforge, OPsound and Vimeo allow users to create, share, distribute and remix music, videos and software. This is where the web stands right now, and this was the approach desktop software should have taken long ago. So, what’s next?

The web has a few shortcomings, which will get amplified rapidly now. Searching for Web 2.0 sites is already a difficult thing. A search engine is made to crawl hyperlinked pages, and throw up the answer. Almost any Google search now gives you a Wikipedia page as the first or second link. Because of the way Web 2.0 has evolved, we are, or soon will be, back to where we started, in the pre-Google days, without some kind of a search engine to make sense of the web. Spiders are not designed to crawl content-rich sites, go to the deep back end of web-based applications, because the web is no longer made up of mere hyperlinks. Consolidating a database of web services is the next big challenge for search engines. There is only so far that the tags and metadata of today can take you. Internet search will have to undergo a radical change to keep pace with Web 2.0. In fact, what it means to search the web itself, might change radically.

Plans for the future

The next generation of social networking will probably use the current services as the back-end. The “front” will have to consolidate the web activities of users across a wide range of web services. Say a common service for updating your status on both Twitter and Facebook is one of the most basic functionality to start off with, something that a service called Plurk already offers. The next step is auto-updating the status message based on the current activity of the user. To a limited extent, there are applications that say monitor your activity on Last.fm and re-submit the scribbles as a status update on Twitter. A new generation of customizable middlemen, engines that let you take feeds, remix them, and feed them back to another service, or the same service are in the offing. Again, bots on Twitter do this to a limited extent. Swearbot, for example, monitors all instances of swearing on the Tweetosphere, and reprimands the swearer for using strong language. Flock is a web browser that lets you plug in to Orkut, Flickr, Facebook, YouTube, Twitter and other Web 2.0 services from one easy interface. Much more of that is headed your way. Theorists are talking about a “lifestream”, a steady flow of data of what you are doing online, used by a multitude of services, for a multitude of applications, but the lifestream itself being invisible as a whole. Another important line that will be breached is the difference between online and offline applications. Web 3.0 applications will fuse seamlessly with desktop applications in a manner that there is little difference between the two. Your data will be accessible from any computer, and you can drag-drop content from the web to your system folders and items from your desktop to say, an attachment in your email. Google Wave, will offer some of these functions when it is released to the common public. Another important line that will get blurred, but over a period of time, is the difference between using your computer and your mobile phone. The interfaces are pretty low-key even on high end mobile phones with customized applications for Web 2.0 services, but these will get better. You can expect to be connected and online all the time, and use the Web 2.0 in a more personalized way. A large role will be played by the search engines of Web 3.0.

Search engines will have to become much more smarter, presenting portals based on the search term as their result, instead of a set of pages. For example, if you search for “India”, the search engine should throw up a page which has snippets from wikis with a different focus (say travel, encyclopedic, and pop culture), feeds from recent blog entries, tweets and plurks, as well as photos from different pools. Possibly, airline tickets to popular airports, festivals in the country, and recent news from the location could also show up. The specifics of this would depend on user preference and history. Additionally, the content will have to change depending on whether the search is made from within India our outside. There are many more demands that users will have from search engines of Web 3.0. If you walk into a store, click a photograph of something on sale, and search using that photograph, the search engine should be able to throw up prices on nearby stores, and directions to the cheapest location of sale. The back end should be pretty robust, so as not to disappoint out of the way requests by users who really need it. Say, a person spots a great looking city while flying, and photographs it from the air, the search engine should be good enough to resolve and identify the location using a pretty low-res mobile phone camera - without any appreciable load time at that. No search engine does all this yet. Wolfram-Alpha is a step in this direction. The functionality of the service is very limited, and falls drastically short of its claims, but that just goes to show the staggering task in designing the next generation of search engines. A single search engine for all these applications would be great, and no guesses for which one will try the hardest to get there, but smaller engines with highly specialized and robust application can get ahead. For example, both iTunes and TrackID on Sony Ericsson phones use Gracenote’s database and search facility to identify songs. Gracenote is well ahead of Google when it comes to searching using snippets of songs, or their tags.

Web 3.0 will require some kind of system where all the applications can pool their resources. Some common language such as XML, that will describe the content generated by the users, and their relation with each other. There are no standards as yet for this language, but when it comes, it will help the web understand us better. There are two divergent paths that the web is taking. One approach is to consolidate everything in one central location, and the other is to distribute everything to the user. Google Docs, On Live and AMD’s Fusion render cloud are examples where data is centrally located, and worked on by remote applications. This includes applications where a game for any console can be played on any console. Another approach, drastically different, is Opera’s Unite browser, and software such as Freenet. Both of these bring the power of web hosting to individual computers. Applications such as Seti@Home have long been using distributed computing for processing large amounts of data. Web 3.0 will incorporate both of these approaches, further blurring the lines between personal and public digital space.

There is a flip side to all of this as well. These trends are not exciting to everyone. There is a small minority of people who don’t really approve of the direction in which the web is going at all. According to these people, blogging just introduced a whole bunch of clutter, Twitter is an irritating phenomenon, and all of these are passing fads. Privacy, and the use of personal data for advertising is another big concern, even if this is done by machines without human interaction. These concerns can not stop the web from going where it will though, and for the digital community, the world continues to grow smaller by the second.