Think (Really, Really Really) Big

By Joe Hildebrand, Presence Enabled

Back before we were storing our photos, music, movies, and other rich media on our computers, it was hard to conceive of the need for a 100GB hard drive on a personal computer — much less the possibility that such a hard drive would be insufficient for one person’s storage needs. But now we know better. I know I’m constantly scrounging for space on my laptop.

Yet sometimes when we talk about presence and the imperative to make it massively scalable, we hear two objections:

1. Even the largest enterprises don’t need to scale beyond a few hundred thousand concurrent users.
2. Even the largest carriers don’t need to scale beyond a few million concurrent users.

Those objections have a surface plausibility, until you realize that many people already have multiple points of presence: mobile phone, PDA, laptop running several presence-aware applications, not to mention services such as a weblog or photo site. Add to that the fact that any addressable application, device, sensor, or service is a potential point of presence, and you begin calculating the need for scale exponentially.

I might have 5, 10, 50, 100 or perhaps many more devices and services associated with my aggregate presence, and you might have just as many devices and services that want to know about my presence. And that’s just presence about people. But standalone devices and applications have presence too. Think presence-enabled vending machines, roads, garage doors, appliances, pets, workflows, inventories, programmable logic controllers in factories, utility grids, news stories, discussion forums, and so on.

It remains to be seen how some of these nodes will create value by generating or consuming presence. But we have seen the number of applications, mashups, services, devices, and sensors proliferate. There is no reason to think that the pace of creating such nodes will do anything but accelerate.

We know that many of these nodes will be presence-enabled because basic and extended availability information gives the consumer awareness of transient state changes that are significant for the purpose of real-time communication and collaboration. Presence is a kind of glue because it unifies communication across nodes, and it is a kind of catalyst because it drives further communication between nodes.

This is true in a wide range of real-time applications, whether based on SIP, XMPP, or some other technology. In today’s real-time world, you don’t necessarily know that you want to initiate a communication session (VoIP, instant messaging, whiteboarding, etc.) until you have presence about other people or applications on the network.

Indeed, presence is already proving useful for delivering on the promise of autonomic computing (fast reactions to changes in computing system state) and for making other processes autonomic (think of the world’s best traffic light timing, real-time supply chains, and the like).

Given all these factors, the conclusion that we will need massive scalability in our presence systems is close to obvious.

Once you think beyond instant messaging as the only presence-enabled application, you start to see presence as a kind of communications middleware. The catch is that if you try to attack the problem with traditional polling or transactional processing middleware, your deployment is going to die a painful, senseless, and untimely death.

Painful because you are very quickly going to feel the need or desire to connect an ever-increasing number of presence points — and a polling-based architecture will collapse on the weight of presence pings alone.

Senseless because full transactional processing is simply overkill for the types of messages these services need to send and receive.

Untimely because your presence architecture will publish stale information, not real-time presence. That someone or something was available is of little value compared to knowing that they are available right now, in real time.

We noted that presence provides transient notification of significant state changes. From an architectural perspective, the key is that presence is transient and timely, so high scale and low latency are both critical.

To achieve scale, presence middleware needs to be event-driven: messages are exchanged only when something changes. To achieve low latency, it needs to dispense with the notion of guaranteed delivery: 99.9% message reliability is plenty good enough when publishing an “on the phone” presence change. And to deliver significant information, it needs to know what it doesn’t know: i.e., it needs to be open and flexible enough to allow for a nearly infinite number of extensions to meet future requirements.

The power of presence for real-time communication and collaboration has only just begun to make itself felt. Innovative enterprises and service providers need to start thinking big if they want to take full advantage of the presence revolution.

Joe Hildebrand is CTO of Jabber, Inc. (news - alert) For more information, please visit the company online at http://www.jabber.com.