Longtime waka waka waka readers will recall that I used to work for an outfit called PubSub. It was an immensely promising idea, with truly revolutionary potential, but despite the fact that first-tier VCs were lining up around the block to give us money, the company perished in a spectacular (and wincingly public) implosion. I think of it as the USS Thresher of Internet startups.
The concept was simple enough; PubSub did “prospective search”, which turns the Google-type search paradigm on its head.
What Google does is crawl around the Web, indexing everything it finds. While it finds its way into an awful lot of places, it can take quite a while to re-index new material, perhaps even weeks. Everything it finds goes, you might say, into a huge bucket, and when a user comes along with a query, the engine fishes around in the bucket, gives the user whatever it finds, and the transaction is over. You get a lot of stuff, but freshness is not guaranteed.
Prospective search is exactly the opposite. It doesn’t deal in persistent data, but persistent queries. The PubSub engine didn’t crawl; it read feeds of various sorts — incoming streams of real-time data. When a user came to us with a query, we didn’t return anything right away: what we did instead was to hang on to the query — a “subscription”, we called it — and from that point forward, whenever any of our incoming data matched that user’s query, we sent out an alert, right away. Prospective search is an ongoing relationship, and you see stuff right when it happens.
The key points to making this sort of thing useful are: reading a lot of feeds, everything you can get your hands on (we started off with pings from bloggers, and then added high-volume feeds about SEC updates, airport delays, and other things besides, with more stuff coming all the time); having a good, fast way to get the alerts out (we had an IM-based application that worked sort of like a stock ticker, as well as being able to alert by email and other channels) — and above all, when you have millions of subscriptions that you have to match against enormous volumes of incoming data, you have to have a very, very fast matching engine. Which we had.
Anyway, the reason I’m bringing this up is because it looks like someone has decided to resurrect the company. I wonder how it will all work out.
3 Comments
Hi Malcolm –
I won’t venture a guess about “how it will all work out,” because my experience with new ventures (i.e., startups) has convinced me that good ideas and hard work count for little unless you happen to be in precisely the right place at precisely the right time; and then you still have to convince the VCs.
Well, Bob, we had all that, and still managed to go off the rails. It was most dispiriting.
That’s a fact!
– M