While Silicon Valley is always awash with new technology buzzwords, one that has become increasingly dominant in recent years is Stream Processing: a type of software designed to transport, process and react in real-time to the massive streams of event data at the heart of large internet companies.
While it isn’t wholly surprising that a company like Netflix–responsible for 36% of US internet traffic–would be interested in technology that processes data as it moves, what may be more surprising is the uptake in traditional markets like finance, which doesn’t typically generate the ‘trillions of events per day’ workloads seen at the giant tech firms.
But this uptake demonstrates a more subtle and intriguing shift in the way companies are reshaping their IT functions. In much the same way that the internet led to a range of products whose value came from connecting people together, the industry may have discovered a second revolution in the way that the applications that make big companies work discover one another, interact and most importantly, share data.