Bookmarks: Emerging Architectures for Modern Data Infrastructure

As an industry, we’ve gotten exceptionally good at building large, complex software systems. We’re now starting to see the rise of massive, complex systems built around data – where the primary business value of the system comes from the analysis of data, rather than the software directly. We’re seeing quick-moving impacts of this trend across the industry, including the emergence of new roles, shifts in customer spending, and the emergence of new startups providing infrastructure and tooling around data.

In fact, many of today’s fastest growing infrastructure startups build products to manage data. These systems enable data-driven decision making (analytic systems) and drive data-powered products, including with machine learning (operational systems). They range from the pipes that carry data, to storage solutions that house data, to SQL engines that analyze data, to dashboards that make data easy to understand – from data science and machine learning libraries, to automated data pipelines, to data catalogs, and beyond. read more

You Don’t Need Big Data – You Need the Right Data

I enjoyed this article from Max Wessel at SAP talking about how big data is not always the answer, but more specifically that you need the right data. There is a lot to unpack when you start apply and adding your business processes to this context.

The term “big data” is ubiquitous. With exabytes of information flowing across broadband pipes, companies compete to claim the biggest, most audacious data sets. And businesses of all varieties — old and new, industrial and digital, big and small — are getting into the game.

Masses of social, weather, and government data are being leveraged to predict supply chain outages. Enormous amounts of user data are being harnessed at scale to identify individuals among a sea of website clicks. And companies are even starting to leverage huge quantities of text exchanges to build algorithms capable of having conversations with customers. read more