Maximizing Returns Series Part 1: Know Your Data

Current technology vendors have doubled down on getting you faster access to insights and results from your data projects. Newer approaches to data management and analytics have made it possible to target specific data and shorten development cycles. Despite these advancements, maximizing your data investment returns will depend on your ability to know your data, not the advancements being offered. In this three-part series, I will lay out steps that can help develop a more informed and rewarding plan for investing in data solutions or making strategic shifts. These steps can be applied to investments in all data solutions, whether the data is at rest or in flight.

Warning: If you have not already defined at least one business objective that is the basis for your investment, you are heading toward a negative return already. This should be step 1.

Know Your Data

You can get the most out of your data investments by understanding the information flowing through your business and what it can realistically offer. This understanding comes from reviewing the data associated with each relevant business process and learning the honest truth about what value it contains.

While a purist-visionary might reject the idea of examining their information flows, I would argue that this discovery exercise expands your vision. You can uncover more value opportunities by examining what flows into and out of a business process. Since an investment is being made to address one or more strategic goals, it makes sense to apply discovery to the information flows that support those goals. Know your data.

Types of Data

Once you have a strategic objective defined, the first step in planning a data investment is to inventory the data flowing through your business. Much has already been written about the different classes of enterprise data, but for now we will focus on the basic categories:

Transactional Data

Transactional data represents events in your day-to-day business, such as orders, shipments, billing, payments, customer inquiries, etc. Business equipment such as engine sensors, meters and other machines can also create transaction data. In most cases, the systems that author and maintain this data are your core business systems or operational systems. If something goes wrong with one of them, everyone knows it. This data typically exists as numeric transactions (e.g. payments) along with their date/time values, and it can be considered the lifeblood of your company.

Master and Reference Data

If transactional data holds your business events, master data represents the persons, places and things that take part in one or more transactions. Master data includes the descriptive information you collect for customers, suppliers, products and other business entities that play a role in your business operation. Although some consider it the same as master data, reference data is data that is primarily static, such as country codes and currency units. This data is used to define sets of valid values that describe other data including master data. Master and reference data are typically shared across different areas in your business, so keeping this data accurate and clean is a priority.

Understanding Data of Value

Sheer data volume is one of the biggest hurdles to a data discovery exercise. The data you really care about for an investment – data of value – is buried with other data you are not concerned with. With transaction data, there is an enormous amount of information collected that can clog up the discovery process. This paralysis can be avoided by linking your strategic goal (e.g. improving customer experience) and the underlying information flows you are reviewing (e.g. user interaction data, orders, returns, etc.). You can remove the excess noise that makes it harder to distinguish the data of value by viewing it through the lens of your strategic goal. If you truly know your data, you can focus on the data of value.

Structured vs Unstructured Data

While transactional and master data are the primary types of data you need to understand, it is worth noting that data of value is stored in both structured and unstructured forms. Typically, we think of many of our operational systems (e.g. billing systems) as systems having data with a definite structure. In technical terms, this data resides in a database with a known structure. Customer data is stored as structured fields describing their billing address, contact information, payment history, marketing preferences and other data. Structured data is often presented as rows and columns of data in reports or spreadsheets floating around your business.

Unstructured data is another form of data you most likely have in your organization. Truth be told, you may find that up to 80% or more of your organization’s information resides in unstructured form. While this data is not organized in any pre-defined manner, it may contain data of value. Unstructured data exists in electronic documents, machine log files, audio and video data, email archives, sensor data and other artifacts created without a defined format. Key elements from unstructured data may find its way into your structured data sources (e.g. response data from customer feedback forms). If you can make a direct link between your strategic goal and these data sources, they belong as part of your discovery. Data of value can exist in forms that lack a neat structure.

Not Losing Focus

So now what? Making an investment in any data solution deserves some upfront analysis so you can get to know your data. The information flow within your company is unique. Getting to know the underlying data is critical to avoid wasted efforts. Part 2 of this series will discuss important stakeholders to include in the discovery and planning of your investment. Involving the right data experts will prevent you from over-hyping your expectations, and will save you from looking for value in all the wrong places.