Five Data Resolutions for the New Year

Once we get past that old joke: “I made a resolution last year to make no more resolutions”, it is time to get serious. This post is for the business and technology professionals who farm and harvest data all year. I am thankful to have been part of a few completed data projects over the past year while continuing work on some others, and while doing so I compiled a list of favors we could do for ourselves in the coming year. These “data resolutions” are meant to supplement your planned activities, and while not all of them may apply to you, I hope they will help make your life in the data world easier in the year ahead.

1. As an organization, work to foster more progressive approaches to using data. This is about collaboration among business and technology professionals to advance the technologies used by your business. I described the rise in Shadow IT in a previous post, and while business units may continue to bypass IT to test drive emerging technologies, there is no reason for them to always go it alone. The business can have a better overall experience if they engage with IT to help facilitate the evaluation and implementation processes, and it usually results in a more cost-effective solution. If a business problem can be addressed using a new technology, efforts should be made to work as a cross-functional team to evaluate the options. For example, a solid and well-formed use case for a predictive analytics solution should be presented as an opportunity to involve all groups. Regrettably, not every IT organization has the bandwidth to take on a new platform, but business teams should still communicate and propose technology solutions that will help the organization.

2. Design new business applications that measure the business – not just automate it. Measuring the business continues to be a high priority for all organizations that collect data. For organizations that choose to build (vs buy) their own business applications, consideration should be given on how the new system will help measure the process, not just facilitate its core functions. Granted, this sounds more like a design principle than a resolution – but too often we roll out new business applications or processes that cut corners on data handling. Aspects such as clean data, standardization, date/time tracking, audit information and proper data structuring often get left out of the design because of a tactical focus that ignores metrics and measurement. Planning for business performance metrics as part of the deliverable can help avoid this scenario. So, when planning implementations for the new year ask yourself, “how will we know precisely that the process is better with the new system?”

3. Remind yourself that networking is not just for job hunting. We hear all the time that you should expand your professional network. And it’s another no-brainer – having more contacts can lead to more career opportunities. Networking is a great habit, but it can help with more than just your career. Events for your industry put you in contact with professionals who face many of the same challenges you do, and attending can be a rare opportunity to learn how other companies have used new data processes or technologies to solve those problems. I have been fortunate to have attended some amazing worldwide conferences, and I have attended others that made me want to fall asleep. But in all cases, I returned home with some ideas I picked up from networking. Local events are also something to consider in the new year, and they require no travel budget. These user groups or industry events offer you the chance to meet professional contacts in your local market, and keeping in touch with those professionals can be easy and more frequent. A big upside to networking is that you can learn from a different perspective that is free from the constraints of your organization.

4. Measure your metrics. Chances are you collect more data than you did three years ago. In some cases, you might have new sources of data – or if your business is growing you may just have more of the same (which is a good thing). In any case, measuring the health and performance of your organization does not need to stay with the same metrics. You may want to explore how some processes can be measured in new ways using different data points. If you have recently implemented a new transaction system, you already know that performance metrics need to be revisited to bake in the data ingredients introduced in the new platform. You should also review the validity of your metrics, especially if there have been material shifts in how your business operates. If for example most of your sales are now online or you handle inventory differently, old methods for measuring sales territories or inventory turnover may no longer tell the real story. As always, be transparent with metrics. My last post on misleading metrics talks about being able to back up your measures with data and details. Performance measurement is all about confidence… when your metrics are defined the right way you can be confident in what you present and how your business is performing.

5. Lose some weight. No, not that kind of weight. What I am talking about is old data weight. Keeping years of old data around adds to the cost of maintenance, and depending on where it resides it can impact the performance of your applications and reporting solutions. But be careful with this one. Personally, I prefer that organizations avoid deleting old data outright. And it is not just because the cost of storage is much lower than it used to be. If there is value in your old data (e.g. it is still suitable for trending and analysis) then you should consider some other options. Developing an archive and purge strategy that makes sense for your organization is step one. It is one of those no-brainers that we end up no-doing. Start by evaluating whether specific sets of data will have value after a certain period. For example, system log files might be a candidate for deletion after a few weeks or months. On the other hand, old transaction data might not be needed for frequent access, but because it will still have value later you should explore options to preserve it in some form. The next step is to evaluate your options. You might store aggregated or summarized versions of old transaction data in your data warehouse, or you could offload it in its raw form to a cheaper storage solution (look into Hadoop, which among other things is a very inexpensive distributed storage platform). Removing historical data from your environments may free up resources, but it should be done carefully with the involvement of all business and IT stakeholders. Pay mind to any regulations you may have to follow and be sure the appropriate people weigh in on any decisions. As a business intelligence practitioner, I must recommend thinking twice before deleting any data. It should never be a hasty decision, and all options to keep it in some form should be considered.

I hope the new year brings you much success both personally and professionally! For me I plan to network more effectively and focus more on the important things. In the meantime, I am thankful to my clients and colleagues for another solid year. Looking forward to taking some time off – and then I’ll see you in 2017!

Data Lens data resolutions