Four Questions to Ask Your Analytics Vendor

No matter the maturity of your analytics deployment, vendor evaluation is an inescapable process. Even if your organization is not actively soliciting quotes and demonstrating platforms, you probably will be when your data volumes spike or user needs evolve.

Vendors are a necessary resource for utilities– according to Black & Veatch’s 2019 Smart Utilities: Strategic Directions Report survey, which polled hundreds of utility leaders, nearly a third of utilities (29 percent) say a lack of resources and expertise is a major barrier to enabling smart infrastructure. Vendors play an important role when it comes to filling this knowledge gap.

To help simplify the process and differentiate between prospective vendors, here are four questions (and accompanying drilldown questions) every CIO should ask during the evaluation process.

1. How will you help us mitigate downtime and increase reliability?

Today, machine learning (ML) and artificial intelligence (AI) tools are used to drive the monitoring and diagnostic processes that improve system reliability. The quicker networks capture, analyze and operationalize information, the less time it takes to address issues that lead to failure. But exercise caution – companies can make the mistake of approaching ML and AI tools in a technical vacuum, looking at them as static formulas for solving X or accomplishing Y.

While there’s nothing inherently wrong with this approach, it neglects two monumental layers of influence that determine a tool’s effectiveness: asset-level domain expertise and operationalization after implementation. You’ll restrict the effectiveness of your monitoring and diagnostic technology if you can’t combine it with real-world knowledge of how an asset should operate. And if you lack the resources to make effective use of your data, your ML/AI returns will flatline.

Before choosing your vendor, take a step back to identify which operational problems you want to address. Ensure the vendor has demonstrated ability in helping solve those problems, and work to align monitoring and diagnostic tools with the level of insight.

• What demonstrated value do you have in operationalizing tools like ML and AI?
• How do you prefer to work with stakeholders (operators, engineers, field personnel, etc.) to operationalize our tools?
• How do you integrate asset data sources like SCADA/DCS, Historian, CMMS and others?
• What tools do you have to help our staff collaborate on issues, questions and reports?
• Are your reports dynamic and customizable?

2. In what ways do you help us make use of our data?

The Utility Analytics Institute reports that nearly half of companies have “very limited use” for the data they collect, and nearly 40 percent admit to “trying to figure out” what do with data. This is a problem that a quality analytics vendor should help solve.

"Before choosing your vendor, take a step back to clearly identify which operational problems you’re trying to solve or prevent"

Ask your vendor about their ability to utilize data through performance analysis and optimization. This process should go beyond tracking standard key performance indicators (KPIs). As beneficial as they can be, standard KPIs can be a pitfall in utility analytics because they only scratch the surface of system performance.

Most KPIs are driven by two factors: efficiency and cost. The less efficient something is, the more money it costs to run it. But there may be a variety of metrics within a system or asset class that contribute to a network’s overall efficiency or cost – granular quantifiers like condenser cleanliness, net turbine rate and percentage of manhours devoted to proactive work. Unpack your KPIs, then ensure that your vendor can provide the level of specificity and granularity your reporting needs.

• How can you help us identify and refine our requisite KPIs?
• How granular can you get with tracking specific metrics?
• How will you help us benchmark asset health across infrastructure?

3. How do you help us manage risk?

We define risk based on two dimensions: How likely something is to fail and how costly it will be when it does. Many variables determine how likely an asset or device is to fail, including age and the timing of repairs or upgrades. Keep in mind that age can be quantified as actual age (years post-asset installation) or effective age (asset health projected from inspection and analysis). A well-rounded risk assessment will track and measure both.

Cost of failure is also a complex metric that goes beyond the cost of the equipment itself. If it fails, how does it impact other assets? How will it impact my business? How much will it cost if my plant shuts down? How much damage will be done to my company’s reputation?

Your vendor should help bolster risk assessment and awareness, aggregate asset data from disparate sources, and synthesize it into actionable intel that leads to informed decisions. Key to this process is being able to evaluate risk across different assets, geographies and time periods based on the metrics that are most impactful for your company and stakeholders.

• What is your process for identifying present and future vulnerabilities in our network?
• In what ways can you help us expose vulnerabilities in redundancy?
• How do you track and consolidate risk factors across our network?
• Can you help us forecast our risk scenarios over time?

4. How will you help us plan for the future?

It’s no secret that data is exploding at a record pace. Add in the evolving needs of staff and the heightened expectations of customers and planning for the future is more important than ever. But the reality is, many utility staff are too busy to perform long-term investment planning as frequently as they should.

Investment planning should be done continuously, and your analytics vendor should facilitate this process. Risk profiles shift based on what’s occurring across the network, and your analytics solution should account for this volatility – meaning your vendor should partner with you as you navigate change. Having real-time access to accurate, useful data is only the start. The right vendor offers tools and features that enable stakeholders to visualize investment scenarios across time– down to the asset level– to make decisions as efficiently and confidently as possible.

• Does your software complement customers’ investment planning process?
• Provide a scenario of how your investment planning tools have benefitted a company.
• How will you help us adapt to market dynamics?
• Can you provide a clear view of task dependencies, resources, manhours and late-breaking changes across time?
• Do you use tools like geospatial mapping and time sequencing to plan investments?

No matter if you’re in the early stages of planning or are looking to fine-tune a more mature deployment, these questions will provide valuable food for thought and bring you closer to solidifying your analytics approach.