The energy system is becoming increasingly complex. There are massive increases in the amount of individual generation units producing power, intermittency is introducing new challenges, time of use tariffs are being promoted by suppliers and there is an enormous amount of smart technology being installed. That’s before you even consider the wave of electrification that’s planned in other sectors such as transport and heat.
This has resulted in a data explosion within the energy market. Companies need to use all this data more effectively and, crucially have information that can be easily understood and quickly acted upon. A prime example of how this new world of data is impacting the market can be seen in power trading. This has always been a relatively active part of the energy market, but traders have traditionally taken fairly long market positions and refined these as the point of delivery approached. However, increased penetration of renewables, smarter systems and more competition has squeezed the margins in these longer-term approaches. This has resulted in a need for traders to be more active in the day ahead, intraday and even balancing markets. Disrupters entering these markets using a variety of approaches, such as smart algorithms guiding trading decisions and completely automated trading, are causing a fundamental shift in the nature of how people operate. We are beginning to see the use of approaches in this area that are much more established in financial markets. It’s going to revolutionise the way the power trading market is going to operate in the future.
"There is a crucial need for generators, suppliers, traders and other stakeholders to be able to compete closer to real-time"
What’s the impact of this? All this change means there is a crucial need for generators, suppliers, traders and other stakeholders to be able to compete closer to real-time. There are going to be challenges for firms of all sizes. Smaller entrants to the market often lack the IT resource to warehouse, process and maintain multiple data sources that its larger competitors have already built, while more established players with legacy systems may struggle to compete with the sophistication and adaptability of new disrupters.
So how do we help the market respond to this increasing complexity and the shortening timescales?
Data integration is the key first step. Energy market data is remarkably disparate, with live data, spreadsheets and PDFs scattered over multiple websites. This means that traders and analysts are having to spend a lot of their invaluable time just on locating and collecting the right data. This is time that could better and more usefully be spent analysing data to make better decisions. After locating the data, it then has to be cleaned. There is still a large amount of inaccurate data that has to be checked which is surprising in a market as real-time as the energy market. The trader must then integrate this data in their head, coming to their own view of the position of the market with the information they’ve managed to gather in the window before trading closes. This involves quickly searching through and analysing the hundreds of thousands of data points pushed out each day, each of which could drastically impact decisions being made by the traders. It’s a long-winded process that hampers clearer analysis and decision making.
This is why we have developed LCP Enact which is a new integration platform that brings all this information together into one place, cleaning it, correcting it and presenting it in a manner that allows instant analysis and interpretation of market activity. Some views are rooted in trader workflows to make real-time decision making much simpler. An example of this is the Balancing Mechanism (BM) dashboard, a visual representation of the BM where users can simulate National Grid actions and identify locational constraints through a clear, interactive map. Other screens are aimed at analysts and managers to allow instantaneous benchmarking and performance monitoring of your units against the market.
However, to really get an edge on the advancing competition, simple integration is not enough. As a trader it’s crucial to develop a clearer view of the upcoming market and position yourself before the market swings. Traditionally, many traders rely on National Grid’s forecasts to make predictions on what might happen later that day. The recent surge in capabilities of data science has allowed much more sophisticated and accurate forecasting to take place. Within LCP Enact, we have developed a cutting-edge proprietary algorithm to forecast the net imbalance volume (NIV) of the system for the next 6 hours, on a real-time basis. Rather than rolling forward the errors in the published NIV forecasts (also-called ‘top down’ approach), we derive imbalance fundamentally as a combination of every factor affecting it such as wind level, demand, plant outages (a ‘bottom up’ approach). Certain aspects of this forecast can be modelled through machine learning techniques. For instance, we use neural networks to create a live wind forecast, while others require expert knowledge of the domain to understand issues around minimum run times, plant outages, and more. As this calculation is live, if any market movements occur (such as a plant declares an engine is faulty, a European interconnector begins importing, the solar forecast changes, and more) our view of imbalance will update. The result is a forecast that is 86 percent accurate in predicting system direction 30 minutes ahead of delivery, compared with National Grid’s own forecast at 59 percent.
Although large amounts of trades are still done manually we see automation having a much bigger role in the future. The potential this has to improve decision making is phenomenal. We see the market continuing to get more complex as it becomes more interconnected, more varied and active market participants continue to join. Getting your data integrated, ensuring your team can focus their time on what matters, and taking advantage of cutting-edge technologies to forecast market changes is the only way to stay ahead of the competition.
