.jpg)
Weather forecasts for the grid depend on supercomputers to calculate the flow of heat, water and radiation in the atmosphere and then spit out predictions about what could happen next.
These supercomputers are powerful. But they are also expensive and slow, relative to how quickly the weather changes. A new class of AI-based weather forecasts could change the game for grid operators and renewable energy developers. Will they take hold?
This week, we explore a variety of emerging applications for artificial intelligence in energy.
First, we’ll look at how machine learning can improve and democratize weather prediction with journalist Emma Woollacott, Jack Kelly of Open Climate Fix and Noelia Otero Felipe of the University of Bern.
Then we’ll hear from a few companies that are using AI for demand response, virtual power plants and EV charging. Latitude producer Erin Hardick explores AI trends at the grid edge with Apoorv Bhargava of WeaveGrid, Jae Beom Bae of Leap, Paul McDonald of Opower at Oracle Energy and Water, and Carlos Nouel of National Grid.
Sign up for Latitude Media’s Frontier Forum on January 31, featuring Crux CEO Alfred Johnson, who will break down the budding market for clean energy tax credits. We’ll dissect current transactions and pricing, compare buyer and seller expectations, and look at where the market is headed in 2024.
Transcript
Stephen Lacey: It has been a year of extremes around the world. Monthly global temperature records have been shattered nearly every month and government prediction agencies in both Europe and the US say this year is virtually certain to be the hottest ever measured. And Europe is warming almost double the global average. The region has been gripped by multiple extreme heat waves over the last couple of summers, roasting even temperate regions like the UK.
Aidan McGivern: This is the Met Office Headquarters, I'm Meteorologist and Weather Presenter Aidan McGivern, and it is an extraordinary spell of weather. I've been a meteorologist for a long time, really this is one of the most remarkable forecasts I've seen in my lifetime.
Emma Woollacott: Well, we're generally not used to terribly high temperatures in the UK, but the last few years we've had some real bursts of hot weather that have lasted anywhere from a few days to a few weeks.
Stephen Lacey: Emma Woollacott is a technology journalist based in the UK. She writes about large tech companies regulation, sustainability, and increasingly artificial intelligence. And she recently wrote a story for Latitude Media on how that heat had direct consequences for the grid operator, National Grid. This June saw a confluence of difficult conditions on the UK grid system. There was an interconnector fault that slashed delivery of electricity from Norway, a nuclear power plant was under planned maintenance, and the summer heat has also increased demand from air conditioners, which are suddenly spiking in popularity in a region where AC is not that common.
Emma Woollacott: Meanwhile, it wasn't especially windy at the time, the weather was very hot and still so there was much greater reliance on solar power. And again, it just wasn't enough. And so the National Grids the electricity system operator had to ask for a coal plant to take up any slack.
Stephen Lacey: And that broke a pretty important streak, did it not?
Emma Woollacott:
Yes, it did. It was rather a shame, 47 days the UK had been without needing any coal plants.
Stephen Lacey:
This return to coal, however brief, was a big deal symbolically. Britain was an early pioneer in the use of coal. As recently as 2012, it generated 40% of electricity there. And in 2015, Britain set a goal of phasing out coal by 2021, but it's had to delay that phase out by three years. And this summer's emergency reliance on coal, even though it had nothing to do with solar performance, brought out the skeptics who claimed it was solar power that failed in the heat wave.
Speaker 2: The latest ridiculous story that we've got here is the National Grid was forced to fire up two coal power stations after solar panels failed to work. I know I may not be a scientist, but you have a solar panel which absorbs the heat from the sun and then transforms that heat into energy so that you can power something else with that electricity. But apparently solar power is only good when it's not too hot.
Stephen Lacey: Over here in the US we saw Fox News and a bunch of other conservative outlets pick up the story, and their claims that solar failed were completely false. But they pointed to an area where actual scientists say we need more improvements: Weather prediction for grids that are increasingly saturated with weather-dependent renewables at a time when weather is increasingly unstable.
Emma Woollacott: You need to know what the weather's going to be like fairly immediately, next few hours, next few days. Keeping power stations on standby can cost hundreds of thousands of dollars a day. The less that needs to happen, the better for everybody.
Stephen Lacey: Weather prediction is this scientific miracle that many of us take for granted and often complain about if it's slightly wrong. The stakes for weather prediction on grids are much higher. And today's numerical weather Forecasts are state-of-the-art. Simulations run via supercomputers that calculate the flow of heat, water, and radiation and spit out predictions about what could happen next. But these supercomputers are expensive and relative to how quickly weather changes, they're actually quite slow.
Emma Woollacott: To do this properly, thoroughly, it means an awful lot of computing power is required and an awful lot of time. So the resolution can't be particularly high, we're looking at about a kilometer square, and they can't be particularly fast because there's a huge amount of number crunching to be done. AI can make a real difference to that.
Stephen Lacey: This is The Carbon Copy, I'm Stephen Lacey. Artificial intelligence is already being used across the grid in all kinds of ways. In this week, we explore a variety of emerging applications for the technology. First, we'll look at how machine learning can improve and democratize weather prediction, and then we'll hear from a few companies that are using AI to democratize how consumers are interacting with the power system.
The question of how to improve weather prediction for power systems brought Emma to a data scientist named Dr. Jack Kelly. He was formerly a technical lead at Google's DeepMind working on an internal wind forecasting service. And Jack had turned his attention to the nuances of solar forecasting, co-founding a nonprofit called Open Climate Fix to bring machine learning to grid operators in the solar industry.
Jack Kelly: For me, it started with a drink down the pub, as is often the case for any projects based in Britain.
Stephen Lacey: It was at this pub where Jack met a guy who worked for National Grid ESO, that's the electricity system operator in the UK. And this guy mentioned something surprising.
Jack Kelly: The biggest source of uncertainty for their demand forecast came from their solar power forecast.
Stephen Lacey: Even though the UK doesn't have that much solar, a short-term forecast that's off by 20 or 30% could mean gigawatts of unexpected drop in capacity. Conventional numerical weather predictions are not great for this application. And that got Jack thinking immediately.
Jack Kelly: And it seems to me that because machine learning is fairly good at dealing with images, that there's a good reason to believe that you could use machine learning to interpret satellite images and interpret the slightly flaky data that we get from solar PV systems and combine those with numerical weather predictions to generate better solar forecasts.
Stephen Lacey: Why are conventional solar forecasts so difficult?
Jack Kelly: Gosh, so conventional-
Stephen Lacey: I can see the list running behind your eyes.
Jack Kelly: Yeah.
Stephen Lacey: Along with being slow and limited resolution, numerical weather prediction models have their eyes closed to the real state of the atmosphere, says Jack, they accumulate a bunch of information about conditions and then they stop.
Jack Kelly: Which is all to say that NWPs, whilst they do a great job of predicting things like humidity and temperature, converting that to a really detailed map of where the clouds are is known to be fairly imperfect at the moment.
Stephen Lacey: So these forecasts are leaving performance on the table. Open Climate Fix's machine learning model takes in satellite imagery, numerical weather prediction data that exists and a bunch of real-time data on solar projects to predict solar output.
Emma Woollacott: And you can just plug all that in and come up with really pretty accurate results a great deal more quickly and requiring a lot less computing power.
Stephen Lacey: And that was good news for the people inside the control room.
Jack Kelly: We halved the error an hour ahead, at 24 hours ahead, I think the era was about 15% lower than their existing forecast. So this is actually still running in National Grid ESO's control room and we've certainly heard some anecdotal evidence that it's saved end users a whole bunch of money in reducing balancing costs.
Stephen Lacey: Is that guy from National Grid you met in the pub a little less stressed?
Jack Kelly: So that guy actually went on to earn a lot more money at an energy trading company. But hopefully in general, there are a bunch of other people that we've been working with at National Grid for a long time, and hopefully they're a bit less stressed, although there's definitely a lot of other stuff happening on the grid that is adding to their stress.
Stephen Lacey: One of the big contributors to the stress of grid operators is the surge of all kinds of extreme weather. And that brought me to Dr. Noelia Otero Felipe, a data scientist based in Germany. She studies climate variability and its impact on energy production. And she studies a specific thing called an energy drought.
Noelia Otero Felipe: And when I started just digging into the topic, I found it really fascinating. I also talked to people from different companies, energy companies, and I realized how important this is. That's why I decided to start working on the so-called energy droughts.
Stephen Lacey: Yeah, what is an energy drought?
Noelia Otero Felipe: Usually the energy drought is defined as a period of low production or also a high electricity demand. And it turns out that when the low production occurs, usually it can happen, occur at the same time as a high electricity demand period. And this is really risky for the energy system.
Stephen Lacey: This is a fairly new field of research that combines atmospheric science, energy modeling, artificial intelligence and data visualization. Noelia is focused on compound extreme events.
Noelia Otero Felipe: For example, if we have precipitation and winds, extreme precipitation and extreme winds, usually when these two events occur at the same time, the impacts are usually larger.
Stephen Lacey: These compound events can have a direct impact on the energy system, lowering solar, wind and hydro output all at the same time across a region. And as these kinds of events get more common, they're also simultaneously shifting demand.
Noelia Otero Felipe: Because of climate change in some countries, the energy patterns are changing.
Speaker 3: The growing use of air conditioners in homes and offices around the world will be one of the top uses for global electricity demand over the next three decades. It's a trend noted by many stores across Europe.
Noelia Otero Felipe: So we can expect, for example, more energy droughts in terms of high demand in summer than in winter, which is something that before didn't happen, it's something that is changing now.
Speaker 3: And that's why Noelia has also turned to machine learning to run simulations faster, often on local computers. And the advancements in this area are surprising even to researchers like her across computer vision, natural language processing, and large language models.
Noelia Otero Felipe: Everything is going fast. So every day we have new models there. For climate applications, I think it's an interesting moment. There are real-world cases that actually are having an impact.
Stephen Lacey: AI is undoubtedly changing the game for weather prediction without necessarily needing a government supercomputer. But in the energy sector, nothing is simple and this brings us to a common theme on the grid. There are a lot of players in the market that are still ill-equipped or uninterested in some of these new forecasts.
Jack Kelly: The skill of the forecast is surprisingly low on the list of priorities when transmission system operators are buying forecasts. So it's more about how understandable is it? How reliable are you? Do you have a track record? Can you deliver the forecast in this arcane format that works with their ancient computer systems, that sort of thing.
Stephen Lacey: So what I'm hearing you describe is a set of problems that plagues almost any new better technology that could be introduced into the power system over the last 20 years.
Jack Kelly:
Yeah, exactly.
Stephen Lacey: So AI forecasting is still a research project for utilities and grid operators, but progress is happening at an astonishing speed. Huawei has a three-dimensional weather prediction system that it says is 10,000 times faster than current methods and it's been successfully tested by the European Center for Medium-Range Weather Forecasts. Nvidia, DeepMind and others are investing heavily in this area as well in an effort to speed up predictions at much higher resolutions.
Emma Woollacott: The abilities of AI are growing all the time, particularly with the expansion in computing power generally. It's finding applications in practically every field of human existence at the moment.
Stephen Lacey: Did you come away from this story feeling like there's a real valuable use case here?
Emma Woollacott: Oh, very much so, very much so indeed. An awful lot of energy companies and utility companies are evaluating this and mostly still in the fairly early stages, but the fact that they are all looking at it shows what an advantage it's likely to be.
Stephen Lacey: Coming up after a quick break, a roundtable conversation on how three tech companies in a major utility are using AI to manage electric cars, smart meters, and other customer-side resources on the grid.
So as I mentioned at the end of our opening story, any discussion of AI in the power sector inevitably comes back to a very familiar set of questions about utility adoption cycles, data availability and how to scale beyond these perpetual pilots. AI doesn't magically make those go away. And when we turn to the technology for managing and dispatching customer-side resources for applications like virtual power plants, this is an area where there's a lot of cool stuff happening right now, we inevitably come back to a similar set of questions about how to work with utilities, how to get good data, and whether customers are willing to go along for the ride. And this is where we're going to focus next. Our producer, Erin Hardick, who's spent her career covering grid trends sat down with Apoorv Bhargava of WeaveGrid, Jae Beom Bae of Leap, Paul McDonald from Opower at Oracle Energy and Water, and Carlos Nouel of National Grid. And they looked at AI trends at the grid edge. So here's that conversation which was recorded live at our transition AI New York event in October.
Apoorv Bhargava: Hi, good morning, good afternoon. I don't know what time zone I'm on anymore. I've been in West Coast, Central and East now in the last forty-eight hours. So my name is Apoorv Bhargava, I'm the co-founder and CEO of WeaveGrid. So my role is I do all the things. And that means that from the first days of thinking about how we take machine learning and artificial intelligence and put it into place to think about what I believe is going to be the most critical transition of this entire next 10, 15 years, which is vehicle electrification, and thinking deeply about how we move to a new grid and a new transportation economy that is decarbonized at its heart.
WeaveGrid basically builds software that works with energy utilities, they are our primary customer. But also deeply partners with automotive and charging companies. And what that means is as we talk about data, as we talk about grid edge, there's a lot of talk about what is the quality of the data, how do you apply machine learning to all those things? And let me tell you if you think utilities are a tough nut, let me tell you how automotive data looks. It makes you utility data look like... No, we're not going to talk about this. I only have hot takes, so I promise I'm going to bring out my hot takes later.
But I think the key thing to understand is we're moving to a world where we're going to have different siloed data streams coming together. And so you might have incredibly high quality data coming from a Tesla and you might have incredibly low quality data coming from somewhere else, and you may have completely ish, okay, data coming out of your smart meter. I saw Landis+Gyr in the audience, you all are great, don't worry. But what that means is we have to somehow bring all of these data sets together, build forecasting, build analytics, I heard something about the analytics pipeline doing it five times over in the last panel, and we have to then build consumer decisions, consumer automations, consumer insights, and importantly consumer savings that benefit each and every customer and also benefit the system overall.
A really critical insight we've had at WeaveGrid is that consumer decisions, the consumer's choices that they make around particularly how they charge their vehicles, have deep operational impacts, operations with a big O going down to the distribution operator. And vice versa, now operators are going to require that data and visibility and insights in order to also keep their system reliable, clean and of course affordable. So that two-way dependency is something that we don't think about a ton because generally the thought has been let the load run free, let the power run. And I'm obviously being a little facetious, but the point is we've never really required that amount of synergistic, both visibility and control at the edge. I have a lot of ex-Nest employees on my team and I love to remind them nobody really cared if it was a smart or dumb thermostat, it was just a thermostat at the end of the day.
Now of course, again, being a little out there, point is EVs are a completely new device type, each of them charging anywhere between seven to 15 kilowatts at home. And that is a tremendous amount of load to be putting onto a cul-de-sac that has eight homes and now suddenly 2.1 cars per household, starting to look pretty hairy pretty quickly. And so that's where my company really focuses on, and we'll talk a lot more about how we use machine learning later. But yeah, I don't know if I'm at time, so I'll shut up and move on.
Jae Beom Bae: So I'm Jae Beom, I am a director of platform operations at Leap. What we do at Leap is kind of similar to WeaveGrid in a sense, but we are more technology-agnostic, whereas EV will be part of the mix of kind of what we provide to the market as a DER aggregator. So really we're a software platform where any sort of device that's enabled by IoT can integrate and we participate on their behalf to the wholesale markets. So our thing is participation in the wholesale markets as well as value stacking and sifting through all the products that are out there from a wholesale perspective and providing the best value for our end customers.
And I think similar to Apoorv, we've seen an explosion of the variety and volume of devices that are proliferating now and willing to participate in the energy market because those are two very different layers. And I think over the next five, 10 years, which is some of the stuff we'll talk about, we're just at the tip of the iceberg of this grand revolution that is DERs, I think.
Carlos Nouel: So good afternoon, good morning, Carlos Nouel with National Grid. I'm part of the business that is here in the US and not the one that Stephen was talking about early on in the session that's in the UK. I actually like this panel, so I have two panels today, but this is one that I'm really interested about primarily because we all talk about DERs, we all talk about modernizing the grid and all these things, but the reality is it only matters if customers actually make different choices. None of this actually matters if people don't make different choices. And a lot of the work that we're trying to is thinking about, how do we actually get customers to make different choices? Even in the places where we're really far ahead, like California and even Australia and other places, penetration of DERs, whether it's EVs, solar batteries, heat pumps, you name it, it's still fairly low. I mean, it might make 20% at best in certain cases.
And we talked about equity before, the reality is a lot of the work that we need to do all as an industry is making sure that we get that 80% that is still left to come along this journey in any way they can, right? Some of it will be with all the things that we want and others will be with pieces of that. And so one of the examples that I wanted to talk about, so the two examples I want to talk about that are related to that.
So the first one is working with Opower quite a bit about, how do we get people to start making different decisions? And we are one of the few utilities actually that probably are here in this conference and probably across the US that we still don't have AMI, we actually are still on AMR. And while that's pretty challenging for us as an organization, it doesn't mean that we need to stop. It doesn't mean that we're in a position where we cannot do anything. And working through analytics, working through the data that we have, we actually have been able to actually triple, for example, adoption of the behavioral program and participation in the EV programs and participation in the weatherization programs. And to me, I think the important part of that is even if you don't have perfect data, even if you don't have all the data that you want, you can still fundamentally start to make changes and actually nudge customers to make a difference. And we need every single customer to make that difference.
The second part is probably the extreme of that curve, which is we're putting new AMI meters and probably if you would've talk to me about five, 10 years ago, I would've said AMI meters are not really that sexy. But what we've done with SANS and Landis+Gyr is really thinking about kind of a completely different platform an AMI would look like. And essentially brings a lot of principles that have been talked about today, which is being able to do a lot of high-resolution sampling, which really allows you to know not just what's happening in the home, but really saying it's actually a Tesla, it's a Chevy Volt, it's a heat pump in real time. And you actually have the ability to process all that information on the grid because that's the only way you can actually get to the point where you can deal with all that data and start to provide meaningful insights for the organization and for the customers.
So those are two things that we're working on that are starting to leverage AI and machine learning to make a difference. And with that, I think that's part of what we're going to talk about today, which is a lot of those things are bringing new challenges for us as an industry, things around what kind of data can we share? How can we engage with customers once we know certain things? How do we protect that data? So that's some of the things we want to talk about today because it's really exciting, but I think it also will require a lot of fundamental changes to this industry that we need to be ready for it.
Paul McDonald: Hey everybody, I'm Paul McDonald and I lead product strategy for the Opower team at Oracle Energy and Water, we like to share all the names of our companies and solutions. The problem that we're focused on is simplifying all the growing complexity and the experience and the service that utilities need to provide their customers. And there's two ways to think about it.
The first is exactly what Carlos just mentioned. In order to navigate the energy transition, we need a lot more people making a lot of different choices and taking a lot more actions like never before. We need people upgrading their homes and buildings and cars, we need them adopting DERs and automation programs, reducing and shifting their use. When infrastructure is constrained, we need them getting financial assistance when they need it. Every single customer that a utility serves has something they need to see and do next, and instrumenting an experience that meets them where they are presents an incredible amount of complexity. So that's thing one.
Thing two is the more success we find in influencing all these actions, the more complex these customers become to serve. You're talking about endless permutations of DERs behind the meter of pricing programs, fuels, languages, income levels, and everyone needs a simple experience and a high quality of service from the utility even as all that stuff changes. So there's tremendous complexity there as well. So what we do with our clients is we leverage artificial intelligence and behavioral science to instrument that experience that will help customers take action and help them feel well cared for and well-served at the end of it.
Like Carlos mentioned, for utilities like National Grid, that means measurably accelerating device program adoption, weatherization program adoption, heat pump purchases. For utilities out west like Arizona Public Service, that means helping customers on really complex pricing plans understand how it works, shift their usage away from expensive peak times, save some money and end up feeling much better about APS as a service provider as a result. For utilities like Washington Gas, they're my local utility in Arlington, Virginia, we're using AI and behavioral science to identify likely limited income customers reach out to them with an experience that helps them feel comfortable seeking assistance and help them get enrolled in really well-funded programs that make the energy they need to live their lives more affordable. So I'm hoping to share more examples like that as we talk, I think this is going to be a good discussion.
Erin Hardick: Okay, Carlos, we'll come back to you. So you mentioned you don't have smart meters, but you're still using AI to motivate behavior. How are you doing that?
Carlos Nouel: Yeah, so there's been a lot of talk about utilities don't have data, but I think someone even said utilities probably have more data than healthcare or I think there was another industry that was called out. And the reality is you can start to piece together the few pieces of data that we have alongside with a bunch of other pieces of data that are relatively available for you.
So things like weather for example, like understanding if it gets really cold, if it gets really hot, how does your usage change? And you can start to correlate, primarily this would be your HVAC system, so when you send a communication to that customer saying, "Look, 20/30% of your usage is actually driven by your HVAC," it actually makes it real for that customer, it actually makes it tangible. And that's one of the simplest things to say, "Oh, I didn't know it was that much." And it doesn't mean that it's perfect, it doesn't mean that it's exactly 20/30%, which is one thing we need to talk about, as an industry we always try to be precise and perfect and the reality is if you can give customers enough guidance, enough direction, that's much better than what they have today, they have zero.
So that's one of examples of trying to leverage weather information, income information, housing information, all those things and trying to couple it all together in a place where you can start to make better decisions and provide better information to that customer without necessarily having what we're going to have in a few months. The next few months are going to bring a whole new kind of sections of data and insights that we're going to have. But you don't have to stop, you don't have to wait until you get there to start giving good insights to your customers.
Erin Hardick: And Paul, you also just mentioned that more choices means more complexity. How does AI help you navigate the complexity?
Paul McDonald: Yeah, that's a great question. The simple answer here is when you have to operate at utility scale and serve millions of customers each with an incredible variety of different data states describing who they are, where they are, what they've done, what they need to do next, what their billing relationship looks like with a utility, you can build a whole pile of rules if you want to and like a CRM marketing automation systems to try to treat each one of those customers as an individual. But you'll just reach a layer of complexity at that point where it becomes impossible to treat everyone as an individual person. So that's where AI can come in to take all these parameters for an individual customer and pinpoint, "This is what this customer needs to see right now, and this is what this customer might want do next right now and here's the next step they can take. And if they do it, here's what it's going to mean for their wallet."
Erin Hardick: Apoorv, we'll come back to you. So EV is a very different type of load. I mentioned managed charging programs, but that's probably just one facet of what you're talking about at WeaveGrid. How does artificial intelligence impact your ability or the ability to manage this new EV load and the customer experience around EV charging?
Apoorv Bhargava: So I think one of the most interesting things about electrification is that more than ever before, we're going to have a moment where the share of your bill is going to be made up more and more by the grid, by the network. There's 280 million cars in the country, there's 120 million households. We designed a century-old grid around those 120 million households. And now as all these cars start going electric, there's just going to be a tremendous change on where the cost to serve actually lives. And increasingly it's only be on distribution, which now means that everyone's time to charge is not 9:00 P.M. My controversial view is there is no right time for everyone to charge, there's only a right time for you to charge, and that is not driven by just your location in the neighborhood and what time you decide to plug in, it's driven by how you choose to drive your car. Not a DER, a car. How you choose to go to school, go to work, pick up groceries from the grocery store. All of those decisions are forecastable to some extent.
And then you have to take that insight of how people drive and hence how they charge to then have an understanding of what assets are going to be fundamentally changed, going to be at risk, have to be managed through intelligence. And what you start to realize is that if there is really a right time for each of us to charge, that's a completely personalized decision. So when we started the company, we didn't talk about AI that much because it was just a given. Of course machine intelligence has to be driving a lot of this personalization. Now it's a buzzword so we say AI. But the truth is it's always been core to our product because you cannot actually derive the right time for an individual to charge if you're not using all of that machine learning to actually figure out, how am I going to create the best economic impact for the customer?
But also, and this is incredibly critical in my view, how do we create the best economic and system outcome? Because people charging at the moments that they think are best because their time of use rate kicked up might actually be a really bad outcome for that local part of the grid. And so if you can just move people around a little bit, suddenly you can now drive both a much better economic outcome for the customer and an economic outcome for the whole system.
So that's how we use AI, it's embedded in every part of our toolkit and there's other machine learning tools we have as well available for our utility customers including finding those EVs before we even start enrolling them in our managed charging programs. But I think predominantly we think about that interaction between customer insight, customer controls, and of course operational controls that utilities need.
Erin Hardick: And Jae, to talk more about other DERs, and sorry if you don't consider EV a DER, but how are you also using machine learning to identify assets on the distribution grid? And are you using other types of AI as well?
Jae Beom Bae: I think for us, our key role in the industry as we see it is really making that as a proof of the best outcome for the end customer as well as the system. And our view is that the best way to do that is through price signaling and just market mechanics. And what we're trying to do is simplify, basically abstract away the market complexities away in such a way that is very easy for these end users and end devices to really participate in any given region or program. So I think we use AI in three critical, I would say, steps of our broader process. What we do is we get all these tiny meters, well tiny or big given how we say it, and we package them up to these things called, as we all know, virtual power plants and we participate into the wholesale markets.
So I think one key area where we use the AI currently is we use it to assess what are the curtailment capabilities of a given category of meter. And for Leap, we probably have the broadest array of different types and sizes of different IoT devices that are able to basically curtail and provide services to a grid. So one big challenge that we're constantly working through refining and polishing is making sure that that model is as accurate as possible. And part of that is, we'll talk about the issues later, but there's a whole set of challenges both on the input side. I mean, if you don't feed ideal data in, ideal data doesn't come out. As well as categorization, I think categorization, I think Carlos has also alluded a little bit to that, but categorization is a massive challenge. I think just know not all pumps are equal, small pumps and big pumps, depending on the region, they all may vary in terms of behavior. Are you clustering them in the same way? So I think that's one set of challenges.
The other very associated with that clustering is aggregation. When we're piecing together these meters into these aggregate resources called virtual power plants that we're participating to the grid, are they synchronous and are we aligning them based on their dispatch profiles or expected dispatch profiles? So that's another key area.
And I think the final one is market participation, real-time market participation. Are we dispatching these meters in a way that is both serving the intent of the customer and best aligned with their economic goals as well as serving the system? So I think that price signaling, making sure that the wheels of the market, so to speak, is as greased as possible is kind of what our main purpose is.
Erin Hardick: Let's actually talk about predictive power and the accuracy of these models. Paul, we'll start with you but definitely open this up to everybody else. How do you handle the tension between data privacy and having accurate models?
Paul McDonald: Yeah, this is a good question, and you all have to forgive me here because I feel like this is going to be a little bit of a wagging my finger answer. It's a straightforward issue. Large volumes of data allow us to train really powerful models. Handling larger and larger volumes of data creates higher and higher stakes for keeping that data secure. Larry Ellison was just talking a few weeks ago about aggregating entire nations worth of electronic health records in order to train models that do things like never-ending clinical trials and personalized medicine use cases.
In our industry, we're handling data that describes people's daily lives in their homes, so the stakes are high. And I think in order to sustain the permission we need, in order to handle the data we need, to train the models we need, to navigate the energy transition, we have to treat the security of that data and the privacy of customer data in particular as a top priority. And that is brutal work. It's getting data usage agreements right, it's getting cybersecurity policies right, it's human and technical controls at every single layer of the stack, it's auditing those things regularly. And it's not as exciting as the work of building a new model to solve a problem we couldn't solve before. But if we don't do that, we won't have the opportunity to build those kinds of models.
Carlos Nouel: Yeah, and I'll just add on that. I think, so I was talking a bit about the new meter we're deploying and the sense application that we're putting in. Before, and I think probably many of the folks here, if you work for utility, you probably have AMI meters, you probably have worked with an AMI meter and you probably can say, "I can predict what is a customer using and what's happening in their home at any given time." And I think that's fairly accurate and what Paul was saying, the more meters you have, the better information you have.
The reality is because of the high level resolution that we're doing on the sampling and using language models, we actually stopped from guessing what's happening in the home to actually knowing what's happening in the home, right? To the point where, I can tell you about my house because I have it in my house, I know when we open the fridge because there's a light in the fridge that comes up and I know when they open the fridge, which is a bad thing for my kids, but I actually know, right? We know when we plug the AV, we know when we turn on certain devices. And we know it in real time, it's not like four hours later, it's real, real time.
And I think the thing we need to do, or at least the way we approach it, is twofold. I think the first one is we need to start building this trust where we say, "You can opt in. This is not a thing we're going to force on every single customer at the beginning," because it is really personal. This is really knowing your house. When people used to say, "Well, we really don't know about what's really happening in your home," that's not the case anymore, we actually know, period. And I think we need to make sure that customers understand that we're doing this for them.
Which ties into my second point, which is we need to show the value to customers of all of this, right? Many of you probably have an Alexa or use Facebook or use Instagram, whatever, Facebook is for old people so I think probably this crowd, people will know what it is. But we all click on the privacy things without reading it. We always say, "Do you agree to the terms and share your data?" And we all say, "Yeah, sure, it doesn't really matter." And part of why you do it is because what's on the other side of that link, it's value, it's connections. You get those reels that are [inaudible 00:39:04] and you spend hours watching them, right? As an industry, we don't have that same sex appeal yet and we need to figure out how we continue to make the value of the data available to customers in a way that they actually are going to click on that link and say, "I'm actually okay with you giving me these insights because I see value in it for me individually. Forget about everything else, I see value for me individually."
And I think as an industry that's something we need to continue to tell the story of why it's important for them to have that data, and then why it's important in the right places to use that data for other purposes that serve the greater good, right? Things that can help the broader system, things that can help your own broader community. I know we've done things with Opower, for example, where we do campaigns at a community level and those tend to be really, really exciting and people get really engaged. So we need to balance privacy and cybersecurity with the value that we're providing to the customers and to the system. And striking that balance, it's going to get more complicated as we have more data as we start to get into more devices that know a lot more things because even when you move away from residential customers, you start to get into commercial customers where in some cases that could be a competitive advantage for some people if I know how and when you run your devices.
So this is going to be a new journey and I think one of the pieces, even from a regulatory perspective, that is a big thing that will need to change. I can tell you right now here in New York, utilities couldn't go and just market if I know you have an EV, I couldn't just go and market people saying, "Here's a great rate for you that can save you money." Even though it's a good thing for customers, that's considered marketing. So how do we deal with those things, how do we work around those things from a customer perspective and a regulatory perspective, is something we really need to get ahead of it so it doesn't become a roadblock later on.
Apoorv Bhargava: I was just going to add one thing. You do have sex appeal, the sex appeal is called being able to drive your electric car somewhere. Honestly, how many of you thought about AT&T before you had a cell phone? Not much. Rotary phones kind of sucked. The reality is the sex appeal is in the value and we are moving to a world where the value is suddenly showing up in not just, "Oh yeah, I used three kWh, I mean kilowatt-hours." I've literally had somebody say, "What's a kWh?" And I was like, "A kilowatt-hour." Three kWh on my fridge, that doesn't matter. Being able to get to a grocery store does matter. Being able to, god forbid, leave a moment of low resilience.
Yesterday we were awarded as part of a coalition with Xcel Energy, we were awarded a GRIP award by the Department of Energy, which is entirely focused around resilience. It's entirely focused around what happens in a moment where a wildfire were to hit and you have to turn off lines. How are we now communicating with people who need to evacuate that area? How are we actually managing the charging so that that line, if it is out now, how are we making sure that somebody is fully charged, ready to go or could do V2H and could actually use their vehicle to power up their home?
The truth is, suddenly the utility industry is going from 40 years of flat and declining load to the first growth moment since the air conditioning unit was invented. I mean, I'm being serious. And what that means is you do have sex appeal. There is an opportunity to suddenly say to customers, "We can help you. We can help you. We can go beyond just being a trusted advisor to actually enabling you to have value from all the things you've got in your home." And I think the EV is going to be the first thing that matters because unlike my thermostat, my car has a name. Unlike a heat pump, I use it every single day in so many different ways. And this is not me saying that EVs are not a DER, I'm just saying they're a completely unique one.
And I think what that means is that you're about to see two very large industries run up against each other because that's the other aspect, which is if you think the utility is constantly monitoring you, wait until you find out what automakers are doing to you, right? They are also constantly taking a lot of that data in account and saying, how do we build the right personalized experience for you, the driver? Now suddenly you, the driver, are also you, the energy customer, and we've maybe got conflicting things happening here, how do we reconcile the experiences that each of these entities is trying to deliver to the customer and make sure that it's a seamless one that actually brings down the cost for everyone, not just the rich few who can buy EVs today?
Erin Hardick: Let's talk a little bit about bias before we transition to talking about how to maybe address these problems. Jae, can you talk about how detrimental or the impacts that bias can have?
Jae Beom Bae: I think the classification challenge that I mentioned initially is a big source of bias that we've been grappling personally at Leap. So I think what that means is you have to be very intentional about understanding what the various different meters that... It's not just a row in a table, right? So you need to understand the context, where that meter is located, what are the characteristics that define how that meter performs or responds to a market event? And I think we've evolved quite a bit as an industry and personally as Leap on how well we understand what are the drivers and what are the different characteristics and parameters that define what that curtailment curve or predicted curtailment curve looks like for a given meter.
So I think getting that classification right has been quite a challenge and it still is challenging because it's partly, is it just meter type and size of the meter that matters and location? Or is there other factors that we're not even capturing today that we should be for one reason or another is hard that would improve how we do that granular mass customization so to speak? Because in the end that will feed from our perspective of how we represent that given meter in participating in the market, how much they're promising, how much they're selling to the market, as well as how we're aggregating those with other, so to speak, residents in their given virtual power plant.
And I think another area where we've seen bias be quite challenging is in a new feature we're rolling out called non-response detection. So a lot of times Leap is an aggregator of aggregators, so we're an aggregator and there will be another business, it's a B2B model where their job is to operate and manage all these different devices. So often when you have that sort of value chain with multiple steps on the way, you inadvertently, I think, have a lot of friction in how much the signal gets lost on the way. So sometimes what some of our customers have faced is they've faced issues where they've gotten market signals but they, for whatever reason, the devices that were supposed to respond to a given market event, they don't end up showing up.
And often these middle players don't know, sometimes aren't aware whether it's data infrastructure or just lack of awareness. And what we're doing is trying to provide that precision at the meter level so that we can go and take action, whether it's reducing the amount of capacity they're promising to the market or resolving whatever onsite issues they have so that we're rectifying whatever we're providing to market. And that's quite important because as an aggregator, we are kind of the spokesman for all these different players, all these different devices that are plugging into our platform and resolving what's really still persisting, it's probably one of the biggest challenges to the DER industry, which is trust from the utility and regulator side on embracing DERs and really accepting them as a reliable resource, whether it's for capacity or for localized capacity. I think establishing that trust is one of the biggest roadblocks to DER, just proliferation into grid integration. And coming back to your question on bias, minimizing that amount of that bias basically that's seeping into our models is one of the biggest challenges that is worthwhile tackling.
Carlos Nouel: By the way, you know you're on a podcast or event in New York when you hear people blasting their horns in the background so that's probably going to get recorded in the podcast.
I think one of the things, I think we mentioned it when we have the prep call, is all the models we talk about are developed by people. And a lot of the people that are developing those models are people that have been in the industry and know the industry pretty well, and a lot of those folks have been trained for years and years and years to be overly cautious, risk-averse, plan for just-in-case, all those bad things we talk about the industry that we don't necessarily like, but actually keep the industry alive, those things are actually getting baked into the models. And I think the challenge with those models and those biases getting into the models is that all the things that we're trying to get away, those inefficiencies we all talk about getting away from like over-planning, over-forecasting, having over-capacity on certain things, the models are probably going to spit out similar responses because they're getting trained in the same way that the people that are doing the work today do it.
And I think it's upon the people that are doing the models, it's the people that are kind of working with the different companies and the utilities kind of start to monitor and tailor those models in a way that we're taking away some of those biases that we've hated over the years and make sure that they don't get embedded into this next generation of things that we're doing. And we actually ended up with a product that we all talk about, which is we want to be more efficient, we want to see real-time data to make better decisions and not necessarily saying, "Well, I'm going to calibrate this model just in case something goes wrong and I'm going to over-plan for it." So that's the other bias that I'm really worried about that if we don't catch it early, models sometimes are black boxes and if we don't know how it's created, we're going to end up with the same results.
Apoorv Bhargava: And very quickly, I mean, the other thing I just think about is societal bias creeping into systematic machine bias, right? I mean, a lot of people who buy these distributed resources today, EVs very much being that, solar and whatever, are generally very wealthy people. And so are we now doing rate design and setting operational constraints based on what the richest 10% want or are we thinking, exactly to what Carlos said earlier, the rest of society? And I think we're seeing those biases creep up in a lot of other machine learning models and use cases and I would hate for the utility industry, which has been an actual really great equitable platform, to start to see that in their use of AI too. So huge issue that we need to think about.
Erin Hardick: Okay, I'll have one more question and that is for each of you to describe what you think could be possible in a five-year or ten-year timeline. Not in the next year or two, but a little further out. Carlos, we'll start with you.
Carlos Nouel: Look, I am really encouraged over the next five years for two things. One, at the beginning I talked about we can do things with limited data, we're actually going to a world where we're going to have a lot of data and that is going to be a game changer for us.
And I think the second piece is we're actually going to be able to process that data on the edge of the grid, which is a big shift as well as not having to bring data and wait years to process it and then make decisions based on that. So I actually feel that, I see a world where we've talked about what's happening on the grid, we'll actually take all the guests out. You talked about how people drive, we'll know how people use things within their homes, when they cool, when they heat their homes, when they charge their EVs. And if you think about load VPPs and things like that, I don't have to guess what loads would I have to modify. Actually, I can forecast exactly what's in every single feeder, what's the makeup of that load? And then based on that I can actually project what that feeder would look like. And 30% is heating, 20%, is EVs, and 30% it's like the rest. Well, I'm not going to worry about certain other things that I don't want to control during that event, I actually want to focus on EVs and other things because I know it's going to come up into my grid, right?
So I think if we actually take this opportunity, we're going to be smarter and being smarter about it and taking some of this just-in-case planning, it's going to allow us to get to that transition. We have to invest in the grid, we definitely have to, but we will be able to do it in a more efficient way rather than having to keep investing and over-investing in certain areas.
Paul McDonald: I'll go back to something Rayford mentioned on the last panel, I think the only reason the example I'm going to give will take five plus years is because of the need for regulatory motions to accelerate. Today, a problem that is just pervasive in the energy transition is anyone out there that needs to get access to an income-qualified clean energy program, financial assistance program gets put through the ringer in terms of paperwork. The amount of time, effort and just sheer focus it takes to get access to some of these benefits that are out there is so crushing that a lot of people don't avail themselves of these programs.
I think it's entirely possible to use artificial intelligence, to use layered data sets to predict, in the same way that we get notifications that we're pre-qualified for a loan or a higher credit card rate, we're pre-qualified for income-eligible financial assistance or clean energy programs. And for customers whom the confidence interval is high enough on the output of that model, skip through all that paperwork completely and just enroll them at a discount rate. Get them limited income energy assistance program benefits, get them access to a free home upgrade or access to subsidized community solar programs. I think it's going to take time for regulatory bodies to get comfortable with that because there is that need to ensure that there's no free ridership in these really well-funded programs, but I think we can convince them, with the use of AI, that we could skip past that step for a lot of customers that need help.
Jae Beom Bae: I think from my perspective, yes, I think one big reason why I would probably hold my hopes and dreams a little bit more grounded to reality is I think the speed of change that's possible from a regulatory and utility standpoint. I think the industry is currently kind of figuring out what it means for a level playing field for all players involved, and that includes like DERs and VPPs. And so I think right now, I think over the next five years, I think the trust issue probably will kind of, not solve itself, but be solved through all these efforts. And I think as part of that, the proliferation of electrification just throughout the country will kind of work in tandem with that to be able to unlock a massive resource as part of the grid that will then thereby accelerate our path to clean, ubiquitous, cheap energy that's just available throughout.
A big chunk of this is going to be how granular we get with our forecasting and obviously how accurate, but also how granular, because I think a big challenge that the industry will face as a whole will be very localized pain points appearing throughout the distribution grid. And I think VPPs have quite a unique role to play in helping alleviating those very localized pain points. And we're not even there right now, we're just talking system-wide capacity is pretty much like the limit as to how much VPPs have been able to contribute in a meaningful way. But I think that's going to go very local and very granular, very real-time by the time we get to five to 10 years.
Apoorv Bhargava: Yeah, I mean, five to 10 years out you're looking at a world where the potential is there for every customer to have a personalized energy mobility experience. It means that you're being able to use AI to not just drive the best outcomes for them where you're automating their charging to the right times for them, but you're also being able to deliver that same visibility of how their vehicle is charging back to the operator and giving the operator confidence at a technical, and then eventually at a regulatory level too, but giving them confidence in the fact that, "Yes, this device is here, this is what it's doing, this is how it could behave, this is how it will behave." And actually, I mean, I don't think that that's that far out. We are providing that today to a lot of our distribution operator colleagues that we work with at utilities. So that's there.
I think the question is, how much do we get to that place of building a truly personalized experience for the customer? And I think that's what AI can unlock. I also hear a lot of things about we're magically going to give everyone real-time rates, I think that's a terrible idea. I think that is a terrible idea. I think real-time rates are an approach to trying to personalize, but actually passing the entirety of the hedge of the energy markets onto customers and saying, "You become an Enron trader, you become an Enron trader, you become an Enron trader." And I think we have a moment right now where we say, "Okay, how do we actually deliver the best experience for our customers where they actually have stability and certainty in what they are getting from an affordability standpoint, from a cleanliness standpoint, and from a reliability standpoint?"
And I think the utility stands to be the most important entity in making that transition happen. And of course, partners like ourselves will enable that. So that's what I'm excited about. But I think also, this isn't just a technology thing, I completely agree with my co-panelists that a lot of it is regulatory, and we have to have regulators that are open and willing to go and try these things out.
Stephen Lacey: And that's going to do it. The Carbon Copy is a co-production of Latitude Media and Canary Media. This episode was produced by me with help from Dalvin Aboagye and Sean Marquand. Sean Marquand is our engineer and technical director. He also wrote our theme song. If you want to learn more about the trends that we covered in this episode, we're covering this a ton at Latitude Media. So just go over to latitudemedia.com and sign up for our newsletter there. Also, don't forget, Canary Media's listener donation drive is happening right now through the end of the year so make a tax-deductible donation. Just go over to canarymedia.com and you can find the donate button.
Latitude Media is supported by Prelude Ventures. Prelude backs visionaries accelerating climate innovation that will reshape the global economy for the betterment of people and planet. And you can learn more about their portfolio and investment strategy at preludeventures.com. Go ahead, give us a review on Apple or Spotify, I know you want to, and send us your thoughts on social media. We'd love to hear from you and pass this link along if you think one of your friends or colleagues would like to hear what we're talking about. I am Stephen Lacey, this is The Carbon Copy, we'll catch you next time.
