Frontier Forum: Inside Crusoe's energy-first approach to data centers

AI is enabling a multitude of solutions across power, industry, and transportation. But AI energy demands are increasingly stressing the electric grid — creating a bottleneck for growth and new challenges for clean energy supply.

The mounting tension highlights the need for an energy-first approach to computing. 

Developer Crusoe is building AI infrastructure that takes advantage of clean energy to power workloads for AI modeling. Likewise, Nvidia, Crusoe’s primary GPU supplier, has been consistently improving the energy efficiency of its GPUs. Both demonstrate the innovation that’s happening in the marketplace to create a 'climate-aligned cloud' for customers.   

In the AI era, how do you build data centers with an energy-first approach?

In this Frontier Forum, Stephen Lacey explores all sides of the AI-energy nexus in conversation with Chase Lochmiller, Crusoe's co-founder and CEO. They discuss innovations in data center design, why the energy demands of AI could be higher than projected — and why that shouldn't scare us.

‍Chase Lochmiller will be speaking at Latitude Media’s Transition-AI 2024 conference on December 3 in Washington, DC. Get your tickets here.

This is a partner episode, brought to you by Crusoe.

Transcript

Stephen Lacey: In the last year or so, you've expanded to focus on these permanent data centers more specifically to serve AI. These are facilities in the hundreds of megawatts, they will have hundreds of thousands of GPUs. What does it mean to apply an energy-first lens to building those larger data centers?

Chase Lochmiller: We are thinking about both the cost and the environmental impact of the energy resources that are going to be powering our data centers. So in the same way that we solve this natural gas flaring problem, we've tried to focus on building data centers and building large clusters for AI computing in locations that have low-cost, clean and abundant energy. So markets like West Texas that have been heavily developed with wind and solar but have massive amounts of curtailment and negatively priced power.

Other markets like upstate New York, we are working on a former Alcoa factory that's powered by a large hydro dam that's being currently massively underutilized. And so a lot of these more brownfield-type assets that can be utilized in a way to power energy-intensive compute workloads, as well as new greenfield development where you can look at NASA and say, "Hey, it sure would make a lot of economic sense if you had a use for the power to build generation in this location." Well, we can say, great, let's partner up with an IPP and develop a data center alongside new greenfield generation capacity. What we do is we partner with renewable energy producers like these IPPs and we can partner with them behind the meter where we're taking a load behind the meter and we leverage the existing substation infrastructure and actually set up a grid connection as sort of a backup resource.

Stephen Lacey: So you looked at this space and you looked at AI specifically and determined that you were in a once-in-a-lifetime moment to meet the computing demands of this technology as cleanly as possible. There are a lot of different projections out there for how big the energy impact will be. IEA, Goldman Sachs both belief electricity demand will double in the next three to five years. Morgan Stanley projects that in 2025, generative AI alone could count for a third of the total computational demand from data centers we saw in 2022. Regulated utilities could see capital investments of five to $10 billion annually to meet this resulting power demand. When you look at these projections and what is actually happening in the industry, how explosive do you think energy demand from data centers will be and why did you decide that this was the area that you wanted to serve?

Chase Lochmiller: I sort of think that the demand is actually under forecast. All that being said, I think there's a lot of incredible opportunities with that. Power usage on the surface isn't necessarily a bad thing, especially if it's advancing humanity forward and advancing and developing all these incredible new technologies including inventing new ways to mitigate climate change and advance an effective energy transition. We work with a number of different companies that are doing just that, ranging from advanced physics modeling systems to develop next generation fusion architectures, developing more advanced climate adaptation strategies with advanced weather modeling techniques, developing new battery chemistries, and we just had a great press release with SES around a lot of the battery chemistry modeling that is fundamentally pushed forward in ways that we can invent new techniques to basically create more effective grid storage battery solutions. We were even talking to someone that's working on a custom engineered material that's empowered by a foundational inorganic chemistry model that can provide more cost-effective and more engineered direct air capture systems.

And I think there's just a lot of very, very interesting innovations that are going to take place from this technology, but we do believe that it's going to consume tremendous amounts of power. I think we kind of view that as an opportunity that load itself, if that load can catalyze net new greenfield generation that's clean, whether it's building data centers in areas like West Texas that have abundant wind and solar and you can deploy large battery clusters as the cost curve comes down or exploring... one area we're spending a tremendous amount of time is on carbon capture and sequestration, so net new gas generation with carbon capture and sequestration where we can use a lot of the existing fossil fuel production and infrastructure to produce power, but capture the negative impact from the CO2 emissions there.

So we're sort of looking at everything and I think the energy demand is immense. One other great example of the type of deployment that we do is we're doing quite a bit in Iceland where geothermal and hydropower is low cost, clean and abundant, and we're able to sort of develop these large GPU clusters in Iceland powered entirely by clean energy and obviously there's benefits there in terms of it being an easier environment to manage from a cooling perspective, but we think that's increasingly going to become an important market to power next generation AI applications.

Stephen Lacey: I appreciate that perspective because I think when you talk to, let's say folks at the large tech companies privately, they'll say, "Yeah, this is going to use a lot of power," but then publicly they'll say, "No, we think we can solve for this problem, but we think the benefits to the society are much greater than the resulting power demand increases," and they hedge a little bit. What you're saying is, yeah, this is going to take a lot of power and yeah, I think that the net benefit is going to be very high for society and let's just deal with the power increases. Is that a fair way to characterize it? I like that you're tackling it head on and not hedging.

Chase Lochmiller: Yeah, I mean we need to address it head on because, and I think what you're seeing is that... I mean if you think about just the power consumption of an H100, which is the current generation of NVIDIA chips, like if every American adult used half an H100 as a form of Copilot for their daily workflows and social interactions, et cetera, that would require 250 gigawatts of power. These are order of magnitudes bigger than the forecast that people are providing and these models are only getting bigger, more complex, and there's a bit of a positive reflexivity to it where it's like as you actually get utility from using these AI applications, whether it's for scientific discovery or social interactions, as they get better, you use them more right? That's like the positive reflexivity aspect to this that is going to therefore drive more demand for more compute and that is going to consume more power.

I don't think this is an unsolvable challenge is my point, and I think it's actually an interesting opportunity for the ICT industry at large to basically help shape what the future of our grid and what our generation infrastructure looks like. Because one of the superpowers of AI that we haven't really talked about is that it is far more tolerant of latency than many, many other applications. So you can actually position where that load goes and you can really take this sort of energy first approach. That's why we're building data centers in West Texas. It's not a traditional data center market, but a lot of people are coming there because the product itself is so compelling from clean and large scale data centers that can power their compute workloads.

Stephen Lacey: So I want to get to some of the solutions. Just another question about the power supply issue. How much of a bottleneck is power supply compared to say chip availability right now for scaling AI? Is power the main constraint?

Chase Lochmiller: Power is the main constraint right now. I would say the in AI have moved around. A year ago today there was this fever of being able to get enough chips and I think Elon Musk famously said something along the lines of like, "NVIDIA H100s are significantly harder to buy than illegal drugs right now." And it was definitely a crazy fever to get the chips, but once the data center capacity was sort of fully absorbed, we ended 2023 sub 2% in vacancy rates across data centers in the US.

When you think about what that is compared to another commercial real estate asset, that's like tiny crumbs of capacity that are sort of spread across people's portfolios. Today, I think it's sub 1%. So there's just not enough data center capacity and what's limiting that data center capacity build oftentimes is the ability to access the power, and there's multiple different reasons for that. It can be grid and our connection cues. It can be high voltage transformers, it can be switch gear, it can be backup generation. There's all these different components in the overall supply chain that prevent us from being able to sort of meet the demand side of the industry.

Stephen Lacey: So talk about the cloud product that you've developed. You call this the climate aligned cloud. It's helping some of these companies actually train their models in a less energy intensive or emissions intensive way. How is your cloud product actually structured for these customers?

Chase Lochmiller: For our customers, what they consume is virtual IT resources. So what we provide is virtual machines and compute storage and networking and a high performance configuration that's really purpose-built and design for AI innovators to run generative AI applications, train large language models and do so with the highest performance compute and networking architectures. Now, what makes it special or one of the things that makes it special for those that care about the emission footprint is that it's powered by clean energy. And like I mentioned, we are making significant investments in Iceland. This is not a new trade, so to speak. The aluminum smelters are famous for going to Iceland to run the very energy intensive process of smelting in Iceland. So they actually ship the raw aluminum ore across the ocean, they do the smelting in Iceland and then they sort of ship the finished products to their end destination.

Well, if you think about a large language model and what needs to be done to train one, this is... training a large language model is a far better manifestation of that trade where you can move data across subsea cables to get to a data center based in Iceland. You can run your back propagation and large-scale pre-training on significant amounts of data to train your LLM, and then you can sort of ship the finished product to wherever you need it to go, or you can run inference by sort of feeding tokens again across those subsea cables and then getting the output back to wherever you need it. So I think there's this philosophy that moving data is quite a bit easier than moving real world physical materials, and it's a way of sort of relocating the energy-intensive compute workload in an area that has that low-cost clean and abundant energy.

Stephen Lacey: What's harder, building the data center, building the network, or building the cloud product?

Chase Lochmiller: They're all hard for different reasons. They all sort of face their own set of challenges and engineering problems. We've sort of brought together a great group of experts across each individual component. Building a data center, you're sort of amassing a lot of different labor and trades to make it happen. And I think that's actually one of the interesting phenomenon of this AI boom that not a lot of people are talking about is sort of the boom in blue-collar labor workforces. So we're experiencing massive shortages of electricians, of welders, of plumbers, these large-scale liquid cooling architectures that we've described that require tremendous amounts of plumbing. So you're building all these different pipes to move water around to cool the data center and a lot of these blue-collar trades are just inherently, we just don't have enough people to do them and it's sort of creating a boom for that whole sector of the economy and a revitalization of areas like the Rust Belt.

So to me that's one exciting trend from a labor perspective. I think there's a lot of challenges from that angle on the data center side. The network, it all depends on where you're at, the complexity there, but there's a bunch of different facets of the network engineering challenges that we face between the long-haul physical network of how we're going to get data in and out of the data center to the high-performance network, the local area network within the data center that enables people to share data from GPU to GPU in a super high-performance capacity with our rail-optimized networking architecture and being able to virtualize all that and do so through our software-defined network is a whole new set of engineering challenges that have been fun to work through as a business.

Stephen Lacey: Are there any big picture changes you see in the data center industry right now that you think are shaping the industry broadly or shaping the size data centers that you're pursuing? What do you think some of those big shifts are right now?

Chase Lochmiller: I think the biggest shift is just the scale. The number of people I talk to that are like, "Yeah, I want a gigawatt data center," I mean it's like a remarkable number of people that are asking for that scale of capacity.

Stephen Lacey: Will you build in that scale or is it the hundreds of megawatts?

Chase Lochmiller: Yeah, that's like in our pipeline. I think we have about 12 gigawatts in our pipeline of either development or in advanced commercial discussions. So it is absolutely in our wheelhouse to go that big, but I do also think that people are trying to find ways because some of the folks asking for one gigawatt actually want 10 gigawatts and that sheer scale, it's a very challenging problem to get in a single location. And so people are looking at ways to sort of link together data centers across different geographies and sort of create a more synchronized, decentralized computing footprint.

I think one thing to remember for folks, especially for training these cutting edge foundational models is that one of the complexities that we've actually had to engineer around is that the cluster itself is moving in sync with one another. So all of the data is sort of being broadcast out across this high performance network to the GPUs. They're running their compute workloads and then publishing the results to everything. So it's almost like the cluster is breathing. You have these big spikes in power and then low spikes in power, and then big spikes in power, and it's like basically every GPU is sort of moving in sync and that frequency and they're doing this hundreds of times or thousands of times a second, and with that crazy frequency of power demand draw, it sort of has introduced other complex challenges in terms of power management.

Stephen Lacey: So you work across the entire value chain. When you look at future efficiencies of chip design, how are you... like NVIDIA chip design, how are you collaborating on data center design to maximize performance and sustainability potential?

Chase Lochmiller: I think the big architectural shifts that we have to engineer around are really this power density aspect. If you look at the last generation NVIDIA chip, it was the Ampere, so the A100, that consumed about 300 watts per chip. The Hopper, the current generation, the H100, H200, those are about 700 watts per chip. And the Blackwell is sort of the next generation, the GB200, that'll be about 1200 watts per chip. So the power density is going up, which I sort of alluded to this earlier, but that's the biggest engineering challenge from the data center perspective of how do you cool those systems effectively. From a sustainability aspect, I think there's the energy usage, but there's also things like water usage. So we've tried to do things with closed loop systems. We're able to sort of minimize the amount of net water that we're consuming from the data center's perspective.

Stephen Lacey: What kind of advancements are firing you up right now? What gets you excited about the future?

Chase Lochmiller: I had a really cool conversation with this company, Orbital Materials and Orbital materials, the team had previously come from Deep Mind working on a lot of the foundational models for inorganic chemistry, and one of the things that they were proposing to me was basically a purpose-built material for our heat profile. One of our big byproducts of running a data center is heat right? It's either hot water or hot air sort of coming out of the cooling systems.

Now that's typically going to waste or we have to chill the water back down or the hot air is kind of going out the tail end of the data center. But what was being proposed is you can actually custom engineer a material, a direct air capture material that can sort of utilize heat at a specific temperature range to basically absorb carbon and also in the water case, chill the water. It's a bit like science-fictiony, but you could imagine a future where you're powering the data center with onsite clean energy cooling loop of the system is actually running a direct air capture system and you're sort of creating a net carbon negative data center running large scale AI workloads.

Stephen Lacey: Are there any other major advancements that you're working on that you will iterate over the course of coming data centers? Do you think we'll see major changes from over the course of your next couple of data centers? How much do you expect things to change technologically?

Chase Lochmiller: Yes, I do. And a lot of it is coming from how do we optimize the process of deploying large scale clusters of GPUs and how do we stand up these data centers in a faster and more cost-effective manner? So a lot of that comes back to this modularity aspect of if you're going to build in a very remote market that has this low cost and clean energy, oftentimes getting labor there for onsite construction can be a huge challenge.

So the more you can do offsite, the faster you can deploy onsite and the lower your cost can be because you're doing a lot of the hard manufacturing work in a controlled manufacturing environment. I think a lot of the work we're doing is directly in that domain. A lot of the other work is in terms of how do we manage cabling in a more cost-effective and speed-effective manner. If you look at one of these big clusters that we're deploying, it's over a million strands of fiber right? That's just like a crazy amount of things to manage. So anything that you can do and value engineer from the data center perspective to streamline a lot of those processes is pretty cool.

Stephen Lacey: So there's a lot of concern about the energy impact of this industry, and what I'm hearing you say is you think we have a lot of the solutions to solve for the problem. Of course, power availability is a huge constraint right now for scale, but it feels to me like you think we have a lot of the business models and clean energy technologies to solve a lot of the problem. Is that right? And is there anything about... you seem pretty optimistic. Is there anything that you want the people who are pessimistic about our ability to solve this challenge, to know about what you're seeing out there that makes you optimistic?

Chase Lochmiller: Look, I'm optimistic in terms of the load itself has control over how we're going to be demanding power just as an industry. And I think people are philosophically aligned with people want the future of AI to be clean and sustainably powered. I think that the pessimists, what I would say is that the technological leap forward that we stand to gain from deploying this technology at scale is so massive in terms of being able to invent any solution we could imagine that would help solve all of our sustainability challenges as a society at large is right there for us. It's like you could take two different perspectives right? One is like, "Oh, let's put the genie back in the bottle and we open Pandora's box and we need to shut this thing down before it destroys us all." My perspective is like, that's not going to happen, so why try to fight it and recognize that the innovative potential of this thing is absolutely enormous in terms of inventing things that fundamentally wouldn't be possible with traditional techniques across every industry, but especially around sustainability and clean energy production.

Stephen Lacey: Chase Lochmiller is the co-founder and CEO of Crusoe. Chase, I really enjoyed this a lot. Thank you.

Chase Lochmiller: Thank you, Stephen. I enjoyed it as well.