Saving frequency scaling in the data center [LWN.net]

Please consider subscribing to LWN
Subscriptions are the lifeblood of LWN.net. If you appreciate this content and would like to see more of it, your subscription will help to ensure that LWN continues to thrive. Please visit this page to join up and keep LWN on the net.

By Jonathan Corbet
May 21, 2020

OSPM

Frequency scaling — adjusting a CPU's operating frequency to save power when the workload demands are low — is common practice across systems supported by Linux. It is, however, viewed with some suspicion in data-center settings, where power consumption is less of a concern and there is a strong emphasis on getting the most performance out of the hardware. At the 2020 Power Management and Scheduling in the Linux Kernel summit (OSPM), Giovanni Gherdovich worried that frequency scaling may be about to go extinct in data centers; he made a plea for improving its behavior for such workloads while there is still time.

He started with a quote from a car-rally driver: "if in doubt, go flat out". This may not actually be the best advice for drivers of motor vehicles, he said, but it may be the right approach for frequency scaling. Users in data centers often switch to the "performance" CPU-frequency governor, which is not frequency scaling. This governor, which simply runs at full speed all the time, reflects one outcome of the tension between energy efficiency and raw performance. Program managers tend to be interested in performance first, and performance is the first thing that customers see. The cost of power usage is only discovered later, resulting in attempts to hack efficiency into a data-center deployment as an afterthought. It would be better to have that efficiency there from the outset, he said.

He asked the audience a question: assume you are a regional bank running an on-premises data center. Which CPU-frequency governor would you choose? The intel_pstate powersave governor would be the smart choice for now. But the intel_pstate performance governor beckons. The "schedutil" governor is the upcoming next generation. Would you pick one of those, or just go with whatever default the distribution picked? The choice is not obvious. Frequency scaling looks like a risk for a data-center user. Can the distribution vendor be trusted to have made the right choice? For distributors, the bank is a customer who must be catered to. Which governor would you set as the default?

Mobile-first considered harmful

He would like to see frequency scaling be the obvious choice for data-center users and make the performance governor obsolete. Eventually he would like to see the schedutil governor win universally; it's "too cool not to win". But that is hampered by the (in his view) mobile-first viewpoint taken by the developers working with frequency scaling. The object is to save every last bit of energy with the idea that the performance governor exists for users who don't share that goal. That results in frequency scaling stagnating on the x86 architecture, which is relatively rare in power-sensitive settings.

(Your editor, who has been watching for a long time, was amused by this. For many years the complaint was that "big iron" dominated kernel-development priorities; that situation would appear to have changed.)

So what happens if distributors default to the performance governor for x86 systems? One advantage would be that the task of getting the powersave governor into shape could be dropped, along with the complexity that governor brings. On the other hand, he said, the x86 community will lose its grip on technology that it will certainly need someday. Avoiding the powersave governor on server systems will simply paper over bugs that, in the long run, need to be fixed. The last time this topic came up at SUSE (where Gherdovich works) the powersave contingent won, but the issue will come up again.

It seems, though, that the performance governor isn't an obvious choice even now. Dhaval Giani said that it can prevent a CPU from going into "turbo" mode, causing some benchmarks to regress. Rafael Wysocki pointed out that frequency scaling is increasingly done in the processor itself, which can cause strange results when the performance governor is selected.

Gherdovich answered that there appears to be some tension here. The schedutil governor is getting smarter, but the "hardware-managed p-states" feature (called HWP) is pushing things the other way and taking the kernel out of the decision loop. It's not clear how things will play out, and whether frequency scaling will ultimately be controlled in the operating system or the firmware. Wysocki said that the two approaches are not mutually exclusive, though; the operating system works at different time scales than HWP does. It is possible to bring the two into agreement, but there aren't many ways to provide feedback between them now. He has a patch that tries to improve the situation; he will attempt to post it soon.

Continuing, Gherdovich said that defaults set by distributors are critically important; they are the first thing that users see. A distribution will be evaluated with its default settings; if the results are bad, users will move on without ever getting to the point where they try to tune things. So distributors tend to emphasize their default settings when running tests, resulting in far fewer bug reports for non-default CPU-frequency governors. If performance is the default, powersave will get little attention. Additionally, regressions are not something that can be tolerated; if frequency scaling is ever turned off, it will be almost impossible to turn it back on. The chances of creating performance regressions would just be too high.

That, he said, leads to a downward spiral for non-performance CPU-frequency governors. The algorithms in those governors will increasingly be tuned for settings outside of the data center, causing data-center users to lose confidence in them entirely. Distributors will just default to performance, there will be no bug reports, bugs won't get fixed, and frequency scaling will just get worse.

Compromise needed

How do we avoid this dark future? Frequency scaling needs to compromise a bit in the direction of performance, he said, if it wants to win the data center. Often the correct choice for the CPU frequency is obvious, and the governor should go with it. But if there is no information available or the indications conflict with each other, that is the time to favor performance. Any other algorithm will be irrelevant on servers.

For now, he said, the process that is making frequency scaling unsuitable for data centers has not advanced far, but he worries that the priority for upstream developers seems to strongly favor saving power, and he would like to change that somewhat. Wysocki said that anybody who sends patches to the kernel has an agenda — it's why they wrote the patch in the first place. What data-center advocates need to do is to respond to patches that show an agenda falling too heavily on the battery side.

Gherdovich was seemingly ready for that; he countered by bringing up this patch merged by Wysocki in 2019. The "I/O boost" heuristic in the powersave governor assumes that, if a task has been waiting for I/O, it will have work to do once that I/O completes, so the governor increases the processor's operating frequency to get that work done quickly. Prior to the patch, it increased the frequency all the way to the maximum; afterward, the frequency ramps up more slowly. This patch regresses the dbench benchmark, Gherdovich said. Wysocki responded that the purpose of the patch was to avoid starving the integrated GPU of power, and to match an equivalent change made to the schedutil governor.

There may be good reasons for the change, Gherdovich said, but that patch is currently reverted in SUSE kernels, which is clearly a stopgap solution. There are a couple of other out-of-tree patches in those kernels as well, as it turns out. The "idle boost" patch works like I/O boost; it temporarily increases the frequency when a processor exits the idle state. The "ramp up faster" change is an old patch that nobody likes; it causes the frequency to ramp up more quickly when utilization hits a threshold. These patches are expensive to maintain, and SUSE would much rather stick with the mainline.

Patrick Bellasi asked whether any attempt had been made to use uclamp to get the desired performance results; that has not been done. Mel Gorman added that uclamp is disabled in SUSE kernels since it imposes a significant (3-4%) overhead even when it is not used. Bellasi (the author of the uclamp work) was evidently surprised by this and asked for further information, so that problem, at least, may eventually be fixed.

Gherdovich concluded by putting up some numbers. Reverting the I/O-boost patch increases dbench performance by 10%, he said. The performance per watt of power used drops by 23%, which is not a big problem on a server system; users typically do not want to lose that 10% of throughput even if it's costly in energy terms. The full "spicy-powersave" patch set — the I/O boost revert plus "idle boost" and "ramp up faster" — improves kernel build times by 10% with no power cost at all.

At the end, Wysocki asked how much performance data-center users were willing to lose to save some power; Gherdovich didn't have a precise answer but did say that 10% is too much. Wysocki expressed a wish that the CPU-frequency governor work would, in the end, converge on a single solution for everybody, probably in the form of the schedutil governor.

[See Gherdovich's slides [PDF] for details and all the performance results.]

Index entries for this article
Kernel	cpufreq
Kernel	Power management/Frequency scaling
Kernel	Schedutil governor
Conference	OS-Directed Power-Management Summit/2020

to post comments

Saving frequency scaling in the data center

Posted May 21, 2020 19:16 UTC (Thu) by josh (subscriber, #17465) [Link] (6 responses)

> The number of operations per watt of power used drops by 23%, which is not a big problem on a server system; users typically do not want to lose that 10% of throughput even if it's costly in energy terms.

That depends on the datacenter. If you own your own datacenter, and you pay for power, performance per watt may still matter. If you're running on someone else's servers and they don't charge you for power, performance per watt stops mattering, and you may want to pay a disproportionate amount of power to get more performance.

Saving frequency scaling in the data center

Posted May 22, 2020 5:22 UTC (Fri) by riking (subscriber, #95706) [Link]

With all the talk about "Total Cost of Ownership" (over 5 or 10 years) coming from AMD, you'd think there'd be more of a focus on performance per watt. Perhaps we need better benchmarks?

Another thing to keep in mind when designing such benchmarks is that servers are not necessarily active all hours of the day - in the traffic trough, there's less work going on and it's okay to slow down and you can C-state some of your cores...

Saving frequency scaling in the data center

Posted May 22, 2020 12:41 UTC (Fri) by ju3Ceemi (subscriber, #102464) [Link] (4 responses)

Power is part of ones datacenter bill

When you get rack space somewhere, two things are marked on the contract :
- how many rackunits
- how many Watts

Saving frequency scaling in the data center

Posted May 22, 2020 14:27 UTC (Fri) by epa (subscriber, #39769) [Link]

But if you use less than your maximum power allowance over the month, do you get some money back?

Saving frequency scaling in the data center

Posted May 22, 2020 20:23 UTC (Fri) by amarao (guest, #87073) [Link] (2 responses)

Nope, no one charge for energy in DC. The bill is usually is an allowed peak load or 95% percentile of that load. Even the electric companies often charge DC based on contracted power (watts), not the energy (joules or kw*hr, like household bills). The reason is that dc reserves specific power from supplier, and that power is often billed as 'take or pay', as it is not oversubscribed on the supplier capacity.

Saving frequency scaling in the data center

Posted May 23, 2020 11:53 UTC (Sat) by mathstuf (subscriber, #69389) [Link]

This sounds like it's also what allows these cloud companies to basically give away their CPU time to CI infrastructure to FOSS for free. Since the extra CPU usage doesn't actually cost them more in resources, they can instead buy some goodwill with the cycles instead (sure there's some personnel overhead, but once it's built out I suspect that it isn't too much to tack it onto the bottom of the sysadmin team's laundry list).

Saving frequency scaling in the data center

Posted May 23, 2020 18:24 UTC (Sat) by bugfood (subscriber, #124228) [Link]

> Nope, no one charge for energy in DC.

This can vary per colo provider. "Metered power" (which should really be called "metered energy") contracts are available. I can't speak to their relative prevalence, but they are common enough to matter.

https://www.greenhousedata.com/blog/colocation-pricing-ex...

My own experience with being on a "metered power" contract, and from looking at several competitive offers, is that the rent was on the order of 2/3 of the total bill. Since the rent is fixed for the duration of the contract, this limited the potential immediate savings from power reduction to 1/3. That said, being at a lower power level is good when it comes time to get the next contract, though of course colo providers will charge lower $/kW for higher kW allotments, so the savings are not linear. Different contracts from different providers in different locations at different times will vary....

Regarding use of frequency scaling on servers, running at higher frequencies means being able to get each unit of work done more quickly, which means being able to spend more time in higher C states. It's been a while since I measured this, but back on Sandy Bridge servers, using the performance governor resulted in substantially more time in higher C states, which presumably partially offset the increased power usage from higher frequencies. Also, as CPU usage gets higher, the difference between governors becomes lower, and it is good to consolidate workloads until CPUs have a meaningful load anyway, in order to avoid wasted servers. With all that, I don't hesitate to use the performance governor on tiers where latency matters at all; on the other hand, anything asynchronous (e.g. kafka) or high-latency (e.g. email) can generally use ondemand with no real downside.

All of this comes down to a business decision: determine your needs, measure the impact, calculate the cost, and make a decision. Having a less conservative governor would simply be another option to measure, and that doesn't sound like a bad thing.

Saving frequency scaling in the data center

Posted May 21, 2020 19:21 UTC (Thu) by josh (subscriber, #17465) [Link] (3 responses)

To what extent could this depend on whether the system is currently running on battery? If you're running on battery, you may want a default policy of "save as much power as possible", albeit that needs to not cause increased power usage overall on a workload (such as making something take 30% longer to save 5% power, which is not a win if you're trying to get that workload done before you run out of battery). If you're running on mains power (and you're not in the "large datacenter with substantial power and cooling costs that you're paying for" use case), you may want more affordances for performance.

Back in the say, making *anything* conditional on "running on battery" was seen as a mistake. Is that still true today, though? Might that help make defaults work for more people?

Saving frequency scaling in the data center

Posted May 22, 2020 10:26 UTC (Fri) by LtWorf (subscriber, #124958) [Link] (1 responses)

Unfortunately the workload of browsers is to take a lot of CPU permanently so that websites can run their useless js.

In the case of browsers, the entire idea of making the CPU running faster to get the work done sooner does not work, because it's just endless loops all over the place.

Saving frequency scaling in the data center

Posted May 24, 2020 0:10 UTC (Sun) by josh (subscriber, #17465) [Link]

That's a problem to be solved in browsers. The OS's job is to run the workloads given to it, and to attempt to balance power and performance in doing so.

Saving frequency scaling in the data center

Posted May 25, 2020 0:39 UTC (Mon) by marcH (subscriber, #57642) [Link]

> Back in the say (day?), making *anything* conditional on "running on battery" was seen as a mistake.

Weird. I love defaults that work in every case but power has obviously very different "prices" depending on the context:

1. Super expensive because I absolutely need my battery to last at least the day
2. Somewhat expensive because my data center charges me - yet I value happy customers more.
3. A bit "expensive" because I don't want to get up from the couch and look for my charger
4. Free because my data center doesn't charge me.
Etc.

OK, we probably don't want 5 or 10 different power settings like some Windows OEMs provide but two settings ("I care" vs "I don't care much") seems like a really bare minimum.

Saving frequency scaling in the data center

Posted May 21, 2020 21:51 UTC (Thu) by flussence (guest, #85566) [Link] (2 responses)

Running a datacentre with no power management at all seems exceptionally short-sighted. Sure, you *can* spec a gargantuan HVAC that can handle a sustained 200-300 watts of TDP per 1U, but if your room's constantly at thermal capacity like that then so are your systems.

I'm surprised the intel_pstate driver performs inconsistently enough to merit all this discourse in the first place. I just tried flipping between schedutil and performance on my Zen2 (using the bog-standard ACPI driver) and the turbostat numbers are more or less identical under load and idle for both, and the chip's overall productivity is too.

Saving frequency scaling in the data center

Posted May 21, 2020 22:16 UTC (Thu) by wmf (guest, #33791) [Link]

No frequency scaling doesn't mean no power management; C-states are still enabled by default. Maximum frequency with C-states enabled is basically the "race to idle" policy which is not a bad policy.

Saving frequency scaling in the data center

Posted May 27, 2020 1:09 UTC (Wed) by da4089 (subscriber, #1195) [Link]

I used to work with HFT systems in colocation space worldwide. We always paid by the cabinet, with tiered options for available power. So you pay for the peak wattage, and the datacenter is built to handle whatever heat load they think is going to eventuate (that was totally not our problem).

There's basically zero incentive to save power, and literally millions of dollars of incentive to ensure that the system is maximally responsive to demand at every microsecond of the day, from 7am Monday in Auckland until 5pm in New York on Friday.

So, yeah, it probably does mean that the datacenter has a massively over-provisioned HVAC, but ... there's no demand to do anything about it.

If you're a FAANG, running your own fleet of datacenters, the calculations are probably different.

Saving frequency scaling in the data center

Posted May 22, 2020 15:09 UTC (Fri) by intelfx (subscriber, #130118) [Link]

> But if there is no information available or the indications conflict with each other, that is the time to favor performance. Any other algorithm will be irrelevant on servers.

> For now, he said, the process that is making frequency scaling unsuitable for data centers has not advanced far, but he worries that the priority for upstream developers seems to strongly favor saving power, and he would like to change that somewhat.

Why is this a dilemma at all?

You'd think there is this weird thing called a "governor", of which there are at least two helpfully named "powersave" and "performance"...

Saving frequency scaling in the data center

Posted May 22, 2020 20:16 UTC (Fri) by amarao (guest, #87073) [Link]

We've just had a lengthy discussion on this topic (dc business). The conclusion was that it's all overcomplicated. C states, p states, power governor, idle governor, power driver, idle driver, cpufreq, intel_pstate, ondeman governor which worse than powersave, etc, etc.

The opinion was that this is just too much for a simple feature to save some power. So we've opted for unconditional performance.

It would be nice to have a way to trim power a bit, but not at cost of years of stidy of that spaghetti bowl of leaking abstractions and hardware wishthinking.

I appreciate efforts of kernel people a lot, but whole powersaving saga demands more that returns back.