If you aren't the CEO, or have a direct line of influence to him, this issue can't be resolved. Your company didn't technically solution itself into this problem, and it won't technically solution itself out of this problem.
It requires painful, swift decisions and a massive culture shift. I would bet you have a CIO(or similar title) who owns the legacy stuff and a CTO where shadow IT lives. One of those people needs to report to the other and if that is the case today, one needs to go (which is likely the case regardless).
Absolutely. And not even for hardware specifically -- with software, too. I've been seeing a big uptick in customers wanting to go from SaaS/cloud to self-hosted/on-prem options. This is actually the main reason I chose to open source my SaaS after 7 years [^0], to provide more ways to self-host it on-prem. There's a lot of other reasons in that blog post, such as the current market, but also touches on things like bus-factor, security, and compliance.
In the end, it kind of boils down to the big questions "what do you own?" and "what do you control?"
When the markets get choppy, people start asking questions that they otherwise wouldn't ask. There's a lot of choppiness right now, and businesses want to reduce risk. Actually owning and controlling hardware, and software, can reduce risk via less reliance on third-parties. But it can also reduce costs, like with 37 Signals [^1], which is yet another risk reduction.
Another question to ask (from a business perspective), in order to reframe your concerns, would be "where is our risk?"
Perhaps they're reducing risk by having one foot outside of the cloud.
[^0]: https://keygen.sh/blog/all-your-licensing-are-belong-to-you/
[^1]: https://world.hey.com/dhh/why-we-re-leaving-the-cloud-654b47...
Have the on prem team start deploying some assets that can utilized by the cloud team via a familiar interface like terraform. (metal as a service, kubevirt, Vsphere, nebula, whatever)
Start small, pick one service at a time (compute this quarter, postgres next quarter, secrets management the next) and use the emerging platform to strangle out paid services
Cloud team gets to stay cloudy, on prem team still gets to farm servers
The choice doesn't have to be solely on-prem OR cloud. We're seeing a hybrid approach often as a first step. More predictable, fixed workloads can move back on-prem for cost-efficiency, while keeping elastic workloads in the cloud for flexibility.
The key to success is having the right on-prem solution to help bring both teams together while driving costs down. Our solution at Triton Datacenter (https://tritondatacenter.com) has helped various enterprises tackle this issue by providing a public cloud-like experience on-prem.
If your management has been letting that wasteful situation go on for a long time, there's not much you can do. It sounds like there's some dysfunction in management. I've seen that in many businesses. Unfortunately, the money they're wasting on maintaining two environments is probably preventing the company from moving forward in other ways.
That said, a number of companies have left the cloud for cost reasons. 37 Signals is a recent high-profile example: https://world.hey.com/dhh/why-we-re-leaving-the-cloud-654b47...
I've read of others leaving for cost reasons as well. Many run niche workloads that don't make sense in the cloud. One company, if I recall correctly, was doing scientific processing that required massive, but not unlimited, CPU and GPU resources. They found it was cheaper to buy a bunch of high-end workstations and run their jobs locally. They didn't need the cloud's 24/7 uptime guarantees, because they were doing offline processing with no public-facing apps.
Ahrefs saved $400M for not going to the cloud
https://tech.ahrefs.com/how-ahrefs-saved-us-400m-in-3-years-...
I just left a $45B company that has many of the same issues you mention. There were efforts around consolidating competing technologies and vendors as well as a very large "digital transformation" effort which I was directly a part of. None of the efforts went very well for a variety of reasons including reorgs, two competing technology organizations, hard economic times, etc. Another underlying issue I saw was lack of accountability and communication. "Technical debt factory" is a great way to phrase it.
Ultimately, leadership has to make tough decisions cut costs while not impacting the company's continued growth.