I'd take the RTX 3090 comparisons with a huge grain of salt for the moment.
First of all, Apple's performance comparisons are against the mobile Nvidia cards, not the desktop ones. For example, the M1 Max comparisons with the "RTX 3080" were against the 100W mobile part, not the 320W desktop part. The mobile version is about 40% slower than the desktop one (something that I do find irritating about Nvidia's marketing). The desktop gaming cards are typically trading power consumption for performance. They're clocked well beyond the optimal efficiency point to squeeze everything possible out of them. Cutting the power target in half may only cost you 20% of your performance.
Another thing to note is that Nvidia pretty much holds all the high-efficiency binned dies for the enterprise cards. In terms of GPGPU, the data center cards are nearly twice the performance per watt of the consumer cards by going wider and dropping the clock rate.
Small edit:
The best benchmarks I could find suggested that the M1 Max was drawing ~105 watts with the GPU under under a large synthetic load and doing about 10 TFLOP/s of FP32. Let's assume the M1 Ultra is exactly double this and can do 20 TFLOP/s at 200 watts. The Nvidia A100 GPUs are the same performance but at 250 watts. The RTX 3090 is upwards of 36 TFLOP/s but approaching 400W of power draw.
By comparison, the fastest supercomputer in the year 2000 was ASCI White, which did 4.9 TFLOP/s. It cost $168M in 2020 dollars, took up 12,000 sqft, weighed 106 tons, and used 3 megawatts (+ another 3MW for cooling... = 6,000,000 watts). Granted, it had 6TB of memory and 160TB storage...but with 7,000 disk drives. (https://www.top500.org/resources/top-systems/asci-white-lawr...)
The M1 Max itself is a the size of a postage stamp, 10 TFLOP/s, costs ~$200 to make, and uses ~100 watts (= a single incandescent lightbulb).
First of all, Apple's performance comparisons are against the mobile Nvidia cards, not the desktop ones. For example, the M1 Max comparisons with the "RTX 3080" were against the 100W mobile part, not the 320W desktop part. The mobile version is about 40% slower than the desktop one (something that I do find irritating about Nvidia's marketing). The desktop gaming cards are typically trading power consumption for performance. They're clocked well beyond the optimal efficiency point to squeeze everything possible out of them. Cutting the power target in half may only cost you 20% of your performance.
Another thing to note is that Nvidia pretty much holds all the high-efficiency binned dies for the enterprise cards. In terms of GPGPU, the data center cards are nearly twice the performance per watt of the consumer cards by going wider and dropping the clock rate.
Small edit:
The best benchmarks I could find suggested that the M1 Max was drawing ~105 watts with the GPU under under a large synthetic load and doing about 10 TFLOP/s of FP32. Let's assume the M1 Ultra is exactly double this and can do 20 TFLOP/s at 200 watts. The Nvidia A100 GPUs are the same performance but at 250 watts. The RTX 3090 is upwards of 36 TFLOP/s but approaching 400W of power draw.