Ethernet Compatibility List for macOS [1GbE, 5GbE, 10GbE, 25GbE, 40GbE, 50GbE, 100GbE]

[PeterMack]

Thanks for all the help, it is finally working! Those last two tutorials were especially useful. I could not be happier now. Well, if iperf would start measuring 25G instead of 11-ish, that would be better. But I'm sure I will sort that out too.

 

[EliteMacx86]

Thanks for all the help, it is finally working! Those last two tutorials were especially useful.

Glad to hear!

I could not be happier now. Well, if iperf would start measuring 25G instead of 11-ish, that would be better. But I'm sure I will sort that out too.

To diagnose that, need to know your exact setup details

- Are you using direct Ethernet to Ethernet or via any Switch/Router? If so, model
- Transceiver model/part number
- Are you using a DAC or LC? What's the cable length? Regardless of both, need the proper model/part number

Other than that, did you test using iperf3? What is the client end computer?

Also, in which PCIe SLOT the NIC has been installed?

 

[PeterMack]

- Are you using direct Ethernet to Ethernet or via any Switch/Router? If so, model
- Transceiver model/part number
- Are you using a DAC or LC? What's the cable length? Regardless of both, need the proper model/part number

Other than that, did you test using iperf3? What is the client end computer?

Also, in which PCIe SLOT the NIC has been installed?

Hi, I have tried to resolve the connection issues, but I'm stuck with it, it seems not even my direct thunderbolt connection works anymore, on which iperf was working fine up until I installed these cards yesterday.

So here are my answers, as much detail as I can think of:
Connecting cable is direct DAC SFP28 to SFP28, both cards are the same, DAC is "10Gtek 25G SFP28 DAC 2m, 30AWG, Passive, for Mellanox"
One machine is MacOS, other is Windows 11 Pro.

In both computers the Mellanox card is in the last (bottom) PCIE16_3 slot.

Ping works both ways, Remote Desktop works.

Iperf only works if Mac is client, Win is server, there is no connection the other way around. But this is also true for my Thunderbolt IP Bridge, which up until yesterday worked fine both ways, running at 10g-ish speeds with iperf. Now it only works if Mac is client, Win is server. Seems like this is an OS level problem.
The iperf speed that works with the Mellanox card is around 10-12Gbit, I can get it up to 15Gbit by using the "-w 500k" switch.
Both cards are configured automatically, with static IPs, MTUs are 1500, no gateway or DNS defined.

 

[EliteMacx86]

Hi, I have tried to resolve the connection issues, but I'm stuck with it, it seems not even my direct thunderbolt connection works anymore, on which iperf was working fine up until I installed these cards yesterday.

So here are my answers, as much detail as I can think of:
Connecting cable is direct DAC SFP28 to SFP28, both cards are the same, DAC is "10Gtek 25G SFP28 DAC 2m, 30AWG, Passive, for Mellanox"
One machine is MacOS, other is Windows 11 Pro.

In both computers the Mellanox card is in the last (bottom) PCIE16_3 slot.

Ping works both ways, Remote Desktop works.

Iperf only works if Mac is client, Win is server, there is no connection the other way around. But this is also true for my Thunderbolt IP Bridge, which up until yesterday worked fine both ways, running at 10g-ish speeds with iperf. Now it only works if Mac is client, Win is server. Seems like this is an OS level problem.
The iperf speed that works with the Mellanox card is around 10-12Gbit, I can get it up to 15Gbit by using the "-w 500k" switch.
Both cards are configured automatically, with static IPs, MTUs are 1500, no gateway or DNS defined.

Provide Transceiver model/part number

Shift the NIC to PCIE16_2 and remove the TB card for now. Reset NVRAM and then check via iperf3. You should be able to get 22-23Gbps throughput.

Also, attach the updated EFI.

 

[PeterMack]

Transceiver is CAB-ZSP/ZSP-P2M, made by 10Gtek. Both cards are in a PCIe X8 slot and both are running at X8 speeds (PCIe v3.0). My motherboards are not restricting the bottom slots as one CPU has 44 lanes, the other has 48. Also I don't have empty PCIe slots and the hardware seems to work fine so I did not start shuffling the cards around.
But I fixed the blocked iperf issue, I was using the wrong iperf version, it turned out that my OS also had an Apple compiled iperf, v3.8.1, but what I could download from their website was only 3.1.3. I also had to find a matching Windows version for it, but now that I knew what I was looking for this was easy. So now iperf works both directions, and I am getting 17Gbit both ways. Now I am guessing that this is due to card settings like MTU or whatnot.

Update: by using the -P switch I made iperf run multithreaded, and now I am getting 23.7Gbit.

It seems iperf was restricted by my 7980XE's single threaded performance, which is not a surprise as I can't even use turbo mode, the CPU can only hit 3.6GHz maximum instead of the official 4.4 (or 4.7 overclocked). This is due to my crappy temporary motherboard, it does not have enough power stages and it is not stable with the 18 core cpus. My replacement motherboard is already on the way, another Prime Deluxe II, most likely getting it tomorrow.

 

[lopess]

Need IOReg to diagnose the issue.

 

[EliteMacx86]

Transceiver is CAB-ZSP/ZSP-P2M, made by 10Gtek. Both cards are in a PCIe X8 slot and both are running at X8 speeds (PCIe v3.0). My motherboards are not restricting the bottom slots as one CPU has 44 lanes, the other has 48. Also I don't have empty PCIe slots and the hardware seems to work fine so I did not start shuffling the cards around.
But I fixed the blocked iperf issue, I was using the wrong iperf version, it turned out that my OS also had an Apple compiled iperf, v3.8.1, but what I could download from their website was only 3.1.3. I also had to find a matching Windows version for it, but now that I knew what I was looking for this was easy. So now iperf works both directions, and I am getting 17Gbit both ways. Now I am guessing that this is due to card settings like MTU or whatnot.

Update: by using the -P switch I made iperf run multithreaded, and now I am getting 23.7Gbit.

It seems iperf was restricted by my 7980XE's single threaded performance, which is not a surprise as I can't even use turbo mode, the CPU can only hit 3.6GHz maximum instead of the official 4.4 (or 4.7 overclocked). This is due to my crappy temporary motherboard, it does not have enough power stages and it is not stable with the 18 core cpus. My replacement motherboard is already on the way, another Prime Deluxe II, most likely getting it tomorrow.

Sounds good. That's the correct throughput. As the original issue is resolved now, create a separate thread for the other issues you might have.

 

[EliteMacx86]

@lopess Seems like VT-d is not working which is must have requirement.

 

[lopess]

@lopess Seems like VT-d is not working which is must have requirement.

VT-d on, SMBIOS changed to MAcpro, and few other minor changes, but now I can't initialize XXV710 and Mellanox at all.

 

[lopess]

As a result of my investigation, I noticed that the problem occurs when installing Broadcom wifi cards (e.g. Fenvi T919). Installing this card requires locking the original IOskywalkfamily kext and running the patched Ventura kext. Support for these cards was probably added in Sonoma. The card tries to install the correct driver, but it is blocked. This means that we either have a working ethernet card using the "com.apple.DriverKit-AppleEthernetMLX5" or "com.apple.DriverKit-AppleEthernetIXL" drivers, or working Wifi from a Broadcom card with airplay, etc. Unfortunately, nowhere in the description of the method of bringing Broadcom wifi cards back to life in Sonoma is there a word about the limits of using this method in relation to other Ethernet cards, like Intel or Mellanox. We can still run wifi cards like Intel AX210/AX211 series, but without additional features like Airplay, handoof, and others.