OPNsense

Given the Linux bridge and QEMU KVM pass-through performance hit, I tried to install OPNsense (based off FreeBSD) and attempted to dual-boot.

Why OPNsense? That was the main reason I originally bought this C3758R unit in the first place – attempting to see if higher performance-to-power efficiency could be achieved with Intel QAT – pfSense’s QAT libraries are only available in their subscription-based pfSense+ product.

In short: a f*cking huge mess…

• install using custom (not “Auto”) partitioning
• the installer was UEFI-compatible so managed to find and write its bootloader into the FAT32 EFI partition – since the Ubuntu bootloader mounts the EFI System Partition (aka “ESP”) as /boot/efi, the full path in Ubuntu was /boot/efi/EFI/freebsd/loader.efi
• but GRUB wasn’t updated, so edit grub:
  • /etc/grub.d/40_custom:

menuentry "OPNsense" {
	insmod ufs2
	insmod zfs
	set root=(hd0,gpt4)
	chainload (hd0,gpt1)/EFI/freebsd/loader.efi
}

• • • changing “gptN” to whatever you see as the correct partition when doing an “ls -l” in grub, where the root is the FreeBSD UFS/ZFS partition, and the chainloader is pointing to the ESP
    • removing one of the two insmod lines where not required (i.e. FreeBSD root is UFS or ZFS formatted)
  • run update-grub
• install the MOK using the UEFI firmware (i.e. the /EFI/freebsd/loader.efi file) – otherwise secure boot would prevent the FreeBSD bootloader from working

That fixed, things went from bad to worse…

Attempting to test the 10GbE NIC on the Qotom C3758R failed spectacularly – no matter what I tried with the “raw” installation of OPNsense, I could not get iperf3 to work on the inbound (i.e. iperf3 -s and get a remote iperf3 client to connect). OPNsense’s own packet capture sees the iperf3 SYNC from the remote end, but there is just no response and everything stops. Outbound iperf3 still sees a max of ~7.5Gb with jumbo packets (i.e. MTU of 9000bytes)… Mind you that inbound traffic works – I am able to access and utilise the web UI.

I even noted that the original if_ix.ko drivers were not loaded (but I have not found out yet which driver was “in charge” of the X553s then), and tried to “force load” the drivers (by using the “System Tunables” and setting “if_ix_load” to “YES“)…

<RANT>

FreeBSD Uncontrollable Upgrades

Since recompiling and using the latest drivers worked for this unit on Linux, I attempted to do the same – but then FreeBSD rears its ugly head… The version OPNsense was built off was 14.3-RELEASE-p1, but I could not find a way to simply freebsd-update my separate, purpose-built FreeBSD VM to “p1“. The latest was “p2” and there was no other way I could easily find a way to do a partial upgrade or to fix this…

This resulted in my compiled if_ix.ko being “incompatible” with the kernel on OPNsense, throwing a “KLD if_ix.ko: depends on kernel - not available or version mismatch” error to my kldload /boot/kernel/if_ix.ko command attempting to manually load it.

Final Straw

Somehow, after playing with the drivers, iperf3 now fails to work either way (although the web UI still worked). Only after removing that “System Tunable” where the original if_ix.ko driver packaged with the OPNsense 25.7.1_1 distribution was “forced” to load, was I able to resume one-way (inbound) iperf3. As mentioned, I still have no way to see which driver in this FreeBSD OPNsense is managing the Intel X553s on this Qotom unit, and making an “educated guess” that it was the if_infiniband.ko driver (with “2” references according to kldstat) resulted in yet more FreeBSD bullsh*t, trying ways and means to disable the loading of the driver.

Note:

• hint.if_infiniband.N.disabled="1" in /boot/device.hints, where N was 4 entries covering 0 to 3, does not work
• if_inifiband_load="NO" from System Tunables does not work
• the only thing that worked was yanking the bloody file out (i.e. renaming or just deleting /boot/kernel/if_infiniband.ko)

But guess what? The interfaces still “work” (for the f*cked up version of the term “work”) – it seems the drivers have been “built in” into the kernel at compile time.

F*CK this. I have wasted more than enough time trying to get this to work, and I’m done with FreeBSD (although until I find a Linux-based firewall as “user friendly” as pfSense, I’m “stuck” with FreeBSD somehow).

Hell, I even found out that the “basic” ability to enforce authorisation key and password for SSH access was not available.

</RANT>

Verdict

Is the Qotom C3758R unit capable to run 10GbE interfaces? Yes, assuming you use OSes that have proper driver support.

Is it fanless? Sorta – you could pull the fan that is sucking all the dust up and into the unit from the bottom (talk about stupid design), but it already gets unbearably hot to touch with the fan – definitely still better to use with a fan, if not adding another USB-powered fan to somehow shoehorn in at the top.

Can it run pfSense/OPNsense? What about Linux in general? Hell no. Not with the current state of these “distros”, the OS underpinning it and particularly the state of the X553 OS drivers available for this unit. Alternatively, you could run pfSense/OPNsense through a Hyper-V VM on Windows, but I believe the CPU is already strained for multi-stream 10Gbps as it is.

In short: NO, just no.