VPS provider's kernel update broke our custom module, how to recover without a full rebuild?

Question

Our VPS provider pushed an automatic kernel update last night, and now our custom-built kernel module for a specific piece of hardware won't load. The server boots, but the service depending on that module is dead. The error is just 'module verification failed' and 'invalid module format'. We've already tried to recompile the module against the new kernel headers using our existing source, but that's failing with a bunch of version mismatch errors. Is our only option to roll back to a snapshot from before the update, or is there a way to get the module building against this new kernel? Downtime is starting to become a problem.

admin · Accepted Answer

You can recover by compiling your module against the exact kernel version running on your VPS, which requires matching kernel headers and build tools. The 'invalid module format' error specifically means the module's vermagic string or symbol versions don't match the running kernel, often because your build environment uses different kernel headers than the VPS's actual kernel binary. First, verify the exact kernel version with `uname -r`, then install the corresponding kernel headers package from your provider's repository, not a generic distribution package. The critical step is ensuring the kernel source configuration (`/boot/config-*`) matches what was used to build the running kernel; providers sometimes apply custom patches that change ABI. Use these commands to attempt a clean rebuild:

KVER=$(uname -r)
sudo apt-get install linux-headers-${KVER} linux-source-${KVER}
cd /usr/src/linux-source-${KVER}
sudo tar -xf linux-source-${KVER}.tar.bz2
cd linux-source-${KVER}
sudo cp /boot/config-${KVER} .config
sudo make modules_prepare
cd /path/to/your/module/source
make -C /lib/modules/${KVER}/build M=$(pwd) modules

If this fails due to provider customizations, your fastest recovery is indeed a snapshot rollback, then immediately setting the kernel to 'manual' updates in your package manager to prevent recurrence.

VPS provider's kernel update broke our custom module, how to recover without a full rebuild?

Who Is Online