Progress!

I've successfully implemented child device creation! Whenever the bus driver receives a IOCTL_SWIVL_CREATE_VM ioctl, it creates a new VM object and registers a new VM device with the PnP (plug-and-play) manager, which is then able to load the function driver. This is a pretty big accomplishment because it means a good portion of the bus driver is already complete - all that's missing is the IOCTL_SWIVL_DELETE_VM ioctl, plus maybe a few other. Most of the actual code will be in the VM instance driver - starting the VM, setting up guest registers/memory, etc.

Now, there's still one bug that's been quite troublesome since the beginning - every once in a while, when uninstalling the hypervisor driver, it crashes with STATUS_ILLEGAL_INSTRUCTION when trying to execute the vmxoff instruction. From what I could discern from the documentation, the main cause for this is if VMX is not enabled when the instruction is executed. I've made several little patches/updates to my code (as well as the 'VMX Basics' post), hoping to fix it; alas, I was only treating the symptoms, not the problem.

Basically, as part of the driver startup, it needs to initialize VMX on every active CPU. As far as I'm aware, the only way to do this is to create a system thread for every CPU, and set the affinity for each thread to only run on its corresponding CPU. Based on KeSetSystemGroupAffinityThread on MSDN, the thread should be moved to the correct CPU by the time that function returns (because the thread's IRQL is at the lowest possible level, which is below APC level). Sometimes, however, the thread remains on the CPU it started on. One time, I even saw it move to the correct CPU and immediately jump back to the original CPU before initializing VMX. Therefore, I believe the root cause of the issue is perhaps the thread's IRQL isn't high enough, some sort of kernel interrupt is preempting it and restoring the affinity mask (and thus possibly moving it back to the original CPU). As a result, it'll try to clean up VMX on a CPU that isn't running VMX anymore, thus resulting in the crash.

I want to test some more on the whole thread affinity issue, and I definitely need to do more testing with the child device creation; however, it's currently about 4:50 AM here, and I should probably get some sleep.

Maybe tomorrow I can figure out how to attach a few pictures showing the registered devices and logs.