Basically, I want to fwrite() n bytes and have < n bytes be written on the actual disk, or whatever other inconsistency artifacts could arise (like sectors being written out-of-order).
I know I could do this at a hardware level either with:
- bare metal by yanking the power cord OR
- with virtualized OS by just issuing a reset via the hypervisor.
.. but I don't like the above cases because:
- I don't want to damage my hardware with actual power loss.
- Actual power loss simulation is manual and/or difficult & ugly to automate.
- Virtualization power loss simulation is better but I guess it would be pretty slow to do 100 runs against the database by waiting for VM boots and ugly having to build logic to re-try logging in via SSH until the VM is booted.
.. so I'm looking for a faster and easier solution to build automation for.
I've had a few ideas:
1) Kill the process with $ kill -9 dbms_pid
This won't probably work, as I guess everything given to fwrite() is added atomically to buffers managed by the kernel (this is speculation), and after process killing the kernel probably just flushes the buffers normally to disk?
2) Unmount the filesystem mid-write
I don't think unmounting works while there are open files on the filesystem.
3) Have the filesystem reside on a loopback device backed by a file and just stop writes to that file at the power cutoff point
I don't think there's a mechanism for that. Renaming the file probably does not stop the writes to the filesystem as the loopback driver probably just refers to an inode or some internal reference.
Unmounting the loopback device probably has the same issue of not being able to do when someone has open files on it.
Copying from the block-device-as-a-file could work (unless it's write-exclusive locked which it probably is) but because copying the contents is not an atomic operation it would probably bring artifacts not related to powering off.
4) Have the filesystem hosted on top of LVM and take a snapshot
This is probably my most workable idea. Taking a snapshot of the LVM volume is atomic and could pretty much simulate power loss? I guess unflushed buffers/pages are not taken into account for the snapshot (which is good for my simulation) and I hope would introduce the same artifacts as a power loss, like fwrite() contents being only half written?
Would there be out-of-order writes or do pages written to LVM volumes always get applied in order?
Do you have any other ideas? What do you think of the options? You agree that 4) is the best idea?
Why am I doing this: basically, I'm developing a database whose durability I would like to test by building an automated script that would take it through a huge round of torture testing by giving it data to save, while simulating a power loss against the disk at least, and then verifying that no committed data was lost and that the database can recover safely.
