I am particularly interested when Occam's razor is applied within the field of computer science and engineering [...]
An software engineer's perspective:
Occam's razor is bit more subtle than 'simplest is always the most correct'.
First: 'simplest' is very subjective. Facing a server crash, I could theorize: a bad piece of code, solar flare, a dead squirrel in the rack, the CPU falling of, or just an act of God. At some level, the first reason is the least simple. However Occam's razor doesn't call for 'simplest', but for the fewest entities. From that perspective, it recommends looking at an explanation involving just the code and the hardware it runs on before introducing a squirrel (or God) in the system.
That said, Occam's razor also says that entities shouldn't be multiplied without necessity. And if after looking at the code and the hardware configuration, I can't find a good explanation for the crash, introducing the squirrel might become necessary (yes, it did happen).
Similarly, it might be necessary to introduce relativity to Newton's gravity when it proves insufficient.
There are some pitfalls: it's easy to get enamored with the most mathematically elegant explanation, or dismiss inconvenient new data that goes against an established (or favorite) theory. But those are issues unrelated to Occam's - a scientist unwilling to consider relativity or a conspiracy theorist unwilling to not consider aliens are equally guilty.
As long as Occam's razor isn't used to justify intellectual dishonesty, it's a pretty useful guiding principle.