If we're upgrading our SAN that has eight switches, an EMC DMX/HP EVA array and multiple operating systems (OS), is it better to run a different OS version in each fabric to let the new version shake out any bugs? Or is it just better to upgrade both in quick order? The switches are Cisco MDS switches, where the upgrade and downgrade are usually not noticed by the hosts.
Assuming that you have separate fabrics, thus redundant access for high availability and performance, I would assume that by design you want to be able to isolate from path failure including adapter, switch, storage interface port and software. Thus, it would make sense that you would upgrade one fabric, make sure it is stable before upgrading the other fabric while being able to fall off the change and revert back to the previous OS. Having said that, there are different schools of thought that say to keep everything the same and if the new OS is good enough to go on one of the paths, it should be good enough for both. Otherwise, why take the risk?
From the description of your environment, it sounds like anything you can do to minimize or reduce risk and complexity will be a plus. Even if the hosts usually do not notice the upgrades, there is that chance as well as the interdependencies of the different HBAs and interaction with the switches and storage devices that need to be factored in. Thus, if you go the route of upgrading one path and leaving the other at the older version, try and get the other path or fabric upgraded to the newer version of the OS as soon as possible. If nothing else, from a support and management standpoint, not to mention late night or early morning troubleshooting, you will be better off getting both fabrics updated as soon as possible.