You're adding a layer of abstraction between physical storage and the infrastructure up above, including switches...
and servers. The fundamental principles of computer science suggest that the added work will take CPU cycles away and draw upon other computing resources that would otherwise be devoted to storage and applications -- undoubtedly adding latency -- and will result in a performance impact.
However, depending on the type of storage virtualization product being deployed, there can actually be performance increases. Consider a simple environment consisting of JBODs. Since IBM SVC is a caching product, it can actually add RAID functionality and increase performance through the storage virtualization product. Overall, it is generally safe to assume that there will be some performance impact, but question the vendor about performance and evaluate the net effect on your environment.
This is why it's so important for large, scalable enterprises to seriously look at intelligent switch-based or purpose-built appliance-based storage virtualization because the switch-based approach mitigates much of the performance impact found in other deployments.
Check out the entire Storage Virtualization FAQ guide.