When several Ethernet switches, operating as layer 2 devices in in terms of the OSI 7-layer network reference model, are interconnected by inter-switch links ( ISLs), the IEEE 802.1D Rapid Spanning Tree (RSTP) protocol or a similar one is used to workout the best paths through the network and avoid things like endless loops. Spanning Tree allows only one link to be active between two Ethernet nodes, so this is achieved using the concept of active and inactive ports, with the former needed to construct a path through the network. Active bridging ports are put into a forwarding state, and un-needed ones into a blocked state.
A blocked link cannot be used for data transport and such ports are present but inactive, thereby literally wasting network bandwidth. This limits the overall bandwidth of a network and means reconfiguration when a port or switch fails takes time. It also limits the size of layer 2 Ethernet domains, and data centres will often have several of then delimited by layer 3 boundaries. We can't be having that; spanning tree approaches are inadequate.
What's needed is a routing protocol in layer 2, then we could get rid of spanning tree limitations, but routing is a layer 3 activity. The developing IEEE Transparent Interconnection of Lots of Links (TRILL) standard should solve this. It also enables multiple access paths at layer 2, which will increase available network bandwidth.
It uses the concept of a routing bridge (pdf) (RBridge) running an IS-IS (Intermediate System - Intermediate System) routing protocol, a link state routing protocol which operates by sending link state information through a network of routers or switches. Each router can then construct a representation of the network’s topology.
DCE plus TRILL
A DCE network, supporting DCB, built with switches supporting TRILL will mean that FCoE traffic can traverse the network, making multiple hops, without losing frames due to them being dropped as a result of congestion, or without taking too long a time, thus breaking the FC link, because the FCoE frames have a high enough priority be unaffected by the congestion. Such a network will also be able to have more than one link between switches, and these links can be simultaneously active, thus increasing effective bandwidth and availability.
A DCE network lacking TRILL can have multi-hop FCoE but it won't be routable and will be limited in scale and reconfigurability.
If the network needs more capacity then more switches and links are added. The IEEE is expected to, or hoped to, finalise DCE and TRILL next year. For now both Brocade and Cisco, the leading FCoE promoters, have announced products that will have a superset of TRILL functionality and, hopefully, support TRILL itself when it is standardised. These are Brocade One and FabricPath.
Combining Ethernet and Fibre Channel intricasies
Storage people considering FCoE will need to be really very well-educated about the intricacies of Ethernet indeed - the text above is simply the most basic overview imaginable - and how it will need to change to properly and effectively support FCoE. You will need to understand how different suppliers' products intercept the emerging standards.
It won't be practicable to have separate storage networking and Ethernet disciplines in your shop; the two have to converge; if they don't then your FCoE implementation will be done by Ethernet people that don't understand Fibre Channel or Fibre Channel people ignorant of Ethernet. You might as well draft your resignation letter now. ®