DFSSSP - A Deadlock-Free Fast Routing Algorithm for InfiniBand/OpenSM

Omnia vincit amor

Home -> Research -> DFSSSP

  Publications
  Awards
  Research
    NB Collectives
    MPI Topologies
    MPI Datatypes
    Netgauge
    Network Topologies
    Ethernet BTL eth
    ORCS
    DFSSSP
    Older Projects
    cDAG
    LogGOPSim
    CoMPIler
  Teaching

Full CV [pdf]
BLOG

Events

Past Events

DFSSSP - A Deadlock-Free Fast Routing Algorithm for InfiniBand/OpenSM

DFSSSP Description: The DFSSSP [1] algorithm is a deadlock-free version of the SSSP algorithm [2], which enables balanced routing along shortest paths. SSSP delivers a significantly higher effective bisection bandwidth [3] than other routing algorithms implemented in OpenSM [1,2].

Download patched OpenSM: opensm-dfsssp.tar.gz - (1302.09 kb) original version (for reproducing results in [1]) opensm-dfsssp-mellanox-patch.tgz - (1311.05 kb) including Mellanox for case ID #47452 See README file in package!

Installing the patched OpenSM tar xzf management.tar.gz cd management export CONFIG_OPTS="--prefix=${HOME}/ofed" export CONFIG_OPTS="${CONFIG_OPTS} LDFLAGS=-L${HOME}/ofed/lib" export CONFIG_OPTS="${CONFIG_OPTS} CPPFLAGS=-I${HOME}/ofed/include" cd libibumad ./autogen.sh && ./configure ${CONFIG_OPTS} make make install cd .. cd libibmad ./autogen.sh && ./configure ${CONFIG_OPTS} make make install cd .. cd opensm ./autogen.sh && ./configure ${CONFIG_OPTS} make make install
Running the patched OpenSM with SSSP (needs ROOT rights): cd $HOME/ofed/sbin ./opensm -R sssp ↪ for additional parameters look at ./opensm --help Run deadlock-free SSSP routing algorithm (needs ROOT rights): cd $HOME/ofed/sbin ./opensm -R dfsssp Run DFSSSP with modified QoS file (to configure SL/VL load): cd $HOME/ofed/sbin ./opensm -R dfsssp --qos_policy_file ${HOME}/ofed/qos.conf Possible content of qos.conf file: qos_max_vls 8 qos_high_limit 0 qos_ca_vlarb_high 0:64,1:64,2:64,3:64,4:64,5:64,6:64,7:64 qos_ca_vlarb_low 0:4,1:4,2:4,3:4,4:4,5:4,6:4,7:4 qos_ca_sl2vl 0,1,2,3,4,5,6,7,0,1,2,3,4,5,6,7 qos_swe_vlarb_high 0:64,1:64,2:64,3:64,4:64,5:64,6:64,7:64 qos_swe_vlarb_low 0:4,1:4,2:4,3:4,4:4,5:4,6:4,7:4 qos_swe_sl2vl 0,1,2,3,4,5,6,7,0,1,2,3,4,5,6,7 ↪ for explanation see management/opensm/doc/QoS_management_in_OpenSM.txt

DFSSSP was developed by Jens Domke at the ZIH at the Technische Universität Dresden and the scientific work was advised by Torsten Hoefler.

References

IPDPS'11	[1] Jens Domke, Torsten Hoefler and W. Nagel:
		Deadlock-Free Oblivious Routing for Arbitrary Topologies In Proceedings of the 25th IEEE International Parallel \& Distributed Processing Symposium (IPDPS), presented in Anchorage, AL, USA, pages 613--624, IEEE Computer Society, ISBN: 0-7695-4385-7, May 2011, (acceptance rate: 19.6%, 112/571)

HotI'09	[2] Torsten Hoefler, Timo Schneider and Andrew Lumsdaine:
		Optimized Routing for Large-Scale InfiniBand Networks In 17th Annual IEEE Symposium on High Performance Interconnects (HOTI 2009), presented in New York, NY, Aug. 2009,

Cluster'08	[3] Torsten Hoefler, Timo Schneider and Andrew Lumsdaine:
		Multistage Switches are not Crossbars: Effects of Static Routing in High-Performance Networks In Proceedings of the 2008 IEEE International Conference on Cluster Computing, presented in Tsukuba, Japan, IEEE Computer Society, ISSN: 1552-5244, ISBN: 978-1-4244-2640, Oct. 2008, (acceptance rate 30%, 28/92)


serving: 18.225.195.4:24043	© Torsten Hoefler