ib_gpudirect is a framework that enables GPUs to communicate directly with the network, bypassing the host memory. Specifically, ib_gpudirect installs a direct data path between a GPU and an Infiniband network device, leaving the host memory out of the critical path and, thus, avoiding unnecessary copies to/from main memory. It currently supports CUDA and the Infiniband verbs programming interface. Preliminary evaluation of our prototype implementation shows that ib_gpudirect achieves significant performance improvement compared to the indirect path which includes main memory.
ib_gpudirect is open-source, available on github.