Katana Graph Engine on Intel Optane DC PMM

Disruptive memory technologies are leading to significant improvements in both the capacity and the bandwidth of memories.

The white paper Katana Graph Engine on Intel Optane DC PMM explores Katana Graph’s ability to tackle large-scale data analytic problems faster while using less power.

The paper identifies graph analytics as an example of the kind of emerging business domains where Intel’s Optane^™ DC Persistent Memory Module (PMM) storage product line excels. Today's graph analytics systems involve almost incomprehensibly massive graphs. Facebook, a prominent example of a huge graph, includes a billion nodes and 200 billion edges. A graph of the indexable web now has roughly 100 billion nodes and trillions of edges.

Two methods are commonly employed to process such a large amount of data. One solution uses distributed-memory machines (clusters) that have sufficient main memory for in-memory processing of the graphs. Another uses secondary storage to house the graphs while using out-of-core algorithms to read a portion of a graph into Dynamic RAM (DRAM) at a time under software control and to process it there.

The Intel Optane DC presents a new option, with persistent memory providing an exciting alternative, since shared-memory graph analytics can be used creatively to process extensive graphs.

Optane DC PMM has two modes of operation. In Memory Mode, Optane PMM is treated as main memory and DRAM functions as a direct-mapped cache termed near-memory. Memory Mode’s DRAM-like performance allows existing code to run without modification, but at a dramatically lower cost. This has particularly compelling potential when analyzing huge graphs. In App Direct Mode, the Optane PPM modules are treated as byte-addressable persistent memory. This allows indices to be stored in persistent memory rather than rebuilding them on reboot, which means a significant reduction in restart time.

Katana Graph identified three key system-level parameters to play a vital role in efficiently using the Optane PMM memory hierarchy and getting superior performance for graph analytics workloads:

• NUMA-aware allocation
• NUMA migrations
• Page size selection

Graph algorithms were analyzed to determine which would work best on machines using the Optane PMM.

Since a given algorithm can usually be implemented in different ways, Katana Graph examined how these differences can have a major effect on parallel performance. For example, implementations with fine-grain locking generally perform better than those with coarse-grain locking. On machines with Optane PMM, Katana recommends using algorithms with non-vertex operators and asynchronous algorithms. Katana Graph's core graph library provides graph analytics applications optimized for Optane PMM machines.

The paper concludes with a performance evaluation of the Katana Graph Engine on a single Xeon machine equipped with Optane DC PMM on graphs that are several terabytes in size. The demonstrated performance advantages stem from the generality of Katana’s non-vertex programming model, support for asynchronous execution, NUMA-aware allocation policies, and provisions for huge memory page sizes along with scale-out provisions for even larger graphs. With the advantages provided by Optane PMM, Katana Graph’s analytics applications have greater scalability, speed, and practical utility.

Please click here to download the full paper.

Interested in what Katana Graph can do for your business? Click to schedule a time to talk.