To evaluate our results, we perform 2 sets of experiments.
These are builds where the input and output vector to perform NTT on is laid out linearly in HBM (i.e., the host doesn’t perform any pre/post-processing). We run experiments with the 8-core, 16-core, 32-core and 64-core variants, yielding different levels of parallelism.
As discussed here, our performance is significantly bound by bandwidth. We conduct 2 builds (32-core and 64-core variant) with a simple data-rearrangement preprocessing step such that the core can stream data in 2048-4096 byte bursts.
We have tested our design and ran builds on a 6-core Intel(R) Core(TM) i5-9600K CPU @ 3.70GHz machine with [Ubuntu 22.04 LTS (GNU/Linux 5.15.0-48-generic x86_64)]. We did not use any special kernel flags / boot parameters to obtain our results. We ran our designs using the Vitis platform Xilinx has provided for the Varium C1100 card. The platform takes up some resources on the FPGA and comes with PCIe gen3 x4 support.
We measured our latency by taking the FPGA-only evaluation latency across 200
NTT runs. Power was measured by sampling
xbutil examine --report electrical
--device <device-pcie-id> 10 times during the latency benchmark.
In this normal layout build, we do not perform any preprocessing or post-processing. Hence, latency below includes only the FPGA NTT evaluation latency.
Latency, Power and Resource Utilisation
The table below depicts our results for various builds:
Here are the available resources on the FPGA. Note that as we are building on top of a Vitis platform, it imposes a non-trivial fixed-cost that we don’t control. The number is reported as “fixed” in the post_route_utilisation.rpt.
|Resource||Available on FPGA||Used by Vitis Platform|
Result from Optimised-Layout Builds
Here is a detailed breakdown of a runtime sample of an optimised 64-core build (the power and utilisation is similar to the normal-layout builds):
Breakdown of a 2^24 optimised-layout 64-core evaluation:
|Preprocessing data rearrangement||0.0213s|
|Copying input points to device||0.0414s|
|Doing Actual NTT work||0.0267s (vs 0.0450s in normal layout)|
|Copying final result to host||0.0552s|
|Copy from internal page-aligned buffer||0.0231s|
Breakdown of a 2^24 optimised-layout 32-core evaluation:
|Preprocessing data rearrangement||0.0217s|
|Copying input points to device||0.0416s|
|Doing Actual NTT work||0.0349s (vs 0.0691s in normal layout)|
|Copying final result to host||0.0554s|
|Copy from internal page-aligned buffer||0.0228s|
By rearranging the data in a much more memory-friendly layout, our NTT evaluation time drops significantly compared to those of a 64-core build in a normal build (0.0267s vs 0.0450s). This comes at the cost of the host doing some data rearrangement.
The bottleneck of our evaluation clearly lies in the host and PCIe latency in this result, both of which can be solved pretty easily:
preprocessing + postprocessing > latency- We can run the preprocessing and post-processing in separate threads. We can set up the input and output buffers such that we don’t run into cache coherency issues. We can also mask some of the preprocessing latency with PCIe latency.
The PCIe latency is larger than the NTT evaluation- This is because the Vitis platform we are using only supports PCIe x4. With PCIe x16, we would have 4 times the bandwidth and could sidestep this problem.
In practice, we believe this is the more scalable design that can achieve low latency and high throughput at the cost of the host machine doing some data rearrangement.