<< Chapter < Page | Chapter >> Page > |
This section presents an implementation of the four-step algorithm [link] that leverages hard-coded sub-transforms to compute larger transforms. The implementation uses an implicit memory transpose (along with vector register transposes) and scales particularly well with VL. In contrast to the fully hard-coded implementation in theprevious section, the four-step implementation requires no new leaf primitives as VL increases, i.e., the code is much the same when $VL>1$ as it is when $VL=1$ .
A transform of size $N$ is decomposed into a two-dimensional array of size ${n}_{1}\times {n}_{2}$ where $N={n}_{1}{n}_{2}$ . Selecting ${n}_{1}={n}_{2}=\sqrt{N}$ (or close) often obtains the best performance results [link] . When either of the factors is larger than the other, it is the larger of the two factors that will determine performance, because the larger factor effectively brings the memory wall closer. The four steps of the algorithm are:
For this out-of-place implementation, steps 2 and 3 are performed as part of step 1. Step 1 reads data from the input array and computes the FFTs, but before storing the data in the final pass, it is multiplied by the twiddle factors from step 2. After this, the data is stored to rows in the output array, and thus the transpose of step 3 is performed implicitly. Step 4 is then computed as usual: FFTs are computed along the columns of the output array.
This method of computing the four-step algorithm in two steps requires only minor modifications in order to support multiple vector lengths: with $VL>1$ , multiple columns are read and computed in parallel without modification of the code, but before storing multiple columns of data to rows, a register transpose is required.
When $VL=1$ , three hard-coded FFTs are elaborated.
In order to generate the code for the four-step sub-transforms, some minor modifications are made to the fully hard-coded code generator that was presented in the previous section.
The first FFT is used to handle the first column of step 1, where there are no twiddle factor multiplications because one of the array coordinates for step 2 is zero, and thus ${\omega}_{N}^{0}$ is unity. This FFT may be elaborated as in "Vector length 1" with the addition of a stride factor for the input address calculation. The second FFT is elaborated as per the first FFT, but with the addition of twiddle factor multiplications oneach register prior to the store operations. The third FFT is elaborated as per the first FFT, but with strided input and output addresses.
const SFFT_D __attribute__ ((aligned(32))) *LUT;
const SFFT_D *pLUT;void sfft_dcf64_fs_x1_0(sfft_plan_t *p, const void *vin, void *vout){
const SFFT_D *in = vin; SFFT_D *out = vout;
SFFT_R r0,r1,r2,r3,r4,r5,r6,r7; L_4(in+0,in+64,in+32,in+96,&r0,&r1,&r2,&r3);
L_2(in+16,in+80,in+112,in+48,&r4,&r5,&r6,&r7);
K_0(&r0,&r2,&r4,&r6);
S_4(r0,r2,r4,r6,out+0,out+4,out+8,out+12); K_N(VLIT2(0.7071,0.7071),VLIT2(0.7071,-0.7071),&r1,&r3,&r5,&r7);
S_4(r1,r3,r5,r7,out+2,out+6,out+10,out+14);}
void sfft_dcf64_fs_x1_n(sfft_plan_t *p, const void *vin, void *vout){ const SFFT_D *in = vin;
SFFT_D *out = vout; SFFT_R r0,r1,r2,r3,r4,r5,r6,r7;
L_4(in+0,in+64,in+32,in+96,&r0,&r1,&r2,&r3);
L_2(in+16,in+80,in+112,in+48,&r4,&r5,&r6,&r7);
K_0(&r0,&r2,&r4,&r6);
r2 = MUL(r2,LOAD(pLUT+4),LOAD(pLUT+6)); r4 = MUL(r4,LOAD(pLUT+12),LOAD(pLUT+14));
r6 = MUL(r6,LOAD(pLUT+20),LOAD(pLUT+22)); S_4(r0,r2,r4,r6,out+0,out+4,out+8,out+12);
K_N(VLIT2(0.7071,0.7071),VLIT2(0.7071,-0.7071),&r1,&r3,&r5,&r7);
r1 = MUL(r1,LOAD(pLUT+0),LOAD(pLUT+2)); r3 = MUL(r3,LOAD(pLUT+8),LOAD(pLUT+10)); r5 = MUL(r5,LOAD(pLUT+16),LOAD(pLUT+18));
r7 = MUL(r7,LOAD(pLUT+24),LOAD(pLUT+26)); S_4(r1,r3,r5,r7,out+2,out+6,out+10,out+14);
pLUT += 28;}
void sfft_dcf64_fs_x2(sfft_plan_t *p, const void *vin, void *vout){ const SFFT_D *in = vin;
SFFT_D *out = vout; SFFT_R r0,r1,r2,r3,r4,r5,r6,r7;
L_4(in+0,in+64,in+32,in+96,&r0,&r1,&r2,&r3);
L_2(in+16,in+80,in+112,in+48,&r4,&r5,&r6,&r7);
K_0(&r0,&r2,&r4,&r6);
S_4(r0,r2,r4,r6,out+0,out+32,out+64,out+96); K_N(VLIT2(0.7071,0.7071),VLIT2(0.7071,-0.7071),&r1,&r3,&r5,&r7);
S_4(r1,r3,r5,r7,out+16,out+48,out+80,out+112);}
void sfft_dcf64_fs(sfft_plan_t *p, const void *vin, void *vout) { const SFFT_D *in = vin;
SFFT_D *out = vout; pLUT = LUT;
int i; sfft_dcf64_fs_x1_0(p, in, out);
for(i=1;i<8;i++) sfft_dcf64_fs_x1_n(p, in+(i*2), out+(i*16));
for(i=0;i<8;i++) sfft_dcf64_fs_x2(p, out+(i*2), out+(i*2));
}
Hard-coded four-step VL-1 size-64 FFT
Notification Switch
Would you like to follow the 'Computing the fast fourier transform on simd microprocessors' conversation and receive update notifications?