Blocks

Thread blocks, as we can see in the preceding diagram, are a collection of threads. The number of threads in each block is limited to the architecture of the GPU. For example, in an NVIDIA GeForce Titan X GPU belonging to the Maxwell architecture, the maximum number of threads in a block is 1,024. But in spite of this limitation, a GPU program can definitely make use of a huge number of blocks instead. This allows the program to maximize thread parallelization, especially on high-performance GPUs.