Dear René,

thank you for the prompt replay. Indeed the problem is quite simple. So wrap it up, the number of cell in each direction should give N % (N_gpu * N_supercells) == 0.

Just for educational purpose: what is so special about the number 128 (I guess it is a size of a cache)? Could in be for instance 256? Would it be possible to specify the same number of super cells in X and Z direction?

Thanks a lot,

Danila.

---

From: picongpu-users@hzdr.de [picongpu-users@hzdr.de] on behalf of René Widera [r.widera@hzdr.de]
Sent: Wednesday, October 19, 2016 4:59 PM
To: picongpu-users@hzdr.de
Subject: Re: [PIConGPU-Users] [PIConGPU-Users] Supercell concept

Dear Danila,

the volume per gpu needs to be a multiple of the superCell size.

In your case 4176/8gpus=522. 522 is not dividable bei 8.
4160 cells in y direction should solve your problem.

Please keep in mind if you change the superCell size to a value smaller than 128cells the most simulation run slower.
The default size with 8x8x4 shows the best result for the most cases.

best,

René

Am 19. Oktober 2016 16:46:28 MESZ, schrieb Khikhlukha Danila <Danila.Khikhlukha@eli-beams.eu>:

Dear all,

I have some troubles trying to specify a computational grid with moving windows using picongpu v0.2.0. We were discussing this topic previously, however I have the same problem again, so likely I misunderstand something from the last time.

So I'm trying to launch the simulation using 4 K80 cards: 8 GPU devices overall. In the memory.param file I have specified the SuperCell layout as (2,8,2). I want to have one GPU in transversal direction and 8 in longitudinal. So in cfg file I specified:

TBG_gpu_x=1

TBG_gpu_y=8

TBG_gpu_z=1

Then I would like my real computational domain to have 256 x 3712 x 256 cell. Since the moving window reduces the real domain by 1 GPU in y direction, I specified my grid as 256 x 4176 x 256. (4176 = 9/8*3712)

TBG_gridSize="-g 256 4176 256"

However, trying to submit such a cfg file I'm receiving an assertion fail:

void picongpu::MySimulation::checkGridConfiguration(PMacc::DataSpace<DIM>,PMacc::GridLayout<DIM>) [with unsigned int DIM = 3u]: Assertion`gridSizeLocal[i] % MappingDesc::SuperCellSize::toRT()[i] == 0' failed.

However 4176 % 8 == 0 and 256 % 2 == 0. 

Could you please guide me how to solve this issue? Looks like I misunderstand the concept of the SuperCell.

Thank you in advance,

Danila.