Return-Path: <a.huebl@hzdr.de>
Received: from [80.187.103.132] (account huebl@hzdr.de HELO [10.18.186.74])
  by hzdr.de (CommuniGate Pro SMTP 6.1.4)
  with ESMTPSA id 11815462 for picongpu-users@hzdr.de; Sat, 08 Aug 2015 13:17:05 +0200
User-Agent: K-9 Mail for Android
In-Reply-To: <list-10865507@cg1.fz-rossendorf.de>
References: <list-10865507@cg1.fz-rossendorf.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain;
 charset=UTF-8
Subject: picongpu in single machine and single gpu
From: Axel Huebl <a.huebl@hzdr.de>
Date: Sat, 08 Aug 2015 13:14:44 +0200
To: picongpu-users@hzdr.de
Message-ID: <2D4071EE-791E-43E0-9798-360100094AB3@hzdr.de>

Hi,

Can you please attach your cfg file and output? I think your example's memory consumption is too high, causing a hang.

If you want to speed up the init, use quiet start (lattice-like in cell positioning of particles) and zero initial temperature.

Two cards, either in the same node or via network/mpi connected will give you nearly a 2x speedup. SLI is not required, cuda can't use and doesn't need it (it's more a frame buffer/gfx/gaming interconnect). Just plug the second card in and use it with -d ... :)

Also, make sure your host=CPU RAM is at least as much as the sum of the RAM of the GPUs in the node (better a bit more).

Best,
Axel

On August 8, 2015 12:37:35 PM CEST, "k.manzoorolajdad" <kazem.manzoor@gmail.com> wrote:
>thanks a lot Mr.Huebl
>the example lwfa is running in my machine with single core with single
>gpu
>since 45 hour ago without output. is it normal?
>how long this code run?
>
>if i use two gpu 670 with SLI can i speed up? how much?
>thanks
>manzoor
>
>
>On Thu, Aug 6, 2015 at 7:34 PM, k.manzoorolajdad
><kazem.manzoor@gmail.com>
>wrote:
>
>> thanks
>> i can run first example :
>>
>> ~/paramSets/lwfa$ tbg -s bash -c submit/0001gpus.cfg -t
>> submit/bash/bash_mpirun.tpl ~/runs/lwfa
>> Running program...
>> tbg/submit.start: line 37: /home/manzoor/picongpu.profile: No such
>file or
>> directory
>>  Data for JOB [64751,1] offset 0
>>
>>  ========================   JOB MAP   ========================
>>
>>  Data for node: manzoor-gpu    Num slots: 4    Max slots: 0    Num
>procs: 1
>>      Process OMPI jobid: [64751,1] App: 0 Process rank: 0
>>
>>  =============================================================
>>  Data for JOB [64762,1] offset 0
>>
>>  ========================   JOB MAP   ========================
>>
>>  Data for node: manzoor-gpu    Num slots: 4    Max slots: 0    Num
>procs: 1
>>      Process OMPI jobid: [64762,1] App: 0 Process rank: 0
>>
>>  =============================================================
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | Sliding Window is OFF
>> [1,0]<stdout>:mem for particles=1078 MiB = 142315 Frames = 36432640
>> Particles
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | max weighting 6955.06
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | Courant c*dt <= 1.74147 ?
>1
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | omega_pe * dt <= 0.1 ?
>0.0142719
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | y-cells per wavelength:
>18.0587
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | macro particles per gpu:
>8388608
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | typical macro particle
>> weighting: 6955.06
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_SPEED 2.99792e+08
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_TIME 8e-17
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_LENGTH 2.39834e-08
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_MASS 6.33563e-27
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_CHARGE 1.11432e-15
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_EFIELD 2.13064e+13
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_BFIELD 71070.4
>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_ENERGY 5.69418e-10
>>
>> i think my run in single core with single gpu.
>>
>>
>> *can i run multi cpu with single gpu?*
>> thanks a lot
>> manzoor
>>
>> On Thu, Aug 6, 2015 at 4:03 PM, k.manzoorolajdad
><kazem.manzoor@gmail.com>
>> wrote:
>>
>>> thanks Mr.Huebl
>>>
>>> i use ubuntu 14.04
>>> cuda 6.0
>>> and install requirements:
>>>
>>> sudo apt-get install build-essential cmake file cmake-curses-gui
>>> libopenmpi-dev zlib1g-dev libboost-program-options-dev
>libboost-regex-dev
>>> libboost-filesystem-dev libboost-system-dev git
>>>
>>> git clone
>https://github.com/ComputationalRadiationPhysics/picongpu.git
>>> $HOME/src/picongpu
>>>
>>>
>>> export PICSRC=$HOME/src/picongpu
>>>
>>> *pngwriter* >= 0.5.5
>>>
>>> mkdir -p ~/src ~/build ~/lib
>>> git clone https://github.com/pngwriter/pngwriter.git
>~/src/pngwriter/
>>>
>>> cd ~/build
>>> cmake -DCMAKE_INSTALL_PREFIX=~/lib/pngwriter ~/src/pngwriter
>>> manzoor@manzoor-gpu:~/build$ make install
>>> Scanning dependencies of target pngwriter_static
>>> [ 16%] Building CXX object
>>> CMakeFiles/pngwriter_static.dir/src/pngwriter.cc.o
>>> Linking CXX static library libpngwriter.a
>>> [ 16%] Built target pngwriter_static
>>> Scanning dependencies of target blackwhite
>>> [ 33%] Building CXX object
>CMakeFiles/blackwhite.dir/tests/blackwhite.cc.o
>>> Linking CXX executable blackwhite
>>> [ 33%] Built target blackwhite
>>> Scanning dependencies of target diamond
>>> [ 50%] Building CXX object CMakeFiles/diamond.dir/tests/diamond.cc.o
>>> Linking CXX executable diamond
>>> [ 50%] Built target diamond
>>> Scanning dependencies of target lyapunov
>>> [ 66%] Building CXX object
>CMakeFiles/lyapunov.dir/examples/lyapunov.cc.o
>>> Linking CXX executable lyapunov
>>> [ 66%] Built target lyapunov
>>> Scanning dependencies of target pngtest
>>> [ 83%] Building CXX object
>CMakeFiles/pngtest.dir/examples/pngtest.cc.o
>>> Linking CXX executable pngtest
>>> [ 83%] Built target pngtest
>>> Scanning dependencies of target pngwriter
>>> [100%] Building CXX object
>CMakeFiles/pngwriter.dir/src/pngwriter.cc.o
>>> Linking CXX shared library libpngwriter.so
>>> [100%] Built target pngwriter
>>> Install the project...
>>> -- Install configuration: ""
>>> -- Installing: /home/manzoor/lib/pngwriter/lib/libpngwriter.so
>>> -- Installing: /home/manzoor/lib/pngwriter/lib/libpngwriter.a
>>> -- Installing: /home/manzoor/lib/pngwriter/include/pngwriter.h
>>>
>>> export CUDA_ROOT=/usr/local/cuda-6.0/
>>> export MPI_ROOT=/usr/local/
>>> export PATH=$PATH:$HOME/src/picongpu/src/tools/bin
>>> export PNGWRITER_ROOT=$HOME/lib/pngwriter
>>>
>>> mkdir -p ~/src ~/build ~/paramSets ~/runs
>>>
>>> ~/src/picongpu/createParameterSet
>~/src/picongpu/examples/LaserWakefield/
>>> paramSets/lwfa/
>>>
>>> cd build/
>>>
>>> manzoor@manzoor-gpu:~/build$ ~/src/picongpu/configure -a sm_30
>>> ../paramSets/lwfa
>>> cmake command: cmake -DCUDA_ARCH=sm_20
>>> -DCMAKE_INSTALL_PREFIX=../paramSets/lwfa
>>> -DPIC_EXTENSION_PATH=../paramSets/lwfa  -DCUDA_ARCH=sm_30
>>> /home/manzoor/src/picongpu
>>> *CMake Error: The source "/home/manzoor/src/picongpu/CMakeLists.txt"
>does
>>> not match the source "/home/manzoor/src/pngwriter/CMakeLists.txt"
>used to
>>> generate cache.  Re-run cmake with a different source directory.*
>>>
>>>
>>> On Thu, Aug 6, 2015 at 2:07 PM, Huebl, Axel <a.huebl@hzdr.de> wrote:
>>>
>>>> Dear Manzoor,
>>>>
>>>> welcome to our user list!
>>>>
>>>> The 670 GTX is a Kepler generation card with sm_30 so you are good
>to go
>>>> from the hardware side (we support sm_20 "Fermi" and upward):
>>>>   https://developer.nvidia.com/cuda-gpus
>>>>
>>>> We would recommend you to install a linux operation system, the
>latest
>>>> CUDA Toolkit
>>>>   https://developer.nvidia.com/cuda-downloads
>>>>
>>>> and the additional required tools and libraries documented here:
>>>>
>>>>
>>>>
>https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/INSTALL.md#requirements
>>>>
>>>> They are all pretty standard and most get shipped with packages
>too,
>>>> e.g., in Debian, Ubuntu and Arch. Please read the instructions we
>>>> provided in this file carefully.
>>>>
>>>> I also recommend you installing pngwriter as described under
>"optional"
>>>> since it allows an easy check of the output with our png (preview)
>>>> plugin.
>>>>
>>>> If you installed the requirements, just scroll a bit down in the
>>>> INSTALL.md guide and set up a simulation case. This is additionally
>>>> documented in a youtube video
>>>>   https://www.youtube.com/watch?v=7ybsD8G4Rsk
>>>>
>>>> With the binary compiled, you can set plugins in the case's *.cfg
>file
>>>> when you are at this point. All available options are documented
>here
>>>>
>>>>
>>>>
>https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/TBG_macros.cfg
>>>>
>>>> and in the wiki
>>>>
>>>>
>>>>
>https://github.com/ComputationalRadiationPhysics/picongpu/wiki/PIConGPU-Plugins
>>>>
>>>> The tbg template you want to use is "bash_mpirun.tpl" and set to "1
>1 1"
>>>> GPUs in your ".cfg" file.
>>>>
>>>> For further resources, please continue to read (links below):
>>>>
>>>>  [0] README.md
>>>>  [1] doc/INSTALL.md
>>>>  [2] our wiki
>>>>  [3] doc/TBG_macros.cfg
>>>>  [4] closed questions in our issue tracker
>>>>
>>>>
>>>> If there should pop up problems along the way, feel free to ask
>again on
>>>> the list!
>>>>
>>>>
>>>> Best regards,
>>>> Axel Huebl
>>>>
>>>> [0]
>>>>
>>>>
>https://github.com/ComputationalRadiationPhysics/picongpu#picongpu---a-many-gpgpu-pic-code
>>>> [1]
>>>>
>>>>
>https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/INSTALL.md
>>>> [2] https://github.com/ComputationalRadiationPhysics/picongpu/wiki
>>>> [3]
>>>>
>>>>
>https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/TBG_macros.cfg
>>>> [4]
>>>>
>>>>
>https://github.com/ComputationalRadiationPhysics/picongpu/issues?q=is%3Aissue+label%3Aquestion+is%3Aclosed
>>>>
>>>> On 06.08.2015 11:21, k.manzoorolajdad wrote:
>>>> > Hi
>>>> > I am new in CUDA andComputational Radiation Physics and want use
>>>> > picongpu but don't have gpu cluster.
>>>> > I have single GPU (Geforce 670 gtx) and want test and run code.
>>>> >
>>>> > How can run code in single machine and single gpu?
>>>> >
>>>> > Thanks
>>>> >
>>>> > Manzoor
>>>> > Ms.student of physic
>>>> > Tehran university,Iran
>>>>
>>>> --
>>>>
>>>> Axel Huebl
>>>> Phone +49 351 260 3582
>>>> https://www.hzdr.de/crp
>>>> Computational Radiation Physics
>>>> Laser Particle Acceleration Division
>>>> Helmholtz-Zentrum Dresden - Rossendorf e.V.
>>>>
>>>> Bautzner Landstrasse 400, 01328 Dresden
>>>> POB 510119, D-01314 Dresden
>>>> Vorstand: Prof. Dr.Dr.h.c. R. Sauerbrey
>>>>           Prof. Dr.Dr.h.c. P. Joehnk
>>>> VR 1693 beim Amtsgericht Dresden
>>>>
>>>>
>>>
>>