picongpu-users@hzdr.de Mailing List Archive

Great to hear that! Memory usage was just too high then. Did you already try the second card? With that, you can simulate twice the size in nearly the same time ("weak scaling"). Or alternatively the same setup in half the time ("strong scaling"). Axel On August 9, 2015 9:36:21 AM CEST, "k.manzoorolajdad" <kazem.manzoor@gmail.com> wrote: >I reduce the gride to 64 128 64 and the run comlete. >Thanks a lot >در تاریخ ۸ اوت ۲۰۱۵ ۲۰:۵۸، "k.manzoorolajdad" <kazem.manzoor@gmail.com> >نوشت: > >> thanks a lot >> this is output in terminal: >> >> manzoor@manzoor-gpu:~/paramSets/lwfa$ tbg -s bash -c >submit/0001gpus.cfg >> -t submit/bash/bash_mpirun.tpl ~/runs/lwfa >> Running program... >> tbg/submit.start: line 37: /home/manzoor/picongpu.profile: No such >file or >> directory >> Data for JOB [64751,1] offset 0 >> >> ======================== JOB MAP ======================== >> >> Data for node: manzoor-gpu Num slots: 4 Max slots: 0 Num >procs: 1 >> Process OMPI jobid: [64751,1] App: 0 Process rank: 0 >> >> ============================================================= >> Data for JOB [64762,1] offset 0 >> >> ======================== JOB MAP ======================== >> >> Data for node: manzoor-gpu Num slots: 4 Max slots: 0 Num >procs: 1 >> Process OMPI jobid: [64762,1] App: 0 Process rank: 0 >> >> ============================================================= >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | Sliding Window is OFF >> [1,0]<stdout>:mem for particles=1078 MiB = 142315 Frames = 36432640 >> Particles >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | max weighting 6955.06 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | Courant c*dt <= 1.74147 ? >1 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | omega_pe * dt <= 0.1 ? >0.0142719 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | y-cells per wavelength: >18.0587 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | macro particles per gpu: >8388608 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | typical macro particle >> weighting: 6955.06 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_SPEED 2.99792e+08 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_TIME 8e-17 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_LENGTH 2.39834e-08 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_MASS 6.33563e-27 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_CHARGE 1.11432e-15 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_EFIELD 2.13064e+13 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_BFIELD 71070.4 >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_ENERGY 5.69418e-10 >> >> >> On Sat, Aug 8, 2015 at 3:44 PM, Axel Huebl <a.huebl@hzdr.de> wrote: >> >>> Hi, >>> >>> Can you please attach your cfg file and output? I think your >example's >>> memory consumption is too high, causing a hang. >>> >>> If you want to speed up the init, use quiet start (lattice-like in >cell >>> positioning of particles) and zero initial temperature. >>> >>> Two cards, either in the same node or via network/mpi connected will >give >>> you nearly a 2x speedup. SLI is not required, cuda can't use and >doesn't >>> need it (it's more a frame buffer/gfx/gaming interconnect). Just >plug the >>> second card in and use it with -d ... :) >>> >>> Also, make sure your host=CPU RAM is at least as much as the sum of >the >>> RAM of the GPUs in the node (better a bit more). >>> >>> Best, >>> Axel >>> >>> On August 8, 2015 12:37:35 PM CEST, "k.manzoorolajdad" < >>> kazem.manzoor@gmail.com> wrote: >>> >thanks a lot Mr.Huebl >>> >the example lwfa is running in my machine with single core with >single >>> >gpu >>> >since 45 hour ago without output. is it normal? >>> >how long this code run? >>> > >>> >if i use two gpu 670 with SLI can i speed up? how much? >>> >thanks >>> >manzoor >>> > >>> > >>> >On Thu, Aug 6, 2015 at 7:34 PM, k.manzoorolajdad >>> ><kazem.manzoor@gmail.com> >>> >wrote: >>> > >>> >> thanks >>> >> i can run first example : >>> >> >>> >> ~/paramSets/lwfa$ tbg -s bash -c submit/0001gpus.cfg -t >>> >> submit/bash/bash_mpirun.tpl ~/runs/lwfa >>> >> Running program... >>> >> tbg/submit.start: line 37: /home/manzoor/picongpu.profile: No >such >>> >file or >>> >> directory >>> >> Data for JOB [64751,1] offset 0 >>> >> >>> >> ======================== JOB MAP ======================== >>> >> >>> >> Data for node: manzoor-gpu Num slots: 4 Max slots: 0 >Num >>> >procs: 1 >>> >> Process OMPI jobid: [64751,1] App: 0 Process rank: 0 >>> >> >>> >> ============================================================= >>> >> Data for JOB [64762,1] offset 0 >>> >> >>> >> ======================== JOB MAP ======================== >>> >> >>> >> Data for node: manzoor-gpu Num slots: 4 Max slots: 0 >Num >>> >procs: 1 >>> >> Process OMPI jobid: [64762,1] App: 0 Process rank: 0 >>> >> >>> >> ============================================================= >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | Sliding Window is OFF >>> >> [1,0]<stdout>:mem for particles=1078 MiB = 142315 Frames = >36432640 >>> >> Particles >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | max weighting 6955.06 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | Courant c*dt <= >1.74147 ? >>> >1 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | omega_pe * dt <= 0.1 ? >>> >0.0142719 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | y-cells per >wavelength: >>> >18.0587 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | macro particles per >gpu: >>> >8388608 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | typical macro particle >>> >> weighting: 6955.06 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_SPEED 2.99792e+08 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_TIME 8e-17 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_LENGTH >2.39834e-08 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_MASS 6.33563e-27 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_CHARGE >1.11432e-15 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_EFIELD >2.13064e+13 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_BFIELD 71070.4 >>> >> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_ENERGY >5.69418e-10 >>> >> >>> >> i think my run in single core with single gpu. >>> >> >>> >> >>> >> *can i run multi cpu with single gpu?* >>> >> thanks a lot >>> >> manzoor >>> >> >>> >> On Thu, Aug 6, 2015 at 4:03 PM, k.manzoorolajdad >>> ><kazem.manzoor@gmail.com> >>> >> wrote: >>> >> >>> >>> thanks Mr.Huebl >>> >>> >>> >>> i use ubuntu 14.04 >>> >>> cuda 6.0 >>> >>> and install requirements: >>> >>> >>> >>> sudo apt-get install build-essential cmake file cmake-curses-gui >>> >>> libopenmpi-dev zlib1g-dev libboost-program-options-dev >>> >libboost-regex-dev >>> >>> libboost-filesystem-dev libboost-system-dev git >>> >>> >>> >>> git clone >>> >https://github.com/ComputationalRadiationPhysics/picongpu.git >>> >>> $HOME/src/picongpu >>> >>> >>> >>> >>> >>> export PICSRC=$HOME/src/picongpu >>> >>> >>> >>> *pngwriter* >= 0.5.5 >>> >>> >>> >>> mkdir -p ~/src ~/build ~/lib >>> >>> git clone https://github.com/pngwriter/pngwriter.git >>> >~/src/pngwriter/ >>> >>> >>> >>> cd ~/build >>> >>> cmake -DCMAKE_INSTALL_PREFIX=~/lib/pngwriter ~/src/pngwriter >>> >>> manzoor@manzoor-gpu:~/build$ make install >>> >>> Scanning dependencies of target pngwriter_static >>> >>> [ 16%] Building CXX object >>> >>> CMakeFiles/pngwriter_static.dir/src/pngwriter.cc.o >>> >>> Linking CXX static library libpngwriter.a >>> >>> [ 16%] Built target pngwriter_static >>> >>> Scanning dependencies of target blackwhite >>> >>> [ 33%] Building CXX object >>> >CMakeFiles/blackwhite.dir/tests/blackwhite.cc.o >>> >>> Linking CXX executable blackwhite >>> >>> [ 33%] Built target blackwhite >>> >>> Scanning dependencies of target diamond >>> >>> [ 50%] Building CXX object >CMakeFiles/diamond.dir/tests/diamond.cc.o >>> >>> Linking CXX executable diamond >>> >>> [ 50%] Built target diamond >>> >>> Scanning dependencies of target lyapunov >>> >>> [ 66%] Building CXX object >>> >CMakeFiles/lyapunov.dir/examples/lyapunov.cc.o >>> >>> Linking CXX executable lyapunov >>> >>> [ 66%] Built target lyapunov >>> >>> Scanning dependencies of target pngtest >>> >>> [ 83%] Building CXX object >>> >CMakeFiles/pngtest.dir/examples/pngtest.cc.o >>> >>> Linking CXX executable pngtest >>> >>> [ 83%] Built target pngtest >>> >>> Scanning dependencies of target pngwriter >>> >>> [100%] Building CXX object >>> >CMakeFiles/pngwriter.dir/src/pngwriter.cc.o >>> >>> Linking CXX shared library libpngwriter.so >>> >>> [100%] Built target pngwriter >>> >>> Install the project... >>> >>> -- Install configuration: "" >>> >>> -- Installing: /home/manzoor/lib/pngwriter/lib/libpngwriter.so >>> >>> -- Installing: /home/manzoor/lib/pngwriter/lib/libpngwriter.a >>> >>> -- Installing: /home/manzoor/lib/pngwriter/include/pngwriter.h >>> >>> >>> >>> export CUDA_ROOT=/usr/local/cuda-6.0/ >>> >>> export MPI_ROOT=/usr/local/ >>> >>> export PATH=$PATH:$HOME/src/picongpu/src/tools/bin >>> >>> export PNGWRITER_ROOT=$HOME/lib/pngwriter >>> >>> >>> >>> mkdir -p ~/src ~/build ~/paramSets ~/runs >>> >>> >>> >>> ~/src/picongpu/createParameterSet >>> >~/src/picongpu/examples/LaserWakefield/ >>> >>> paramSets/lwfa/ >>> >>> >>> >>> cd build/ >>> >>> >>> >>> manzoor@manzoor-gpu:~/build$ ~/src/picongpu/configure -a sm_30 >>> >>> ../paramSets/lwfa >>> >>> cmake command: cmake -DCUDA_ARCH=sm_20 >>> >>> -DCMAKE_INSTALL_PREFIX=../paramSets/lwfa >>> >>> -DPIC_EXTENSION_PATH=../paramSets/lwfa -DCUDA_ARCH=sm_30 >>> >>> /home/manzoor/src/picongpu >>> >>> *CMake Error: The source >"/home/manzoor/src/picongpu/CMakeLists.txt" >>> >does >>> >>> not match the source >"/home/manzoor/src/pngwriter/CMakeLists.txt" >>> >used to >>> >>> generate cache. Re-run cmake with a different source >directory.* >>> >>> >>> >>> >>> >>> On Thu, Aug 6, 2015 at 2:07 PM, Huebl, Axel <a.huebl@hzdr.de> >wrote: >>> >>> >>> >>>> Dear Manzoor, >>> >>>> >>> >>>> welcome to our user list! >>> >>>> >>> >>>> The 670 GTX is a Kepler generation card with sm_30 so you are >good >>> >to go >>> >>>> from the hardware side (we support sm_20 "Fermi" and upward): >>> >>>> https://developer.nvidia.com/cuda-gpus >>> >>>> >>> >>>> We would recommend you to install a linux operation system, the >>> >latest >>> >>>> CUDA Toolkit >>> >>>> https://developer.nvidia.com/cuda-downloads >>> >>>> >>> >>>> and the additional required tools and libraries documented >here: >>> >>>> >>> >>>> >>> >>>> >>> > >>> >https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/INSTALL.md#requirements >>> >>>> >>> >>>> They are all pretty standard and most get shipped with packages >>> >too, >>> >>>> e.g., in Debian, Ubuntu and Arch. Please read the instructions >we >>> >>>> provided in this file carefully. >>> >>>> >>> >>>> I also recommend you installing pngwriter as described under >>> >"optional" >>> >>>> since it allows an easy check of the output with our png >(preview) >>> >>>> plugin. >>> >>>> >>> >>>> If you installed the requirements, just scroll a bit down in >the >>> >>>> INSTALL.md guide and set up a simulation case. This is >additionally >>> >>>> documented in a youtube video >>> >>>> https://www.youtube.com/watch?v=7ybsD8G4Rsk >>> >>>> >>> >>>> With the binary compiled, you can set plugins in the case's >*.cfg >>> >file >>> >>>> when you are at this point. All available options are >documented >>> >here >>> >>>> >>> >>>> >>> >>>> >>> > >>> >https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/TBG_macros.cfg >>> >>>> >>> >>>> and in the wiki >>> >>>> >>> >>>> >>> >>>> >>> > >>> >https://github.com/ComputationalRadiationPhysics/picongpu/wiki/PIConGPU-Plugins >>> >>>> >>> >>>> The tbg template you want to use is "bash_mpirun.tpl" and set >to "1 >>> >1 1" >>> >>>> GPUs in your ".cfg" file. >>> >>>> >>> >>>> For further resources, please continue to read (links below): >>> >>>> >>> >>>> [0] README.md >>> >>>> [1] doc/INSTALL.md >>> >>>> [2] our wiki >>> >>>> [3] doc/TBG_macros.cfg >>> >>>> [4] closed questions in our issue tracker >>> >>>> >>> >>>> >>> >>>> If there should pop up problems along the way, feel free to ask >>> >again on >>> >>>> the list! >>> >>>> >>> >>>> >>> >>>> Best regards, >>> >>>> Axel Huebl >>> >>>> >>> >>>> [0] >>> >>>> >>> >>>> >>> > >>> >https://github.com/ComputationalRadiationPhysics/picongpu#picongpu---a-many-gpgpu-pic-code >>> >>>> [1] >>> >>>> >>> >>>> >>> > >>> >https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/INSTALL.md >>> >>>> [2] >https://github.com/ComputationalRadiationPhysics/picongpu/wiki >>> >>>> [3] >>> >>>> >>> >>>> >>> > >>> >https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/TBG_macros.cfg >>> >>>> [4] >>> >>>> >>> >>>> >>> > >>> >https://github.com/ComputationalRadiationPhysics/picongpu/issues?q=is%3Aissue+label%3Aquestion+is%3Aclosed >>> >>>> >>> >>>> On 06.08.2015 11:21, k.manzoorolajdad wrote: >>> >>>> > Hi >>> >>>> > I am new in CUDA andComputational Radiation Physics and want >use >>> >>>> > picongpu but don't have gpu cluster. >>> >>>> > I have single GPU (Geforce 670 gtx) and want test and run >code. >>> >>>> > >>> >>>> > How can run code in single machine and single gpu? >>> >>>> > >>> >>>> > Thanks >>> >>>> > >>> >>>> > Manzoor >>> >>>> > Ms.student of physic >>> >>>> > Tehran university,Iran >>> >>>> >>> >>>> -- >>> >>>> >>> >>>> Axel Huebl >>> >>>> Phone +49 351 260 3582 >>> >>>> https://www.hzdr.de/crp >>> >>>> Computational Radiation Physics >>> >>>> Laser Particle Acceleration Division >>> >>>> Helmholtz-Zentrum Dresden - Rossendorf e.V. >>> >>>> >>> >>>> Bautzner Landstrasse 400, 01328 Dresden >>> >>>> POB 510119, D-01314 Dresden >>> >>>> Vorstand: Prof. Dr.Dr.h.c. R. Sauerbrey >>> >>>> Prof. Dr.Dr.h.c. P. Joehnk >>> >>>> VR 1693 beim Amtsgericht Dresden >>> >>>> >>> >>>> >>> >>> >>> >> >>> >>> >>> ############################################################# >>> This message is sent to you because you are subscribed to >>> the mailing list <picongpu-users@hzdr.de>. >>> To unsubscribe, E-mail to: <picongpu-users-off@hzdr.de> >>> To switch to the DIGEST mode, E-mail to ><picongpu-users-digest@hzdr.de> >>> To switch to the INDEX mode, E-mail to ><picongpu-users-index@hzdr.de> >>> Send administrative queries to <picongpu-users-request@hzdr.de> >>> >>> >> >> ############################################################# >> This message is sent to you because you are subscribed to >> the mailing list <picongpu-users@hzdr.de>. >> To unsubscribe, E-mail to: <picongpu-users-off@hzdr.de> >> To switch to the DIGEST mode, E-mail to ><picongpu-users-digest@hzdr.de> >> To switch to the INDEX mode, E-mail to <picongpu-users-index@hzdr.de> >> Send administrative queries to <picongpu-users-request@hzdr.de> >> >> >>