picongpu-users@hzdr.de Mailing List Archive

Hi Manzoor, can you retry the old size again but set the value totalFreeGpuMemory in param/memory.param to 400 and 450 MB? Currently it's 350 MB and we would like to see if we can set a more reasonable default for 2GB // GForce Cards. Thanks! Axel On 09.08.2015 13:09, Axel Huebl wrote: > Great to hear that! Memory usage was just too high then. > > Did you already try the second card? > With that, you can simulate twice the size in nearly the same time ("weak scaling"). Or alternatively the same setup in half the time ("strong scaling"). > > Axel > > On August 9, 2015 9:36:21 AM CEST, "k.manzoorolajdad" <kazem.manzoor@gmail.com> wrote: >> I reduce the gride to 64 128 64 and the run comlete. >> Thanks a lot >> در تاریخ ۸ اوت ۲۰۱۵ ۲۰:۵۸، "k.manzoorolajdad" <kazem.manzoor@gmail.com> >> نوشت: >> >>> thanks a lot >>> this is output in terminal: >>> >>> manzoor@manzoor-gpu:~/paramSets/lwfa$ tbg -s bash -c >> submit/0001gpus.cfg >>> -t submit/bash/bash_mpirun.tpl ~/runs/lwfa >>> Running program... >>> tbg/submit.start: line 37: /home/manzoor/picongpu.profile: No such >> file or >>> directory >>> Data for JOB [64751,1] offset 0 >>> >>> ======================== JOB MAP ======================== >>> >>> Data for node: manzoor-gpu Num slots: 4 Max slots: 0 Num >> procs: 1 >>> Process OMPI jobid: [64751,1] App: 0 Process rank: 0 >>> >>> ============================================================= >>> Data for JOB [64762,1] offset 0 >>> >>> ======================== JOB MAP ======================== >>> >>> Data for node: manzoor-gpu Num slots: 4 Max slots: 0 Num >> procs: 1 >>> Process OMPI jobid: [64762,1] App: 0 Process rank: 0 >>> >>> ============================================================= >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | Sliding Window is OFF >>> [1,0]<stdout>:mem for particles=1078 MiB = 142315 Frames = 36432640 >>> Particles >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | max weighting 6955.06 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | Courant c*dt <= 1.74147 ? >> 1 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | omega_pe * dt <= 0.1 ? >> 0.0142719 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | y-cells per wavelength: >> 18.0587 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | macro particles per gpu: >> 8388608 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | typical macro particle >>> weighting: 6955.06 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_SPEED 2.99792e+08 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_TIME 8e-17 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_LENGTH 2.39834e-08 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_MASS 6.33563e-27 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_CHARGE 1.11432e-15 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_EFIELD 2.13064e+13 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_BFIELD 71070.4 >>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_ENERGY 5.69418e-10 >>> >>> >>> On Sat, Aug 8, 2015 at 3:44 PM, Axel Huebl <a.huebl@hzdr.de> wrote: >>> >>>> Hi, >>>> >>>> Can you please attach your cfg file and output? I think your >> example's >>>> memory consumption is too high, causing a hang. >>>> >>>> If you want to speed up the init, use quiet start (lattice-like in >> cell >>>> positioning of particles) and zero initial temperature. >>>> >>>> Two cards, either in the same node or via network/mpi connected will >> give >>>> you nearly a 2x speedup. SLI is not required, cuda can't use and >> doesn't >>>> need it (it's more a frame buffer/gfx/gaming interconnect). Just >> plug the >>>> second card in and use it with -d ... :) >>>> >>>> Also, make sure your host=CPU RAM is at least as much as the sum of >> the >>>> RAM of the GPUs in the node (better a bit more). >>>> >>>> Best, >>>> Axel >>>> >>>> On August 8, 2015 12:37:35 PM CEST, "k.manzoorolajdad" < >>>> kazem.manzoor@gmail.com> wrote: >>>>> thanks a lot Mr.Huebl >>>>> the example lwfa is running in my machine with single core with >> single >>>>> gpu >>>>> since 45 hour ago without output. is it normal? >>>>> how long this code run? >>>>> >>>>> if i use two gpu 670 with SLI can i speed up? how much? >>>>> thanks >>>>> manzoor >>>>> >>>>> >>>>> On Thu, Aug 6, 2015 at 7:34 PM, k.manzoorolajdad >>>>> <kazem.manzoor@gmail.com> >>>>> wrote: >>>>> >>>>>> thanks >>>>>> i can run first example : >>>>>> >>>>>> ~/paramSets/lwfa$ tbg -s bash -c submit/0001gpus.cfg -t >>>>>> submit/bash/bash_mpirun.tpl ~/runs/lwfa >>>>>> Running program... >>>>>> tbg/submit.start: line 37: /home/manzoor/picongpu.profile: No >> such >>>>> file or >>>>>> directory >>>>>> Data for JOB [64751,1] offset 0 >>>>>> >>>>>> ======================== JOB MAP ======================== >>>>>> >>>>>> Data for node: manzoor-gpu Num slots: 4 Max slots: 0 >> Num >>>>> procs: 1 >>>>>> Process OMPI jobid: [64751,1] App: 0 Process rank: 0 >>>>>> >>>>>> ============================================================= >>>>>> Data for JOB [64762,1] offset 0 >>>>>> >>>>>> ======================== JOB MAP ======================== >>>>>> >>>>>> Data for node: manzoor-gpu Num slots: 4 Max slots: 0 >> Num >>>>> procs: 1 >>>>>> Process OMPI jobid: [64762,1] App: 0 Process rank: 0 >>>>>> >>>>>> ============================================================= >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | Sliding Window is OFF >>>>>> [1,0]<stdout>:mem for particles=1078 MiB = 142315 Frames = >> 36432640 >>>>>> Particles >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | max weighting 6955.06 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | Courant c*dt <= >> 1.74147 ? >>>>> 1 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | omega_pe * dt <= 0.1 ? >>>>> 0.0142719 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | y-cells per >> wavelength: >>>>> 18.0587 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | macro particles per >> gpu: >>>>> 8388608 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | typical macro particle >>>>>> weighting: 6955.06 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_SPEED 2.99792e+08 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_TIME 8e-17 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_LENGTH >> 2.39834e-08 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_MASS 6.33563e-27 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_CHARGE >> 1.11432e-15 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_EFIELD >> 2.13064e+13 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_BFIELD 71070.4 >>>>>> [1,0]<stdout>:PIConGPUVerbose PHYSICS(1) | UNIT_ENERGY >> 5.69418e-10 >>>>>> >>>>>> i think my run in single core with single gpu. >>>>>> >>>>>> >>>>>> *can i run multi cpu with single gpu?* >>>>>> thanks a lot >>>>>> manzoor >>>>>> >>>>>> On Thu, Aug 6, 2015 at 4:03 PM, k.manzoorolajdad >>>>> <kazem.manzoor@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> thanks Mr.Huebl >>>>>>> >>>>>>> i use ubuntu 14.04 >>>>>>> cuda 6.0 >>>>>>> and install requirements: >>>>>>> >>>>>>> sudo apt-get install build-essential cmake file cmake-curses-gui >>>>>>> libopenmpi-dev zlib1g-dev libboost-program-options-dev >>>>> libboost-regex-dev >>>>>>> libboost-filesystem-dev libboost-system-dev git >>>>>>> >>>>>>> git clone >>>>> https://github.com/ComputationalRadiationPhysics/picongpu.git >>>>>>> $HOME/src/picongpu >>>>>>> >>>>>>> >>>>>>> export PICSRC=$HOME/src/picongpu >>>>>>> >>>>>>> *pngwriter* >= 0.5.5 >>>>>>> >>>>>>> mkdir -p ~/src ~/build ~/lib >>>>>>> git clone https://github.com/pngwriter/pngwriter.git >>>>> ~/src/pngwriter/ >>>>>>> >>>>>>> cd ~/build >>>>>>> cmake -DCMAKE_INSTALL_PREFIX=~/lib/pngwriter ~/src/pngwriter >>>>>>> manzoor@manzoor-gpu:~/build$ make install >>>>>>> Scanning dependencies of target pngwriter_static >>>>>>> [ 16%] Building CXX object >>>>>>> CMakeFiles/pngwriter_static.dir/src/pngwriter.cc.o >>>>>>> Linking CXX static library libpngwriter.a >>>>>>> [ 16%] Built target pngwriter_static >>>>>>> Scanning dependencies of target blackwhite >>>>>>> [ 33%] Building CXX object >>>>> CMakeFiles/blackwhite.dir/tests/blackwhite.cc.o >>>>>>> Linking CXX executable blackwhite >>>>>>> [ 33%] Built target blackwhite >>>>>>> Scanning dependencies of target diamond >>>>>>> [ 50%] Building CXX object >> CMakeFiles/diamond.dir/tests/diamond.cc.o >>>>>>> Linking CXX executable diamond >>>>>>> [ 50%] Built target diamond >>>>>>> Scanning dependencies of target lyapunov >>>>>>> [ 66%] Building CXX object >>>>> CMakeFiles/lyapunov.dir/examples/lyapunov.cc.o >>>>>>> Linking CXX executable lyapunov >>>>>>> [ 66%] Built target lyapunov >>>>>>> Scanning dependencies of target pngtest >>>>>>> [ 83%] Building CXX object >>>>> CMakeFiles/pngtest.dir/examples/pngtest.cc.o >>>>>>> Linking CXX executable pngtest >>>>>>> [ 83%] Built target pngtest >>>>>>> Scanning dependencies of target pngwriter >>>>>>> [100%] Building CXX object >>>>> CMakeFiles/pngwriter.dir/src/pngwriter.cc.o >>>>>>> Linking CXX shared library libpngwriter.so >>>>>>> [100%] Built target pngwriter >>>>>>> Install the project... >>>>>>> -- Install configuration: "" >>>>>>> -- Installing: /home/manzoor/lib/pngwriter/lib/libpngwriter.so >>>>>>> -- Installing: /home/manzoor/lib/pngwriter/lib/libpngwriter.a >>>>>>> -- Installing: /home/manzoor/lib/pngwriter/include/pngwriter.h >>>>>>> >>>>>>> export CUDA_ROOT=/usr/local/cuda-6.0/ >>>>>>> export MPI_ROOT=/usr/local/ >>>>>>> export PATH=$PATH:$HOME/src/picongpu/src/tools/bin >>>>>>> export PNGWRITER_ROOT=$HOME/lib/pngwriter >>>>>>> >>>>>>> mkdir -p ~/src ~/build ~/paramSets ~/runs >>>>>>> >>>>>>> ~/src/picongpu/createParameterSet >>>>> ~/src/picongpu/examples/LaserWakefield/ >>>>>>> paramSets/lwfa/ >>>>>>> >>>>>>> cd build/ >>>>>>> >>>>>>> manzoor@manzoor-gpu:~/build$ ~/src/picongpu/configure -a sm_30 >>>>>>> ../paramSets/lwfa >>>>>>> cmake command: cmake -DCUDA_ARCH=sm_20 >>>>>>> -DCMAKE_INSTALL_PREFIX=../paramSets/lwfa >>>>>>> -DPIC_EXTENSION_PATH=../paramSets/lwfa -DCUDA_ARCH=sm_30 >>>>>>> /home/manzoor/src/picongpu >>>>>>> *CMake Error: The source >> "/home/manzoor/src/picongpu/CMakeLists.txt" >>>>> does >>>>>>> not match the source >> "/home/manzoor/src/pngwriter/CMakeLists.txt" >>>>> used to >>>>>>> generate cache. Re-run cmake with a different source >> directory.* >>>>>>> >>>>>>> >>>>>>> On Thu, Aug 6, 2015 at 2:07 PM, Huebl, Axel <a.huebl@hzdr.de> >> wrote: >>>>>>> >>>>>>>> Dear Manzoor, >>>>>>>> >>>>>>>> welcome to our user list! >>>>>>>> >>>>>>>> The 670 GTX is a Kepler generation card with sm_30 so you are >> good >>>>> to go >>>>>>>> from the hardware side (we support sm_20 "Fermi" and upward): >>>>>>>> https://developer.nvidia.com/cuda-gpus >>>>>>>> >>>>>>>> We would recommend you to install a linux operation system, the >>>>> latest >>>>>>>> CUDA Toolkit >>>>>>>> https://developer.nvidia.com/cuda-downloads >>>>>>>> >>>>>>>> and the additional required tools and libraries documented >> here: >>>>>>>> >>>>>>>> >>>>>>>> >>>>> >>>> >> https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/INSTALL.md#requirements >>>>>>>> >>>>>>>> They are all pretty standard and most get shipped with packages >>>>> too, >>>>>>>> e.g., in Debian, Ubuntu and Arch. Please read the instructions >> we >>>>>>>> provided in this file carefully. >>>>>>>> >>>>>>>> I also recommend you installing pngwriter as described under >>>>> "optional" >>>>>>>> since it allows an easy check of the output with our png >> (preview) >>>>>>>> plugin. >>>>>>>> >>>>>>>> If you installed the requirements, just scroll a bit down in >> the >>>>>>>> INSTALL.md guide and set up a simulation case. This is >> additionally >>>>>>>> documented in a youtube video >>>>>>>> https://www.youtube.com/watch?v=7ybsD8G4Rsk >>>>>>>> >>>>>>>> With the binary compiled, you can set plugins in the case's >> *.cfg >>>>> file >>>>>>>> when you are at this point. All available options are >> documented >>>>> here >>>>>>>> >>>>>>>> >>>>>>>> >>>>> >>>> >> https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/TBG_macros.cfg >>>>>>>> >>>>>>>> and in the wiki >>>>>>>> >>>>>>>> >>>>>>>> >>>>> >>>> >> https://github.com/ComputationalRadiationPhysics/picongpu/wiki/PIConGPU-Plugins >>>>>>>> >>>>>>>> The tbg template you want to use is "bash_mpirun.tpl" and set >> to "1 >>>>> 1 1" >>>>>>>> GPUs in your ".cfg" file. >>>>>>>> >>>>>>>> For further resources, please continue to read (links below): >>>>>>>> >>>>>>>> [0] README.md >>>>>>>> [1] doc/INSTALL.md >>>>>>>> [2] our wiki >>>>>>>> [3] doc/TBG_macros.cfg >>>>>>>> [4] closed questions in our issue tracker >>>>>>>> >>>>>>>> >>>>>>>> If there should pop up problems along the way, feel free to ask >>>>> again on >>>>>>>> the list! >>>>>>>> >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Axel Huebl >>>>>>>> >>>>>>>> [0] >>>>>>>> >>>>>>>> >>>>> >>>> >> https://github.com/ComputationalRadiationPhysics/picongpu#picongpu---a-many-gpgpu-pic-code >>>>>>>> [1] >>>>>>>> >>>>>>>> >>>>> >>>> >> https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/INSTALL.md >>>>>>>> [2] >> https://github.com/ComputationalRadiationPhysics/picongpu/wiki >>>>>>>> [3] >>>>>>>> >>>>>>>> >>>>> >>>> >> https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/doc/TBG_macros.cfg >>>>>>>> [4] >>>>>>>> >>>>>>>> >>>>> >>>> >> https://github.com/ComputationalRadiationPhysics/picongpu/issues?q=is%3Aissue+label%3Aquestion+is%3Aclosed >>>>>>>> >>>>>>>> On 06.08.2015 11:21, k.manzoorolajdad wrote: >>>>>>>>> Hi >>>>>>>>> I am new in CUDA andComputational Radiation Physics and want >> use >>>>>>>>> picongpu but don't have gpu cluster. >>>>>>>>> I have single GPU (Geforce 670 gtx) and want test and run >> code. >>>>>>>>> >>>>>>>>> How can run code in single machine and single gpu? >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> Manzoor >>>>>>>>> Ms.student of physic >>>>>>>>> Tehran university,Iran >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Axel Huebl >>>>>>>> Phone +49 351 260 3582 >>>>>>>> https://www.hzdr.de/crp >>>>>>>> Computational Radiation Physics >>>>>>>> Laser Particle Acceleration Division >>>>>>>> Helmholtz-Zentrum Dresden - Rossendorf e.V. >>>>>>>> >>>>>>>> Bautzner Landstrasse 400, 01328 Dresden >>>>>>>> POB 510119, D-01314 Dresden >>>>>>>> Vorstand: Prof. Dr.Dr.h.c. R. Sauerbrey >>>>>>>> Prof. Dr.Dr.h.c. P. Joehnk >>>>>>>> VR 1693 beim Amtsgericht Dresden >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>>> >>>> ############################################################# >>>> This message is sent to you because you are subscribed to >>>> the mailing list <picongpu-users@hzdr.de>. >>>> To unsubscribe, E-mail to: <picongpu-users-off@hzdr.de> >>>> To switch to the DIGEST mode, E-mail to >> <picongpu-users-digest@hzdr.de> >>>> To switch to the INDEX mode, E-mail to >> <picongpu-users-index@hzdr.de> >>>> Send administrative queries to <picongpu-users-request@hzdr.de> >>>> >>>> >>> >>> ############################################################# >>> This message is sent to you because you are subscribed to >>> the mailing list <picongpu-users@hzdr.de>. >>> To unsubscribe, E-mail to: <picongpu-users-off@hzdr.de> >>> To switch to the DIGEST mode, E-mail to >> <picongpu-users-digest@hzdr.de> >>> To switch to the INDEX mode, E-mail to <picongpu-users-index@hzdr.de> >>> Send administrative queries to <picongpu-users-request@hzdr.de> >>> >>> >>> > > > ############################################################# > This message is sent to you because you are subscribed to > the mailing list <picongpu-users@hzdr.de>. > To unsubscribe, E-mail to: <picongpu-users-off@hzdr.de> > To switch to the DIGEST mode, E-mail to <picongpu-users-digest@hzdr.de> > To switch to the INDEX mode, E-mail to <picongpu-users-index@hzdr.de> > Send administrative queries to <picongpu-users-request@hzdr.de> > -- Axel Huebl Phone +49 351 260 3582 https://www.hzdr.de/crp Computational Radiation Physics Laser Particle Acceleration Division Helmholtz-Zentrum Dresden - Rossendorf e.V. Bautzner Landstrasse 400, 01328 Dresden POB 510119, D-01314 Dresden Vorstand: Prof. Dr.Dr.h.c. R. Sauerbrey Prof. Dr.Dr.h.c. P. Joehnk VR 1693 beim Amtsgericht Dresden