Return-Path: Received: from mx2.fz-rossendorf.de ([149.220.142.12] verified) by hzdr.de (CommuniGate Pro SMTP 6.1.12) with ESMTP id 16382240 for picongpu-users@cg.hzdr.de; Mon, 13 Feb 2017 16:17:34 +0100 Received: from localhost (localhost [127.0.0.1]) by mx2.fz-rossendorf.de (Postfix) with ESMTP id 1C7BF42E98 for ; Mon, 13 Feb 2017 16:17:34 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at mx2.fz-rossendorf.de Received: from mx2.fz-rossendorf.de ([127.0.0.1]) by localhost (mx2.fz-rossendorf.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MYGcnqNmwOVk for ; Mon, 13 Feb 2017 16:17:29 +0100 (CET) Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=147.231.234.10; helo=mailgw.eli-beams.eu; envelope-from=prvs=1217dec4b3=danila.khikhlukha@eli-beams.eu; receiver=picongpu-users@hzdr.de Received: from mailgw.eli-beams.eu (mailgw.eli-beams.eu [147.231.234.10]) by mx2.fz-rossendorf.de (Postfix) with ESMTPS id 9634B42ECC for ; Mon, 13 Feb 2017 16:17:29 +0100 (CET) Received: from mail.eli-beams.eu ([10.1.5.17]) by mailgw.eli-beams.eu with ESMTP id v1DEl9X8006035-v1DEl9XA006035; Mon, 13 Feb 2017 15:47:09 +0100 Received: from BRAUN.eli-beams.eu ([::1]) by braun.eli-beams.eu ([::1]) with mapi id 14.03.0319.002; Mon, 13 Feb 2017 15:47:09 +0100 From: Khikhlukha Danila To: "picongpu-users@hzdr.de" Subject: RE: [PIConGPU-Users] [PIConGPU-Users] Restart failure Thread-Topic: [PIConGPU-Users] [PIConGPU-Users] Restart failure Thread-Index: AQHShgcINeXoCKt3PkqbMsohYQ4X/6FnA5XY Date: Mon, 13 Feb 2017 14:47:08 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US, cs-CZ Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.36.30.5] Content-Type: multipart/mixed; boundary="_002_BA7C853FEE430847B9C35FFCC6E5B2A52A32CBEDbraunelibeamseu_" MIME-Version: 1.0 --_002_BA7C853FEE430847B9C35FFCC6E5B2A52A32CBEDbraunelibeamseu_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Ren=E9,=0A= sure, pls. see the attachment. Please let me know if more information is ne= eded. =0A= =0A= D.=0A= ________________________________________=0A= From: picongpu-users@hzdr.de [picongpu-users@hzdr.de] on behalf of Ren=E9 W= idera [r.widera@hzdr.de]=0A= Sent: Monday, February 13, 2017 3:39 PM=0A= To: picongpu-users@hzdr.de=0A= Subject: Re: [PIConGPU-Users] [PIConGPU-Users] Restart failure=0A= =0A= Dear Danila,=0A= =0A= could you please send use the `stdout`, `stderr` and the files from the=0A= `tbg` folder?=0A= =0A= best,=0A= =0A= Ren=E9=0A= =0A= On 02/13/2017 03:11 PM, Khikhlukha Danila wrote:=0A= > Dear all,=0A= > currently I was trying to setup PoG in the Jureca machine. It all worked= =0A= > fine for the LWFA example, however when I tried to restart the=0A= > simulation I received a segfault almost immediately.=0A= > My tool chain is as follows=0A= >=0A= > GCC/5.4.0=0A= > CUDA/8.0.44=0A= > MVAPICH2/2.2-GDR=0A= > HDF5/1.8.17=0A= > Boost/1.61.0=0A= >=0A= > So, the first run didn't have any problems -- pictures, save points and= =0A= > data dumps were created. When I tried to launch the restart it crashes=0A= > although I explicitly specify the savepoint directory.=0A= >=0A= > test$ diff -r 0002/submit/ 0002_restart/submit/=0A= > diff -r 0002/submit/0008gpus.cfg 0002_restart/submit/0008gpus.cfg=0A= > 39c39=0A= > < TBG_steps=3D"-s 1024"=0A= > ---=0A= >> TBG_steps=3D"-s 2048"=0A= > 41a42=0A= >> TBG_restart=3D"--restart --restart-directory=0A= > /work/hhh20/hhh20z/run_0002/simOutput/checkpoints"=0A= > 67a69=0A= >> !TBG_restart \=0A= >=0A= > I also checked that it exists and accessible. I tried to switch on some= =0A= > debug information, with the following command:=0A= >=0A= > $PICSRC/configure -c"-DCMAKE_VERBOSE_MAKEFILE=3DON -DPIC_VERBOSE_LVL=3D29= =0A= > -DPMACC_VERBOSE_LVL=3D7"=0A= >=0A= > however I didn't find any information except a standard message:=0A= > [jrc0007:mpi_rank_4][error_sighandler] Caught error: Segmentation fault= =0A= > (signal 11)=0A= >=0A= > Could you please advice me if there are another way how to diagnose the= =0A= > problem (except launching a gdb). may be I'm doing something wrong?=0A= > However restart used to work on other machines...=0A= >=0A= >=0A= > Thank you in advance,=0A= > Danila.=0A= >=0A= =0A= --=0A= Ren=E9 Widera=0A= Abteilung Laser-Teilchenbeschleunigung (FWKT)=0A= Helmholtz-Zentrum Dresden-Rossendorf=0A= Tel: +49 (0351) 260 3543=0A= r.widera@hzdr.de=0A= http://www.hzdr.de=0A= =0A= Vorstand: Prof. Dr. Dr. h. c. Roland Sauerbrey,=0A= Prof. Dr. Dr. h. c. Peter Joehnk=0A= Vereinsregister: VR 1693 beim Amtsgericht Dresden=0A= =0A= #############################################################=0A= This message is sent to you because you are subscribed to=0A= the mailing list .=0A= To unsubscribe, E-mail to: =0A= To switch to the DIGEST mode, E-mail to =0A= To switch to the INDEX mode, E-mail to =0A= Send administrative queries to =0A= =0A= --_002_BA7C853FEE430847B9C35FFCC6E5B2A52A32CBEDbraunelibeamseu_ Content-Type: application/x-compressed-tar; name="restart_failure.tgz" Content-Description: restart_failure.tgz Content-Disposition: attachment; filename="restart_failure.tgz"; size=3234; creation-date="Mon, 13 Feb 2017 14:46:53 GMT"; modification-date="Mon, 13 Feb 2017 14:46:53 GMT" Content-Transfer-Encoding: base64 H4sIAD/GoVgAA+1be3PaSBLP33yKtpOq2EkAvTA2WXYLE/y48oMDvFlvLsUO0iAUC0mrGRnbdR/+ ukcIsION88C7m6OrEqSZ7p539697ZCEdHsfPlkoa0pZl0a9ZNk38NSxNU+9Ehq6Zz3RTK2mmuWWW tGearmtl6xloy+1WSomQLAZ4NhgMDO3mfj5V/xQdeloKLm0bEsEFuK9fAxMgBxwc3meJL2EQCgl2 OIw8n8cFOOzDdZjAiAUSZEhS4NmRDWEMkWun4uy2kBKwWQB+yByle1IzDJ3E58ACR2mijnRTNahP vSnlG/iWt+2eF1Q/qOo3VPxxswB7WEPafC55wIVQquwwuOSBxwN7rJKG5WGdL0KI4vDSc7hTgKbP GTUaSo69YhLqZ+9qsE2cWKbUMun1sH8jTw7gMJDcB0PTy5P+i9yHT7GN27dcwYJuzIKLbunjBzxM YdwVnjvA3iDbR6izxB1IUBUVaHN3yAOJysMA0lneQO6A+aDrm5nOralObQk6zSXo1Jeg0/heOmfW qLwEnVtL0Gl9s04RJ0El483mFiQTFwK0vDlPfp5MOZOx8uW5Mn+1CVvRN5CQTpjI5baxwP/rllWa +H+jbJL/143yyv8/BbWSIPACl1yjG7NhoVDINQ/rYbDfPPuVx70QnWTz4Lx9WG9v6JvwX2j7nkP8 773ACUfkL09PHpaoh0lMiMF+5Uj4qQp6Adff2IFfQH9YUETc9hCY8AqEQ+6ybgSvINWhFXSU1wq4 kco7ZethPdd5m/u+gAhBx4hdcp8HrhxUQN8uaKXt8sPCQ2bHCBxYLD3b56kSN0pQeqtcLhv61sPi 8jrybLTHt9XAiKNRlziPFdjaKZUK2gI1ZyeHnW672Wi8A6Ows1PeMfhrbfsRMp3D4wbOuLnD84u6 qviPGif7nQOwCjg83eD5RzVyXGu3YatgmqUtk+eNBTOqROoHtdY+9UzXLRPb0UuPEGrsHTaO3qGQ YaCd4K918xFCu6mQpe1oZuExo2mcNFr751AqbO1Y+jZ2TfuRPZzsucVlt0FGvlwq3Wf/iSb2Xy/p aP81yyg9g9KyO0b0f27/af1F0ht6skATsRQksMD/m1bJvLP+JoKClf9/Cnq+VkxEXMTwusiDS+gx Mcg9h8xK4ru0ByDs2Isk9DHg/tdZq1GvQfvorHWc1V4LyYc5FLPINaY+MggdjiUU2dvc8ylfYDM7 8Znkqk4gf3u31qkfQD6v/CJFFFWSn6mQ3pBXDbNS0iqaNlOuNFSN2RIVo1S3Z4rcGHnIV1szhaMw vnC8uFqkh6Ja1PT/myKGPl1KR3VRjo7CVCqEFCZPSziVYJiUy3F7EMLLz2HUy1zOduAxzQgESDaH Fwenxw21EFHodlFR3/M5zpLgkqYxhpgwg1CZl2o8unrrVuP81duwms/nc8kQhw+otZzLDWmAILzh aSKjRILxMxQdflkMEt+nPk1qaMlGHFisEiEgEnzw+mmWJoljDPOghzJXGJMW0ekWrIKJi4hLTjjI CwLuwJAPw/ga7BDZbelfo0ZARVEUxhIQ61xg5z901G/FwEOtlQzD0hHz69tbHyGNNlE9DG0GfZ+5 IlXAObbswkDKSFSKRdeTg6RXsMNhsR4Osecq/GR+izmeemwOroVniyKCrTDAJS9G2O+iZW7n+JXq yulx87B7XK91Kb72OaLAbjqCqkazgMuXZoBwPBKXRW3XAbcvAA/BSwEDznw5UNHxY5Z02g9aTjtx WHesuKCOFzWXHbEczvgHePEL7qk/QYOP8JYWIMgBfF1r2QvkHdBhG//dR3kXjNIW6GiN1cN9bAIM zdq+Vw1yDOeW5sf9mz7lcWPiNqEd88C4ipMNWlRrEIVegPv+3tbzuJaBW0Cr44UOmMakhF1heHJ9 NXkXvmfzJmnD0KE0Ke6HPh5lwMeGj72Lw0Cc//bAcB/X7M3XNPv7I5odOP1S1qpubGclZC3oaL9j ki3UgkKzU1vSDYpV8NSFqWHoe3gq5ADHMTb9IybAjjlabyfNiGJQ40NxgIFZ7/YqyjD0Zw6iiO1x EW1ORBu0n0TqOPL2fA24WYq3NvgYoVDhNnmIgt1HPfILpT+hfbNZvn/zqXixrWUWtiAj/1Gn7MeM Aqb4r2aTJUUD9d3bWID/Dd20pvgPgR/iP8tY4b8nofn4rx5G1wpu0J2Hmcf/tqB2xX04SHjPfwMt HnB476H9Ym9gj/veFbTtAW4ihEgo3iHLoQwS/hK4g7A/cXgFxTJBmMQZo+URYV+OEIpUJldGMXc8 IWOvlyBm9CRd7xRDdW/k9QloYFESkAklwCJ5PBTUDL3sn5zBPnYxZj40kx7aXzhCGxwgGEU7FlGJ GKAl65EaEtijHrTHPYC9EPUqXPEGOBo7bOKSx4Iy3mbWxFjfGwhj1LHBJHU7hjAisU3s6zUQ1p1I fj7q6eAc8AKldBBG4+soHNvIQwvb44T9+on/BqWRF94fdg5OzzpQOzmH97VWq3bSOX+rTDKabuCX PNXkDSPfI1vNYkq+XWO3UcFxo1U/QIna7uHRYeecLtr2DjsnjXYb9k5bUINmrdU5rJ8d1VrQPGs1 T9uNAkCbU6cIzT8wr321MjHdHUrm+SId7zkupcCe+Q4MEHPhkmIwcIn9YggZo+vF64U6mI+eJHU6 M1uILiMRtL5RWPEnAoqIE0ejUcENkkIYu0U/VSCKP1NXcs+fpy0iDkuB3RipC0LELzq7+11HyCZD kJfC58yB5ewI8nHKgSKf0FMTFx2YhTxeYPuJwxfypfZ3yqYQYb7/OaM9ZBd8j2AyfMyNUSLqfIhv ohRdOjXPUm50oIoTN1zgEbIOcCV6bi7TltUTdiVoQTt5Xr1A5M+juxyaevoxHeYPRjP5HwRCy2lj 4fcfln4n/2NY1ur+50loqfmfApkKKmry+AQLqn+kwS4Vp3fKeVeiVBb0gkqmWG+B++hS1MuU9y0i ij8eTimp9tLc0PqLjY0NmDb0Gu70BfI6bELxbvHm5vrjM1NrJDzCSKiDb5+lp9Ymvfk8T7U26dm8 hNXanU7NyV+tzbqs75CouqVvlZH6R2WkZtfuO6ae7lc7yTGtjZEHbaUmIzj1WR7hrzZwK3qQZvw/ YrrltLHA/1slfYv8v1U2tbJhmcr/0/3fyv8vn74t1oeFwT6kTN8a7sN3ivfhewT88LiIf87QvzDm h28O+uFbo374wrB/POhvDvzhnsgf4Mtif9Uf/O85ZQDUVqVY13OTePwJ43jj4oyr/ZE1hC4WfRtQ 6lq5XpX7BkZaPsPEkxVGrynS75LZeEJQKeI/xUqtzZyMNCNxtysC0qQBQTTKsLNLVMLoO+RLFnv0 INSwxwOaS05oF8kvq2+OVK6cuCkFsoBIY5urNHAFj/2fiYdnEn7NGgbFsUhHbhYTV9ezy9v1tAJB Q/eqqk+er6vbk+ebrDz2nLZ3g8J3bqjWVTUGGxHi+/G11FjvMLzEKUs/SsO6Yco6Tt1jwfe9i1r/ 0sk8jVJ4+FWTGQXu+e80hmVcOK1PmvhtcRNfd5WWNkGXVNTCI66vUoGZ+Sa5O/dV43WP/MT1AmRY mwxj3pH4T+7W64R57n3bPGbq49zDNo/59k75wq3STNE0KDjN0Z98yVbBoArXheZrHNGp8zZ9vJ4+ 3mQzOIvex/M4VjN3hDPjzA7qAjZ1YO+Zrxm22SP8AFt2kBdoG28MOqnQOe3UjiBIhj3cm2iC0fqK 3CQAV6kCmE7WK5jO1vT5BlRmYH02+7l+9+bsHxjsEP6fSQIvpY0F+F+jv/lS+N8oqw+/EP+XSsYK /z8Fpbf8yvrSX5lwH0G12g53wUmo7kRyz0c8RXRoN9CPIQZX+MxnsYsgkL49CBH/gn68Sx8QJILS OWPw1PNclw5gqr6X9Pv4tqFr2vHuZq4n/W5a042dIetGXoShRsC7ggdON/1kG6oIBcZ0v0A/Zm5X kFmaZf+r5/nvSir+n73EWUIbi87/rfx/ev+vG6vz/yTUrp+2Gs1uq7F/eHrSPakdN9rdXXw5Qc/a +K1+dPauAZRFbGCQKf+dcEqtJjMvlMXsoEvOzVPUOHm3Onh/b6LzP3PJu5Q2Fp1/SzfunH+jZK2+ /3kSEtKpVF5hFAZa7lUvDIWcvDaPmW1XKo3g0sNAjv7q83YF890wRuc/FFMNyjJkL5ltyN4DPsoe +RW3E8npFd+Y74c2w1hcvUPuVQeB+UQLCxjihklVzALBFOIeSzc9ugBydsOrqUgmjgIIEuJMN1VS hNmOmM2zAvrJ6ytDtaIVrWhFK1rRila0ohWtaEUr+oHpf3D+RYAAUAAA --_002_BA7C853FEE430847B9C35FFCC6E5B2A52A32CBEDbraunelibeamseu_--