http://www.k-wave.org/forum/topic/ultrasound-simulation-on-hpc Support for the k-Wave MATLAB toolbox en-US Tue, 12 May 2026 23:02:09 +0000 http://bbpress.org/?v=1.0.2 q http://www.k-wave.org/forum/search.php http://www.k-wave.org/forum/topic/ultrasound-simulation-on-hpc#post-7402 Fri, 17 Apr 2020 15:18:21 +0000 Jiri Jaros 7402@http://www.k-wave.org/forum/ <p>Hi Cyrill,<br /> Unfortunately, you'll have to create an independent file for every simulation. The OMP code has no logic for the steering angle, this is left to Matlab.</p> <p>Best<br /> Jiri </p> http://www.k-wave.org/forum/topic/ultrasound-simulation-on-hpc#post-7399 Wed, 15 Apr 2020 13:57:07 +0000 Cyrill 7399@http://www.k-wave.org/forum/ <p>Thanks Jiri for the detailed feedback, this makes for a very simple solution while maintaining a lot of node/thread allocation! This however brings me to a new challenge. As far as defining the input .h5 files for kspaceFirstOrder3D-OMP, the only variation between jobs is the transducer.steering_angle. Is it possible to parsed this parameter as and input flag or will I have to create individual (potentially very large) files for each angle? Or am I just approaching this problem wrong entirely?</p> <p>Best,</p> <p>Cyrill </p> http://www.k-wave.org/forum/topic/ultrasound-simulation-on-hpc#post-7394 Tue, 14 Apr 2020 11:38:07 +0000 Jiri Jaros 7394@http://www.k-wave.org/forum/ <p>Hi Cyrill,<br /> Take a look at SLURM job arrays.<br /> <a href="https://slurm.schedmd.com/job_array.html" rel="nofollow">https://slurm.schedmd.com/job_array.html</a></p> <p>You can specify there, I've got 512 scan lines (simulations), each needs one node and 12 threads. The job array system will automatically distribute the workload over all free nodes. </p> <p>Some scaling issues to consider (need some performance measurement, though):</p> <p>- Download k-Wave 1.3, it has several performance improvements.</p> <p>- You can specify the number of threads, a single simulation uses, by -t (e.g. -t 1 for a single thread), or the OMP_NUM_THREADS environmental variable.</p> <p>- Its an open question whether it is better to run multiple simulations on a single node or not. It depends on the simulation size, CPU specs, etc. Sometimes, it might be useful to use 4 threads per a simulation and run 3 simulations on a single node (considering you have a 12 core CPU). For small simulations, the best could be to run as many single-threaded simulations as there are cores, for big ones the run only a single simulation per CPU (NUMA domain - but make sure you use proper thread binding) or the whole node. Here you need to run a few experiments. </p> <p>Best<br /> Jiri </p> http://www.k-wave.org/forum/topic/ultrasound-simulation-on-hpc#post-7384 Sat, 11 Apr 2020 12:51:47 +0000 Cyrill 7384@http://www.k-wave.org/forum/ <p>We have been successfully running sound-field simulations on an HPC using kspaceFirstOrder3D-OMP for some time now. Currently I am trying to implement an ultrasound imaging simulation and am looking for advice on how to best parallelize the approach. Based on my experience using the code so far, I noticed diminishing returns on parallelization across nodes and have thus maximized thread counts while restricting to a single node. For the imaging purposes I would have to now re-run the code for each scan-line i.e.:</p> <p>for (( i=1; i&lt;=$NLines; i++ ))<br /> mpirun -np $SLURM_NTASKS kspaceFirstOrder3D-OMP ….</p> <p>This however would sequentially do one scan-line after the other not being able to capitalize on multiple nodes. I could alternatively create multiple jobs but this seems like a poor approach.<br /> Question: How would I best go about parallelizing lines across nodes while parallelizing simulations across threads on a single node?</p> <p>Is adapting the source code to include and manage the MPI allocations my only option?<br /> Any guidance is appreciated!</p> <p>Cheers, Cyrill </p>