k-Wave User Forum » Topic: Creating the HDF5 File in C++

k-Wave User Forum » Topic: Creating the HDF5 File in C++ http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c Support for the k-Wave MATLAB toolbox en-US Tue, 12 May 2026 21:32:00 +0000 http://bbpress.org/?v=1.0.2 <![CDATA[Search]]> q http://www.k-wave.org/forum/search.php GavinMacaulay on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c/page/2#post-8142 Wed, 28 Apr 2021 04:09:03 +0000 GavinMacaulay 8142@http://www.k-wave.org/forum/ Thanks, that link works for me (the original works now too, strangely...) Filip Vaverka on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c/page/2#post-7944 Thu, 26 Nov 2020 11:56:22 +0000 Filip Vaverka 7944@http://www.k-wave.org/forum/ The link appears to work for me, but its not directly clickable. Does this one <a href="https://www.fit.vut.cz/~ivaverka/k-wave-gen-lite.zip">https://www.fit.vut.cz/~ivaverka/k-wave-gen-lite.zip</a> help? GavinMacaulay on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c/page/2#post-7919 Wed, 11 Nov 2020 00:25:18 +0000 GavinMacaulay 7919@http://www.k-wave.org/forum/ Hi Filip and Jiri, The link in the post immediately above isn't working. Do you have an updated link to your python scripts? Thanks Gavin Filip Vaverka on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-7543 Wed, 03 Jun 2020 15:16:53 +0000 Filip Vaverka 7543@http://www.k-wave.org/forum/ Hi djps, The issue with Python/numpy is that it defaults to C ordered arrays, whereas MATLAB is F ordered. It seems that there are two options to deal with this: 1) Reorder the data that are stored in the file (this leads to transpositions as "array.transpose(2, 1, 0)). 2) Flip whole simulation instead (this is done by swapping x/z axis and keeping the data in C order). Either way, we prepared new version (its complete rewrite - hence the delay) of the Python script with some examples, which should work with latest k-Wave release. The script should support both "Flip" and "Reorder" (by default and bit more tested) modes. The scripts require Python 3, h5py, numpy, scipy and matplotlib for "driver example" and can be downloaded from <a>https://www.fit.vut.cz/~ivaverka/k-wave-gen-lite.zip</a>. Once again this is not thoroughly tested and therefore provided "as is". Any comments and questions can be posted here or you can email me directly. djps on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-7496 Wed, 13 May 2020 01:09:57 +0000 djps 7496@http://www.k-wave.org/forum/ I am trying to run some diagnostic simulations without matlab, and can't write hdf5 files in octave, so have to do pretty much everything in python. It makes cross-checking between things generated pretty difficult! For clarity: from the files provided, where is it necessary to remove the transpositions - in the <code>example_generate_input.py</code> or <code>kwave_input_file.py</code>? If this is removed, shouldn't arrays be defined as <code>np.zeros((Nz,Ny,Nx))</code> though? Thanks, david Patrick on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6984 Sat, 27 Jul 2019 05:39:03 +0000 Patrick 6984@http://www.k-wave.org/forum/ Dr. Jaros, Fantastic, that's great news. I look forward to the next release. I will read through both generated files and see where the differences were and try to track down my error. Thanks for your help! PC Jiri Jaros on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6981 Wed, 24 Jul 2019 13:01:24 +0000 Jiri Jaros 6981@http://www.k-wave.org/forum/ Hi pcook, And are there any plans for full support of this Python script? - Yes, I've reduced the complexity of the input file so that the derivative operators, pml and shift variables are generated inside the c++ code. This will be in the next release. The more complicated question is about wrong results. There must be nonsence in the data. NaN means some variables are ill-defined leading to divisions by zero, etc. I'd suspect some PML setting or derivative operators ddx_shift.... I'd recommend cross-checking the python input file with the Matlab input file. Simply generate those two files, then open matlab and read dataset by dataset using h5read and compare the contents. Best Jiri Patrick on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6976 Sun, 14 Jul 2019 05:31:09 +0000 Patrick 6976@http://www.k-wave.org/forum/ Hi, I have attempted to use your Python script to generate the input file, however, running the exact same configuration in Matlab gives drastically different results. (p_max nearly 100% different). Moreover, with identical timesteps, the input file generated by the Python script will produce NaN pressure values, while the Matlab code will run without problem. What could be the cause of this? And are there any plans for full support of this Python script? I know many of us would highly appreciate it. Thanks, PC Filip Vaverka on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6794 Wed, 20 Mar 2019 15:31:32 +0000 Filip Vaverka 6794@http://www.k-wave.org/forum/ Yes, you are right it is missing from that method. Also "self.input_data_sets['simulation_flags']['ux_source_flag'].value" (and other "X_source_flag") are wrongly set to length of the signal series instead to index of number of timesteps before the source series begins to be introduced in the domain. Tristan2 on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6759 Wed, 20 Feb 2019 16:17:09 +0000 Tristan2 6759@http://www.k-wave.org/forum/ Hi Jiri Jaros, thank you very much for the python input generator. I get some errors if i try to use the set_u_source method. I'm not sure if i completely understand what the script is doing but is it possible that something like "self.input_data_sets['velocity_source_terms']['u_source_many'].value = 1 if ux_source_series.shape[0] > 1 or ux_source_series.shape[0] > 1 or ux_source_series.shape[0] > 1 else 0" is missing from the set_u_source method? Best regards, Tristan lbruju on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6445 Wed, 02 May 2018 16:20:32 +0000 lbruju 6445@http://www.k-wave.org/forum/ Hi Jiri Jaros, Thanks a lot for your answer, I removed all the transpositions and swapped x and z axis, and my simulation seems to be correct now. Best regards, lbruju Jiri Jaros on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6442 Mon, 30 Apr 2018 14:14:49 +0000 Jiri Jaros 6442@http://www.k-wave.org/forum/ Hi lbruju, HDF5 is transparent to C++ (row-major ordering) / Matlab (column-major ordering) and respects the convention of the host language. Consequently, when creating a matrix in Matlab/Fortran, you provide the dimensions in the Matlab ordering (Nx, Ny, Nz). However, when doing the same thing in C++ you work with (Nz, Ny, Nz). HDF5 is responsible for data rotation when reading from different languages. The k-Wave manual is meant for Matlab users and thus respects the Matlab conventions. I'd recommend to use h5ls and h5dump commandline utilities to see the organization. Now the question is what about Python. And the answer is, it depends :) If you use NumPy, you can choose whether you want to work with C-like of Fortran-like ordering by the parameter order. When using the C-ordering, which is default, you provide dimension size in this order [Nz, Ny, Nx]. We have now realized that there are unnecessary transpositions in the script I shared. Simply said, the X-Z are swapped and the data rotated, which is actually the same thing as before. So this is not necessary (you don't need to transpose data, rotate indices, etc.) Best Jiri lbruju on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6432 Fri, 20 Apr 2018 14:24:30 +0000 lbruju 6432@http://www.k-wave.org/forum/ Hi, Thanks for the files. I am lost regarding indexing and shape. It's written in the k-wave_user_manual in the chapter "Format of the C++ HDF5 Files" that the dimension sizes are given following the MATLAB indexing convention (which is row-major ordering) and for creating files outside MATLAB, dataset dimensions should be given as <code>(Nz, Ny, Nx)</code> . But in the python script you shared, the shape are given and for example, for <code>ddx_k_shift_neg</code> the shape <code>(1,1,Nz)</code> is indicated. I don't understand why the shape is not <code>(Nz,1,1)</code> ? After executing kspaceFirstOrder3D-OMP.exe, the resulting acoustic pressure field is in a matrix of size <code>(np.size(p_source_input), Nx*Ny*Nz)</code> . I want to reshape the acoustic pressure field to have in <code>final_press[t][x,y,z]</code> the resulting pressure at time t, at point <code>(x,y,z)</code> . For each <code>time</code>, do I need to reshape among <code>(Nz,Ny,Nx)</code> (i.e <code>p.reshape(pressure[time,0:Nx*Ny*Nz],(Nz,Ny,Nx))</code>) or among <code>(Nx,Ny,Nz)</code> (i.e <code>p.reshape(pressure[time,0:Nx*Ny*Nz],(Nz,Ny,Nx))</code>)? Thanks a lot, lbruju Jiri Jaros on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6371 Fri, 23 Mar 2018 09:01:46 +0000 Jiri Jaros 6371@http://www.k-wave.org/forum/ Hi Ibruju, My PhD student, Filip Vaverka, has created a prototype of python generator of k-wave input files. You can download it here. <a href="http://www.fit.vutbr.cz/~jarosjir/download/k-wave-generator-example.zip" rel="nofollow">http://www.fit.vutbr.cz/~jarosjir/download/k-wave-generator-example.zip</a> Jiri Bradley Treeby on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6364 Wed, 21 Mar 2018 21:49:38 +0000 Bradley Treeby 6364@http://www.k-wave.org/forum/ Hi lbruju, The transducer source allows a single time series to be used for multiple grid points within the simulation. It always uses a velocity source in the x-direction (<code>u_source_index</code> is for a velocity source, and <code>p_source_index</code> is for a mass or pressure source). For details, see the Ultrasound Imaging examples. I wouldn't recommend using this, as we will likely implement this functionality in a slightly different (and more general) way in a future release. So in short, define your source using <code>p_source_index</code> and <code>p_source_input</code>. Brad. lbruju on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6357 Tue, 20 Mar 2018 13:16:34 +0000 lbruju 6357@http://www.k-wave.org/forum/ Hi again, I have another question. I don't get the difference between defining a transducer and defining a source mask and a time varying pressure? In my study case, I want to mimic the evolution of pressure with a focused transducer. Do I need to define a transducer (with <code>transducer_source_flag = 1</code>) or to define a source (with 'p_source_index' and 'p_source_input')? If I need to define a transducer, can you confirm that "u_source_index" is equivalent to "p_source_index" and "transducer_source_input" is equivalent to "p_source_input". But what is the role of "delay_mask"? Is that for defining the shape of the transducer? Thanks a lot, lbruju lbruju on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6356 Tue, 20 Mar 2018 11:43:55 +0000 lbruju 6356@http://www.k-wave.org/forum/ Hi, Thanks a lot for your answer. I think I manage to write a script which generates a hdf5 file. But I obtain surprising results. In 'p_source_index' are the index of half of a sphere. In 'p_source_input', I defined a list 'source' by: 'source=amp*amp*np.sin(2*np.pi*freq*t_array)' with 'amp=10.0 #Pa'. I defined all other dataset. At the end, I obtained a pressure field around 1E8, that seems to be very high... Do you know what could be the origin of the error? Thanks lbruju Jiri Jaros on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6355 Tue, 20 Mar 2018 09:40:53 +0000 Jiri Jaros 6355@http://www.k-wave.org/forum/ Hi Jakob, Ibruju, I will upload a sample matlab script later today. But first, let's answer your questions. (1) Yes, it is necessary to add HDF5 attributes to all datasets. The reason is to double check the data has been generated correctly. (2) ddx_k_shift_neg_r - these are the spatial shifts in the x, y and z direction for staggered grids. The _r suffix says, the size of the domain is reduced in this direction. Since the pressure/velocity matrices are real, the FFTs are symmetric, and we can only store the first half. Consequently, the ddx_k_shift_neg is truncated from N to N/2 + 1 complex numbers. The reason why this is generated by Matlab not C++ is that we wanted some freedom in the staggered grid definition. - the code to generate these values will be part of the script I'm gonna upload. (3) The C++ code only supports an index sensor mask. You will have to identify the grid points on the arc manually and fill the index sensor mask by yourself. (4) There's no diffusion code in C++ at the moment. We thought it was so fast it wasn't worth to reimplemented it in C++ at that moment. Best Jiri JakobD on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6343 Wed, 14 Mar 2018 21:07:59 +0000 JakobD 6343@http://www.k-wave.org/forum/ Hi everyone I don't have matlab, so I also want to generate a hdf5 file with python (for after execute kSpaceFirstOrder3D-OMP). But I have problems when generating this file. (1) Is it necessary to add one by one all the attributes for each element? It seems to be quite long and not very optimized... (2) I found a hdf5 generated with matlab (on the forum) and I don't understand what représente the variables "ddx_k_shift_neg_r" (and same with neg and for the other axis y and z) (3) With the matlab code, lots of functions permit to create different shape of source or sensor (makeArc, makeBowl...). How it could be possible to mimic these functions in hdf5 file? (4) and last question, did I well understand that there is no equivalent function of kWaveDiffusion in C++? Finally, I will be very grateful if someone can share an example of python script generating a hdf5 file ! Jakob lbruju on "Creating the HDF5 File in C++" http://www.k-wave.org/forum/topic/creating-the-hdf5-file-in-c#post-6341 Tue, 13 Mar 2018 11:03:41 +0000 lbruju 6341@http://www.k-wave.org/forum/ Hi Jiri Jaros, I am also working on a Python code to generate input files for k-wave. As you propose, could you please post your Python code which generates the input files for inspiration? Thanks a lot, lbruju