k-Wave User Forum » User Favorites: bastg

k-Wave User Forum » User Favorites: bastg bastg Support for the k-Wave MATLAB toolbox en-US Tue, 12 May 2026 23:29:57 +0000 http://bbpress.org/?v=1.0.2 <![CDATA[Search]]> q http://www.k-wave.org/forum/search.php bastg on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-7544 Wed, 03 Jun 2020 17:32:21 +0000 bastg 7544@http://www.k-wave.org/forum/ Hi Jiri -- below is the output of the "--version -g 0" command line. Let me know if this gives you some insight... Thanks for your help! >> !/autofs/space/guerin/USneuromod/KWAVE_1.2.1/src/kspaceFirstOrder-CUDA/kspaceFirstOrder-CUDA --version -i kwave3D_N104_650KHZ_0p75MM_PAR.h5 -o kwave3D_N104_650KHZ_0p75MM_SOL.h5 -g 0 ; ┌───────────────────────────────────────────────────────────────┐ │ kspaceFirstOrder-CUDA v1.3 │ ├───────────────────────────────────────────────────────────────┤ │ Selected GPU device id: Failed │ ├───────────────────────────────────────────────────────────────┤ │ Build information │ ├───────────────────────────────────────────────────────────────┤ │ Build number: kspaceFirstOrder v3.6 │ │ Build date: Jun 3 2020 │ │ Build time: 12:25:49 │ │ Git hash: 468dc31c2842a7df5f2a07c3a13c16c9b0b2b770 │ ├───────────────────────────────────────────────────────────────┤ │ Operating system: Linux x64 │ │ Compiler name: GNU C++ 7.3.1 20180303 (Red │ │ Processor name: Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz │ │ Instruction set: Intel AVX 2 │ ├───────────────────────────────────────────────────────────────┤ │ CUDA runtime: 10.1 │ │ CUDA driver: 10.1 │ │ CUDA code arch: 0.0 │ ├───────────────────────────────────────────────────────────────┤ │ CUDA device id: 0 │ │ CUDA device name: N/A │ │ CUDA capability: 0.0 │ ├───────────────────────────────────────────────────────────────┤ │ Contact email: <a href="mailto:jarosjir@fit.vutbr.cz">jarosjir@fit.vutbr.cz</a> │ │ Contact web: <a href="http://www.k-wave.org" rel="nofollow">http://www.k-wave.org</a> │ ├───────────────────────────────────────────────────────────────┤ │ Copyright (C) 2011-2020 SC@FIT Research Group, BUT, Czech Rep │ └───────────────────────────────────────────────────────────────┘ ┌───────────────────────────────────────────────────────────────┐ │ !!! K-Wave experienced a fatal error !!! │ ├───────────────────────────────────────────────────────────────┤ │ Error: CUDA device id 0 is busy or unavailable. │ ├───────────────────────────────────────────────────────────────┤ │ Execution terminated │ └───────────────────────────────────────────────────────────────┘ Segmentation fault (core dumped) /autofs/space/guerin/USneuromod/KWAVE_1.2.1/src/kspaceFirstOrder-CUDA/kspaceFirstOrder-CUDA --version -i kwave3D_N104_650KHZ_0p75MM_PAR.h5 -o kwave3D_N104_650KHZ_0p75MM_SOL.h5 -g 0 ;: Segmentation fault >> Jiri Jaros on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-7521 Thu, 21 May 2020 22:08:56 +0000 Jiri Jaros 7521@http://www.k-wave.org/forum/ Hi bastg, the driver and compile parameters look good to me. Even, nvidia-smi says the GPU is in default mode thus anyone can use it. could you try to run the code with -g 0 parameter. This will explicitly select GPU no 0. Could you also try to run ./kspaceFirstOrder-CUDA --version -g 0 Best Jiri bastg on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-7499 Fri, 15 May 2020 16:18:35 +0000 bastg 7499@http://www.k-wave.org/forum/ Hi all -- a follow up on the previous messages of this thread. I have the same error, i.e. "All CUDA-capable devices are busy or unavailable". I have tried to follow the recommendations given in various threads of this forum, but to no avail. My GPU is a Tesla P100, which is a Pascal architecture with compute capability 60 ("sm_60" or "compute_60"), and therefore requires CUDA 8.0 or newer. I am using the CUDA 10.1, which is almost the latest (10.2 is the latest). This is the CUDA_ARCH variable in my Makefile (entire Makefile is shown at the end of this post, which I have modified a little from the original K-WAVE Makefile): CUDA_ARCH = --generate-code arch=compute_30,code=sm_30 \ --generate-code arch=compute_32,code=sm_32 \ --generate-code arch=compute_35,code=sm_35 \ --generate-code arch=compute_37,code=sm_37 \ --generate-code arch=compute_50,code=sm_50 \ --generate-code arch=compute_52,code=sm_52 \ --generate-code arch=compute_53,code=sm_53 \ --generate-code arch=compute_60,code=sm_60 \ --generate-code arch=compute_61,code=sm_61 \ --generate-code arch=compute_62,code=sm_62 \ --generate-code arch=compute_70,code=sm_70 \ --generate-code arch=compute_72,code=sm_72 \ --generate-code arch=compute_75,code=sm_75 I am not sure what I am missing... Thanks for your input! Bastien ************************* ERROR MESSAGE ************************* deepbrain:guerin[152] ../../KWAVE_1.2.1/src/kspaceFirstOrder-CUDA/kspaceFirstOrder-CUDA -i kwave3D_N104_650KHZ_0p75MM_PAR.h5 -o kwave3D_N104_650KHZ_0p75MM_SOL.h5 ┌───────────────────────────────────────────────────────────────┐ │ kspaceFirstOrder-CUDA v1.3 │ ├───────────────────────────────────────────────────────────────┤ │ Reading simulation configuration: Done │ │ Selected GPU device id: Failed │ └───────────────────────────────────────────────────────────────┘ ┌───────────────────────────────────────────────────────────────┐ │ !!! K-Wave experienced a fatal error !!! │ ├───────────────────────────────────────────────────────────────┤ │ Error: All CUDA-capable devices are busy or unavailable. │ ├───────────────────────────────────────────────────────────────┤ │ Execution terminated │ └───────────────────────────────────────────────────────────────┘ deepbrain:guerin[153] ***************************************************************** ************************* OUTPUT OF NVIDIA-SMI ************************* Fri May 15 10:57:31 2020 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 418.67 Driver Version: 418.67 CUDA Version: 10.1 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 Tesla P100-PCIE... Off | 00000000:04:00.0 Off | 0 | | N/A 33C P0 26W / 250W | 0MiB / 16280MiB | 0% Default | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+ ***************************************************************** ************************* MAKEFILE ************************* # Select compiler # GNU is default due to Intel 2018's compatibility issues with Ubuntu 18.04 COMPILER = GNU #COMPILER = Intel # SEMI static lining is default since it is expected the binary will run on the # same system. # Everything will be linked statically, may not work on all GPUs #LINKING = STATIC # Everything will be linked dynamically #LINKING = DYNAMIC # Everything but CUDA will be linked statically LINKING = SEMI # Set up paths: If using modules, the paths are set up automatically, # otherwise, set paths manually CUDA_DIR = /usr/pubsw/packages/CUDA/10.1 HDF5_DIR = /autofs/space/guerin/USneuromod/KWAVE_1.2.1/src/hdf5-1.12.0/hdf5 ZLIB_DIR = /autofs/space/guerin/USneuromod/KWAVE_1.2.1/src/zlib-1.2.11 SZIP_DIR = /autofs/space/guerin/USneuromod/KWAVE_1.2.1/src/szip-2.1.1 # Select CPU architecture (what instruction set to be used). # The native architecture will compile and optimize the code for the underlying # processor. CPU_ARCH = native #CPU_ARCH = AVX #CPU_ARCH = AVX2 #CPU_ARCH = AVX512 ############################### Common flags ################################### # Git hash of release 1.3 GIT_HASH = -D__KWAVE_GIT_HASH__=\"468dc31c2842a7df5f2a07c3a13c16c9b0b2b770\" # Replace tabs by spaces .RECIPEPREFIX += # What CUDA GPU architectures to include in the binary CUDA_ARCH = --generate-code arch=compute_30,code=sm_30 \ --generate-code arch=compute_32,code=sm_32 \ --generate-code arch=compute_35,code=sm_35 \ --generate-code arch=compute_37,code=sm_37 \ --generate-code arch=compute_50,code=sm_50 \ --generate-code arch=compute_52,code=sm_52 \ --generate-code arch=compute_53,code=sm_53 \ --generate-code arch=compute_60,code=sm_60 \ --generate-code arch=compute_61,code=sm_61 \ --generate-code arch=compute_62,code=sm_62 \ --generate-code arch=compute_70,code=sm_70 \ --generate-code arch=compute_72,code=sm_72 \ --generate-code arch=compute_75,code=sm_75 # What libraries to link and how ifeq ($(LINKING), STATIC) LDLIBS = $(HDF5_DIR)/lib/libhdf5_hl.a \ $(HDF5_DIR)/lib/libhdf5.a \ $(CUDA_DIR)/lib64/libcufft_static.a \ $(CUDA_DIR)/lib64/libculibos.a \ $(CUDA_DIR)/lib64/libcudart_static.a \ $(ZLIB_DIR)/lib/libz.a \ $(SZIP_DIR)/lib/libsz.a \ -ldl else ifeq ($(LINKING), DYNAMIC) LDLIBS = -lhdf5 -lhdf5_hl -lz -lcufft else ifeq ($(LINKING), SEMI) LDLIBS = $(HDF5_DIR)/lib/libhdf5_hl.a \ $(HDF5_DIR)/lib/libhdf5.a \ $(ZLIB_DIR)/lib/libz.a \ $(SZIP_DIR)/lib/libsz.a \ -lcufft \ -ldl endif ############################## NVCC + GNU g++ ################################## ifeq ($(COMPILER), GNU) # C++ compiler for CUDA CXX = /usr/pubsw/packages/CUDA/10.0/bin/nvcc # C++ standard CPP_STD = -std=c++11 # Enable OpenMP OPENMP = -fopenmp # Set CPU architecture # Sandy Bridge, Ivy Bridge ifeq ($(CPU_ARCH), AVX) CPU_FLAGS = -m64 -mavx # Haswell, Broadwell else ifeq ($(CPU_ARCH), AVX2) CPU_FLAGS = -m64 -mavx2 # Skylake-X, Ice Lake, Cannon Lake else ifeq ($(CPU_ARCH), AVX512) CPU_FLAGS = -m64 -mavx512f # Maximum performance for this CPU else CPU_FLAGS = -m64 -march=native -mtune=native endif # Use maximum optimization CPU_OPT = -O3 -ffast-math -fassociative-math # Use maximum optimization GPU_OPT = -O3 # CPU Debug flags CPU_DEBUG = # Debug flags GPU_DEBUG = # Profile flags PROFILE = # C++ warning flags WARNING = -Wall # Add include directories INCLUDES = -I$(HDF5_DIR)/include -I. # Add library directories LIB_PATHS = -L$(HDF5_DIR)/lib -L$(CUDA_DIR)/lib64 # Set compiler flags and header files directories CXXFLAGS = -Xcompiler="$(CPU_FLAGS) $(CPU_OPT) $(OPENMP) \ $(CPU_DEBUG) $(PROFILE) $(WARNING)"\ $(GPU_OPT) $(CPP_STD) $(GPU_DEBUG) \ $(GIT_HASH) \ $(INCLUDES) \ --device-c --restrict # Set linker flags and library files directories LDFLAGS = -Xcompiler="$(OPENMP)" \ -Xlinker="-rpath,$(HDF5_DIR)/lib:$(CUDA_DIR)/lib64" \ -std=c++11 \ $(LIB_PATHS) endif ############################ NVCC + Intel icpc ################################# ifeq ($(COMPILER), Intel) # C++ compiler for CUDA CXX = /usr/pubsw/packages/CUDA/10.0/bin/nvcc # C++ standard CPP_STD = -std=c++11 # Enable OpenMP OPENMP = -qopenmp # Set CPU architecture # Sandy Bridge, Ivy Bridge ifeq ($(CPU_ARCH), AVX) CPU_FLAGS = -m64 -xAVX # Haswell, Broadwell else ifeq ($(CPU_ARCH), AVX2) CPU_FLAGS = -m64 -xCORE-AVX2 # Skylake-X, Ice Lake, Cannon Lake else ifeq ($(CPU_ARCH), AVX512) CPU_FLAGS = -m64 -xCORE-AVX512 # Maximum performance for this CPU else CPU_FLAGS = -m64 -xhost endif # Use maximum optimization CPU_OPT = -Ofast # Use maximum optimization GPU_OPT = -O3 # CPU Debug flags CPU_DEBUG = # Debug flags GPU_DEBUG = # Profile flags PROFILE = # C++ warning flags WARNING = -Wall # Add include directories INCLUDES = -I$(HDF5_DIR)/include -I. # Add library directories LIB_PATHS = -L$(HDF5_DIR)/lib -L$(CUDA_DIR)/lib64 # Set compiler flags and header files directories CXXFLAGS = -Xcompiler="$(CPU_FLAGS) $(CPU_OPT) $(OPENMP) \ $(CPU_DEBUG) $(PROFILE) $(WARNING)" \ $(GPU_OPT) $(CPP_STD) $(GPU_DEBUG) \ $(GIT_HASH) \ $(INCLUDES) \ --device-c --restrict -ccbin=icpc # Set linker flags and library files directories ifneq ($(LINKING), DYNAMIC) LDFLAGS = -Xcompiler="$(OPENMP) -static-intel -qopenmp-link=static" else LDFLAGS = -Xcompiler="$(OPENMP)" endif LDFLAGS += -std=c++11 -ccbin=icpc \ -Xlinker="-rpath,$(HDF5_DIR)/lib:$(CUDA_DIR)/lib64" \ $(LIB_PATHS) endif ################################### Build ###################################### # Target binary name TARGET = kspaceFirstOrder-CUDA # Units to be compiled DEPENDENCIES = main.o \ Containers/MatrixContainer.o \ Containers/CudaMatrixContainer.o \ Containers/OutputStreamContainer.o \ Hdf5/Hdf5File.o \ Hdf5/Hdf5FileHeader.o \ KSpaceSolver/KSpaceFirstOrderSolver.o \ KSpaceSolver/SolverCudaKernels.o \ Logger/Logger.o \ MatrixClasses/BaseFloatMatrix.o \ MatrixClasses/BaseIndexMatrix.o \ MatrixClasses/CufftComplexMatrix.o \ MatrixClasses/ComplexMatrix.o \ MatrixClasses/IndexMatrix.o \ MatrixClasses/RealMatrix.o \ MatrixClasses/TransposeCudaKernels.o \ OutputStreams/BaseOutputStream.o \ OutputStreams/IndexOutputStream.o \ OutputStreams/CuboidOutputStream.o \ OutputStreams/WholeDomainOutputStream.o \ OutputStreams/OutputStreamsCudaKernels.o \ Parameters/CommandLineParameters.o \ Parameters/Parameters.o \ Parameters/CudaParameters.o \ Parameters/CudaDeviceConstants.o # Build target all: $(TARGET) # Link target $(TARGET): $(DEPENDENCIES) $(CXX) $(LDFLAGS) $(DEPENDENCIES) $(LDLIBS) -o $@ # Compile CPU units %.o: %.cpp $(CXX) $(CXXFLAGS) -o $@ -c $< # Compile CUDA units %.o: %.cu $(CXX) $(CXXFLAGS) $(CUDA_ARCH) -o $@ -c $< # Clean repository .PHONY: clean clean: rm -f $(DEPENDENCIES) $(TARGET) (END) Jiri Jaros on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-7209 Mon, 10 Feb 2020 16:39:37 +0000 Jiri Jaros 7209@http://www.k-wave.org/forum/ I'll do so next week. For now, this is my vcxproj file <pre><code><?xml version="1.0" encoding="utf-8"?> <Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> <ItemGroup Label="ProjectConfigurations"> <ProjectConfiguration Include="Debug|x64"> <Configuration>Debug</Configuration> <Platform>x64</Platform> </ProjectConfiguration> <ProjectConfiguration Include="Release|x64"> <Configuration>Release</Configuration> <Platform>x64</Platform> </ProjectConfiguration> </ItemGroup> <ItemGroup> <ClCompile Include="Containers\MatrixContainer.cpp" /> <ClCompile Include="Containers\OutputStreamContainer.cpp" /> <ClCompile Include="GetoptWin64\Getopt.cpp" /> <ClCompile Include="Hdf5\Hdf5File.cpp" /> <ClCompile Include="Hdf5\Hdf5FileHeader.cpp" /> <ClCompile Include="KSpaceSolver\KSpaceFirstOrderSolver.cpp" /> <ClCompile Include="Logger\Logger.cpp" /> <ClCompile Include="main.cpp" /> <ClCompile Include="MatrixClasses\BaseFloatMatrix.cpp" /> <ClCompile Include="MatrixClasses\BaseIndexMatrix.cpp" /> <ClCompile Include="MatrixClasses\ComplexMatrix.cpp" /> <ClCompile Include="MatrixClasses\CufftComplexMatrix.cpp" /> <ClCompile Include="MatrixClasses\IndexMatrix.cpp" /> <ClCompile Include="MatrixClasses\RealMatrix.cpp" /> <ClCompile Include="OutputStreams\BaseOutputStream.cpp" /> <ClCompile Include="OutputStreams\CuboidOutputStream.cpp" /> <ClCompile Include="OutputStreams\IndexOutputStream.cpp" /> <ClCompile Include="OutputStreams\WholeDomainOutputStream.cpp" /> <ClCompile Include="Parameters\CommandLineParameters.cpp" /> <ClCompile Include="Parameters\CudaParameters.cpp" /> <ClCompile Include="Parameters\Parameters.cpp" /> </ItemGroup> <ItemGroup> <ClInclude Include="Containers\CudaMatrixContainer.cuh" /> <ClInclude Include="Containers\MatrixContainer.h" /> <ClInclude Include="Containers\MatrixRecord.h" /> <ClInclude Include="Containers\OutputStreamContainer.h" /> <ClInclude Include="GetoptWin64\Getopt.h" /> <ClInclude Include="Hdf5\Hdf5File.h" /> <ClInclude Include="Hdf5\Hdf5FileHeader.h" /> <ClInclude Include="KSpaceSolver\KSpaceFirstOrderSolver.h" /> <ClInclude Include="KSpaceSolver\SolverCudaKernels.cuh" /> <ClInclude Include="Logger\ErrorMessages.h" /> <ClInclude Include="Logger\ErrorMessagesLinux.h" /> <ClInclude Include="Logger\ErrorMessagesWindows.h" /> <ClInclude Include="Logger\Logger.h" /> <ClInclude Include="Logger\OutputMessages.h" /> <ClInclude Include="Logger\OutputMessagesLinux.h" /> <ClInclude Include="Logger\OutputMessagesWindows.h" /> <ClInclude Include="MatrixClasses\BaseFloatMatrix.h" /> <ClInclude Include="MatrixClasses\BaseIndexMatrix.h" /> <ClInclude Include="MatrixClasses\BaseMatrix.h" /> <ClInclude Include="MatrixClasses\ComplexMatrix.h" /> <ClInclude Include="MatrixClasses\CufftComplexMatrix.h" /> <ClInclude Include="MatrixClasses\IndexMatrix.h" /> <ClInclude Include="MatrixClasses\RealMatrix.h" /> <ClInclude Include="MatrixClasses\TransposeCudaKernels.cuh" /> <ClInclude Include="OutputStreams\BaseOutputStream.h" /> <ClInclude Include="OutputStreams\CuboidOutputStream.h" /> <ClInclude Include="OutputStreams\IndexOutputStream.h" /> <ClInclude Include="OutputStreams\OutputStreamsCudaKernels.cuh" /> <ClInclude Include="OutputStreams\WholeDomainOutputStream.h" /> <ClInclude Include="Parameters\CommandLineParameters.h" /> <ClInclude Include="Parameters\CudaDeviceConstants.cuh" /> <ClInclude Include="Parameters\CudaParameters.h" /> <ClInclude Include="Parameters\Parameters.h" /> <ClInclude Include="Utils\CudaUtils.cuh" /> <ClInclude Include="Utils\DimensionSizes.h" /> <ClInclude Include="Utils\MatrixNames.h" /> <ClInclude Include="Utils\TimeMeasure.h" /> </ItemGroup> <ItemGroup> <CudaCompile Include="Containers\CudaMatrixContainer.cu" /> <CudaCompile Include="KSpaceSolver\SolverCudaKernels.cu" /> <CudaCompile Include="MatrixClasses\TransposeCudaKernels.cu" /> <CudaCompile Include="OutputStreams\OutputStreamsCudaKernels.cu" /> <CudaCompile Include="Parameters\CudaDeviceConstants.cu" /> </ItemGroup> <PropertyGroup Label="Globals"> <ProjectGuid>{B040477E-5790-4F5D-B010-B7B57EE26FED}</ProjectGuid> <RootNamespace>k_wave_fluid_cuda</RootNamespace> <WindowsTargetPlatformVersion>10.0.17763.0</WindowsTargetPlatformVersion> </PropertyGroup> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" /> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration"> <ConfigurationType>Application</ConfigurationType> <UseDebugLibraries>true</UseDebugLibraries> <CharacterSet>MultiByte</CharacterSet> <PlatformToolset>v141</PlatformToolset> <WholeProgramOptimization>true</WholeProgramOptimization> </PropertyGroup> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration"> <ConfigurationType>Application</ConfigurationType> <UseDebugLibraries>false</UseDebugLibraries> <WholeProgramOptimization>true</WholeProgramOptimization> <CharacterSet>MultiByte</CharacterSet> <PlatformToolset>v141</PlatformToolset> <UseOfMfc> </UseOfMfc> </PropertyGroup> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" /> <ImportGroup Label="ExtensionSettings"> <Import Project="$(VCTargetsPath)\BuildCustomizations\CUDA 10.2.props" /> </ImportGroup> <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|x64'"> <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" /> </ImportGroup> <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|x64'"> <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" /> </ImportGroup> <PropertyGroup Label="UserMacros" /> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'"> <LinkIncremental> </LinkIncremental> <IncludePath>$(VC_IncludePath);$(WindowsSDK_IncludePath);c:\Program Files\HDF_Group\HDF5\1.10.6\include;.;..\Sources</IncludePath> <LibraryPath>$(VC_LibraryPath_x64);$(WindowsSDK_LibraryPath_x64);$(NETFXKitsDir)Lib\um\x64;c:\Program Files\HDF_Group\HDF5\1.10.6\lib</LibraryPath> </PropertyGroup> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'"> <IncludePath>$(VC_IncludePath);$(WindowsSDK_IncludePath);c:\Program Files\HDF_Group\HDF5\1.10.6\include;.;..\Sources</IncludePath> <LibraryPath>$(VC_LibraryPath_x64);$(WindowsSDK_LibraryPath_x64);$(NETFXKitsDir)Lib\um\x64;c:\Program Files\HDF_Group\HDF5\1.10.6\lib</LibraryPath> </PropertyGroup> <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'"> <ClCompile> <WarningLevel>Level3</WarningLevel> <PreprocessorDefinitions>WIN32;WIN64;_DEBUG;_CONSOLE;%(PreprocessorDefinitions);__KWAVE_GIT_HASH__="b6413962eefd51c642c627d5ceff5093c7ad8da4"</PreprocessorDefinitions> <FavorSizeOrSpeed> </FavorSizeOrSpeed> <FloatingPointModel>Precise</FloatingPointModel> <LanguageStandard> </LanguageStandard> <MultiProcessorCompilation>true</MultiProcessorCompilation> <EnableEnhancedInstructionSet> </EnableEnhancedInstructionSet> <OpenMPSupport>false</OpenMPSupport> <CompileAs>Default</CompileAs> <BasicRuntimeChecks>Default</BasicRuntimeChecks> <WholeProgramOptimization>false</WholeProgramOptimization> <OpenMP>GenerateParallelCode</OpenMP> <CCppSupport>Cpp11Support</CCppSupport> </ClCompile> <Link> <GenerateDebugInformation>true</GenerateDebugInformation> <SubSystem>Console</SubSystem> <AdditionalDependencies>cudart_static.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies);libszip.lib;libzlib.lib;libhdf5.lib;libhdf5_hl.lib;cufft.lib</AdditionalDependencies> <AdditionalLibraryDirectories>%(AdditionalLibraryDirectories);$(CudaToolkitLibDir);</AdditionalLibraryDirectories> </Link> <CudaCompile> <TargetMachinePlatform>64</TargetMachinePlatform> <GenerateRelocatableDeviceCode>true</GenerateRelocatableDeviceCode> <PtxAsOptionV>true</PtxAsOptionV> <FastMath>true</FastMath> <CodeGeneration> </CodeGeneration> <AdditionalCompilerOptions> </AdditionalCompilerOptions> </CudaCompile> </ItemDefinitionGroup> <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'"> <ClCompile> <WarningLevel>Level3</WarningLevel> <Optimization>MaxSpeed</Optimization> <FunctionLevelLinking>true</FunctionLevelLinking> <IntrinsicFunctions>true</IntrinsicFunctions> <PreprocessorDefinitions>WIN32;WIN64;NDEBUG;_CONSOLE;%(PreprocessorDefinitions);__KWAVE_GIT_HASH__="5d1e866ae6bd2f83b97f5f98c1081076d2052828"</PreprocessorDefinitions> <MultiProcessorCompilation>true</MultiProcessorCompilation> <EnableEnhancedInstructionSet>AdvancedVectorExtensions</EnableEnhancedInstructionSet> <LanguageStandard>stdcpp14</LanguageStandard> <FloatingPointModel>Fast</FloatingPointModel> <OpenMPSupport>false</OpenMPSupport> <FavorSizeOrSpeed>Speed</FavorSizeOrSpeed> </ClCompile> <Link> <GenerateDebugInformation>false</GenerateDebugInformation> <EnableCOMDATFolding>true</EnableCOMDATFolding> <OptimizeReferences>true</OptimizeReferences> <SubSystem>Console</SubSystem> <AdditionalDependencies>cudart_static.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies);libszip.lib;libzlib.lib;libhdf5.lib;libhdf5_hl.lib;cufft.lib</AdditionalDependencies> <Version>1.3</Version> </Link> <CudaCompile> <TargetMachinePlatform>64</TargetMachinePlatform> <GenerateRelocatableDeviceCode>true</GenerateRelocatableDeviceCode> <CodeGeneration>compute_30,sm_30;compute_32,sm_32;compute_35,sm_35;compute_37,sm_37;compute_50,sm_50;compute_52,sm_52;compute_60,sm_60;compute_61,sm_61;compute_70,sm_70;compute_75,sm_75;</CodeGeneration> <FastMath>true</FastMath> <MaxRegCount /> </CudaCompile> </ItemDefinitionGroup> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" /> <ImportGroup Label="ExtensionTargets"> <Import Project="$(VCTargetsPath)\BuildCustomizations\CUDA 10.2.targets" /> </ImportGroup> </Project></code></pre> mckao on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-7208 Mon, 10 Feb 2020 16:26:30 +0000 mckao 7208@http://www.k-wave.org/forum/ Could K-wave post an updated binary build of the CUDA code? That would be very much appreciated, thank you Joehansenshearer on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-7130 Mon, 18 Nov 2019 15:55:12 +0000 Joehansenshearer 7130@http://www.k-wave.org/forum/ Hi there, I was getting the same errors as qjk. I applied all the suggested fixes and they solved my problems. Unfortunately now I have run into a further errors when I use the MakeFile. nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified make: *** [Makefile:214: KSpaceSolver/SolverCudaKernels.o] Error 1 make: *** Waiting for unfinished jobs.... nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified make: *** [Makefile:220: Parameters/CudaDeviceConstants.o] Error 1 nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified make: *** [Makefile:217: OutputStreams/OutputStreamsCudaKernels.o] Error 1 Does anyone know how to solve this problem? Any help would be appreciated. So far I have; Downloaded source file from <a href="http://www.k-wave.org/download.php" rel="nofollow">http://www.k-wave.org/download.php</a> Edited MakeFile removing --generate-code arch=compute_20 ... --generate-code arch=compute_21... and added --generate-code arch=compute_60,code=sm_60 \ --generate-code arch=compute_61,code=sm_61 \ --generate-code arch=compute_62,code=sm_62 \ --generate-code arch=compute_70,code=sm_70 \ --generate-code arch=compute_72,code=sm_72 \ --generate-code arch=compute_75,code=sm_75 \ as suggested. I have then added appropriate paths for linking. I had an additional problem where in Hdf5File.h where #include <hdf5.h> was unable to find file so i added full path here #include "/usr/include/hdf5/serial/hdf5.h" #include "/usr/include/hdf5/serial/hdf5_hl.h" Now I am running into the errors specified above when running make. I am using a GeForce RTX 2060 Super, CUDA 10.1 and Ubuntu 19.10 Thanks for any help in advance. Regards Joseph Jiri Jaros on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6980 Wed, 24 Jul 2019 12:55:28 +0000 Jiri Jaros 6980@http://www.k-wave.org/forum/ Wow! do you compile it with the NVidia Nsight? It looks like the compiler does not know some CUDA stuff. Best Jiri andrewFrizado on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6974 Fri, 12 Jul 2019 16:49:22 +0000 andrewFrizado 6974@http://www.k-wave.org/forum/ Hey Jiri, I followed your instructions as well as getting the original vcxproj to adhere to the correct paths and library paths etc (including the HDF5 libraries) - only to uncover over 100 new errors in the code. Almost all are related to expecting a semicolon (;) or unaware of the "constexp" term in the code. Any tips to get around this? Here are a few of the 139 errors (same ones repeated multiple times throughout the build): <pre><code>1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2070): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2071): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2070): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2071): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2070): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2071): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2070): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2071): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2070): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2071): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2061): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2062): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2070): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2071): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2280): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2280): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2281): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2281): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2300): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2310): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2327): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2336): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2356): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2367): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2384): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2395): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2280): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2280): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2281): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2281): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2300): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2310): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2327): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2336): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2356): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2367): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2384): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2395): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2280): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2280): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2281): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2281): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2300): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2310): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2327): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2336): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2356): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2367): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2384): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2395): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2280): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2280): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2281): warning : expression has no effect 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2281): error : expected a ";" 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2300): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2310): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2327): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2336): error : identifier "outPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2356): error : identifier "inPad" is undefined 1>C:/Users/Andrew/Documents/Test/k-wave-fluid-cuda/KSpaceSolver/SolverCUDAKernels.cu(2367): error : identifier "outPad" is undefined</code></pre> Thanks for the help! Andrew Jiri Jaros on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6955 Tue, 09 Jul 2019 11:11:23 +0000 Jiri Jaros 6955@http://www.k-wave.org/forum/ Hi andrewFrizado, Windows is always a bit tricky and it takes me a day to figure out how to compile every new version. I assume you've installed HDF5 library (any version newer than 1.8.10 is fine) and you know where the .dll and .h files are. Here is a dump from my *.vcxproj. I would recommend to start a new CUDA project, add all source codes in the project and then set up the libraries, folders, etc. <pre><code><?xml version="1.0" encoding="utf-8"?> <Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> <ItemGroup Label="ProjectConfigurations"> <ProjectConfiguration Include="Debug|x64"> <Configuration>Debug</Configuration> <Platform>x64</Platform> </ProjectConfiguration> <ProjectConfiguration Include="Release|x64"> <Configuration>Release</Configuration> <Platform>x64</Platform> </ProjectConfiguration> </ItemGroup> <ItemGroup> <ClCompile Include="..\Sources\Containers\MatrixContainer.cpp" /> <ClCompile Include="..\Sources\Containers\MatrixRecord.cpp" /> <ClCompile Include="..\Sources\Containers\OutputStreamContainer.cpp" /> <ClCompile Include="..\Sources\GetoptWin64\Getopt.cpp" /> <ClCompile Include="..\Sources\Hdf5\Hdf5File.cpp" /> <ClCompile Include="..\Sources\Hdf5\Hdf5FileHeader.cpp" /> <ClCompile Include="..\Sources\KSpaceSolver\KSpaceFirstOrderSolver.cpp" /> <ClCompile Include="..\Sources\Logger\Logger.cpp" /> <ClCompile Include="..\Sources\main.cpp" /> <ClCompile Include="..\Sources\MatrixClasses\BaseFloatMatrix.cpp" /> <ClCompile Include="..\Sources\MatrixClasses\BaseIndexMatrix.cpp" /> <ClCompile Include="..\Sources\MatrixClasses\ComplexMatrix.cpp" /> <ClCompile Include="..\Sources\MatrixClasses\CufftComplexMatrix.cpp" /> <ClCompile Include="..\Sources\MatrixClasses\IndexMatrix.cpp" /> <ClCompile Include="..\Sources\MatrixClasses\RealMatrix.cpp" /> <ClCompile Include="..\Sources\OutputStreams\BaseOutputStream.cpp" /> <ClCompile Include="..\Sources\OutputStreams\CuboidOutputStream.cpp" /> <ClCompile Include="..\Sources\OutputStreams\IndexOutputStream.cpp" /> <ClCompile Include="..\Sources\OutputStreams\WholeDomainOutputStream.cpp" /> <ClCompile Include="..\Sources\Parameters\CommandLineParameters.cpp" /> <ClCompile Include="..\Sources\Parameters\CudaParameters.cpp" /> <ClCompile Include="..\Sources\Parameters\Parameters.cpp" /> </ItemGroup> <ItemGroup> <ClInclude Include="..\Sources\Containers\MatrixContainer.h" /> <ClInclude Include="..\Sources\Containers\MatrixRecord.h" /> <ClInclude Include="..\Sources\Containers\OutputStreamContainer.h" /> <ClInclude Include="..\Sources\GetoptWin64\Getopt.h" /> <ClInclude Include="..\Sources\Hdf5\Hdf5File.h" /> <ClInclude Include="..\Sources\Hdf5\Hdf5FileHeader.h" /> <ClInclude Include="..\Sources\KSpaceSolver\KSpaceFirstOrderSolver.h" /> <ClInclude Include="..\Sources\KSpaceSolver\SolverCudaKernels.cuh" /> <ClInclude Include="..\Sources\Logger\ErrorMessages.h" /> <ClInclude Include="..\Sources\Logger\ErrorMessagesLinux.h" /> <ClInclude Include="..\Sources\Logger\ErrorMessagesWindows.h" /> <ClInclude Include="..\Sources\Logger\Logger.h" /> <ClInclude Include="..\Sources\Logger\OutputMessages.h" /> <ClInclude Include="..\Sources\Logger\OutputMessagesLinux.h" /> <ClInclude Include="..\Sources\Logger\OutputMessagesWindows.h" /> <ClInclude Include="..\Sources\MatrixClasses\BaseFloatMatrix.h" /> <ClInclude Include="..\Sources\MatrixClasses\BaseIndexMatrix.h" /> <ClInclude Include="..\Sources\MatrixClasses\BaseMatrix.h" /> <ClInclude Include="..\Sources\MatrixClasses\ComplexMatrix.h" /> <ClInclude Include="..\Sources\MatrixClasses\CufftComplexMatrix.h" /> <ClInclude Include="..\Sources\MatrixClasses\IndexMatrix.h" /> <ClInclude Include="..\Sources\MatrixClasses\RealMatrix.h" /> <ClInclude Include="..\Sources\OutputStreams\BaseOutputStream.h" /> <ClInclude Include="..\Sources\OutputStreams\CuboidOutputStream.h" /> <ClInclude Include="..\Sources\OutputStreams\IndexOutputStream.h" /> <ClInclude Include="..\Sources\OutputStreams\OutputStreamsCudaKernels.cuh" /> <ClInclude Include="..\Sources\OutputStreams\WholeDomainOutputStream.h" /> <ClInclude Include="..\Sources\Parameters\CommandLineParameters.h" /> <ClInclude Include="..\Sources\Parameters\CudaDeviceConstants.cuh" /> <ClInclude Include="..\Sources\Parameters\CudaParameters.h" /> <ClInclude Include="..\Sources\Parameters\Parameters.h" /> <ClInclude Include="..\Sources\Utils\CudaUtils.cuh" /> <ClInclude Include="..\Sources\Utils\DimensionSizes.h" /> <ClInclude Include="..\Sources\Utils\MatrixNames.h" /> <ClInclude Include="..\Sources\Utils\TimeMeasure.h" /> </ItemGroup> <ItemGroup> <CudaCompile Include="..\Sources\KSpaceSolver\SolverCudaKernels.cu" /> <CudaCompile Include="..\Sources\OutputStreams\OutputStreamsCudaKernels.cu" /> <CudaCompile Include="..\Sources\Parameters\CudaDeviceConstants.cu" /> </ItemGroup> <PropertyGroup Label="Globals"> <ProjectGuid>{B040477E-5790-4F5D-B010-B7B57EE26FED}</ProjectGuid> <RootNamespace>k_wave_fluid_cuda</RootNamespace> <WindowsTargetPlatformVersion>10.0.17763.0</WindowsTargetPlatformVersion> </PropertyGroup> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" /> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration"> <ConfigurationType>Application</ConfigurationType> <UseDebugLibraries>true</UseDebugLibraries> <CharacterSet>MultiByte</CharacterSet> <PlatformToolset>v141</PlatformToolset> <WholeProgramOptimization>true</WholeProgramOptimization> </PropertyGroup> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration"> <ConfigurationType>Application</ConfigurationType> <UseDebugLibraries>false</UseDebugLibraries> <WholeProgramOptimization>true</WholeProgramOptimization> <CharacterSet>MultiByte</CharacterSet> <PlatformToolset>v141</PlatformToolset> </PropertyGroup> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" /> <ImportGroup Label="ExtensionSettings"> <Import Project="$(VCTargetsPath)\BuildCustomizations\CUDA 10.0.props" /> </ImportGroup> <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|x64'"> <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" /> </ImportGroup> <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|x64'"> <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" /> </ImportGroup> <PropertyGroup Label="UserMacros" /> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'"> <LinkIncremental> </LinkIncremental> <IncludePath>$(VC_IncludePath);$(WindowsSDK_IncludePath);c:\Program Files\HDF_Group\HDF5\1.8.21\include;.;..\Sources</IncludePath> <LibraryPath>$(VC_LibraryPath_x64);$(WindowsSDK_LibraryPath_x64);$(NETFXKitsDir)Lib\um\x64;c:\Program Files\HDF_Group\HDF5\1.8.21\lib</LibraryPath> </PropertyGroup> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'"> <IncludePath>$(VC_IncludePath);$(WindowsSDK_IncludePath);c:\Program Files\HDF_Group\HDF5\1.8.21\include;.;..\Sources</IncludePath> <LibraryPath>$(VC_LibraryPath_x64);$(WindowsSDK_LibraryPath_x64);$(NETFXKitsDir)Lib\um\x64;c:\Program Files\HDF_Group\HDF5\1.8.21\lib</LibraryPath> </PropertyGroup> <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'"> <ClCompile> <WarningLevel>Level3</WarningLevel> <PreprocessorDefinitions>WIN32;WIN64;_DEBUG;_CONSOLE;%(PreprocessorDefinitions);__KWAVE_GIT_HASH__="b6413962eefd51c642c627d5ceff5093c7ad8da4"</PreprocessorDefinitions> <FavorSizeOrSpeed> </FavorSizeOrSpeed> <FloatingPointModel>Precise</FloatingPointModel> <LanguageStandard> </LanguageStandard> <MultiProcessorCompilation>true</MultiProcessorCompilation> <EnableEnhancedInstructionSet> </EnableEnhancedInstructionSet> <OpenMPSupport>false</OpenMPSupport> <CompileAs>Default</CompileAs> <BasicRuntimeChecks>Default</BasicRuntimeChecks> <WholeProgramOptimization>false</WholeProgramOptimization> <OpenMP>GenerateParallelCode</OpenMP> <CCppSupport>Cpp11Support</CCppSupport> </ClCompile> <Link> <GenerateDebugInformation>true</GenerateDebugInformation> <SubSystem>Console</SubSystem> <AdditionalDependencies>cudart_static.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies);libszip.lib;libzlib.lib;libhdf5.lib;libhdf5_hl.lib;cufft.lib</AdditionalDependencies> <AdditionalLibraryDirectories>%(AdditionalLibraryDirectories);$(CudaToolkitLibDir);</AdditionalLibraryDirectories> </Link> <CudaCompile> <TargetMachinePlatform>64</TargetMachinePlatform> <GenerateRelocatableDeviceCode>true</GenerateRelocatableDeviceCode> <PtxAsOptionV>true</PtxAsOptionV> <FastMath>true</FastMath> <CodeGeneration>compute_50,sm_50</CodeGeneration> <AdditionalCompilerOptions> </AdditionalCompilerOptions> </CudaCompile> </ItemDefinitionGroup> <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'"> <ClCompile> <WarningLevel>Level3</WarningLevel> <Optimization>MaxSpeed</Optimization> <FunctionLevelLinking>true</FunctionLevelLinking> <IntrinsicFunctions>true</IntrinsicFunctions> <PreprocessorDefinitions>WIN32;WIN64;NDEBUG;_CONSOLE;%(PreprocessorDefinitions);__KWAVE_GIT_HASH__="b6413962eefd51c642c627d5ceff5093c7ad8da4"</PreprocessorDefinitions> <MultiProcessorCompilation>true</MultiProcessorCompilation> <EnableEnhancedInstructionSet> </EnableEnhancedInstructionSet> <LanguageStandard>stdcpp14</LanguageStandard> <FloatingPointModel> </FloatingPointModel> <OpenMPSupport>false</OpenMPSupport> </ClCompile> <Link> <GenerateDebugInformation>false</GenerateDebugInformation> <EnableCOMDATFolding>true</EnableCOMDATFolding> <OptimizeReferences>true</OptimizeReferences> <SubSystem>Console</SubSystem> <AdditionalDependencies>cudart_static.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies);libszip.lib;libzlib.lib;libhdf5.lib;libhdf5_hl.lib;cufft.lib</AdditionalDependencies> </Link> <CudaCompile> <TargetMachinePlatform>64</TargetMachinePlatform> <GenerateRelocatableDeviceCode>true</GenerateRelocatableDeviceCode> <CodeGeneration>compute_75,sm_75</CodeGeneration> <FastMath>true</FastMath> </CudaCompile> </ItemDefinitionGroup> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" /> <ImportGroup Label="ExtensionTargets"> <Import Project="$(VCTargetsPath)\BuildCustomizations\CUDA 10.0.targets" /> </ImportGroup> </Project></code></pre> andrewFrizado on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6948 Wed, 03 Jul 2019 13:22:57 +0000 andrewFrizado 6948@http://www.k-wave.org/forum/ Hi Jiri, I have also been trying to get the same setup on a PC running Windows 7 with a NVIDIA RTX 2080 Ti. I am able to drop in the binaries for Pascal architecture and get good performance, though not quite as good as the setup on the Linux machine. I am assuming that this is related to the mismatch of the Turing architecture etc. To workaround, I am trying to recompile the source codes in Visual Studio 2015 (I have also tried in Visual Studio 2013 with v120 platform toolset) but continuously am running into new errors with the library dependencies (directly related to the kWave folders that are in the directory). I have looked at the vcxproj file in a text editor and have updated some of the variables to fit my compilation paramters (HDF1.8.18 instead of HDF5 1.8.19, and CUDA 10.1). My unfamiliarity in Windows and VS definitely lend itself to my troubles, but I was wondering what aleterations I would need to make to the vcxproj file such that I can simply build the code from the provided .sln file. I also assumed that for strictly compiling GPU code the intel compiler and library would not be needed. Is this correct? What else is needed to get these binaries compiled? Attached are my errors using VS 2015 BuildTools (they are repeated from each k-Wave file in the zip directories): <pre><code>"C:\Users\Andrew\Documents\kWaveSourceTest\k-wave-fluid-cuda.sln" (default targ et) (1) -> "C:\Users\Andrew\Documents\kWaveSourceTest\k-wave-fluid-cuda\k-wave-fluid-cuda. vcxproj" (default target) (2) -> "C:\Users\Andrew\Documents\kWaveSourceTest\k-wave-fluid-cuda\k-wave-fluid-cuda. vcxproj" (CudaBuildCore target) (2:2) -> (CudaBuildCore target) -> C:/Users/Andrew/Documents/kWaveSourceTest/k-wave-fluid-cuda/KSpaceSolver/Solv erCUDAKernels.cu(34): fatal error C1083: Cannot open include file: 'KSpaceSolve r/SolverCudaKernels.cuh': No such file or directory [C:\Users\Andrew\Documents\ kWaveSourceTest\k-wave-fluid-cuda\k-wave-fluid-cuda.vcxproj]</code></pre> Jiri Jaros on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6945 Mon, 01 Jul 2019 11:03:12 +0000 Jiri Jaros 6945@http://www.k-wave.org/forum/ Hi qjk, When building the codes, I always download sources of SZIP, HDF5 and CUDA and compile them with the same compiler as the k-Wave binary. This saves a lot of problems with different versions, compile parameters, etc. I run on Ubuntu 18.04 and CUDA 10.1 without any problems (installed manually form Nvidia website). Best Jiri qjk on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6883 Fri, 14 Jun 2019 04:20:10 +0000 qjk 6883@http://www.k-wave.org/forum/ Solved the above problem by making an AWS instance with Ubuntu 19.04. The other one I had was 18.04 which is the version Amazon makes the most obvious. 19.04 "Disco" installs CUDA 10.1 with apt-get. All is well in V100 land now. Considerably faster than K80. qjk on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6882 Fri, 14 Jun 2019 01:55:28 +0000 qjk 6882@http://www.k-wave.org/forum/ I was able to get the: (.text+0x618): undefined reference to `aec_buffer_decode' error to go away by adding $(SZIP_DIR)/libaec.a to the definition of the LIBS variable. Now the linker completes. I added the --generate-code arch=... lines to the Makefile. Now when I try to run I am back to "Error: All CUDA-capable devices are busy or unavailable." This could be due to building on a K80 instance with nvcc v9.1. I need to sort out which version of CUDA runs on which hardware. apt-get doesn't seem to want to give me a newer CUDA installation and I'd prefer to use that rather than getting an installation directly from Nvidia since it deals with dependencies. qjk on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6877 Sat, 08 Jun 2019 16:09:42 +0000 qjk 6877@http://www.k-wave.org/forum/ Thanks Andrew, Understood that the error I am getting must be connected to the compression libraries. I tried to apt-get every package that might be relevant to sz2 and zlib, but without success. Chris andrewFrizado on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6875 Thu, 06 Jun 2019 15:12:07 +0000 andrewFrizado 6875@http://www.k-wave.org/forum/ Thanks Jiri, I have got kWave running on Turing RTX 2080 Ti thanks to this. I had to also download the szip and zlib library dependancies and add them to the path in the makefile, plus removed --generate-code arch=compute_20 ... --generate-code arch=compute_21... as there was no support in CUDA 10.1 for these architectures (or, at least, that was the error that was the error reported upon compilation) qjk - your error may be associated with not having the full szip and zlib library dependancies - just a guess though qjk on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6857 Wed, 22 May 2019 01:54:01 +0000 qjk 6857@http://www.k-wave.org/forum/ I finally got round to trying to build from the latest sources on ubuntu 18.04 using LINKING=SEMI. I was able to compile everything but the linker found an undefined reference to 'aec_buffer_encode' (see below). I did "grep -r" in the source to locate this function but did not find it. Is it saying that the object file sz_compat.o inside the archive libsz.a calls a function 'aec_buffer_encode' which is not found? The same issue happens if I say LINKING=STATIC. This is still on the K80 system. I wanted to learn how to compile before going to the V100. Chris <pre><code>nvcc -Xcompiler="-fopenmp" -Xlinker="-rpath,/usr/lib/x86_64-linux-gnu/hdf5/serial/lib:/lib64:." -std=c++11 -L/usr/lib/x86_64-linux-gnu/hdf5/serial/lib -L/lib64 --generate-code arch=compute_60,code=sm_60 --generate-code arch=compute_61,code=sm_61 --generate-code arch=compute_62,code=sm_62 --generate-code arch=compute_70,code=sm_70 --generate-code arch=compute_72,code=sm_72 main.o \ Containers/MatrixContainer.o \ Containers/MatrixRecord.o \ Containers/OutputStreamContainer.o \ Hdf5/Hdf5File.o \ Hdf5/Hdf5FileHeader.o \ KSpaceSolver/KSpaceFirstOrder3DSolver.o \ KSpaceSolver/SolverCudaKernels.o \ Logger/Logger.o \ MatrixClasses/BaseFloatMatrix.o \ MatrixClasses/BaseIndexMatrix.o \ MatrixClasses/CufftComplexMatrix.o \ MatrixClasses/ComplexMatrix.o \ MatrixClasses/IndexMatrix.o \ MatrixClasses/RealMatrix.o \ OutputStreams/BaseOutputStream.o \ OutputStreams/IndexOutputStream.o \ OutputStreams/CuboidOutputStream.o \ OutputStreams/WholeDomainOutputStream.o \ OutputStreams/OutputStreamsCudaKernels.o \ Parameters/CommandLineParameters.o \ Parameters/Parameters.o \ Parameters/CudaParameters.o \ Parameters/CudaDeviceConstants.o \ /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_hl.a /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.a /usr/lib/x86_64-linux-gnu/libz.a /usr/lib/x86_64-linux-gnu/libsz.a -lcufft -ldl \ -o kspaceFirstOrder3D-CUDA /usr/lib/x86_64-linux-gnu/libsz.a(sz_compat.o): In function</code></pre> SZ_BufftoBuffCompress': (.text+0x149): undefined reference to `aec_buffer_encode' (.text+0x339): undefined reference to `aec_buffer_encode' /usr/lib/x86_64-linux-gnu/libsz.a(sz_compat.o): In function `SZ_BufftoBuffDecompress': (.text+0x51a): undefined reference to `aec_buffer_decode' (.text+0x618): undefined reference to `aec_buffer_decode' collect2: error: ld returned 1 exit status Makefile:174: recipe for target 'kspaceFirstOrder3D-CUDA' failed make: *** [kspaceFirstOrder3D-CUDA] Error 1` qjk on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6825 Sat, 06 Apr 2019 18:19:08 +0000 qjk 6825@http://www.k-wave.org/forum/ Thank you for explaining the solution! DLam on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6824 Fri, 05 Apr 2019 15:20:07 +0000 DLam 6824@http://www.k-wave.org/forum/ Thanks, worked for me! Jiri Jaros on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6822 Thu, 04 Apr 2019 13:37:52 +0000 Jiri Jaros 6822@http://www.k-wave.org/forum/ Hi Chris, the binary, you probably have, was compiled with CUDA 8.0 and there was no support for V100 by that time. Please download the source codes and recompile the binary with CUDA 10.1. In order to add support for Pascal, Volta and Turing GPUs, you also have to modify the makefile. Just simply add these lines below the similar ones: --generate-code arch=compute_60,code=sm_60 \ --generate-code arch=compute_61,code=sm_61 \ --generate-code arch=compute_62,code=sm_62 \ --generate-code arch=compute_70,code=sm_70 \ --generate-code arch=compute_72,code=sm_72 \ --generate-code arch=compute_75,code=sm_75 \ qjk on "Running on AWS Tesla instances" http://www.k-wave.org/forum/topic/running-on-aws-tesla-instances#post-6815 Sat, 30 Mar 2019 05:51:26 +0000 qjk 6815@http://www.k-wave.org/forum/ Hi, I have been attempting to use k-wave on AWS. I was able to get it running on an instance with a K80 Tesla card. Then I tried a V100 instance, using the V390 Nvidia driver on Ubuntu 18.04. I received the error: "Error: All CUDA-capable devices are busy or unavailable" The AWS instance which gave this error is a p3.2xlarge which has an 8-core Xeon CPU and a single V100 Tesla card. What am I doing wrong here? Thanks, Chris