# HG changeset patch # User Rik # Date 1382326211 25200 # Node ID 5415a9cd61d4148406ca8cdaa4666aa0d8884a22 # Parent f10b7a578e2cb2cc5e89323a88c28201e73b6be6 Implement faster partial loading of HDF5 files. * libinterp/corefcn/load-save.cc(do_load): Call read_hdf5_data with extra arguments for variables to load. * libinterp/corefcn/ls-hdf5.h: Re-define read_Hdf5_data function prototype. * libinterp/corefcn/ls-hdf5.cc: Check for any variable names to load, instead of whole file. diff -r f10b7a578e2c -r 5415a9cd61d4 libinterp/corefcn/load-save.cc --- a/libinterp/corefcn/load-save.cc Tue Jul 02 14:17:33 2013 +0100 +++ b/libinterp/corefcn/load-save.cc Sun Oct 20 20:30:11 2013 -0700 @@ -380,7 +380,8 @@ #ifdef HAVE_HDF5 case LS_HDF5: - name = read_hdf5_data (stream, orig_fname, global, tc, doc); + name = read_hdf5_data (stream, orig_fname, global, tc, doc, + argv, argv_idx, argc); break; #endif /* HAVE_HDF5 */ diff -r f10b7a578e2c -r 5415a9cd61d4 libinterp/corefcn/ls-hdf5.cc --- a/libinterp/corefcn/ls-hdf5.cc Tue Jul 02 14:17:33 2013 +0100 +++ b/libinterp/corefcn/ls-hdf5.cc Sun Oct 20 20:30:11 2013 -0700 @@ -580,7 +580,8 @@ // and error. std::string read_hdf5_data (std::istream& is, const std::string& /* filename */, - bool& global, octave_value& tc, std::string& doc) + bool& global, octave_value& tc, std::string& doc, + const string_vector& argv, int argv_idx, int argc) { std::string retval; @@ -599,6 +600,37 @@ #endif H5Gget_num_objs (group_id, &num_obj); H5Gclose (group_id); + + // For large datasets and out-of-core functionality, + // check if only parts of the data is requested + bool load_named_vars = argv_idx < argc; + while (load_named_vars && hs.current_item < static_cast (num_obj)) + { + std::vector var_name; + bool found = false; + size_t len = 0; + + len = H5Gget_objname_by_idx (hs.file_id, hs.current_item, 0, 0); + var_name.resize (len+1); + H5Gget_objname_by_idx( hs.file_id, hs.current_item, &var_name[0], len+1); + + for (int i = argv_idx; i < argc; i++) + { + glob_match pattern (argv[i]); + if (pattern.match (std::string (&var_name[0]))) + { + found = true; + break; + } + } + + if (found) + break; + + hs.current_item++; + } + + if (hs.current_item < static_cast (num_obj)) H5Giterate_retval = H5Giterate (hs.file_id, "/", &hs.current_item, hdf5_read_next_data, &d); diff -r f10b7a578e2c -r 5415a9cd61d4 libinterp/corefcn/ls-hdf5.h --- a/libinterp/corefcn/ls-hdf5.h Tue Jul 02 14:17:33 2013 +0100 +++ b/libinterp/corefcn/ls-hdf5.h Sun Oct 20 20:30:11 2013 -0700 @@ -180,7 +180,8 @@ extern OCTINTERP_API std::string read_hdf5_data (std::istream& is, const std::string& filename, bool& global, - octave_value& tc, std::string& doc); + octave_value& tc, std::string& doc, + const string_vector& argv, int argv_idx, int argc); extern OCTINTERP_API bool save_hdf5_data (std::ostream& os, const octave_value& tc,