Utilities
Submodules
kale.utils.download module
Data downloading and compressed data extraction functions, Based on https://github.com/pytorch/vision/blob/master/torchvision/datasets/utils.py https://github.com/pytorch/pytorch/blob/master/torch/hub.py
- kale.utils.download.download_file_by_url(url, output_directory, output_file_name, file_format=None)
Download file/compressed file by url.
- Parameters
url (string) – URL of the object to download
output_directory (string, optional) – Full path where object will be saved Abosolute path recommended. Relative path also works.
output_file_name (string, optional) – File name which object will be saved as
file_format (string, optional) – File format For compressed file, support [“tar.xz”, “tar”, “tar.gz”, “tgz”, “gz”, “zip”]
- Example: (Grab the raw link from GitHub. Notice that using “raw” in the URL.)
>>> url = "https://github.com/pykale/data/raw/main/videos/video_test_data/ADL/annotations/labels_train_test/adl_P_04_train.pkl" >>> download_file_by_url(url, "data", "a.pkl", "pkl")
>>> url = "https://github.com/pykale/data/raw/main/videos/video_test_data.zip" >>> download_file_by_url(url, "data", "video_test_data.zip", "zip")
- kale.utils.download.download_file_gdrive(id, output_directory, output_file_name, file_format=None)
Download file/compressed file by Google Drive id.
- Parameters
id (string) – Google Drive file id of the object to download
output_directory (string, optional) – Full path where object will be saved Abosolute path recommended. Relative path also works.
output_file_name (string, optional) – File name which object will be saved as
file_format (string, optional) – File format For compressed file, support [“tar.xz”, “tar”, “tar.gz”, “tgz”, “gz”, “zip”]
Example
>>> gdrive_id = "1U4D23R8u8MJX9KVKb92bZZX-tbpKWtga" >>> download_file_gdrive(gdrive_id, "data", "demo_datasets.zip", "zip")
>>> gdrive_id = "1SV7fmAnWj-6AU9X5BGOrvGMoh2Gu9Nih" >>> download_file_gdrive(gdrive_id, "data", "dummy_data.csv", "csv")
kale.utils.logger module
Logging functions, based on https://github.com/HaozhiQi/ISONet/blob/master/isonet/utils/logger.py
- kale.utils.logger.out_file_core()
Creates an output file name concatenating a formatted date and uuid, but without an extension.
- Returns
A string to be used in a file name.
- Return type
string
- kale.utils.logger.construct_logger(name, save_dir)
Constructs a logger. Saves the output as a text file at a specified path. Also saves the output of git diff HEAD to the same folder.
Reference: https://docs.python.org/3/library/logging.html
- Parameters
name (str) – the logger name, typically the method name
save_dir (str) – the path to save the log file (.txt)
kale.utils.print module
Screen printing functions, from https://github.com/HaozhiQi/ISONet/blob/master/isonet/utils/misc.py
- kale.utils.print.tprint(*args)
Temporarily prints things on the screen so that it won’t be flooded
- kale.utils.print.pprint(*args)
Permanently prints things on the screen to have all info displayed
- kale.utils.print.pprint_without_newline(*args)
Permanently prints things on the screen, separated by space rather than newline
kale.utils.seed module
Setting seed for reproducibility
- kale.utils.seed.set_seed(seed=1000)
Sets the seed for generating random numbers to get (as) reproducible (as possible) results.
The CuDNN options are set according to the official PyTorch guidance on reproducibility: https://pytorch.org/docs/stable/notes/randomness.html. Another references are https://discuss.pytorch.org/t/difference-between-torch-manual-seed-and-torch-cuda-manual-seed/13848/6 https://pytorch.org/docs/stable/cuda.html#torch.cuda.manual_seed https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/utils.py#L58
- Parameters
seed (int, optional) – The desired seed. Defaults to 1000.