Importing and Exporting Systems

1. All Infrasys systems can be serialized and deserialized using the to_json and from_json methods.

2. The DistributionSystem class can also be exported to GeoDataFrame and GeoJSON formats using the to_gdf and to_geojson methods.

We start by loading a sample distribution system.

from gdm.distribution import DistributionSystem
from gdmloader.constants import GCS_CASE_SOURCE
from gdmloader.source import SystemLoader

gdm_loader = SystemLoader()
gdm_loader.add_source(GCS_CASE_SOURCE)

distribution_system: DistributionSystem = gdm_loader.load_dataset(
    source_name=GCS_CASE_SOURCE.name,
    system_type=DistributionSystem,
    dataset_name="p5r",
)
distribution_system.name = "P5R"

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
File /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/gdmloader/source.py:119, in SystemLoader.load_dataset(self, system_type, source_name, dataset_name, version)
    118 try:
--> 119     source.fs.get(
    120         remote_folder,
    121         str(local_folder),
    122         recursive=True,
    123     )
    124 except FileNotFoundError:

File /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/fsspec/asyn.py:118, in sync_wrapper.<locals>.wrapper(*args, **kwargs)
    117 self = obj or args[0]
--> 118 return sync(self.loop, func, *args, **kwargs)

File /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/fsspec/asyn.py:103, in sync(loop, func, timeout, *args, **kwargs)
    102 elif isinstance(return_result, BaseException):
--> 103     raise return_result
    104 else:

File /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/fsspec/asyn.py:56, in _runner(event, coro, result, timeout)
     55 try:
---> 56     result[0] = await coro
     57 except Exception as ex:

File /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/fsspec/asyn.py:645, in AsyncFileSystem._get(self, rpath, lpath, recursive, callback, maxdepth, **kwargs)
    644 rpath = self._strip_protocol(rpath)
--> 645 rpaths = await self._expand_path(
    646     rpath, recursive=recursive, maxdepth=maxdepth
    647 )
    648 if source_is_str and (not recursive or maxdepth is not None):
    649     # Non-recursive glob does not copy directories

File /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/fsspec/asyn.py:883, in AsyncFileSystem._expand_path(self, path, recursive, maxdepth)
    882 if isinstance(path, str):
--> 883     out = await self._expand_path([path], recursive, maxdepth)
    884 else:

File /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/fsspec/asyn.py:912, in AsyncFileSystem._expand_path(self, path, recursive, maxdepth)
    911 if not out:
--> 912     raise FileNotFoundError(path)
    913 return sorted(out)

FileNotFoundError: ['gdm_data/data/DistributionSystem/2_1_4/p5r']

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[1], line 8
      5 gdm_loader = SystemLoader()
      6 gdm_loader.add_source(GCS_CASE_SOURCE)
----> 8 distribution_system: DistributionSystem = gdm_loader.load_dataset(
      9     source_name=GCS_CASE_SOURCE.name,
     10     system_type=DistributionSystem,
     11     dataset_name="p5r",
     12 )
     13 distribution_system.name = "P5R"

File /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/gdmloader/source.py:126, in SystemLoader.load_dataset(self, system_type, source_name, dataset_name, version)
    124     except FileNotFoundError:
    125         msg = f"{remote_folder=} not found! Check the URL: {source.url}/{remote_folder}"
--> 126         raise ValueError(msg)
    128 system_file = list(dataset_folder.rglob("*.json"))[0]
    129 return system_type.from_json(system_file)

ValueError: remote_folder='gdm_data/data/DistributionSystem/2_1_4/p5r' not found! Check the URL: https://storage.googleapis.com/gdm_data/gdm_data/data/DistributionSystem/2_1_4/p5r

The DistributionSystem can be serialized to disk using the to_json method. If the model already exists, set the overwrite parameter to True. Serialization ensures no loss of information. The JSON file can be deserialized using the from_json method.

distribution_system.to_json("test.json", overwrite=True)
system = DistributionSystem.from_json("test.json")

The DistributionSystem can also be exported in GeoDataFrame and GeoJSON formats.

Warning

Limited model information is exported in these formats. Model deserialization is not supported for these formats.

system.to_gdf(“test.csv”) system.to_geojson(“test.geojson”)

system.to_gdf("test.csv")
system.to_geojson("test.geojson")