def from_zip_csv(cls, fp: PathSpecifier, config: StoreConfigMapInitializer = None) -> 'Bus': store = StoreZipCSV(fp) return cls(cls._deferred_series(store.labels()), store=store, config=config)
def to_zip_csv(self, fp: PathSpecifier, config: StoreConfigMapInitializer = None) -> None: ''' Write the complete :obj:`Bus` as a zipped archive of CSV files. {args} ''' store = StoreZipCSV(fp) config = config if not config is None else self._config store.write(self.items(), config=config)
def to_zip_csv(self, fp: PathSpecifier, *, config: StoreConfigMapInitializer = None) -> None: ''' Write the complete :obj:`Bus` as a zipped archive of CSV files. {args} ''' store = StoreZipCSV(fp) config = self._filter_config(config) store.write(self._items_store(), config=config)
def test_store_zip_csv_b(self) -> None: f1, *_ = get_test_framesA() with temp_file('.zip') as fp: st = StoreZipCSV(fp) st.write((f.name, f) for f in (f1,)) # this now uses a default config f = st.read(f1.name) self.assertEqual(f.to_pairs(), (('__index0__', ((0, 'x'), (1, 'y'))), ('a', ((0, 1), (1, 2))), ('b', ((0, 3), (1, 4)))))
def test_store_zip_csv_b(self) -> None: f1 = Frame.from_dict(dict(a=(1, 2), b=(3, 4)), index=('x', 'y'), name='foo') with temp_file('.zip') as fp: st = StoreZipCSV(fp) st.write((f.name, f) for f in (f1, )) with self.assertRaises(ErrorInitStore): # config is required _ = st.read(f1.name)
def from_zip_csv( cls, fp: PathSpecifier, config: StoreConfigMapInitializer = None) -> 'StoreClientMixin': ''' Given a file path to zipped CSV :obj:`Bus` store, return a :obj:`Bus` instance. {args} ''' store = StoreZipCSV(fp) return cls._from_store(store, config) #type: ignore
def test_store_zip_csv_a(self) -> None: f1 = Frame.from_dict(dict(a=(1, 2), b=(3, 4)), index=('x', 'y'), name='foo') f2 = Frame.from_dict(dict(a=(1, 2, 3), b=(4, 5, 6)), index=('x', 'y', 'z'), name='bar') f3 = Frame.from_dict(dict(a=(10, 20), b=(50, 60)), index=('p', 'q'), name='baz') with temp_file('.zip') as fp: st = StoreZipCSV(fp) st.write((f.name, f) for f in (f1, f2, f3)) labels = tuple(st.labels(strip_ext=False)) self.assertEqual(labels, ('foo.csv', 'bar.csv', 'baz.csv')) config = StoreConfig(index_depth=1) for label, frame in ((f.name, f) for f in (f1, f2, f3)): frame_stored = st.read(label, config=config) self.assertEqual(frame_stored.shape, frame.shape) self.assertTrue((frame_stored == frame).all().all()) self.assertEqual(frame.to_pairs(0), frame_stored.to_pairs(0))
def from_zip_csv( cls, fp: PathSpecifier, config: StoreConfigMapInitializer = None, max_persist: tp.Optional[int] = None, ) -> 'StoreClientMixin': ''' Given a file path to zipped CSV :obj:`Bus` store, return a :obj:`Bus` instance. {args} ''' store = StoreZipCSV(fp) return cls._from_store( store, #type: ignore config=config, max_persist=max_persist, )
def from_zip_csv(cls, fp: PathSpecifier, *, config: StoreConfigMapInitializer = None, max_persist: tp.Optional[int] = None, index_constructor: IndexConstructor = None, ) -> 'Bus': ''' Given a file path to zipped CSV :obj:`Bus` store, return a :obj:`Bus` instance. {args} ''' store = StoreZipCSV(fp) return cls._from_store(store, config=config, max_persist=max_persist, index_constructor=index_constructor, )
def from_zip_csv(cls, fp: PathSpecifier, *, config: StoreConfigMapInitializer = None, axis: int = 0, retain_labels: bool, deepcopy_from_bus: bool = False, max_persist: tp.Optional[int] = None, ) -> 'Quilt': ''' Given a file path to zipped CSV :obj:`Quilt` store, return a :obj:`Quilt` instance. {args} ''' store = StoreZipCSV(fp) return cls._from_store(store, config=config, axis=axis, retain_labels=retain_labels, deepcopy_from_bus=deepcopy_from_bus, max_persist=max_persist, )
def from_zip_csv( cls, fp: PathSpecifier, *, config: StoreConfigMapInitializer = None, max_workers: tp.Optional[int] = None, chunksize: int = 1, use_threads: bool = False, ) -> 'Batch': ''' Given a file path to zipped CSV :obj:`Batch` store, return a :obj:`Batch` instance. {args} ''' store = StoreZipCSV(fp) return cls._from_store( store, config=config, max_workers=max_workers, chunksize=chunksize, use_threads=use_threads, )
def test_store_zip_csv_a(self) -> None: f1, f2, f3 = get_test_framesA() with temp_file('.zip') as fp: st = StoreZipCSV(fp) st.write((f.name, f) for f in (f1, f2, f3)) labels = tuple(st.labels(strip_ext=False)) self.assertEqual(labels, ('foo.csv', 'bar.csv', 'baz.csv')) for label, frame in ((f.name, f) for f in (f1, f2, f3)): for read_max_workers in (1, 2): config = StoreConfig(index_depth=1, read_max_workers=1) frame_stored = st.read(label, config=config) self.assertEqual(frame_stored.shape, frame.shape) self.assertTrue((frame_stored == frame).all().all()) self.assertEqual(frame.to_pairs(0), frame_stored.to_pairs(0))
def to_zip_csv(self, fp: PathSpecifier, config: StoreConfigMapInitializer = None) -> None: store = StoreZipCSV(fp) config = config if not None else self._config store.write(self.items(), config=config)