Skip to content

nmb10/ambryfdw

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DEPRECATED: Merged into ambry_sources . Also it is broken because format of the packed partition files changed.

Ambry partition msgpack files Foreign Data Wrapper (multicorn)

This FDW can be used to access data stored in the ambry partitions packed with msgpack. Each column defined in the table will be mapped, in order, against columns in the ambry partition file.

Installation.

Install multicorn:

wget https://github.com/Kozea/Multicorn/archive/v1.2.3.zip
unzip v1.2.3.zip
cd Multicorn-1.2.3
make && sudo make install

Install ambryfdw:

pip install ambryfdw

Run tests.

python setup.py test

Options

filename (required)

The full path to the gzipped partition file containing the data in msgpack format. This file must be readable to the postgres user.

Usage

Supposing you want to parse the following msgpack partition file, located in /tmp/test.msg::

Year,Make,Model,Length
1997,Ford,E350,2.34
2000,Mercury,Cougar,2.38

You can declare the following table:

    CREATE SERVER partition_srv foreign data wrapper multicorn options (
        wrapper 'ambryfdw.PartitionMsgpackForeignDataWrapper'
    );

    CREATE FOREIGN TABLE partition_test (
           year numeric,
           make character varying,
           model character varying,
           length numeric
    ) SERVER partition_srv OPTIONS (
           filename '/tmp/test.msg');

Now you can query ambry partition

    SELECT * FROM partition_test;
     year |  make   | model  | length
    ------+---------+--------+--------
     1997 | Ford    | E350   |   2.34
     2000 | Mercury | Cougar |   2.38
    (2 lines)

About

This FDW (Foreign Data Wrapper) can be used to access data stored in the ambry partitions packed with msgpack.

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages