Nautilus connectors kit is a tool which aim is getting raw data from different sources and store them as-is into different destinations (GCS, BQ, local files, etc.).

• Google DoubleClick Manager (DBM / DV360)
• Google Campaign Manager (CM / DCM)
• Google Search Ads 360 (SA360)
• Google Analytics
• Google Search Console
• Google Sheets
• Google Cloud Storage
• Google Adwords
• Google Search Console
• Facebook Business Manager
• Amazon S3
• Oracle
• SalesForce
• MySQL
• Radarly
• Adobe Analytics 1.4
• Yandex

• Google BigQuery
• Google Cloud Storage
• Amazon S3
• Local File
• Console (Debug)

NCK is divided in three main components : Readers, Streams, and Writers.

• Readers role is to read data from distant sources and transform it into stream object
• Streams role is to be read as file or line by line. There are local objects usable by writers
• Writers role is to write stream into distant location

nck could be consumed through a docker image or can be installed as package and then be used as library or a binary.

1. Build Docker image using make build_base_image
2. Run image to get help docker run --rm nautilus-connector-kit:latest --help

First install dependencies with:

pip install -r requirements.txt

Then run:

python nck/entrypoint.py

• Exec cmd make dist (it generates a source distrib and a wheel in the created directory dist/)

It is advised to do the following in a virtual env

python3 -m venv testenv; source testenv/bin/activate

pip wheel --wheel-dir=wheels -r requirements.txt (that creates folder of wheels for packages in requierements)

pip install --no-index --find-links=./wheels dist/[nck-file-generated].whl

pip install -e .

python setup.py install

• Run cmd nckrun --help (which is equivalent to python nck/entrypoint.py)

from nck.readers.dbm_reader import DbmReader

• https://manikos.github.io/a-tour-on-python-packaging
• http://lucumr.pocoo.org/2014/1/27/python-on-wheels/
• https://pip.readthedocs.io/en/1.4.1/cookbook.html#controlling-setup-requires