Home Explore Help
Sign In
code
/
transport
mirror of https://github.com/lnyemba/data-transport.git
2
0
Fork 0
Files Issues 0 Wiki

No Description

Tree: f9aba5db1c
Branches Tags
transport
ZIP TAR.GZ
Steve L. Nyemba f9aba5db1c Merge pull request #14 from lnyemba/v2.0.4 11 months ago
bin f5187790ce refactor: etl,better reusability & streamlined and threaded 11 months ago
info f5187790ce refactor: etl,better reusability & streamlined and threaded 11 months ago
notebooks 2b5c038610 documentation ... 11 months ago
transport 870c1caed3 bug fix: use plugins to refer to plugins 11 months ago
.gitignore 92bf0600c3 .. 1 year ago
README.md 5adbb5a61e bug fixes and documentation 1 year ago
requirements.txt 8d4ecd7a9f S3 Requirments file 7 years ago
setup.py f5187790ce refactor: etl,better reusability & streamlined and threaded 11 months ago

README.md

Introduction

This project implements an abstraction of objects that can have access to a variety of data stores, implementing read/write with a simple and expressive interface. This abstraction works with NoSQL, SQL and Cloud data stores and leverages pandas.

Why Use Data-Transport ?

Mostly data scientists that don't really care about the underlying database and would like a simple and consistent way to read/write and move data are well served. Additionally we implemented lightweight Extract Transform Loading API and command line (CLI) tool. Finally it is possible to add pre/post processing pipeline functions to read/write

  1. Familiarity with pandas data-frames
  2. Connectivity drivers are included
  3. Mining data from various sources
  4. Useful for data migrations or ETL

Installation

Within the virtual environment perform the following :

pip install git+https://github.com/lnyemba/data-transport.git

Learn More

We have available notebooks with sample code to read/write against mongodb, couchdb, Netezza, PostgreSQL, Google Bigquery, Databricks, Microsoft SQL Server, MySQL ... Visit data-transport homepage