forked from intelie/pycollector
isabella232/pycollector
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
_ _ _ _ __ _ _ ___ ___ | || | ___ ___ | |_ ___ _ __ | '_ \ | | | | / __|/ _ \ | || | / _ \ / __|| __|/ _ \ | '__| | |_) || |_| || (__| (_) || || || __/| (__ | |_| (_) || | | .__/ \__, | \___|\___/ |_||_| \___| \___| \__|\___/ |_| |_| |___/ by Intelie Collecting data should be simple. -------- | Goal | -------- Moving data from A to B is a well known task in computing. Our aim here is to make it easier to 'develop' and 'configure' a piece of software to collect data. ------------------- | Main components | ------------------- There are two main components of any collecting process: - Readers Piece of code that takes the responsability of going to the source of the data and prepare it to be written somewhere else. - Writers Piece of code that takes a read message and delivers to the destination. A pair reader/writer is called collecting unit. Readers and writers may make any transformation to the original data. ---------------- | Architecture | ---------------- Collecting unit architecture: 1. ask source_______ ________ 5. write destination <-------| | | |------------------> | Reader | | Writer | ------->|________| +--->|________| 2. answer | | | __________ | 4. get stored msgs +---------->| |--+ 3.store | internal | | queue | |__________| It is possible to configure as many collecting units as needed. -------------- | How to use | -------------- There are two ways to use it. - configuring you may configure a predefined reader/writer for you needs. - development you may adapt an existing reader/writer or code a new one. --------------- | Configuring | --------------- A reader/writer always have a 'type' property. This 'type' will define how the data will be read by a reader or written by a writer. Moreover, readers and writers have properties in common, regardless of their types: * 'blockable' if 'blockable : True' is set to a reader, the message will wait for a slot in queue. if 'blockable : True' is set to a writer, it will try to rewrite the message. default value: True * 'period' if 'period : N' is set to a reader, the method read() will be called periodically, with period N (in seconds) if 'period : N' is set to a writer, the method write(msg) will be called periodically, with period N (in seconds) default value: None, default is to be async, read()/write() are called just once. ... (TODO: complete list) ---------------- | Current step | ---------------- Improving core; Increasing test coverage.
About
No description, website, or topics provided.
Resources
License
Code of conduct
Stars
Watchers
Forks
Packages 0
No packages published
Languages
- Python 99.4%
- Other 0.6%