Skip to content

tpeng/python-hubstorage

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

HubStorage service client library

https://badge.fury.io/py/hubstorage.png https://secure.travis-ci.org/scrapinghub/python-hubstorage.png?branch=master

Note

This module is experimental and its API may change without previous notice.

Overview

This is the HubStorage client library, which contains:

  • Full client api trough hubstorage.HubstorageClient

Requirements

Basic API

Example creating a new job:

>>> from hubstorage import HubstorageClient
>>> hs = HubstorageClient(auth=apikey)
>>> job = hs.new_job(projectid='1111111', spider='foo')
>>> job.key
'1111111/1/1'

>>> job.metadata['state']
'pending'

>>> job.items.write({'title': 'my first item'})
>>> job.logs.info('lorem impsum message are cool')
>>> job.logs.error('but sometimes s**t happens')
>>> job.finished()

Example getting job data later:

>> job = hs.get_job('1111111/1/1')
>> job.metadata['state']
'finished'

>> list(job.items.list(count=1))
[{'title': 'my first item'}]

...

About

HubStorage client library

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 100.0%