# Setting up the database

- Install postgres
- Create a database named 'sociallists'
- In python, do:

  >>> from sociallists import db
  >>> db.Base.metadata.create_all()

# Some notes on asynchrony

I spent some time trying to convert this codebase to asyncio so that feed
updates can be SUPER PARALLEL. What I discovered is that if I do this the HTTP
stack begins to misbehave and timeout and stuff, and I'm not entirely sure why.

So aiohttp doesn't seem to be a good choice for me.

Using coroutines but a fundamentally blocking HTTP stack (asyncio + requests)
doesn't actually buy me anything because all my threads will be blocked on the
same fixed number of HTTP requests anyway, so it will be a lot of complexity
for absolutely no efficiency gain.

(In addition, the async/await stuff is currently very ugly and the docs are
complicated and I just like this one better for now. So we're sticking with
the slow but useful. Rewriting the feed update process to be massively parallel
is still something I want to do but it will have to wait.)

# TODO:

UI:
- Error boxes
- Enable jumping between refresh sessions:
  - Mark the top refresh session on all rivers when we start a refresh.
  - Put navigation buttons in the boundaries between refreshes that jump up
    and down. (i.e., build a meta-FeedUpdate construct that is "refresh
    session"; it will be river-specific. This might be complex.)
- Add/remove rivers
- Add/remove feeds from rivers
- Add/remove river sets
- Re-order rivers

Shipping:
- Re-visit use of requirements and packages and vendoring and the like
- Re-pack font awesome fonts
- Use packed/minified/precompiled source if it exists

Feed Stuff:
- Extract enclosures and the like, store locally (in the DB) and present them
  in the river. (Then do UI work to expose audio enclosures and the like.)
- Put river items in the DB so that we can efficiently re-run the processing
  steps for items that are out of the feed
- For the future, we'll probably need options on individual feeds for
  processing to make them better. It is possible to over-engineer this so be
  careful.
- Implement ATOM cache timeout and RSS skip hours and the like to poll more
  intelligently.
- Implement pubsubhubbub
- Clip titles that are too long. (Because seriously.)

Bugs:
- Can't add feed to river:
  stderr: 2016-07-18 10:15:23,113 127.0.0.1 - - [18/Jul/2016 10:15:23] "POST /wat/9 HTTP/1.1" 404 -
- Fix Syblmoon bug where no title means broken rendering
- b0rk's feed refreshes completely every time there's a new article; fix it!
- Bug here:

  stderr: 2016-07-18 10:13:01,881 Error updating feed http://feeds.feedburner.com/buckblog: Traceback (most recent call last):
    File "/Users/doty/src/sociallists/sociallists/feed.py", line 244, in do_update_feed
      update = self.get_feed_update(history)
    File "/Users/doty/src/sociallists/sociallists/feed.py", line 191, in get_feed_update
      f, self.feed.next_item_id, update_time, self.http_session
    File "/Users/doty/src/sociallists/sociallists/river.py", line 186, in feed_to_river_update
      for e,i in zip(feed.entries, count(start_id))
    File "/Users/doty/src/sociallists/sociallists/river.py", line 186, in <listcomp>
      for e,i in zip(feed.entries, count(start_id))
    File "/Users/doty/src/sociallists/sociallists/river.py", line 152, in entry_to_river
      image = get_entry_thumbnail_image(entry, session)
    File "/Users/doty/src/sociallists/sociallists/river.py", line 131, in get_entry_thumbnail_image
      thumbnail_image = media.get_url_image(link, size, session)
    File "/Users/doty/src/sociallists/sociallists/media.py", line 37, in get_url_image
      thumbnail_url, http_session, referer=url)
    File "/Users/doty/src/sociallists/sociallists/media.py", line 74, in _fetch_url
      result = (response.url, response.headers['Content-Type'], response.content)
    File "/Users/doty/src/sociallists/venv/lib/python3.5/site-packages/requests/structures.py", line 54, in __getitem__
      return self._store[key.lower()][1]
  KeyError: 'content-type'