def issues_mvt(db, z, x, y): x2,y1 = num2deg(x,y,z) x1,y2 = num2deg(x+1,y+1,z) dx = (x2 - x1) / 256 dy = (y2 - y1) / 256 params = query._params() params.bbox = [y1-dy*8, x1-dx*32, y2+dy*8, x2+dx] if (not params.users) and (not params.source) and (params.zoom < 6): return params.limit = 50 params.full = False expires = datetime.datetime.now() + datetime.timedelta(days=365) path = '/'.join(request.fullpath.split('/')[0:-1]) response.set_cookie('last_zoom', str(params.zoom), expires=expires, path=path) response.set_cookie('last_level', str(params.level), expires=expires, path=path) response.set_cookie('last_item', str(params.item), expires=expires, path=path) response.set_cookie('last_tags', str(','.join(params.tags)) if params.tags else '', expires=expires, path=path) response.set_cookie('last_fixable', str(params.fixable) if params.fixable else '', expires=expires, path=path) response.content_type = 'application/vnd.mapbox-vector-tile' return _errors_mvt(db, params, z, y1, x1, y2, x2, 50)
def issues_mvt(db, z, x, y): lon1, lat2 = tiles.tile2lonlat(x, y, z) lon2, lat1 = tiles.tile2lonlat(x + 1, y + 1, z) dlon = (lon2 - lon1) / 256 dlat = (lat2 - lat1) / 256 params = query._params() params.tilex = x params.tiley = y params.zoom = z if params.zoom > 18: return if (not params.users) and (not params.source) and (params.zoom < 6): return params.limit = 50 params.full = False tile = _errors_mvt(db, params, z, lon1, lat1, lon2, lat2, 50) if tile: response.content_type = 'application/vnd.mapbox-vector-tile' return tile else: return HTTPError(404)
def user(db, lang, username=None, format=None): params = query._params() if username: params.users = utils.pg_escape(username.decode("utf-8")).split(",") params.limit = 500 params.full = True username = "******".join(params.users) if not params.users: return template('byuser/index') errors = query._gets(db, params) count = len(errors) if request.path.startswith("/api") or format == "json": out = OrderedDict() out["description"] = ["id", "item", "lat", "lon", "source", "class", "elems", "subclass", "subtitle", "comment", "title", "level", "timestamp", "menu", "username", "date"] for res in errors: res["timestamp"] = str(res["timestamp"]) res["date"] = str(res["date"]) out["byusers"] = errors return out elif format == 'rss': response.content_type = "application/rss+xml" return template('byuser/byuser.rss', username=username, users=params.users, count=count, errors=errors, translate=utils.translator(lang), website=utils.website) else: return template('byuser/byuser', username=username, users=params.users, count=count, errors=errors, translate=utils.translator(lang), website=utils.website, main_website=utils.main_website, remote_url_read=utils.remote_url_read, html_escape=html_escape)
def user(db, lang, username=None, format=None): params = query._params() if username: params.users = utils.pg_escape(username.decode("utf-8")).split(",") params.limit = 500 params.full = True username = "******".join(params.users) if not params.users: return template('byuser/index', translate=utils.translator(lang)) errors = query._gets(db, params) count = len(errors) if request.path.startswith("/api") or format == "json": out = OrderedDict() for res in errors: res["timestamp"] = str(res["timestamp"]) res["lat"] = float(res["lat"]) res["lon"] = float(res["lon"]) out["issues"] = map(dict, errors) return out elif format == 'rss': response.content_type = "application/rss+xml" return template('byuser/byuser.rss', username=username, users=params.users, count=count, errors=errors, translate=utils.translator(lang), website=utils.website) else: return template('byuser/byuser', username=username, users=params.users, count=count, errors=errors, translate=utils.translator(lang), website=utils.website, main_website=utils.main_website, remote_url_read=utils.remote_url_read, html_escape=html_escape)
def errors(db, lang): params = query._params(max_limit=10000) results = query._gets(db, params) translate = utils.translator(lang) out = [] for res in results: i = { 'lat': float(res["lat"]), 'lon': float(res["lon"]), 'id': res["id"], 'item': str(res["item"]), } if params.full: i.update({ 'lat': float(res["lat"]), 'lon': float(res["lon"]), 'id': res["id"], 'item': str(res["item"]), 'source': res["source"], 'classs': res["class"], 'elems': res["elems"], 'subclass': res["subclass"], 'subtitle': translate.select(res["subtitle"]), 'title': translate.select(res["title"]), 'level': res["level"], 'update': str(res["timestamp"]), 'username': res["username"] or None, }) out.append(i) return {'issues': out}
def markers(db): params = query._params() if (not params.users) and (not params.source) and (params.zoom < 6): return params.limit = 200 params.full = False return errors._errors_geo(db, params)
def get_data(db, options): sqlbase = """ SELECT date, SUM(count) FROM ( SELECT date_trunc('day', timestamp) AS date, AVG(count) AS count FROM ( SELECT marker.source, marker.class, marker.count, generate_series( lower(timestamp_range), coalesce(upper(timestamp_range) - '23 hour'::interval, now()), '1 day'::interval )::timestamp without time zone AS timestamp FROM %s WHERE %s ) AS t WHERE %s GROUP BY source, class, date_trunc('day', timestamp) ) AS t GROUP BY date ORDER BY date """ params = query._params() join, where = query._build_param(None, params.source, params.item, params.level, None, params.classs, params.country, params.useDevItem, None, params.tags, None, stats=True, start_date=params.start_date, end_date=params.end_date) where2 = ["1 = 1"] if params.start_date: where2.append("timestamp >= '%s'" % params.start_date.isoformat()) if params.end_date: where2.append("timestamp < '%s'" % params.end_date.isoformat()) where2 = " AND ".join(where2) sql = sqlbase % (join, where, where2) if len(sys.argv)>1: print sql result = [] db.execute(sql) for r in db.fetchall(): result.append((r[0],r[1])) return result
def get_data(db, options): sqlbase = """ SELECT date, SUM(count) FROM ( SELECT date_trunc('day', marker.timestamp) AS date, AVG(marker.count) AS count FROM %s WHERE %s GROUP BY marker.source, marker.class, date ) AS t GROUP BY date ORDER BY date """ params = query._params() join, where = query._build_param(None, None, None, params.source, params.item, params.level, None, params.classs, params.country, params.useDevItem, None, params.tags, None, stats=True, start_date=params.start_date, end_date=params.end_date) sql = sqlbase % (join, where) if len(sys.argv) > 1: print sql result = [] db.execute(sql) for r in db.fetchall(): result.append((r[0], r[1])) return result
def _user_count(db, username=None): params = query._params() if username: params.users = utils.pg_escape(username.decode("utf-8")).split(",") if not params.users: return res = query._count(db, params, ['class.level'], ['class.level']) ret = {1:0, 2:0, 3:0} for (l, c) in res: ret[l] = c return ret
def _user_count(db, username=None): params = query._params() if username: params.users = utils.pg_escape(username.decode("utf-8")).split(",") if not params.users: return res = query._count(db, params, ['dynpoi_class.level'], ['dynpoi_class.level']) ret = {1:0, 2:0, 3:0} for (l, c) in res: ret[l] = c return ret
def user(db, lang, username=None, format=None): params = query._params() if username: params.users = utils.pg_escape(username.decode("utf-8")).split(",") params.limit = 500 params.full = True username = "******".join(params.users) if not params.users: return template('byuser/index') errors = query._gets(db, params) count = len(errors) if request.path.startswith("/api") or format == "json": out = OrderedDict() out["description"] = [ "id", "item", "lat", "lon", "source", "class", "elems", "subclass", "subtitle", "comment", "title", "level", "timestamp", "menu", "username", "date" ] for res in errors: res["timestamp"] = str(res["timestamp"]) res["date"] = str(res["date"]) out["byusers"] = errors return out elif format == 'rss': response.content_type = "application/rss+xml" return template('byuser/byuser.rss', username=username, users=params.users, count=count, errors=errors, translate=utils.translator(lang), website=utils.website) else: return template('byuser/byuser', username=username, users=params.users, count=count, errors=errors, translate=utils.translator(lang), website=utils.website, main_website=utils.main_website, remote_url_read=utils.remote_url_read, html_escape=html_escape)
def markers(db): params = query._params() if (not params.users) and (not params.source) and (params.zoom < 6): return params.limit = 200 params.full = False expires = datetime.datetime.now() + datetime.timedelta(days=365) path = '/'.join(request.fullpath.split('/')[0:-1]) response.set_cookie('last_zoom', str(params.zoom), expires=expires, path=path) response.set_cookie('last_level', str(params.level), expires=expires, path=path) response.set_cookie('last_item', str(params.item), expires=expires, path=path) response.set_cookie('last_tags', str(','.join(params.tags)) if params.tags else '', expires=expires, path=path) response.set_cookie('last_fixable', str(params.fixable) if params.fixable else '', expires=expires, path=path) return errors._errors_geo(db, params)
def markers(db, lang): params = query._params() if (not params.users) and (not params.source) and (params.zoom < 6): return params.limit = 200 params.full = False expires = datetime.datetime.now() + datetime.timedelta(days=365) path = '/'.join(request.fullpath.split('/')[0:-1]) response.set_cookie('last_lat', str(params.lat), expires=expires, path=path) response.set_cookie('last_lon', str(params.lon), expires=expires, path=path) response.set_cookie('last_zoom', str(params.zoom), expires=expires, path=path) response.set_cookie('last_level', str(params.level), expires=expires, path=path) response.set_cookie('last_item', str(params.item), expires=expires, path=path) response.set_cookie('last_tags', str(','.join(params.tags)) if params.tags else '', expires=expires, path=path) response.set_cookie('last_fixable', str(params.fixable) if params.fixable else '', expires=expires, path=path) return errors._errors_geo(db, lang, params)
def issues_mvt(db, z, x, y, format): lon1, lat2 = tiles.tile2lonlat(x, y, z) lon2, lat1 = tiles.tile2lonlat(x + 1, y + 1, z) dlon = (lon2 - lon1) / 256 dlat = (lat2 - lat1) / 256 params = query._params(max_limit=50 if z > 18 else 10000) params.tilex = x params.tiley = y params.zoom = z params.lat = None params.lon = None params.full = False if params.zoom > 18: return if (not params.users) and (not params.source) and (params.zoom < 6): return results = query._gets(db, params) if z >= 6 else None if format == 'mvt': tile = _errors_mvt(db, results, z, lon1, lat1, lon2, lat2, params.limit) if tile: response.content_type = 'application/vnd.mapbox-vector-tile' return tile else: return HTTPError(404) elif format in ('geojson', 'json'): # Fall back to GeoJSON tile = _errors_geojson(db, results, z, lon1, lat1, lon2, lat2, params.limit) if tile: response.content_type = 'application/vnd.geo+json' return tile else: return [] else: return HTTPError(404)
def issues_mvt(db, z, x, y, format): lon1,lat2 = tiles.tile2lonlat(x,y,z) lon2,lat1 = tiles.tile2lonlat(x+1,y+1,z) dlon = (lon2 - lon1) / 256 dlat = (lat2 - lat1) / 256 params = query._params(max_limit=50 if z > 18 else 10000) params.tilex = x params.tiley = y params.zoom = z params.lat = None params.lon = None params.full = False if params.zoom > 18: return if (not params.users) and (not params.source) and (params.zoom < 6): return results = query._gets(db, params) if z >= 6 else None if format == 'mvt': tile = _errors_mvt(db, results, z, lon1, lat1, lon2, lat2, params.limit) if tile: response.content_type = 'application/vnd.mapbox-vector-tile' return tile else: return HTTPError(404) elif format in ('geojson', 'json'): # Fall back to GeoJSON tile = _errors_geojson(db, results, z, lon1, lat1, lon2, lat2, params.limit) if tile: response.content_type = 'application/vnd.geo+json' return tile else: return [] else: return HTTPError(404)
def get_data(db, options): sqlbase = """ SELECT date, SUM(count) FROM ( SELECT date_trunc('day', marker.timestamp) AS date, AVG(marker.count) AS count FROM %s WHERE %s GROUP BY marker.source, marker.class, date_trunc('day', marker.timestamp) ) AS t GROUP BY date ORDER BY date """ params = query._params() join, where = query._build_param(None, params.source, params.item, params.level, None, params.classs, params.country, params.useDevItem, None, params.tags, None, stats=True, start_date=params.start_date, end_date=params.end_date) sql = sqlbase % (join, where) if len(sys.argv)>1: print sql result = [] db.execute(sql) for r in db.fetchall(): result.append((r[0],r[1])) return result
def index(db, lang, format=None): if "false-positive" in request.path: title = _("False positives") gen = "false-positive" elif "done" in request.path: title = _("Fixed issues") gen = "done" else: title = _("Informations") gen = "error" if not format in ('rss', 'gpx', 'kml', 'josm', 'csv'): format = None countries = query_meta._countries(db, lang) if format == None else None items = query_meta._items(db, lang) params = query._params() params.status = {"error":"open", "false-positive": "false", "done":"done"}[gen] params.limit = None params.fixable = None if format == None and params.item: errors_groups = query._count(db, params, [ "dynpoi_class.item", "marker.source", "marker.class", "source.country", "source.analyser", "dynpoi_update_last.timestamp"], [ "dynpoi_item", "class"], [ "min(dynpoi_item.menu::text)::hstore AS menu", "min(class.title::text)::hstore AS title"], ) total = 0 for res in errors_groups: if res["count"] != -1: total += res["count"] else: errors_groups = [] total = 0 params.limit = request.params.get('limit', type=int, default=100) if params.limit > 10000: params.limit = 10000 if (total > 0 and total < 1000) or params.limit: params.full = True errors = query._gets(db, params) if gen in ("false-positive", "done"): opt_date = "date" else: opt_date = "-1" else: opt_date = None errors = None if format == 'rss': response.content_type = 'application/rss+xml' tpl = 'errors/list.rss' elif format == 'gpx': response.content_type = 'application/gpx+xml' tpl = 'errors/list.gpx' elif format == 'kml': response.content_type = 'application/vnd.google-earth.kml+xml' tpl = 'errors/list.kml' elif format == 'josm': objects = [] for res in errors: if res["elems"]: elems = res["elems"].split("_") for e in elems: m = re.match(r"([a-z]+)([0-9]+)", e) if m: cur_type = m.group(1) objects.append(cur_type[0] + m.group(2)) response.status = 302 response.set_header('Location', 'http://localhost:8111/load_object?objects=%s' % ','.join(objects)) return elif format == 'csv': output = StringIO.StringIO() writer = csv.writer(output) h = ['id', 'source', 'item', 'class', 'subclass', 'level', 'title', 'subtitle', 'country', 'analyser', 'timestamp', 'username', 'lat', 'lon', 'elems'] writer.writerow(h) for res in errors: writer.writerow(map(lambda a: res[a], h)) response.content_type = 'text/csv' return output.getvalue() else: tpl = 'errors/index' return template(tpl, countries=countries, items=items, errors_groups=errors_groups, total=total, errors=errors, query=request.query_string, country=params.country, item=params.item, level=params.level, translate=utils.translator(lang), gen=gen, opt_date=opt_date, title=title, website=utils.website, main_website=utils.main_website, remote_url_read=utils.remote_url_read)
def heat(db, z, x, y): COUNT=32 lon1,lat2 = tiles.tile2lonlat(x,y,z) lon2,lat1 = tiles.tile2lonlat(x+1,y+1,z) params = query._params() items = query._build_where_item(params.item, "dynpoi_item") params.tilex = x params.tiley = y params.zoom = z params.lat = None params.lon = None if params.zoom > 18: return db.execute(""" SELECT SUM((SELECT SUM(t) FROM UNNEST(number) t)) FROM dynpoi_item WHERE """ + items) limit = db.fetchone() if limit and limit[0]: limit = float(limit[0]) else: return HTTPError(404) join, where = query._build_param(None, params.source, params.item, params.level, params.users, params.classs, params.country, params.useDevItem, params.status, params.tags, params.fixable, tilex=params.tilex, tiley=params.tiley, zoom=params.zoom) join = join.replace("%", "%%") where = where.replace("%", "%%") sql = """ SELECT COUNT(*), ((lon-%(lon1)s) * %(count)s / (%(lon2)s-%(lon1)s) + 0.5)::int AS latn, ((lat-%(lat1)s) * %(count)s / (%(lat2)s-%(lat1)s) + 0.5)::int AS lonn, mode() WITHIN GROUP (ORDER BY dynpoi_item.marker_color) AS color FROM """ + join + """ WHERE """ + where + """ GROUP BY latn, lonn """ db.execute(sql, {"lon1":lon1, "lat1":lat1, "lon2":lon2, "lat2":lat2, "count":COUNT}) features = [] for row in db.fetchall(): count, x, y, color = row count = max( int(math.log(count) / math.log(limit / ((z-4+1+math.sqrt(COUNT))**2)) * 255), 1 if count > 0 else 0 ) if count > 0: count = 255 if count > 255 else count features.append({ "geometry": Polygon([(x, y), (x - 1, y), (x - 1, y - 1), (x, y - 1)]), "properties": { "color": int(color[1:], 16), "count": count} }) response.content_type = 'application/vnd.mapbox-vector-tile' return mapbox_vector_tile.encode([{ "name": "issues", "features": features }], extents=COUNT)
def heat(db, z, x, y): COUNT = 32 lon1, lat2 = tiles.tile2lonlat(x, y, z) lon2, lat1 = tiles.tile2lonlat(x + 1, y + 1, z) params = query._params() items = query._build_where_item(params.item, "dynpoi_item") params.tilex = x params.tiley = y params.zoom = z params.lat = None params.lon = None if params.zoom > 18: return db.execute(""" SELECT SUM((SELECT SUM(t) FROM UNNEST(number) t)) FROM dynpoi_item WHERE """ + items) limit = db.fetchone() if limit and limit[0]: limit = float(limit[0]) else: return HTTPError(404) join, where = query._build_param(None, params.source, params.item, params.level, params.users, params.classs, params.country, params.useDevItem, params.status, params.tags, params.fixable, tilex=params.tilex, tiley=params.tiley, zoom=params.zoom) join = join.replace("%", "%%") where = where.replace("%", "%%") sql = """ SELECT COUNT(*), ((lon-%(lon1)s) * %(count)s / (%(lon2)s-%(lon1)s) + 0.5)::int AS latn, ((lat-%(lat1)s) * %(count)s / (%(lat2)s-%(lat1)s) + 0.5)::int AS lonn, mode() WITHIN GROUP (ORDER BY dynpoi_item.marker_color) AS color FROM """ + join + """ WHERE """ + where + """ GROUP BY latn, lonn """ db.execute(sql, { "lon1": lon1, "lat1": lat1, "lon2": lon2, "lat2": lat2, "count": COUNT }) features = [] for row in db.fetchall(): count, x, y, color = row count = max( int( math.log(count) / math.log(limit / ((z - 4 + 1 + math.sqrt(COUNT))**2)) * 255), 1 if count > 0 else 0) if count > 0: count = 255 if count > 255 else count features.append({ "geometry": Polygon([(x, y), (x - 1, y), (x - 1, y - 1), (x, y - 1)]), "properties": { "color": int(color[1:], 16), "count": count } }) response.content_type = 'application/vnd.mapbox-vector-tile' return mapbox_vector_tile.encode([{ "name": "issues", "features": features }], extents=COUNT)
def errors(db, lang): params = query._params() return _errors(db, lang, params)
def index(db, lang, format=None): if request.path.endswith("false-positive"): title = _("False positives") gen = "false-positive" elif request.path.endswith("done"): title = _("Fixed issues") gen = "done" else: title = _("Informations") gen = "error" if not format in ('rss', 'gpx', 'josm'): format = None countries = query_meta._countries(db, lang) if format == None else None items = query_meta._items(db, lang) params = query._params() params.status = {"error":"open", "false-positive": "false", "done":"done"}[gen] params.limit = None if format == None and params.item: errors_groups = query._count(db, params, [ "dynpoi_class.item", "dynpoi_class.source", "dynpoi_class.class", "source.country", "source.analyser"], ["dynpoi_item"], [ "first(dynpoi_item.menu) AS menu", "first(dynpoi_class.title) AS title"], orderBy = True) total = 0 for res in errors_groups: if res["count"] != -1: total += res["count"] else: errors_groups = [] total = 0 params.limit = request.params.get('limit', type=int, default=100) if params.limit > 10000: params.limit = 10000 if (total > 0 and total < 1000) or params.limit: params.full = True errors = query._gets(db, params) if gen in ("false-positive", "done"): opt_date = "date" else: opt_date = "-1" else: opt_date = None errors = None if format == 'rss': response.content_type = 'application/rss+xml' tpl = 'errors/list.rss' elif format == 'gpx': response.content_type = 'application/gpx+xml' tpl = 'errors/list.gpx' elif format == 'josm': objects = [] for res in errors: if res["elems"]: elems = res["elems"].split("_") for e in elems: m = re.match(r"([a-z]+)([0-9]+)", e) if m: cur_type = m.group(1) objects.append(cur_type[0] + m.group(2)) response.status = 302 response.set_header('Location', 'http://localhost:8111/load_object?objects=%s' % ','.join(objects)) return else: tpl = 'errors/index' return template(tpl, countries=countries, items=items, errors_groups=errors_groups, total=total, errors=errors, query=request.query_string, country=params.country, item=params.item, translate=utils.translator(lang), gen=gen, opt_date=opt_date, title=title, website=utils.website)
def _users(db): params = query._params() return query._count(db, params, ["marker_elem.username"])
def heat(db, z, x, y): x2,y1 = num2deg(x,y,z) x1,y2 = num2deg(x+1,y+1,z) params = query._params() params.bbox = [y1, x1, y2, x2] items = query._build_where_item(params.item, "dynpoi_item") db.execute(""" SELECT SUM((SELECT SUM(t) FROM UNNEST(number) t)) FROM dynpoi_item WHERE """ + items) max = db.fetchone() if max and max[0]: max = float(max[0]) else: response.content_type = 'image/png' return static_file("images/tile-empty.png", root='static') join, where = query._build_param(None, None, params.bbox, params.source, params.item, params.level, params.users, params.classs, params.country, params.useDevItem, params.status, params.tags, params.fixable) join = join.replace("%", "%%") where = where.replace("%", "%%") COUNT=32 sql = """ SELECT COUNT(*), (((lon-%(y1)s))*%(count)s/(%(y2)s-%(y1)s)-0.5)::int AS latn, (((lat-%(x1)s))*%(count)s/(%(x2)s-%(x1)s)-0.5)::int AS lonn FROM """ + join + """ WHERE """ + where + """ GROUP BY latn, lonn """ db.execute(sql, {"x1":x1, "y1":y1, "x2":x2, "y2":y2, "count":COUNT}) im = Image.new("RGB", (256,256), "#ff0000") draw = ImageDraw.Draw(im) transparent_area = (0,0,256,256) mask = Image.new('L', im.size, color=255) mask_draw = ImageDraw.Draw(mask) mask_draw.rectangle(transparent_area, fill=0) for row in db.fetchall(): count, x, y = row count = int(math.log(count) / math.log(max / ((z-4+1+math.sqrt(COUNT))**2)) * 255) count = 255 if count > 255 else count r = [(x*256/COUNT,(COUNT-1-y)*256/COUNT), ((x+1)*256/COUNT-1,((COUNT-1-y+1)*256/COUNT-1))] mask_draw.rectangle(r, fill=count) im.putalpha(mask) del draw buf = StringIO.StringIO() im.save(buf, 'PNG') response.content_type = 'image/png' return buf.getvalue()
def index(db, lang, format=None): if request.path.endswith("false-positive"): title = _("False positives") gen = "false-positive" elif request.path.endswith("done"): title = _("Fixed issues") gen = "done" else: title = _("Informations") gen = "error" if not format in ('rss', 'gpx', 'josm'): format = None countries = query_meta._countries(db, lang) if format == None else None items = query_meta._items(db, lang) params = query._params() params.status = { "error": "open", "false-positive": "false", "done": "done" }[gen] params.limit = None if format == None and params.item: errors_groups = query._count( db, params, [ "dynpoi_class.item", "dynpoi_class.source", "dynpoi_class.class", "source.country", "source.analyser" ], ["dynpoi_item"], [ "min(dynpoi_item.menu::text)::hstore AS menu", "min(dynpoi_class.title::text)::hstore AS title" ], orderBy=True) total = 0 for res in errors_groups: if res["count"] != -1: total += res["count"] else: errors_groups = [] total = 0 params.limit = request.params.get('limit', type=int, default=100) if params.limit > 10000: params.limit = 10000 if (total > 0 and total < 1000) or params.limit: params.full = True errors = query._gets(db, params) if gen in ("false-positive", "done"): opt_date = "date" else: opt_date = "-1" else: opt_date = None errors = None if format == 'rss': response.content_type = 'application/rss+xml' tpl = 'errors/list.rss' elif format == 'gpx': response.content_type = 'application/gpx+xml' tpl = 'errors/list.gpx' elif format == 'josm': objects = [] for res in errors: if res["elems"]: elems = res["elems"].split("_") for e in elems: m = re.match(r"([a-z]+)([0-9]+)", e) if m: cur_type = m.group(1) objects.append(cur_type[0] + m.group(2)) response.status = 302 response.set_header( 'Location', 'http://localhost:8111/load_object?objects=%s' % ','.join(objects)) return else: tpl = 'errors/index' return template(tpl, countries=countries, items=items, errors_groups=errors_groups, total=total, errors=errors, query=request.query_string, country=params.country, item=params.item, translate=utils.translator(lang), gen=gen, opt_date=opt_date, title=title, website=utils.website, main_website=utils.main_website, remote_url_read=utils.remote_url_read)
def heat(db, z, x, y): COUNT=32 x2,y1 = num2deg(x,y,z) x1,y2 = num2deg(x+1,y+1,z) params = query._params() params.bbox = [y1, x1, y2, x2] items = query._build_where_item(params.item, "dynpoi_item") db.execute(""" SELECT SUM((SELECT SUM(t) FROM UNNEST(number) t)) FROM dynpoi_item WHERE """ + items) max = db.fetchone() if max and max[0]: max = float(max[0]) else: response.content_type = 'image/png' return static_file("images/tile-empty.png", root='static') join, where = query._build_param(None, None, params.bbox, params.source, params.item, params.level, params.users, params.classs, params.country, params.useDevItem, params.status, params.tags, params.fixable) join = join.replace("%", "%%") where = where.replace("%", "%%") sql = """ SELECT COUNT(*), (((lon-%(y1)s))*%(count)s/(%(y2)s-%(y1)s)-0.5)::int AS latn, (((lat-%(x1)s))*%(count)s/(%(x2)s-%(x1)s)-0.5)::int AS lonn, mode() WITHIN GROUP (ORDER BY dynpoi_item.marker_color) AS color FROM """ + join + """ WHERE """ + where + """ GROUP BY latn, lonn """ db.execute(sql, {"x1":x1, "y1":y1, "x2":x2, "y2":y2, "count":COUNT}) im = Image.new("RGB", (256,256), "#ff0000") draw = ImageDraw.Draw(im) transparent_area = (0,0,256,256) mask = Image.new('L', im.size, color=255) mask_draw = ImageDraw.Draw(mask) mask_draw.rectangle(transparent_area, fill=0) for row in db.fetchall(): count, x, y, color = row count = int(math.log(count) / math.log(max / ((z-4+1+math.sqrt(COUNT))**2)) * 255) count = 255 if count > 255 else count count = count r = [(x*256/COUNT,(COUNT-1-y)*256/COUNT), ((x+1)*256/COUNT-1,((COUNT-1-y+1)*256/COUNT-1))] mask_draw.rectangle(r, fill=count) draw.rectangle(r, fill=color) im.putalpha(mask) del draw buf = StringIO.StringIO() im.save(buf, 'PNG') response.content_type = 'image/png' return buf.getvalue()
def heat(db, z, x, y): COUNT=32 x2,y1 = num2deg(x,y,z) x1,y2 = num2deg(x+1,y+1,z) params = query._params() params.bbox = [y1, x1, y2, x2] items = query._build_where_item(params.item, "dynpoi_item") db.execute(""" SELECT SUM((SELECT SUM(t) FROM UNNEST(number) t)) FROM dynpoi_item WHERE """ + items) limit = db.fetchone() if limit and limit[0]: limit = float(limit[0]) else: global MVT_EMPTY if not MVT_EMPTY: MVT_EMPTY = mapbox_vector_tile.encode([]) response.content_type = 'application/vnd.mapbox-vector-tile' return MVT_EMPTY join, where = query._build_param(params.bbox, params.source, params.item, params.level, params.users, params.classs, params.country, params.useDevItem, params.status, params.tags, params.fixable) join = join.replace("%", "%%") where = where.replace("%", "%%") sql = """ SELECT COUNT(*), ((lon-%(y1)s) * %(count)s / (%(y2)s-%(y1)s) + 0.5)::int AS latn, ((lat-%(x1)s) * %(count)s / (%(x2)s-%(x1)s) + 0.5)::int AS lonn, mode() WITHIN GROUP (ORDER BY dynpoi_item.marker_color) AS color FROM """ + join + """ WHERE """ + where + """ GROUP BY latn, lonn """ db.execute(sql, {"x1":x1, "y1":y1, "x2":x2, "y2":y2, "count":COUNT}) features = [] for row in db.fetchall(): count, x, y, color = row count = max( int(math.log(count) / math.log(limit / ((z-4+1+math.sqrt(COUNT))**2)) * 255), 1 if count > 0 else 0 ) if count > 0: count = 255 if count > 255 else count features.append({ "geometry": Polygon([(x, y), (x - 1, y), (x - 1, y - 1), (x, y - 1)]), "properties": { "color": int(color[1:], 16), "count": count} }) response.content_type = 'application/vnd.mapbox-vector-tile' return mapbox_vector_tile.encode([{ "name": "issues", "features": features }], extents=COUNT)
def index(db, lang, format=None): if "false-positive" in request.path: title = _("False positives") gen = "false-positive" elif "done" in request.path: title = _("Fixed issues") gen = "done" else: title = _("Informations") gen = "error" if not format in ('rss', 'gpx', 'kml', 'josm', 'csv'): format = None countries = query_meta._countries(db, lang) if format == None else None items = query_meta._items(db, lang) params = query._params() params.status = {"error":"open", "false-positive": "false", "done":"done"}[gen] params.limit = None params.fixable = None if format == None and params.item: errors_groups = query._count(db, params, [ "dynpoi_class.item", "marker.source", "marker.class", "source.country", "source.analyser", "dynpoi_update_last.timestamp"], [ "dynpoi_item", "class"], [ "min(dynpoi_item.menu::text)::hstore AS menu", "min(class.title::text)::hstore AS title"], orderBy = True) total = 0 for res in errors_groups: if res["count"] != -1: total += res["count"] else: errors_groups = [] total = 0 params.limit = request.params.get('limit', type=int, default=100) if params.limit > 10000: params.limit = 10000 if (total > 0 and total < 1000) or params.limit: params.full = True errors = query._gets(db, params) if gen in ("false-positive", "done"): opt_date = "date" else: opt_date = "-1" else: opt_date = None errors = None if format == 'rss': response.content_type = 'application/rss+xml' tpl = 'errors/list.rss' elif format == 'gpx': response.content_type = 'application/gpx+xml' tpl = 'errors/list.gpx' elif format == 'kml': response.content_type = 'application/vnd.google-earth.kml+xml' tpl = 'errors/list.kml' elif format == 'josm': objects = [] for res in errors: if res["elems"]: elems = res["elems"].split("_") for e in elems: m = re.match(r"([a-z]+)([0-9]+)", e) if m: cur_type = m.group(1) objects.append(cur_type[0] + m.group(2)) response.status = 302 response.set_header('Location', 'http://localhost:8111/load_object?objects=%s' % ','.join(objects)) return elif format == 'csv': output = StringIO.StringIO() writer = csv.writer(output) h = ['id', 'source', 'item', 'class', 'subclass', 'level', 'title', 'subtitle', 'country', 'analyser', 'timestamp', 'username', 'lat', 'lon', 'elems'] writer.writerow(h) for res in errors: writer.writerow(map(lambda a: res[a], h)) response.content_type = 'text/csv' return output.getvalue() else: tpl = 'errors/index' return template(tpl, countries=countries, items=items, errors_groups=errors_groups, total=total, errors=errors, query=request.query_string, country=params.country, item=params.item, level=params.level, translate=utils.translator(lang), gen=gen, opt_date=opt_date, title=title, website=utils.website, main_website=utils.main_website, remote_url_read=utils.remote_url_read)