def getreviews(logindata, gamedata,page=1): if page == 1: for game in gamedata: if "review" in gamedata[game]: del gamedata[game]["reviewed"] try: appinfofile = urllib2.urlopen(utils.getprofileurl(logindata) + "/recommended/?p=" + str(page)) doc = bs(appinfofile) for el in doc.findAll("div", {"class": "review_box"}): # header = el.find("div", {"class": "header"}) # try: # print re.search(r'([0-9]+) of ([0-9]+) people \(([0-9]+)%',header.text).groups() # except: appid = re.search(r'\d+',el.find("div",{"class":"leftcol"}).find("a")["href"]).group() review = re.search(r'(?<=thumbs).*(?=\.)',el.find("div",{"class":"rightcol"}).find("div",{"class":"thumb"}).find("a").find("img")["src"]).group() gamedata[appid]["reviewed"] = review.lower() paging = doc.find("div", {"class": "workshopBrowsePagingInfo"}) revpos = re.search(r'([0-9]+) of ([0-9]+)',paging.text) if revpos.groups()[0] != revpos.groups()[1]: getreviews(logindata, gamedata, page+1) except: pass
def getprofile(logindata, gamedata): def getval(xmlval, default): if xmlval is not None: return xmlval.text else: return default usock = urllib2.urlopen(utils.getprofileurl(logindata) + "/games/?tab=all&xml=1") try: xmldoc = ET.parse(usock) root = xmldoc.getroot() for game in root.find("games").findall("game"): appid = game.find("appID").text if appid in gamedata: hrs = getval(game.find("hoursOnRecord"), None) if hrs is not None: gamedata[appid]["timeplayed"] = hrs hrs2wks = getval(game.find("hoursLast2Weeks"), None) if hrs2wks is not None: gamedata[appid]["timeplayed2wk"] = hrs2wks except: pass