def _checkBaseClassInit(moduleFilename, c, func_code, funcInfo) : """ Return a list of warnings that occur for each base class whose __init__() is not called @param funcInfo: triple of functions called, code objects, return values @type funcInfo: triple """ warnings = [] functionsCalled, _, returnValues = funcInfo for line, stackItem, dummy in returnValues : if stackItem.data != None : if not stackItem.isNone() or cfg().returnNoneFromInit : warn = Warning(func_code, line, msgs.RETURN_FROM_INIT) warnings.append(warn) classInit = getattr(c.classObject, utils.INIT, None) if cfg().baseClassInitted and classInit is not None : classInitInfo = _get_func_info(classInit) for base in getattr(c.classObject, '__bases__', None) or (): if not _baseInitCalled(classInitInfo, base, functionsCalled): warn = Warning(moduleFilename, func_code, msgs.BASE_CLASS_NOT_INIT % utils.safestr(base)) warnings.append(warn) return warnings
def updateCheckerArgs(argStr, func, lastLineNum, warnings): """ @param argStr: list of space-separated options, as passed on the command line e.g 'blacklist=wrongname initattr no-classdoc' @type argStr: str @param func: 'suppressions' or code object @type func: str or {function.FakeCode} or {types.CodeType} @param lastLineNum: the last line number of the given function; compare to func.co_firstlineno if exists @type lastLineNum: int or {types.CodeType} @param warnings: list of warnings to append to @type warnings: list of L{Warning} @rtype: int @returns: 1 if the arguments were invalid, 0 if ok. """ try: argList = string.split(argStr) # if func is code, might trigger # TypeError: code.__cmp__(x,y) requires y to be a 'code', not a 'str' if argList and not type(func) == str: debug('func %r: pychecker args %r', func, argStr) # don't require long options to start w/--, we can add that for them for i in range(0, len(argList)): if argList[i][0] != '-': argList[i] = '--' + argList[i] cfg().processArgs(argList) return 1 except Config.UsageError, detail: # this gets triggered when parsing a bad __pychecker__ declaration warn = Warning(func, lastLineNum, msgs.INVALID_CHECKER_ARGS % detail) warnings.append(warn) return 0
def _getUnused(module, globalRefs, dict, msg, filterPrefix=None): """ Return a list of warnings for unused globals. For example, for modules: from twisted.internet import defer would result in 'defer' -> PyCheckerModule at 0x...: twisted.internet.defer @param globalRefs: dict of token alias -> full name of token aliases that have been used ? @type globalRefs: dict of str -> str @param dict: dict of token alias -> token containing the tokens to check for use @type dict: dict of str -> token @param msg: the message template to use to create warnings @type msg: str from L{msgs} """ import pprint warnings = [] for ref in dict.keys(): check = not filterPrefix or utils.startswith(ref, filterPrefix) # FIXME: do a ref in globalRefs instead? if check and globalRefs.get(ref) == None: lineInfo = module.moduleLineNums.get(ref) if lineInfo: warnings.append(Warning(lineInfo[0], lineInfo[1], msg % ref)) return warnings
def _checkOverridenMethods(func, baseClasses, warnings) : for baseClass in baseClasses : if func.func_name != utils.INIT and \ not function.same_signature(func, baseClass) : err = msgs.METHOD_SIGNATURE_MISMATCH % (func.func_name, utils.safestr(baseClass)) warnings.append(Warning(func.func_code, func.func_code, err)) break
def load(self, warnings=None): try: # there's no need to reload modules we already have global _output, _statusDlg, _count txt = _("Loading Module %s\n") % self.moduleName _output.AddLines(txt) _count += 1 if _count == 100: _count = 95 _statusDlg.Update(_count, txt) module = sys.modules.get(self.moduleName) if module: if not _allModules[self.moduleName].module: return self._initModule(module) return 1 return self._initModule(self.setupMainCode()) except (SystemExit, KeyboardInterrupt): exc_type, exc_value, exc_tb = sys.exc_info() raise exc_type, exc_value except SyntaxError, (message, (fileName, line, col, text)): # ActiveGrid: added this for better feedback when module couldn't be loaded. w = Warning( self.fullpath, line, _("Syntax Error: %s\n%s\n%s^error near here") % (message, text, ' ' * (col - 1))) warnings.append(w) return 0
def processFiles(files, cfg=None, pre_process_cb=None): # insert this here, so we find files in the local dir before std library if sys.path[0] != '': sys.path.insert(0, '') # ensure we have a config object, it's necessary global _cfg if cfg is not None: _cfg = cfg elif _cfg is None: _cfg = Config.Config() if _cfg.ignoreImportErrors: install_ignore__import__() warnings = [] utils.initConfig(_cfg) for file, (moduleName, moduleDir) in zip(files, getModules(files)): if callable(pre_process_cb): pre_process_cb("module %s (%s)" % (moduleName, file)) oldsyspath = sys.path[:] sys.path.insert(0, moduleDir) module = PyCheckerModule(moduleName, moduleDir=moduleDir) if not module.load(): w = Warning(module.filename(), 1, msgs.Internal("NOT PROCESSED UNABLE TO IMPORT")) warnings.append(w) sys.path = oldsyspath utils.popConfig() return warnings
def _getUnused(module, globalRefs, dict, msg, filterPrefix = None) : "Return a list of warnings for unused globals" warnings = [] for ref in dict.keys() : check = not filterPrefix or utils.startswith(ref, filterPrefix) if check and globalRefs.get(ref) == None : lineInfo = module.moduleLineNums.get(ref) if lineInfo: warnings.append(Warning(lineInfo[0], lineInfo[1], msg % ref)) return warnings
def processFiles(files, cfg=None, pre_process_cb=None): """ @type files: list of str @type cfg: L{Config.Config} @param pre_process_cb: callable notifying of module name, filename @type pre_process_cb: callable taking (str, str) """ warnings = [] # insert this here, so we find files in the local dir before std library if sys.path[0] != '': sys.path.insert(0, '') # ensure we have a config object, it's necessary global _cfg if cfg is not None: _cfg = cfg elif _cfg is None: _cfg = Config.Config() if _cfg.ignoreImportErrors: install_ignore__import__() utils.initConfig(_cfg) utils.debug('Processing %d files' % len(files)) for file, (moduleName, moduleDir) in zip(files, getModules(files)): if callable(pre_process_cb): pre_process_cb("module %s (%s)" % (moduleName, file)) # create and load the PyCheckerModule, tricking sys.path temporarily oldsyspath = sys.path[:] if moduleDir is not None: sys.path.insert(0, moduleDir) pcmodule = pcmodules.PyCheckerModule(moduleName, moduleDir=moduleDir) loaded = pcmodule.load() sys.path = oldsyspath if not loaded: w = Warning(pcmodule.filename(), 1, msgs.Internal("NOT PROCESSED UNABLE TO IMPORT")) warnings.append(w) utils.debug('Processed %d files' % len(files)) utils.popConfig() return warnings
def updateCheckerArgs(argStr, func, lastLineNum, warnings) : try : argList = string.split(argStr) # don't require long options to start w/--, we can add that for them for i in range(0, len(argList)) : if argList[i][0] != '-' : argList[i] = '--' + argList[i] cfg().processArgs(argList) return 1 except Config.UsageError, detail : warn = Warning(func, lastLineNum, msgs.INVALID_CHECKER_ARGS % detail) warnings.append(warn) return 0
def _findFunctionWarnings(module, globalRefs, warnings, suppressions): for func in module.functions.values(): func_code = func.function.func_code utils.debug("function:", func_code) name = '%s.%s' % (module.moduleName, func.function.__name__) suppress = getSuppression(name, suppressions, warnings) if cfg().noDocFunc and func.function.__doc__ == None: err = msgs.NO_FUNC_DOC % func.function.__name__ warnings.append(Warning(module.filename(), func_code, err)) _checkNoSelfArg(func, warnings) _updateFunctionWarnings(module, func, None, warnings, globalRefs) if suppress is not None: utils.popConfig()
def _findFunctionWarnings(module, globalRefs, warnings, suppressions) : """ @type module: L{pychecker.checker.PyCheckerModule} """ for func in module.functions.values() : func_code = func.function.func_code utils.debug("function:", func_code) name = '%s.%s' % (module.moduleName, func.function.__name__) suppress = getSuppression(name, suppressions, warnings) if cfg().noDocFunc and func.function.__doc__ == None : err = msgs.NO_FUNC_DOC % func.function.__name__ # FIXME: is there a good reason why this passes func_code as line ? warnings.append(Warning(module.filename(), func_code, err)) _checkNoSelfArg(func, warnings) _updateFunctionWarnings(module, func, None, warnings, globalRefs) if suppress is not None : utils.popConfig()
def removeWarnings(warnings, blacklist, std_lib, cfg): if std_lib is not None: std_lib = normalize_path(std_lib) for index in range(len(warnings) - 1, -1, -1): filename = normalize_path(warnings[index].file) if filename in blacklist or (std_lib is not None and utils.startswith(filename, std_lib)): del warnings[index] elif cfg.only: # ignore files not specified on the cmd line if requested if os.path.abspath(filename) not in cfg.files: del warnings[index] # filter by warning/error level if requested if cfg.level and warnings[index].level < cfg.level: del warnings[index] if cfg.limit: # sort by severity first, then normal sort (by file/line) warnings.sort(lambda a, b: cmp(a.level, b.level) or cmp(a, b)) # strip duplicates lastWarning = None for index in range(len(warnings) - 1, -1, -1): warning = warnings[index] # remove duplicate warnings if lastWarning is not None and cmp(lastWarning, warning) == 0: del warnings[index] else: lastWarning = warning num_ignored = len(warnings) - cfg.limit if num_ignored > 0: del warnings[:-cfg.limit] msg = msgs.TOO_MANY_WARNINGS % num_ignored warnings.append(Warning('', 0, msg)) return warnings
def processFiles(files, cfg=None, pre_process_cb=None): # insert this here, so we find files in the local dir before std library if sys.path[0] != '': sys.path.insert(0, '') # ensure we have a config object, it's necessary global _cfg if cfg is not None: _cfg = cfg elif _cfg is None: _cfg = Config.Config() warnings = [] utils.initConfig(_cfg) for moduleName in getModules(files): if callable(pre_process_cb): pre_process_cb(moduleName) module = PyCheckerModule(moduleName) if not module.load(): w = Warning(module.filename(), 1, msgs.Internal("NOT PROCESSED UNABLE TO IMPORT")) warnings.append(w) utils.popConfig() return warnings
def _checkSelfArg(method, warnings) : """Return a Warning if there is no self parameter or the first parameter to a method is not self.""" if not cfg().methodArgName: return code = method.function.func_code err = None if method.isStaticMethod(): if code.co_argcount > 0 and cfg().methodArgName == code.co_varnames[0]: err = msgs.SELF_IS_ARG % 'staticmethod' elif code.co_argcount < 1: err = msgs.NO_METHOD_ARGS % cfg().methodArgName else: if method.isClassMethod(): if code.co_varnames[0] not in cfg().classmethodArgNames: err = msgs.SELF_NOT_FIRST_ARG % \ (cfg().classmethodArgNames, 'class') elif code.co_varnames[0] != cfg().methodArgName: err = msgs.SELF_NOT_FIRST_ARG % (cfg().methodArgName, '') if err is not None : warnings.append(Warning(code, code, err))
def find(moduleList, initialCfg, suppressions=None): "Return a list of warnings found in the module list" if suppressions is None : suppressions = {}, {} utils.initConfig(initialCfg) utils.debug('Finding warnings in %d modules' % len(moduleList)) warnings = [] before = 0 for module in moduleList : if module.moduleName in cfg().blacklist : continue modSuppress = getSuppression(module.moduleName, suppressions, warnings) globalRefs, classCodes = {}, {} # mainCode can be null if there was a syntax error if module.mainCode != None : utils.debug("module:", module) before = len(warnings) funcInfo = _updateFunctionWarnings(module, module.mainCode, None, warnings, globalRefs, 1) if before != len(warnings): utils.debug("module: %r __main__ triggered %d warnings", module, len(warnings) - before) for code in funcInfo[1] : classCodes[code.co_name] = code before = len(warnings) _findFunctionWarnings(module, globalRefs, warnings, suppressions) if before != len(warnings): utils.debug("module: %r functions triggered %d warnings", module, len(warnings) - before) before = len(warnings) for c in module.classes.values(): _findClassWarnings(module, c, classCodes.get(c.name), globalRefs, warnings, suppressions) if before != len(warnings): utils.debug("module: %r classes triggered %d warnings", module, len(warnings) - before) if cfg().noDocModule and \ module.module != None and module.module.__doc__ == None: warnings.append(Warning(module.filename(), 1, msgs.NO_MODULE_DOC)) utils.debug("module: %r module doc triggered 1 warning") before = len(warnings) if cfg().allVariablesUsed or cfg().privateVariableUsed: prefix = None if not cfg().allVariablesUsed: prefix = "_" for ignoreVar in cfg().variablesToIgnore + cfg().unusedNames: globalRefs[ignoreVar] = ignoreVar warnings.extend(_getUnused(module, globalRefs, module.variables, msgs.VAR_NOT_USED, prefix)) if before != len(warnings): utils.debug("module: %r unused variables triggered %d warnings", module, len(warnings) - before) before = len(warnings) if cfg().importUsed: if module.moduleName != utils.INIT or cfg().packageImportUsed: # always ignore readline module, if [raw_]input() is used if globalRefs.has_key('input') or \ globalRefs.has_key('raw_input'): globalRefs['readline'] = 0 warnings.extend(_getUnused(module, globalRefs, module.modules, msgs.IMPORT_NOT_USED)) if before != len(warnings): utils.debug("module: %r unused imports triggered %d warnings", module, len(warnings) - before) # we have to do this here, b/c checkFunction doesn't popConfig for # classes this allows us to have __pychecker__ apply to all methods # when defined at class scope if module.mainCode != None: utils.popConfig() if modSuppress is not None: utils.popConfig() std_lib = None if cfg().ignoreStandardLibrary: std_lib = getStandardLibraries() ret = removeWarnings(warnings, getBlackList(cfg().blacklist), std_lib, cfg()) utils.debug('Found %d warnings in %d modules' % (len(ret), len(moduleList))) return ret
def _findClassWarnings(module, c, class_code, globalRefs, warnings, suppressions) : utils.debug("class:", class_code) try: className = utils.safestr(c.classObject) except TypeError: # goofy __getattr__ return classSuppress = getSuppression(className, suppressions, warnings) baseClasses = c.allBaseClasses() for base in baseClasses : baseModule = utils.safestr(base) if '.' in baseModule : # make sure we handle import x.y.z packages = string.split(baseModule, '.') baseModuleDir = string.join(packages[:-1], '.') globalRefs[baseModuleDir] = baseModule # handle class variables if class_code is not None : func = function.create_fake(c.name, class_code) _updateFunctionWarnings(module, func, c, warnings, globalRefs, 0, 1) filename = module.filename() func_code = None for method in c.methods.values() : if method == None : continue func_code = method.function.func_code utils.debug("class %s: method:" % className, func_code) try: name = utils.safestr(c.classObject) + '.' + method.function.func_name except AttributeError: # func_name may not exist continue methodSuppress = getSuppression(name, suppressions, warnings) if cfg().checkSpecialMethods: funcname = method.function.func_name if funcname[:2] == '__' == funcname[-2:] and \ funcname != '__init__': err = None argCount = python.SPECIAL_METHODS.get(funcname, -1) if argCount != -1: # if the args are None, it can be any # of args if argCount is not None: minArgs = maxArgs = argCount err = CodeChecks.getFunctionArgErr(funcname, func_code.co_argcount, minArgs, maxArgs) else: err = msgs.NOT_SPECIAL_METHOD % funcname if err is not None: warnings.append(Warning(filename, func_code, err)) if cfg().checkOverridenMethods : _checkOverridenMethods(method.function, baseClasses, warnings) if cfg().noDocFunc and method.function.__doc__ == None : err = msgs.NO_FUNC_DOC % method.function.__name__ # FIXME: is there a good reason why this passes func_code as line ? warnings.append(Warning(filename, func_code, err)) _checkSelfArg(method, warnings) tmpModule = _getModuleFromFilename(module, func_code.co_filename) funcInfo = _updateFunctionWarnings(tmpModule, method, c, warnings, globalRefs) if func_code.co_name == utils.INIT : # this is a constructor if utils.INIT in dir(c.classObject) : warns = _checkBaseClassInit(filename, c, func_code, funcInfo) warnings.extend(warns) elif cfg().initDefinedInSubclass : err = msgs.NO_INIT_IN_SUBCLASS % c.name warnings.append(Warning(filename, c.getFirstLine(), err)) if methodSuppress is not None : utils.popConfig() if c.memberRefs and cfg().membersUsed : memberList = c.memberRefs.keys() memberList.sort() err = msgs.UNUSED_MEMBERS % (string.join(memberList, ', '), c.name) warnings.append(Warning(filename, c.getFirstLine(), err)) try: newStyleClass = issubclass(c.classObject, object) except TypeError: # FIXME: perhaps this should warn b/c it may be a class??? newStyleClass = 0 slots = c.statics.get('__slots__') if slots is not None and cfg().slots: lineNum = c.lineNums['__slots__'] if not newStyleClass: err = msgs.USING_SLOTS_IN_CLASSIC_CLASS % c.name warnings.append(Warning(filename, lineNum, err)) elif cfg().emptySlots: try: if len(slots.data) == 0: err = msgs.EMPTY_SLOTS % c.name warnings.append(Warning(filename, lineNum, err)) except AttributeError: # happens when slots is an instance of a class w/o __len__ pass if not newStyleClass and property is not None and cfg().classicProperties: for static in c.statics.keys(): if type(getattr(c.classObject, static, None)) == property: err = msgs.USING_PROPERTIES_IN_CLASSIC_CLASS % (static, c.name) warnings.append(Warning(filename, c.lineNums[static], err)) coerceMethod = c.methods.get('__coerce__') if newStyleClass and coerceMethod: lineNum = coerceMethod.function.func_code.co_firstlineno err = msgs.USING_COERCE_IN_NEW_CLASS % c.name warnings.append(Warning(filename, lineNum, err)) for newClassMethodName in python.NEW_STYLE_CLASS_METHODS: newClassMethod = c.methods.get(newClassMethodName) if not newStyleClass and newClassMethod: lineNum = newClassMethod.function.func_code.co_firstlineno err = msgs.USING_NEW_STYLE_METHOD_IN_OLD_CLASS % (newClassMethodName, c.name) warnings.append(Warning(filename, lineNum, err)) if cfg().noDocClass and c.classObject.__doc__ == None : method = c.methods.get(utils.INIT, None) if method != None : func_code = method.function.func_code # FIXME: check to make sure this is in our file, # not a base class file??? err = msgs.NO_CLASS_DOC % c.classObject.__name__ warnings.append(Warning(filename, func_code, err)) # we have to do this here, b/c checkFunction doesn't popConfig for classes # this allows us to have __pychecker__ apply to all methods # when defined at class scope if class_code is not None : utils.popConfig() if classSuppress is not None : utils.popConfig()
def _checkNoSelfArg(func, warnings) : "Return a Warning if there is a self parameter to a function." code = func.function.func_code if code.co_argcount > 0 and cfg().methodArgName in code.co_varnames: warnings.append(Warning(code, code, msgs.SELF_IS_ARG % 'function'))
def removeWarnings(warnings, blacklist, std_lib, cfg): """ @param blacklist: list of absolute paths not to warn for @type blacklist: str @param std_lib: list of standard library directories @type std_lib: list of str or None """ utils.debug('filtering %d warnings with blacklist', len(warnings)) if std_lib is not None: std_lib = [normalize_path(p) for p in std_lib] for index in range(len(warnings) - 1, -1, -1): filename = normalize_path(warnings[index].file) # the blacklist contains paths to packages and modules we do not # want warnings for # when we find a match, make sure we continue the warnings for loop found = False for path in blacklist: if not found and filename.startswith(path): found = True del warnings[index] if found: continue if std_lib: found = False for path in std_lib: if not found and utils.startswith(filename, path) : found = True del warnings[index] if found: continue elif cfg.only: # ignore files not specified on the cmd line if requested if os.path.abspath(filename) not in cfg.files: del warnings[index] continue # filter by warning/error level if requested if cfg.level and warnings[index].level < cfg.level: del warnings[index] if cfg.limit: # sort by severity first, then normal sort (by file/line) warnings.sort(lambda a, b: cmp(a.level, b.level) or cmp(a, b)) # strip duplicates lastWarning = None for index in range(len(warnings)-1, -1, -1): warning = warnings[index] # remove duplicate warnings if lastWarning is not None and cmp(lastWarning, warning) == 0: del warnings[index] else: lastWarning = warning num_ignored = len(warnings) - cfg.limit if num_ignored > 0: del warnings[:-cfg.limit] msg = msgs.TOO_MANY_WARNINGS % num_ignored warnings.append(Warning('', 0, msg)) utils.debug('kept %d warnings with blacklist', len(warnings)) return warnings
def find(moduleList, initialCfg, suppressions=None): "Return a list of warnings found in the module list" if suppressions is None: suppressions = {}, {} utils.initConfig(initialCfg) warnings = [] for module in moduleList: if module.moduleName in cfg().blacklist: continue modSuppress = getSuppression(module.moduleName, suppressions, warnings) globalRefs, classCodes = {}, {} # main_code can be null if there was a syntax error if module.main_code != None: funcInfo = _updateFunctionWarnings(module, module.main_code, None, warnings, globalRefs, 1) for code in funcInfo[1]: classCodes[code.co_name] = code _findFunctionWarnings(module, globalRefs, warnings, suppressions) for c in module.classes.values(): _findClassWarnings(module, c, classCodes.get(c.name), globalRefs, warnings, suppressions) if cfg().noDocModule and \ module.module != None and module.module.__doc__ == None : warnings.append(Warning(module.filename(), 1, msgs.NO_MODULE_DOC)) if cfg().allVariablesUsed or cfg().privateVariableUsed: prefix = None if not cfg().allVariablesUsed: prefix = "_" for ignoreVar in cfg().variablesToIgnore + cfg().unusedNames: globalRefs[ignoreVar] = ignoreVar warnings.extend( _getUnused(module, globalRefs, module.variables, msgs.VAR_NOT_USED, prefix)) if cfg().importUsed: if module.moduleName != utils.INIT or cfg().packageImportUsed: # always ignore readline module, if [raw_]input() is used if globalRefs.has_key('input') or \ globalRefs.has_key('raw_input'): globalRefs['readline'] = 0 warnings.extend( _getUnused(module, globalRefs, module.modules, msgs.IMPORT_NOT_USED)) if module.main_code != None: utils.popConfig() if modSuppress is not None: utils.popConfig() std_lib = None if cfg().ignoreStandardLibrary: std_lib = getStandardLibraries() return removeWarnings(warnings, getBlackList(cfg().blacklist), std_lib, cfg())
return self._initModule(self.setupMainCode()) except (SystemExit, KeyboardInterrupt): exc_type, exc_value, exc_tb = sys.exc_info() raise exc_type, exc_value except SyntaxError, (message, (fileName, line, col, text)): # ActiveGrid: added this for better feedback when module couldn't be loaded. w = Warning( self.fullpath, line, _("Syntax Error: %s\n%s\n%s^error near here") % (message, text, ' ' * (col - 1))) warnings.append(w) return 0 except: exc_type, exc_value, exc_tb = sys.exc_info() w = Warning( self.moduleName, 1, "%s: %s.\nUnable to import module %s." % (exc_type, exc_value, self.moduleName)) warnings.append(w) importError(self.moduleName) return 0 def initModule(self, module): if not self.module: filename = _getPyFile(module.__file__) if string.lower(filename[-3:]) == '.py': try: file = open(filename) except IOError: pass else: self._setupMainCode(file, filename, module)