def diff_behavior(ctxfn1: FunctionInfo, ctxfn2: FunctionInfo, options: AnalysisOptions) -> Union[str, List[BehaviorDiff]]: fn1, sig1 = ctxfn1.callable() fn2, sig2 = ctxfn2.callable() debug("Resolved signature:", sig1) all_diffs: List[BehaviorDiff] = [] half1, half2 = options.split_limits(0.5) with condition_parser(options.analysis_kind), Patched(): # We attempt both orderings of functions. This helps by: # (1) avoiding code path explosions in one of the functions # (2) using both signatures (in case they differ) all_diffs.extend(diff_behavior_with_signature(fn1, fn2, sig1, half1)) all_diffs.extend( diff.reverse() for diff in diff_behavior_with_signature(fn2, fn1, sig2, half2)) debug("diff candidates:", all_diffs) # greedily pick results: result_diffs = [] opcodeset1 = set(i.offset for i in dis.get_instructions(fn1.__code__)) opcodeset2 = set(i.offset for i in dis.get_instructions(fn2.__code__)) while all_diffs: scorer = diff_scorer(opcodeset1, opcodeset2) selection = max(all_diffs, key=scorer) (num_opcodes, _) = scorer(selection) debug("Considering input", selection.args, " num opcodes=", num_opcodes) if num_opcodes == 0: break all_diffs.remove(selection) result_diffs.append(selection) coverage1, coverage2 = selection.coverage1, selection.coverage2 if coverage1 is not None and coverage2 is not None: opcodeset1 -= coverage1.offsets_covered opcodeset2 -= coverage2.offsets_covered return result_diffs
def diffbehavior(args: argparse.Namespace, options: AnalysisOptions, stdout: TextIO, stderr: TextIO) -> int: def checked_load(qualname: str) -> Optional[FunctionInfo]: try: objs = list(load_files_or_qualnames([qualname])) except Exception as exc: print(f'Unable to load "{qualname}": {exc}', file=stderr) return None obj = objs[0] if not isinstance(obj, FunctionInfo): print(f'"{qualname}" does not target a function.', file=stderr) return None return obj (fn_name1, fn_name2) = (args.fn1, args.fn2) fn1 = checked_load(fn_name1) fn2 = checked_load(fn_name2) if fn1 is None or fn2 is None: return 2 options.stats = collections.Counter() diffs = diff_behavior(fn1, fn2, options) debug("stats", options.stats) if isinstance(diffs, str): print(diffs, file=stderr) return 2 elif len(diffs) == 0: num_paths = options.stats["num_paths"] exhausted = options.stats["exhaustion"] > 0 stdout.write( f"No differences found. (attempted {num_paths} iterations)\n") if exhausted: stdout.write( "All paths exhausted, functions are likely the same!\n") else: stdout.write( "Consider trying longer with: --per_condition_timeout=<seconds>\n" ) return 0 else: width = max(len(fn_name1), len(fn_name2)) + 2 for diff in diffs: inputs = ", ".join(f"{k}={v}" for k, v in diff.args.items()) stdout.write(f"Given: ({inputs}),\n") result1, result2 = diff.result1, diff.result2 differing_args = result1.get_differing_arg_mutations(result2) stdout.write( f"{fn_name1.rjust(width)} : {result1.describe(differing_args)}\n" ) stdout.write( f"{fn_name2.rjust(width)} : {result2.describe(differing_args)}\n" ) return 1
def diff_behavior_with_signature( fn1: Callable, fn2: Callable, sig: inspect.Signature, options: AnalysisOptions ) -> Iterable[BehaviorDiff]: search_root = SinglePathNode(True) condition_start = time.monotonic() for i in range(1, options.max_iterations): debug("Iteration ", i) itr_start = time.monotonic() if itr_start > condition_start + options.per_condition_timeout: debug( "Stopping due to --per_condition_timeout=", options.per_condition_timeout, ) return options.incr("num_paths") space = StateSpace( execution_deadline=itr_start + options.per_path_timeout, model_check_timeout=options.per_path_timeout / 2, search_root=search_root, ) with StateSpaceContext(space): output = None try: (verification_status, output) = run_iteration(fn1, fn2, sig, space) except UnexploredPath: verification_status = VerificationStatus.UNKNOWN debug("Verification status:", verification_status) top_analysis, space_exhausted = space.bubble_status( CallAnalysis(verification_status) ) if ( top_analysis and top_analysis.verification_status == VerificationStatus.CONFIRMED ): debug("Stopping due to code path exhaustion. (yay!)") options.incr("exhaustion") break if output: yield output
def cover(args: argparse.Namespace, options: AnalysisOptions, stdout: TextIO, stderr: TextIO) -> int: ctxfn = checked_load(args.fn, stderr) if ctxfn is None: return 2 options.stats = collections.Counter() paths = path_cover(ctxfn, options, args.coverage_type) fn, _ = ctxfn.callable() example_output_format = args.example_output_format if example_output_format == ExampleOutputFormat.ARGUMENT_DICTIONARY: return output_argument_dictionary_paths(fn, paths, stdout, stderr) elif example_output_format == ExampleOutputFormat.EVAL_EXPRESSION: return output_eval_exression_paths(fn, paths, stdout, stderr) if example_output_format == ExampleOutputFormat.PYTEST: return output_pytest_paths(fn, paths, stdout, stderr) assert False
def diffbehavior(args: argparse.Namespace, options: AnalysisOptions, stdout: TextIO, stderr: TextIO) -> int: (fn_name1, fn_name2) = (args.fn1, args.fn2) fn1 = checked_load(fn_name1, stderr) fn2 = checked_load(fn_name2, stderr) if fn1 is None or fn2 is None: return 2 options.stats = collections.Counter() diffs = diff_behavior(fn1, fn2, options) debug("stats", options.stats) if isinstance(diffs, str): print(diffs, file=stderr) return 2 elif len(diffs) == 0: num_paths = options.stats["num_paths"] exhausted = options.stats["exhaustion"] > 0 stdout.write( f"No differences found. (attempted {num_paths} iterations)\n") if exhausted: stdout.write( "All paths exhausted, functions are likely the same!\n") else: stdout.write( "Consider trying longer with: --per_condition_timeout=<seconds>\n" ) return 0 else: width = max(len(fn_name1), len(fn_name2)) + 2 for diff in diffs: inputs = ", ".join(f"{k}={v}" for k, v in diff.args.items()) stdout.write(f"Given: ({inputs}),\n") result1, result2 = diff.result1, diff.result2 differing_args = result1.get_differing_arg_mutations(result2) stdout.write( f"{fn_name1.rjust(width)} : {result1.describe(differing_args)}\n" ) stdout.write( f"{fn_name2.rjust(width)} : {result2.describe(differing_args)}\n" ) return 1