def main(): args=argparse.Namespace() try: # get all available modules args.modules=get_modules() # parse args from cli and configuration file # arguments parsed from cli will cover origin arguments in configuration file configfile_parse(args) cli_parse(args) begin_banner() init_logger(args.logfile,args.verbose,args.disable_col) except SwarmBaseException as e: print str(e) end_banner() return # now use logger instead of simple print try: m=MSwarm(args) # wake slaves up now m.waken_swarm() m.parse_distribute_task() except SwarmBaseException as e: LOG.critical(str(e)) finally: end_banner()
def _parse_charset(self): try: charset=self._args.domain_charset while True: index=charset.find('-') if index==-1: break begin_chr=charset[index-1] end_chr=charset[index+1] dst='' for x in range(ord(begin_chr),ord(end_chr)+1): dst+=chr(x) charset=charset.replace(begin_chr+'-'+end_chr,dst) ret = ''.join(x for i, x in enumerate(charset) if charset.index(x) == i) LOG.debug('charset: %s'%ret) return ret except Exception, e: LOG.critical('invalid subdomain name charset, or format error') # raise SwarmUseException('invalid subdomain name charset, or format error') raise
def _parse_charset(self): try: charset = self._args.domain_charset while True: index = charset.find('-') if index == -1: break begin_chr = charset[index - 1] end_chr = charset[index + 1] dst = '' for x in range(ord(begin_chr), ord(end_chr) + 1): dst += chr(x) charset = charset.replace(begin_chr + '-' + end_chr, dst) ret = ''.join(x for i, x in enumerate(charset) if charset.index(x) == i) LOG.debug('charset: %s' % ret) return ret except Exception, e: LOG.critical('invalid subdomain name charset, or format error') # raise SwarmUseException('invalid subdomain name charset, or format error') raise
def getlist(target='',target_file=''): """ Return integrated ip and domain name list from target list and file, with network segment parsed. """ try: LOG.info('begin to parse target list') iplist=[] if target!='': iplist.extend(target) if target_file!='': f=open(target_file,'r') targets=f.read() iplist.extend(targets.splitlines()) f.close() # parse network segment and check iplist=_unite_list(iplist) LOG.info('parse completed') return iplist except socket.timeout, e: LOG.critical('time out when parsing target') raise SwarmNetException('time out when parsing target')
def getswarmlist(swarm='',swarm_file=''): """ Return integrated ip and domain name list with port list from swarm list and file like (['127.0.0.1','127.0.0.2','github.com'],[80,90,90]). """ try: LOG.info('begin to parse swarm list') rawlist=[] iplist=[] portlist=[] if swarm!='': rawlist.extend(swarm) if swarm_file!='': f=open(swarm_file,'r') swarm=f.read() rawlist.extend(swarm.splitlines()) f.close() iplist,portlist=_unite_swarmlist(rawlist) LOG.info('parse completed') return iplist,portlist except socket.timeout, e: LOG.critical('time out when parsing target') raise SwarmNetException('time out when parsing target')
def scan_domain(self): """ Decomposition domain name scan task and distribute tasks, get result from swarm. Task granularity should not be too small or too huge Task format: __doms__:task_index:domain name:dict:dict_path:start_line:scan_lines __doms__:task_index:domain name:comp:charset:begin_str:end_str Result format: __doms__:task_index:result Example: put task: __doms__:26:github.com:dict:2000:3000 __doms__:980:github.com:comp:ABCDEFGHIJKLMN8980:DAAA:D000 get result: __doms__:980:gist.github.com;XX.github.com __doms__:980:no subdomain """ scan_result=[] domain_list=removeip(self._target_list) if len(domain_list)==0: LOG.critical('domain name must be provided') raise SwarmUseException('domain name must be provided') if self._args.domain_maxlevel<=0: LOG.critical('subdomain name max level must be positive') raise SwarmUseException('subdomain name max level must be positive') # begin to discomposition for curlevel in range(self._args.domain_maxlevel): self._init_task_statistics() if self._args.domain_compbrute==True: # parse interval of subdomain name length try: midindex=self._args.domain_levellen.find('-') minlen=int(self._args.domain_levellen[:midindex],10) maxlen=int(self._args.domain_levellen[midindex+1:],10) except Exception, e: LOG.critical('invalid subdomain name length interval, or format error') raise SwarmUseException('invalid subdomain name length interval, or format error') # parse char set charset=self._parse_charset() task_granularity=4 for cur_target in domain_list: begin_str='' if maxlen<task_granularity: begin_str=minlen*charset[0] end_str=maxlen*charset[-1] task=':'.join([cur_target,'comp',charset,begin_str,end_str]) self._put_task('__doms__',task) continue if minlen<task_granularity: begin_str=minlen*charset[0] end_str=(task_granularity-1)*charset[-1] task=':'.join([cur_target,'comp',charset,begin_str,end_str]) self._put_task('__doms__',task) bflist=generate_bflist(charset,charset[0],(maxlen-task_granularity+1)*charset[-1]) for pre_str in bflist: begin_str=pre_str+(task_granularity-1)*charset[0] end_str=pre_str+(task_granularity-1)*charset[-1] task=':'.join([cur_target,'comp',charset,begin_str,end_str]) self._put_task('__doms__',task) # use dictionary else: if self._args.domain_dict=='': LOG.critical('domain name dictionary need to be provided') raise SwarmUseException('domain name dictionary need to be provided') try: # get total number of dictionary lines with open(self._args.domain_dict) as fp: sumline=sum(1 for i in fp) except IOError, e: LOG.critical('can not open dictionary for domain scan, path:%s'%(self._args.domain_dict)) raise SwarmUseException('can not open dictionary for domain scan, path:%s'%(self._args.domain_dict)) task_granularity=1000 for cur_target in domain_list: # begin from fisrt line cur_line=1 while True: # after next calculation, cur_line point to next start line if (cur_line+task_granularity)>sumline: lines=sumline-cur_line+1 else: lines=task_granularity task=':'.join([cur_target,'dict',self._args.domain_dict,str(cur_line),str(lines)]) self._put_task('__doms__',task) # update current line cur_line+=task_granularity if cur_line>sumline: break
def scan_domain(self): """ Decomposition domain name scan task and distribute tasks, get result from swarm. Task granularity should not be too small or too huge Task format: __doms__:task_index:domain name:dict:dict_path:start_line:scan_lines __doms__:task_index:domain name:comp:charset:begin_str:end_str Result format: __doms__:task_index:result Example: put task: __doms__:26:github.com:dict:2000:3000 __doms__:980:github.com:comp:ABCDEFGHIJKLMN8980:DAAA:D000 get result: __doms__:980:gist.github.com;XX.github.com __doms__:980:no subdomain """ scan_result = [] domain_list = removeip(self._target_list) if len(domain_list) == 0: LOG.critical('domain name must be provided') raise SwarmUseException('domain name must be provided') if self._args.domain_maxlevel <= 0: LOG.critical('subdomain name max level must be positive') raise SwarmUseException( 'subdomain name max level must be positive') # begin to discomposition for curlevel in range(self._args.domain_maxlevel): self._init_task_statistics() if self._args.domain_compbrute == True: # parse interval of subdomain name length try: midindex = self._args.domain_levellen.find('-') minlen = int(self._args.domain_levellen[:midindex], 10) maxlen = int(self._args.domain_levellen[midindex + 1:], 10) except Exception, e: LOG.critical( 'invalid subdomain name length interval, or format error' ) raise SwarmUseException( 'invalid subdomain name length interval, or format error' ) # parse char set charset = self._parse_charset() task_granularity = 4 for cur_target in domain_list: begin_str = '' if maxlen < task_granularity: begin_str = minlen * charset[0] end_str = maxlen * charset[-1] task = ':'.join( [cur_target, 'comp', charset, begin_str, end_str]) self._put_task('__doms__', task) continue if minlen < task_granularity: begin_str = minlen * charset[0] end_str = (task_granularity - 1) * charset[-1] task = ':'.join( [cur_target, 'comp', charset, begin_str, end_str]) self._put_task('__doms__', task) bflist = generate_bflist(charset, charset[0], (maxlen - task_granularity + 1) * charset[-1]) for pre_str in bflist: begin_str = pre_str + (task_granularity - 1) * charset[0] end_str = pre_str + (task_granularity - 1) * charset[-1] task = ':'.join( [cur_target, 'comp', charset, begin_str, end_str]) self._put_task('__doms__', task) # use dictionary else: if self._args.domain_dict == '': LOG.critical('domain name dictionary need to be provided') raise SwarmUseException( 'domain name dictionary need to be provided') try: # get total number of dictionary lines with open(self._args.domain_dict) as fp: sumline = sum(1 for i in fp) except IOError, e: LOG.critical( 'can not open dictionary for domain scan, path:%s' % (self._args.domain_dict)) raise SwarmUseException( 'can not open dictionary for domain scan, path:%s' % (self._args.domain_dict)) task_granularity = 1000 for cur_target in domain_list: # begin from fisrt line cur_line = 1 while True: # after next calculation, cur_line point to next start line if (cur_line + task_granularity) > sumline: lines = sumline - cur_line + 1 else: lines = task_granularity task = ':'.join([ cur_target, 'dict', self._args.domain_dict, str(cur_line), str(lines) ]) self._put_task('__doms__', task) # update current line cur_line += task_granularity if cur_line > sumline: break
iplist.extend(target) if target_file!='': f=open(target_file,'r') targets=f.read() iplist.extend(targets.splitlines()) f.close() # parse network segment and check iplist=_unite_list(iplist) LOG.info('parse completed') return iplist except socket.timeout, e: LOG.critical('time out when parsing target') raise SwarmNetException('time out when parsing target') except IOError, e: LOG.critical('can not open target file') raise SwarmUseException('can not open target file') except Exception, e: LOG.critical('invalid target or swarm address, or format error') raise SwarmUseException('invalid target or swarm address, or format error') def getswarmlist(swarm='',swarm_file=''): """ Return integrated ip and domain name list with port list from swarm list and file like (['127.0.0.1','127.0.0.2','github.com'],[80,90,90]). """ try: LOG.info('begin to parse swarm list') rawlist=[] iplist=[] portlist=[]