Offline Threat Intelligence Analyzer for extracting artifacts and IoCs from file/dump into readable format
- Runs locally (Offline)
- Analyze buffer, file or full folder
- Intime analysis (Session is saved)
- 2 modes (Interactive and silent)
- Generates HTML or JSON as output
- Dump output file with details to mongodb
- Save raw json result to mongodb
- Basic file information MD5, charset, mime, ssdeep
- Different string/patterns analysis methods
- NL English words detection
- OCR words detection
- IPS hints and countries description
- Ports hints
- World IPS world image and flags
- DNS servers description (Top servers)
- Websites similarity detection (Top 10000)
- Artifacts force directed image
- Cross references force directed image and table
- MITRE att&ck tools and patterns detection (could be FP)
- Similarity image divided to classes
- YARA module and YARA rules included (Downloaded a copy from yara-rules-github)
- YARA module includes conditions
- Yara tags by index
- URL/EMAIL/TEL/Tags patterns extraction
- Credit Cards patterns extraction
- Credential patterns extraction
- Encryption patterns (base64, md5, sha1..) extraction
- DGA (Domain Generation Algorithm) patterns extraction
- BOM (Byte Order Mark) detection
- URL shorteners extraction
- ASCII extraction from UNICODE
- Whitelist implemented (Windows7, 8 and 10 files)
- Check WAF and bypass proxy
- Free/Fake email extraction
- Spelling and punctuation check
- Top phishing words included
- Snort support
- Web interface
- Supports threat intelligence platform feeds
- Linux (wrapper)
- ELF information
- API functions descriptions
- System commands descriptions
- Sections descriptions
- Lib descriptions
- Encrypted section detection
- Symbols extraction
- MITRE artifacts mapped to detection
- Cross references detection
- Behavior detection
- macOS (wrapper)
- DMG extraction
- Shell code detection
- PLIST information
- MITRE artifacts mapped to detection
- macOS information
- Windows (wrapper)
- PE information
- Encrypted section detection
- Sections descriptions
- DLL descriptions
- Symbols extraction
- Signature extraction and validation
- API descriptions
- PE ASLR, DEP, SEH and CFG detection
- MITRE artifacts mapped to detection
- API Behavior detection (DLL injection, Process Hollowing, Process Doppelganging etc..)
- Cross references detection
- Icon extraction
- Extract String file info (FileDescription, FileDescription etc..)
- Android (wrapper)
- APK information
- DEX information
- Manifest descriptions
- Intent descriptions
- Resources extraction
- Symbols extraction
- Classes extraction
- Big functions identification
- Cross references detection
- API Behavior detection
- IPhone (built-in)
- IPA information
- BlackBerry (COD) (built-in)
- COD information
- Functions extraction
- Strings extraction
- PCAP (wrapper)
- Frame filter
- HTTP filter
- DNS filter
- ARP filter
- WAF detection
- DGA detection
- Snort parsing
- PDF (built-in)
- Objects enumeration
- Keys (javascript, js, OpenAction) extraction
- Streams parsing
- String analysis
- Office (built-in and wrapper)
- Meta info extraction
- Hyper and target links extraction
- Bin printable parser
- Extract Text
- Extract DDE
- Macros extraction
- OLE (wrapper)
- Number of objects
- Object extraction
- Macros extraction
- EMAIL (built-in and wrapper)
- Header information
- Attachment extraction and parsing
- Extract body
- Phishing patterns check
- Archives (wrapper)
- Extract mimes and guess by extensions
- Finding patterns in all unpacked files
- Encrypted archives detection
- HTML (wrapper)
- Extract scripts, iframes, links and forms
- Decode/analyze links
- Script entropy
- Online TIPs (Required tokens, Moving to different project)
- HybridAnalysis
- MalShare
- MetaDefender
- VirusTotal
- AlienVault
- PulseDive
- ☑
Reduce file I/O - ☑
PDF module - ☑
RTF module - ☑
Fix htmlmaker (return concat(self.root_render_func(self.new_context(vars))) MemoryError) due to rendering large objects.. this happened due to yara module appending too many results that caused htmlmaker to hang . Solved by grouping yara results into one - ☑
HTML module - ☑
Refactoring modules v2 - ☑
Converting some yara rules into individual modules (Requested by users) - ☑
Whitelist (Requested by users) - ☑
Switching to mongodb (Requested by users) - ☑
Phishing module - ☑
Web service and API - ☑
Web interface (Requested by users) - ☑
Curling some TIPs (Requested by users) - ☑
MS office module - ☑
Snort wrapper (Requested by users) - ☑
Machine learning modules - Moving to different project - ☑
Offline multiscanner - Moving to different project - Java analysis (Requested by users)
- Web detection
- Adding username and password wrappers to databases
git clone https://github.com/qeeqbox/analyzer.git
cd analyzer
chmod +x run.sh
./run.sh auto_configure
The project interface http://127.0.0.1:8000/login/ will open automatically after finishing the initialization process
docker-compose -f docker-compose-dev.yml up --build
Then open http://127.0.0.1:8000/login/
apt-get install -y python3 python3-pip curl libfuzzy-dev yara libmagic-dev libjansson-dev libssl-dev libffi-dev tesseract-ocr libtesseract-dev libssl-dev swig p7zip-full radare2 dmg2img mongodb redis
pip3 install pyelftools macholib python-magic nltk Pillow jinja2 ssdeep pefile scapy r2pipe pytesseract M2Crypto requests tld tldextract bs4 psutil pymongo flask pyOpenSSL oletools extract_msg
Prerequisites packages are required for some modules (If you are having issues using those packages, I might be able to share with you my own alternatives that I developed in the past in C#\C)
- Linux documentation
- MacOS documentation
- Windows documentation
- Android documentation
- software77
- MITRE ATT&CK™
- sc0ty
- hexacorn
- PEID
- cisco umbrella
- yara rules community
- TONS OF RESEARCHES.. (Please let me know if i missed a resource or dependency)
By using this framework, you are accepting the license terms of each package listed below:
- https://github.com/arbor/yara/blob/master/LICENSE
- https://github.com/Yara-Rules/rules/blob/master/LICENSE
- https://github.com/tesseract-ocr/tesseract/blob/master/LICENSE
- http://www.swig.org/Release/LICENSE
- https://www.radare.org/r/license.html
- http://vu1tur.eu.org/tools/LICENSE
- https://github.com/decalage2/oletools/wiki/License
- https://www.mongodb.com/community/licensing
- https://github.com/Supervisor/supervisor/blob/master/COPYRIGHT.txt
- https://github.com/mattgwwalker/msg-extractor/blob/master/LICENSE.txt
- https://www.snort.org/license
- https://github.com/eliben/pyelftools/blob/master/LICENSE
- https://bitbucket.org/ronaldoussoren/macholib/src/default/LICENSE
- https://github.com/erocarrera/pefile/blob/master/LICENSE
- https://github.com/secdev/scapy/blob/master/LICENSE
- https://github.com/ahupp/python-magic/blob/master/LICENSE
- https://flask.palletsprojects.com/en/0.12.x/license/
- https://github.com/pallets/werkzeug/blob/master/LICENSE.rst
- https://github.com/benoitc/gunicorn/blob/master/LICENSE
- https://github.com/MongoEngine/flask-mongoengine/blob/master/LICENSE
- https://github.com/flask-admin/flask-admin/blob/master/LICENSE
- https://github.com/maxcountryman/flask-login/blob/master/LICENSE
- https://github.com/maxcountryman/flask-bcrypt/blob/master/LICENSE
- https://github.com/pyca/pyopenssl/blob/master/LICENSE
- https://github.com/dcolish/flask-markdown/blob/master/LICENSE
- https://github.com/barseghyanartur/tld/blob/master/LICENSE_GPL2.0.txt
- https://github.com/giampaolo/psutil/blob/master/LICENSE
- https://github.com/gevent/gevent/blob/master/LICENSE
- https://github.com/dateutil/dateutil/blob/master/LICENSE
- https://requests.readthedocs.io/en/master/user/intro/
- https://github.com/mher/pymongo/blob/master/LICENSE
- https://www.crummy.com/software/BeautifulSoup/
- https://github.com/john-kurkowski/tldextract/blob/master/LICENSE
- https://gitlab.com/m2crypto/m2crypto/-/blob/master/LICENCE
- https://github.com/madmaze/pytesseract/blob/master/LICENSE
- https://github.com/radareorg/radare2-r2pipe/blob/master/dotnet/LICENSE
- https://ssdeep-project.github.io/ssdeep/index.html
- https://github.com/Kronuz/jinja2/blob/master/LICENSE
- https://github.com/python-pillow/Pillow/blob/master/LICENSE
- https://github.com/nltk/nltk/blob/develop/LICENSE.txt
- http://p7zip.sourceforge.net/
- https://redislabs.com/legal/licenses/
- https://github.com/andymccurdy/redis-py/blob/master/LICENSE
- Do not deploy without proper configuration
- Setup some security group rules and remove default credentials
- This project is NOT an anti malware project and does not quarantine or delete malicious files