Skip to content

thanhbinhdoan1993/dns-traffic-statistics-agent

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DNS Traffic Statistic Agent

Architecture

Architecture

  • In which, the DNS packets are classified by:
    • Each DNS client IP address – from the incoming queries.
    • Each authoritative DNS server IP address – for the outgoing queries.
  1. Sniffer:

    • Sniff the DNS packets on port 53 for the configured interface.
    • Then put the sniffed packets into Decoder module.
  2. Decoder:

    • Process the message in single channel.
    • Then put the decoded packets into Statistics module
  3. Statistics:

    • Create the map contains all interfaces of the running server.
    • Create the map to store all messages that belong to the client/AS.
    • When the message received, triggers the process to calculate and update the metrics.
    • At the end of the interval time:
      • Trigger the function to continue to do more extra process if need.
      • Export the statistics.
      • Prepare the message then send to the SNMP sub-agent via REST API.
      • Create the message queue to save the out message if there's issue when sending to the SNMP sub-agent.
    • Check the message queue to see if any message need to be sent.
    • Limit the message in queue/disk by using interval time/number of items.
    • Note for Average response time calculation in Statistics module as following:
      • msg_1: avg = value_msg_1
      • msg_2: avg = (avg + value_msg_2)/2
      • msg_3: avg = (avg + value_msg_3)/3
      • msg_4: avg = (avg + value_msg_4)/4
  4. New SNMP Sub-agent:

    • Be responsible to update value to MIB counters whenever receiving the statistics message sent from "Statistics" module in Packetbeat.
    • Collect Per-view statistics using URL provided by BIND statistics-channels and write to MIB counters

Packetbeat installation and usage

Extract the dns-traffic-statistic-agent.tar.gz to a folder in BDDS, then follow the below steps:

Compile

  1. Install golang dependency
    go get ./...
    
  2. Build
    go build
    

Setup packetbeat service

  1. Install packetbeat.deb in setup-package
    dpkg -i packetbeat.deb
    
  2. Open postDeploy.sh from the link /usr/local/bluecat/postDeploy.sh and add line python /usr/share/packetbeat/bin/announcement_bam_deploy.py at the bottom of this file

Configuration Details

  1. packetbeat.yml in /etc/packetbeat/
  • Logging configuration is configured at Logging session in packetbeat.yml file.
  • Configuring packetbeat capture packet at Transaction protocols session
    • On type dns: it will be listen at ports: [53]
    • On type http: it will be listen at ports: [51415], port 51415 (PORT) is SNMP Sub Agent Http port
  1. statistics_config.json in /usr/share/packetbeat/bin/
Key Value Description
statistics_destination http://[IP]:[PORT]/counter [String] IP and PORT of SNMP Sub Agent Http Server
statistics_interval [integer] Interval collecting and sending DNS statistics
maximum_clients [integer] maximum number of clients for statistics, 200 clients is required.
url_announcement_bam_deploy http://[IP]:[PORT]/announcement-deploy-from-bam IP and PORT of SNMP Sub Agent Http Server
interval_clear_outstatis_cache [integer] Interval In Second for cleaning data cached of data statistics which are sending to SNMP Agent

SNMP Subagent installation and usage

Running Packetbeat

  1. Make sure the SNMP SubAgent started first
  2. Running
    /etc/init.d/packetbeat start
    

SNMP Subagent

Configure and install

  1. Copy BCN-DNS-AGENT-MIB.mib into folder /usr/share/snmp/mibs and append "BCN-DNS-AGENT-MIB" end of /etc/snmp/snmp.conf.

  2. Configure snmpd.conf. There are 2 options:

    • Enable and config snmp's information in BAM and deploy to BDDS.
    • Copy snmpd.conf from dns-snmp-agent/ to /etc/snmpd/ and restart snmp service. Note: Run following cmd to prevent snmpd.conf be overwritten automatically via BAM or when snmpd service is restarted:
       /usr/local/bluecat/PsmClient node set manual-override=snmp
      
  3. Configure Master agent to plugin new sub-agent

  • Before running python dns_stat_agent, need to configuare AGENT_CONFIGURATION in config.py matched host and port in snmpd.conf.

    • HTTP_CONFIGURATION:
    Key Value
    host IP address for http server [String]
    port port of http server [integer]
    • AGENT_CONFIGURATION:
    Key Value
    agent_name The agent's name used for registration with net-snmp. [String]
    master_socket defines the address the master agent listens at, or the subagent should connect to. The default is the Unix Domain socket "/var/agentx/master". [String]
    persistence_dir The directory to use to store persistence information. Change this if you want to use a custom snmpd instance, eg. for automatic testing.[String]
    • BIND_CONFIGURATION:
    Key Value
    host Host of bind API. [String]
    port Port of bind API. [String]
    stats_path Path to get statistics in bind.[String]
  1. Install wheel in setup-package
    pip install wheel-0.33.4-py2.py3-none-any.whl
    
  2. Install netsnmpagent in setup-package
    pip install netsnmpagent-0.6.0.tar.gz
    
  3. Configure dns_stat_agent service
    • Copy service to /lib/systemd/system/
       cp dns_stat_agent.service /lib/systemd/system/
      
    • Set permission for dns_stat_agent.service
       chmod -R 644 /lib/systemd/system/dns_stat_agent.service
      
    • In dns_stat_agent.service file, need to change path of dns_stat_agent.py at ExecStart to where dns_stat_agent is stored.
    • Reload daemon and enable
       systemctl daemon-reload
       systemctl enable dns_stat_agent.service
      

Run subagent

  • Start dns_stat_agent service.
service dns_stat_agent start
  • Note: Log files are stored at /var/log/dns_stat_agent/

Get statistic data from mib

Test to get statistic data

  • Table
     snmptable [OPTIONS] <IP> BCN-DNS-AGENT-MIB::<Statistic table name>
    
  • Walk
     snmpwalk [OPTIONS] <IP> BCN-DNS-AGENT-MIB::<Object name>
    
  • Get
     snmpget [OPTIONS] <IP> BCN-DNS-AGENT-MIB::<Object name>.\"<ClientIP/AsIP>\".<Metric type>
    

BAM ACLs Configuration

Create ACLs for Clients and Server

  • The name of Client ACLs have to start with _TrafficStatisticsAgent_Clients Example:

    • _TrafficStatisticsAgent_Clients
    • _TrafficStatisticsAgent_Clients_Region_1
    • _TrafficStatisticsAgent_Clients_test
  • The name of Client ACLs have to start with _TrafficStatisticsAgent_Servers Example:

    • _TrafficStatisticsAgent_Servers
    • _TrafficStatisticsAgent_Servers_Region_1
    • _TrafficStatisticsAgent_Servers_test

ACLs

  • the ACL contains individual host IPs:
    • snmp agent create table entries with all statistics equal to zero
  • the ACL contains only networks:
    • there are no SNMP OIDs be created.

About

DNS Traffic Statistics SNMP Agent

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 85.1%
  • Python 13.6%
  • Thrift 1.2%
  • Other 0.1%