This repository includes scripts to analyze local traffic of smart home IoT devices for our IMC23 paper.
For more details about our paper: https://github.com/Android-Observatory/IoT-LAN

To be updated soon!

• devices.txt: a list of all devices with their MAC addresses. Add router manually.
• analysis.py: the main script to call.
• tagged_analysis:
• protocol_statistics.py:
• analyser: analysis files
  • protocols_analysis.py: protocl-wise analysis
  • plotting.py: plots charts.
  • flow_extraction_new.py: extract flows from traffic
  • utils.py, constants.py: auxiliary files.
  • periodic_analysis.py: peroidicity analysis
  • protocol_identification.py: automatically identify top-layer protocols from pcaps.
  • all_device_analysis.py: deprecated. protocol distribution, basic analysis, plotting, etc.
  • protocols: !TODO
  • backups.py, flow_extraction.py: backup file.
  • vis.py: graph visualization helper.
• scripts:
  • extract_ca.py: extracts certificates from TLS traffic.
• helper: some other scripts
  • setup-log.py: get device-ip mapping from DHCP logs
  • device-to-phone.py: which device communicates with mobile phone
• connectedGraph:
  • graph_generator.py: generates connected graph visualizations.
• logs: nohup log files. To remove
• vis: deprecated. html graph visualizations for all_device_analysis
• outputs: extracted certificates. Need to rename this folder.

python3 analysis.py ~/2022-datasets/idle-dataset-dec ~/local_output/idle-dataset-dec-new/

python3 tagged_analysis.py ~/2022-datasets/tagged-local-new ~/local_output/tagged-dec/

python3 analysis.py ~/2022-datasets/idle-dataset-dec ~/local_output/idle-dataset-dec/ -a bcmc/eth_unicast/unicast/ipv6

python3 protocol_statistics.py ~/2022-datasets/idle-dataset-dec ~/local_output/idle-dataset-dec/ -plot _overall_manual_processed.txt

python3 -u analysis.py ~/2022-datasets/idle-dataset-dec ~/local_output/idle-dataset-dec-new/ -f PROTOCOL

python3 analyser/protocol_parser/dhcp.py python3 analyser/protocol_parser/PROTOCOL.py

python3 analyser/periodic_analysis.py /home/hutr/local_output/idle-dataset-dec-new/flow_burst/ /home/hutr/local_output/idle-dataset-dec-new/periodic_detection/

python3 connectedGraph/graph_generator.py ~/local_output/idle-dataset-dec/tcp_output vis python3 connectedGraph/graph_generator.py ~/local_output/idle-dataset-dec/udp_output vis

python3 connectedGraph/merged_html.py

python3 scripts/cdf.py

scripts/figure/ipynb

python3 scripts/scanning_response.py ~/local_output/idle-dataset-dec-new/flow_burst/ ~/local_output/idle-dataset-dec-new/scanning_response_2/

scanning_table_maker.ipynb

Scanning device parser: BCMC protocols per device. Input: bcmc/bcmc/new_packet_count python3 scripts/scanning_device_parser.py

Scanning response results parser: python3 scripts/scanning_response_results_parser.py

python3 scripts/ip_traffic_exclude_router.py ~/local_output/idle-dataset-dec-new/tcp_output/ ~/local_output/idle-dataset-dec-new/udp_output/ ~/local_output/idle-dataset-dec-new/ip_traffic_exclude_router

python3 scripts/periodic_log_parser.py ~/local_output/idle-dataset-dec-new/periodic_detection/ ~/local_output/idle-dataset-dec-new/periodic_detection_results/

python3 scripts/protocol_ports.py

python3 scripts/extract_ca.py /home/hutr/2022-datasets/idle-dataset-dec /home/hutr/local-traffic-analysis/outputs/tls_dec.txt

python3 scripts/raw_tshark_flow_protocol.py in_dir out_dir

python3 scripts/ndpi_results_parser.py

python3 scripts/tagged_graph_difference.py ~/local_output/idle-dataset-dec-new/ ~/local_output/tagged-dec/ ./output_diff

python3 scripts/tagged_dataset_stat.py

python3 helper/device-to-phone.py ~/local_output/idle-dataset-dec/packet_count/ .

python3 helper/get_device_ip_addr.py

python3 helper setup_log.py

python3 analysis.py ~/2022-datasets/idle-dataset ~/local_output/idle-dataset/ -b