This blog will largely focus on my SURF about Botnet detection under Prof. Julian Bunn. I’m just getting started but my first task (which I am still far from completing after 2 days) was to develop a diagnostic tool that takes a pcap file as input and outputs a network/multigraph with machines (IP addresses) as nodes and the packet transfers + date/time + protocol + payload length as edges.
First I looked up existing softwares that take in pcap files as input. There are many of them (https://github.com/caesar0301/awesome-pcaptools). As an example, CapAnalysis outputs the following (like many of the others):
- Source IP vs Flows, Destination IP vs Flows, Source IP vs Data Sent/Received, Destination IP vs Data Sent/Received, Protocols vs Flows, Country vs Flows, Duration vs Flows
- Timeline of Connections, Data, Data Received, Data Sent
Some of the other softwares also have anomaly-based inspection (not necessarily related to botnets) and thus identify potential threats. Some of the other softwares that seem somewhat useful are Libcrafter (https://github.com/pellegre/libcrafter), Snort (https://www.snort.org/), Bro – yes that is the name of the software (https://www.bro.org/), and AIEngine (https://bitbucket.org/camp0/aiengine).
None of these softwares really seemed to display a graph of the network and I’m not sure how to leverage the other components though some of these are open source, so I decided to start from scratch.
Ideally, this diagnostic tool would be in Java since it would like nicer and be cleaner than Python for example. And thus began a near ten-hour enterprise accomplishing little:
I would probably use the Java package jNetPcap to read in the pcap files and process them. There are lots of libraries that supposedly visualize graphs such as JGraph, JUNG, GraphStream, Gephi, Java 3D, etc. I decided to use jzy3d since it sounded snazzy and seemed pretty. Eventually I’d probably use Swing to create the GUI, somehow…
I spent hours simply installing Eclipse Neon for Java on my laptop (Ubuntu 16.04). There is some bug such that the installer fails with no warning or error message; you click install and it makes you wait for a minute and accomplishes nothing except for printing out a log file. Finally I found the solution 3/4ths of the way down an Eclipse Community forum:
“You can download eclipse in the old-fashion way, that is, download the zip containing some specific package. If you go to https://www.eclipse.org/downloads, click on “Download Packages” just below of “Eclipse Neon” – “Download 64 bit”. On the next page, you will be able to select your preferred package (C/C++, java, php, etc) and download it in Neon eclipse version.” (https://www.eclipse.org/forums/index.php/t/1083012/).
Next I spent a few hours trying to figure out how to install/import packages required for jzy3d from https://github.com/jzy3d/jzy3d-graphs and https://github.com/jzy3d/jzy3d-api – following the instructions from http://www.jzy3d.org/plugins-graphs.php. Unfortunately I made no actual progress on this front: I learned there were a few extra things I needed to take care of but they ultimately didn’t solve all the problems. I’ll write them anyway:
- Clone/download and extract the repositories from the github links above
- Add version – get version by running
mvn -Dplugin=groupId:artifactId help:describe– for
build-helper-maven-plugin (version 3.0.0) and maven-surefire-plugin (version 2.20) to the pom.xml for jzy3d-graphs folder only (not the jzy3d-api folder)
mvn install jzy3d-api
- I couldn’t get the
mvn install jzy3d-graphto work and so got stuck. I have no idea how to really lay out the Build Path for this package.
I tried the tutorial on https://wadeawalker.wordpress.com/2010/10/09/tutorial-a-cross-platform-workbench-program-using-java-opengl-and-eclipse/ – I got 288 errors and 31 warnings.
Out of desperation now I asked my very first question on StackOverflow. Within 30 minutes and 5 views, someone downvoted my question and told me to make it precise and readable (see above for a screenshot). That was the end of my experience with jzy3d.
Now I began to second-guess my installation of Java. I have version 1.8.0_131, which I think I installed with some pain also (because the widely written about method no longer works since the links that were accessed from the terminal have been removed from the official website). I am pretty sure that’s the newest update from jdk 8, so I concluded I was fine here.
Now I’m debating continuing with Java and perhaps using JGraph for the visualization (which again will be difficult because it is Java) and JGraphT for the graph analysis (which I realized I didn’t think about at all before while trying to use jzy3d). My other option is to use Python; I’m fairly confident I can read pcap files and generate a graph of the network using Python but it won’t look good and it won’t be as clean as I first imagined.