The Beginning – already full of setbacks

This blog will largely focus on my SURF about Botnet detection under Prof. Julian Bunn. I’m just getting started but my first task (which I am still far from completing after 2 days) was to develop a diagnostic tool that takes a pcap file as input and outputs a network/multigraph with machines (IP addresses) as nodes and the packet transfers + date/time + protocol + payload length as edges.

First I looked up existing softwares that take in pcap files as input. There are many of them (https://github.com/caesar0301/awesome-pcaptools). As an example, CapAnalysis outputs the following (like many of the others):

  • Source IP vs Flows, Destination IP vs Flows, Source IP vs Data Sent/Received, Destination IP vs Data Sent/Received, Protocols vs Flows, Country vs Flows, Duration vs Flows
  • Timeline of Connections, Data, Data Received, Data Sent

Some of the other softwares also have anomaly-based inspection (not necessarily related to botnets) and thus identify potential threats. Some of the other softwares that seem somewhat useful are Libcrafter (https://github.com/pellegre/libcrafter), Snort (https://www.snort.org/), Bro – yes that is the name of the software (https://www.bro.org/), and AIEngine (https://bitbucket.org/camp0/aiengine).

None of these softwares really seemed to display a graph of the network and I’m not sure how to leverage the other components though some of these are open source, so I decided to start from scratch.

Ideally, this diagnostic tool would be in Java since it would like nicer and be cleaner than Python for example. And thus began a near ten-hour enterprise accomplishing little:

I would probably use the Java package jNetPcap to read in the pcap files and process them. There are lots of libraries that supposedly visualize graphs such as JGraph, JUNG, GraphStream, Gephi, Java 3D, etc. I decided to use jzy3d since it sounded snazzy and seemed pretty. Eventually I’d probably use Swing to create the GUI, somehow…

I spent hours simply installing Eclipse Neon for Java on my laptop (Ubuntu 16.04). There is some bug such that the installer fails with no warning or error message; you click install and it makes you wait for a minute and accomplishes nothing except for printing out a log file. Finally I found the solution 3/4ths of the way down an Eclipse Community forum:

You can download eclipse in the old-fashion way, that is, download the zip containing some specific package. If you go to https://www.eclipse.org/downloads, click on “Download Packages” just below of “Eclipse Neon” – “Download 64 bit”. On the next page, you will be able to select your preferred package (C/C++, java, php, etc) and download it in Neon eclipse version.” (https://www.eclipse.org/forums/index.php/t/1083012/).

Next I spent a few hours trying to figure out how to install/import packages required for jzy3d from https://github.com/jzy3d/jzy3d-graphs and https://github.com/jzy3d/jzy3d-api ā€“ following the instructions from http://www.jzy3d.org/plugins-graphs.php. Unfortunately I made no actual progress on this front: I learned there were a few extra things I needed to take care of but they ultimately didn’t solve all the problems. I’ll write them anyway:

  1. Clone/download and extract the repositories from the github links above
  2. Add version ā€“ get version by running mvn -Dplugin=groupId:artifactId help:describe ā€“ for
    build-helper-maven-plugin (version 3.0.0) and maven-surefire-plugin (version 2.20) to the pom.xml for jzy3d-graphs folder only (not the jzy3d-api folder)
  3. mvn install jzy3d-api
  4. I couldn’t get the mvn install jzy3d-graph to work and so got stuck. I have no idea how to really lay out the Build Path for this package.

I tried the tutorial on https://wadeawalker.wordpress.com/2010/10/09/tutorial-a-cross-platform-workbench-program-using-java-opengl-and-eclipse/ – I got 288 errors and 31 warnings.

Out of desperation now I asked my very first question on StackOverflow. Within 30 minutes and 5 views, someone downvoted my question and told me to make it precise and readable (see above for a screenshot). That was the end of my experience with jzy3d.

Now I began to second-guess my installation of Java. I have version 1.8.0_131, which I think I installed with some pain also (because the widely written about method no longer works since the links that were accessed from the terminal have been removed from the official website). I am pretty sure that’s the newest update from jdk 8, so I concluded I was fine here.

Now I’m debating continuing with Java and perhaps using JGraph for the visualization (which again will be difficult because it is Java) and JGraphT for the graph analysis (which I realized I didn’t think about at all before while trying to use jzy3d). My other option is to use Python; I’m fairly confident I can read pcap files and generate a graph of the network using Python but it won’t look good and it won’t be as clean as I first imagined.

Advertisements

Author: kapilsinha

Student at Caltech (2020) Computer Science Major

2 thoughts on “The Beginning – already full of setbacks”

  1. Hi Kapil! I saw your reference to my tutorial at https://wadeawalker.wordpress.com/2010/10/09/tutorial-a-cross-platform-workbench-program-using-java-opengl-and-eclipse/, so I double-checked it to make sure it still works properly. I had to make a small update to the code in GitHub so it would launch properly on Eclipse Neon 3, and I updated the blog to reflect this. But it shouldn’t have been showing you any warnings or errors with Java 1.8.0_131, since that’s the same version I just tested it with. Perhaps your Java or Eclipse installations are messed up? Let me know if you’d like some help šŸ™‚

    Liked by 1 person

    1. Hey Wade, for now I decided to try to get a basic version of my code to work on Python since I’ll ultimately be doing some ML; if I have time or the GUI I’m trying to make doesn’t work out in Python, I’ll let you know. Thanks!

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s