Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I recently did the following:

- had a giant pcap

- wrote a perl script to output some of the key value from the dump (e.g. IP and UDP packet lengths) into csv

- loaded the csv into sqlite3 database

- ran several queries to identify microbursts of bandwidth etc

The younger/more junior folks were blown away that you could do this with <100 lines of code and it was pretty fast.

Btw, above was inspired by this: https://adamdrake.com/command-line-tools-can-be-235x-faster-...



If I were looking for bursts, SQL is not the first thing that comes to mind! Could you elaborate on this or sketch out the query?


Basically, doing a group by at millisecond resolution with a sum on the IP packet length to get a rough metric for bandwidth.

Once you have that, you can see the milliseconds with the highest bandwidth. Some extra math can also get you to Gigabits/second in a more network engineer friendly format.


I did a histogram-type thing in the same way by using a window function (similarly sqlite table scraped off pcap recordings). I can't remember if it was a fixed-width window (number of samples) or within some time window

Dropped it in datasette with datasette-vega and got a nice little plot


Aha, group by millisecond! Thanks.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: