John Bourke
2018-04-22 18:10:20 UTC
Hi,
I have a network with about 50000 IP addresses, 2Gbps of traffic. A lot of these ordered into groups, and groups of groups.
I'd like to generate some real time statistics for these, and by real time, I mean every 6 seconds. I don't need flows just RX/TX bytes/packets.
I need these statistics by IP address and aggregated up the group hierarchy. I'd like "something" to do this aggregation. If it can be done at the collection point, that would efficient.
I would like to be able to stream/push some of these statistics every six seconds to applications which are interested. These applications certainly would not want all statistics all of the time. Maybe the aggregations all of the time, and the specific IP addresses some of the time.
So the question is, from a ntop and associated architecture point of view
1. Were and how do I do the aggregation ?
2. Can I selectivity push the aggregations ?
The simple answer is to dump all current statistics to disk every 6 seconds, and batch process the aggregation, and then push the statistics - meaning I do all the work. 50000x4 data points every six seconds to disk does not sound very efficient.
Is there a better way in the ntop architecture ?
Thanks
john
I have a network with about 50000 IP addresses, 2Gbps of traffic. A lot of these ordered into groups, and groups of groups.
I'd like to generate some real time statistics for these, and by real time, I mean every 6 seconds. I don't need flows just RX/TX bytes/packets.
I need these statistics by IP address and aggregated up the group hierarchy. I'd like "something" to do this aggregation. If it can be done at the collection point, that would efficient.
I would like to be able to stream/push some of these statistics every six seconds to applications which are interested. These applications certainly would not want all statistics all of the time. Maybe the aggregations all of the time, and the specific IP addresses some of the time.
So the question is, from a ntop and associated architecture point of view
1. Were and how do I do the aggregation ?
2. Can I selectivity push the aggregations ?
The simple answer is to dump all current statistics to disk every 6 seconds, and batch process the aggregation, and then push the statistics - meaning I do all the work. 50000x4 data points every six seconds to disk does not sound very efficient.
Is there a better way in the ntop architecture ?
Thanks
john