[Ntop] Multiple Netflow senders sending to Nprobe and Ntop on same server

[Warren Daly (OPUS)]

Hello,
Goal: Multiple Netflow senders sending to Nprobe and Ntop on same
server. Ability to view traffic in each subnet, or view traffic from
individual netflow sources.

I would like some assistance please. I have read the user guide (ntop &
nprobe)
I have also read this thread
http://www.gossamer-threads.com/lists/ntop/misc/38960
and this excellent article
http://www.ntop.org/ntopng/creating-a-hierarchical-cluster-of-ntopng-instances/

This is what I have, and what I would like to achieve.

Remote Network A - 192.168.0.0/24
Netflow streams from Cisco ASA (192.168.0.254) sending to nprobe on
192.168.2.1:2055

Remote Network B - 192.168.1.0/24
Netflow streams from Cisco ASA (192.168.1.254) sending to nprobe on
192.168.2.1:2055

Local Network C - 192.168.2.0/24
Netflow streams from Cisco ASA (192.168.2.254) sending to nprobe on
192.168.2.1:2055

The server 192.168.2.1 runs both nprobe and ntopng

I would like to check on traffic in each subnet. So I want to check
which nodes are doing what in each subnet.
I don't want all the traffic mix together.

This is what I have tried.

Start ntopng using
-i=tcp://127.0.0.1:5556
-i=tcp://127.0.0.1:5557
-i=tcp://127.0.0.1:5558

I try to start multiple nprobes to listen on port 2055. But I need to
filter traffic so I tried

nprobe -f src ip 192.168.0.254 --zmq tcp://*:5556 -i none -n none
--collector-port 2055 -b 2
But you can't use BPF filtering as a collector...

if I use this
nprobe --zmq tcp://*:5556 -i eth1 -n none --collector-port 2055 -b 2
This shows all traffic in all the netflows... if I select the interface
*5556 in ntop it shows me all traffic....

I also tried this
nprobe --zmq tcp://*:5556 -i none -n none --collector-port 2055 -b 2
but this will *not* show any incoming netflows being decoded on port
2055. A tcpdump shows they are arriving.

So I decided, to change the ports of each netflow stream to make it
easier for nprobe.

Remote Network A - 192.168.0.0/24
Netflow streams from Cisco ASA (192.168.0.254) sending to nprobe on
192.168.2.1:2055

Remote Network B - 192.168.1.0/24
Netflow streams from Cisco ASA (192.168.1.254) sending to nprobe on
192.168.2.1:2056

Local Network C - 192.168.2.0/24
Netflow streams from Cisco ASA (192.168.2.254) sending to nprobe on
192.168.2.1:2057

But again if I do this
nprobe --zmq tcp://*:5556 -i eth1 -n none --collector-port 2055 -b 2
nprobe --zmq tcp://*:5557 -i eth1 -n none --collector-port 2056 -b 2
nprobe --zmq tcp://*:5558 -i eth1 -n none --collector-port 2057 -b 2

no matter which interface I select on the ntopng interface I see all
traffic aggregated. I can't view the traffic from just one nprobe instance.
e.g if I select *5556 interface in ntopng, I should only see traffic in
the 192.168.0.0 subnet, but I see all traffic.
e.g if I select *5557 interface in ntopng, I should only see traffic in
the 192.168.1.0 subnet, but I see all traffic.
e.g if I select *5558 interface in ntopng, I should only see traffic in
the 192.168.2.0 subnet, but I see all traffic.

I'm obviously doing something silly. Any assistance is greatly
appreciated. I am about to purchase a pro license, and a nprobe license,
I just want to show management this works before proceeding.

Best Regards,
Warren

[Warren Daly (OPUS)]

Hi Yuri,
thank you for your reply.
I have started 2 Nprobes, and reconfigured the ASAs...

ASA #1 to port 2055
nprobe --zmq tcp://*:5556 -i none -n none --collector-port 2055 --verbose 2
ASA #2 to port 2056
nprobe --zmq tcp://*:5557 -i none -n none --collector-port 2056 --verbose 2

I quickly check the netflow packets are arriving....

$sudo tcpdump -n dst port 2055
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 8192 bytes
04:33:39.776112812 IP 192.168.0.254.18656 > 192.168.13.7.2055: UDP,
length 1424
04:33:39.942931812 IP 192.168.0.254.18656 > 192.168.13.7.2055: UDP,
length 1400

$tcpdump -n dst port 2056
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 8192 bytes
04:33:49.856934812 IP 192.168.1.254.14780 > 192.168.13.7.2056: UDP,
length 1456
04:33:50.716154812 IP 192.168.1.254.14780 > 192.168.13.7.2056: UDP,
length 1452

But I do not see any verbose logging output of the Netflow stream (maybe
this does not happen?)

When I login to the NtopNg interface I see these messages...

No packet has been received yet on interface tcp://127.0.0.1:5556.
Please wait 6 seconds until this page reloads.

I change interface....
No packet has been received yet on interface tcp://127.0.0.1:5557.
Please wait 10 seconds until this page reloads.

I have confirmed the Cisco is exporting Netflow V9. I have changed the
nprobe start up to
nprobe --zmq tcp://*:5556 -i none -n none --collector-port 2055
--verbose 2 --flow-version 9

Still 'No packet has been received yet on interface tcp://127.0.0.1:5557"

So I changed the nprobe to utilized the IP address (not *)
nprobe --zmq tcp://192.168.2.1:5556 -i none -n none --collector-port
2055 --verbose 2
nprobe --zmq tcp://192.168.2.1:5557 -i none -n none --collector-port
2056 --verbose 2

and change the ntopng to start with
-i=tcp://192.168.2.1:5556
-i=tcp://192.168.2.1:5557

The message about No packets is *not* shown any more.

I waited 30 minutes... and All Hosts show "No Results Found", Active
Flows "No Results Found"

Any assistance is greatly appreciated.
Regards,
Warren

Warren,
what about use different collector port and, of course, reconfigure
your ASAs to send the traffic to the right port.
Something like
ASA #1 to port 2055
nprobe --zmq tcp://*:5556 -i none -n none --collector-port 2055
ASA #2 to port 2056
nprobe --zmq tcp://*:5557 -i none -n none --collector-port 2056
ASA #3 to port 2057
nprobe --zmq tcp://*:5558 -i none -n none --collector-port 2057
and then ntopng as you did.
Regards, Yuri
###############################################
http://www.ntop.org
"Simplicity is the ultimate sophistication" - Leonardo da Vinci
###############################################

Post by Warren Daly (OPUS)
Hello,
Goal: Multiple Netflow senders sending to Nprobe and Ntop on same
server. Ability to view traffic in each subnet, or view traffic from
individual netflow sources.
I would like some assistance please. I have read the user guide (ntop
& nprobe)
I have also read this thread
http://www.gossamer-threads.com/lists/ntop/misc/38960
and this excellent article
http://www.ntop.org/ntopng/creating-a-hierarchical-cluster-of-ntopng-instances/
This is what I have, and what I would like to achieve.
Remote Network A - 192.168.0.0/24
Netflow streams from Cisco ASA (192.168.0.254) sending to nprobe on
192.168.2.1:2055
Remote Network B - 192.168.1.0/24
Netflow streams from Cisco ASA (192.168.1.254) sending to nprobe on
192.168.2.1:2055
Local Network C - 192.168.2.0/24
Netflow streams from Cisco ASA (192.168.2.254) sending to nprobe on
192.168.2.1:2055
The server 192.168.2.1 runs both nprobe and ntopng
I would like to check on traffic in each subnet. So I want to check
which nodes are doing what in each subnet.
I don't want all the traffic mix together.
This is what I have tried.
Start ntopng using
-i=tcp://127.0.0.1:5556
-i=tcp://127.0.0.1:5557
-i=tcp://127.0.0.1:5558
I try to start multiple nprobes to listen on port 2055. But I need to
filter traffic so I tried
nprobe -f src ip 192.168.0.254 --zmq tcp://*:5556 -i none -n none
--collector-port 2055 -b 2
But you can't use BPF filtering as a collector...
if I use this
nprobe --zmq tcp://*:5556 -i eth1 -n none --collector-port 2055 -b 2
This shows all traffic in all the netflows... if I select the
interface *5556 in ntop it shows me all traffic....
I also tried this
nprobe --zmq tcp://*:5556 -i none -n none --collector-port 2055 -b 2
but this will *not* show any incoming netflows being decoded on port
2055. A tcpdump shows they are arriving.
So I decided, to change the ports of each netflow stream to make it
easier for nprobe.
Remote Network A - 192.168.0.0/24
Netflow streams from Cisco ASA (192.168.0.254) sending to nprobe on
192.168.2.1:2055
Remote Network B - 192.168.1.0/24
Netflow streams from Cisco ASA (192.168.1.254) sending to nprobe on
192.168.2.1:2056
Local Network C - 192.168.2.0/24
Netflow streams from Cisco ASA (192.168.2.254) sending to nprobe on
192.168.2.1:2057
But again if I do this
nprobe --zmq tcp://*:5556 -i eth1 -n none --collector-port 2055 -b 2
nprobe --zmq tcp://*:5557 -i eth1 -n none --collector-port 2056 -b 2
nprobe --zmq tcp://*:5558 -i eth1 -n none --collector-port 2057 -b 2
no matter which interface I select on the ntopng interface I see all
traffic aggregated. I can't view the traffic from just one nprobe instance.
e.g if I select *5556 interface in ntopng, I should only see traffic
in the 192.168.0.0 subnet, but I see all traffic.
e.g if I select *5557 interface in ntopng, I should only see traffic
in the 192.168.1.0 subnet, but I see all traffic.
e.g if I select *5558 interface in ntopng, I should only see traffic
in the 192.168.2.0 subnet, but I see all traffic.
I'm obviously doing something silly. Any assistance is greatly
appreciated. I am about to purchase a pro license, and a nprobe
license, I just want to show management this works before proceeding.
Best Regards,
Warren
_______________________________________________
Ntop mailing list
http://listgateway.unipi.it/mailman/listinfo/ntop

_______________________________________________
Ntop mailing list
http://listgateway.unipi.it/mailman/listinfo/ntop

--
Warren Daly
Chief Technical Officer
[phone] Skype: warrendaly

OPUS
[phone]
www.opus.com.kh
Suite 3FN1 - VTrust Office Centre
Parkway Square | Phnom Penh, Cambodia

[Warren Daly (OPUS)]

Hello,
I wish to give user friendly names to 'Interfaces' on NTOPNG
Professional v.2.0.150531

I have 4 interfaces defined in
/etc/ntopng/ntopng.conf

Defined like this:
-i=tcp://192.168.1.7:5556
-i=tcp://192.168.1.7:5557
-i=tcp://192.168.1.7:5558
-i=tcp://192.168.1.7:5559

1. If I select any interface from the dropdown - the first page to load
is the 'top talkers'
2. If I re-select the interface from the dropdown again, I am shown a
page with the menu
Overview
Protocols
Historical Activity
Packet Dump

A) I am wondering why the dropdown operates like this. Why am I shown
different pages depending on how many times I select it?

3. If I click on 'Overview' I see there is a custom name fields with a
'save name' button. I input a simple identified such as GreenOffice. I
click on save name.

4. I then click on the Interfaces dropdown to change interface. (so I
can rename it to a user friendly name) The first interface is the only
interface that changes name. Even if I confirm I using
-i=tcp://192.168.1.7:5559, when I save the name the interface
-i=tcp://192.168.1.7:5556 has it's name changed.

Can you help with this please?

NTOPNG Professional v.2.0.150531
Running on Ubuntu 12.04 LTS
Client Web browser 40.0.2 on Windows 7
--
Warren Daly
Chief Technical Officer
[phone] Skype: warrendaly

OPUS
[phone]
www.opus.com.kh
Suite 3FN1 - VTrust Office Centre
Parkway Square | Phnom Penh, Cambodia

[Warren Daly (OPUS)]

Upgrading to ntopng Professional v.2.0.150820 seems to fix this minor bug.

Post by Warren Daly (OPUS)
Hello,
I wish to give user friendly names to 'Interfaces' on NTOPNG
Professional v.2.0.150531
I have 4 interfaces defined in
/etc/ntopng/ntopng.conf
-i=tcp://192.168.1.7:5556
-i=tcp://192.168.1.7:5557
-i=tcp://192.168.1.7:5558
-i=tcp://192.168.1.7:5559
1. If I select any interface from the dropdown - the first page to
load is the 'top talkers'
2. If I re-select the interface from the dropdown again, I am shown a
page with the menu
Overview
Protocols
Historical Activity
Packet Dump
A) I am wondering why the dropdown operates like this. Why am I shown
different pages depending on how many times I select it?
3. If I click on 'Overview' I see there is a custom name fields with a
'save name' button. I input a simple identified such as GreenOffice. I
click on save name.
4. I then click on the Interfaces dropdown to change interface. (so I
can rename it to a user friendly name) The first interface is the only
interface that changes name. Even if I confirm I using
-i=tcp://192.168.1.7:5559, when I save the name the interface
-i=tcp://192.168.1.7:5556 has it's name changed.
Can you help with this please?
NTOPNG Professional v.2.0.150531
Running on Ubuntu 12.04 LTS
Client Web browser 40.0.2 on Windows 7

--
Warren Daly
Chief Technical Officer
[phone] Skype: warrendaly

OPUS
[phone]
www.opus.com.kh
Suite 3FN1 - VTrust Office Centre
Parkway Square | Phnom Penh, Cambodia

[Warren Daly (OPUS)]

With ntopng Professional v.2.0.150820 under 'interfaces' drop down, I
could select the interface. The dashboard.lua would display

Top Local Talkers
Top Remote Destinations
Realtime Top Application Traffic
Realtime Traffic
etc...etc..


After upgrading to ntopng Professional v.2.0.150830, the dashboard will
not load, it says 'interface has not received any packets...'

I down graded to v.2.0.150820 and it operates ok.

All interfaces are in Netflow collection mode.
Ubuntu 12.04 LTS

[Warren Daly (OPUS)]

Hi,
I wish to enable historical data. If I select 'interfaces' dropdown menu
I see the historical menu item. If I hover over this item is says 'In
order to enable this interface, you have to start ntopng with -F option.

I've looked in the version 2.3 of the user guide. There is no reference
to the -F (historical option)

I've tried -F=1 and -F in the ntopng.conf
I've also tried --F in ntop.start file... but nothing works.

Should I wait until NTOPNG version 2.1 is released?

NTOPNG Professional v.2.0.150531
Running on Ubuntu 12.04 LTS

Thanks.
Warren

[Albert K]

Hi Luca,

Is -F db (sqlite) being deprecated? It is still useful for lightweight
post processing without using a full featured Database. Thanks.

Post by Warren Daly (OPUS)
Warren
With -F mysql... you have historical view but it is still under
development, the old historical interfaces gone as it turned not to be a
good idea based on the feedback received so we are rewriting it from scratch
Regards Luca
Sent from my iPad

Post by Warren Daly (OPUS)
Hi,
I wish to enable historical data. If I select 'interfaces' dropdown menu

I see the historical menu item. If I hover over this item is says 'In order
to enable this interface, you have to start ntopng with -F option.

Post by Warren Daly (OPUS)
I've looked in the version 2.3 of the user guide. There is no reference

to the -F (historical option)

Post by Warren Daly (OPUS)
I've tried -F=1 and -F in the ntopng.conf
I've also tried --F in ntop.start file... but nothing works.
Should I wait until NTOPNG version 2.1 is released?
NTOPNG Professional v.2.0.150531
Running on Ubuntu 12.04 LTS
Thanks.
Warren
_______________________________________________
Ntop mailing list
http://listgateway.unipi.it/mailman/listinfo/ntop

_______________________________________________
Ntop mailing list
http://listgateway.unipi.it/mailman/listinfo/ntop