New module to monitor firewall parameters

[israel-lugo]

We need to create something that knows how to monitor for events like the connection table being full, and so on, and react to them.

[israel-lugo]

For the Linux implementation, we can just read from a /proc/sys file to get the connection table count. For other operations, we can go for one of two approaches:

1. Use a Python binding to interact with Netlink at a low level, e.g. pynetfilter_conntrack. This is more flexible and lets us do everything from within our firewall module. On the other hand, it's yet another external Python module, with its own update cycle (and this one in particular wasn't updated for 6 years, only gained a new maintainer recently).

2. Use an existing CLI tool such as conntrack. This requires generating commands and executing them, but should be safe since we're the ones generating everything. Just be sure to validate the settings. Also, conntrack has an option to output in XML format, so we can take advantage of that instead of screen scraping. xml.etree.ElementTree gives us a pretty practical way to process the XML.

pynetfilter_conntrack is actually a binding for libnetfilter_conntrack, so that needs to be installed too. Might as well use conntrack, I think (which in itself also uses libnetfilter_conntrack).

[israel-lugo]

>>> import xml.etree.ElementTree
>>> tree = xml.etree.ElementTree.parse("/tmp/bla.xml")
>>> root = tree.getroot()
>>> root[0].find("./meta[@direction='original']/layer3/src").text
'10.10.1.81'

We could store this information in an sqlite database, for quickly finding things such as who has the most connections, and so on.

[israel-lugo]

If we do go with sqlite, be sure to convert IP addresses to numeric form, to store them as integers. Comparisons should be much faster than strings. IPv6 will require splitting into two 8-byte columns, since sqlite only has up to 8-byte integers.

[israel-lugo]

Come to think of it, unless we want some fancy features, we don't really need to model the conntrack table per se. Just to get the connection counts by IP, we can use a Python dict, which is already very well optimized. We could have a flowcount_by_src and a flowcount_by_dst. Of course, we may end up wanting fancier features... we'll see.