ClearFoundation Tracker - ClearOS
View Issue Details
0009541ClearOSapp-attack-detector - Attack Detectorpublic2016-07-08 06:512019-02-23 15:45
NickH 
tracker 
normalminoralways
closedsuspended 
7.2.0 
7.6.0 Updates 
0009541: Restarting the firewall wipes all f2b chains and rules
If you install and start app-attack-detector all the usual fail2ban (f2b) iptables rules and chains are created. If you restart the firewall they all get deleted and not not recreated. This is a standard f2b issue. You need to drop a 10-f2b type of rule (perhaps with a higher number so it runs later) into /etc/clearos/firewall.d with the systemd equivalent of "service fail2ban reload" in it. This will recreate the rules and chains on each firewall restart.
Install and start app-attack-detector with some rules enabled.
Restart the firewall.
No tags attached.
? 90-attack-detector (1,829) 2017-07-19 14:09
https://tracker.clearos.com/file_download.php?file_id=421&type=bug
? clearos-sshd.conf (271) 2017-07-19 14:09
https://tracker.clearos.com/file_download.php?file_id=431&type=bug
? clearos-sshd-ddos.conf (281) 2017-07-19 14:09
https://tracker.clearos.com/file_download.php?file_id=441&type=bug
Issue History
2016-07-08 06:51NickHNew Issue
2016-07-08 07:08NickHNote Added: 0003601
2016-07-09 07:07NickHNote Added: 0003631
2016-08-17 09:47user2Statusnew => acknowledged
2016-10-20 07:18NickHNote Added: 0004051
2017-04-19 12:05NickHNote Added: 0005401
2017-05-01 07:12NickHNote Edited: 0005401bug_revision_view_page.php?bugnote_id=5401#r1631
2017-05-01 07:13NickHNote Added: 0005471
2017-06-20 13:29NickHNote Added: 0005761
2017-07-14 13:04NickHNote Added: 0005871
2017-07-17 08:37user2Note Added: 0005881
2017-07-17 08:37user2Statusacknowledged => resolved
2017-07-17 08:37user2Resolutionopen => fixed
2017-07-17 08:37user2Assigned To => user2
2017-07-18 12:26user2Target Version => 7.4.0 Beta 1
2017-07-18 13:27NickHNote Added: 0005931
2017-07-18 13:27NickHStatusresolved => feedback
2017-07-18 13:27NickHResolutionfixed => reopened
2017-07-18 13:29NickHNote Edited: 0005931bug_revision_view_page.php?bugnote_id=5931#r1721
2017-07-18 23:49NickHNote Added: 0005941
2017-07-18 23:49NickHStatusfeedback => assigned
2017-07-18 23:51NickHNote Edited: 0005931bug_revision_view_page.php?bugnote_id=5931#r1731
2017-07-19 14:09NickHFile Added: 90-attack-detector
2017-07-19 14:09NickHFile Added: clearos-sshd.conf
2017-07-19 14:09NickHFile Added: clearos-sshd-ddos.conf
2017-07-19 14:12NickHNote Added: 0005951
2017-07-20 14:18NickHNote Added: 0005961
2017-07-21 10:56NickHNote Added: 0005981
2017-08-11 20:32user2Target Version7.4.0 Beta 1 => 7.4.0 Updates
2017-08-15 13:02NickHNote Added: 0006301
2018-04-11 10:27user2Target Version7.4.0 Updates => 7.5.0 Updates
2018-10-30 18:11user2Target Version7.5.0 Updates => 7.6.0 Updates
2018-12-14 12:10user2Assigned Touser2 => tracker
2019-02-23 15:45dloperNote Added: 0009471
2019-02-23 15:45dloperStatusassigned => closed
2019-02-23 15:45dloperResolutionreopened => suspended

Notes
(0003601)
NickH   
2016-07-08 07:08   
Actually the "service fail2ban reload" is not ideal as you don't want to reload it if you have it disabled but there is not a condrestart option so you'd need to do your own pidof test (or some other means) to check if it was running in the first place before reloading.
(0003631)
NickH   
2016-07-09 07:07   
If fail2ban is not running then doing a reload is safe as it does not start the process, but throws an error because it is not running.

pidof does not work, probably because it is a python process rather than a process in its own right.

The following works but would be better converted to systemd format:

[ -e /run/fail2ban/fail2ban.pid ] && service fail2ban reload
(0004051)
NickH   
2016-10-20 07:18   
I've just got 7.x up and running. Using the systemctl command:

systemctl reload fail2ban.service

in /etc/clearos/firewall.d/20-something does not work. The process just hangs. Appending a "&" allows it to work:

systemctl reload fail2ban.service &

I have not yet checked out a conditional command to see if fail2ban is already running. Really it should not be reloaded if it is not running.
(0005401)
NickH   
2017-04-19 12:05   
(edited on: 2017-05-01 07:11)
Please can this be escalated? To me it is more than a minor bug and it makes the app useless. In my VM, the firewall clearly restarts at least once during boot up so the app is immediately useless as its firewall chains are wiped during the boot up.

There is currently a job ticket, 553549, where app-attack-detector would really help - if it worked.

(0005471)
NickH   
2017-05-01 07:13   
In 7.x the following works in my /etc/clearos/firewall.d/30-fail2ban:
[ -e /var/run/fail2ban/fail2ban.pid ] && systemctl reload fail2ban.service &
(0005761)
NickH   
2017-06-20 13:29   
The fix for this was pushed to the Community quite a while back. Is it ready yet for Business?
(0005871)
NickH   
2017-07-14 13:04   
I notice this did go through to Business but the bug has not been updated.

I am not convinced about the bug fix. It only appears to work on the enabled jails in app-attack-detector and not on all enabled jails. This means that anyone running fail2ban conventionally will still need to restart/reload f2b to get the non-ClearOS jail firewall rules added. Can the script not be opened up to reload the firewall rules from all jails returned by the script line:

JAILS=`LANG=en_US fail2ban-client status | grep "Jail list" | sed -e 's/.*Jail list://' [^] -e 's/,//g'`
(0005881)
user2   
2017-07-17 08:37   
Thanks for the heads up. The fail2ban implementation was migrated over to the "ipset" way of doing things ( see 0014761 ) so this item should be resolved.

> I am not convinced about the bug fix. It only appears to work on the enabled jails in app-attack-detector and not on all enabled jails.

The script picks up on all enabled jails using "fail2ban-client status - that includes non-app rules as well. In principle, it should work on both ipset and iptables implementations in fail2ban.

I'll mark this issue as resolved for now, but feel free to change it back if you find an issue!
(0005931)
NickH   
2017-07-18 13:27   
(edited on: 2017-07-18 23:51)
Reopening for multiple issues but not all totally related to the bug.
1 - both /etc/fail2ban/jail.d/clearos-sshd-ddos.conf and /etc/fail2ban/jail.d/clearos-sshd.conf have a bug. In /etc/fail2ban/jail.d/clearos-sshd-ddos.conf, the line reading "iptables-ipset-proto6-allports[name=sshd-ddos][name=sshd]" should read "action = iptables-ipset-proto6-allports[name=sshd-ddos]". Note the addition of the "action =" and the removal of the duplicate key word. It is similar in the other jail.
2 - I was getting the case that enabling sshd-ddos through the webconfig also enabled sshd but it has stopped that for the moment. I'll see if I can get a reproducible test case.
3 - The script adds the firewall rules to the INPUT chains but does nothing about adding the banned IP's to f2b-XXX chains. This is only necessary for iptables type blocks (cyrus-imap, postfix-sasl and proftpd). Personally I would have thought it may be easier that for any jail not using ipset just to reload the jail. This is detected very early on in the script (ACTION=`fail2ban-client get $JAIL actions | tail -n 1`) so it would be easy to test for it and branch.

[edit]
... and I am not sure if the [name= keyword] is necessary in 1 above
[/edit]

(0005941)
NickH   
2017-07-18 23:49   
Elaborating on 1 above, you can see the line is not working as the jail is created with the jail's default iptables-multiport rule and an f2b-sshd/sshd-dos chain is also created in iptables.
I think (so untested) 2 was caused when I added the "action =" to the action line but there were still 2 [name = ] bits to the rule causing the wrong sshd jail to be created for the sshd-ddos rule which confused ClearOS. I am also pretty certain the [name =] bit is not needed at all and it then defaults to using the jail name. I'll try to test tonight.
(0005951)
NickH   
2017-07-19 14:12   
I cannot reproduce issue 0000002, but please find attached my fixes to the firewall script and the two jails. The [name = .....] is unnecessary.
(0005961)
NickH   
2017-07-20 14:18   
More comments. I've just done a fresh installation and the two files /etc/fail2ban/jail.d/clearos-sshd-ddos.conf and /etc/fail2ban/jail.d/clearos-sshd.conf have come in in a workable state i.e. correctly, with [name = ...]. This leads me to believe my two files became corrupted somewhere during the upgrade when ClearOS changed the rules over to ipset. Can we look at older systems which have been through the ssh upgrade cycle to see if any of them are wrong. They are not files I'd have fiddled with apart from disabling the rules. I've checked my backups from Jan '17 and the action line was not there at all so it must have come in from an update of some sort. If you want I can go through my weekly backups to pin down when it happened.
(0005981)
NickH   
2017-07-21 10:55   
More research.
Sometime prior to 27/05/2017 the two .conf files were updated correctly. They were wrong in my backup of 03/06/2017. Between these two dates, on 31/05/2017 app-ssh-server-core and app-ssh-server were updated to 2.3.23-1. on 03/06/2017 there are two new .conf.rpmnew files which are fine and the original .conf files go wrong. It looks like the culprit is the post install script in app-ssh-server-core-2.3.23-1.v7.noarch.rpm which executes /usr/clearos/apps/ssh_server/deploy/upgrade and this is clearly knocking out the "action =" and also leads to the [name=...] bit appearing twice.

Possible conclusion: any installation from prior to 31/05 will now have erroneous clearos-sshd-ddos.conf and clearos-sshd.conf files which will leave the jails using the iptables default actions. This should easily be correctable in another post-install script.
(0006301)
NickH   
2017-08-15 13:02   
In my solution, I'd avoid reloading the jails asynchronously (where I put an "&" after the jail reload rule). Deleting two firewalls in quick succession restart sthe firewall twice and the asynchronous jobs risk colliding. I potentially hit that as an issue or there is something else going on in f2b which caused the script to not find any jails at all.
(0009471)
dloper   
2019-02-23 15:45   
Migrated to: https://gitlab.com/clearos/clearfoundation/app-attack-detector/issues/2 [^]