|
For the past several weeks, we have experienced almost daily denial of
service attacks on our www servers. Actually, the word attack is probably
too harsh, maybe I should say event. The nature of the event finds a
remote client somewhere has opened a number of TCP connections to LVS
that have absolutely no traffic whatsoever, save a single keepalive packet
every two minutes. I have seen few as 3 and as many as 120 connections
in the various incidents over the weeks.
These open connections are counted in the algorithm LVS uses to schedule
servers, so the server that has all these open connections receives
proportionately fewer new connections, in most cases taking the target
server completely out of rotation.
Yesterday, I noticed an event coming from 130.80.XXX.XXX which is the
Chronicle firewall address. Three connections were being held open from
a machine inside the Chronicle network. After some investigation, I
discovered the culprit was my own workstation. The moment I killed my
browser, the connections went away. I then fired up my browser again
and tried to retrace my steps to duplicate the situation.
To make a long story short, it appears that Opera version 6.0 beta will
leave a connection open to a server even after the window that was used
for that connection has been closed. The only time the connection is
closed is when Opera exits.
I will submit a problem report to Opera, in the meantime, there could be
hundreds if not thousands of beta Opera browsers out there that could
lock up ports on our servers for hours or days or longer.
This morning I made a configuation change to LVS that seems to have
solved the problem. The masquerading portion of the Linux kernel
(using LVS_NAT) uses default times to keep connections open, one for
TCP connections, one for closed TCP connections that have received a
FIN, and one for UDP connections. These defaults are 15, 2, and 5
minutes respectively. I changed the TCP timeout from 15 minutes to 110
seconds, which is shorter that the two minute intervals that the
keepalive packets occur, yet long enough for any imaginable connection
to a web server.
The change I made was:
ipchains -M -S 110 0 0
--
Peter F. Mastren | Houston Chronicle | If you're happy
Peter@xxxxxxxxxxx | Peter.Mastren@xxxxxxxxx | you're
See Our Twins | Phone: (713) 220-7689 | successful
http://www.Mastren.org/Twins | Fax: (713) 354-3114 |
|
