Diary

 

DNS queries for
Share |
Published: 2009-01-18,
Last Updated: 2009-01-22 19:48:29 UTC
by Daniel Wesemann (Version: 6)
51 comment(s)

Several folks are reporting odd queries hitting their DNS servers at a steady rate of about two per second.  The queries invariably ask for the name server of the domain "." (NS query for a single dot).   Since "." is a query for the root name servers, it has a very short query packet but a pretty long answer. Our current theory therefore is that this is a denial of service (DoS) attack in progress, where the DNS servers are used as "amplifiers" and unwittingly flood the (spoofed) source by providing a long answer to a system which never asked.

Update 0118 UTC: Correlations of logs and captures submitted by readers suggests that 69.50.142.11 and 76.9.16.171 are the two IPs from which most queries appear to originate... which would mean that these two sites are under a DoS attack.  ISC reader Chris used reverse DNS/passive DNS to determine that both IP addresses seem to be associated with porn sites.

Update 0253 UTC: The NOC of one of the netblocks has confirmed to ISC reader Steven that a DDoS attack is in progress against one of their clients. If you have queries for "." in your DNS log, best verify by use of a sniffer whether your DNS server actually responds and contributes to the DOS.  Normally, an internet-facing authoritative DNS server should not respond to recursive 3rd party queries, but we have received reports that some servers apparently respond to these "." queries even when recursion is disabled.

Update 1150 UTC: Several readers wrote in to comment that the answer to a "." query can come from the root-hints file or from the cache of the server itself. BIND has several options to control this behaviour (additional-from-auth, additional-from-cache, allow-query-cache).

Update 1520 UTC: 76.9.31.42,  69.50.142.110 and a couple more IPs are also being spoofed. Looks like more targets are being added into the fray.

Update 2117 UTC: We now provide an online tool on isc.sans.org that you can use to verify if your DNS server responds to these "." queries with a full list of root name servers and thus potentially contributes to the ongoing DDoS attack.  You can do this test readily on your own by issuing a "dig . NS @yournameserver" command, but since you need to run this from outside of your network to get the "real" picture, we are providing the online tool to help out.

Update: If you're looking for advice on securing your DNS servers, here's are configuration guidelines for BIND (thanks for the pointer, Hal Pomeranz!) and for Microsoft DNS.

Keywords: dns
51 comment(s)

Comments

brief experimentation with my own nameservers seems to indicate that if you don't allow recursion for queries from external hosts, this theory falls over... or am i missing something?
posted by Ken, Sun Jan 18 2009, 23:43
Seems to be a fixed length query. After putting both feet in my mouth and reporting it to an abuse@ without thinking about UDP spoofs (and got a really good reply, I might add), filtering on UDP dest port 53 with packet length 45 seems to stop it dead. For now, until whoever is pulling this DDoS stunt changes the query...
posted by Matt, Mon Jan 19 2009, 16:25
I take it that DJB would love seeing mentioned that tinydns - unless explicitly set up to do so - does not answer such requests.
posted by rpdenid, Mon Jan 19 2009, 18:54
So far, I've been getting these queries from 76.9.31.42, 76.9.16.171, 69.50.142.110, and 69.50.142.11. It started on Jan 16 with 69.50.142.11.
posted by Dshield, Mon Jan 19 2009, 20:06
It started with a few probes from 63.217.28.226 on the 6th of Januar, then I see a lot of requests from 216.201.82.19 on the 8th of January before the real flooding for the mentioned IPs hit my server at 15th of January at 12:26:37 UTC.
posted by tyldis, Mon Jan 19 2009, 20:58
I am also being hit by this.

I have blocked port 53 for packets for size 45, but named is still not working.

any other ways to block this ?

My firewall rule :
iptables -I INPUT 2 -p udp --dport 53 -m length --length 45:45 -j DROP
posted by andy, Tue Jan 20 2009, 09:22
Getting hit by most of the listed ips too. \"Solved\" the issue by a new regex for fail2ban (bans after 10 failures of regex \"Not authoritative for(.*), sending servfail to <HOST>(.*)\" - this regex works for powerdns.
Its a dynamic solution, so you dont need to update iptables rules by hand on any new ip involved.
posted by Torsten, Tue Jan 20 2009, 11:53
We had an misconfigured Windows 2008 DNS server serving the queries from spoofed addresses. I did by the instructions from Microsoft and set the option "Disable recursion" (as was set on our other servers). I used "Clear Cache" from DNS managment console, restarted the service (not the server).

After doing this I found out that the server was still happily serving the spoofed queries with root hints.

After spending an hour between banging my head to a wall, trying to figure out what I was doing wrong and restarting services ... I found out that there was a windowssystem32dnscache.dns file which contained nothing but the root hints. After uncommenting the lines, the server stopped serving the root hints and started to work as it should.

I do not know if restarting the physical server would have helped, but I didn't have this luxury of an option available..

Just to let others know that if they stumble into the same problem.. uncomment the lines or maybe rename the file..
posted by Paul (.FI), Tue Jan 20 2009, 12:15
EmergingThreats now has a snort sig to alert on this, sid:2009030. That was added to my ruleset at 7:00 UTC today. Although no servers here are responding, I have seen ~100 queries per hour from the four addresses mentioned by Dshield. As of 12:35 UTC, all have ceased.
posted by Ken, Tue Jan 20 2009, 13:34
It was only temporary... As of about an hour later, 13:32 UTC, I'm seeing alerts on these again.
posted by Ken, Tue Jan 20 2009, 13:46
On Jan 20, 2009 I'm seeing that the targets tailed off and the new spoofed IP addresses are:

66.230.160.1
66.230.128.15

which also are IP addresses related to a porn site.
posted by Andrew from Vancouver, Tue Jan 20 2009, 17:43
Looking for a quick fix. Using BIND9; have disabled all recursion and it now (correctly) won't resolve non-authoritative hosts - instead it returns the root hints and I don't know how to stop this!

Thanks
posted by John, Tue Jan 20 2009, 18:43
for an authoritative (only) name server, replying with the list of root name servers when asked for a domain it is not authoritative for, is the normal reaction. So, I would say the name server replies to the query "NS .", but rather, the name server says to "ask them (the root name servers)".

This being said, the problem remains that there is a multiplication factor of *10 ... *15 (45 bytes in - I didn't count them, 500+ bytes out).
To reduce that multiplication factor, one could consider using a dedicated view (Bind 9) to hold authoritative data, and within that view use a different (short) list of root name servers : namely with only that authoritative name server itself. That would probably reduce the amplification factor to max 2. "innocent" name servers that accidentily ask there question, should ignore that list of root name servers anyway ...

Kind regards,

Marc Lampo
author and teacher of
"DNS Explained" at John Cordier Academy, Belgium
posted by MarcL, Tue Jan 20 2009, 20:16
Thanks
posted by John, Tue Jan 20 2009, 20:37
There isn't a quick fix yet. I posted the above length 45, which is the datagram length. The packet length was, originally, 60 and the payload length is 17. It all depends on which bit of the UDP packet your firewall matches against. All of this can be had from tcpdump and the man pages for your firewall. <br> The easiest "fix" is to use something like snort_inline and the rule Ken posted earlier in block or sblock mode. You may want to increase the logging frequency in the rule if you use block or newsyslog is going to be busy. snort_inline is ideal if you're running iptables or ipfw. You can also use allow-recursive and allow-query-cache in BIND with IP ranges to limit recursive queries to a known subset of hosts, but in this case the target is still having to deal with your recursive refusal reply which, ironically, is the same length as the query. You can also try snortsam with Ken's rule, which works with a wider range of firewall packages, but has the disadvantage that it installs permanent block rules and is a possible DoS vector against your own network. The snort solutions are about the best you're going to get. <br> Be aware that the targets are changing (I have about six in my logs now) so simply blocking on source address is not sufficient. I can confirm that the snort rule SID: 20090030 from Emerging Threats works without hindering legitimate traffic.
posted by Matt, Tue Jan 20 2009, 21:19
If your Bind 9 is authoritative-only, globally deny queries and recursion (in the options section), and then allow queries in each zone definition. Queries for the \".\" hint zone are implicitly denied.
posted by Alex, Wed Jan 21 2009, 10:11
new target is 63.217.28.226 which is listed within SpamHaus as SBL Ref: SBL68873 netblock 63.217.28.128/25 as a Russian & Polish pharmacy spam operation
posted by Andrew from Vancouver, Fri Jan 23 2009, 18:52
Started receiving queries from 206.71.158.30 around 18:26 UTC (01/24/2009).
posted by Dshield, Sat Jan 24 2009, 20:14
Have been seeing this for several weeks from all the mentioned IP's. If you are using the Cisco IPS sensors, the 4003/0 sig fires. The "quick" fix for us has been to simply block inbound traffic at the firewall from the IP address being spoofed until the attack stops.
posted by sdewndr, Tue Jan 27 2009, 12:13
Checked my logs this morning and 3 more addresses show up: 67.192.144.0 (rackspace); 64.57.246.146 (4T Networks); 208.69.32.12 (OpenDNS). Looks like the Denial of Service attack is spreading/migrating.
posted by tjrical, Tue Jan 27 2009, 15:40
For what it is worth, I'm not seeing the OpenDNS IP that tjrical saw.

I do see that the bag guys don't completely shift to new targets.

The top spoofed DNS requests I'm seeing are now from: 63.217.28.226 (started 4 days ago) then 64.57.246.146 (started today) then 76.9.16.171 (started 9 days ago) and last, 67.192.144.0 (started today).
posted by Andrew from Vancouver, Tue Jan 27 2009, 17:38
re: my earlier post, the OpenDNS IP was not part of the attack. It was querying the same domain almost 200 times in about 1 minute, but it was not querying for the \".\" domain.
posted by tjrical, Tue Jan 27 2009, 19:23
IP's I've seen so far:
63.217.28.226
64.57.246.146
66.230.128.15
66.230.160.1
67.192.144.0
69.50.137.175
69.50.142.11
69.50.142.110
70.86.80.98 (started this morning around 6:20am est)
76.9.16.171
204.238.82.20
206.71.158.30
216.240.131.173
posted by sdewndr, Wed Jan 28 2009, 11:56
Started seeing 64.57.246.123 at 6:07am EST.
posted by Dave, Wed Jan 28 2009, 17:16
Also seeing 72.20.3.82; and 72.249.127.168 now
posted by sdewndr, Thu Jan 29 2009, 12:27
I'm seeing the same two new addresses as sdewndr 72.20.3.82 and 72.249.127.168 today and also 69.64.87.156 ... all three new addresses are name and webservers for pharmacy spammers. Perhaps the bad guys are expanding their campaign to squeeze competitors, or new players with different goals are entering the fray.
posted by Andrew from Vancouver, Thu Jan 29 2009, 16:57
I'm seeing quite a few IPs hitting my DNS servers. One such example: 72.249.127.168
posted by John Menerick, Thu Jan 29 2009, 17:26
Seen nearly all of the IPs, thanks people for the list, please keep it coming. I carefully compared the lot with my list and I've no new ones. Right this second I'm seeing same three as Andrew from Vancouver. I block on input in iptables, and then use the 45 rule with a --limit log, hope it lasts (the rule that is!).
posted by Dave from Scotland, Fri Jan 30 2009, 02:34
Now seeing 78.140.132.25 as of about 5:30am est
posted by sdewndr, Fri Jan 30 2009, 11:59
seeing 76.9.20.244 as of 6:10am est
posted by csmith, Fri Jan 30 2009, 15:10
also saw one hit pointed at 65.173.218.96 (isc.sans.org) on 1/29/09 at about 4:30pm est
posted by csmith, Fri Jan 30 2009, 15:11
My servers saw 78.140.132.25 and 76.9.20.244 this morning but for only 14 and 17 minutes, respectively, and was finished very close to 4:00 AM PST. Both targets are pill spammers' web and/or DNS service, and one of them is a shared webhost. Meanwhile, the packet rate dropped from a very consistent buzz down to ZERO and near-zero for periods as long as several hours. Perhaps the bad guys are tinkering with their network, or perhaps they are switching to concentrated bursts to show off the level of control they have of their network.
posted by Andrew from Vancouver, Fri Jan 30 2009, 18:03
Looks like 208.76.253.253 was added as a target. Started seeing queries on 01/30/2009 at 22:07 UTC.
posted by Dshield, Sat Jan 31 2009, 00:05
208.76.253.253 is the address for the email server and webserver for ScamFraudAlert.com and response time to that site is suffering. This time, it's not a DNS attack per se, as the spoofed packets are not heading to a DNS server. It is a straightforward bandwidth-based denial of service.
posted by Andrew, Sat Jan 31 2009, 00:49
Starting around 8pm est or so Friday night, we started seeing these: 208.76.253.253; 64.27.1.194; 70.86.80.98; 71.6.131.81; 65.23.129.220. The last four started around 4am est at about the same time. First time I've seen that. Intensifying?
posted by sdewndr, Sat Jan 31 2009, 10:49
Funny thing is, my server has never amped, yet I still see a ton of these queries. I suspect the attacker is satisfied even when attacks are unamped since the refused response makes it essentially an even trade with the benefit of added indirection.

You can block the inbound query

# and provide a level of indirection
iptables -N logdrop
iptables -A logdrop -j LOG
iptables -A logdrop -j DROP
iptables -I INPUT -i eth0 -p udp -s ! 192.168.0.0/16 --destination-port 53 -m string --algo kmp --from 30 --hex-string \"|010000010000000000000000020001|\" -j logdrop

or blocked the reject outbound altogether:

iptables -A OUTPUT -p udp -d ! 192.168.0.0/16 --source-port 53 -m string --algo kmp --from 30 --to 31 --hex-string \"|8105|\" -j logdrop

I have to do more thinking on the latter. It seems lame delegation could introduce inefficiencies to the DNS system if this were widely adopted, though negative caching might take care of that.
posted by sharingfoo, Sat Jan 31 2009, 18:27
Actaully the 8105 rule would only effect reject packets in reply to recursive queries so it's fine. It essentially turns off the reject response behavior when recursion is off. iptables -A OUTPUT -p udp -d ! 192.168.0.0/16 --source-port 53 -m string --algo kmp --from 30 --to 31 --hex-string "|8105|" -j DROP
posted by sharingfoo, Sat Jan 31 2009, 18:36
We have seen these ip's so far:
63.217.28.226
66.230.128.15
66.230.160.1
69.50.142.11
69.50.142.110
69.64.87.156
72.20.3.82
72.249.127.168
76.9.16.171
76.9.31.42
204.9.88.13
204.9.94.145
206.71.158.30
208.76.253.253
216.201.82.19
220.181.168.251

208.76.x.x and 220.181.x.x being the most recent. Also a few mins after our system started dropping 208.76.253.253 querys through a script, we saw this entry once in our log which resolves to isc/bind ????

client 149.20.52.143#56538: query 'VERSION.BIND/CH' denied

posted by Tom, Sat Jan 31 2009, 23:01
Regarding the queries from 149.20.52.143,
see: https://www.dns-oarc.net/node/109
posted by Dshield, Sun Feb 01 2009, 22:05
After a hiatus of several days the method has resumed with spoofing 89.149.221.182 which is the website for seopharmacy.com ... perhaps we can also expect to see 89.149.209.161 which is the sole NS address listed for that domain.
posted by Andrew from Vancouver, Wed Feb 04 2009, 16:43
After half an hour of warm-up, I am not seeing any new spoofed addresses, and the total packets per second is less than half of what it was in late January. It is roughly 1 packet per second now whereas it was roughly 2.2 packets per second, regardless of how many targets there were.

The spoofed targets I am seeing are: 76.9.16.171 (one of the initial targets which began on Jan 17), 69.64.87.156 (a pharmacy spam site which began on Jan 29) and then 89.149.209.161 (which is the pharmacy spam site I saw that began today Feb 04 at 8:31 AM)
posted by Andrew from Vancouver, Wed Feb 04 2009, 17:11
I'm now also seeing 89.149.221.182
posted by sdewndr, Wed Feb 04 2009, 17:39
Now seeing 195.68.176.4 as of 2/10/2009 21:47 UTC
posted by Dshield, Wed Feb 11 2009, 01:56
Quiet for a time, now seeing 82.146.35.143 firstvds.ru, since 11:59:37 Valentine's day
posted by Dave from Scotland, Sat Feb 14 2009, 14:44
New spoofed sources as targets are firstvds.ru [82.146.35.143] back on Feb 14th and 15th, and invest-pool.ru [62.109.4.89] starting at 3:43 PM PST (1143 UDT).
posted by Andrew from Vancouver, Tue Feb 17 2009, 00:15
As of a month ago i have been seeing the spoofed queries to my dns. but i have different ips. Currently they are using 62.109.4.89, 195.68.176.4, 65.173.218.96. i have talked to 2 of the ip maintainers and both have revealed DDOS attacks on those ips.
posted by slimebox, Mon Feb 23 2009, 18:32
ah! the last ip seems to be this site! oh boy!
posted by slimebox, Mon Feb 23 2009, 18:34
scratch that, i did the query test from this site . im an idiot.
posted by slimebox, Mon Feb 23 2009, 18:35
Atacks continue from 62.109.4.89
posted by grafik, Sun Mar 01 2009, 18:31
i am seeing these attacks again... "origin" is from networks belonging to mozilla and macrovision.
posted by joeblow, Tue Jul 07 2009, 03:42
Yeah we are seeing them again. They had all but stopped for a few months and now we are seeing "client 204.123.28.55#41293: query(cache) 'a338.g.akamaitech.net/IN' denied". All of them say a338.g.akamaitech.net/IN' denied from many different addresses.
posted by Tom, Sat Jul 11 2009, 04:57
Login here to post a comment. Diary Archive