Threat Level: green Handler on Duty: Didier Stevens

SANS ISC: Are you a "Hunter"? - SANS Internet Storm Center SANS ISC InfoSec Forums


Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!
Are you a "Hunter"?

It sound like an interesting question, isn't it? But what I'm referring to is us analyst that searches for unusual activity or you just wait for a trigger from an IDS/IPS or that a rule will trigger something from the SIEM.

I watched the opening keynote by Amit Yoran President of RSA at the RSA Singapore conference [1] and he was made reference to large organizations who have cutting edge security software/hardware and how bad they are still failing at catching bad actors still go undetected for a long time. He shared five points to go by to help catch bad actors in a network: Does it really Help (this shiny new device or software), Visibility, Identity, Intelligence and Prioritize. The fourth point Intelligence is where he talks about "CISO that gives their security team the time to hunt and learn their environment to understand what normal looks like are much more rapidly identifying unusual patterns (23:53m)"[1]

I do go "hunting" looking for unusual activity and pattern IDS/IPS or even the SIEM doesn't know about. There is a lot of threat intelligence out there that can be used to detect unusual pattern of activity. Maybe you have a security device that use some form of feeds to detect bad actors (i.e. some vendors use DShield feeds), reviewing what they trigger might yield interesting data. How about taking the time to review if the systems communicating with the HR server(s) are part of the allowed list? This example could be added to a SIEM to trigger for unusual activity.

If you are a “hunter”, what do you look for?

[1] http://www.rsaconference.com/media/the-game-has-changed?

-----------

Guy Bruneau IPSS Inc. gbruneau at isc dot sans dot edu

Guy

424 Posts
ISC Handler
I'm most certainly a hunter. When I'm not doing other things I will usually login to random machines inspect process lists, look at web requests coming in. Usually if the IDS hasn't spotted something I usually do when I take a look around the network in what I call my "Combat Patrols".

Most of the time, when the IDS does report something, I'll investigate and find something else entirely, either because the rule has found some other kind of malicious behaviour, or it was a false positive, but something else unrelated catches my eye.
Yinette

12 Posts Posts
I also watched this, and was shocked to see the claim that SIEM misses over 99% of APT's! The claim itself was quoted from verizons 2015 DBIR. Part of the claim was that traditional perimeter defences are not enough and we are losing the arms race to the bad guys...and although there is undoubtedly a lot of truth in this, my question is, what can we do to make SIEM better?

I believe that part of making SIEM better is allowing the analysts time to fully understand the environment, to know its norms and nuances. Above and beyond that the SIEM itself must have an excellent asset and network model that is clearly defined and kept up to to date. Also as has already been said, samples must be regularly taken and tested to ensure that everything is as it should be. Processes muse be defined, tested, refined and improved.

Another issue is, if you have too many rules the analysts can be overwhelmed, ddos'ed if you like. The rules need to be well defined and noise and false positives must be eradicated, but above all the analysts must fully understand the environment and the traffic flowing over it.

A framework for reaching operational maturity must also be standardised and worked on openly by a group of experienced and engaged professionals.

Finally SIEM must become part of the standard curricular for Security course in universities and colleges.
TheJulyPlot

5 Posts Posts
I think that hunting is unique to human beings since the beginning ;)
I call this "active defense". As said Yinette, I'm also investigating a lot, keeping an eye on my logs and running honeypots... Paste sites are also a nice source of juicy content
Xme

394 Posts Posts
ISC Handler
Xme, what are "Paste sites"? Are you referring to sites like Pastebin?
AAInfoSec

49 Posts Posts
Yes, love the challenge of hunt. I do have a question though and was going to start a new thread around it. How long do the readers/members of this forum keep their logs to chart to progression of "the hunt"?

Regards,
ICI2I
ICI2I

63 Posts Posts
Yes, pastebin.com is the most known but they are plenty of others (and some much more obscure).
Examples:
pastie.org
codepad.org
nopaste.net
pasteguru.com
postits4tga4cqts.onion
Xme

394 Posts Posts
ISC Handler
Longer is better but it has a cost (in terms of storage)
IMHO, it's important to make a difference between events and incidents.
- An event is "an observable change to the normal behaviour of a system, environment, process, workflow or person".
- A security incident is "a series of events that adversely affects the information assets of an organization".
I'm keeping 3 months of events (to have time to investigate and rollback to them)
Incidents (read: alerts based on correlation rules / filters) are kept forever... (until I've enough storage)
Of course, when you drop oldest events, you also drop potential evidences or interesting stuff... Keep in mind that, for compliance reasons, you can be forced to keep them x months.
Xme

394 Posts Posts
ISC Handler
Hunter for sure! I like to log into the SIEM on a daily basis and start with a view of all logs for the last 5 mins. At this point I start removing the common events. Once I have it really narrowed down I start to expand the time frame and boom all sorts of new easter eggs start to show up... sometimes... ;) Sitting and waiting for an event to trigger or happen from any security tool can be a disastrous mistake. Being a Hunter requires a very active imagination, and a mindset of thinking outside the box.
Scott3Boy

3 Posts Posts
@Xme:

Fortunately my SW emails me 2X daily in text, crushing them down is not an issue for size. I try to follow trends posted here and other sites and tag the ones that show up >5X in a week. So far all has been quiet. Fingers X!

Thanks
ICI2I

63 Posts Posts
Combat Patrols - I love that and I'm gonna shamelessly steal it. :-)

I wholeheartedly agree - it's when things are suddenly quiet that we should be paying extra attention to what's going on. Is it quiet because you're being left alone for a change (unlikely) or because the bad actors are using new tools all those fancy defenses we have in place don't detect.

We recently went on a phish education campaign at my $DAYJOB$ and it's paying off. I'm often getting phish reports and can often use them to not only check if anyone fell for them (DNS query logs, snort/firewall logs, etc), but can proactively prevent them by updating DNS filters, updating firewalls, etc. Best yet, my employer lets me spend time digging further. For instance, given a piece of malware found in some phish, often a downloader, I'll obtain a copy of what it's trying to download and run and then run some malware analysis tools on THAT (I really like www.hybrid-analysis.com for instance). Then I'll see what THAT malware does - who it talks to, what DNS queries it makes, etc. That gives me a whole 'nother batch of indicators that I can make snort rules for. That way the next round of phish that uses some new file-dropper that fetches the same secondary malware, I've already got either blocked and/or being watched for.

And don't forget the logs! When I go to the trouble of blocking hostnames that resolve to a particular IP or network, I also have a job that tells me every morning what hostnames were blocked because of one of these filters. That has occasionally led to "interesting" (tm) - stuff that nobody is detecting yet.
Brent

103 Posts Posts
@Brent:

Great information! We've been hit 3 times in the last week with malware in emails. I recently started my position as a 1-man security team, so I have so much to do...from policy work to running firewall & vuln assessment, etc.
I'm trying to set up a sandbox machine to do something similar to what you're doing. Where do you view DNS query logs? Your DNS servers? How's your phishing campaign set up? Are you using any specific tools/vendors? Also where are you running the job to block hostnames? Your firewall?
AAInfoSec

49 Posts Posts
Excellent idea and I think, given the option, most security professionals would want to hunt around the network to find out where the problems are and hopefully find that sneaky APT they have a feeling exists somewhere.

The problem isn't the people, it is the organisations. Few big businesses maintain a large enough, skilled enough, security team to deal with the daily incidents let alone have the time to properly pro-actively search the network looking for incidents.

Unfortunately should an IR team get the downtime needed to scan, odds are business pressures will have them downsized into other roles.
AAInfoSec
1 Posts Posts
My recommendation is start putting rules/use case according to kill chain phase (google cyber kill chain). A lot of analyst always gets to focus on protecting the perimeters and forgot about every other thing that the attacker can do while navigate through the network. There is literally dozens of way that an attacker can enter through the network; but their options are severely limited once their in the organization. Several examples, bundled tools(installation), lateral movement (internal scanning), c2 like outbound behavior and ex filtration of data. I couldn't give a whole list due to NDA, but that should give some ideas. And internal honey pots is always a great option.
Mostropi

27 Posts Posts
My recommendation is start putting rules/use case according to kill chain phase (google cyber kill chain). A lot of analyst always gets to focus on protecting the perimeters and forgot about every other thing that the attacker can do while navigate through the network. There is literally dozens of way that an attacker can enter through the network; but their options are severely limited once their in the organization. Several examples, bundled tools(installation), lateral movement (internal scanning), c2 like outbound behavior and ex filtration of data. I couldn't give a whole list due to NDA, but that should give some ideas. And internal honey pots is always a great option.
Mostropi

27 Posts Posts
Quoting Mostropi:And internal honey pots is always a great option.


Oh yeah! I forgot to mention Attivo. For anyone interested in setting up internal honeypots, you might just pass an eye over their "Botsink" product. It's a whole lot easier than rolling your own and has some really slick features.
Brent

103 Posts Posts
Quoting AAInfoSec: Where do you view DNS query logs? Your DNS servers? How's your phishing campaign set up? Are you using any specific tools/vendors? Also where are you running the job to block hostnames? Your firewall?


Hmmm, my previous reply might've just been held up or too long or it got held up for review because it had a URL or something else... But just in case it doesn't make it, here's a shorter response...

Feel free to drop me an email and I can give you some details on what's worked for us. The username part of my email is "bbice" and the domain is sgi.com.

We're running BIND for DNS, some custom tools for logging/searching, and RPZ (response policy zones) in BIND for DNS filtering.
Brent

103 Posts Posts
Been security analyst in SOC for more than 3 years. Besides looking for the alerts triggering from the device and waiting for SIEM to pick up following I look for :

1. Traffic violating security standards.
2. Traffic going to or coming from countries where customer does not do business.
3. Access/running executables or confidential files on hosts/servers - this can be done by acceptable use monitoring.
4. I do have active lists in my SIEM containing IP address and domains that continously checks for any match. This list is regularly updated.
5. I also randomly access websites on top-level domains to identify any suspicious/malicious re-direction - the traffic is passed to the interface being monitored on Security Onion.
6. Check spam emails and run it over Security Onion to see if any alerts gets trigger. Extract domains and IP address and use them in the SIEM.
7. Check specific parameters in the URL/HTTP requests that can be used to exploit web applications - although the IDPS may trigger the rule for it but there are instances where we do not get IDPS logs.
8. Lastly, honeypot to identify bad actors.

Hope this helps. Happy hunting.
makflwana

17 Posts Posts
@makflwana
I hope you are aware that attackers are also watching forums like this and using it to their advantage. I recall blogging about Dridex/EK and they hastily changes the traffic within weeks.

I also think there is a lot of people who get too caught are in using activelist for indicators (as reference to your point #3), and forgot they don't help in detecting targeted attacks.
Mostropi

27 Posts Posts
Quoting Mostropi:@makflwana
I hope you are aware that attackers are also watching forums like this and using it to their advantage. I recall blogging about Dridex/EK and they hastily changes the traffic within weeks.


That is a given, Ever heard of cat and mouse? And no matter what we try to stop, we are always in "reactive mode"
ICI2I

63 Posts Posts
@Mastropi
I believe you wanted to reference point 4 rather than 3.

Please note, using active lists is one way of getting SIEM to perform well and steering away from multiple filters within a rule. Active lists in SIEM can be updated regularly with interfering SIEM system.

To identify threat actors for targeted attack I believe you need combination of points that I have mentioned and other external/vendor intelligence and obvious understanding of target - as ICI2I suggested we need to be in reactive mode - means a continuous monitoring will indeed help.

Forums such as this are used for sharing intelligence. A person can use this intelligence for good or bad, that does not mean we stop sharing our thoughts or strategies. In this cyber age we need sharing as much as possible.
makflwana

17 Posts Posts

Sign Up for Free or Log In to start participating in the conversation!