Jekyll requires blog post files to be named according to the following format:

YEAR-MONTH-DAY-title.MARKUP

Where YEAR is a four-digit number, MONTH and DAY are both two-digit numbers, and MARKUP is the file extension representing the format used in the file. After that, include the necessary front matter. Take a look at the source for this post to get an idea about how it works.

Jekyll also offers powerful support for code snippets:

def print_hi(name)
  puts "Hi, #{name}"
end
print_hi('Tom')
#=> prints 'Hi, Tom' to STDOUT.

Check out the Jekyll docs for more info on how to get the most out of Jekyll. File all bugs/feature requests at Jekyll’s GitHub repo. If you have questions, you can ask them on Jekyll Talk.

There have been more and more “discussions” recently that have crossed my radar in the Twitter-sphere talking about OST (offensive security tools), and the morality surrounding such things. The basic crux of the argument is that if someone were to create an offensive/hacking tool and make it freely available on a platform like Github, that they are inherently making the world a less secure place due to the ease with which attackers will be able to leverage such tooling for malicious means. This is a very complex topic, and the “discussion” on Twitter does not lend itself towards making meaningful progress on the topic. Twitter provides an easy mechanism to try to distill exceptionally complex topics into witty platitudes, which inherently means that most of the points that are made completely gloss over necessary details. Then, as soon as one side makes such a statement, the opposing side can point at all of the details that were glossed over, make their own platitudes, and the cycle continues.

This post is meant to dictate my actual thoughts on the topic. I will warn you up front that I don’t have a warm-fuzzy answer for the best path forward for this. However, I do believe the topic is sufficiently complex that if anyone does offer a simple answer, there is a good chance they are full of shit.

First we’re gonna talk about death

You heard me, we’re going there. If we’re going to address a topic that boils down to moral judgement, let’s take a look at a concrete example anyone can understand. Let’s start by making a statement that most people will probably agree with:

It is better to not be directly responsible for the death of another person.

Morbid, but sure. That sounds like a reasonable statement to make, and I imagine most people would agree with the sentiment. Let’s take a look at the extreme manifestation of a situation that contradicts that statement. Someone is a psychopathic serial killer, and they kill other people in cold blood with no justification and with no motivation other than to just do it; legitimate malevolence. I imagine that most people would look at that situation and say, “That is wrong.”, and rightfully so.

What would happen if we took a look at the extreme use case of cold blooded murder and attempted to use that as the foundation for a policy framework on how to handle any situation where someone dies? You quickly enter exceptionally dangerous territory. What about if someone died in a situation of self-defense? What if there were 2 people attempting to drive as safe as they could, but inclement weather causes a fatal crash? We don’t live in a world with binary outcomes, binary motivations, and binary circumstances. If you attempt to handle exceptionally complex circumstances based on the extreme edge case, the only people that will take you seriously are those that will blindly gloss over all of the nitty-gritty details of reality.

Good intentions gone awry

Moving on to a slightly more palatable subject, I think it’s worth exploring the gap between ideal outcomes and the downfalls of human nature. Let’s take a super quick look at the world of vulnerabilities in technology. If we were to distil what the best outcome would be about vulnerabilities, we could probably make a statement like:

The world would be a safer and more secure place if any time a vulnerability in a product was discovered, the vendor was contacted with the details and they immediately fixed the vulnerability.

Yeah, sure. If that happened in reality, we would end up in a world that would become increasingly more secure over time, which would benefit everyone. I imagine that this is the fundamental mindset of people that actively espouse the “responsible disclosure” model. I am willing to venture that many people that discover vulnerabilities began with such good intentions as well.

However, it doesn’t take much effort to find a string of vulnerability researchers with broken souls. Many that have become jaded from things like companies ignoring their critical findings, companies suing them from their research, companies silently fixing the vulns without offering any credit to the research, etc. etc. If you go through those experiences enough times, it is only natural to not feel morally obligated to freely share the fruits of your labor with a company that may feel inclined to persecute you for your efforts.

“But didn’t we just agree above that the world would be a better place if everyone did responsible disclosure?!”

Yeah, but responsible disclosure only works when everyone involved acts with the best of intentions. Maybe that periodically happens, but it is not always the case. To suggest that each vuln researcher must abide by such standards without equally stringent expectations of each and every company that they interact with is just naive. We live in a messy world, and Santa isn’t real. Sorry to break it to you.

OST - the lay of the land

Let’s take a step back into reality and explore what that means to offensive tooling that gets released on the wild internet. Fundamentally, I think that a sentiment that most people can agree with could be:

It is not optimal to take people with malicious intentions, and give them easy access to tools that enable them to act upon their malicious intentions; especially in a manner that could be difficult to detect or defend against.

Believe me, I have zero interest in giving an angsty-teenager or malicious person easy access to ./break-the-internet.py. So, what are we to do about this? People post tools and code to github that could potentially enable such things, and that is a use case we need to protect against, right? Well, this also is a massively gray area, where concentrating solely on the extreme edge cases does no one real justice. What are the various shades of gray in this equation? They include items such as:

• Learning offsec - For anyone that wants to get into pentesting or red teaming, providing open access to such offensive tooling gives them the ability to learn how certain technologies work and how certain technologies can be taken advantage of. If you restrict access to such tooling, you cut off a viable learning mechanism for a pipeline of people that could grow into this field. How do we solve for this issue? How do we propose new talent learns and hones these skills? Do we expect some kid in India that barely has access to a computer to start paying premiums for authorized offensive training for a potential career path?

• Open detection options - By creating open source/free offensive tooling it also provides benefits to defenders. There are a couple of ways that this manifests itself. First, it enables them to play with tooling that some attackers on the Internet use. It enables them to understand the limitations of the tooling, what artifacts it creates and how to potentially detect it, and how it could manifest itself in their networks. If you remove open offensive tooling from the equation, what is the mechanism for defenders to test the defenses of their network? Shall blue teams start to budget for approved offensive tooling for these things?

• TTP funneling - There is something to be said about understanding what kinds of tooling a defender should even prepare for. If your threat model includes the 5-Eyes Intelligence Community or the Mossad, then good luck. There are ways to defend your network from such threats, but may the deities be on your side, because you have an up-hill battle. But that’s not what we’re talking about here. This entire discussion is around putting tools into the hands of people that don’t have the skills to build it themselves and all they have to do it run it maliciously. Understanding this as our threat model, it means we can more easily understand what tooling we need to defend against. “What tooling could I easily find on Github to attack my network?” The types of tools that you find are probably going to be very similar to the tools attackers find, which can better inform defenders on what tools and techniques to prioritize creating detections for. For example, the moment subTee tweets something like, “I just had a really good Hawaiian burger”, you can bet your ass I’m immediately starting to wonder how I can detect pineapples in network traffic.

• An OST by any other name - There is something to be said about who gets to define what is truly offensive in this case. For example, is nmap an offensive tool? It is used maliciously in some cases, absolutely. But when a sys admin uses it to check if a port is open or reachable, is that offensive as well? What about netcat? Because it can be used as an offensive tool, does that mean that it is an “offensive tool”? What about powercat? The list goes on. Now, there are some things that I think are easily definable in this category, such as a proof-of-concept for a RCE. There are few legitimate ways that this could be used (even though there is still an argument that a sysadmin could use such a proof-of-concept to confirm whether his/her infrastructure is vulnerable to attack). But there is a very broad category of tools that fall into muddy waters and are not easily defined. Who gets to define what is “offensive” or not?

OST - the discussion

So what are we to do? Many of the conversations that I’ve seen on Twitter are basically guaranteed to fail. In my naive observations, the conversations have gone kinda like this (yes, I am oversimplifying a bit, but below is my rough interpretation):

A: “It is bad that people are open sourcing offensive tooling, because it makes it easier for attackers to up their game with little to no effort.”

B: “Aiding attackers is obviously not optimal, but there are a number of other details about this subject that need to be considered…(at which point they talk about any of the above bullet points). What do you propose is actually done about this?”

A: “We need more regulations on people releasing OST’s!”

B: “Okay, that sounds like it could turn unreasonable/dangerous very quickly. How do you propose this actually gets accomplished without neutering all of the benefits that have already been listed?”

A: “No one is able to provide a meaningful argument that we SHOULDN’T regulate OST to keep it out of the hands of attackers!”

B: “Whaaaa?”

So here we are. There are people that appear to be in the camp of “we should regulate OSTs to make the world a safer place”. Yet, to the best of my understanding, no one has actually provided concrete recommendations of what, exactly, should be done. I imagine that most of us would agree with the intention of the argument, but see an equally dangerous slippery slope of where “regulations” could lead. A few examples of potential slippery slopes that could be introduced would be things like:

• What does “regulation” even mean? Do people not get to post certain kinds of code to Github anymore? Are there technical restrictions imposed to prevent people from posting certain things in the first place, or social/legal ramifications for doing it?

• Who are the gatekeepers of these regulations? Who gets to decide what is right or wrong? What is offensive or sys admin? Who gets to decide what the ramifications are for breaking the rules?

• Whoever these gatekeepers are for regulations around OST, what standards are they held at? How is the community ensured that everyone’s best interest is actually accounted for? Looking at vulnerability researchers and “responsible disclosure” you can get a glimpse of what can happen when you try to dictate the actions of the security community based off of moral decisions that only apply to them. You drive talent underground. Is that what we want?

• Let’s pretend there is an agreed upon group of gatekeepers and they even create a set of regulations that is as reasonable as possible. What do you think would happen from there? Do you think people will stop people from developing OST’s in general? The answer is likely not. They will just change how they socialize and share the code that they write, and assuredly this will manifest itself in more disparate and closed venues. As you end up with more tooling that is shared in this way, will that really provide a net positive for defenders? Also, to clarify, if your fundamental position boils down to, “people just shouldn’t create hacking tools in the first place!”, then I don’t know what to say to you. People will do that. It’s the itch to figure out how certain things work. It’s the curiosity to poke what no one else pokes. Saying people just shouldn’t do such things, you’re effectively saying, “why can’t everyone be like I expect them to be”, at which point you are already in a losing argument, and trying to create suggestions based off of such viewpoints will never stand reliably.

And here we are

Yup, I don’t know exactly what to do about this. Is there a way to make the world a safer place without sacrificing all of the benefits of our current system? Probably, but assuredly it’s not gonna come from an arrogant person that pounds their fist on the table and says, “we need arbitrary regulations to solve this, and if you disagree with me you are part of the problem!”

One thing that I think could be a reasonable step forward could be saying something like, “Hey, if you are creating and openly distributing an offensive tool, please publish it with a corresponding way to detect it (or, if you don’t have a detection, please be open to posting a detection for your tool created by someone else)”. I feel like that could be a reasonable expectation we could socialize within the community. Some OST developers may even be kind enough to build certain detections right into their tooling.

But who knows, maybe that isn’t the next best step. I don’t have the answers myself. The one thing that I’m sure of is that we, as an Infosec community, will not make progress on this subject if we can’t agree on our actual goals or have conversations with the appropriate appreciation for the complexity at hand. If you’re the kind of person who wants to continue handwaving nuance while just spouting empty moral platitudes, well, you deserve the inevitable hand-wavy snark that is coming your way.

One benefit of being around the InfoSec industry for a while is that you get a longer snapshot of time to observe trends, mindsets, and methodologies. You get to watch, over the long-haul, what tends to work and what doesn’t. For example, looking back at security conferences 10-15 years ago, most conversations and presentations felt like a blue team pity party, where the tone boiled down to, “woe is the security industry, attackers will always win, defenders will always lose.” Over a period of time, there developed pockets of blue teams/defenders that started to alter their mindset of the situation.

“Maybe we can’t drastically alter the initial barrier to entry into our network for attackers, but maybe we could become exceptionally good at identifying them and kicking them out of our network before before they can do anything meaningful.”

By altering how they view the situation, some teams were able to build tooling and processes to do just that. Teams that get good at this can drastically raise the cost for attackers to operate effectively, which I would count as a win.

What does this have to do with phishing?

If you’ve been in any security meeting/conference/talk in recent years, you’ll find another manifestation of the above woeful chant. As soon as the speaker asks the question, “What is the weakest link in any network?” you can almost feel the entire audience think in unison:

“The user is the weakest link.”

And we tacitly accept this. We just accept that users will click on every single link that shows up in every single email. Every attacker is confident users will click their links, and every defender is confident there’s gonna be a mess to clean up afterwards.

I don’t believe this needs to remain the case. I believe that we, as an industry, could alter how we approach the problem to make meaningful progress in this area.

The Old Way

The way that many companies attempt to solve this problem traditionally contains some mix of the below items:

• As employees are onboarded, part of their onboarding may include a phishing awareness training

• Employees are required to periodically (like annually?) complete their phishing awareness training

• InfoSec teams or security consultants are brought in to phish the workforce, identifying who would still fall for phishing attempts from real attackers

• Those employees might be disciplined, put on a watch list, put back in a queue for more phishing campaigns, or required to take more rounds of phishing awareness training

We do such things, people continue to click on links, then we sit back and say, “see, these people never learn, they keep clicking on links in phishing emails…“

Why do we do these things?

There are completely valid reasons for us to be doing things like these, but I’m not sure if most of us easily understand or could articulate what they actually are. If you ask yourself “why” enough times, you can get to the heart of it. Such as,

“Why do we do phishing awareness training?”

Because phishing is a dominant method for attackers to get access to our network, and we don’t want our employees to fall for such things.

“Why do we perform periodic phishing campaigns?”

Because we want to identify and validate who didn’t understand the phishing awareness training and still leaves the company vulnerable to such attacks.

“What should we do with those people?”

We want to enable and empower them to not be as vulnerable to such attacks.

“How should we accomplish that?”

Now THAT is a fantastic question. If, after completing any arbitrary number of phishing awareness trainings people continue to fall for the same phishing techniques, why do we conclude that more of the same training will yield any better results? If a child has a hard time understanding basic addition, we expect their teacher to find a different way of teaching them in a way they can understand.

How is it that we desire behavioral changes from our coworkers (the act of better recognizing phishing emails and NOT clicking on the links), yet when our “industry standard” training fails to elicit the behavioral changes we desire, we just throw our hands up in the air and write off the users themselves as opposed to exploring better ways of reaching them?

You know what they say about assumptions…

In my proposal there are some caveats and assumptions that I am going to make. The ones that I can think of are:

• This will not address every use case and edge case that we care about, and I am not attempting to do so. This is primarily geared at the very basic use case of, “we want to ensure our co-workers are as resilient as possible in identifying and not falling for link-based spear phishing”.
• You have employees that generally want to do the right thing. This may sound odd, but if you have malicious (or maliciously indifferent) coworkers that don’t care about downloading all malware under the sun, then no difference in approach may matter.
• You have an InfoSec team that is willing to work with employees in investigating potential phishing. Remembering that our goal is creating resilient co-workers who don’t fall for phishing. There will absolutely be times where they will know enough to be suspicious of an email, but not know enough to determine if it is truely malicious. In those cases, there is a good chance they may reach out to Infosec for a sanity check. If the Infosec team responds in ways that negatively reinforce that interaction (being gruff, dismissive, condescending, etc), that will deter from the exact behaviors we are trying to develop.
• These phishing campaigns are approached in legitimate good faith with the intention of educating and changing the behaviors of our co-workers, and not with a “gotcha!” mentality. No body likes to be tricked, especially in bad faith. If this is performed in a manner that comes off as condescending, then general trust will be lost with the people whose behavior you are seeking to change. That is never a good place to be.

Dude, get on with the proposal…

Given the fact that we are seeking behavioral changes, what if we target our phishing campaigns to provide feedback to our coworkers that is immediate and relevant.

What if we created our phishing campaigns in such a way that, when a user clicks on a link they are taken to a page that immediately notifies them that they got caught by a phishing campaign. But not only that, the page contains easily digestable information for how they were caught by that exact phishing campaign. This would include things like screen shots taken directly from the phishing email they just clicked on and probably still have open in a tab that highlight things like:

“Here’s a screenshot of someone’s name you probably know, but coming from an email address that TOTALLY doesn’t match.”

“Here’s a screenshot of a link that looks fine, but when you mouse over it, you can see it’s gonna take you somewhere totally different.”

etc.

Here are a couple reasons why I feel like this approach is so critically important:

• Timely - This feedback needs to be timely. For example, if you are potty training a puppy, if it pees on the carpet, you can’t wait until the next day and then punish it. The association between the behavior you are trying to change and your feedback is totally lost. How in the world can we phish coworkers, put them through follow-on training at an arbitrary time in the future, and expect them to ever understand what they should have done differently at the time that really mattered? Also, even if they remember the phishing email in question, almost assuredly the context of why they clicked will be lost or muddled (such as recognizing things like, “I clicked because I felt a sense of urgency because it looked like it came from my CEO, but I forgot to make sure the email address matched…”)
• Relevant - The feedback needs to be relevant. People can alter best if they can understand, concretely in that moment, what was wrong. Even if you immediately inform them they got phished and link them back to your general phishing awareness page, if it’s not pointed out to them directly, how are they expected to decipher exactly which tactics tricked them? For example, I am trying to learn how to work on cars. If I mess something up on my car, it would not be helpful to hand me a 300 page manual for everything there is to know about my car. Sure, the answer is assuredly in there, but I don’t know enough to know exactly where I went wrong and where I should look to correct myself. I need someone to take me by the hand and say, “Dude, you were trying to change the oil. The bolt you took out drained the coolant, not the oil. You should put that back and take out that other one…“ This is why it is critically important for us to provide the feedback to our coworkers in the form of the phishing email that they still have up right in front of their faces to highlight what they could have caught and what should have jumped out as being suspicious to them.

I see your logical fallacy, and raise you another logical fallacy

I am wrong about many things, and perhaps this is another example of where I am wrong. Upon some very brief socialization of this thought process, below are some of the counter arguments (both that I have actually heard, as well as those that I internally imagine that could be presented), along with my counter-counter arguments:

“Well, if you provide this immediate feedback, it may happen to one person, then they tell their coworkers about it, and those people won’t click your email so it will skew your metrics.”

I mean, yes that is true. But my take is “coworkers socializing recognizing phishing campaigns should be a positive behavior we reinforce, not a negative one because it potentially throws off the numbers we get to present to management”. Remember, the entire goal of this is to elicit positive behavioral changes.

“If phishing is one small link in an overall red-team chain of attack/engagement, then of course, you wouldn’t leverage this approach.”

That is absolutely a true statement. If the intention is ‘phishing as part of a larger operation’, don’t do this. But if we are phishing as an exercise to create a more resilient workforce, then I think we should do this.

“This will require more effort on the teams creating the phishing campaigns, because you have to tailor each landing page to the campaign”

That is true, but it is important to keep in mind what problem we are solving for. The entire point of this is to solve for trying to change end-user behaviors via direct educational feedback. If “phishing engineer time” is what we are trying to optimize for, then I would argue that we are focusing on solving the wrong problems.

“This kind of education could result in the workforce looking more suspiciously at their email in general, resulting in more frequently pinging Infosec for sanity checks on potentially non-malicious things.”

Again, in my mind, that would be a successful outcome of an effort like this where the relationships between Infosec and coworkers could be groomed over time to make the situation better. However, as called out in my assumptions, some InfoSec teams may not want that level/kind of interaction/engagement.

“You don’t know if this will work!”

That is absolutely a true statement. But the problem is that I don’t know if this would work because I am not aware of anyone that is actually trying this approach. I would love it if someone actually tried this and was able to measure if it elicited the kinds of changes that we, as an industry, are desiring.

“…C is for Conclusion…”

I believe that, as an industry, we have been going through the motions with all of these phishing awareness trainings and phishing campaigns. We are fantastic at “doing what’s always been done” and measuring how poorly our coworkers are at recognizing phishing emails. We are just as fantastic at complaining and blaming them as opposed to taking the onus on ourselves to figure out a better way to address the real issues at hand.

I think there are real tangible ways that we can make meaningful progress on this front, but the required changes will have to start with ourselves.

If you have any feedback on this, please feel free to shoot me an email, shoot me a tweet, or even shoot me with a rubber band. I’m usually always happy to chat and reconsider my thoughts on these things where warranted!

(p.s. I realize that I didn’t mock up an example of what I am thinking about the kind of educational landing page for a phish that I described. I am horrible when it comes to UI design and wouldn’t trust myself to create an example of what a ‘gold standard’ would look like, but if it would be helpful as a starting example, feel free to ping me and I’ll be happy to give it a go.)

Let’s be honest, I am biased. You are biased. Repeat after me kiddos, points to the blackboard “he, she, we are biased”.

There are a plethora of things that contribute to this, including the culmination of our lived experience and perceptions of reality at any given point in time. These color the lens through which we view the world. The product of all of these factors results in the biases we have, the unconscious filters through which we interpret the world and inputs around us.

Why am I talking about this? Because I am very interested in analysis, and as pointed out here, analysis is the observation of a data point and making an inference off of that observation. If you are not careful, the biases that you bring to the table can jade how you interpret data, which can result in you coming to incorrect conclusions and making incorrect decisions because of it.

For example, in The Psychology of Intelligence Analysis they discuss a fascinating study. The gist is this, someone is shown only a small portion of a picture, and they need to guess what the picture is of. Slowly, the picture is completely revealed. As the picture becomes slowly more revealed, the subject has to keep guessing what it is of. The result was pretty fascinating:

The earlier someone had to form an opinion of what the picture was, the longer it took and the more concrete data was required for them to change their mind. The longer someone waited before they had to form an opinion on it, the easier it was for them to change their minds based on the increasing amount of data presented to them.

Why does this matter to us? Well, if we don’t recognize the biases that we have towards the data we work with and how to counter them, it can make our jobs/lives a lot harder.

Story time of how I almost failed

There I was, about to sip some coffee, starting out the day. “Hrm, what do we have here…an alert for user brute forcing?” Yup, there it was, an alert that there was a workstation on our network that had generated roughly ~16k failed authentication attempts in the past few hours.

“Shit, ok, gotta figure out what’s going on…something trying to brute-force a user’s password? Something trying to brute force a bunch of users’ passwords?!”

The clock is spinning…

“Okay, take a step back, something must be running on this machine that is trying to do all of these lookups. Let me see if I can find the process in ${EDR tool}….shit, the workstation isn’t there!”

The clock starts spinning a little faster…

“The workstation is on a VPN IP. Okay, I have the username of the person that’s VPN’d in (we’ll call her Sally). Okay, workstation name looks like a personal computer they used to VPN in…damn, that’s probably infected to hell. What did they download recently? We don’t have ${EDR tool} on their personal machine, so no way for me to figure that out. Should I kill their VPN? Lock their account? Send them a message trying to figure out what they installed lately?”

The clock is working in overtime…

“Okay, these are failed authentications. Let me go check the Active Directory Domain Controller logs…if there are thousands of failed authentication events for this user’s account maybe someone is trying to brute-force their password. They’re VPN’d in, so they already had to successfully connect with creds, why would malware be trying to brute-force their creds if they just successfully logged into our network? I dunno, I’ve seen malware do dumber shit…”

…or maybe…

“Maybe if I see this machine failing authentication for a ton of different accounts! I can see what accounts it’s trying to break into! Then, maybe I could take the list of accounts and figure out more about this malware! If it looks like legit internal user accounts, this could be a well informed attacker! If it looks like generic usernames, maybe I could take those and try to figure out what malware does that. So even though I have no access to this person’s computer, maybe I can get a clue about what’s going on…”

…my body is ready, give me them DC logs…

And I see ~16k failed logon events in all their glory. Looking through them, every single one of them was a Windows EventID 4625 (failed logon attempt) with a Status Code of 0xC0000064 (username does not exist). The username of every single one of them was for Sally (like, SALLY-WORKSTATION/sally).

…wait, u fokken wut m8? What is going on?

After consulting with far wiser people in my IT department, it turns out that this is a known thing. If someone VPN’s into our domain on a personal machine, there are some domain resources Windows will try to fetch. However, the client’s Windows OS won’t prompt the user to enter domain creds to access those resources, it’ll just try to use the user’s currently logged in creds in the background. Well, obviously we don’t create AD objects for people’s personal computers/user accounts, so in the background of the user just being VPN’d in, they were racking up failed authentication events on our domain controller like they won the jackpot in Vegas!

Miley Cyrus remix - I (could’ve) CAME IN LIKE A WRECKING BALL!!!

There were multiple times throughout this process that I could have (in semi-good-conscious) ruined Sally’s day. She pops up as a culprit for account brute forcing, and there were a number of times that I was about to kill vpn/lock out account/etc until I figured out what malware was running on her machine. Not having all of the information made it exceptionally easy for me to jump that conclusion. It doesn’t help that the alert that I was responding to, account brute forcing, colored how I was interpreting the situation. (Side note: this is why it is so important to do everything you can to understand the logic behind the alerts you investigate, so you are familiar with where they might fail, as described in my blog here )

I have seen/heard of a number of times when the SOC takes nuclear measures before they have all of the data. I have also seen analysts that look at data and initially come to a conclusion about what is going on, and no matter what new information is presented to them that contradicts their conclusion, they never give it up.

Do not let this happen to you. Learn to recognize the biases you start investigations with, because if you can’t even recognize them, how will you ever know when they take you down the wrong path?

Again, hit me up with any feedback/comments/questions/etc. Go forth and do good shit fam!

There are a couple different ways that people can work with detections. This is a follow-on article to my other post on detections. There are different ways that you can build and use detections, but it’s important to be cognizant of what they are and when to use them.

Detections are like tools in your work shop, there are different kinds, and some are more appropriate for certain jobs than others.

In this article, we’ll be exploring a couple types of detections - atomic and composite. Full disclosure, these are just the terms I’ve become accustomed to amongst my circle of friends when we talk about these things. If there are “industry accepted vernacular that represent these concepts in far superior way to best reflect the intellectual elegance of what is being discussed”, then I don’t know it. :)

“Atomic” and “composite” is what you’re stuck with now from me.

Lay the groundwork

One of the primary goals of a SOC/blue team/etc. is to catch attackers as soon as possible. The idea is that the sooner you catch them, the sooner you are able to figure out exactly what they’ve done, and respond appropriately and kick them out. The concept of this is sounds pretty simple on the surface:

Figure out what things an attacker is likely to do, then detect that stuff.

I know, this sounds so dreadfully simple, but bear with me, because this concept really matters in how you approach detections.

Atomic Detections

The concept of an atomic detection is pretty simple, you create a detection that relies on a single event that matches whatever criteria you are looking for. Below are some examples of what that may look like:

• A computer makes a network connection to a known-bad domain that hosts malware
• Anti-Virus identifies a known malicious file on a host
• Something like msword process spawns cmd.exe

There are a plethora of other examples, but these highlight the concept of an atomic detection. It basically boils down to an event that, if it were to happen in isolation, its mere presence alone warrants showing up in front of a SOC analyst’s eyes.

Composite Detections

The idea of a composite detection is that it will only fire if it sees a culmination of pre-defined events. As noted in the article I wrote on analysis, one form of analysis is not only making an inference off of observing a single data point, but creating a more complex inference based off of observing multiple data points. An example of this showed up in the example I used talking about biases on account brute forcing. Seeing a single failed logon attempt by itself may not warrant an investigation, but seeing 1,000 (or another arbitrary number) of failed logon attempts within a small time frame would warrant a detection to fire for an analyst to investigate.

Exemplar Example of Exampleness

Let’s look at an example that has worked before. But before looking at the example detection, let’s briefly look at the mindset behind it.

We want to detect attackers as soon as possible. If our detections don’t catch them getting a foothold in our network, what do we think they would do next, and can we detect that?

Cool, so we have a bunch of detections looking for attackers getting into our network (IDS/AV/phishing/etc), but if they somehow get past all of that, it may be reasonable to assume they will perform some sort of reconnaissance (obligatory shout-out to Mitre ATT&CK Framework ). What would that recon look like? Well, maybe some of the commands could be:

• whoami
• id
• ipconfig / ifconfig
• netstat
• ping
• net user
• tasklist
• systeminfo
• cmdkey /list etc.

Cool, here’s the problem. If you just create alerts for the execution of those commands, you and your analysts will wish for the sweet consequence of committing seppuku multiple times a day. It will be healthy for no one. However, if you were to take the culmination of all of these events, the chance that your EDR product would see a single user running 3+ of these commands in a short timeframe is (probably) exceptionally low. This could be a fantastic method of catching those squirrelly actors that seem to find an imaginative way to get into your network. (Quick anecdote, a long time ago in a former life I was looking at some data where a threat actor was able to hack supply chain to get access to a target’s network. Honestly, good luck catching initial access via supply chain, but once the actor got on the network, they ran a handful of recon commands to see where the landed, and it was exactly an analytic like this that initially caught them.)

Caveats

Yes, I know. Building a combination of atomic and composite alerts may be great, but that will not catch absolutely everything. This is a cat and mouse game, the back and forth between attackers and defenders will never stop. Great attackers who understand defenders will always be exceptionally dangerous and hard to catch. Our goal is to make the bar as high as possible to increase the cost to attackers for looking in our direction.

Outtie like a belly button

Here we explored a couple methodologies for creating detections to catch attackers. Basically they fall into two categories:

• Atomic - “If this certain singular event happens, notify someone”
• Composite - “If a combination of <x,y,z> events happens in a timeframe, notify someone”

While they are both aimed at identifying attackers as soon as possible, they approach it in slightly different ways. There is value in both of these ways, and neither should be used to the exclusion of the other. The question you should be consistently asking yourself is, “If I were attacking me, what would I be doing?”, and the answer to that question can help guide what you should be looking for. Sometimes there is a single thing you’ll expect an attacker to do, and sometimes there is a combination of (otherwise benign) things.

The ideology of not catching every single hacker on keyboard should never stop you from making progress where you can. Go forth and raise the bar one notch higher every time you can, because it’s the culmination of all of these little steps in defense that will make your organization an absolute pain in the ass for an attacker to get in and stay in.

<3 you all. Please feel free to shoot me any feedback/comments/questions/concerns/etc.

Let’s take a look at a potential scenario some companies may find themselves in…

• Data is coming in ${somewhere}…like a SIEM or something like that. Let’s say you have EDR data (Sysmon, CarbonBlack, etc.), AV logs, server logs, network logs, etc.
• Alerts are being generated off of that data
• Those alerts make their way to the eye balls of analysts to evaluate how legitimate they are

Cool, sounds great. If something bad happens, it should generate an alert, that’ll get in front of an analyst. If it’s truly bad, they should be able to figure that out and respond appropriately. Remember, our goal as defenders isn’t to create detections to catch every single thing an attacker does; it is to catch something an attacker does that leads us down the trail of breadcrumbs to find everything they did and kick them out as soon as possible. (a good article on this was written by @thegrugq here )

That’s a wonderful fairy tale that probably isn’t as close to reality as many of us hope. Why is that? First, let’s discuss where detections come from.

Rough Types of Detections

Where do our detections come from? You see, when a mommy detection and a daddy detection love each other very much…

Oh, sorry, wrong blog…

There are probably a couple ways you get detections in your environment. There are at least a couple more posts I want to write on detections, so I’ll try to contain myself here. But let’s look at how detections will probably manifest in your environment:

• Vendor created - An example of this may be using some anti-virus software. If someone downloads this_totally_isnt_a_bitcoin_miner.exe, AV may catch that and create an alert saying it caught it. This could also be an EDR tool running on your endpoints that comes pre-configured to create an alert if a process does something it is told to look out for. What gets detected here is purely based on the logic of the tool that is evaluating the data. You may or may not be able to figure out how valid/trustworthy this detection is.
• Threat intel - I have some strong opinions about threat intel, but the world doesn’t revolve around me, so it’s still part of the cYb0r-3c0noMy. Companies pay for feeds of threat intel that they then try to match against the logs they have coming into their SIEM. If something in your logs matches something from a threat intel feed, that may generate an alert. With this as well, there is an exceptionally high chance they will have no idea what logic/data was used to generate that alert.
• Custom alerts - These are the ones you/your team create yourself. These are the ones where you’re like, “Hey, if you see Bob from HR accessing our sensitive source code repo, alert on that”. This is also the custom detections you may write against your EDR data, a very common example would be looking for whenever cmd.exe or powershell.exe are spawned from msword.exe (as this would be a very strong indicator someone got phished). Depending on how well you play with others, you may develop some friends that will share the custom detections they wrote with you.

Cool, now we’re sitting pretty with these detections rolling in. They show up in the system, and an analyst responds to them. Life is good…or is it?

Story time with Cthulhu

Quite a while ago a handful of friends from different companies were chatting about detections they use. We each shared some methodologies and detections with each other (rising tide raises all boats and all that).

“Hell yeah!” was my general take away from that, “We’ll be able to detect so many moar things!” (I was so naive in my youth…those were the days…)

I slept even easier knowing that if shit were to pop off, there was a great chance we’d catch it! Fast forward some time, and I was going to test out some attacker tool/technique in our environment. “This will create an alert, but I want to see what an attacker could do with this on our network anyway”, I thought. Well, I ran the tool and …drumroll… no detection fired.

Going back and looking through the (rather complex) logic for what should have fired, it turns out the logic got fucked up, and there was almost no circumstance in which it would ever fire.

Yeah, my world-view was shattered. It raised the question…all that time that some detections weren’t firing, was that because there was no malicious activity happening? or because the detections weren’t working the way we thought they were?

Knee-jerk reactions aren’t known for their subtlety

So, what do you do now? Well, luckily, the methodology to fix something like this is already a solved problem in the software world (I’m looking at YOU, unit testing). The concept of unit testing is pretty simple; there are tests created to ensure that bits of code work as anticipated. Sadly, I haven’t seen or heard this applied much in the context of detection and response, even though I think we absolutely should.

What would this look like? Well, here is how I’ve seen it implemented before:

• You have one (or more) test machines/VMs in your network
• It is imperative these machines have the exact same policies/tools applied to them that every other machine does
• Those machines feed their logs to a system that uses the exact same detections as what you have running in prod
• You run an attack that you think you will detect
• You observe if a detection actually fires
• You reset the machine and start over

To do this in a way that is continuous and scales it will require you to have a corpus of “attacks” you can run on your test boxes. There are products out there that offer suites of threat emulation that could be built into a pipeline like this. But even if you can’t get your hands on something like that, you could still run most of these manually.

The beauty of doing these things continuously and building a pipeline for this is you could get immediate feedback and notice changes over time. For example, did IT make a change to group policy last week that took an attack that didn’t used to work in your environment, and now it will work? Having a system like this will be able to give you that feedback.

I have my goals but the Universe is playing goal-line defense…

What are your real goals from doing something like this? There are a couple of them:

1. Identify detections that aren’t detecting activity you think they should be. For example, let’s say you have a detection looking for psexec; if you run psexec and that detection doesn’t fire, it is time to go back to the drawing board on that one.
2. Provide a feedback loop to improve detections that don’t work very well/are too niche. Let’s say you have a detection for Mimikatz, and that detection is looking for any process run from C:\Windows\Syystem32\Mimikatz.exe. “Awesome, now I can sleep well knowing that we’ll detect Mimikatz in our network…“, you may be thinking to yourself. One question you should always be asking yourself is, “Under what circumstances would an attacker do this thing, but my detection not catch it?” For this example, what if the attacker runs Mimikatz from a temp or downloads folder? What if “Mimikatz.exe” is renamed to “Mimidogz.exe”? Or “update.exe”? Attackers can be crafty, and just renaming an attack tool happens relatively frequently. If you slightly alter one of these attacks on your test box, would your detection still work?
3. You may find yourself in a situation where you are running certain attacks on your test box and realize that you don’t even have the right logs to be able to identify the activity/build detections off of. This is a fantastic and concrete data point that you can raise with business stakeholders to try to get buy-in for that kind of visibility!
4. It’s not always feasible to build detections for all permutations of how an attacker may manifest themselves in your environment. But over time you will create better detections and you will have to get far more creative to pull off attacks in a way to evade them. That will have raised the bar high enough that attackers will have to expend more and more effort to do the same attacks, which is part of the point of all of this.

Outtro

Trust me, the idea of a big CI/CD system for unit testing your detections may sound daunting and unrealistic for most people. But consider something like that a north star; most people should be able to coordinate doing some manual testing of attacks in their network to ensure you’re able to see and detect some of the low hanging fruit you may expect to see from attackers.

Don’t become complacent in the concept of “I don’t have to worry, we have a detection for that…” if you haven’t actively seen that detection fire on what you expect. Too many people (myself included), have made that mistake.

There’s at least a couple more blogs in the pipeline about detection methodologies, but I’m happy to respond to any questions/comments/concerns. Go make good shit happen today!

If you look at the make-up of a blade, it consists of a piece of metal where two opposing sides of it are carefully filed to specific angles to create an edge at a specific degree. If the two sides of the metal are grinded/filed at specific degrees, they meet in the middle and can create an exceptionally sharp edge.

To me, this is a fair representation of our skills as InfoSec professionals from the offensive and defensive perspectives.

I believe that to be truly effective as an InfoSec professional, you need to hone both your offensive AND defensive skills.

I mean, like, duh. This conversation has been played out ad nauseum in some ways. “Think like an attacker” is the catch-phrase for all of that offensive training that permeates the InfoSec world. The concept is that, as a defender, if you learn how to “hack things” you’ll learn how to defend those things. There is also the fantastic article about how Defenders think in lists, attackers think in graphs. The goal of this is to take the attacker mindset and implant that upon the defenders to better equip them with what to expect and how to operate.

One thing that I think is an interesting omission in this conversation is what attackers can learn from defenders. For example, as an attacker, if you are intimately familiar with the biases of defenders, you can better prepare your tactics to take advantage of that. For example, looking at the netflow example in this analysis post, the natural inclination of many analysts would be to view that as SSH until proven otherwise. If you just have some version of netflow it can be hard to determine what protocol is actually being used on a port, so it’s easy to go with that assumption.

An analyst worth their salt may then want to check if SSH is actually open. In the example we’ll say the machine with SSH open is in their network, so they may have the access to determine if it a running service on that machine. Lo and behold, we’ll say SSH isn’t running. That’s odd, could that imply that port 22126 is open on the other machine for this connection? Let’s say you use nmap/masscan/shodan to scan that port on that IP, and it’s closed. Then what? As a defender, it can be difficult to determine where to go from there.

Without getting into the details at the moment, you could customize your C2 and malware to accommodate for these defender biases and thought processes to make your implants exceptionally harder to find.

As an industry we have very much normalized defenders learning the attackers’ tricks of the trade; but if attackers truly understood the biases and presuppositions of defenders, they could find very interesting ways to up their game as well.

But that’s the entire point, you have to sharpen both sides of a blade for it to be truly effective.

Go to any SOC or blue team or similar org and look at multiple analysts. Over time you will probably identify some analysts “do better” than others. There are a variety of reasons why this may be, but we don’t usually have good solid advice for what to do about it. Or even if you remove the inter-personal comparison aspect of it, if you were the only SOC analyst in an organization, what’s the answer to the question, “How do I get better at this?”

For those analysts that don’t appear to be performing as well as they can be, how can they improve?

If someone is just beginning in their career, advice like “just work more tickets” and “git gud nooblol” aren’t helpful.

Strap in your cliche seatbelt and get ready to bear with me here. I think that we don’t ask the question, “why am I here?” enough of ourselves in regards to our jobs, and what that means for the work we do. This is meant as a follow-on to the previous article on analysis and most specifically targeting people in traditional InfoSec positions, but hopefully could be interesting in other ways to other people.

Answering the question

Here’s the trap I see many InfoSec professionals sink into…it’s like the scene from the Office Space when the Bobs are asking Glen, “What would you say you do here?” Then the security guy says, “I take the alert from ${tool A} and I close it in ${tool B}!”

“Why am I here….like, really?”

• I monitor for alerts from ${x} system…
• I look for threats in our network…
• I attack our own network to dunk on the blue team…
• etc. etc. etc.

If your answers are anything like the above, it may be worth doing some deeper thinking about that question. For most of us that work in the security field, whether directly for a company or contracting for customers, really that answer boils down to “protecting the company”.

Thanks Captain Obvious for the astounding observation…

Hear me out, because I think there are aspects of this most people don’t think about. I think many people view their work functions without the full context of where they fit in the big picture.

The Big Picture

Companies do stuff. They generally hope to do stuff to make money. Ideally, if they can do stuff efficiently and well, they will be better at making money. If a bunch of unanticipated roadblocks get thrown in the way of how the company hopes to operate, that can impede their ability to make money. That is where we come in. Generally, we attempt to identify these areas where things can/have gone wrong, and either fix them where we can, or inform the company of the situation where we don’t have the authority/permission/buy-in to fix it. With the company informed, they can make the decision about balancing the risk versus reward of addressing or not addressing the issue we are raising.

Why am I still reading this? Where is this going?

As mentioned above, most people don’t consider the work they do in the proper context of the big picture. Here is an example of what the big picture may look like:

• Company uses systems (networks, workstations, servers, etc.) to perform the functions they hope will make them money
• Those systems generate logs about what they’re doing, and they get sent ${somewhere}
• Some threat hunter-like person somewhere develops a detection that says, “Hey, in these kinds of logs, if you see the pattern ${x}, that is bad”
• Inevitably that pattern matches, and it generates an alert
• There is an analyst that looks at that alert and has to decide, “Is this really bad? and if so, how bad?”
• Depending on the answer to that question, they decide how to proceed from there

Mixed in with all of this, we may also have a pentester or red teamer that is also attacking the company’s resources with the goal of understanding things like:

• If an attacker used tool/technique ${a}, would it actually work?
• If an attacker used tool/technique ${a}, will it get caught?
• When gaps are identified, work with the blue team/defensive team to fill that gap so no other true adversaries can take advantage of it

An example as concrete as silly putty

Let’s say you’re a SOC analyst. It is not abnormal for many SOC analysts to view their job as clicking through tickets. However, if we were to slightly switch the context, we could reframe that as:

my role is to evaluate data points to determine risks to our company and react appropriately

What would something like that really look like? Let’s say you get an alert (maybe it’s an anti-virus alert, maybe it’s some behavioral detection, whatever…), your goal is to analyze how accurate that is. When you see that, below are some questions you can start asking:

• How accurate is the detection that lead to this alert?
• Do you know the logic behind this alert?
• How much do you trust that alert?
• What other data can you use to determine how legitimate that alert is? Do you have access to that data?
• What assumptions are you making about this alert? (this is a huge one with many sub-topics and will probably be the subject of another post)
• What possible explanations are there for this alert firing? How can you test those explanations?

By being able to answer questions like these you will be able to better assess the actual risk and optimal outcomes of that alert. For example, you may be able to identify this alert is a false positive and prevent some poor user from having their machine isolated/re-imaged/etc. You may investigate this alert and identify a chain of legitimately malicious activity. Or, you may dig into the alert and identify the fact that there is another set of data that you’d need to fully determine how legitimate it is; which would be a fantastic use case to raise with the business/${other teams} about getting this data going forward.

…and now for a diatribe…

Ha! Suggesting we can make other teams give us data we want. That’s funny, we (security people) are pretty powerless in the kinds of data we can get our hands on. We don’t have buy-in from the business and are mostly an afterthought when it comes to the business.

Trust me, been there, know that feeling. Here’s a secret.

Lean in a little closer to the monitor and I’ll tell you….

…

…closer…

…ready?

As an InfoSec professional, maybe half of the problems you’ll have to solve are technical. The rest are social, relationship-based, and communication-based.

Yup, I took what this post was supposed to be about, got distracted on a diatribe, and took you all into left field with me. But even after ~pretending to~ proofread this, I feel like this is important enough of a point to diverge from what I really wanted to tell you. Many times, the reason that the business doesn’t give you buy-in on logging that one thing, or getting telemetry on that data is because they haven’t been given a good enough reason to.

“B/c ScUriTy!” will very rarely work, yet that seems to be the tone of a number of justifications that I’ve seen/heard in the past. Remember, the business exists to make money, and more than likely they aren’t making money by selling your work in security (i’m handwaving infosec consulting firms atm, even though their own internal security teams count as sunk cost (assuming they have their own internal security team…)).

If you frame the conversation in the light of helping the business continuing to make money, that can drastically increase the chances of success. An example of this could be something like:

“We get ${x} number of anti-virus alerts in our Windows server environment a week. However, because we don’t get Sysmon-like logs (as an example) from these servers, we don’t have the ability to quickly identify when these alerts are valid. This could result in an inability to detect or investigate a large-scale compromise in our server environment that could affect our users’ ability to use our product.”

This approach still pleads the case for what you need to most effectively do your job, but it presents it in a fashion that justifies the monetary/man-power expense to the business to get it done.

Why are you here?

You are not here to push tickets. You aren’t here to mindlessly click buttons. That kind of work (generally) isn’t fulfilling on a fundamental human level and will inevitably be automated away. You are here to analyze data and make decisions that keep the company safe so it can do what it’s been created to do (yes, the phrase “analyze data” can have a plethora of meanings which can be explored at a later date).

Now that we’ve explored what analysis actually is, and how that roughly fits into our world of InfoSec, next we can start to explore what it means to get better at it.

Outtros are overrated

I will say this now and always, but I am happy for anyone to hit me up with feedback. Negative, positive, ad hominems, whatever you wanna send my direction.

Take care of yourself fam.

There are a plethora of specialities within InfoSec, many of which have a number of learning resources dedicated to them. However, one aspect that I believe is fundamental to many of them, yet rarely receives any amount of attention, is analysis. There are entire career paths within InfoSec that, by their very name, imply that analysis is the fundamental activity (SOC analyst, threat intel analyst, etc). However, there is almost no shared understanding of what analysis actually is. If someone is an ‘analyst’, but cannot define what ‘analysis’ actually is, how can they expect to get better and develop their skills in this undefined activity? Analysis is an actual skill that can be learned, practiced, and honed, however, that requires us to define what that looks like.

Caveats

I am not even close to the first person (nor the last) to ponder and write on the essence of analysis. There have been numerous before me to explore and document their findings on this process. One of the most notable that comes to mind is The Psychology of Intelligence Analysis. While that explores analysis through the lens of government intelligence analysts whose drive it is to inform policy makers, it covers many cognitive biases that impacts how we do analysis in other fields.

This post is not a comprehensive view of analysis and how it can apply to our functions in the InfoSec world, but my goal is to provide an initial introduction to the subject. How can people beginning in the field better identify a path to grow their skills? How can someone established in the field better evaluate the accuracy of their own work? These are a couple of the topics this introduction seeks to touch on.

Definition of Analysis

In order to have a meaningful discussion of how to get better at analysis, we need to define what the act of analysis actually is. I currently define it as:

“The observation of a data point, and making an inference from that observation.”

While this may sound like a simple definition, I believe there are multiple aspects of it to unpack in how that applies to the act of ‘analysis’ in the InfoSec world. In my mind, a few of those aspects are:

• Our level of expertise in the data we are observing
• Our ability to synthesize inferences from multiple data points together
• Our ability to understand logical chains that lead to our data point
• Our imagination to account for unintended results (or “be prepared to be wrong”)

Data Expertise

Our ability to ‘analyze’ data can be directly correlated to our level of understanding of that data. For example, let’s say you walked up to my desk and handed me a magazine written in Hindi and asked me to tell you what it says. Depending on my level of understanding of Hindi, I may have any of the following results for you:

1. If I don’t know Hindi at all, I may say, “Sorry, I got nothing for you”.
2. If I have the most basic understanding of the language, I may know enough to look up each word in the dictionary and give you a very rough translation in broken English.
3. If I have a well versed understanding of the language, I may be able to give you a translation in coherent English.
4. If I have an expert level understanding of the language, I may be able to not only give you a proper English translation, but also provide insights such as, even though they said “A”, that is actually a Hindi saying that actually means “B”.

Given this example, it becomes more obvious how our ability to infer data grows as our expertise/understanding of the data in question grows.

Synthesizing Multiple Data Points

Our ability to build a complete picture of reality rarely hinges on a single data point. However, when analyzing technical data, sometimes we fall into the trap of drawing conclusions off a single data point (a single network connection, a single process execution, etc.) without looking at what else was going on at the time of the event that can change the context. Imagine the below scenario:

• There is a man knocking at your front door, and he is holding a knife in his hand.

How would you interpret this situation? Different people will evaluate this scenario differently; some may feel innately threatened, while others may not. However, what if we add the following data points:

• There is a man knocking at your front door, and he is holding a knife in his hand.
• This man is your neighbor that you have known for years and have a positive relationship with.
• You let him borrow a kitchen knife last week, which he said he would return. That is the knife he is holding.

Adding these data points can completely change the context of the scenario and how we interpret what is going on. This is where it is important to get a composite view of multiple data points to provide clearer context for your initial data point of interest.

Understanding Logical Chains of Data Points

This one is slightly tied to the previous two. Sometimes you can observe a data point that allows you to infer other data points, even though you may not be able to observe them. This is recognizing that your data point of interest didn’t happen in a vaccuum and requires certain circumstances to get to the point that it got to (even though you may not have access to other data points to back that up). Below are a couple examples for this:

You observe a computer on your network making a network connection to ‘downloadingmalware.com’ (or whatever example malicious domain you want to use here)

There are multiple other data points you can infer from this:

1. Computers don’t make network connections because they just feel like it. There must be a process on that computer that initiated that network connection.
2. That process was not started in a vaccuum, it was started by something…potentially this was started automatically by other software, or perhaps it was started manually by a human.
3. There is a fair chance that there is a binary file (like a MZ executable on a Windows system) on the computer associated with the process that made this network connection.
4. The computer had to be able to resolve the domain ‘downloadingmalware.com’. Are you logging DNS, and did it make a DNS lookup for that domain? If there is evidence that a DNS lookup did not happen, that could imply the domain could have been added to the computers hosts file. Is there evidence of that? If so, there may be another process that made that change.
5. (assuredly more here)

Looking at that, you can take a single data point and identify a number of other data points that would be related that would have had to happen to get to the data point that you initially observed. You can then start to build a trail of other data points to begin investigating to create a more complete picture of what happened.

Below is another example of this from a slightly different perspective. Caveat: I MADE UP THE BELOW DATA FOR THE SAKE OF ILLUSTRATION, IT IS NOT REAL. Let’s say all you have to evaluate is a single line of netflow data:

Date flow start Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes

2010-09-01 17:29:00.459 TCP 159.93.12.79:22 -> 103.148.117.12:22126 1 61488

There are multiple ways people could interpret that data. Some reactions could be, “that is just a single packet and not a whole session, so it doesn’t tell us much”, or “That is SSH because of port 22, so it is encrypted so we don’t have any idea what’s going on” . But you could also infer some of the following data:

1. That is TCP port 22, which implies it is SSH, but it is going in the server -> client direction. Given this, we can immediately imply that SSH (or a process listening on port 22) is actually running on the server.
2. Given what we discussed above, the server SSH process will not randomly reach out to the client, so we are probably seeing a reflection of the server responding to traffic initiated by the client.
3. Looking at the size of the packet, we can imply that the client has successfully SSH’d to the server.
   • Thinking about how SSH works, a client will attempt to connect either via pki or intending to use a username and password.
   • After the 3-way TCP handshake, the server will traditionally display it’s banner and negotiate the authentication, requesting a username and password.
   • If the client fails 3 password attempts, the connection will be killed and they’ll have to start over.
   • Thinking about that, in the circumstance where the client does not know how to correctly authenticate to the server, the amount of data transferred from the server back to the client is very small (the only real variability in it is the size of the banner).
   • Given that the size of the packet is 61,488 bytes, that can reasonably be assumed to be larger than any banner that a SSH server would return. This implies that the client has completed the authentication to the server, and ran a command that elicited the server to return that many bytes of data.
4. The server IP address belongs to AS2875, which is the Joint Institute for Nuclear Research in Russia, and the client IP address belongs to AS207529 (Adnan Rafique) which is in Pakistan.

Given an understanding of the protocol at play in the above data point, you can go from a perception of, “this is just a packet”, to “an entity in Pakistan appears to have successful SSH access to a server in Russia’s Joint Institute for Nuclear Research”, which is a far more comprehensive view of the situation, all based off of the implications of what would be required for a single data point to exist.

Imagination to Account for the Unexplained

Sometimes technology doesn’t work the way we think it should. For example, one time I was looking at building detections based on Windows logon types and RDP. I figured there were some kinds of accounts we’d expect to see for interactive logons (hands on keyboard, type 2), some network logons (type 3), and some remoteinteractive (rdp, type 10). To my surprise, when I was doing testing, I realized that the way RDP was implemented at the time, a single RDP session created a type 2, type 3, and type 10 logon event. This took me by surprise and I had to completely alter the way that I intended to start building detections and alerts.

This is where our internal biases and the very expertise that I espouse we develop in ourselves in the first point, can get in our own way. The very expertise we develop within ourselves may give us a false sense of confidence in our initial interpretation of data. I have had a number of conversations with analysts where they observed a data point which did not fully conform to their expectation of how a protocol/process should work. Sometimes they got so far deep in the rabbit hole trying to defend their expectations of how things should work, that they lose sight of what the data points are actually telling them.

One great way that I have found to check a conclusion that I come to is to ask myself the question, “What data, if presented, would render my hypothesis incorrect?” For example, let’s look at the SSH netflow example above; what if we determine that the SSH service isn’t running on 159.93.12.79:22, but it is actually HTTP? That would negate every other follow-on conclusion we made based off of that data point and we would have to develop new conclusions. Then, what if we said 159.93.12.79 wasn’t acting as a server at all, but 103.148.117.12 was hosting a service on TCP port 22126? That would again change our entire interpretation of the data point and force us to come to new conclusions. But if we don’t actively look for data points that can disprove our theories, it can be far more difficult to identify instances where our analysis is incorrect.

While this is just a single example of identifying biases in our thinking and analysis, there are many more that may arise that are covered in other writings, such as the Psychology of Intelligence Analysis noted in the beginning. Some of the most successful analysts that I have observed have been able to take data points that did not conform to their preconceived notions, identify where it diverged from their expectations, research the other data points that contextualized why the divergence happened, and then reincorporated this new information into their experience/expectations going forward.

TL;DR at the End

It is odd that analysis has been largely undiscussed and undefined from an industry perspective, despite being such an integral skill in many positions. If we don’t actually understand what it is, how can we get better at it? Analysis could be defined as “the observation of a data point, and making an inference from that observation.” One’s strength and ability to do that can be deepened by some of the following skills:

• Our level of expertise in the data we are observing - If you are analyzing network data, learn networking as well as you can. If you are analyzing processes, learn operating system internals the best you can. etc.
• Our ability to synthesize inferences from multiple data points together - How often do you look for alternate data points that are related to what you are looking at to build a more complete picture of what is going on?
• Our ability to understand logical chains that lead to our data point - Data points almost never happen in complete isolation. What other factors must exist (whether you have the data points to support them or not) for your initial data point to exist?
• Our imagination to account for unintended results - Sometimes technology (and the world) does not work as we think it should. When you come across data that breaks your expectations, how able are you to incorporate other data points to compensate for that and adjust your expectations? Are you flexible enough to adjust your expectations of reality at all?
• Sometimes we are wrong and our expertise can trick us into consistently assuming we are right. What are you doing to identify when this happens?

Hopefully this is useful for anyone. Feel free to hit me up with any feedback (positive or negative), questions, or comments. Hopefully more to come soon. Peace out fam!