Red Team Guide: External Recon

Published Apr 4, 2022

Contents


Introduction

In this phase we are going to:

  • Find all the companies inside the scope.
  • Find all the assets belonging to the companies.
  • Find all the domains belonging to the companies.
  • Find all the subdomains of the domains.
  • Find all the web servers and take a screenshot of them.
  • Find all the leaked secrets from the company's github repositories.

Active Scanning

Adversaries may execute active reconnaissance scans to gather information that can be used during targeting. Active scans are those where the adversary probes victim infrastructure via network traffic, as opposed to other forms of reconnaissance that do not involve direct interaction.

Scanning IP Blocks

Adversaries may scan victim IP blocks to gather information that can be used during targeting. Public IP addresses may be allocated to organizations by block, or a range of sequential addresses.

  • nmap: Nmap is used to discover hosts and services on a computer network by sending packets and analyzing the responses.
  • massscan: TCP port scanner, spews SYN packets asynchronously, scanning entire Internet in under 5 minutes.
  • mapcidr: A utility program to perform multiple operations for a given subnet/cidr ranges.
  • naabu: A fast port scanner written in go with a focus on reliability and simplicity.
  • Smap: A drop-in replacement for Nmap powered by shodan.io.

Vulnerability Scanning

Adversaries may scan victims for vulnerabilities that can be used during targeting. Vulnerability scans typically check if the configuration of a target host/application (ex: software and version) potentially aligns with the target of a specific exploit the adversary may seek to use.

  • nuclei: Fast and customizable vulnerability scanner based on simple YAML based DSL.
  • reNgine: reNgine is a web application reconnaissance suite with focus on highly configurable streamlined recon process via Engines, recon data correlation, continuous monitoring, recon data backed by database and simple yet intuitive User Interface.
  • Osmedeus: A Workflow Engine for Offensive Security.
  • Sn1per Professional: Discover the attack surface and prioritize risks with our continuous Attack Surface Management (ASM) platform.

Gather Victim Identity Information

Adversaries may gather information about the victim's identity that can be used during targeting. Information about identities may include a variety of details, including personal data (ex: employee names, email addresses, etc.) as well as sensitive details such as credentials.

Credentials

Adversaries may gather credentials that can be used during targeting. Account credentials gathered by adversaries may be those directly associated with the target victim organization or attempt to take advantage of the tendency for users to use the same passwords across personal and business accounts.

  • DeHashed: DeHashed is described as the largest & fastest data breach search engine, its API Key can be used to integrate with other tools like dehashQuery to download breach results as shown below.
  • IntelligenceX: IntelligenceX is a search engine and data archive. Search Tor, I2P, data leaks and the public web by email, domain, IP, CIDR, Bitcoin address and more.
  • Have I Been Pwned?: Have I Been Pwned? is a website that allows Internet users to check whether their personal data has been compromised by data breaches.

Email Addresses

Adversaries may gather email addresses that can be used during targeting. Even if internal instances exist, organizations may have public-facing email infrastructure and addresses for employees.

  • Hunter.io: Hunter is the leading solution to find and verify professional email addresses.
  • EmailHarvester: A tool to retrieve Domain email addresses from Search Engines.
  • Infoga: Infoga is a tool that gathering email accounts informations (ip,hostname,country,…) from different public source (search engines, pgp key servers and shodan).
  • Skymen: Find email addresses of companies and people.

Employee Names

Adversaries may gather employee names that can be used during targeting. Employee names be used to derive email addresses as well as to help guide other reconnaissance efforts and/or craft more-believable lures.

  • linkedin-employee-scraper: Extract all employees from LinkedIn. Especially useful for companies with thousands of pages and employees. Script is run as a userscript, running in e.g. Chromes Tampermonkey or Firefox’s Greasemonkey.

Gather Victim Network Information

Adversaries may gather information about the victim's networks that can be used during targeting. Information about networks may include a variety of details, including administrative data (ex: IP ranges, domain names, etc.) as well as specifics regarding its topology and operations.

Domain Properties

Adversaries may gather information about the victim's network domain(s) that can be used during targeting. Information about domains and their properties may include a variety of details, including what domain(s) the victim owns as well as administrative data (ex: name, registrar, etc.) and more directly actionable information such as contacts (email addresses and phone numbers), business addresses, and name servers.

  • AADInternals: AADInternals can gather information about a tenant’s domains using public Microsoft APIs.
# Get login information for a domain
Get-AADIntLoginInformation -Domain company.com

DNS

Adversaries may gather information about the victim's DNS that can be used during targeting. DNS information may include a variety of details, including registered name servers as well as records that outline addressing for a target’s subdomains, mail servers, and other hosts.

  • dig: dig is a network administration command-line tool for querying the Domain Name System.
dig google.com
dig google.com -t mx +short # grab mail server information
  • host: the host command is a DNS lookup utility, finding the IP address of a domain name.
host google.com
  • dnsenum: dnsenum is a perl script that enumerates DNS information.
dnsenum --no-reverse google.com
  • dns-brute-script: Nmap will attempt to enumerate DNS hostnames by brute forcing popular subdomain names.
nmap -T4 -p 53 --script dns-brute google.com
  • dnsrecon: Check all NS Records for Zone Transfers. Enumerate General DNS Records for a given Domain (MX, SOA, NS, A, AAAA, SPF and TXT). Perform common SRV Record Enumeration. Top Level Domain (TLD) Expansion.
dnsrecon -d google.com
dnsrecon -r <DNS Range> -n <IP_DNS>   # DNS reverse of all of the addresses
dnsrecon -d facebook.com -r 157.240.221.35/24 # Using facebooks dns
dnsrecon -r 157.240.221.35/24 -n 1.1.1.1 # Using cloudflares dns
dnsrecon -r 157.240.221.35/24 -n 8.8.8.8 # Using google dns
  • dnsx: dnsx is a fast and multi-purpose DNS toolkit allow to run multiple DNS queries of your choice with a list of user-supplied resolvers.
dnsx -l dnsx.txt -resp -a -aaaa -cname -mx -ns -soa -txt
  • ffuf: Sometimes you will find pages that only return the header Access-Control-Allow-Origin when a valid domain/subdomain is set in the Origin header. In these scenarios, you can abuse this behavior to discover new subdomains.
ffuf -w subdomains-top1million-5000.txt -u http://10.10.10.208 -H 'Origin: http://FUZZ.crossfit.htb' -mr "Access-Control-Allow-Origin" -ignore-body
  • gobuster: If you suspect that some subdomain can be hidden in a web server you could try to brute force it.
gobuster vhost -u https://mysite.com -t 50 -w subdomains.txt

wfuzz -c -w /usr/share/wordlists/SecLists/Discovery/DNS/subdomains-top1million-20000.txt --hc 400,404,403 -H "Host: FUZZ.example.com" -u http://example.com -t 100

# From https://github.com/allyshka/vhostbrute
vhostbrute.py --url="example.com" --remoteip="10.1.1.15" --base="www.example.com" --vhosts="vhosts_full.list"

# https://github.com/codingo/VHostScan
VHostScan -t example.com

Network Trust Dependencies

Adversaries may gather information about the victim's network trust dependencies that can be used during targeting. Information about network trusts may include a variety of details, including second or third-party organizations/domains (ex: managed service providers, contractors, etc.) that have connected (and potentially elevated) network access.

  • crunchbase: search for the main company, and click on "acquisitions". There you will see other companies acquired by the main one.
  • wikipedia: Visit the page of the main company and search for acquisitions.

IP Addresses

Adversaries may gather the victim's IP addresses that can be used during targeting. Public IP addresses may be allocated to organizations by block, or a range of sequential addresses. Information about assigned IP addresses may include a variety of details, such as which IP addresses are in use. IP addresses may also enable an adversary to derive other details about a victim, such as organizational size, physical location(s), Internet service provider, and or where/how their publicly-facing infrastructure is hosted.

First we need to find the companies owned by the main company and then all the assets of these companies.

  • Find the acquisitions of the main company, this will give us the companies inside the scope.
  • Find the ASN (if any) of each company, this will give us the IP ranges owned by each company
  • Use reverse whois lookups to search for other entries (organisation names, domains...) related to the first one (this can be done recursively)
  • Use other techniques like shodan org and ssl filters to search for other assets (the ssl trick can be done recursively).

An autonomous system number (ASN) is a unique number assigned to an autonomous system (AS) by the Internet Assigned Numbers Authority (IANA).

An AS consists of blocks of IP addresses which have a distinctly defined policy for accessing external networks and are administered by a single organisation but may be made up of several operators.

It's interesting to find if the company have assigned any ASN to find its IP ranges. It will be interested to perform a vulnerability test against all the hosts inside the scope and look for domains inside these IPs.

  • Hurricane Electric BGP Toolkit: Hurricane Electric operates the largest Internet Protocol version 4 (IPv4) and Internet Protocol version 6 (IPv6) transit networks globally, as measured by the count of peering interconnections to other networks. You can search by company name, by IP, or by domain name. Depending on the region of the company this links could be useful to gather more data: AFRINIC (Africa), Arin (North America), APNIC (Asia), LACNIC (Latin America), RIPE NCC (Europe). Anyway, probably all the useful information (IP ranges and Whois) appears already in the first link.
  • NetblockTool: Find netblocks owned by a company.
  • SurfaceBrowser: Know the external Internet surface area of any company through a simple web-based interface.
  • ipinfo.io: Comprehensive IP address data, IP geolocation API.

Search Open Technical Databases

Adversaries may search freely available technical databases for information about victims that can be used during targeting. Information about victims may be available in online databases and repositories, such as registrations of domains/certificates as well as public collections of network data/artifacts gathered from traffic and/or scans.

WHOIS

Adversaries may search public WHOIS data for information about victims that can be used during targeting. WHOIS data is stored by regional Internet registries (RIR) responsible for allocating and assigning Internet resources such as domain names. Anyone can query WHOIS servers for information about a registered domain, such as assigned IP blocks, contact information, and DNS nameservers.

Digital Certificates

Adversaries may search public digital certificate data for information about victims that can be used during targeting. Digital certificates are issued by a certificate authority (CA) in order to cryptographically verify the origin of signed content. These certificates, such as those used for encrypted web traffic (HTTPS SSL/TLS communications), contain information about the registered organization such as name and location.

  • crt.sh: crt.sh is a web interface to a distributed database called the certificate transparency logs.

CDN

Adversaries may search content delivery network (CDN) data about victims that can be used during targeting. CDNs allow an organization to host content from a distributed, load balanced array of servers. CDNs may also allow organizations to customize content delivery based on the requestor’s geographical region.

  • findcdn - findCDN is a tool created to help accurately identify what CDN a domain is using.

Search Open Websites/Domains

Adversaries may search freely available websites and/or domains for information about victims that can be used during targeting. Information about victims may be available in various online sites, such as social media, new sites, or those hosting information about business operations such as hiring or requested/rewarded contracts.

  • subfinder - Subfinder is a subdomain discovery tool that discovers valid subdomains for websites.
  • assetfinder - Find domains and subdomains related to a given domain.
  • knockknock - A simple reverse whois lookup tool which returns a list of domains owned by people or companies.
  • findomain - The complete solution for domain recognition. Supports screenshoting, port scan, HTTP check, data import from other tools, subdomain monitoring, alerts via Discord, Slack and Telegram, multiple API Keys for sources and much more.
  • hakrevdns - Small, fast tool for performing reverse DNS lookups en masse.
  • Amass - In-depth Attack Surface Mapping and Asset Discovery.
amass intel -org 'Sony Corporation of America'  #fetch ASN & CIDR IP Range of a Company
amass intel -active -asn 3725 -ip   #enumerate subdomains & IP Address from ASN
amass intel -active -asn 3725    #enumerate subdomains only from ASN
amass intel -active -cidr 160.33.96.0/23   #enumerate subdomains from cidr range
amass intel -asn 3725 -whois -d sony.com   #enumerate subdomains using asn & whois
amass enum -d sony.com -active -cidr 160.33.99.0/24,160.33.96.0/23 -asn 3725   #enumerate subdomains using cidr & asn

Other Techniques

Web Servers Hunting

At this point we have found all the companies and their assets and we know IP ranges, domains and subdomains inside the scope. It's time to search for web servers.

A fast method to discover open ports related to web servers is using masscan. Another friendly tool to look for web servers is httpx. You just pass a list of domains and it will try to connect to port 80 (http) and 443 (https).

  • Screenshots: Now that you have discovered all the web servers present in the scope (among the IPs of the company and all the domains and subdomains) you probably don't know where to start. So, let's make it simple and start just taking screenshots of all of them. Just by taking a look at the main page you can find weird endpoints that are more prone to be vulnerable. To perform the proposed idea you can use EyeWitness, HttpScreenshot, and Aquatone.
  • Cloud Assets: Just with some specific keywords identifying the company it's possible to enumerate possible cloud assets belonging to them with tools like cloud_enum, CloudScraper, or cloudlist.

GitHub Leaked Secrets

We can search for some OSINT low-hanging fruits.

Automatic Tools

There are several tools out there that will perform part of the proposed actions against a given scope. You can get inspired by them and build your own tool.