Reconnaissance / Passive Information Gathering / OSINT

Use the search engines to perform reconnaissance on websites and web applications. Identify what sensitive design and configuration information of the application, system, or organization is exposed directly (on the organization’s website) or indirectly (via third-party services).

WSTG-INFO-01: Conduct Search Engine Discovery Reconnaissance for Information Leakage (OWASP Testing Guide)

Objective

Identify what sensitive design and configuration information of the application, system, or organization is exposed directly (on the organization’s website) or indirectly (via third-party services).

Look for: network diagrams and configurations, archived posts and emails by administrators or other key staff, logon procedures and username formats, usernames, passwords, and private keys, third-party, or cloud service configuration files, revealing error message content; and non-public applications (development, test, User Acceptance Testing (UAT), and staging versions of sites).

Domain

whois $DOMAIN

Subdomains

💡 Copy this table in your favorite note program to keep track of information about subdomains.

Subdomain	Status	Hosted by	Source	Comments
subdomain.example.com	UP/down	Amazon, hosting provider	Security Trails, website	nginx version displayed on page, any other observations

• sublist3r
• Netcraft Search DNS
• GHDB: search for site:”domain.com” -site:”www.domain.com”

crt.sh – subdomains from SSL certificates

DOMAIN=example.com
curl --no-progress-meter https://crt.sh/?q=${DOMAIN} | grep -o "<TD>.*${DOMAIN}</TD>" | sed  's/<BR>/\n/g' | sed -E 's/(<TD>)|(<\/TD>)|(\*\.)//g' | sort -u

SecurityTrails

SecurityTrails – DNS records, list of subdomains. Create an account and generate an API key.

APIKEY=<API KEY>

DOMAIN=example.com

curl https://api.securitytrails.com/v1/domain/$DOMAIN/subdomains?children_only=true\&include_inactive=true --header "APIKEY: $APIKEY" --header "accept: application/json" > subdomains-st.txt

for i in $(cat subdomains-st.txt | grep "    " | cut -d '"' -f2); do echo $i.$DOMAIN; done | sort

People & Email addresses

💡 Copy this table in your favorite note program to keep track of information about people.

Email address	Name	Source	Comments
john.smith@example.com	John Smith	Hunter.io, LinkedIn	Developer, left the company

• Find email addresses:
  • Hunter.io, then guess other people’s email addresses based on this format
  • GHDB: search for “@domain.com”
• Find people:
  • Linkedin – also look for technologies and information in profile
  • Facebook
  • Social Searcher
  • Other resources: Canada 411, Yellow Pages, Flight Radar 24 (flights in real time)

💡 For every email address previously found, look for leaked information (username, password, address, etc.) on Have I Been Pwned.

Leaked Information

• Search GitHub Repositories
• Google Hacking Database (GHDB)

Leaked private keys

• https://github.com/sec-consult/houseofkeys
• What can an attacker do?

git clone https://github.com/sec-consult/houseofkeys.git
cd houseofkeys

List files containing vendor name

grep -Ril "vendor name" ./

List vendor products affected

for file in $(grep -Ril "vendor name" ~/houseofkeys/); do grep "name" $file; done;

Search Engines

• Baidu (China)
• Bing (Microsoft)
• binsearch.info (binary Usernet newsgroups)
• DuckDuckGo (results from different sources)
• Shodan (IoT)

Search in Archived Versions

Search the web application or organization’s website URL in Wayback Machine.

Search through the Google cache

cache: URL [string]

Remediation

• Carefully consider the sensitivity of design and configuration information before it is posted online.
• Periodically review the sensitivity of existing design and configuration information that is posted online.