Use the search engines to perform reconnaissance on websites and web applications. Identify what sensitive design and configuration information of the application, system, or organization is exposed directly (on the organization’s website) or indirectly (via third-party services).
WSTG-INFO-01: Conduct Search Engine Discovery Reconnaissance for Information Leakage (OWASP Testing Guide)
Objective
Identify what sensitive design and configuration information of the application, system, or organization is exposed directly (on the organization’s website) or indirectly (via third-party services).
Look for: network diagrams and configurations, archived posts and emails by administrators or other key staff, logon procedures and username formats, usernames, passwords, and private keys, third-party, or cloud service configuration files, revealing error message content; and non-public applications (development, test, User Acceptance Testing (UAT), and staging versions of sites).
Domain
whois $DOMAINSubdomains
Copy this table in your favorite note program to keep track of information about subdomains.
| Subdomain | Status | Hosted by | Source | Comments |
|---|---|---|---|---|
| subdomain.example.com | UP/down | Amazon, hosting provider | Security Trails, website | nginx version displayed on page, any other observations |
crt.sh – subdomains from SSL certificates
DOMAIN=example.com
curl --no-progress-meter https://crt.sh/?q=${DOMAIN} | grep -o "<TD>.*${DOMAIN}</TD>" | sed 's/<BR>/\n/g' | sed -E 's/(<TD>)|(<\/TD>)|(\*\.)//g' | sort -uSecurityTrails
SecurityTrails – DNS records, list of subdomains. Create an account and generate an API key.
APIKEY=<API KEY>DOMAIN=example.comcurl https://api.securitytrails.com/v1/domain/$DOMAIN/subdomains?children_only=true\&include_inactive=true --header "APIKEY: $APIKEY" --header "accept: application/json" > subdomains-st.txtfor i in $(cat subdomains-st.txt | grep " " | cut -d '"' -f2); do echo $i.$DOMAIN; done | sortPeople & Email addresses
Copy this table in your favorite note program to keep track of information about people.
| Email address | Name | Source | Comments |
|---|---|---|---|
| john.smith@example.com | John Smith | Hunter.io, LinkedIn | Developer, left the company |
- Find email addresses:
- Hunter.io, then guess other people’s email addresses based on this format
- GHDB: search for “@domain.com”
- Find people:
- Linkedin – also look for technologies and information in profile
- Social Searcher
- Other resources: Canada 411, Yellow Pages, Flight Radar 24 (flights in real time)
For every email address previously found, look for leaked information (username, password, address, etc.) on Have I Been Pwned.
Leaked Information
Leaked private keys
git clone https://github.com/sec-consult/houseofkeys.git
cd houseofkeysList files containing vendor name
grep -Ril "vendor name" ./List vendor products affected
for file in $(grep -Ril "vendor name" ~/houseofkeys/); do grep "name" $file; done;Search Engines
- Baidu (China)
- Bing (Microsoft)
- binsearch.info (binary Usernet newsgroups)
- DuckDuckGo (results from different sources)
- Shodan (IoT)
Search in Archived Versions
Search the web application or organization’s website URL in Wayback Machine.
Search through the Google cache
cache: URL [string]Remediation
- Carefully consider the sensitivity of design and configuration information before it is posted online.
- Periodically review the sensitivity of existing design and configuration information that is posted online.