|
Table of Contents |
5 |
|
|
About the Authors |
13 |
|
|
About the Technical Reviewer |
14 |
|
|
Acknowledgments |
15 |
|
|
Introduction |
16 |
|
|
Chapter 1: The Evolution of Open Source Intelligence |
21 |
|
|
Open Source Information Categories |
23 |
|
|
OSINT Types |
25 |
|
|
Digital Data Volume |
25 |
|
|
OSINT Organizations |
26 |
|
|
Government Organizations |
27 |
|
|
Open Source Center |
27 |
|
|
BBC Monitoring |
27 |
|
|
Private Sector |
27 |
|
|
Jane’s Information Group |
28 |
|
|
Economist Intelligence Unit |
28 |
|
|
Oxford Analytica |
28 |
|
|
Gray Literature Vendors |
28 |
|
|
Factiva |
29 |
|
|
LexisNexis |
30 |
|
|
Parties Interested in OSINT Information |
30 |
|
|
Government |
30 |
|
|
International Organizations |
31 |
|
|
Law Enforcement Agencies |
31 |
|
|
Business Corporations |
32 |
|
|
Penetration Testers and Black Hat Hackers/Criminal Organizations |
32 |
|
|
Privacy-Conscious People |
33 |
|
|
Terrorist Organizations |
33 |
|
|
Information Gathering Types |
34 |
|
|
Passive Collection |
34 |
|
|
Semipassive |
34 |
|
|
Active Collection |
35 |
|
|
Benefits of OSINT |
35 |
|
|
Challenges of Open Source Intelligence |
36 |
|
|
Legal and Ethical Constraints |
37 |
|
|
Summary |
38 |
|
|
Notes |
39 |
|
|
Chapter 2: Introduction To Online Threats and Countermeasures |
41 |
|
|
Online Threats |
42 |
|
|
Malware |
42 |
|
|
Black Hat Hackers |
43 |
|
|
Pharming |
43 |
|
|
Phishing |
44 |
|
|
Ransomware |
47 |
|
|
Adware and Spyware |
48 |
|
|
Trojan |
49 |
|
|
Virus |
49 |
|
|
Worms |
49 |
|
|
Scareware |
49 |
|
|
Distributed Denial of Service |
50 |
|
|
Rootkits |
50 |
|
|
Juice Jacking |
50 |
|
|
Wi-Fi Eavesdropping |
50 |
|
|
Security Software |
51 |
|
|
Antivirus |
51 |
|
|
Firewall |
52 |
|
|
Anti-malware |
53 |
|
|
Securing the Operating System |
53 |
|
|
Hardening the Windows OS |
54 |
|
|
Updating Windows |
54 |
|
|
Updating All Installed Programs |
55 |
|
|
Locking Your PC Using a USB Drive |
55 |
|
|
Using a Less-Privileged User Account |
55 |
|
|
Using a Strong Password for Windows |
55 |
|
|
Keeping Your User Account Control Turned On |
56 |
|
|
Disabling Remote Assistance |
56 |
|
|
Making Hidden Files Visible |
57 |
|
|
Freezing the Hard Disk |
57 |
|
|
Setting a Password for BIOS/UEFI |
58 |
|
|
Disabling Unnecessary Ports/Protocols and Services |
58 |
|
|
Staying Private in Windows 10 |
59 |
|
|
Destroying Digital Traces |
61 |
|
|
General Privacy Settings |
65 |
|
|
Covering Your Laptop Camera |
65 |
|
|
Avoiding Pirated Software |
65 |
|
|
Handling Digital Files Metadata |
66 |
|
|
Physically Securing Computing Devices |
70 |
|
|
Online Tracking Techniques |
72 |
|
|
Tracking Through IP Address |
72 |
|
|
What Is an IP Address? |
72 |
|
|
How Is an IP Address Used to Track You Online? |
74 |
|
|
Cookies |
75 |
|
|
Digital Fingerprinting |
77 |
|
|
Script-Based Fingerprinting |
77 |
|
|
Canvas Fingerprinting |
77 |
|
|
HTML5 |
78 |
|
|
Checking Your Digital Footprint |
78 |
|
|
Browserleaks |
78 |
|
|
Panopticlick |
78 |
|
|
Secure Online Browsing |
79 |
|
|
Configuring Firefox to Become More Private |
79 |
|
|
Turning On Private Browsing |
79 |
|
|
Changing the Firefox Settings to Become More Private |
80 |
|
|
Firefox Privacy Extensions |
83 |
|
|
Fighting Against Digital Fingerprinting and Browser Leak |
84 |
|
|
Secure Online Communication |
84 |
|
|
VPN |
85 |
|
|
Proxies |
86 |
|
|
DNS Leak Test |
87 |
|
|
Online Anonymity |
89 |
|
|
Using the TOR Network |
89 |
|
|
Tor Browser |
90 |
|
|
Hiding Tor Usage |
91 |
|
|
Using a VPN |
92 |
|
|
Using Tor Bridges |
92 |
|
|
Using Pluggable Transports |
94 |
|
|
Using the Tails OS and Other Security OSs |
96 |
|
|
Sharing Files Securely |
97 |
|
|
OnionShare |
97 |
|
|
Making Anonymous Payments |
99 |
|
|
Prepaid Gift Card |
99 |
|
|
Cryptocurrency |
100 |
|
|
Encryption Techniques |
101 |
|
|
Securing Your Passwords |
101 |
|
|
Encrypting Your Hard Drive/USB Sticks |
102 |
|
|
Cloud Storage Security |
102 |
|
|
Secure E-mail Communications |
103 |
|
|
Secure E-mail Providers |
104 |
|
|
Secure IM and Online Calling Services |
105 |
|
|
Virtualization Technology |
106 |
|
|
Android and iOS Emulator |
108 |
|
|
Essential Prerequisites |
108 |
|
|
Drawing Software and Data Visualization |
109 |
|
|
Mind Mapping and Idea Generation Tools |
109 |
|
|
FreeMind |
109 |
|
|
Storytelling Tools |
109 |
|
|
Diagramming Software |
109 |
|
|
Apache OpenOffice Draw |
109 |
|
|
Google Drawings |
110 |
|
|
Note Management |
110 |
|
|
TagSpaces |
110 |
|
|
KeepNote |
110 |
|
|
Data Visualization |
110 |
|
|
Microsoft Excel |
110 |
|
|
Business Intelligence and Reporting Tools |
110 |
|
|
Dradis CE |
110 |
|
|
Bookmarking |
111 |
|
|
Free Translation Services |
112 |
|
|
Final Tips |
112 |
|
|
Use a False Identity to Register on Some Websites |
112 |
|
|
Be Anonymous |
113 |
|
|
Destroy Your Digital Traces Upon Finishing |
113 |
|
|
Use Linux |
113 |
|
|
Summary |
114 |
|
|
Chapter 3: The Underground Internet |
115 |
|
|
Layers of the Internet |
116 |
|
|
Darknet Users |
123 |
|
|
Accessing the Darknet |
124 |
|
|
Security Checks When Accessing the Darknet |
124 |
|
|
Accessing the Darknet from Within the Surface Web |
126 |
|
|
Using Tor |
127 |
|
|
Using the Tails OS |
129 |
|
|
Warning When Using the Tails OS |
134 |
|
|
Searching the Tor Network |
135 |
|
|
Other Anonymity Networks |
136 |
|
|
I2P |
137 |
|
|
Using I2P |
137 |
|
|
I2P vs. Tor |
142 |
|
|
Freenet |
143 |
|
|
Going Forward |
143 |
|
|
Summary |
144 |
|
|
Notes |
145 |
|
|
Chapter 4: Search Engine Techniques |
146 |
|
|
Keywords Discovery and Research |
148 |
|
|
Using Search Engines to Locate Information |
149 |
|
|
Google |
149 |
|
|
Google Advanced Operators |
152 |
|
|
Google Hacking Database |
155 |
|
|
Search Engines Powered by Google |
157 |
|
|
Bing |
157 |
|
|
Privacy-Oriented Search Engines |
159 |
|
|
Other Search Engines |
160 |
|
|
Business Search Sites |
161 |
|
|
Find Business Annual Records |
162 |
|
|
Business Information (Profiles) |
164 |
|
|
Metadata Search Engines |
166 |
|
|
Code Search |
169 |
|
|
FTP Search Engines |
170 |
|
|
Automated Search Tools |
171 |
|
|
SearchDiggity |
171 |
|
|
SearchDome |
171 |
|
|
Jeviz |
172 |
|
|
Internet Of Things (IoT) Device Search Engines |
172 |
|
|
Web Directories |
173 |
|
|
Translation Services |
175 |
|
|
Website History and Website Capture |
177 |
|
|
Website Monitoring Services |
179 |
|
|
RSS Feed |
181 |
|
|
News Search |
182 |
|
|
Customize Google News |
183 |
|
|
News Websites |
185 |
|
|
Fake News Detection |
185 |
|
|
Searching for Digital Files |
189 |
|
|
Document Search |
189 |
|
|
DOC and DOCX |
189 |
|
|
HTML and HTM |
189 |
|
|
ODT |
189 |
|
|
XLS and XLSX |
190 |
|
|
ODS |
190 |
|
|
PPT and PPTX |
190 |
|
|
ODP |
190 |
|
|
TXT |
190 |
|
|
PDF |
191 |
|
|
File Search Engines |
191 |
|
|
Fagan Finder |
191 |
|
|
General-Search |
192 |
|
|
ShareDir |
192 |
|
|
Custom Search Engine |
193 |
|
|
Gray Literature |
198 |
|
|
Data Leak Information |
201 |
|
|
Document Metadata |
202 |
|
|
Image |
202 |
|
|
Basic Image Search |
202 |
|
|
Reverse Image Search |
206 |
|
|
Image Manipulation Check |
207 |
|
|
OCR Tools |
208 |
|
|
Video |
210 |
|
|
Basic Video Search |
211 |
|
|
Video Analysis |
213 |
|
|
File Extension and File Signature List |
215 |
|
|
Productivity Tools |
215 |
|
|
Screen Capture |
215 |
|
|
Download Online Video |
216 |
|
|
Easy YouTube Video Downloader Express |
216 |
|
|
YooDownload |
216 |
|
|
Dredown |
217 |
|
|
Video/Audio Converter |
217 |
|
|
File Search Tools |
218 |
|
|
Summary |
220 |
|
|
Notes |
220 |
|
|
Chapter 5: Social Media Intelligence |
221 |
|
|
What Is Social Media Intelligence? |
223 |
|
|
Social Media Content Types |
224 |
|
|
Classifications of Social Media Platforms |
226 |
|
|
Popular Social Networking Sites |
228 |
|
|
Investigating Social Media Sites |
229 |
|
|
Facebook |
229 |
|
|
Facebook Graph Search |
231 |
|
|
Other Useful Facebook Graph Search Commands |
240 |
|
|
Tracking Photos Downloaded from Facebook to Its Source Profile |
240 |
|
|
Using Google to Search Facebook Content |
242 |
|
|
Search for Hashtags on Facebook |
242 |
|
|
Using Automated Services to Facilitate Facebook Graph Search |
242 |
|
|
Facebook Scanner |
243 |
|
|
Graph |
243 |
|
|
peoplefindThor |
244 |
|
|
Socmint |
244 |
|
|
Online Facebook Search Tools/Services |
245 |
|
|
Collecting Local Copy of Target Facebook Data |
246 |
|
|
Twitter |
249 |
|
|
Twitter Search |
250 |
|
|
Twitter Advanced Search Operators |
251 |
|
|
Twitter Advanced Search Page |
255 |
|
|
Online Twitter Search Tools/Services |
256 |
|
|
Google+ |
259 |
|
|
Searching Google+ |
261 |
|
|
Google+ Advanced Search Operator |
261 |
|
|
Using Google to Search Within Google+ |
264 |
|
|
Searching Google+ Using a Google Custom Search Engine |
265 |
|
|
Other Useful Services for Google+ |
265 |
|
|
LinkedIn |
265 |
|
|
LinkedIn Search |
267 |
|
|
Advanced LinkedIn Search Operators |
270 |
|
|
Searching LinkedIn Using a Google Custom Search |
270 |
|
|
General Resources for Locating Information on Social Media Sites |
271 |
|
|
Other Social Media Platforms |
272 |
|
|
Pastebin Sites |
273 |
|
|
Social Media Psychological Analysis |
274 |
|
|
Tone Analyzer |
275 |
|
|
Watson Tone Analyzer |
275 |
|
|
Facebook and Twitter Prediction |
276 |
|
|
Fake Sport |
276 |
|
|
Review Meta |
276 |
|
|
TweetGenie |
276 |
|
|
Summary |
276 |
|
|
Notes |
277 |
|
|
Chapter 6: People Search Engines and Public Records |
279 |
|
|
What Is a People Search Engine? |
279 |
|
|
What Are Public Records? |
280 |
|
|
Example of Public Records |
281 |
|
|
Searching for Personal Details |
282 |
|
|
General People Search |
282 |
|
|
TruthFinder |
282 |
|
|
411 |
283 |
|
|
Pipl |
283 |
|
|
Other |
283 |
|
|
Online Registries |
286 |
|
|
Vital Records |
287 |
|
|
Criminal and Court Search |
290 |
|
|
Property Records |
291 |
|
|
Tax and Financial Records |
292 |
|
|
Social Security Number Search |
293 |
|
|
Username Check |
293 |
|
|
E-mail Search and Investigation |
293 |
|
|
Data Compromised Repository Websites |
295 |
|
|
Phone Number Search |
297 |
|
|
Employee Profiles and Job Websites |
298 |
|
|
Dating Website Search |
299 |
|
|
Other Public Records |
301 |
|
|
Summary |
302 |
|
|
Notes |
302 |
|
|
Chapter 7: Online Maps |
303 |
|
|
The Basics of Geolocation Tracking |
303 |
|
|
How to Find the GPS Coordinates of Any Location on a Map |
304 |
|
|
How to Find the Geocode Coordinates from a Mailing Address |
306 |
|
|
General Geospatial Research Tools |
306 |
|
|
Commercial Satellites |
312 |
|
|
Date/Time Around the World |
312 |
|
|
Location-Based Social Media |
313 |
|
|
YouTube |
313 |
|
|
Facebook |
314 |
|
|
Using Facebook Graph in the Location Search |
314 |
|
|
Facebook Live |
315 |
|
|
Twitter |
316 |
|
|
Search for Tweets in a Specific Geographical Location |
316 |
|
|
Tweet Mapper |
318 |
|
|
One Million Tweet Map |
319 |
|
|
Qtr Tweets |
319 |
|
|
Tweet Map |
319 |
|
|
Periscope Map |
319 |
|
|
Other Social Media Platforms |
320 |
|
|
Strava Heat Map |
320 |
|
|
Conducting Location Searches on Social Media Using Automated Tools |
321 |
|
|
Country Profile Information |
322 |
|
|
Transport Tracking |
322 |
|
|
Air Movements |
323 |
|
|
Maritime Movements |
325 |
|
|
Vehicles and Railway |
327 |
|
|
Package Tracking |
328 |
|
|
Webcams |
329 |
|
|
Digital File Metadata |
330 |
|
|
Summary |
330 |
|
|
Chapter 8: Technical Footprinting |
331 |
|
|
Investigate the Target Website |
332 |
|
|
Investigate the Robots.txt File |
334 |
|
|
Mirror the Target Website |
335 |
|
|
Extract the Links |
335 |
|
|
Check the Target Website’s Backlinks |
336 |
|
|
Monitor Website Updates |
336 |
|
|
Check the Website’s Archived Contents |
336 |
|
|
Identify the Technologies Used |
337 |
|
|
Web Scraping Tools |
340 |
|
|
theHarvester |
340 |
|
|
Web Data Extractor |
342 |
|
|
Email Extractor |
342 |
|
|
Investigate the Target Website’s File Metadata |
342 |
|
|
Website Certification Search |
343 |
|
|
Website Statistics and Analytics Tools |
343 |
|
|
Website Reputation Checker Tools |
344 |
|
|
Passive Technical Reconnaissance Activities |
345 |
|
|
WHOIS Lookup |
345 |
|
|
Subdomain Discovery |
347 |
|
|
Using Google Search Operator |
348 |
|
|
Using VirusTotal.com |
348 |
|
|
DNSdumpster |
349 |
|
|
DNS Reconnaissance |
350 |
|
|
Route Mapping |
350 |
|
|
Common DNS Record Types |
351 |
|
|
nslookup Command |
352 |
|
|
Netcraft |
354 |
|
|
IP Address Tracking |
355 |
|
|
Summary |
357 |
|
|
Chapter 9: What’s Next? |
358 |
|
|
Where Will OSINT Go Next? |
358 |
|
|
OSINT Process |
360 |
|
|
Final Words |
361 |
|
|
Index |
362 |
|