Hiding In Plain Site
Misunderstood as a dark bastion for drug lords and hit men, the Deep Web comprises the majority of the Internet — a vast digital landscape of databases, social-media repositories, and personalized content where users can unearth government reports, scour academic research, and build an itinerary for their next trip abroad.
WORDS BY Antoinette Siu // ART BY Miriam Taylor
Runa Sandvik wanted to be a family lawyer growing up in Oslo, Norway, but plans changed when her mom bought a computer. Reading about hacking tools and seeing people break into systems on the big screen only drew her closer to a future in security. To a 15-year-old with a new 4GB HP Compaq, the endless gratification of connecting with a world outside of secondary school proved powerful. “It looks really cool in the movies, right?” she remembers. “I’m still as curious as I was back then, but I think I’m more careful now in how I go about doing things.” Sandvik, 27, specializes in privacy and security research. Twelve years and several computers later, she is developing privacy software and giving the keynote about protecting personal data at EUNIS 2014, an information technology conference held in Sweden. Sandvik employs specialty browsers to ensure security and to gain access to information. However, a much larger Web exists that encompasses legal and illegal activities that are open to anyone and where online safety and information retrieval can coexist.
Welcome to the Deep Web.
Everyone talks about it, yet this Wild West, black-market Internet making headlines and landing roles has, by and large, been misrepresented. This corner of the Internet is neither evil nor dangerous, and we’re confusing the Deep Web for its constituent, the dark Web. The surface Web, Deep Web, and dark Web, all make up the World Wide Web, which, as it turns out, isn’t too different from our physical world. The earth is a sphere with three layers, each one enveloping the core at the center; our online world also contains an outer surface layer and a few unseen layers underneath. Most of us are familiar with the visible surface Web, which includes anything accessible through standard search engines like Google or Yahoo. Surface Web content could be anything from a Yelp restaurant page to a Wikipedia entry on photosynthesis, but standard search engines skim less than 1 percent of all pages. The rest of it: invisible. “If people embrace it, it could be one of the new ways of searching the average Internet,” says Jeremy Martin, a computer-forensics instructor at InfoSec Institute, an Illinois-based company that trains IT professionals.
“Very little has been written or known of the Deep Web. Estimates of size and importance have been anecdotal at best and certainly underestimate scale,” Bergman says in his study.
In October 2013, when the FBI arrested Ross Ulbricht, owner of the anonymous online drug market, the Silk Road, media covered the Deep Web’s disturbing hit men and arms dealers lurking the Internet. Political TV dramas House of Cards and The Good Wife’s characters tracked down powerful hackers in Washington, D.C., and a fictional Silk Road creator in Chicago. Thomas Pynchon’s latest detective novel follows a fraud investigator hunting Russian gangsters and shady billionaires in a Deep Web virtual reality as 9-11 unfolds. Rapper Childish Gambino just finished The Deep Web Tour, promoting his album, Because the Internet, through hidden Web pages and singles about hashtags and memes. Director Alex Winter even raised $78,700 to produce Deep Web’s first documentary, Deep Web: The Untold Story of Bitcoin and Silk Road, in 2015.
Getting to the Deep Web, however, doesn’t require a special browser, criminal intent, or Bitcoins. Most of us already interact with and see Deep Web content, albeit unknowingly, by checking the weather or downloading a JSTOR article. About 96 percent of the Web is considered the Deep Web, but its exponential growth makes size estimates nearly impossible. The one time someone attempted measuring its size was in 2001 when BrightPlanet founder Michael Bergman discovered the Deep Web to be 400 to 550 times the size of the surface Web. Bergman, who coined the term “Deep Web,” noted in his study, “Very little has been written or known of the Deep Web. Estimates of size and importance have been anecdotal at best and certainly underestimate scale.”
At BrightPlanet, a Midwest Deep Web intelligence company, Data Acquisition Engineer Jamie Martin, 25, points out our generally flawed logic in seeing the Deep Web as an exclusive part of the larger Internet. “A lot of times people today are using the Deep Web without having any idea of what it is exactly,” he says. Martin gives another example of how we might book flights on Hotwire.com. You choose a date and destination, and Hotwire pulls up a list of flight options. “You’ll never see that page on a Google search just because you have to go directly to a search form to access it,” Martin explains. When you enter search terms into Google, you’re not seeing every site out there because results are based on what its Web crawlers can, well, crawl. We browse websites by clicking links from the homepage to point us to other pages; that’s similar to how Google’s crawlers gather data and bring it back to its servers (more on search). Within the enormous Deep Web, some Web pages are built to hide in plain sight — that’s the dark Web, where you can ostensibly hire contract killers and sell meth under the radar.
In actuality, the dark Web appeals to more than offenders: Domestic abuse and stalking victims, minors, protesters, and oppressed voices gain from anonymity, too. Added privacy aside, the dark Web’s advantage is its platform for free speech, without the real life repercussions of wire taps, data mines, and location trackers. “Unfortunately, a lot of people do not care about privacy,” Jeremy Martin of InfoSec Institute says. “That’s proven with smartphones and all the information leakage. I do computer forensics, and it’s amazing how much personal data we get just by listening to the network.” As a proponent of Deep Web usage, Martin writes in The Beginner’s Guide to the Internet Underground, “You never know who is watching. It could be your employer, your government, or your government’s enemies.”
Within the enormous Deep Web, some Web pages are built to hide in plain sight — that’s the dark Web, where you can ostensibly hire contract killers and sell meth under the radar.
Call it paranoia, or call it precaution, but unwanted surveillance and data collection have cornered even non-security experts into utilizing hidden services. Browsers like Tor or I2P let users connect to the dark Web’s secret networks. Since its release 11 years ago, the Tor browser has amassed more than two million steady users globally — the U.S., Germany, and France top the charts. Sandvik, now based in D.C., spent four years at the Tor Project and faces a predicament when it comes to educating users on Tor. “I once had a journalist ask me how I feel about people using Tor to do bad things,” Sandvik says. “I feel like it’s sort of the same as an Apple developer will feel when someone uses an iPhone to do something bad. At the end of the day, Tor is a tool, and how people choose to use it is up to them. It’s not for me to decide.”
No matter what you think you know about the Deep Web, this underground — and much bigger — part of the Web can be used to solve offline problems from predicting election winners to identifying counterfeit drugs. Now people, companies, and governments use the Deep Web to do what they want faster and better, and it starts with accessing the right information when they need it. The Web’s volume and depth keep expanding, and the fact that Google is no longer efficient enough for discovering quality information will become a starker reality. When Abe Lederman, CEO of Web Technologies, bought moss balls — a species of algae that grows in large, green balls — for his aquarium, he wondered: Are they toxic for my fish? Do the fish eat it? His curiosity sent him on the same Google quest with millions of search results many of us end up poring over. “What I got mostly was people trying to sell me stuff, and I wasn’t interested in buying them,” Lederman says. “I think I gave up.”
Lederman’s company mines the Deep Web for academic and medical libraries, providing one interface for organizations to search all their content. Their clients need optimized searches where advertising does not distract from the information or influence results, unlike on Google, where the search net revenue spiked from $10 billion in 2007 to $27 billion in 2013. “Google has become much more commercial,” Lederman says. “[They are] much more focused on bringing back results that they can have ads in. There are a lot of people trying to game the system by doing a lot of search optimization. The quality of Google has been going down.”
“I feel like it’s sort of the same as an Apple developer will feel when someone uses an iPhone to do something bad,” Sandvik says. “At the end of the day, Tor is a tool, and how people choose to use it is up to them. It’s not for me to decide.”
Even media claimed a spot in the Deep Web market. Launched in 2012 and headed by CEO Scott Cohen, Vocativ calls itself news from the Deep Web.Its intelligence technology, OpenMIND, picks up on open broadcasts and unprotected data — public databases, social media posts, forum threads. Theoretically, this gives them the ability to monitor online conversations and content and plan their reporting around buzz-worthy topics. That sort of online listening gave Vocativ a head start in reporting on the Middle East Respiratory Syndrome outbreak in May 2013. Cohen says they were tracking the information before the news became mainstream, and their story on the virus beat U.S. news outlets by nearly a month. “Source building in the traditional realm is you on the phone working a source, going out to lunch with someone,” Cohen says. “What we’re doing is a mass scale based in the Deep Web. You can call one person at a time. We can listen to hundreds of thousands of people via the Deep Web and data points.” It may seem intrusive, but how Vocativ goes about monitoring the online space likens to a coffee shop barista picking up on customers’ conversations in a shared space.
Companies aren’t the only ones uncovering information with sophisticated Deep Web searches, and on occasion, specific inquiries demand specific portals. John Miller, a 25-year-old IT professional, wanted an instructional guide to jailbreaking his phone, but when he went online to find that information, he didn’t “Google it.” “Maybe I could have found it on the [regular Web] but just happened to choose to look on the Deep Web because I knew I could find it there,” Miller says. The Deep Web delivered instructions on how to switch his phone’s operating system.
Applying Miller’s success on a larger scale opens Deep Web content to even greater possibilities. When confronted with natural disasters, FEMA or the Red Cross comes to mind. But emergencies can benefit from on-the-screen proficiency as much as they can on-the-ground relief. When Toronto flooded in July 2013, power cut off on more than 300,000 people, and property damage exceeded millions of dollars, but the tweets kept pouring in. To help insurance companies prioritize the hardest hit areas and identify potentially fraudulent claims, BrightPlanet harvested and filtered all tweets within Greater Toronto to create real-time heat maps and track results of the flooding. Jamie Martin assisted with collecting the data and filtering them in the map that July. Areas with fewer tweets, for instance, demanded further investigating. “Rather than it being caused by actual flooding, maybe it is actually caused a week earlier by someone’s sprinkler system, and they just wanted to piggyback off of the claims before,” Martin says.
Constant social media usage through tweets, posts, and shares makes the Deep Web behemoth boundless in growth and speed. Unlike hidden databases or classified documents, this sector of the Deep Web never stops producing content and never stops talking.
Sandvik’s best friend is getting married in August, and sharing links to dresses and flowers in a private Facebook message ended up flooding her ads with wedding merchandise over the next few days. Professionally or personally, maybe not everyone has a stake in Deep Web technologies, but there are days we want to make it through the mall without people trying to sell their products or to talk on the phone without people eavesdropping on the conversation. And sometimes you appreciate knowing you’re truly alone — just you and your search results.
Kathleen Rubino, Ally Balcerzak, and Donato DiRenzo contributed to research.
WORDS BY Ashley Branch
The newest trend in travel, selfie hotels, encourages everyone to snap a great vacation.
WORDS BY Lauren Boudreau
On the quest for cheap rent and cultural diversity, 20-somethings are reinventing the burbs. Here are seven of the nation’s finest.
WORDS BY Chris Landers
What to read, watch, and listen to now.