TMCnet Feature Free eNews Subscription
July 17, 2019

Who's the Best WHOIS Database Download Provider?

A quick Google (News - Alert) search on the term “best WHOIS database download” will turn up around 1.2 million results. That’s a whole lot of information to read and services to choose from. Frankly, I don’t think anyone would want to do that much reading, so I decided to narrow down the search for you.



The first thing you need to know is what WHOIS is. Let me just clarify. Though WHOIS is spelled in capital letters, it’s not an acronym. In the simplest terms, it’s the system that answers the question, “Who is responsible for a domain name or an IP address?”

In this article, I will define what a WHOIS database is, identify what kinds of people use a WHOIS database, explain how a WHOIS database works, enumerate the characteristics of a good WHOIS database, and name vendors who offer WHOIS database download services. Only then can we compare vendor offerings to find out who the best WHOIS database download provider is. A total of six WHOIS database download services were profiled and vetted in this article.

Table of Contents

What is a WHOIS database?

Who can benefit from WHOIS database downloads?

How do WHOIS database downloads work?

What makes a good WHOIS database product?

What are the best WHOIS database download products in the market?

  • WhoisXML API WHOIS Database Download
  • IQWhois WHOIS Database Download
  • JsonWHOIS WHOIS Database Download
  • Domain Name Stat WHOIS Database Download
  • Whoisology WHOIS Database Download
  • WDD WHOIS Database Download
  • DomainIndex WHOIS Database Download

The Verdict

What is a WHOIS database?

A WHOIS database is a comprehensive or exhaustive set of data that any person can find on a domain. It has millions of WHOIS records that contain information related to a specific domain registered in any part of the world at any given point in time. Note that no website can exist without registering the domain it’s hosted on—it’s part of the law.

Each WHOIS report typically contains the following information:

  • Domain name
  • Domain age
  • Domain status
  • Registration date
  • Date(s) when the domain was updated
  • Expiration date
  • Name servers
  • Other related servers
  • Registrar
  • Registrar’s contact details
  • Registrant’s name
  • Registrant’s contact details
  • Administrative contact’s name
  • Administrative contact’s details
  • Billing contact’s name
  • Billing contact’s details
  • Technical contact’s name
  • Technical contact’s details

The registrant’s and administrative, billing, and technical contacts’ details include their name, company, complete street address, email address, and phone and fax numbers. For a lot of domains, all these contacts may be a single person but no one is allowed to use fake information on his domain registration. Anyone found in violation of this will have his domain ownership and rights revoked. Some violators are even blacklisted or worse banned from owning domains ever again.

In some cases though, a domain registrant may opt for anonymous or private registration. This could be due to various reasons, including to protect his privacy or prevent unwanted solicitation. It’s a perfectly legal practice. That’s why a lot of web hosts and domain registrars offer this option for an additional fee, of course. In such a case, the service provider would give out an authorized representative’s contact details instead of the actual registrant’s on the WHOIS record.

The data in a WHOIS database can be filtered in various ways, depending on the kind of information a user wishes to focus on. The kinds of WHOIS reports that can be created include but are not limited to the following:

  • List of all the domains a registrant owns
  • Domain name activities recorded on a specific date
  • Top N companies in terms of domain ownership
  • List of all the domains a company owns
  • Top-level domain or TLD registration and/or termination trends within a given period of time
  • Historical data on a particular domain
  • Essential attributes of certain domain registrations (registrant’s name, email address, registrar, name servers, etc.)
  • List of all the servers related to a domain
  • Top N registrants with the greatest number of registered domains
  • List of all the domains a person owns

Who Can Benefit from WHOIS Database Downloads?

WHOIS reports have a lot of practical uses. One you’ve probably heard or read about in the news would be as a forensic investigation tool in cybercrime investigations. To provide a more or less comprehensive list, I surveyed the websites of the vendors whose products we will take a closer look at in this article. A thorough scan of their WHOIS database download pages resulted in the following list of potential beneficiaries:

  • Cybersecurity professionals: Those in the IT security industry can use WHOIS data when conducting threat analyses or detecting fraudulent or malicious activities. With it, they can also supplement known indicators of compromise or IoCs to enhance their collection of threat intelligence, improve their security information and event management or SIEM platforms, and innovate their existing automation and orchestration tools.
  • Domain registrars: Those in the domain registry business, meanwhile, can use WHOIS data to keep track of the domains they are responsible for. This is especially true for domain registrars that have multiple offices and subsidiaries worldwide who may work independently of each other and so would find it hard to track all of the domains they registered or own.
  • Law enforcement agents: Those in charge of policing the Internet can use WHOIS data to identify all domains, websites, and IP addresses associated with fraudulent or suspicious activities and cybercriminals. This is especially useful due to the fact that law enforcement agents need to collect evidence before they can launch an actual investigation.
  • Marketing researchers and analysts: Chief marketing officers or CMOs, marketing executives, and marketing professionals can also use WHOIS data to examine domain registration trends in order to create market forecasts and come up with data-driven forward-looking strategies for their companies.
  • Brand agents: Those in charge of protecting their business’s intellectual property can also check for potential trademark infringements by scoping out domain name similarities, duplicates, or copycats using a WHOIS database.
  • Payment processors, banks, and fraud solution vendors: Financial institutions and service providers can also enhance their machine learning- or ML-based research and modeling tools for fraud detection with the aid of a WHOIS database.
  • Contact data warehouses: Those in the business of providing contact data to companies can verify and enrich their contact database offerings with available WHOIS data.
  • Investment banking institutions: Investment bankers can also use WHOIS data to enhance their business intelligence on portfolio companies as well as monitor corporation domain assets.
  • Domainers: Domainers can also identify growth areas for investment by looking at WHOIS data in order to beat their competitors.
  • Digital marketers: To fuel marketing campaigns, digital marketers would also do well to look at WHOIS data. The information will help them learn what competitors plan to do in the future. Domains can, after all, give them hints on possible new products or services their company’s competitors may launch.

How Do WHOIS Database Downloads Work?

Now, there are several companies that offer either access to their WHOIS databases via interactive Web portals or their entire WHOIS databases via downloads. Depending on a user’s system infrastructure, he can choose from at least three download formats—MySQL, MySQL dump, or CSV. You’re probably asking how these three formats differ. Read on to find out.

  • A MySQL file uses the .FRM extension with FoRMat or FRM files for tables that can link to MySQL cross-platform relational databases. FRM files can be read by programs such as MySQL, Microsoft (News - Alert) Visual Basic, Corel WordPerfect, iBlaze, and MYOB. When opened, it looks like a regular text editor file. For the nontechies out there, these files can be opened and read by Notepad on Windows or TextEdit on Mac OS.
  • A MySQL dump, meanwhile, is also part of the MySQL relational database package that allows a user to “dump” a database or a collection of databases for either backup or transfer to another SQL server. Note that the server that imports the database does not have to be a MySQL server.
  • Finally, a Comma Separated Values or CSV file is a plain text file that contains a list of data. It’s often used to exchange data between different applications and is readable via any spreadsheet application—Excel on a Windows or Numbers on Mac OS. Here's an example.

The first step in downloading a WHOIS database is to register to your desired service. Once done, look for the Download button. You should be asked to choose what format you would like to download if that is an option. Some providers don’t offer that many choices as you’ll see in our vendor comparison later on. Click Download, save the file on your computer, and you’re done. You can start using the WHOIS database.

What Makes a Good WHOIS Database Download Product?

Keep in mind that not all WHOIS databases are created equal. When searching for the best WHOIS database download product, look for the following characteristics:

  • Size does matter. The bigger the number of domains in the database, the better. The best WHOIS database download product can be determined by how exhaustive or complete the information it contains is. To date, the magic number should be around 7 billion WHOIS records. That about corresponds to how big the Internet currently is.
  • Keep up or get left behind. The latest VeriSign Domain Name Industry Brief highlighted that the first quarter of 2019 alone closed at 351.8 million domain name registrations across all TLDs, indicating a year-over-year increase of 18 million or 5.4 percent. New TLDs are added to the space regularly, and the provider that has the greatest number of TLDs (generic or gTLDs and country code or ccTLDs) should be at the top of your mind. There are over a thousand gTLDs and hundreds of ccTLDs. The closer to these numbers a WHOIS database vendor has, the better. A regularly updated database should also be an indicator that the provider is keeping pace with the Internet’s constant growth.
  • Quality data translates to better insights. In data analytics, volume isn’t the only important thing though. Comparisons are easier to do, after all, when the information you’re working on is consistently formatted. As such, the better-parsed and better-structured the data your provider gives, the less time you’ll need to spend on filtering and cleanup before diving into actual data analysis.
  • A trusted name goes a long way. In the same way that you check out product reviews before buying anything, you should also see what customers have to say about the vendor you’re eyeing. Companies that have been in the business for a long while typically provide good products and services, especially if they have big brand names vetting for them. Check out customer reviews before making the final decision. Look for independent user reviews of the vendors’ products as well.
  • A tool can only be truly useful if it’s easy to use. Companies use a wide variety of applications or programs in their network. The best WHOIS database download providers should thus offer files in formats that are not just easy for anyone to open and decipher, but also can be easily integrated into and used with already-existing frameworks and systems.

What Are the Best WHOIS Database Download Products in the Market?

NamePros, a Domaining-recommended reference for all things that have to do with domain names, provided a list of the best WHOIS database download providers back in late 2018. Let’s see if I can provide you with the latest updates. But first, let’s get to know the WHOIS database download providers better.

PROVIDER 1: WhoisXML API

WhoisXML API’s WHOIS Database Download touts a unified and consistent database that contains billions of historic WHOIS records, domains and subdomains and thousands of TLDs (both gTLDs and ccTLDs). It also has records of more than 9.1 million IP netblocks, translating to a 99.5% coverage of all the IP addresses in use today worldwide with 7+ B WHOIS records, 330+ M active domains, and 2,864+ TLDs.

WhoisXML API has been collecting and parsing domain WHOIS records for more than a decade now. It has served Fortune 500 companies, threat intelligence and infosec companies, anti-malware and security vendors, cybercrime units, government agencies, brand protection agencies, domain registries and registrars, domain investors and brokers, banks, payment processors, telcos, marketing researchers, big data warehouses, web analytics firms, investment funds, web developers, and many more.

It has a 52,000-strong list of satisfied customers that includes big name brands like F-Secure, HSBC, IBM, Symantec, RSA Security, Amazon, Apple, Cisco, Google, American Express, AT&T (News - Alert), DuckDuckGo, eBay, just to name a few. It has also been recognized as one of Inc. 5000’s fastest-growing Top IT Companies, ranking #268 in 2017 and #651 in 2018.

WhoisXML API’s WHOIS Database Download provides a well-parsed and well-structured database that can be downloaded in three formats—MySQL, MySQL dump, and CSV. Because the information in the database is consistently formatted, it can be easily integrated into a company’s existing applications and systems as well.

PROVIDER 2: IQWhois

IQWhois is a reverse WHOIS domain name ownership database provider. Its database boasts about containing billions of domain name WHOIS records that are regularly updated. It also has a constantly growing data archive on several domain names. All the information in its database — 5 B WHOIS records, 300 M active domains, hundreds of TLDs — can be used to conduct research on individual domains and owners or entire portfolios of domain names.

IQWhois’s WHOIS Database Download is exceptionally useful for tracking connections between domain names, cross-referencing ownership details, and brand monitoring. Its WHOIS database can be downloaded in three formats—MySQL, MySQL dump, and CSV.

PROVIDER 3: JsonWHOIS

JsonWHOIS is a domain API services provider that, unlike others in the field, provides social media statistics such as Google, Twitter, Facebook (News - Alert), LinkedIn, Pinterest, and Stumbleupon count, along with WHOIS data, which contains 300+ M active domains, and 1,000+ TLDs.

JsonWHOIS’s WHOIS Database Download also allows users to take screenshots of the historical states of the company websites in its database. This feature can give you additional insights into a company, allowing you to create a better profile or picture in your head. Its database comes in three formats too—MySQL, MySQL dump, and CSV.

PROVIDER 4: Domain Name Stat

Domain Name Stat offers a comprehensive set of properly parsed information on all the domains registered via WHOIS Database Download —which like its predecessors include 5+ B WHOIS records, 300+ M active domains, and hundreds of TLDs.

The provider has been monitoring the Internet for domain registrations since 2008 and is also a recipient of the Domaining Excellence Award. Its database is downloadable in the MySQL, MySQL dump, and CSV file formats as well.

PROVIDER 5: Whoisology

Whoisology is a domain name ownership archive with billions of searchable and cross-referenced domain name WHOIS records that primarily focuses on providing reverse WHOIS lookups through 5+ B WHOIS records, 300+ M active domains, and close to 2000 TLDs.

Whoisology’s WHOIS Database Download provides information in the usual three formats—MySQL, MySQL dump, and CSV.

PROVIDER 6: WHOIS Database Download (WDD)

Whois Database Download (WDD) specializes in providing WHOIS information that’s updated daily. As such, its database includes even newly registered domains. It also gives subscribers free access to 30 days’ worth of historical data for 40+ M, active domains, and hundreds of TLDs. Like most providers, WDD offers customized data packages by country, region, and more.

PROVIDER 7: The Domain Research Corporation

The Domain Research Corporation’s DomainIndex.com contains WHOIS records that are not older than 4 months, most of which are up to date at the time of download. Its data comes in both parsed and raw forms, and includes several million gTLD WHOIS records. The database can be downloaded as a MySQL dump file.

DomainIndex.com requires users to have a server infrastructure and technical understanding to unpack and run a database such as theirs.

Bottom line: All of the providers profiled above have customized packages that would suit any individual’s or company’s requirements.

The Verdict

Now that we have all the information we need to do a comparison of the services out in the market today, it’s time to rank them. Let’s do that by answering the questions below.

Which vendor has the biggest volume of WHOIS records?

1. WhoisXML API, with more than 7+ billion

2. Domain Name Stat and Whoisology, with more than 5 billion

3. IQWhois, with 5 billion

4. DomainIndex.com, with more than 135 million

Which vendor has the greatest number of active domains?

1. WhoisXML API, more than 330 million

2. Whoisology, JsonWHOIS, and Domain Name Stat, and IQWhois with more than 300 million domains

3. WDD, more than 40 million

Which vendor has the widest gTLD coverage?

1. WhoisXML API, 1,260

2. Domain Name Stat and JsonWHOIS, 1250

3. IQWhois and Whoisology, 1,246

4. WDD, 514

5. DomainIndex.com, 5
 

Which vendor has the widest ccTLD coverage?

1. WhoisXML API, 1,592

2. IQWhois, Whoisology, and JsonWHOIS, 1,200

3. Domain Name Stat, 322

4. WDD, 240

5. DomainIndex.com, 16

Which vendor is the most versatile with regard to download formats?

1. WhoisXML API, IQWhois, JsonWHOIS, Domain Name Stat and Whoisology, 3 formats

2. WDD and DomainIndex.com, 1 format

Which vendor answers the needs of the greatest number of user types?

1. WhoisXML API, 10

2. Domain Name Stat, 7

3. JsonWHOIS and Whoisology, 5
4. IQWhois, WDD, DomainIndex.com, 4
 

---

All in all, WhoisXML API topped the vendor list for most items. For more information, you might want to read this extensive WHOIS Database Download review or access the company’s WHOIS Database Download Product Sheet.

About the Author

Jonathan Zhang is the founder and CEO of Threat Intelligence Platform (TIP) — a data, tool, and API provider that specializes in automated threat detection, security analysis, and threat intelligence solutions for Fortune 1000 and cybersecurity companies. TIP is part of the Whois API Inc. family which is a trusted intelligence vendor by over 50,000 clients.


 
» More TMCnet Feature Articles
Get stories like this delivered straight to your inbox. [Free eNews Subscription]
SHARE THIS ARTICLE

LATEST TMCNET ARTICLES

» More TMCnet Feature Articles