Cloud Provider and abusers
I’ve always wondered what Public Cloud Provider is the favorite of the ‘bad guys’ to do the things the bad guys do. There are many factors that can make a provider a favorite compared to others, and I can think of them myself:
- Use SSH access per user and password instead of public and private key exchange.
- Allow root access.
- Default versions of operating systems that are not updated at the kernel and library level.
- Poor management of the perimeter firewall or no perimeter firewall.
This is true about the Operating System side. But there are also factors relating to the cloud computing provider that can make a provider more ‘bad guy’ friendly than others:
- Easy onboarding, allowing you to automate the creation of new accounts.
- Free trial and generous credit accounts.
- Easy to find documentation explaining the use of the platform.
- Proactive verification by the Cloud Providers that there are no services from their IP address pool that abuse third parties.
Ranking based on real data and not assumptions
With all this information we can make a ranking that is probably more fruit of our personal tastes and affinities than really the result of empirical tests. But now with Apility.io we can know with a high level of accuracy which of the suppliers is more and less tolerant thanks to the data we have in our Data Lake.
So I wanted to test the ‘Big Three’ Public Cloud Providers: Amazon Web Services, Google Cloud Platform, and Microsoft Azure. And as an extra bonus, I wanted to test two cheap Service Providers very popular in the industry: DigitalOcean and OVH.
A very simple approach is testing every single IP address of the pool allocated for each provider against our API, count the result and then compare. All the big three players have API requests to obtain this information. Sadly, neither DigitalOcean and OVH do have such an API request to return their pool of allocable IP addresses. So, for DigitalOcean and OVH I will use the prefixes returned by their Autonomous Systems. So the total count of IP addresses per Service Provider in their Computing services are:
- Amazon Web Services: 31.294.171 IP addresses.
- Microsoft Azure:13.079.822 IP addresses.
- Google Cloud Platform: 2.824.906 IP addresses.
- DigitalOcean: 1.454.505 IP addresses.
- OVH: 2.736.033 IP addresses.
So we have to send 51.389.437 requests. We can send batch requests and send 1.000 queries per API request but even sending batch requests it would take too long to get the results. We need to find a different approach.
Download the full Apility.io database in memory
A much better approach is to download the full Apility.io database, then load into memory and test every single IP in our database against the list of prefixes available for each Cloud Provider. Our subscribers of the Professional and Enterprise Plans can download the database of IP addresses, Domains and Emails in CSV, JSON and SQL formats. For each IP address of our database, we look up if the up falls into the intervals. The number of IP addresses and CIDR blocks in our database rounds between 700.000 and 1.000.000. The implementation is much more complex. I had to implement the download of the database and how to load into memory, but also I had to use an Interval Tree library to implement an efficient search between the ranges of the prefixes of the computing pool.
You can find the full implementation here.
This approach is much faster, taking just a few seconds to run in a 2015 i7 MacBook Pro with SSD.
After running the program for each Cloud Provider we obtained the following count of malicious IP addresses:
- Amazon Web Services: 3.678 IP addresses. There s 1.1 malicious IPv4 address for every 10000 IP addresses of the EC2 pool.
- Microsoft Azure:680 IP addresses. That’s 0.5 malicious IPv4 address for every 10000 IP addresses of the computing pool.
- Google Cloud Platform: 972 IP addresses. That’s 3.4 malicious IPv4 address for every 10000 IP addresses of the computing pool.
- DigitalOcean: 10.945 IP addresses. That’s 75 malicious IPv4 address for every 10000 IP addresses.
- OVH: 6.409 IP addresses. That’s 23.5 malicious IPv4 address for every 10000 IP addresses.
If we take the results as they are, we can draw several conclusions: First, Microsft Azure is the Public Cloud Provider with the lower number of IP addresses in our Data Lake of malicious IP addresses. AWS triples the number of malicious IP addresses and Google Cloud Platform almost triples the number of AWS. I’m not quite sure why this happens, but I think the case of GCP has to do with a strategy of very easy onboarding and generous trial and free credit compared to AWS and Azure.
The case for DigitalOcean and OVH is completely different. Due to its very aggressive pricing strategy, OVH looks like a very good target for the bad guys, but surprisingly DigitalOcean more than triples its figures. Again my personal opinion is DigitalOcean has (or maybe had) a strategy of easy onboarding and easy access to free credit.
A very important question is: How accurate are these results? If we trust in our Data Lake of malicious IP addresses the result is accurate IF we assume that the usage density of IP addresses for each pool looks similar. And I don’t think this is the case. What does it mean? We are assuming that each provider has a very similar number of IP addresses in use versus IP addresses allocated but not assigned to any service. This value should be applied for each provider to obtain a very accurate result. Otherwise, we have to assume that the usage density is equal for each provider and this is too naive. So, if you know how to obtain this value, please let me know and I will run again the tests applying this factor.
Tracking your IP addresses and pools in real time
In Apility.io we have recently implemented a new feature that allows you to be informed in real-time if the IP addresses pool of your services has been included in any of the blacklists we track. It is very important to know if your reputation has been compromised as soon as it happens because a connection to your services could be banned from some DNS providers that implement anti-malware features, Firewalls, Mailing services and API services that deny connections from suspicious sites. But the most important reason is your own security and safety: if your servers have been compromised you need to act as soon as you know it, and our Real-Time alerts help you to know it as it happens.