OSINT Tip: Bulk URL HTTP Status Code, Header & Redirect Checker

#LookDeeper: HTTP Status Code Checker - Professional Tool for Online Investigations

In online investigations, there are many cases where the investigator does not have to look at just one website. Particularly in cases where more than 20 domains are involved, it makes sense to conduct information queries in bulk.

This is then a quick way to get important information about all domains through a single query. There is a lot of information that is important when viewing and evaluating a website. For this there are tools with which a larger number of domains can be queried at the same time. In this post I introduce the HTTP Status Code Checker as my favorite tool for checking HTTP status codes and redirect information.

The HTTP Status Code Checker checks the following for the website:

  1. status code
  2. response header
  3. possible redirect location
  4. redirect chain

The page is visually tidy and easy to use. The use is free of charge and free of advertising.

Via the input field up to 100 domains can be directly copied into the field and queried. It does not matter whether the URL starts with http:// or only the domain name is in the list

My tip would be to always work with a smaller number of domains (e.g. 30) to keep the waiting time short.

When I query a single domain, the full “HTTP response body” to the query is shown.

Any User Agent can be selected in the “Set Request Headers” settings. For example the Google Bot or a mobile browser.

It is also possible to enter login information or create an optional HTTP request.

As a result of a query of several domains I get a clear list with the status codes and the number of redirects.

By mouse-over over the status codes I can also retrieve the details and follow the redirect chain.

It is helpful to be able to export the results as CSV. This makes it easy to process the information in your own system.

We use the tool when we check a larger number of domains for availability. Especially if we want to process this information with our own intelligence solutions.

As an alternative solution a comparable tool, the “urlchecker.com”, can be used.

This solution provides similar possibilities as HTTP Status Code Checker. In addition, exports can also be downloaded as .json. You can check up to 5000 URL’s in one process and save your session and continue later.

As input the tool expects a complete URL like “http://domain.com”.

The result is a well structured overview. This can be extended per URL and per click by the detailed information.

The website operator also provides an informative overview of the most common HTTP status codes here. More detailed information can be found at Wikipedia.

For domains that have not been resolved or for which there is no response code, I recommend a manual check. In this article I describe how you can use the “Geo Screenshot” to quickly check whether a website is only accessible from certain countries. Geotargeting can limit the visibility of the site or content to selected countries.

The solution is error-free, clearly structured and easy to use. Results for redirect checks and http response can be exported as CSV.

A status or progress display would be desirable so that the user can see how long the query will still take.

Conclusion

The tool is absolutely recommendable. The information provided is to be used well for the further determination.

You must decide for yourself whether you want to enter your case information on this website. The Dutch operator offers a privacy statement for this purpose.

For a free tool, the scope of services is excellent.

Follow me on Twitter for up-to-date information on OSINT, cyber investigations, investigations & research.

How can we help you?

Want to discuss a confidential matter in more detail?
Need a private investigator in Germany or Europe?
Get in touch and we’ll work out a free tailor-made proposal for you.

Send us an email, use our or

Contact
Share on facebook
Facebook
Share on xing
XING
Share on twitter
Twitter
Share on linkedin
LinkedIn
Share on email
Email