Free utility · No signup · Plain results
/tools/robots
Robots + Sitemap
robots.txt + discovered sitemap URLs.
robots.txt + sitemaps for wdhdomain.com
robots.txt
<!DOCTYPE html>
<html lang="en">
<head><meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>404 - Page Not Found - WDH Domain</title>
<!-- Styling -->
<link href="https://fonts.googleapis.com/css2?family=Poppins:wght@100;300;400;500;700;900&display=swap" rel="stylesheet">
<link href="/templates/kohost-professional/css/all.min.css?v=ab8c2e" rel="stylesheet">
<link href="/assets/css/fontawesome-all.min.css" rel="stylesheet">
<link href="/templates/kohost-professional/img/favicon.png" rel="shortcut icon">
<link href="/templates/kohost-professional/css/custom.css?v=ab8c2e" rel="stylesheet">
<!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
<script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
<![endif]-->
<script type="text/javascript">
var csrfToken = '291ce6e669d1a3bb4b4704964735cd3525a3a1ff',
markdownGuide = 'Markdown Guide',
locale = 'en',
saved = 'saved',
saving = 'autosaving',
saved = 'saved',
saving = 'autosaving',
whmcsBaseUrl = "",
requiredText = 'Required',
recaptchaSiteKey = "";
</script>
<script src="/templates/kohost-professional/js/scripts.min.js?v=ab8c2e"></script>
</head>
<body data-phone-cc-input="1" class="body-wraper ">
<header id="standard" class="white-bg header-main-menu">
<div class="container">
<div class="row">
<section id="main-menu">
<nav id="nav" class="navbar navbar-default navbar-main" role="navigation">
<!-- Brand and toggle get grouped for better mobile display -->
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#primary-nav">
<span class="sr-only">toggleNav</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<!--
aClass = additional class if you want to pass add here or skip
logo = option: priamry or secondary. Primary will be color logo and Secondary will be white version logo
-->
<a href="/index.php" class="logo navbar-brand"><img src="/templates/kohost-professional/img/logo-primary.png" class="img-responsive" alt="WDH Domain"></a>
</div>
<!-- Collect the nav links, forms, and other content for toggling -->
<div class="collapse navbar-collapse" id="primary-nav">
<ul class="nav navbar-nav navbar-center">
<li menuItemName="Home" class="" id="Primary_Navbar-Home">
<a href="/index.php">
<!-- Check if the item label is 'Store' and replace it -->
Home
</a>
</li>
<li menuItemName="Store" class="dropdown" id="Primary_Navbar-Store">
<a class="dropdown-toggle" data-toggle="dropdown" href="#">
<!-- Check if the item label is 'Store' and replace it -->
Domain & Hosting
<b class="caret"></b> </a>
<ul class="dropdown-menu">
<li menuItemName="Browse Products Services" id="Primary_Navbar-Store-Browse_Products_Services">
<a href="/index.php?rp=/store">
Browse All
</a>
</li>
<li menuItemName="Shop Divider 1" class="nav-divider" id="Primary_Navbar-Store-Shop_Divider_1">
<a href="">
-----
</a>
</li>
<li menuItemName="cPanel Hosting USA" id="Primary_Navbar-Store-cPanel_Hosting_USA">
<a href="/index.php?rp=/store/cpanel-hosting-usa">
cPanel Hosting USA
</a>
</li>
<li menuItemName="Reseller Hosting WHM" id="Primary_Navbar-Store-Reseller_Hosting_WHM">
<a href="/index.php?rp=/store/reseller-hosting-whm">
Reseller Hosting WHM
</a>
</li>
<li menuItemName="VPS Hosting USA" id="Primary_Navbar-Store-VPS_Hosting_USA">
<a href="/index.php?rp=/store/vps-hosting">
VPS Hosting USA
</a>
</li>
<li menuItemName="Managed Storage VPS" id="Primary_Navbar-Store-Managed_Storage_VPS">
<a href="/index.php?rp=/store/managed-storage-vps">
Managed Storage VPS
</a>
</li>
<li menuItemName="Dedicated Server USA" id="Primary_Navbar-Store-Dedicated_Server_USA">
<a href="/index.php?rp=/store/dedicated-server-usa">
Dedicated Server USA
</a>
</li>
<li menuItemName="Data Backup - Storage" id="Primary_Navbar-Store-Data_Backup_-_Storage">
<a href="/index.php?rp=/store/data-backup-storage">
Data Backup - Storage
</a>
</li>
<li menuItemName="Web Hosting Transfer" id="Primary_Navbar-Store-Web_Hosting_Transfer">
<a href="/index.php?rp=/store/hosting-migration">
Web Hosting Transfer
</a>
</li>
<li menuItemName="Google Business Email Transfer" id="Primary_Navbar-Store-Google_Business_Email_Transfer">
<a href="/index.php?rp=/store/google-business-email-migration">
Google Business Email Transfer
</a>
</li>
<li menuItemName="Business Email Accounts" id="Primary_Navbar-Store-Business_Email_Accounts">
<a href="/index.php?rp=/store/business-email-accounts">
Business Email Accounts
</a>
</li>
<li menuItemName="Register a New Domain" id="Primary_Navbar-Store-Register_a_New_Domain">
<a href="/cart.php?a=add&domain=register">
Register a New Domain
</a>
</li>
<li menuItemName="Transfer a Domain to Us" id="Primary_Navbar-Store-Transfer_a_Domain_to_Us">
<a href="/cart.php?a=add&domain=transfer">
Transfer Domains to Us
</a>
</li>
</ul>
</li>
<li menuItemName="Announcements" class="" id="Primary_Navbar-Announcements">
<a href="/index.php?rp=/announcements">
<!-- Check if the item label is 'Store' and replace it -->
News
</a>
</li>
<li menuItemName="Knowledgebase" class="" id="Primary_Navbar-Knowledgebase">
<a href="/index.php?rp=/knowledgebase">
<!-- Check if the item label is 'Store' and replace it -->
Knowledgebase
</a>
</li>
<li menuItemName="Contact Us" class="" id="Primary_Navbar-Contact_Us">
<a href="/contact.php">
<!-- Check if the item label is 'Store' and replace it -->
Contact Us
</a>
</li>
</ul>
<ul class="nav navbar-nav navbar-right secondary-nav">
<li class="primary-action">
<a href="/cart.php?a=view"> <i class="fas fa-shopping-basket" data-toggle="tooltip" data-placement="top" title="" data-original-title="View Cart"></i></a>
</li>
<!--currency dropdown start-->
<li class="dropdown" id="currency-dropdown">
<a class="dropdown-toggle" data-toggle="dropdown" href="#">
<img alt="USD"src="/templates/kohost-professional/img/flags/USD.svg" class="paises">
<span>$</span> <b class="caret"></b>
</a>
<ul class="dropdown-menu">
<li><a href="https://wdhdomain.com/robots.txt?currency=2"><img alt="BDT"src="/templates/kohost-professional/img/flags/BDT.svg" class="paises"><span>৳</span></a></li>
</ul>
</li>
<!--currency dropdown end-->
<li class="primary-action">
<a href="/register.php" class="sign-up-btn">Sign Up</a>
</li>
<li class="primary-action">
<a href="/clientarea.php" class="login-btn">Login</a>
</li>
</ul>
</div><!-- /.navbar-collapse -->
</nav>
</section>
</div>
</div>
</header>
<section class="page-header-section">
<div class="container">
<div class="row align-items-center">
<div class="col-md-7 col-lg-6">
<div class="page-header-content text-white">
<h1 class="text-white mb-2">
404 - Page Not Found
</h1>
</div>
<div class="custom-breadcrumb">
<ol class="d-inline-block bg-transparent list-inline py-0 pl-0">
</ol>
</div>
</div>
</div>
</div>
</section>
<section id="main-body">
<div class="container-fluid without-padding">
<div class="row">
<!-- Container for main page display content -->
<div class="col-xs-12 main-content">
<div class="container">
<div class="error-container boxed">
<h1><i class="fas fa-exclamation-triangle"></i> Oops!</h1>
<h3>We couldn't find that page</h3>
<p>Please try navigating using the options below.</p>
<div class="buttons">
<a href="https://wdhdomain.com/" class="btn btn-default">
Homepage
</a>
<a href="https://wdhdomain.com/submitticket.php" class="btn btn-default">
Contact Support
</a>
</div>
</div>
</div>
</div><!-- /.main-content -->
<div class="clearfix"></div>
</div>
</div>
</section>
<!--footer section start-->
<footer id="footer" class="footer-section">
<!--footer top start-->
<div class="footer-top gradient-bg">
<div class="container">
<div class="row">
<div class="col-md-3">
<div class="row footer-top-wrap">
<div class="col-sm-12">
<div class="footer-nav-wrap text-white">
<!--
aClass = additional class if you want to pass add here or skip
logo = option: priamry or secondary. Primary will be color logo and Secondary will be white version logo
-->
<a href="/index.php" class="logo "><img src="/templates/kohost-professional/img/logo-secondary.png" class="img-responsive" alt="WDH Domain"></a>
<p class="mt-20">Dynamically re-engineer high standards in functiona with alternative paradigms resource maximizing initiatives.</p>
<ul class="list-inline social-list background-color social-hover-2 mt-2">
<li class="list-inline-item tooltip-hover"><a class="twitter" href="#"><i class="fab fa-twitter"></i></a><div class="tooltip-item">Twitter</div></li>
<li class="list-inline-item tooltip-hover"><a class="youtube" href="#"><i class="fab fa-youtube"></i></a><div class="tooltip-item">Youtube</div></li>
<li class="list-inline-item tooltip-hover"><a class="linkedin" href="#"><i class="fab fa-linkedin-in"></i></a><div class="tooltip-item">Linkedin</div></li>
<li class="list-inline-item tooltip-hover"><a class="dribbble" href="#"><i class="fab fa-dribbble"></i></a><div class="tooltip-item">Dribbble</div></li>
</ul>
</div>
</div>
</div>
</div>
<div class="col-md-9">
<div class="row footer-top-wrap">
<div class="col-md-3 col-sm-6">
<div class="footer-nav-wrap text-white">
<h4 class="text-white">PRODUCTS</h4>
<ul class="nav flex-column">
<li class="nav-item">
<a class="nav-link" href="#">WordPress Hosting</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Shared Hosting</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Virtual Private Servers</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Dedicated Servers</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Domain Names</a>
</li>
</ul>
</div>
</div>
<div class="col-md-3 col-sm-6">
<div class="footer-nav-wrap text-white">
<h4 class="text-white">COMPANY</h4>
<ul class="nav flex-column">
<li class="nav-item">
<a class="nav-link" href="#">About</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Blog</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Affiliates</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Careers</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Community</a>
</li>
</ul>
</div>
</div>
<div class="col-md-3 col-sm-6">
<div class="footer-nav-wrap text-white">
<h4 class="text-white">LEGAL</h4>
<ul class="nav flex-column">
<li class="nav-item">
<a class="nav-link" href="#">Security</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Legal Information</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Privacy Policy</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Terms of Service</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">WHOIS Lookup</a>
</li>
</ul>
</div>
</div>
<div class="col-md-3 col-sm-6">
<div class="footer-nav-wrap text-white">
<h4 class="text-white">SUPPORT</h4>
<ul class="nav flex-column">
<li class="nav-item">
<a class="nav-link" href="#">Contact</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Knowledge Base</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Forums</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">System Status</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">Support Team</a>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="container">
<div class="back-to-top-btn">
<a href="#" class="back-to-top"><i class="fas fa-chevron-up"></i></a>
</div>
</div>
</div>
<!--footer top end-->
<!--footer copyright start-->
<div class="footer-bottom py-3">
<div class="container">
<div class="row align-items-center justify-content-between display-flex">
<div class="col-md-8 col-lg-8 col-sm-7 col-xs-12">
<p class="copyright-text pb-0 mb-0 text-white">Copyright © 2026 WDH Domain. All Rights Reserved.</p>
</div>
<div class="col-md-4 col-lg-4 col-sm-5 col-xs-12">
<div class="payment-method text-lg-right text-md-right text-left">
<ul class="list-inline">
<li class="list-inline-item"><a class="small-text" href="#">Language :</a></li>
<li class="list-inline-item">
<div class="dropup language-chooser">
<button class="btn btn-default dropdown-toggle" type="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
English
<span class="caret"></span>
</button>
<ul class="dropdown-menu" aria-labelledby="languageChooser" id="languageChooserContent" class="hidden">
<li>
<a href="/robots.txt?language=arabic">العربية</a>
</li>
<li>
<a href="/robots.txt?language=azerbaijani">Azerbaijani</a>
</li>
<li>
<a href="/robots.txt?language=catalan">Català</a>
</li>
<li>
<a href="/robots.txt?language=chinese">中文</a>
</li>
<li>
<a href="/robots.txt?language=croatian">Hrvatski</a>
</li>
<li>
<a href="/robots.txt?language=czech">Čeština</a>
</li>
<li>
<a href="/robots.txt?language=danish">Dansk</a>
</li>
<li>
<a href="/robots.txt?language=dutch">Nederlands</a>
</li>
<li>
<a href="/robots.txt?language=english">English</a>
</li>
<li>
<a href="/robots.txt?language=estonian">Estonian</a>
</li>
<li>
<a href="/robots.txt?language=farsi">Persian</a>
</li>
<li>
<a href="/robots.txt?language=french">Français</a>
</li>
<li>
<a href="/robots.txt?language=german">Deutsch</a>
</li>
<li>
<a href="/robots.txt?language=hebrew">עברית</a>
</li>
<li>
<a href="/robots.txt?language=hungarian">Magyar</a>
</li>
<li>
<a href="/robots.txt?language=italian">Italiano</a>
</li>
<li>
<a href="/robots.txt?language=macedonian">Macedonian</a>
</li>
<li>
<a href="/robots.txt?language=norwegian">Norwegian</a>
</li>
<li>
<a href="/robots.txt?language=portuguese-br">Português</a>
</li>
<li>
<a href="/robots.txt?language=portuguese-pt">Português</a>
</li>
<li>
<a href="/robots.txt?language=romanian">Română</a>
</li>
<li>
<a href="/robots.txt?language=russian">Русский</a>
</li>
<li>
<a href="/robots.txt?language=spanish">Español</a>
</li>
<li>
<a href="/robots.txt?language=swedish">Svenska</a>
</li>
<li>
<a href="/robots.txt?language=turkish">Türkçe</a>
</li>
<li>
<a href="/robots.txt?language=ukranian">Українська</a>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<!--footer copyright end-->
</footer>
<!--footer section end-->
<div id="fullpage-overlay" class="hidden">
<div class="outer-wrapper">
<div class="inner-wrapper">
<img src="/assets/img/overlay-spinner.svg">
<br>
<span class="msg"></span>
</div>
</div>
</div>
<div class="modal system-modal fade" id="modalAjax" tabindex="-1" role="dialog" aria-hidden="true">
<div class="modal-dialog">
<div class="modal-content panel-primary">
<div class="modal-header panel-heading">
<button type="button" class="close" data-dismiss="modal">
<span aria-hidden="true">×</span>
<span class="sr-only">Close</span>
</button>
<h4 class="modal-title"></h4>
</div>
<div class="modal-body panel-body">
Loading...
</div>
<div class="modal-footer panel-footer">
<div class="pull-left loader">
<i class="fas fa-circle-notch fa-spin"></i>
Loading...
</div>
<button type="button" class="btn btn-default" data-dismiss="modal">
Close
</button>
<button type="button" class="btn btn-primary modal-submit">
Submit
</button>
</div>
</div>
</div>
</div>
<form action="#" id="frmGeneratePassword" class="form-horizontal">
<div class="modal fade" id="modalGeneratePassword">
<div class="modal-dialog">
<div class="modal-content panel-primary">
<div class="modal-header panel-heading">
<button type="button" class="close" data-dismiss="modal" aria-label="Close"><span aria-hidden="true">×</span></button>
<h4 class="modal-title">
Generate Password
</h4>
</div>
<div class="modal-body">
<div class="alert alert-danger hidden" id="generatePwLengthError">
Please enter a number between 8 and 64 for the password length
</div>
<div class="form-group">
<label for="generatePwLength" class="col-sm-4 control-label">Password Length</label>
<div class="col-sm-8">
<input type="number" min="8" max="64" value="12" step="1" class="form-control input-inline input-inline-100" id="inputGeneratePasswordLength">
</div>
</div>
<div class="form-group">
<label for="generatePwOutput" class="col-sm-4 control-label">Generated Password</label>
<div class="col-sm-8">
<input type="text" class="form-control" id="inputGeneratePasswordOutput">
</div>
</div>
<div class="row">
<div class="col-sm-8 col-sm-offset-4">
<button type="submit" class="btn btn-check-all btn-sm">
<i class="fas fa-plus fa-fw"></i>
Generate new password
</button>
<button type="button" class="btn btn-default btn-sm copy-to-clipboard" data-clipboard-target="#inputGeneratePasswordOutput">
<img src="/assets/img/clippy.svg" alt="Copy to clipboard" width="15">
Copy
</button>
</div>
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn outline-btn" data-dismiss="modal">
Close
</button>
<button type="button" class="btn primary-solid-btn" id="btnGeneratePasswordInsert" data-clipboard-target="#inputGeneratePasswordOutput">
Copy to clipboard & Insert
</button>
</div>
</div>
</div>
</div>
</form>
<script src="/templates/kohost-professional/js/custom.js?v=ab8c2e"></script>
</body>
</html>
Sitemaps discovered
No sitemap declared in robots.txt and the common paths returned no 200.
About Robots + Sitemap
This tool fetches robots.txt from the host and displays it as-is. It also extracts every Sitemap: directive from the file and probes the two standard sitemap paths (/sitemap.xml and /sitemap_index.xml) to find any sitemaps that exist but were not declared. The result gives you a complete picture of what the site invites crawlers to read and where to find the structured URL inventory.
When to use it
Use this when migrating a site to confirm the new server is serving the right crawl rules. SEO professionals check it to verify they have not accidentally disallowed important paths. Bug bounty researchers read it for hints about admin paths or staging endpoints that are listed only here. Webmasters use it to discover where competitor sitemaps live, useful for content strategy and indexable URL counts.
How to read the results
Disallow lines tell crawlers what not to visit. A bare Disallow: with no path means crawl everything. Allow: overrides Disallow: for specific paths. User-agent: * applies to all crawlers, named agents take their own rules. Sitemap: lines list authoritative sitemap locations. The crawl-delay directive (some crawlers ignore it) suggests a minimum gap between requests in seconds.
Frequently asked questions
Does robots.txt enforce access control? ▾
No. It is a polite request, not security. Well-behaved crawlers like Google and Bing respect it. Malicious crawlers and scrapers ignore it freely. For real access control, use authentication, firewall rules, or rate limits.
Why does the tool show sitemaps that were not in robots.txt? ▾
Many sites publish sitemaps at the conventional /sitemap.xml path without declaring them in robots.txt. Search engines check this path automatically. The tool probes these standard locations and reports any that return a 200 response.
Should I block AI crawlers in robots.txt? ▾
That is a policy choice. Many publishers now block GPTBot, ClaudeBot, Google-Extended, and similar named user agents. Be aware that some crawlers ignore robots.txt, and blocking does not prevent content from appearing in training sets assembled before the block.
How big can robots.txt be? ▾
Google reads up to 500 KiB. Other crawlers may have different limits. Files larger than that get truncated, which can lead to unpredictable interpretation. Keep robots.txt focused on directives that matter and offload large URL lists to sitemaps.
Try with the same input