Robots + Sitemap
robots.txt + discovered sitemap URLs.
robots.txt + sitemaps for intertune.com
robots.txt
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="">
<meta name="author" content="Coderthemes">
<!-- Favicon icon -->
<link rel="shortcut icon" href="images/favicon.png">
<!-- Site Title -->
<title>Intertune - The cloud that runs your business</title>
<!-- Google fonts -->
<link href='https://fonts.googleapis.com/css?family=Roboto:400,300,500,700' rel='stylesheet' type='text/css'>
<link href='https://fonts.googleapis.com/css?family=Montserrat:400,700' rel='stylesheet' type='text/css'>
<!-- Bootstrap core CSS -->
<link href="css/bootstrap.min.css" rel="stylesheet">
<!-- Animate -->
<link href="css/animate.css" rel="stylesheet">
<!-- Icon-font -->
<link rel="stylesheet" type="text/css" href="css/material-design-iconic-font.min.css">
<!-- Custom styles for this template -->
<link href="css/style.css" rel="stylesheet">
<!-- Color css -->
<link href="css/colors/default.css" rel="stylesheet">
<script src="//scripts.iconnode.com/115039.js"></script>
<!-- HTML5 shim and Respond.js IE8 support of HTML5 tooltipss and media queries -->
<!--[if lt IE 9]>
<script src="js/html5shiv.js"></script>
<script src="js/respond.min.js"></script>
<![endif]-->
<!-- Global site tag (gtag.js) - Google Analytics -->
<!-- gtag('config', 'UA-57962765-6'); old google id -->
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-3330888-1"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-3330888-1');
gtag('config', 'AW-432417745');
</script>
<!-- Facebook Pixel Code -->
<script>
!function(f,b,e,v,n,t,s)
{if(f.fbq)return;n=f.fbq=function(){n.callMethod?
n.callMethod.apply(n,arguments):n.queue.push(arguments)};
if(!f._fbq)f._fbq=n;n.push=n;n.loaded=!0;n.version='2.0';
n.queue=[];t=b.createElement(e);t.async=!0;
t.src=v;s=b.getElementsByTagName(e)[0];
s.parentNode.insertBefore(t,s)}(window, document,'script',
'https://connect.facebook.net/en_US/fbevents.js');
fbq('init', '1619912328193757');
fbq('track', 'PageView');
</script>
<noscript><img height="1" width="1" style="display:none"
src="https://www.facebook.com/tr?id=1619912328193757&ev=PageView&noscript=1"
/></noscript>
<!-- End Facebook Pixel Code -->
</head>
<body data-spy="scroll" data-target="#navbar-menu">
<!-- <script>
function initFreshChat() {
window.fcWidget.init({
token: "9022a178-e024-4e68-9840-459c85508a48",
host: "https://wchat.freshchat.com"
});
}
function initialize(i,t){var e;i.getElementById(t)?initFreshChat():((e=i.createElement("script")).id=t,e.async=!0,e.src="https://wchat.freshchat.com/js/widget.js",e.onload=initFreshChat,i.head.appendChild(e))}function initiateCall(){initialize(document,"freshchat-js-sdk")}window.addEventListener?window.addEventListener("load",initiateCall,!1):window.attachEvent("load",initiateCall,!1);
</script> -->
<div class="navbar navbar-custom navbar-fixed-top sticky" role="navigation">
<div class="container">
<!-- Navbar-header -->
<div class="navbar-header">
<!-- Responsive menu button -->
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
<i class="zmdi zmdi-menu"></i>
</button>
<!-- LOGO -->
<a class="navbar-brand logo" href="index.html">
<img src="images/intertune_200.png" alt="Home">
</a>
</div>
<!-- end navbar-header -->
<!-- menu -->
<div class="navbar-collapse collapse" id="navbar-menu">
<!-- Navbar right -->
<ul class="nav navbar-nav navbar-right">
<li>
<a href="https://cloud.intertune.com/angularjs/#/login">Login</a>
<!-- <ul class="dropdown-menu arrow">
<li><a href="https://cloud.intertune.com/angularjs/#/login">Backup MX</a></li>
<li><a href="https://cloud.intertune.com/angularjs/#/login">ClearMX</a></li>
<li><a href="https://admin.payperhost.com/en/login/index/">PayPerHost</a></li>
</ul> -->
</li>
<li>
<a href="https://cloud.intertune.com/angularjs/#/signUp?service=hosting" class="btn btn-inverse btn-bordered navbar-btn">Sign Up</a>
<!-- <ul class="dropdown-menu arrow">
<li><a href="https://cloud.intertune.com/angularjs/#/signUp?service=backupmx">Backup MX</a></li>
<li><a href="https://cloud.intertune.com/angularjs/#/signUp?service=filter">ClearMX</a></li>
<li><a href="https://www.payperhost.com/en/web-hosting">PayPerHost</a></li>
</ul> -->
</li>
<!-- <li><a href="#" onclick="window.fcWidget.open();window.fcWidget.show();">24/7 Chat Support</a><script>window.fcWidget.init({ token: "9022a178-e024-4e68-9840-459c85508a484", host: "https://wchat.freshchat.com"});window.fcWidget.hide();</script></li> -->
</ul>
</div>
<!--/Menu -->
</div>
<!-- end container -->
</div>
<!-- End navbar-custom -->
<!-- HOME -->
<section class="home" id="home">
<!-- <div class="bg-overlay"></div> -->
<div class="container">
<div class="row">
<div class="col-sm-12">
<div class="home-wrapper text-center">
<div class="clearfix"></div>
</div><!-- home wrapper -->
</div> <!-- end col -->
</div> <!-- end row -->
</div> <!-- end container -->
</section>
<!-- END HOME -->
<!-- FEATURES -->
<section class="section" id="features">
<div class="container">
<div class="row">
<div class="col-sm-12 text-center">
<div class="title-box">
<!-- <p class="title-alt">Services</p> -->
<h3 class="fadeIn animated wow" data-wow-delay=".1s">Our Cloud Services</h3>
<div class="border"></div>
</div>
</div>
</div> <!-- end row -->
<div class="row text-center">
<div class="col-sm-4" style="width:100%;">
<div class="service-item animated fadeInDown wow" data-wow-delay=".1s">
<a href="https://www.sendmetric.com">
<img src="images/sendmetric_logo.png" width="250" height="65" alt="img">
</a>
<div class="service-detail">
<h4>Outbound SMTP Filter</h4>
<div style="margin-left:25%;margin-right:20%;text-align:left;">
Sendmetric provides an outbound smarthost built for web hosting companies, ISP's & organizations with advanced spam filtering to stop spam outbreaks from your users.
</div>
</div> <!-- /service-detail -->
</div> <!-- /service-item -->
</div> <!-- /col -->
<div class="col-sm-4" style="width:100%;">
<div class="service-item animated fadeInDown wow" data-wow-delay=".1s">
<a href="http://www.backupmxservice.com">
<img src="images/backupmx.png" width="250" alt="img">
</a>
<div class="service-detail">
<h4>Email Disaster Recovery</h4>
<div style="margin-left:25%;margin-right:20%;text-align:left;">
The insurance policy for email at a small annual fee. Never lose an email due to an outage!
</div>
</div> <!-- /service-detail -->
</div> <!-- /service-item -->
</div> <!-- /col -->
<div class="col-sm-4" style="width:100%;">
<div class="service-item animated fadeInLeft wow" data-wow-delay=".3s">
<!-- <a href="https://www.payperhost.com">
<img src="images/payperhost.png" width="250" alt="img">
</a> -->
<div class="service-detail">
<h4><a href="web_hosting.html">Web Hosting</a></h4>
<div style="margin-left:25%;margin-right:20%;text-align:left;">
We have provided web hosting services for almost 2 decades. Our motto is simple. We acquire hosting businesses from legacy providers and set them up on our intuitive, easy-to-use control panel running on infrastructure from the leading cloud hosting providers worldwide.
</div>
</div> <!-- /service-detail -->
</div> <!-- /service-item -->
</div> <!-- /col -->
<!-- <div class="col-sm-4" style="width:100%;">
<div class="service-item animated fadeInLeft wow" data-wow-delay=".2s">
<a href="http://www.clearmx.com">
<img src="images/clearmx.png" width="250" alt="img">
</a>
<div class="service-detail">
<h4>Email Security</h4>
<div style="margin-left:25%;margin-right:20%;text-align:left;">
Barracuda powered email filtering hosted in the Amazon cloud. All without any per user fees.
</div>
</div>
</div>
</div> -->
<!-- <div class="col-sm-4" style="width:100%;">
<div class="service-item animated fadeInLeft wow" data-wow-delay=".3s">
<a href="https://facilityvision.com">
<img src="images/facilityvision.png" alt="img">
</a>
<div class="service-detail">
<h4>CMMS</h4>
<div style="margin-left:25%;margin-right:20%;text-align:left;">
Track work orders, locations and assets for a wide array of industries requiring a computerized maintenance manangement system.
</div>
</div> <!-- /service-detail -->
</div> <!-- /service-item -->
</div> <!-- /col -->
</div> <!--end row -->
</div> <!-- end container -->
</section>
<!-- END FEATURES -->
<!-- FOOTER -->
<footer class="section bg-gray footer" style="padding-top:0px;">
<div class="container">
<div class="row">
<div class="col-md-3 col-sm-6">
<h5>Intertune</h5>
<ul class="list-unstyled">
<li><a href="index.html">Home</a></li>
<li><a href="company.html">About Us</a></li>
<li><a href="https://www.sendmetric.com">Outbound Email Filter</a></li>
<li><a href="web_hosting.html">Web Hosting</a></li>
<li><a href="http://www.backupmxservice.com">Backup MX</a></li>
</ul>
</div>
<!-- <div class="col-md-3 col-sm-6">
<h5>Social</h5>
<ul class="list-unstyled">
<li><a href="https://www.facebook.com/intertune/">Facebook</a></li>
<li><a href="https://twitter.com/intertune/">Twitter</a></li>
</ul>
</div> -->
<div class="col-md-3 col-sm-6">
<h5>Support</h5>
<ul class="list-unstyled">
<li><a href="http://help.intertune.com/">Help & Support</a></li>
<li><a href="https://www.site24x7.eu/sv.do?id=6CRZz1hrhcqtRm8ofc%2B0s1NgEXtCItSrvvpSk3sTnZaVCocyYEts15%2BxEW7Uj0MsFQVokuI31Uhy%0AuStf3YtrP4olisBMHhskwsbLwOEC1%2Fc%3D">Network Status</a></li>
<li><a href="privacy.html">Privacy Policy</a></li>
<li><a href="http://www.intertune.com/web_hosting_agreement.html">Web Hosting Terms</a></li>
</ul>
</div>
<div class="col-md-3 col-sm-6">
<h5>Contact</h5>
<address>
<!-- 5663 Balboa Avenue, Suite 379<br>
San Diego, California 92111<br>
<abbr title="Phone">U.S:</abbr> 855.TUNE.468<br/>
<abbr title="Phone">U.K:</abbr> +44.2038.6TUNE9<br/> -->
<a href="mailto:help@intertune.com">help@intertune.com</a><br />
<!-- <a href="#" onclick="window.fcWidget.open();window.fcWidget.show();">24/7 Live Chat</a> -->
</address>
</div>
</div>
<div class="row">
<div class="col-sm-12">
<div class="footer-alt">
<p class="pull-right text-muted m-b-0">2026 © Intertune, Inc.</p>
<a class="navbar-brand logo" href="index.html">
<!-- <img src="images/intertune_200.png" width="100" height="31" border="0"> -->
</a>
</div>
</div>
</div>
</div>
</footer>
<!-- END FOOTER -->
<!-- Back to top -->
<!-- <a href="#" class="back-to-top"> <i class="zmdi zmdi-chevron-up"> </i> </a> -->
<!-- js placed at the end of the document so the pages load faster -->
<script src="js/jquery.js"></script>
<script src="js/bootstrap.min.js"></script>
<!-- Jquery easing -->
<script type="text/javascript" src="js/jquery.easing.1.3.min.js"></script>
<script src="js/wow.min.js"></script>
<!--sticky header-->
<script type="text/javascript" src="js/jquery.sticky.js"></script>
<!--common script for all pages-->
<script src="js/jquery.app.js"></script>
<script type="text/javascript">
_linkedin_partner_id = "8125820";
window._linkedin_data_partner_ids = window._linkedin_data_partner_ids || [];
window._linkedin_data_partner_ids.push(_linkedin_partner_id);
</script>
<script type="text/javascript">
(function(l) {
if (!l){window.lintrk = function(a,b){window.lintrk.q.push([a,b])};
window.lintrk.q=[]}
var s = document.getElementsByTagName("script")[0];
var b = document.createElement("script");
b.type = "text/javascript";b.async = true;
b.src = "https://snap.licdn.com/li.lms-analytics/insight.min.js";
s.parentNode.insertBefore(b, s);})(window.lintrk);
</script>
<noscript>
<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=8125820&fmt=gif" />
</noscript>
</body>
</html>
Sitemaps discovered
No sitemap declared in robots.txt and the common paths returned no 200.
About Robots + Sitemap
This tool fetches robots.txt from the host and displays it as-is. It also extracts every Sitemap: directive from the file and probes the two standard sitemap paths (/sitemap.xml and /sitemap_index.xml) to find any sitemaps that exist but were not declared. The result gives you a complete picture of what the site invites crawlers to read and where to find the structured URL inventory.
When to use it
Use this when migrating a site to confirm the new server is serving the right crawl rules. SEO professionals check it to verify they have not accidentally disallowed important paths. Bug bounty researchers read it for hints about admin paths or staging endpoints that are listed only here. Webmasters use it to discover where competitor sitemaps live, useful for content strategy and indexable URL counts.
How to read the results
Disallow lines tell crawlers what not to visit. A bare Disallow: with no path means crawl everything. Allow: overrides Disallow: for specific paths. User-agent: * applies to all crawlers, named agents take their own rules. Sitemap: lines list authoritative sitemap locations. The crawl-delay directive (some crawlers ignore it) suggests a minimum gap between requests in seconds.
Frequently asked questions
Does robots.txt enforce access control? ▾
No. It is a polite request, not security. Well-behaved crawlers like Google and Bing respect it. Malicious crawlers and scrapers ignore it freely. For real access control, use authentication, firewall rules, or rate limits.
Why does the tool show sitemaps that were not in robots.txt? ▾
Many sites publish sitemaps at the conventional /sitemap.xml path without declaring them in robots.txt. Search engines check this path automatically. The tool probes these standard locations and reports any that return a 200 response.
Should I block AI crawlers in robots.txt? ▾
That is a policy choice. Many publishers now block GPTBot, ClaudeBot, Google-Extended, and similar named user agents. Be aware that some crawlers ignore robots.txt, and blocking does not prevent content from appearing in training sets assembled before the block.
How big can robots.txt be? ▾
Google reads up to 500 KiB. Other crawlers may have different limits. Files larger than that get truncated, which can lead to unpredictable interpretation. Keep robots.txt focused on directives that matter and offload large URL lists to sitemaps.