Free utility · No signup · Plain results
/tools/robots
Robots + Sitemap
robots.txt + discovered sitemap URLs.
robots.txt + sitemaps for medianet.mv
robots.txt
<!doctype html>
<html lang="en">
<head>
<title>Medianet</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="author" content="Medianet">
<meta name="description" content="Official website of Medianet" />
<meta itemprop="name" content="Medianet">
<meta itemprop="description" content="Official website of Medianet">
<meta itemprop="image" content="https://medianet.mv/share.jpg">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:site" content="@medianetmv">
<meta name="twitter:title" content="Medianet">
<meta name="twitter:description" content="Official website of Medianet">
<meta name="twitter:creator" content="@medianetmv">
<meta name="twitter:image:src" content="https://medianet.mv/share.jpg">
<meta property="og:title" content="Medianet" />
<meta property="og:type" content="website" />
<meta property="og:url" content="Medianet" />
<meta property="og:image" content="https://medianet.mv/share.jpg" />
<meta property="og:description" content="Official website of Medianet" />
<meta property="og:site_name" content="Medianet" />
<script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-W332JG5');</script>
<script src="https://www.google.com/recaptcha/api.js"></script>
<link rel="icon" href="/img/favicon.png" type="image/png">
<link rel="stylesheet" href="/css/bootstrap.min.css">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.2.0/css/all.css">
<link href="https://fonts.googleapis.com/css?family=Nunito:400,700,900&display=swap" rel="stylesheet">
<link rel="stylesheet" href="/css/meanmenu.css">
<link rel="stylesheet" href="/css/magnific-popup.min.css">
<link rel="stylesheet" href="/css/aos.min.css">
<link rel="stylesheet" href="/css/slick.css">
<link rel="stylesheet" href="/style.css?d=20240707041906">
<link rel="stylesheet" href="/css/responsive.css?d=20240707041906">
</head>
<body>
<!-- Google Tag Manager (noscript) -->
<noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-W332JG5" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>
<!-- End Google Tag Manager (noscript) -->
<!--[if lt IE 8]>
<p class="browserupgrade">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p>
<![endif]-->
<!-- Preloader Start ========================-->
<div id='preloader'>
<div id='status'>
<img src='/img/loading.gif' alt='LOADING....!!!!!' />
</div>
</div>
<!--=========== Main Header area ===============-->
<header id="home">
<!-- Personal Business About Menu -->
<nav class="top-navigation navbar navbar-expand bg-dark navbar-dark">
<div class="container">
<ul class="navbar-nav">
<li class="nav-item active">
<a class="nav-link" href="/">Personal</a>
</li>
<li class="nav-item ">
<a class="nav-link" href="/business">Business</a>
</li>
<li class="nav-item ">
<a class="nav-link" href="/aboutus">About us</a>
</li>
</ul>
</div>
</nav>
<div class="main-navigation-1">
<div class="container">
<div class="row">
<!-- logo-area-->
<div class="col-xl-2 col-lg-3 col-md-3">
<div class="logo-area">
<a href="/"><img src="/img/medianet_logo.png" alt="MediaNet"></a>
</div>
</div>
<!-- mainmenu-area-->
<div class="col-xl-10 col-lg-9 col-md-9">
<div class="main-menu f-right">
<nav id="mobile-menu">
<ul>
<li>
<a href="/">Home</a>
</li>
<li>
<a href="/products_services">Products</a>
</li>
<li>
<a href="#">Services <i class="fas fa-angle-down"></i></a>
<ul class="dropdown">
<li>
<a href="/vas/medianet_tv">Medianet TV</a>
</li>
</ul>
</li>
<!-- dropdown menu-area-->
<li>
<a href="#">more <i class="fas fa-angle-down"></i></a>
<ul class="dropdown">
<li><a href="/channels">Our Channels</a></li>
<li><a href="/blog">Blog</a></li>
<li><a href="/vas/find_your_ms_dealer">Find Your MS Dealer</a></li>
<li><a href="/shop">merchandise</a></li>
<li><a href="/live">Live Events</a></li>
<li><a href="/epg">TV Guide</a></li>
</ul>
</li>
</ul>
</nav>
</div>
<!-- mobile menu-->
<div class="mobile-menu"></div>
<!--Search-->
<div class="search-box-area">
<!-- <div class="search-icon-area">
<a href='https://my.medianet.mv/'>
<img src="/img/quickpay.png" alt="">
</a>
</div> -->
<!--div id="search" class="fade">
<a href="#" class="close-btn" id="close-search">
<em class="fa fa-times"></em>
</a>
<-- <input placeholder="what are you looking for?" id="searchbox" type="search" /> ->
<div class="container">
<div class="row">
<div class="col-sm" style="text-align: center;">
<a href="https://mymedianet.mv/quickpay"><img src="img/payment/mquickpay.png" alt="" width="250px" /></a>
</div>
<div class="col-sm" style="text-align: center;">
<a href="http://ore.do/mm"><img src="img/payment/m-faisaa.png" alt="" width="250px" /></a>
</div>
</div>
</div>
</!--div -->
<div class="d-none d-lg-block top-buttons-area section-button ">
<!-- <a href='#search'>
<img src="img/quickpay.png" style="width: 50px;">
</a> -->
<a href="https://register.medianet.mv" target="_blank" tabindex="0">New Connection</a>
<a href="/contact" tabindex="1" target="_blank">Get Help</a>
<a href="https://mymedianet.mv/quickpay" tabindex="2" target="_blank">Quick Pay</a>
</div>
</div>
</div>
</div>
</div>
</div>
</header>
<div class="error-page-area">
<div class="container">
<div class="row">
<!--404 text image-->
<div class="col-xl-12">
<div class="error-text">
<!-- <img src="img/404.png" alt=""> -->
<script src="https://unpkg.com/@lottiefiles/lottie-player@latest/dist/lottie-player.js"></script>
<lottie-player src="/img/404.json" background="transparent" speed="1" style="width: 300px; height: 300px;" loop autoplay>
</lottie-player>
<h2>sorry we can't find anything</h2>
<a href="/">back to home</a>
</div>
</div>
</div>
</div>
</div>
<section>
<div class="home-blog-area">
<div class="container">
<div class="row">
<!-- blog heading text-->
<div class="col-xl-12">
<div class="section-heading-3">
<h3>Latest Blogs</h3>
</div>
</div>
</div>
<div class="row">
<!-- single blog-->
<div class="col-xl-4 col-lg-4 col-md-4 col-sm-6">
<div class="home-single-blog">
<a href="/article/medianet_launches_dedicated_fifa_world_cup_2026_website">
<div class="s-blog-image">
<img src="m.gif" class="lazy" data-src="/media/block/04fc00be553793f22ec35450f03076b6x1.jpg" alt="">
<div class="blog-post-date">
<span>01 Jun</span>
</div>
</div>
<div class="s-blog-content">
<h4>Medianet Launches Dedicated FIFA World Cup 2026 Website</h4>
</div>
</a>
</div>
</div>
<div class="col-xl-4 col-lg-4 col-md-4 col-sm-6">
<div class="home-single-blog">
<a href="/article/medianet_eid_offer_">
<div class="s-blog-image">
<img src="m.gif" class="lazy" data-src="/media/block/c4299d89c95c0ae215f22d3de9bcfb03x1.jpg" alt="">
<div class="blog-post-date">
<span>09 May</span>
</div>
</div>
<div class="s-blog-content">
<h4>Medianet Eid Offer! </h4>
</div>
</a>
</div>
</div>
<div class="col-xl-4 col-lg-4 col-md-4 col-sm-6">
<div class="home-single-blog">
<a href="/article/pl4k_pass_now_just_mvr199_for_the_rest_of_the_season">
<div class="s-blog-image">
<img src="m.gif" class="lazy" data-src="/media/block/5cd8fec3cb3a531d48c660129b67754ex1.jpg" alt="">
<div class="blog-post-date">
<span>03 May</span>
</div>
</div>
<div class="s-blog-content">
<h4>PL4K Pass - Now Just MVR199 for the Rest of the Season</h4>
</div>
</a>
</div>
</div>
</div>
</div>
</div>
</section>
<div class="blog-share expertise-area">
<h4>Share It on</h4>
<ul>
<li><a class="footer-socials share" id="facebook" href="#"><i class="fab fa-facebook"></i></a></li>
<li><a class="footer-socials share" id="twitter" href="#"><i class="fab fa-twitter"></i></a></li>
<li><a class="footer-socials share" id="whatsapp" href="#"><i class="fab fa-whatsapp"></i></a></li>
<li><a class="footer-socials share" id="telegram" href="#"><i class="fab fa-telegram"></i></a></li>
<li><a class="footer-socials share" id="viber" href="#"><i class="fab fa-viber"></i></a></li>
</ul>
</div>
<div class="callto-action">
<div class="container">
<div class="row">
<div class="col-xl-8 col-lg-8 col-md-8 col-sm-7">
<div class="callto-action-text">
<h5>Any thoughts on our services? Your feedback will help us improve and <span>serve you better.</span> </h5>
</div>
</div>
<div class="col-xl-4 col-lg-4 col-md-4 col-sm-5">
<div class="callto-action-btn">
<a href="/feedback">Feedback</a>
</div>
</div>
</div>
</div>
</div>
<section id="footer-fixed">
<div class="big-footer">
<div class="container">
<div class="row">
<!--footer logo-->
<div class="col-xl-3 col-lg-3 col-md-3 col-sm-6">
<div class="footer-logo">
<a href="#">
<img src="/img/medianet_logo_color.png" alt="">
</a>
<p> Your World of Entertainment.</p>
<p>Tel: (960) 332 0800<br> Fax: (960) 331 6567</p>
</div>
<!--footer social-->
<div class="social">
<ul>
<li><a class="footer-socials" href="https://www.facebook.com/medianetmv"><i class="fab fa-facebook"></i></a></li>
<li><a class="footer-socials" href="https://www.instagram.com/medianetmv"><i class="fab fa-instagram"></i></a></li>
<li><a class="footer-socials" href="https://www.twitter.com/medianetmv"><i class="fab fa-twitter"></i></a></li>
<li><a class="footer-socials" href="https://www.youtube.com/medianetdigital"><i class="fab fa-youtube"></i></a></li>
</ul>
</div> </div>
<!--footer quick links-->
<div class="col-xl-2 col-lg-2 col-md-2 col-sm-4">
<div class="footer-heading">
<h3>quick links</h3>
</div>
<div class="footer-content">
<ul>
<li><a href="/contact">contact</a></li>
<li><a href="/downloads">downloads</a></li>
<li><a href="/faq">FAQs</a></li>
<li><a href="/news">news</a></li>
<li><a href="/careers">careers</a></li>
</ul>
</div>
</div>
<!--footer latest work-->
<div class="col-xl-4 col-lg-4 col-md-4 col-sm-8">
<div class="footer-heading">
<h3>News & Events</h3>
</div>
<div class="footer-content">
<ul>
<li><a href="/article/public_announcement_regarding_fifa_world_cup_2026_">Public announcement regarding FIFA World Cup 2026™</a></li><li><a href="/article/terms_and_conditions_medianet_sofa_inn_dhandah_promotion">Terms and Conditions - Medianet Sofa inn Dhandah Promotion</a></li><li><a href="/article/terms_and_conditions_for_medianet_fpl_2024_25">Terms and Conditions for Medianet FPL 2024-25</a></li><li><a href="/article/announcement">Announcement</a></li><li><a href="/article/announcement_scheduled_maintenance_in_quickpay_portal">Announcement: Scheduled maintenance in Quickpay portal</a></li> </ul>
</div>
</div>
<!--footer subscribe-->
<div class="col-xl-3 col-lg-3 col-md-3 col-sm-6">
<div class="footer-heading">
<h3>Get Updates</h3>
</div>
<div class="footer-content footer-cont-mar-40">
<form action="https://medianet.us4.list-manage.com/subscribe/post?u=a3c9d40612519fbe3c2faaba4&id=c58f357533" method="post" id="mc-embedded-subscribe-form" name="mc-embedded-subscribe-form" class="validate" target="_blank" novalidate>
<!-- <input id="leadgenaration" type="email" placeholder="Enter your email"> -->
<input type="email" value="" name="EMAIL" class="email" id="leadgenaration" placeholder="email address" required>
<div style="position: absolute; left: -5000px;" aria-hidden="true"><input type="text" name="b_a3c9d40612519fbe3c2faaba4_c58f357533" tabindex="-1" value=""></div>
<!-- <input id="subscribe" type="submit" value="Subscribe"> -->
<div class="clear"><input type="submit" value="Subscribe" name="subscribe" id="subscribe" class="button"></div>
</form>
</div>
</div>
</div>
</div>
</div>
<footer>
<p>Copyright © 2026 Medianet, Maldives. All rights reserved.</p>
</footer>
<!-- Floating Add Button -->
<div class="fab-container d-lg-none">
<div class="fab shadow">
<div class="fab-content">
<i class="fa fa-plus"></i>
</div>
</div>
<div class="sub-button shadow">
<a href="https://register.medianet.mv" target="_blank">New Connection</a>
</div>
<div class="sub-button shadow">
<a href="/contact" target="_blank">
Get Help
</a>
</div>
<div class="sub-button shadow">
<a href="https://mymedianet.mv/quickpay" target="_blank">
Quick Pay
</a>
</div>
</div>
<!-- End Floating Add Button -->
</section>
<!-- Load Facebook SDK for JavaScript -->
<div id="fb-root"></div>
<script>
window.fbAsyncInit = function() {
FB.init({
xfbml : true,
version : 'v5.0'
});
};
(function(d, s, id) {
var js, fjs = d.getElementsByTagName(s)[0];
if (d.getElementById(id)) return;
js = d.createElement(s); js.id = id;
js.src = 'https://connect.facebook.net/en_US/sdk/xfbml.customerchat.js';
fjs.parentNode.insertBefore(js, fjs);
}(document, 'script', 'facebook-jssdk'));
</script>
<!-- Your customer chat code -->
<div class="fb-customerchat" attribution="install_email" page_id="115565131797726"></div>
<!-- Jquery JS -->
<script src="/js/vendor/jquery-2.2.4.min.js"></script>
<!-- Proper JS -->
<script src="/js/popper.min.js"></script>
<!-- Bootstrap Js -->
<script src="/js/bootstrap.min.js"></script>
<!-- Video popup Js -->
<script src="/js/magnific-popup.min.js"></script>
<!-- Waypoint Up Js -->
<script src="/js/waypoints.min.js"></script>
<!-- Counter Up Js -->
<script src="/js/counterup.min.js"></script>
<!-- Meanmenu Js -->
<script src="/js/meanmenu.min.js"></script>
<!-- Animation Js -->
<script src="/js/aos.min.js"></script>
<!-- Filtering Js -->
<script src="/js/isotope.min.js"></script>
<!-- Background Move Js -->
<script src="/js/jquery.backgroundMove.js"></script>
<!-- Slick Carousel Js -->
<script src="/js/slick.min.js"></script>
<!-- Main Js -->
<script src="/js/public.js?d=20240303044222"></script>
<script>
$(document).ready(function(){
$('.sliders').slick({
prevArrow: '<button type="button" class="slick-prev"><i class="fas fa-long-arrow-alt-left"></i></button>',
nextArrow: '<button type="button" class="slick-next"><i class="fas fa-long-arrow-alt-right"></i></button>',
autoplay:true,
loop: true,
dots: true
});
});
</script>
</body>
</html>
Sitemaps discovered
About Robots + Sitemap
This tool fetches robots.txt from the host and displays it as-is. It also extracts every Sitemap: directive from the file and probes the two standard sitemap paths (/sitemap.xml and /sitemap_index.xml) to find any sitemaps that exist but were not declared. The result gives you a complete picture of what the site invites crawlers to read and where to find the structured URL inventory.
When to use it
Use this when migrating a site to confirm the new server is serving the right crawl rules. SEO professionals check it to verify they have not accidentally disallowed important paths. Bug bounty researchers read it for hints about admin paths or staging endpoints that are listed only here. Webmasters use it to discover where competitor sitemaps live, useful for content strategy and indexable URL counts.
How to read the results
Disallow lines tell crawlers what not to visit. A bare Disallow: with no path means crawl everything. Allow: overrides Disallow: for specific paths. User-agent: * applies to all crawlers, named agents take their own rules. Sitemap: lines list authoritative sitemap locations. The crawl-delay directive (some crawlers ignore it) suggests a minimum gap between requests in seconds.
Frequently asked questions
Does robots.txt enforce access control? ▾
No. It is a polite request, not security. Well-behaved crawlers like Google and Bing respect it. Malicious crawlers and scrapers ignore it freely. For real access control, use authentication, firewall rules, or rate limits.
Why does the tool show sitemaps that were not in robots.txt? ▾
Many sites publish sitemaps at the conventional /sitemap.xml path without declaring them in robots.txt. Search engines check this path automatically. The tool probes these standard locations and reports any that return a 200 response.
Should I block AI crawlers in robots.txt? ▾
That is a policy choice. Many publishers now block GPTBot, ClaudeBot, Google-Extended, and similar named user agents. Be aware that some crawlers ignore robots.txt, and blocking does not prevent content from appearing in training sets assembled before the block.
How big can robots.txt be? ▾
Google reads up to 500 KiB. Other crawlers may have different limits. Files larger than that get truncated, which can lead to unpredictable interpretation. Keep robots.txt focused on directives that matter and offload large URL lists to sitemaps.
Try with the same input