Free utility · No signup · Plain results
/tools/robots
Robots + Sitemap
robots.txt + discovered sitemap URLs.
robots.txt + sitemaps for obosnett.no
robots.txt
<!DOCTYPE html><html id="__next_error__"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="preload" as="script" fetchPriority="low" href="/_next/static/chunks/0~7w28q-qk4b7.js"/><script src="/_next/static/chunks/0e2w8~_z-bygc.js" async=""></script><script src="/_next/static/chunks/0tloj~egz2.~v.js" async=""></script><script src="/_next/static/chunks/0~meb8-1l1ysv.js" async=""></script><script src="/_next/static/chunks/0-d15g6ij46ni.js" async=""></script><script src="/_next/static/chunks/15guj0id6.za5.js" async=""></script><script src="/_next/static/chunks/turbopack-0m1rts~8~pnrc.js" async=""></script><script src="/_next/static/chunks/0l~a4ahuyd8s8.js" async=""></script><script src="/_next/static/chunks/0ddc71fsnmi.y.js" async=""></script><script src="/_next/static/chunks/0qnj4vy-qaqlu.js" async=""></script><link rel="preload" href="/_next/static/chunks/0rtg2s6aam-m..css" as="style"/><meta name="robots" content="noindex"/><title>Beklager, men vi finner ikke siden du leter etter | OBOS | OBOS</title><meta name="robots" content="index"/><link rel="canonical" href="https://www.obos.no/styrearbeid/nettrobots.txt"/><meta property="og:title" content="Beklager, men vi finner ikke siden du leter etter | OBOS | OBOS"/><meta property="og:site_name" content="OBOS"/><meta property="og:image" content="https://cdn.sanity.io/media-libraries/mln4u7f3Hc8r/images/containers/333ofPaSyUuTSNI8FVFSmBA76P9/obos-logo-socialmeta.jpg"/><meta property="og:image:width" content="1200"/><meta property="og:image:height" content="630"/><meta name="twitter:card" content="summary_large_image"/><meta name="twitter:title" content="Beklager, men vi finner ikke siden du leter etter | OBOS | OBOS"/><meta name="twitter:image" content="https://cdn.sanity.io/media-libraries/mln4u7f3Hc8r/images/containers/333ofPaSyUuTSNI8FVFSmBA76P9/obos-logo-socialmeta.jpg"/><meta name="twitter:image:width" content="1200"/><meta name="twitter:image:height" content="630"/><meta name="sentry-trace" content="0111d3b38c3cbeddd48902b3ec6eff69-3b256ba6e3d77a0e"/><meta name="baggage" content="sentry-environment=prod,sentry-public_key=bc683e1a1db73a9b8afd93b4f6768031,sentry-trace_id=0111d3b38c3cbeddd48902b3ec6eff69,sentry-org_id=4507446359097344"/><script src="/_next/static/chunks/03~yq9q893hmn.js" noModule=""></script></head><body><div hidden=""><!--$--><!--/$--></div><script src="/_next/static/chunks/0~7w28q-qk4b7.js" id="_R_" async=""></script><script>(self.__next_f=self.__next_f||[]).push([0])</script><script>self.__next_f.push([1,"1:\"$Sreact.fragment\"\n2:I[714448,[\"/_next/static/chunks/0l~a4ahuyd8s8.js\",\"/_next/static/chunks/0ddc71fsnmi.y.js\"],\"default\"]\n3:I[128696,[\"/_next/static/chunks/0l~a4ahuyd8s8.js\",\"/_next/static/chunks/0ddc71fsnmi.y.js\"],\"default\"]\n6:I[295895,[\"/_next/static/chunks/0l~a4ahuyd8s8.js\",\"/_next/static/chunks/0ddc71fsnmi.y.js\"],\"OutletBoundary\"]\n7:\"$Sreact.suspense\"\na:I[295895,[\"/_next/static/chunks/0l~a4ahuyd8s8.js\",\"/_next/static/chunks/0ddc71fsnmi.y.js\"],\"ViewportBoundary\"]\nc:I[295895,[\"/_next/static/chunks/0l~a4ahuyd8s8.js\",\"/_next/static/chunks/0ddc71fsnmi.y.js\"],\"MetadataBoundary\"]\ne:I[563491,[\"/_next/static/chunks/0qnj4vy-qaqlu.js\"],\"default\"]\n10:I[530283,[\"/_next/static/chunks/0qx3tgtdg.oqc.js\",\"/_next/static/chunks/0hni-~9dd455c.js\",\"/_next/static/chunks/0qt0cy3kue~w..js\",\"/_next/static/chunks/09~6ndh.3pwu_.js\",\"/_next/static/chunks/140v0syuvt69x.js\"],\"\"]\n11:I[571183,[\"/_next/static/chunks/0qx3tgtdg.oqc.js\",\"/_next/static/chunks/0hni-~9dd455c.js\",\"/_next/static/chunks/0qt0cy3kue~w..js\",\"/_next/static/chunks/09~6ndh.3pwu_.js\",\"/_next/static/chunks/140v0syuvt69x.js\"],\"A\"]\n12:I[169986,[\"/_next/static/chunks/0qx3tgtdg.oqc.js\",\"/_next/static/chunks/0hni-~9dd455c.js\",\"/_next/static/chunks/0qt0cy3kue~w..js\",\"/_next/static/chunks/09~6ndh.3pwu_.js\",\"/_next/static/chunks/140v0syuvt69x.js\"],\"O\"]\n13:I[925586,[\"/_next/static/chunks/0qx3tgtdg.oqc.js\",\"/_next/static/chunks/0hni-~9dd455c.js\",\"/_next/static/chunks/0qt0cy3kue~w..js\",\"/_next/static/chunks/09~6ndh.3pwu_.js\",\"/_next/static/chunks/140v0syuvt69x.js\"],\"isInternalLink\"]\n14:I[416587,[\"/_next/static/chunks/0qx3tgtdg.oqc.js\",\"/_next/static/chunks/0hni-~9dd455c.js\",\"/_next/static/chunks/0qt0cy3kue~w..js\",\"/_next/static/chunks/09~6ndh.3pwu_.js\",\"/_next/static/chunks/140v0syuvt69x.js\"],\"I18nClientProvider\"]\n15:I[852093,[\"/_next/static/chunks/0qx3tgtdg.oqc.js\",\"/_next/static/chunks/0hni-~9dd455c.js\",\"/_next/static/chunks/0qt0cy3kue~w..js\",\"/_next/static/chunks/09~6ndh.3pwu_.js\",\"/_next/static/chunks/140v0syuvt69x.js\"],\"ClientProviders\"]\n16:I[79425,[\"/_next/static/chunks/0qx3tgtdg.oqc.js\",\"/_next/static/chunks/0hni-~9dd455c.js\",\"/_next/static/chunks/0qt0cy3kue~w..js\",\"/_next/static/chunks/09~6ndh.3pwu_.js\",\"/_next/static/chunks/140v0syuvt69x.js\"],\"EventTracker\"]\n17:I[11316,[\"/_next/static/chunks/0qx3tgtdg.oqc.js\",\"/_next/static/chunks/0hni-~9dd455c.js\",\"/_next/static/chunks/0qt0cy3kue~w..js\",\"/_next/static/chunks/09~6ndh.3pwu_.js\",\"/_next/static/chunks/140v0syuvt69x.js\",\"/_next/static/chunks/0yd-o2d3w17gj.js\"],\"default\"]\n:HL[\"/_next/static/chunks/0rtg2s6aam-m..css\",\"style\"]\n:HL[\"/_next/static/chunks/0oujqcb.uvkjq.css\",\"style\"]\n:HL[\"https://www.obos.no/fonts/OBOSText-Regular.woff2\",\"font\",{\"crossOrigin\":\"\",\"type\":\"font/woff2\"}]\n:HL[\"https://www.obos.no/fonts/OBOSText-Medium.woff2\",\"font\",{\"crossOrigin\":\"\",\"type\":\"font/woff2\"}]\n:HL[\"https://www.obos.no/fonts/OBOSDisplay-SemiBold.woff2\",\"font\",{\"crossOrigin\":\"\",\"type\":\"font/woff2\"}]\n:Hm\"https://www.obos.no/appshell/app-shell-no.js\"\n"])</script><script>self.__next_f.push([1,"0:{\"P\":null,\"c\":[\"\",\"styrearbeid\",\"nettrobots.txt\"],\"q\":\"\",\"i\":false,\"f\":[[[\"\",{\"children\":[\"no\",{\"children\":[[\"slug\",\"styrearbeid/nettrobots.txt\",\"c\",null],{\"children\":[\"__PAGE__\",{}]}]},\"$undefined\",\"$undefined\",16]}],[[\"$\",\"$1\",\"c\",{\"children\":[null,[\"$\",\"$L2\",null,{\"parallelRouterKey\":\"children\",\"error\":\"$undefined\",\"errorStyles\":\"$undefined\",\"errorScripts\":\"$undefined\",\"template\":[\"$\",\"$L3\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":[[[\"$\",\"title\",null,{\"children\":\"404: This page could not be found.\"}],[\"$\",\"div\",null,{\"style\":{\"fontFamily\":\"system-ui,\\\"Segoe UI\\\",Roboto,Helvetica,Arial,sans-serif,\\\"Apple Color Emoji\\\",\\\"Segoe UI Emoji\\\"\",\"height\":\"100vh\",\"textAlign\":\"center\",\"display\":\"flex\",\"flexDirection\":\"column\",\"alignItems\":\"center\",\"justifyContent\":\"center\"},\"children\":[\"$\",\"div\",null,{\"children\":[[\"$\",\"style\",null,{\"dangerouslySetInnerHTML\":{\"__html\":\"body{color:#000;background:#fff;margin:0}.next-error-h1{border-right:1px solid rgba(0,0,0,.3)}@media (prefers-color-scheme:dark){body{color:#fff;background:#000}.next-error-h1{border-right:1px solid rgba(255,255,255,.3)}}\"}}],[\"$\",\"h1\",null,{\"className\":\"next-error-h1\",\"style\":{\"display\":\"inline-block\",\"margin\":\"0 20px 0 0\",\"padding\":\"0 23px 0 0\",\"fontSize\":24,\"fontWeight\":500,\"verticalAlign\":\"top\",\"lineHeight\":\"49px\"},\"children\":404}],[\"$\",\"div\",null,{\"style\":{\"display\":\"inline-block\"},\"children\":[\"$\",\"h2\",null,{\"style\":{\"fontSize\":14,\"fontWeight\":400,\"lineHeight\":\"49px\",\"margin\":0},\"children\":\"This page could not be found.\"}]}]]}]}]],[]],\"forbidden\":\"$undefined\",\"unauthorized\":\"$undefined\"}]]}],{\"children\":[[\"$\",\"$1\",\"c\",{\"children\":[[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/chunks/0rtg2s6aam-m..css\",\"precedence\":\"next\",\"crossOrigin\":\"$undefined\",\"nonce\":\"$undefined\"}],[\"$\",\"script\",\"script-0\",{\"src\":\"/_next/static/chunks/0qx3tgtdg.oqc.js\",\"async\":true,\"nonce\":\"$undefined\"}],[\"$\",\"script\",\"script-1\",{\"src\":\"/_next/static/chunks/0hni-~9dd455c.js\",\"async\":true,\"nonce\":\"$undefined\"}],[\"$\",\"script\",\"script-2\",{\"src\":\"/_next/static/chunks/0qt0cy3kue~w..js\",\"async\":true,\"nonce\":\"$undefined\"}],[\"$\",\"script\",\"script-3\",{\"src\":\"/_next/static/chunks/09~6ndh.3pwu_.js\",\"async\":true,\"nonce\":\"$undefined\"}],[\"$\",\"script\",\"script-4\",{\"src\":\"/_next/static/chunks/140v0syuvt69x.js\",\"async\":true,\"nonce\":\"$undefined\"}]],\"$L4\"]}],{\"children\":[[\"$\",\"$1\",\"c\",{\"children\":[null,[\"$\",\"$L2\",null,{\"parallelRouterKey\":\"children\",\"error\":\"$undefined\",\"errorStyles\":\"$undefined\",\"errorScripts\":\"$undefined\",\"template\":[\"$\",\"$L3\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":\"$undefined\",\"forbidden\":\"$undefined\",\"unauthorized\":\"$undefined\"}]]}],{\"children\":[[\"$\",\"$1\",\"c\",{\"children\":[\"$L5\",[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/chunks/0oujqcb.uvkjq.css\",\"precedence\":\"next\",\"crossOrigin\":\"$undefined\",\"nonce\":\"$undefined\"}],[\"$\",\"script\",\"script-0\",{\"src\":\"/_next/static/chunks/0gaxic1gg8if9.js\",\"async\":true,\"nonce\":\"$undefined\"}],[\"$\",\"script\",\"script-1\",{\"src\":\"/_next/static/chunks/0wti607wwxelz.js\",\"async\":true,\"nonce\":\"$undefined\"}]],[\"$\",\"$L6\",null,{\"children\":[\"$\",\"$7\",null,{\"name\":\"Next.MetadataOutlet\",\"children\":\"$@8\"}]}]]}],{},null,false,null]},null,false,\"$@9\"]},null,false,null]},null,false,\"$@9\"],[\"$\",\"$1\",\"h\",{\"children\":[null,[\"$\",\"$La\",null,{\"children\":\"$Lb\"}],[\"$\",\"div\",null,{\"hidden\":true,\"children\":[\"$\",\"$Lc\",null,{\"children\":[\"$\",\"$7\",null,{\"name\":\"Next.Metadata\",\"children\":\"$Ld\"}]}]}],null]}],false]],\"m\":\"$undefined\",\"G\":[\"$e\",[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/chunks/0rtg2s6aam-m..css\",\"precedence\":\"next\",\"crossOrigin\":\"$undefined\",\"nonce\":\"$undefined\"}]]],\"S\":false,\"h\":null,\"s\":\"$undefined\",\"l\":\"$undefined\",\"p\":\"$undefined\",\"d\":\"$undefined\",\"b\":\"jepEq6ctxlrSOaf2oXYpt\"}\n"])</script><script>self.__next_f.push([1,"f:[]\n9:\"$Wf\"\n"])</script><script>self.__next_f.push([1,"4:[\"$\",\"html\",null,{\"lang\":\"nb-NO\",\"className\":\"\",\"children\":[[\"$\",\"head\",null,{\"children\":[[\"$\",\"link\",null,{\"rel\":\"preconnect\",\"href\":\"https://res.cloudinary.com\"}],[\"$\",\"link\",null,{\"rel\":\"dns-prefetch\",\"href\":\"https://res.cloudinary.com\"}],[\"$\",\"meta\",null,{\"name\":\"theme-color\",\"content\":\"#ffffff\"}],[[\"$\",\"link\",null,{\"rel\":\"icon\",\"href\":\"/favicon.ico\",\"sizes\":\"32x32\"}],[\"$\",\"link\",null,{\"rel\":\"icon\",\"href\":\"/favicon.svg\",\"type\":\"image/svg+xml\"}],[\"$\",\"link\",null,{\"rel\":\"apple-touch-icon\",\"href\":\"/apple-touch-icon.png\"}],[\"$\",\"link\",null,{\"rel\":\"manifest\",\"href\":\"/manifest.webmanifest\"}]],[\"$\",\"$L10\",null,{\"type\":\"module\",\"src\":\"/ad-info.js\",\"strategy\":\"lazyOnload\"}],[\"$\",\"$L10\",null,{\"src\":\"https://www.obos.no/collector.js\",\"strategy\":\"afterInteractive\",\"type\":\"module\",\"id\":\"collector-script\",\"data-api-key\":\"4426822b50f2893bb0bc6cd74521848c\",\"data-app-name\":\"nettsted-frontend\",\"data-collect-url\":\"$undefined\",\"data-dangerously-ignore-consent\":\"$undefined\",\"data-session-replay-collect-url\":\"$undefined\",\"data-enable-session-replay\":true,\"crossOrigin\":\"\"}],[\"$\",\"link\",null,{\"rel\":\"preload\",\"href\":\"https://www.obos.no/fonts/OBOSText-Regular.woff2\",\"as\":\"font\",\"type\":\"font/woff2\",\"crossOrigin\":\"\"}],[\"$\",\"link\",null,{\"rel\":\"preload\",\"href\":\"https://www.obos.no/fonts/OBOSText-Medium.woff2\",\"as\":\"font\",\"type\":\"font/woff2\",\"crossOrigin\":\"\"}],[\"$\",\"link\",null,{\"rel\":\"preload\",\"href\":\"https://www.obos.no/fonts/OBOSDisplay-SemiBold.woff2\",\"as\":\"font\",\"type\":\"font/woff2\",\"crossOrigin\":\"\"}],[[\"$\",\"link\",null,{\"rel\":\"modulepreload\",\"href\":\"https://www.obos.no/appshell/app-shell-no.js\"}],[\"$\",\"script\",null,{\"id\":\"app-shell\",\"type\":\"module\",\"crossOrigin\":\"\",\"src\":\"https://www.obos.no/appshell/app-shell-no.js\"}],[\"$\",\"$L11\",null,{}]]]}],[\"$\",\"body\",null,{\"children\":[[\"$\",\"a\",null,{\"href\":\"#main\",\"className\":\"sr-only focus:not-sr-only\",\"children\":\"Hopp til innhold\"}],[\"$\",\"div\",null,{\"className\":\"page-layout\",\"children\":[[\"$\",\"$L12\",null,{\"isInternalLink\":\"$13\"}],[\"$\",\"main\",null,{\"className\":\"page-layout-main\",\"id\":\"main\",\"children\":[\"$\",\"$L14\",null,{\"locale\":\"nb\",\"messages\":{\"1yMmBK\":[\"Pris fra \u003c0/\u003e\"],\"2AFIS_\":[\"Jobb i OBOS\"],\"5nG4EH\":[\"Du er i forhåndsvisningsmodus. Trykk på dette banneret for å slå av forhåndsvisning\"],\"6lIXbJ\":[\"KI-generert bilde\"],\"8KfCRt\":[\"Relaterte artikler\"],\"8PkFM4\":[\"Lukk\"],\"8qFoqZ\":[\"Nye boliger \",[\"0\"]],\"A2gC-j\":[\"Se alle boliger\"],\"AYnPO0\":[\"Tekst\"],\"J3pMce\":[\"Side \",[\"page\"]],\"MimVzt\":[\"Les mer\"],\"NPCxte\":[\"Neste\"],\"O7SW7R\":[\"Nyheter\"],\"P00ECa\":[\"Laster inn antall tilgjengelige boliger...\"],\"PUn7qZ\":[\"Forside\"],\"PrX-B3\":[[\"0\"],\" ledige boliger\"],\"REM-Iz\":[\"Vi har ingen ledige stillinger akkurat nå.\"],\"ShrPvX\":[\"Søk\"],\"UNP6Ro\":[\"Om OBOS\"],\"Ue7fOG\":[\"Gå til forrige side\"],\"WBxuMf\":[\"Publisert\"],\"WZZDGo\":[\"Frist for forkjøp \u003c0/\u003e\"],\"Xxr1vx\":[\"Ledige stillinger\"],\"Z46vy0\":[\"Innflyttingsklare boliger\"],\"bdlnI6\":[\"Størrelse fra \",[\"0\"],\" – \",[\"1\"],\" m²\"],\"bwpLEf\":[\"\u003c0\u003e\u003c1\u003eSjekk at søkeordet er stavet riktig\u003c/1\u003e\u003c2\u003ePrøv mer generelle søkeord\u003c/2\u003e\u003c3\u003eBruk færre ord i søket ditt, for eksempel \\\"bolig\\\" i stedet for \\\"kjøp ny bolig i Oslo\\\"\u003c/3\u003e\u003c4\u003eIkke del personopplysninger når du bruker søket. Søkeordene lagres for å forbedre tjenesten.\u003c/4\u003e\u003c/0\u003e\"],\"cgyJEa\":[\"Forrige\"],\"dak_77\":[\"Det oppstod en feil under søket. Vennligst prøv igjen senere.\"],\"e2hpe8\":[\"Sist endret\"],\"egXu_Q\":[\"Laster inn pris...\"],\"ewQyCC\":[\"Paginering for nyheter\"],\"gB5mB7\":[\"Viser \",[\"0\"],\" av \",[\"1\"],\" resultat.\"],\"iI2BXG\":[\"Gå til neste side\"],\"irz-qZ\":[\"Hos oss er det store muligheter for en spennende karriere.\\n Vi søker etter deg som både er engasjert og faglig dyktig, og som tør å ta OBOS ett steg lenger.\\n Kanskje er det deg?\"],\"jk9b30\":[\"Vis mer\"],\"k5FD4B\":[\"Laster inn ferdigstillelsesdato...\"],\"lYXZGt\":[\"Laster inn prosjekt...\"],\"nBWrGq\":[\"Nye boliger\"],\"nSZD2l\":[\"Salgsstart \u003c0/\u003e\"],\"oSCUjQ\":[\"Temasider\"],\"qfAJQ3\":[\"Se alle innflyttingsklare boliger\"],\"ryb3Cy\":[\"Vi fant ingen resultater for ditt søk.\"],\"siC3QE\":[\"Tips for bedre søk\"],\"t5tfDT\":[\"Laster inn størrelsesområde...\"],\"tNsBaF\":[\"Kortversjonen\"],\"vXagu6\":[\"Innflytting fra \",[\"0\"],\" \",[\"1\"]],\"yAoVDV\":[\"Spill av video\"]},\"children\":[\"$\",\"$L15\",null,{\"locale\":\"nb\",\"isInternalLink\":\"$13\",\"children\":[[\"$\",\"$L16\",null,{}],false,[\"$\",\"$L2\",null,{\"parallelRouterKey\":\"children\",\"error\":\"$17\",\"errorStyles\":[],\"errorScripts\":[[\"$\",\"script\",\"script-0\",{\"src\":\"/_next/static/chunks/0yd-o2d3w17gj.js\",\"async\":true}]],\"template\":\"$L18\",\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":\"$L19\",\"forbidden\":\"$undefined\",\"unauthorized\":\"$undefined\"}]]}]}]}],\"$L1a\"]}]]}],\"$L1b\"]}]\n"])</script><script>self.__next_f.push([1,"18:[\"$\",\"$L3\",null,{}]\n19:[[\"$\",\"div\",null,{\"className\":\"prose container-prose my-16\",\"children\":[[\"$\",\"h1\",null,{\"className\":\"text-center\",\"children\":\"Beklager, men vi finner ikke siden du leter etter\"}],[\"$\",\"p\",null,{\"className\":\"lead text-center\",\"children\":\"Kanskje siden har blitt slettet eller funnet seg en ny bolig her på OBOS sine sider.\"}],[\"$\",\"p\",null,{\"children\":[\"Her kan du \",[\"$\",\"a\",null,{\"href\":\"/\",\"children\":\"gå tilbake til forsiden\"}],\".\"]}],[\"$\",\"p\",null,{\"children\":\"Du kan også velge en av disse lenkene for å komme deg videre:\"}],[\"$\",\"div\",null,{\"className\":\"grid gap-4\",\"children\":[[\"$\",\"a\",null,{\"href\":\"/bolig/prosjekt\",\"children\":\"Se hva vi bygger der du vil bo\"}],[\"$\",\"a\",null,{\"href\":\"/brukt-bolig\",\"children\":\"Brukt bolig\"}],[\"$\",\"a\",null,{\"href\":\"/medlem/medlemsfordeler\",\"children\":\"Medlemsfordeler\"}]]}]]}],[]]\n1a:[\"$\",\"obos-footer\",null,{\"suppressHydrationWarning\":true}]\n1b:[\"$\",\"$L10\",null,{\"id\":\"gtm-script\",\"strategy\":\"afterInteractive\",\"dangerouslySetInnerHTML\":{\"__html\":\"\\n (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':\\n new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],\\n j=d.createElement(s),dl=l!='dataLayer'?'\u0026l='+l:'';j.async=true;j.src=\\n 'https://stm.obos.no/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);\\n })(window,document,'script','dataLayer', 'GTM-P4W9NTL');\\n \"}}]\nb:[[\"$\",\"meta\",\"0\",{\"charSet\":\"utf-8\"}],[\"$\",\"meta\",\"1\",{\"name\":\"viewport\",\"content\":\"width=device-width, initial-scale=1\"}]]\n"])</script><script>self.__next_f.push([1,"8:null\nd:[[\"$\",\"meta\",\"0\",{\"name\":\"robots\",\"content\":\"index\"}],[\"$\",\"link\",\"1\",{\"rel\":\"canonical\",\"href\":\"https://www.obos.no/styrearbeid/nettrobots.txt\"}],[\"$\",\"meta\",\"2\",{\"name\":\"twitter:card\",\"content\":\"summary\"}]]\n"])</script><script>self.__next_f.push([1,"5:E{\"digest\":\"NEXT_HTTP_ERROR_FALLBACK;404\"}\n"])</script></body></html>
Sitemaps discovered
No sitemap declared in robots.txt and the common paths returned no 200.
About Robots + Sitemap
This tool fetches robots.txt from the host and displays it as-is. It also extracts every Sitemap: directive from the file and probes the two standard sitemap paths (/sitemap.xml and /sitemap_index.xml) to find any sitemaps that exist but were not declared. The result gives you a complete picture of what the site invites crawlers to read and where to find the structured URL inventory.
When to use it
Use this when migrating a site to confirm the new server is serving the right crawl rules. SEO professionals check it to verify they have not accidentally disallowed important paths. Bug bounty researchers read it for hints about admin paths or staging endpoints that are listed only here. Webmasters use it to discover where competitor sitemaps live, useful for content strategy and indexable URL counts.
How to read the results
Disallow lines tell crawlers what not to visit. A bare Disallow: with no path means crawl everything. Allow: overrides Disallow: for specific paths. User-agent: * applies to all crawlers, named agents take their own rules. Sitemap: lines list authoritative sitemap locations. The crawl-delay directive (some crawlers ignore it) suggests a minimum gap between requests in seconds.
Frequently asked questions
Does robots.txt enforce access control? ▾
No. It is a polite request, not security. Well-behaved crawlers like Google and Bing respect it. Malicious crawlers and scrapers ignore it freely. For real access control, use authentication, firewall rules, or rate limits.
Why does the tool show sitemaps that were not in robots.txt? ▾
Many sites publish sitemaps at the conventional /sitemap.xml path without declaring them in robots.txt. Search engines check this path automatically. The tool probes these standard locations and reports any that return a 200 response.
Should I block AI crawlers in robots.txt? ▾
That is a policy choice. Many publishers now block GPTBot, ClaudeBot, Google-Extended, and similar named user agents. Be aware that some crawlers ignore robots.txt, and blocking does not prevent content from appearing in training sets assembled before the block.
How big can robots.txt be? ▾
Google reads up to 500 KiB. Other crawlers may have different limits. Files larger than that get truncated, which can lead to unpredictable interpretation. Keep robots.txt focused on directives that matter and offload large URL lists to sitemaps.
Try with the same input