User-agent: * Allow: / Allow: /pages/ Allow: /register Allow: /login # Authenticated app — no user profiles or private pages Disallow: /app/ Disallow: /admin/ Disallow: /api/ Disallow: /onboarding/ Disallow: /set-language Disallow: /sso/ Disallow: /sw.js Disallow: /logout Disallow: /forgot-password Disallow: /reset-password # Prevent indexing of user photos (served via API) Disallow: /api/v1/photos/ Disallow: /api/v1/users/ # ── AI crawlers / LLM training bots ────────────────────────────── # We allow indexing of public marketing copy + the city/country # landing pages (those are SEO real estate, same answer for every # crawler), but block private routes and user-generated photos. # llms.txt at the root carries a structured summary of where to # look — see https://llmstxt.org/ for the convention. User-agent: GPTBot Allow: /$ Allow: /dating-in/ Allow: /pages/ Allow: /register Allow: /login Disallow: /app/ Disallow: /api/ Disallow: /admin/ Disallow: /onboarding/ User-agent: ChatGPT-User Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ # OAI-SearchBot is the crawler that builds ChatGPT's *search* index — # this is what makes the site show up when ChatGPT answers dating # questions. Allow the public marketing + landing pages so we're # discoverable there; keep the app/API private. User-agent: OAI-SearchBot Allow: /$ Allow: /dating-in/ Allow: /pages/ Allow: /register Allow: /login Disallow: /app/ Disallow: /api/ Disallow: /admin/ Disallow: /onboarding/ User-agent: ClaudeBot Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ User-agent: Claude-Web Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ User-agent: PerplexityBot Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ User-agent: Google-Extended Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ Disallow: /admin/ # Other AI assistants / answer engines — same policy: index the public # marketing + landing pages so we're discoverable when users ask these # assistants for a dating app; keep the app/API/admin private. # (Apple Intelligence + Siri, Microsoft Copilot / Bing Chat, Meta AI, # Amazon / Alexa, DuckDuckGo AI, Mistral, You.com.) User-agent: Applebot-Extended Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ Disallow: /admin/ User-agent: bingbot Allow: /$ Allow: /dating-in/ Allow: /pages/ Allow: /register Allow: /login Disallow: /app/ Disallow: /api/ Disallow: /admin/ User-agent: meta-externalagent Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ Disallow: /admin/ User-agent: Amazonbot Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ Disallow: /admin/ User-agent: DuckAssistBot Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ Disallow: /admin/ User-agent: MistralAI-User Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ Disallow: /admin/ User-agent: YouBot Allow: /$ Allow: /dating-in/ Allow: /pages/ Disallow: /app/ Disallow: /api/ Disallow: /admin/ # Crawlers that scrape solely for training without contributing to # search visibility — block. User-agent: CCBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: cohere-ai Disallow: / Sitemap: https://luvu.plus/sitemap.xml Sitemap: https://luvu.plus/llms.txt