User-agent: * Disallow: /pages/ Disallow: /api/ Disallow: /files/