Project Structure

42crawl is built with modern web technologies, prioritizing speed, scalability, and developer experience. This guide helps you navigate the repository to contribute to our SEO crawler or implement custom technical SEO logic.

---

Core Directories

• src/: The heart of the application. Contains the React source code, components, and the core crawling engine logic.
• docs/: The documentation site you are currently reading, powered by VitePress.
• public/: Static assets like logos, icons, and fonts served directly.
• supabase/: Database schemas, edge functions, and migrations for our backend.
• scripts/: Utility scripts for data processing and build automation, including our internal link audit tools.

---

Key Configuration Files

• keystatic.config.ts: Configuration for the Keystatic headless CMS used to manage our blog.
• tailwind.config.ts: Our design system and styling configuration.
• vite.config.ts: The build and development server setup.

For more details on how to contribute or run 42crawl locally, see the README.md file in the root directory. Mastering the project structure is the first step toward building the future of technical SEO.