🚀 AI API Contract Extractor

An intelligent backend system that clones a GitHub/GitLab repository and automatically extracts REST API endpoints along with LLM-inferred request/response schemas.

🔥 Features

Automatic API detection across multiple frameworks (Spring Boot, Express.js), implemented via the Strategy design pattern — adding a new framework means adding one new FrameworkEndpointStrategy bean, no existing code changes.
AI-powered schema inference using an LLM (via LangChain4j), with a heuristic fallback that kicks in automatically if the LLM call fails, times out, or returns unparseable output — the pipeline never returns a hard failure just because the LLM had a bad response.
Bounded, concurrent schema generation via a fixed-size thread pool, so a large repo doesn't spawn unbounded LLM calls (cost + rate-limit safety).
Deduplicated endpoint list (by METHOD:PATH).
Automatic cleanup of cloned repositories after each request — no disk leaks.
Host allowlisting on clone URLs (SSRF protection) and shallow, time-bounded clones.

🏗️ Architecture

POST /api/extract {repoUrl}
        ↓
   GitService            (validates host, shallow clones, always cleans up)
        ↓
   RepoScannerService     (walks the repo, finds candidate .java/.js/.ts files)
        ↓
   EndpointExtractorService
        ↓  (delegates per-file to whichever strategy supports it)
   FrameworkEndpointStrategy impls: SpringEndpointStrategy, ExpressEndpointStrategy
        ↓
   CodeSnippetExtractorService  (grabs a code window around each endpoint)
        ↓
   SchemaGeneratorService  (LLM inference → heuristic fallback on failure)
        ↓
   RepositoryStructure (JSON response: endpoints + request/response schemas)

🛠️ Tech Stack

Java 21, Spring Boot 3.3
LangChain4j for LLM integration (any OpenAI-compatible endpoint — defaults to Groq)
JGit for repository cloning
java.util.concurrent (ExecutorService) for bounded parallel schema generation
JUnit 5 + Mockito + AssertJ for testing

📦 Project Structure

service/
 ├── ExtractionService.java            # orchestrates the full pipeline
 ├── EndpointExtractorService.java     # dispatches to strategies
 ├── strategy/
 │    ├── FrameworkEndpointStrategy.java   # Strategy interface
 │    ├── AbstractRegexEndpointStrategy.java
 │    ├── SpringEndpointStrategy.java
 │    └── ExpressEndpointStrategy.java
 ├── SchemaGeneratorService.java       # LLM + heuristic fallback
 ├── CodeSnippetExtractorService.java
 ├── RepoScannerService.java
 └── GitService.java                   # clone / cleanup / URL validation

agent/
 ├── RepoAgent.java                    # LangChain4j AI service interface
 └── RepoTools.java                    # sandboxed file read/list tools for the agent

dto/
 └── ExtractRequest.java               # validated request body

exception/
 ├── InvalidRepoUrlException.java
 ├── ExtractionFailedException.java
 └── GlobalExceptionHandler.java       # consistent JSON error responses

model/
 ├── ApiEndpoint.java
 ├── GeneratedSchema.java
 └── RepositoryStructure.java

▶️ How to Run

1. Set your LLM API key (never commit it)

export GROQ_API_KEY=your_key_here

2. Build and run

./mvnw clean spring-boot:run

Server starts at http://localhost:8080.

3. Call the API

curl -X POST http://localhost:8080/api/extract \
  -H "Content-Type: application/json" \
  -d '{"repoUrl": "https://github.com/some-user/some-repo"}'

Only github.com and gitlab.com URLs are accepted by default (configurable via repo.allowed-hosts in application.properties).

4. Run tests

./mvnw test

⚙️ Configuration

All tunables live in application.properties:

Property	Default	Purpose
`extraction.schema-generation-limit`	5	Max endpoints sent to the LLM per request
`extraction.schema-generation-threads`	2	Max concurrent LLM calls
`extraction.llm-call-delay-millis`	250	Delay between LLM calls per thread
`repo.allowed-hosts`	github.com,gitlab.com	Allowed clone source hosts
`repo.clone-timeout-seconds`	60	Max time allowed for a clone

📌 Example Response

{
  "endpoints": [
    {
      "method": "POST",
      "path": "/jobPost",
      "sourceFile": "/tmp/repo_scan_.../JobController.java",
      "requestSchema": "{\"title\":\"string\",\"description\":\"string\"}",
      "responseSchema": "{\"id\":\"integer\",\"status\":\"string\"}"
    }
  ]
}

⚠️ Known Limitations

Endpoint detection is regex-based, not a full AST parse for JS/TS (Java uses JavaParser in SchemaExtractor for field typing, but endpoint detection is still regex-driven).
Schema accuracy depends on code clarity and how much context fits in the snippet window.
Only a bounded sample of endpoints (extraction.schema-generation-limit) get LLM-generated schemas per request, to control cost — the rest are still listed, just without inferred schemas.

🧩 Extending to a New Framework

Implement FrameworkEndpointStrategy (or extend AbstractRegexEndpointStrategy if regex-based detection is enough).
Annotate it @Component.
Done — Spring auto-injects it into EndpointExtractorService, no other code changes needed.

FilesExpand file tree

README.md

Latest commit

History