Imagine being able to type:

“Where is the Terraform config for our staging RDS?”
“What’s the on-call schedule for the payments service?”
“List all services deployed in the last 24 hours.”

…and getting back a clear, actionable answer.

In this article, we’ll build a full-stack LangChain-based prototype that turns natural language into useful developer queries — across service metadata, deployment info, runbooks, and more. You’ll walk away with a working tool that can run in your terminal, Slack, or even your internal dev portal.

What We’ll Build

A LangChain app that:

1. Accepts natural language developer questions
2. Uses semantic search to retrieve relevant metadata from your system
3. Builds a structured prompt and sends it to OpenAI
4. Returns the answer in plain English — and optionally in JSON or Markdown

We’ll build this in a modular way, so you can plug it into Slack, VS Code, or your internal web tools.

Stack Overview

• LangChain for orchestration
• ChromaDB as the vector database
• FastAPI as the backend
• OpenAI as the LLM backend
• Frontends: CLI and (optionally) Slack or web

Step 1: Setup and Install

pip install langchain openai chromadb fastapi uvicorn

Set your API key:

export OPENAI_API_KEY=sk-...

Step 2: Ingest Developer Metadata

Let’s say we have structured JSON like:

{
  "name": "checkout-service",
  "owner": "team-payments",
  "oncall": "@sarah",
  "docs": "See internal/wiki/checkout",
  "last_deploy": "2024-04-01T16:22Z",
  "infra": "Terraform module: rds.tf"
}

Create vector_store.py:

import json
import chromadb
from langchain.embeddings import OpenAIEmbeddings

def build_vector_store(data_path="data/services.json"):
    client = chromadb.Client()
    collection = client.create_collection(name="services")
    embeddings = OpenAIEmbeddings()

    with open(data_path) as f:
        services = json.load(f)

    for svc in services:
        doc = f"""
        Service: {svc['name']}
        Owner: {svc['owner']}
        On-call: {svc['oncall']}
        Docs: {svc['docs']}
        Infra: {svc['infra']}
        Last Deployed: {svc['last_deploy']}
        """
        collection.add(documents=[doc], ids=[svc['name']])

    return collection

Step 3: LangChain App

Create query_engine.py:

from langchain.chains import RetrievalQA
from langchain.llms import OpenAI
from langchain.vectorstores import Chroma
from langchain.embeddings import OpenAIEmbeddings

def build_query_engine():
    vectordb = Chroma(persist_directory="chroma", embedding_function=OpenAIEmbeddings())
    llm = OpenAI(model_name="gpt-4")
    qa = RetrievalQA.from_chain_type(llm=llm, retriever=vectordb.as_retriever())
    return qa

Create a simple CLI:

qa = build_query_engine()

while True:
    query = input("🧠 Ask DevGPT: ")
    if query.lower() in ['exit', 'quit']:
        break
    answer = qa.run(query)
    print(f"\n{answer}\n")

Step 4: Optional FastAPI Wrapper

Want to expose it over HTTP?

from fastapi import FastAPI
from query_engine import build_query_engine

app = FastAPI()
qa = build_query_engine()

@app.post("/ask")
def ask_question(q: str):
    return {"answer": qa.run(q)}

Run it with:

uvicorn app:app --reload

Now you can hit:

POST /ask?q=Who owns the checkout service?

Step 5: Slackbot Integration (Bonus)

Use Slack Bolt to wire this backend into a Slack command:

@app.command("/askdev")
def handle_askdev(ack, body, say):
    query = body['text']
    answer = qa.run(query)
    say(answer)

Prompt Engineering Tip

To keep responses clear and accurate, prepend metadata to your query like this:

prompt = f"""
You are an internal developer assistant. Use the following service metadata to 
answer the user's question:

User: {query}
Metadata:
{docs}
"""

You can even have the LLM respond in structured JSON like:

{
  "owner": "@team-core-infra",
  "last_deploy": "2024-04-02T12:32Z",
  "status": "Healthy"
}

Limitations and Future Work

• You'll want regular re-indexing of metadata as services change
• Add caching for repeated queries
• Fine-tune with your internal terminology and acronyms
• Add memory or chat history via LangChain ConversationChain

Conclusion

With LangChain, OpenAI, and a bit of creativity, you can build internal tools that make developer metadata searchable, understandable, and actionable via natural language. This is how we move beyond dashboards — and toward true dev productivity assistants.

Want to see this in action across Backstage, GitHub, and Slack? Explore our AI tooling stack on Slaptijack