Kayak - Travel Metasearch

A team-built, 3-tier travel metasearch and booking platform: a React client over Node/Express services (API gateway, user, search, booking) backed by MySQL, MongoDB, and Redis, with a separate Python AI service for chat, deal scoring, and bundles.

ROLE
Developer
PERIOD
2025
DOMAIN
Distributed Systems
STATUS
Published
CODE
GitHub ↗

OVERVIEW

A team-built, 3-tier travel metasearch and booking platform modeled on Kayak (flights, hotels, car rentals). A React client talks through an API gateway (JWT auth, rate limiting, routing) to Node/Express middleware services, user, search (MongoDB + Redis cache), and booking (transactional, with inventory control), over a data tier that splits work across MySQL (OLTP), MongoDB (analytics and logs), and Redis (caching). A separate Python FastAPI service adds the AI layer: a concierge chat agent, deal scoring, and bundle and price-analysis APIs, fronted by a semantic cache that reuses LLM responses for similar queries and a rule-based intent parser that uses OpenAI only when available. Kafka carries cross-service events, with an interface-based in-memory implementation for local development.

ARRIVED AS

A Kayak-style metasearch site has to serve flights, hotels, and car rentals to clients, keep bookings transactionally correct, run analytics, and layer on AI recommendations, all without those concerns tangling together. The project's goal was a clean 3-tier distributed system: a client tier, a middleware tier of independent services, and a data tier split by job, with an AI service kept off to the side.

This was a team final project: a distributed system modeling Kayak's travel metasearch and booking, with flights, hotels, and car rentals, user bookings and payments, admin and analytics, and an agentic AI recommendation layer. It is organized as a textbook 3-tier system, client, middleware, data, with an additional Python AI service. My work centered on the back-end services and the AI layer (the semantic cache, intent parsing, and Kafka integration).

WHAT I BUILT

  1. 01A 3-tier architecture: a React client, a middleware tier of Node/Express services, and a data tier of MySQL, MongoDB, and Redis, each chosen for what it is good at.
  2. 02Middleware services behind an API gateway that handles JWT auth, rate limiting, and routing: a user service, a search service (MongoDB + Redis caching), and a booking service with transactional inserts and inventory decrements.
  3. 03Data split by purpose: MySQL for transactional bookings, billing, and listings; MongoDB for analytics, reviews, and event logs; Redis for caching search results.
  4. 04Kafka for event streaming between services, with an in-memory Kafka implementation behind the same interface so the team could develop independently before the real broker existed.
  5. 05A separate Python FastAPI AI service: a concierge chat agent, a deals-scoring background worker, plus bundles, price-analysis, watches, and quotes APIs, with a semantic cache for LLM responses and rule-based intent parsing that uses OpenAI only when available.

WHAT CHANGED

  • Clean separation of concerns across three tiers, so transactional booking (MySQL), analytics (MongoDB), and caching (Redis) each use the right store instead of forcing one database to do everything.
  • A semantic cache that returns a stored answer when a new query is similar enough (cosine > 0.85), cutting repeat LLM calls instead of re-asking the model every time.
  • Designed for parallel team development: an interface-based Kafka client lets the system run on an in-memory queue locally and switch to a real broker by config alone.

Data flow

click a stage

The React client calls the API gateway, which authenticates (JWT), rate-limits, and routes to the right middleware service.

COMPONENT

React client (Tier 1)

The metasearch UI plus AI components (chat widget, bundle cards, price analysis, watches), talking to the gateway over an API layer.

Decisions, with the cost of each.

A decision without its trade-off is marketing. Each row says what was chosen, why, and what it gave up.

Split the data tier by workload

Bookings need transactions and joins (MySQL), analytics and logs are append-heavy and schema-flexible (MongoDB), and repeated searches just need to be fast (Redis). Using one database for all three would compromise at least one of those jobs, so each store is matched to its workload.

A single database for everything (transactions and analytics and caching all fighting in one engine).

Put a semantic cache in front of the LLM

Travel queries repeat with small variations. Embedding each query and returning a cached answer when similarity clears a threshold (0.85) avoids paying for an LLM call on every near-duplicate, which matters for both latency and cost.

Exact-match caching (misses paraphrases); no cache (an LLM call for every query, however similar).

Hide Kafka behind an interface with an in-memory implementation

On a team, not everyone can stand up a broker on day one. A KafkaInterface with an in-memory queue lets each person develop and test against simulated topics, then switch to the real broker by changing config, with no code changes.

Require a real Kafka broker for any local work (blocks teammates and slows iteration).

The part that mattered.

The numbers behind the work, and the code that produced them.

client · services · data
3 tiers
gateway + Node services + MySQL/Mongo/Redis
for LLM responses
Semantic cache
cosine > 0.85, Redis-backed
split by workload
3 stores
MySQL OLTP · MongoDB analytics · Redis cache
event streaming
Kafka
in-memory stand-in for local dev
A semantic cache in front of the LLMpython
class SemanticCache:
    """Semantic cache for LLM responses (cosine similarity > 0.85, Redis-backed)."""

    def __init__(self, redis_client, embedding_service, threshold: float = 0.85,
                 ttl_days: int = 7):
        self.redis = redis_client
        self.embeddings = embedding_service   # embeddings via a local model
        self.threshold = threshold

    def get(self, query: str):
        query_embedding = self.embeddings.embed_query(query)
        cache_keys = self.redis.smembers(self.index_key)
        # compare against stored embeddings; if cosine similarity > threshold,
        # return the cached LLM response instead of calling the model again
        ...

Each query is embedded and compared against cached embeddings; if a previous query is similar enough (cosine > 0.85) its stored response is returned, so a near-duplicate question never hits the model again. Entries live in Redis with a TTL, so the cache is fast and self-expiring.

Rule-based intent parsing, OpenAI optionalpython
try:
    from openai import OpenAI
    OPENAI_AVAILABLE = True
except ImportError:
    OPENAI_AVAILABLE = False

class IntentParser:
    """Parses natural-language travel queries into structured intents.
    Uses rule-based parsing with optional LLM enhancement."""
    def __init__(self):
        self.airport_codes = {
            "san francisco": "SFO", "new york": "JFK",
            "mumbai": "BOM", "delhi": "DEL", "bangalore": "BLR",
            # ...
        }

Intent parsing works without any model: rules and an airport-code map turn a query into structured intent, and OpenAI is used only as an optional enhancement when it is installed. The parser degrades to pure rules rather than failing when the LLM is unavailable.

Kafka behind an interface, with an in-memory stand-inpython
class MemoryKafka(KafkaInterface):
    """In-memory Kafka for independent development.
    - Loads test data from JSON files
    - Uses a Python Queue to simulate topics
    - Swap to the real Kafka client by changing config
    """
    def __init__(self, data_dir: str = "data/mock"):
        ...

Both the in-memory and real-broker clients implement the same KafkaInterface, so the rest of the system does not know or care which is running. The team could build and test against simulated topics, then flip to a real broker by config when it was ready.

✓ LEARNED

  1. A 3-tier split only pays off if the data tier is split too: matching MySQL, MongoDB, and Redis to transactions, analytics, and caching is most of the design.

  2. A semantic cache turns an LLM from a per-request cost into something closer to a memoized function for similar queries, which is a cheap, large win on a chat-style feature.

  3. Hiding infrastructure like Kafka behind an interface with an in-memory implementation is what makes parallel team development actually work.