Data at scale
StayMatch
Natural-language search over two million enriched listings
A two-person course project: describe a stay in plain words, a language model turns the wish into a target profile, and nearest-neighbour search ranks about two million listings enriched with public safety, income, health and air-quality data.
Key result Built over 2,098,880 listing rows, as reported in the project notebook. Matching quality was not measured, so no accuracy is claimed.
Software system Status: Prototype Status: Completed Academic · team of 2
Schematic, not data
Purpose
Rental search runs on filters, but people describe what they want in words: quiet, safe, near the beach, good for remote work. StayMatch matches the description itself.
My Role & Leadership
Project Owner · Technical Lead · Developer. Owned the project and led its technical direction in a two-person course project, and worked hands-on in its development.
- Led the system design: a Spark pipeline that cleans and enriches about 2.1 million listings and partitions them by state, feeding a language-model step and a nearest-neighbour ranking.
- Set the central design decision: the language model writes a target listing profile instead of a filter, so the ranking stays an ordinary, inspectable distance.
What the team built
- A Spark pipeline that cleans about 2.1 million listing rows, with a combined hard and adaptive price-outlier rule.
- Enrichment joins at city and county level with public crime, road-accident, income, health (CDC PLACES) and air-quality (EPA) data.
- Parquet output partitioned by state, so each search reads one partition.
- A language-model step that turns a request (location, budget and free text) into a target listing profile in JSON.
- Standardised nearest-neighbour ranking, and a Streamlit interface that keeps favourites out of later results.
How it works
The language model does not filter listings: it writes a target record, filling what the user did not specify with column means. The candidate pool for the chosen state, price and city is standardised, and the closest listings by Euclidean distance are returned.
Clean the listing table on Spark and join neighbourhood data at city and county level.
Write the enriched table as Parquet, partitioned by state.
Turn the user's location, budget and free text into a target listing profile with GPT-4o.
Standardise the candidate pool and rank it by distance to the target profile.
Design decisions
- A target profile instead of a generated filter: the model describes the ideal listing, and the ranking stays an ordinary, inspectable distance.
- Partitioning by state, so a request touches one partition instead of the whole table.
- Enrichment before matching, so neighbourhood qualities are columns the search can use, not something the model has to guess.
Key technologies
Outcomes and limitations
Known limits
A prototype that runs inside Databricks only. The target profile comes from a language model at temperature 0.7, so the same request can rank differently; every numeric column weighs the same after scaling; matching quality was not measured. The dataset's terms are unconfirmed.
What was provided
- Access to the course's listing dataset and the Spark configuration came from the instructor.
- GPT-4o and Databricks are third-party services.
Evidence
- The team's Databricks notebook and application code.
- The code repository is not linked from this page.