GCP – A deep dive into AlloyDB’s vector search enhancements

If you’re building a generative AI application or an AI agent, there’s a high likelihood you’ll need to perform simultaneous searches on structured and unstructured data. For example, the prompt “Show me all pictures of sunsets I took in the past month” includes a structured part (the date is within the past month) and an unstructured part (the picture contains a sunset). In recent years, modern relational databases such as AlloyDB for PostgreSQL have added vector search capabilities to cover the unstructured part.

At Google Cloud Next 2025, we announced a series of key innovations in AlloyDB AI’s ScaNN index to improve performance and quality of search over structured and unstructured data. By deeply integrating with the AlloyDB query planner, the ScaNN index is able to optimize the ordering of SQL filters in vector search based on your workload characteristics. Let’s dive into what filter selectivity is and how AlloyDB ScaNN’s index leverages it to improve the performance and quality of your search.

Filtered vector search

To illustrate the power of filtered vector search in AlloyDB, imagine you’re an online retailer managing a product catalog within AlloyDB. With more than 100,000 items, this product catalog includes references to images, textual descriptions, inventory information, and catalog metadata in your products table.

To search through this data, you can leverage vector search with SQL filters to enable search across unstructured and structured data, providing users with higher quality search results. In the metadata, there may be fields such as color, gender, size and price stored in your table that you can leverage as search filters.

Say a user searches for a “maroon puffer jacket”. You might use “maroon” as a filter and “puffer jacket” as the part of the query upon which you perform a vector search. So you might have a SQL statement like:

In the products table, we have set a vector index on our text_embedding column and a B-tree index on our metadata column, color.

Depending on how commonly maroon appears in the color column of the dataset, which is called selectivity in database terminology, the AlloyDB query planner may choose to apply the filter before, after, or in-line with the vector search query. Let’s dive into why the planner may choose one option over the other. 

High selectivity

When a filter is highly selective, it means that only a small percentage of your data meets the specified criteria. In our example, “maroon” is a rare color, with only 0.2% of the 100,000 products in the catalog being that color. 

In cases where we have highly selective filters, the AlloyDB query planner often chooses to apply a pre-filter, i.e, applying the filtering conditions prior to the vector search. In our example, we would apply our filter condition WHERE color=’maroon’ before the vector search. Since “maroon” is rare, the B-tree index on the color column efficiently identifies a small subset of products (e.g., 200 out of 100,000). Subsequently, the computationally intensive vector search is performed only on this significantly reduced set of candidates. This strategy utilizes a K-Nearest Neighbors (KNN) vector search, which delivers results with 100% recall, i.e., the exact closest neighbors, within the set of results after the filter is replied.

Low selectivity

Conversely, if the filter isn’t highly selective (e.g., if 90% of products are “blue”), pre-filtering is inefficient because it doesn’t significantly narrow down the search space. In such cases, when a large proportion of your data satisfies the filtering conditions, a filter is considered to have low selectivity.

Say you’re searching for “blue puffer jackets”; if 90% of our catalog is blue, applying the filter first isn’t beneficial because it doesn’t narrow down our list of candidates all that much. If you applied a pre-filter, you would end up performing a KNN vector search against the majority of the dataset, which would be computationally expensive. Therefore, the AlloyDB query planner would choose to apply a post-filter.

Post-filtering means performing the vector search first, leveraging an Approximate Nearest Neighbors (ANN) vector index such as ScaNN on the text_embedding column to quickly identify a set of candidate results. Only after retrieving these initial candidates — the top 100 based on vector similarity — is the filter condition, WHERE color=’blue’, applied.

If your filter had high selectivity, there’s a risk this approach would yield very few candidates meeting your filter criteria. However because the condition WHERE color=’blue’ has low selectivity, you would likely obtain the approximate top-100 results. In the unlikely case you do not retrieve 100 results, the vector search would need to perform additional scans on the vector index to retrieve more candidates until the desired limit was reached. While effective for filters with low selectivity, post-filtering can become less efficient with highly selective filters, as the vector index might need to scan through many non-matching candidates.

Medium selectivity

When a filter has medium selectivity, the AlloyDB query planner may choose to apply either a pre-filter or a post-filter. However in cases of medium selectivity that range from 0.5-10% selectivity (such as, say, the color “purple”), AlloyDB supports a method called inline filtering, or in-filtering. Inline filtering applies the filter conditions in tandem with the vector search. With in-line filtering, AlloyDB leverages a bitmap from a B-tree index to select candidates matching the filter condition in tandem with the vector search in one pass. 

So in this example, while the plan evaluates which candidates are purple, AlloyDB is simultaneously searching for the approximate neighbors of the search query against items in the data catalog. This approach balances the benefits of reducing the search space, as pre-filtering does, without the risk of returning too few results, a potential issue with post-filtering when combined with a highly selective filter. 

Adaptive filtration

While the cases detailed above seem to clearly partition the search space across three different kinds of filtering, in practice it’s not so simple. At times the query planner may misjudge the selectivity of a filter due to outdated statistics, resulting in the vector search and filtering conditions being applied in a suboptimal order, for less high-quality results. This is where AlloyDB ScaNN’s latest innovation, adaptive filtration, comes in. With adaptive filtration, AlloyDB learns the selectivity of your filters at query time based on actual observed statistics and can adaptively change its execution plan. This results in more optimal ordering of filters and vector search, and greatly mitigates cases of planner misestimations. 

In summary, real-world workloads are complex, and distinct filtering conditions have different selectivities that may change over time as your data and workloads grow. That’s where an intelligent database engine powering your vector search can make a difference — by optimizing and adapting filtering for your workload, helping to ensure consistently high-quality and performant search results as your data evolves.

Get started today

Get started with vector search leveraging AlloyDB’s ScaNN index today. Then, learn how you can use AlloyDB AI’s latest features to power multimodal vector search. Adaptive filtration is available in preview; get started by turning on the feature flag. 

You can also sign up for a 30-day AlloyDB free trial.