Pinecone is introducing an open-source extension for PostgreSQL called pg_pinecone. This lets developers perform vector searches without using a separate vector database. Pg_pinecone enables faster searches and reduces memory usage, particularly in AI applications.
Pg_pinecone is based on the search technology that Pinecone already uses for its own vector database. This extension makes that technology directly available in PostgreSQL. This lets companies keep using their existing database for applications that search by meaning, such as AI assistants.
Currently, developers often use pgvector for this purpose. This is an earlier and widely used extension for PostgreSQL. Pinecone states that the HNSW indexing used in pgvector requires a relatively large amount of memory. Furthermore, search accuracy can decrease when users apply specific filters.
Smaller index
Pg_pinecone uses a different method for indexing vector data. Instead of a network of interconnected vectors, it groups and stores data in a compressed format. This way, the index takes up much less memory space.
According to Pinecone’s benchmarks, the index size for a dataset of one million vectors is only 136 MB. With pgvector using a traditional HNSW index, it is 4.1 GB. In addition, the company reports that building the index takes less time, 45 seconds versus 79 seconds.
For search queries where 99 percent of the relevant results are found, Pinecone reports a response time of 2.3 milliseconds, compared to 6.7 milliseconds for pgvector. The results are based on Pinecone’s own tests.
Another difference concerns how search results are filtered. Pg_pinecone applies filters during the search, while pgvector, using HNSW, generally applies them afterward. According to Pinecone, this prevents strict-filter queries from missing relevant results.
Switch to the cloud
The extension is available under the Apache 2.0 license and runs directly within PostgreSQL. No Pinecone account or external API is required to use the local search engine. Existing SQL queries also remain usable.
Developers can also switch from this extension to Pinecone’s managed cloud service. Search queries are then executed on the company’s infrastructure, while PostgreSQL remains the central data repository.
With pg_pinecone, the company is primarily targeting developers who want to keep their AI applications within PostgreSQL but may eventually need a separate search infrastructure. The first version of the extension is available on GitHub.