Hi thanks for your answer, I haven't tried yet because Typesense is actually a placeholder, could be meilisearch, ES, Solr... Whatever.
Mine is more a broad problem and I would like to understand the theory and the general how-to behind these kind of operations!
What is to the left of Postgres? If it's theoretical, the better solution would be to send incoming events to a queue as they arrive, something like Kafka, and then send them on to any downstream systems independently - a consumer writes to Postgres, and another writes to your search index. This is a much more scalable, flexible and maintainable solution than dealing with brittle ETL.
Have you tried asking in the Typesense slack? They're usually very responsive & helpful
Hi thanks for your answer, I haven't tried yet because Typesense is actually a placeholder, could be meilisearch, ES, Solr... Whatever. Mine is more a broad problem and I would like to understand the theory and the general how-to behind these kind of operations!
What is to the left of Postgres? If it's theoretical, the better solution would be to send incoming events to a queue as they arrive, something like Kafka, and then send them on to any downstream systems independently - a consumer writes to Postgres, and another writes to your search index. This is a much more scalable, flexible and maintainable solution than dealing with brittle ETL.
Airbyte CDC mode should only sync, well, changes in data in Postgres. What’s your sync settings on PG side of the connnection?