What are the patterns for integrating AI with databases and data lakes?

AI-database integration serves two purposes: providing data to AI models for inference and storing AI outputs for application use. Inference integration: implement vector search in the database (pgvector for PostgreSQL, Atlas Vector Search for MongoDB, OpenSearch knn), enable text-to-SQL querying through AI agents, and use database triggers to invoke AI processing on data changes. Output storage: store AI predictions alongside source data for historical tracking, maintain prediction metadata (model version, confidence, timestamp), and implement materialized views that combine raw data with AI enrichment. Architecture patterns: in-database AI (run models directly in the database using extensions like pgml or BigQuery ML), alongside-database (model serving endpoint called from application code, results stored back), and data lake AI (models process data in S3/GCS, results loaded into serving databases). Real-time enrichment: database CDC streams changes to an AI processor that enriches records and writes results back (e.g., new product listing triggers AI category classification, description generation, and image tagging). Batch enrichment: scheduled jobs process historical data through AI models for backfill or re-scoring. Performance: index AI-generated columns (vector indexes for embeddings, B-tree for scores), partition tables by processing status, and implement caching for frequently-accessed AI results. Data lake integration: use Apache Iceberg or Delta Lake for ACID transactions on AI training data, maintain data catalogs (DataHub, Amundsen) for discoverability, and implement access controls per dataset.