[epic] aibridge scalability

[dannykopping]

Notes

Issues:

• Writing individual records to postgres saturates connection pool (see also: Allow configuration of max database connections per coderd coder#21360), adds resource pressure on postgres
  • Batch postgres writes
• JSON marshaling in AI libs is a source of high allocs
  • Fork libs and use more efficient JSON library
    • perf: optimise appendCompact usage anthropics/anthropic-sdk-go#262
• If CachedBridgePool fills up and thrashes, it can add significant latency
  • Set pool size high by default: perf: increase bridge pool cache size limit coder#21399