Files
dotfiles/dot_agents/skills/cloudflare/references/vectorize/gotchas.md
T
Prad Nukala 6406208b35 shell: fish functions -> ~/.local/bin executables + bash/zsh ports; manage ~/.agents
- 16 cross-shell utilities converted to sh/bash chezmoi executables:
  ai c2p cp-appid extract_agents fab gdc gh-sonr-org-migrate ghg-clone
  ghn gho git-summary launch_omp_livediff_overlay lc-elir mdcp mdp pj
  (ghg-clone not named ghg: unrelated cargo binary owns that name)
- cwd/history helpers stay per-shell: yy c cd clip-last + ghg/ghn/gho
  cd-wrappers in bash/functions/ and zsh/functions/, loaded by rc files
- fish keeps only shell-state functions; converted ones now hit PATH
- cp-appid ignore gating moved to .local/bin path
- dot_agents/skills: 82 skills materialized from dotstate storage so
  dot_claude skill symlinks resolve on fresh bootstrap; dotstate
  profile/symlinks.json released .agents/skills .bashrc .zshrc
2026-08-10 15:15:06 -04:00

2.1 KiB

Vectorize Gotchas

Critical Warnings

Async Mutations

Insert/upsert/delete return immediately but vectors aren't queryable for 5-10 seconds.

Batch Size Limit

Workers API: 500 vectors max per call (undocumented, silently truncates)

// ✅ Chunk into 500
for (let i = 0; i < vectors.length; i += 500) {
  await env.VECTORIZE.upsert(vectors.slice(i, i + 500));
}

Metadata Truncation

returnMetadata: "indexed" returns only first 64 bytes of strings. Use "all" for complete metadata (but max topK drops to 20).

topK Limits

returnMetadata returnValues Max topK
"none" / "indexed" false 100
"all" any 20
any true 20

Metadata Indexes First

Create BEFORE inserting - existing vectors not retroactively indexed.

# ✅ Create index FIRST
wrangler vectorize create-metadata-index my-index --property-name=category --type=string
wrangler vectorize insert my-index --file=data.ndjson

Index Config Immutable

Cannot change dimensions/metric after creation. Must create new index and migrate.

Limits (V2)

Resource Limit
Vectors per index 10,000,000
Max dimensions 1536
Batch upsert (Workers) 500
Indexed string metadata 64 bytes
Metadata indexes 10
Namespaces 50,000 (paid) / 1,000 (free)

Common Mistakes

  1. Wrong embedding shape: Extract result.data[0] from Workers AI
  2. Metadata index after data: Re-upsert all vectors
  3. Insert vs upsert: insert ignores duplicates, upsert overwrites
  4. Not batching: Individual inserts ~1K/min, batched ~200K+/min

Troubleshooting

No results?

  • Wait 5-10s after insert
  • Check namespace spelling (case-sensitive)
  • Verify metadata index exists
  • Check dimension mismatch

Metadata filter not working?

  • Index must exist before data insert
  • Strings >64 bytes truncated
  • Use dot notation for nested: "product.category"

Model Dimensions

  • @cf/baai/bge-small-en-v1.5: 384
  • @cf/baai/bge-base-en-v1.5: 768
  • @cf/baai/bge-large-en-v1.5: 1024