Remove reference to phi3_vision

leon-seidel · web-flow · commit ebc6c0d24306 · 2025-01-25T18:46:40.000+01:00
diff --git a/examples/multimodal_vision/idefics3_example.py b/examples/multimodal_vision/idefics3_example.py
@@ -77,8 +77,7 @@ def tokenize(sample):
     )
 
 
-# long data lengths produced by the phi3_vision processor
-# can lead to integer overflows when mapping, avoid with writer_batch_size
+# avoid errors with writer_batch_size
 ds = ds.map(tokenize, writer_batch_size=1, remove_columns=ds.column_names)
 
 # Perform oneshot

Original file line number	Diff line number	Diff line change
`@@ -77,8 +77,7 @@ def tokenize(sample):`
`77`	`77`	`)`
`78`	`78`
`79`	`79`
`80`		`-# long data lengths produced by the phi3_vision processor`
`81`		`-# can lead to integer overflows when mapping, avoid with writer_batch_size`
	`80`	`+# avoid errors with writer_batch_size`
`82`	`81`	`ds = ds.map(tokenize, writer_batch_size=1, remove_columns=ds.column_names)`
`83`	`82`
`84`	`83`	`# Perform oneshot`