Imagenet_fake_data_benchmark.py has no split info

[mattiasmar]

What are the prerequisites for running the Imagenet_fake_data_benchmark.py?
The tfds.builder used in train.py doesn't generate any split info and thus the method create_input_iter in the flax example fails.

Without a disk quota to download all of imagenet, do you have a suggestion on how to run the fake data example?

[mattiasmar]

Seems the expected way is to manually download the metadata file dataset_info.json to the folder os.path.join(data_dir,imagenet2012/5.1.0) (where data_dir is defined in the beginning of the method test_fake_data in imagenet_fake_data_benchmark.py, line 42, and imagenet2012/5.1.0 is defined in the example's config file.

@avital @marcvanzee We could automate this or append the example source code with the metadata file (for each of the flax examples). I could submit a PR, but maybe first you could share your thoughts on design / necessity / pitfalls of this issue?

[avital]

That's excellent. Then let's use your proposal! Can you please file a pull request (that also removes all references and dependencies to svn in our scripts?) Thanks!

[mattiasmar]

I'll give it a shot. Will begin next week.

[avital]

Excellent, thanks Mattias!

[mattiasmar]

Change done, but I'm not able to run the coverage tests (independent of my own changes).

Exception occurred:
  File "/usr/lib/python3/dist-packages/sphinx/jinja2glue.py", line 233, in get_source
    raise TemplateNotFound(template)

Do you want to go on a call to quickly sort it out?

[marcvanzee]

@mattiasmar I think we totally dropped the ball on this one, apologies! Is this problem still occurring?

[mattiasmar]

Don't know. Haven't worked on it since then.

…

________________________________ From: Marc van Zee ***@***.***> Sent: Monday, August 29, 2022 9:15:45 AM To: google/flax ***@***.***> Cc: Marder, Mattias ***@***.***>; Mention ***@***.***> Subject: Re: [google/flax] Imagenet_fake_data_benchmark.py has no split info (Discussion #1773) @mattiasmar<https://github.com/mattiasmar> I think we totally dropped the ball on this one, apologies! Is this problem still occurring? — Reply to this email directly, view it on GitHub<#1773 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALPSDLD25BE6DMMRWAB6I4TV3RPSDANCNFSM5LVUEKDQ>. You are receiving this because you were mentioned.Message ID: ***@***.***> --------------------------------------------------------------------- Intel Israel (74) Limited This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.