How to get the textured mesh in gradio?

[lililuya]

Hi, really inspiring work! I encountered a problem when use the gradio demo. Firstly, I upload the texture img and corresponding mask to get the texture. When I try to rendered the mesh I upload to gradio from the 'example/result' , I got this result in the "Textured results" region.

It seemed like that I got 8 screenshot of the page I was operating. Is there something wrong with my processing. Appreciating for your reply!!!

[ZhanxyR]

'Outputs' shows the inpainted result. If you want to get the textured mesh, please put the correct mesh to 'Mesh Input' (Noted that it will take the inputs as the texture image) and click ‘Render’ later.
Or you can download the inpainted result and modify the texture image path in the '.mtl' file.

[lililuya]

'Outputs' shows the inpainted result. If you want to get the textured mesh, please put the correct mesh to 'Mesh Input' (Noted that it will take the inputs as the texture image) and click ‘Render’ later. Or you can download the inpainted result and modify the texture image path in the '.mtl' file.

Thanks for your quick reply, I've tried the gradio again, put the correct mesh in mesh input, the expected result should be the textured mesh, but the 'Textured Result' still the screenshot.

BTW, To try the second way you mentioned, I replace the map_Kd in texture.mtl, and how should I get the result, should I use the texture_projection function? I am new in rendering, hope this not bother you.

Looking for your replay!

[ZhanxyR]

That's right, you should replace the path first. And make sure the mesh file contains the correct vts information. You can refer here to add the uv information to a mesh file and bind it with the modified texture.mtl.

You can take a look at the partial_colored.obj in our temp results.

# An .obj file may contains such information :
mtllib texture.mtl # to specific the material file
usemtl texture

v -0.019645 0.195721 0.010508 # vertex information

vt 0.458691 0.749025 # uv information

f 4/1 37498/40198 37499/40199 # triangle mesh

After that, you can directly use software like meshlab or blender to check the textured mesh. Or you can use open3d to render the result manually.
Here is the corresponding code we used in the gradio demo. It will load the mesh and texture image first. Then render the images with open3d. I'm also curious about why it shows the screenshot rather than the mesh even if you have already provided the mesh. Can you please check the temp results of this part. And it will be helpful to provide your environment for me to check later.

[lililuya]

Thanks for reply,!

1. I have check the temp_result partial_colored.obj , it contains the keys you mentioned
   
2. The partial_colored.obj in meshlab is as the following, it contained the uv information.
   
3. And my environment is export as:

name: shert
channels:
  - defaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2024.9.24=h06a4308_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libffi=3.4.4=h6a678d5_1
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.4=h6a678d5_0
  - openssl=3.0.15=h5eee18b_0
  - pip=24.2=py38h06a4308_0
  - python=3.8.20=he870216_0
  - readline=8.2=h5eee18b_0
  - sqlite=3.45.3=h5eee18b_0
  - tk=8.6.14=h39e8969_0
  - xz=5.4.6=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1
  - pip:
      - absl-py==2.1.0
      - accelerate==1.0.1
      - adabound==0.0.5
      - aiofiles==23.2.1
      - aiohappyeyeballs==2.4.3
      - aiohttp==3.10.10
      - aiosignal==1.3.1
      - altair==5.4.1
      - annotated-types==0.7.0
      - antlr4-python3-runtime==4.9.3
      - anyio==4.5.2
      - argon2-cffi==23.1.0
      - argon2-cffi-bindings==21.2.0
      - arrow==1.3.0
      - asttokens==2.4.1
      - async-lru==2.0.4
      - async-timeout==4.0.3
      - attrs==24.2.0
      - babel==2.16.0
      - backcall==0.2.0
      - beautifulsoup4==4.12.3
      - bleach==6.1.0
      - boto3==1.35.55
      - botocore==1.35.55
      - cachetools==5.5.0
      - certifi==2022.12.7
      - cffi==1.17.1
      - charset-normalizer==2.1.1
      - chumpy==0.70
      - click==8.1.7
      - coloredlogs==15.0.1
      - comm==0.2.2
      - compel==2.0.3
      - compress-pickle==2.1.0
      - contourpy==1.1.1
      - cupy==12.3.0
      - cycler==0.12.1
      - cython==0.29.37
      - debugpy==1.8.7
      - decorator==5.1.1
      - defusedxml==0.7.1
      - diffusers==0.31.0
      - docker-pycreds==0.4.0
      - einops==0.8.0
      - exceptiongroup==1.2.2
      - executing==2.1.0
      - face-alignment==1.4.1
      - facenet-pytorch==2.5.3
      - fastapi==0.112.4
      - fastjsonschema==2.20.0
      - fastrlock==0.8.2
      - ffmpy==0.4.0
      - filelock==3.13.1
      - flatbuffers==24.3.25
      - fonttools==4.54.1
      - fqdn==1.5.1
      - frozenlist==1.5.0
      - fsspec==2024.2.0
      - future==1.0.0
      - fvcore==0.1.5.post20221221
      - gevent==24.2.1
      - gitdb==4.0.11
      - gitpython==3.1.43
      - google-auth==2.36.0
      - google-auth-oauthlib==1.0.0
      - gradio==4.44.1
      - gradio-client==1.3.0
      - gradio-toggle==0.0.2
      - greenlet==3.1.1
      - grpcio==1.67.1
      - h11==0.14.0
      - httpcore==1.0.6
      - httpx==0.27.2
      - huggingface-hub==0.26.2
      - humanfriendly==10.0
      - idna==3.4
      - imageio==2.35.1
      - imgaug==0.4.0
      - importlib-metadata==8.5.0
      - importlib-resources==6.4.5
      - iopath==0.1.10
      - ipykernel==6.29.5
      - ipython==8.12.3
      - ipywidgets==8.1.5
      - isoduration==20.11.0
      - jax==0.4.13
      - jaxlib==0.4.13
      - jedi==0.19.1
      - jinja2==3.1.3
      - jmespath==1.0.1
      - joblib==1.4.2
      - json5==0.9.25
      - jsonpointer==3.0.0
      - jsonschema==4.23.0
      - jsonschema-specifications==2023.12.1
      - jupyter-client==8.6.3
      - jupyter-core==5.7.2
      - jupyter-events==0.10.0
      - jupyter-lsp==2.2.5
      - jupyter-server==2.14.2
      - jupyter-server-terminals==0.5.3
      - jupyterlab==4.2.5
      - jupyterlab-pygments==0.3.0
      - jupyterlab-server==2.27.3
      - jupyterlab-widgets==3.0.13
      - kiwisolver==1.4.7
      - kornia==0.7.3
      - kornia-rs==0.1.7
      - lazy-loader==0.4
      - lightning-utilities==0.11.8
      - llvmlite==0.41.1
      - markdown==3.7
      - markdown-it-py==3.0.0
      - markupsafe==2.1.5
      - matplotlib==3.7.5
      - matplotlib-inline==0.1.7
      - mdurl==0.1.2
      - mediapipe==0.10.11
      - mistune==3.0.2
      - ml-dtypes==0.2.0
      - mpmath==1.3.0
      - msgpack==1.1.0
      - multidict==6.1.0
      - munch==4.0.0
      - narwhals==1.13.2
      - nbclient==0.10.0
      - nbconvert==7.16.4
      - nbformat==5.10.4
      - nest-asyncio==1.6.0
      - networkx==3.0
      - notebook==7.2.2
      - notebook-shim==0.2.4
      - numba==0.58.1
      - numpy==1.23.1
      - oauthlib==3.2.2
      - omegaconf==2.3.0
      - onnxruntime==1.19.2
      - open3d==0.10.0.0
      - opencv-contrib-python==4.10.0.84
      - opencv-python==4.10.0.84
      - opencv-python-headless==4.10.0.84
      - opt-einsum==3.4.0
      - orjson==3.10.11
      - overrides==7.7.0
      - packaging==24.1
      - pandas==2.0.3
      - pandocfilters==1.5.1
      - parso==0.8.4
      - pexpect==4.9.0
      - pickleshare==0.7.5
      - pillow==10.2.0
      - pkgutil-resolve-name==1.3.10
      - platformdirs==4.3.6
      - pooch==1.8.2
      - portalocker==2.10.1
      - prometheus-client==0.21.0
      - prompt-toolkit==3.0.48
      - propcache==0.2.0
      - protobuf==3.20.3
      - psutil==6.1.0
      - ptyprocess==0.7.0
      - pure-eval==0.2.3
      - pyasn1==0.6.1
      - pyasn1-modules==0.4.1
      - pybind11==2.13.6
      - pycparser==2.22
      - pydantic==2.9.2
      - pydantic-core==2.23.4
      - pydeprecate==0.3.1
      - pydub==0.25.1
      - pyembree==0.2.11
      - pygments==2.18.0
      - pymatting==1.1.12
      - pyparsing==3.1.4
      - python-dateutil==2.9.0.post0
      - python-json-logger==2.0.7
      - python-multipart==0.0.17
      - pytorch-lightning==1.5.0
      - pytz==2024.2
      - pywavelets==1.4.1
      - pyyaml==6.0.2
      - pyzmq==26.2.0
      - referencing==0.35.1
      - regex==2024.11.6
      - rembg==2.0.59
      - requests==2.32.3
      - requests-oauthlib==2.0.0
      - rfc3339-validator==0.1.4
      - rfc3986-validator==0.1.1
      - rich==13.9.4
      - rpds-py==0.20.1
      - rsa==4.9
      - rtree==1.3.0
      - ruff==0.7.2
      - s3transfer==0.10.3
      - safetensors==0.4.5
      - scikit-image==0.21.0
      - scikit-learn==1.3.2
      - scipy==1.10.1
      - semantic-version==2.10.0
      - send2trash==1.8.3
      - sentry-sdk==2.18.0
      - setproctitle==1.3.3
      - setuptools==60.10.0
      - shapely==2.0.6
      - shellingham==1.5.4
      - six==1.16.0
      - smmap==5.0.1
      - smplx==0.1.28
      - sniffio==1.3.1
      - sounddevice==0.5.1
      - soupsieve==2.6
      - stack-data==0.6.3
      - starlette==0.38.6
      - sympy==1.13.1
      - tabulate==0.9.0
      - tensorboard==2.14.0
      - tensorboard-data-server==0.7.2
      - termcolor==2.4.0
      - terminado==0.18.1
      - threadpoolctl==3.5.0
      - tifffile==2023.7.10
      - tinycss2==1.4.0
      - tinyobjloader==0.0.0
      - tokenizers==0.20.3
      - tomli==2.0.2
      - tomlkit==0.12.0
      - torch==2.1.0+cu121
      - torchaudio==2.1.0+cu121
      - torchfile==0.1.0
      - torchmetrics==1.5.1
      - torchvision==0.16.0+cu121
      - tornado==6.4.1
      - tqdm==4.67.0
      - traitlets==5.14.3
      - transformers==4.46.2
      - trimesh==3.23.5
      - triton==2.1.0
      - typer==0.12.5
      - types-python-dateutil==2.9.0.20241003
      - typing-extensions==4.12.2
      - tzdata==2024.2
      - uri-template==1.3.0
      - urllib3==2.2.3
      - uvicorn==0.32.0
      - wandb==0.18.6
      - wcwidth==0.2.13
      - webcolors==24.8.0
      - webencodings==0.5.1
      - websocket-client==1.8.0
      - websockets==11.0.3
      - werkzeug==3.0.6
      - wheel==0.37.1
      - widgetsnbextension==4.0.13
      - yacs==0.1.8
      - yarl==1.15.2
      - zerorpc==0.6.3
      - zipp==3.20.2
      - zope-event==5.0
      - zope-interface==7.1.1
prefix: /home/aiart/.conda/envs/shert