Dynamic fetching of script info based on Metadata JSON files

[BramSuurdje]

Proposal for Script Metadata Management

• Objective: To implement dynamic metadata fetching when a user views a script on a static site, ensuring accurate and up-to-date information without manual refreshes.

Approach

1. Dynamic Fetching:

   • Scripts should fetch metadata dynamically when viewed.
   • This may be challenging with a static site, so I’ll explore implementation options to determine feasibility.

2. Static Metadata Storage:

   • Core metadata can be stored in the ProxmoxVE repository as it changes infrequently.
   • Metadata for each script is structured in a JSON file as follows:

{
  "slug": "nginxproxymanager",
  "logo": "https://raw.githubusercontent.com/loganmarchione/homelab-svg-assets/main/assets/nginxproxymanager.svg",
  "description": "Nginx Proxy Manager is a tool that provides a web-based interface to manage Nginx reverse proxies. It enables users to easily and securely expose their services to the internet by providing features such as HTTPS encryption, domain mapping, and access control. It eliminates the need for manual configuration of Nginx reverse proxies, making it easy for users to quickly and securely expose their services to the public.",
  "date_created": "2024-05-02",
  "website": "https://nginxproxymanager.com/",
  "documentation": "",
  "default_credentials": {
    "username": "admin",
    "password": "admin"
  },
  "alerts": [
    {
      "alert": "Since there are hundreds of Certbot instances, it's necessary to install the specific Certbot of your preference."
    },
    {
      "alert": "This is another example of an alert."
    }
  ]
}

3. Metadata API Route:

   • Create an API route that, during the build process, fetches all JSON files in the metadata directory.
   • The API will iterate through each script in the scripts directory and match it with its corresponding metadata.

4. Final JSON Structure:

   • For each script, the resulting JSON object will contain all relevant details:

{
  "name": "Nginx Proxy Manager",
  "slug": "nginxproxymanager",
  "logo": "https://raw.githubusercontent.com/loganmarchione/homelab-svg-assets/main/assets/nginxproxymanager.svg",
  "description": "Nginx Proxy Manager is a tool that provides a web-based interface to manage Nginx reverse proxies. It enables users to easily and securely expose their services to the internet by providing features such as HTTPS encryption, domain mapping, and access control. It eliminates the need for manual configuration of Nginx reverse proxies, making it easy for users to quickly and securely expose their services to the public.",
  "date_created": "2024-05-02",
  "type": "LXC",
  "updateable": true,
  "website": "https://nginxproxymanager.com/",
  "documentation": "",
  "interface_port": "81",
  "install_methods": [
    {
      "type": "default",
      "resources": {
        "cpu": 1,
        "ram": 1024,
        "hdd": 3
      }
    }
  ],
  "default_credentials": {
    "username": "admin",
    "password": "admin"
  },
  "alerts": [
    {
      "alert": "Since there are hundreds of Certbot instances, it's necessary to install the specific Certbot of your preference."
    }
  ]
}

5. Automation:
   • A nightly GitHub Action (cron job) can update the metadata to ensure it remains current.

This structure should allow for both static storage of rarely-changing data and dynamic updates for accuracy in the user interface.

[BramSuurdje]

@havardthom Could you take a look at this and tell me what you think? If i have to go more in-depth on the specifics on how it would retrieve the information let me know.

[havardthom]

Interesting approaches, I don't think we need Dynamic Fetching since the documentation will change relatively seldom. What about storing the json files in this repo? Then the github action will build and deploy every time they change and we only need your 5th point. The action could be extended to update the json files on releases/changes in ProxmoxVE.

[newzealandpaul]

@havardthom that was what I was going to suggest.

@BramSuurdje regarding point #4. What would the JSON structure look like if there was two possible installation methods eg. zigbee2mqtt and alpine-zigbee2mqtt?

[BramSuurdje]

@newzealandpaul the json structure would look something like the following:

{
    "name": "Grafana",
    "slug": "grafana",
    "category": "8",
    "date_created": "2024-05-02",
    "type": "LXC",
    "updateable": true,
    "privileged": false,
    "interface_port": "3000",
    "documentation": "",
    "website": "https://grafana.com/",
    "logo": "https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fdocs.checkmk.com%2Flatest%2Fimages%2Fgrafana_logo.png&f=1&nofb=1",
    "description": "Grafana is a data visualization and monitoring platform that enables users to query, visualize, alert on and understand metrics, logs, and other data sources. It integrates with various data sources, including Prometheus, InfluxDB, Elasticsearch, and many others, to present a unified view of the data and enable users to create insightful and interactive dashboards.",
    "install_methods": [
      {
        "type": "default",
        "installer": "ct/grafana.sh",
        "resources": {
          "cpu": 1,
          "ram": 2048,
          "hdd": 2,
          "os": "debian",
          "version": "12"
        }
      },
      {
        "type": "alpine",
        "installer": "ct/alpine-grafana.sh",
        "resources": {
          "cpu": 1,
          "ram": 256,
          "hdd": 1,
          "os": "alpine",
          "version": "3.19"
        }
      }
    ],
    "default_credentials": {
      "username": "admin",
      "password": "admin"
    },
    "alerts": [
      {
        "alert": "Example necessary information about install/product here"
      }
    ]
}

contains all necessary information to render the script on the website

can show multiple variations of installer, each with their own config and installer path

can build installer url

bash -c "$(wget -qLO - https://github.com/community-scripts/ProxmoxVE/raw/main/${installer})"

would require a separate categories list as such

{
  "1": "Proxmox VE Tools",
  "2": "AdBlocker - DNS",
  "3": "Automation",
  [...]
  "8": "Monitoring - Analytics",
}

[newzealandpaul]

@BramSuurdje this has been implemented in community-scripts/ProxmoxVE#86

I think you can close this discussion now.