Upgrade website to Docsy 0.5.1 #531

SayakMukhopadhyay · 2024-09-21T15:02:35Z

~~This PR updates Docsy to the latest version available and makes it compatible to use with the latest Hugo.~~

This PR was initially made to be upgraded to Docsy 0.10.0 but has since been modified to upgrade only upto 0.5.1. Further upgrade will be done in another PR. I have crossed out the items that was planned in this PR but won'
t happen in this PR either due to being fixed by another PR or being siloed to another PR.

In details, the following are the changes:

Updated the docsy submodule to use ~~v0.10.0 tag~~ v0.5.1 tag
~~Updated dependencies in the package.json to the latest autoprefixer and postcss-cli and added postcss which is needed by postcss-cli now.~~
~~Update the Hugo version in netlify to 0.134.3 and~~ added NODE_VERSION = 20.16.0 env var to force netlify to use the current LTS of Node.js as older versions won't be able to compile native code in some of the dependencies. (If this value is configured on the infra side, we should look into either updating the value there and remove it from here, or remove it from infra and keep it here.)
Update the Makefile to add a target to download the dependencies of docsy. ~~(WIP: right now, only added as a dependency of the preview-build target to get the CI in the PR to work)~~
Partials removed:
- head.html: it was the same as the theme except without the _internal/google_news.html template. This template has since been removed.
- navbar.html: it was the same as the theme except removing the minification of the navbar logo due to Hugo not having tdewolff/minify >= 2.7.3. The latest Hugo has v2.20.37 so that removal is not needed.
- sidebar-html: it was the same as the theme except wrapping an or conditional method with cond method to work with newer versions of Hugo which Docsy was not supporting. The latest Docsy version does support the latest version of Hugo and this wrapping is no longer needed.
Layouts removed:
- docs & layout: this layout was an exact copy from the theme with the only difference being the page-meta-lastmod.html partial was removed in the override. But the partial itself doesn't activate unless both .GitInfo and Site.Params.github_repo are truthy, which is not, hence the override is not needed.
- events: this layout was unnecessary as all the content files in the events folder are using type: docs in its front matter and so the events layout will never be used. And anyway, docs layout and events layout were exactly the same.
Layouts updated:
- calendar: updated to the current docs layout provided by the theme while keeping the customization of including some scripts.
- community: updated to the current docs layout provided by the theme as it was the same before. (The only reason this layout is needed is because all generated files in content/en/community doesn't contain the type or layout in its front matter and thus will default to type: page if a community layout is not created.)
- resources: updated to the current docs layout provided by the theme as it was the same before. (The only reason this layout is needed is because of a generated file content/en/resource/release/index.md which doesn't contain the type or layout in its front matter and thus will default to type: page if a resources layout is not created.)
_variables.scss has been updated from the theme. This change is introduced by docsy updating to bootstrap 5 in 0.7.0. This change introduces multiple changes to the default layout, including margins, padding, colors, font sizes among others. _variables.scss has been replace by _variables_project.scss which is the recommended way to customize the variables of Docsy. This file only lists the properties that need to be customized and not everything in Docsy's _variables.scss
~~_styles_project.scss has been added to include the CSS to make the copyright statement visible. This is the file that all customisations should go to.~~
~~In the cover page, the color parameter is removed else it is setting the wrong text color.~~
~~The Hugo config has been changed from TOML to YAML~~ (This has been done in feat: move from a toml hugo config to a yaml one #556). ~~Some properties which are redundant have also been commented out. Also, since FontAwesome was upgraded to v6 in Docsy, the icon names have been updated. Also fixes Update X logo to 𝕏 on the website's footer #503 ~~
Migrated from using Docsy as a Git submodule to using it as an NPM module. Fixes Migrate to using Docsy as an npm module #557. This migration also updates the Makefile to not use any git submodule commands and update the Dockerfile to use npm ci to install the dependencies instead of installing them globally.

k8s-ci-robot · 2024-09-21T15:02:41Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: SayakMukhopadhyay
Once this PR has been reviewed and has the lgtm label, please assign jberkus for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2024-09-21T15:02:44Z

Welcome @SayakMukhopadhyay!

It looks like this is your first PR to kubernetes/contributor-site 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/contributor-site has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

SayakMukhopadhyay · 2024-09-22T14:41:34Z

The biggest "change" in this upgrade is the fact that Bootstrap was updated to v5 by Docsy. This has led to changes to stylistic elements, mostly padding, margins, text sizes and some colours. I will try to document some of them here. (Left is the current site and right is the PR site)

The page title is no longer capitalised by default (see navbar title follows config.toml file (removes uppercase by default) google/docsy#797) but the title and menu items are now aligned.
In the welcome page, margins are calculated based on screensize instead of being harcoded in bootstrap. Should these margins be changed to reflect the current state? Do note, the current site is using defaults. Also, the link colours are a bit different.
The blog list also has very minor margin differences. Also, note the slight difference in the link colours. Another difference that is present in all list pages is that the left hand side navigation looks a little different with an underline below the page title instead of the items being indented and the search box being squarish.
Disabled buttons in pagination also have a different colour
Feedback buttons look a bit different

These are all that I have found out. I am not too sure if there is a need to make the upgrade look pixel-accurate to the current site.

Also, I have another branch that adds some changes that might need more discussion. These changes include

~~Moving to YAML config for Hugo instead of TOML as Docsy itself has moved and well...YAML is the language of K8s right 😄~~ (This has been done in feat: move from a toml hugo config to a yaml one #556) ? Also, while moving, I figured out which config variables are redundant due to them just repeating the default values. Removing those properties can help make it more maintainable. I have commented those right now but depending on the consensus, I can remove them.
Deletion of _variables.scss as it just redefines values in the Docsy theme with maybe a couple of changes. Instead, I have added _variables_project.scss which is the recommended way to customise variables by Docsy. This file only has what is needed to be overridden.

If the maintainers agree with the above changes, I will include them in this PR.

SayakMukhopadhyay · 2024-09-22T15:17:56Z

/assign @SayakMukhopadhyay

tokt · 2024-09-23T03:35:57Z

Thank you so much!! This will take awhile to review, sorry.
Personally: +1 on moving from toml to yaml and also on deletion of _variables.scss.

SayakMukhopadhyay · 2024-09-23T05:45:09Z

Alright, I will bring in those changes from the other branch to make it easier to review. The config file will contain commented properties which are not needed but I have kept them for now to make it easier to review the YAML->TOML change. I can remove them before merging this PR once the format change itself is reviewed.

chris-short · 2024-09-27T20:29:40Z

This is amazing! Thank you so much. As mentioned, we'll need some time to review all this but, I greatly appreciate the time, effort, and energy.

SayakMukhopadhyay · 2024-09-28T08:39:11Z

I have tried to make the commits as atomic as possible so if any changes are required, we can drop those commits.

assets/scss/_variables_project.scss

sftim · 2024-09-28T09:28:23Z

@SayakMukhopadhyay an aside: if you'd be willing to work on part of the equivalent change for https://k8s.io/, I can make time to pair up with you on that.

I'm @sftim on Kubernetes' Slack workspace.

sftim · 2024-09-28T09:29:14Z

Makefile

+dependencies:
+	npm ci
+	cd themes/docsy/ && npm i


Does this need adapting for make container-server?

Yeah, the Makefile needs to be updated. The dependencies target itself is ok but needs to be added as dependencies on docker-server, container-server and production-build. I have not added them as I wanted to know the sentiment regarding overhauling the Makefile. There is a lot of repetition (like git submodule update --init --recursive --depth 1) that can be moved to its own target and used as a dependency. Let me know if its ok to do that and I will make the changes to this PR.

Where possible, split out refactorings like that into their own (separate, smaller) PR.

Which is why I haven't made any changes to the Makefile. The dependencies target though is necessary for the latest Docsy and it was needed to get the preview-build target to work and will be needed for the other targets. Essentially, that command creates 2 folders inside themes, FortAwesome and twbs.

Does this PR break local container previewing? If so, let's update the README and call out the issue in the PR description.

By local container previewing, do you mean using make container-server? It doesn't work in master itself. It seems there is a permission issue on the /tmp folder. The following error is thrown:

...
rsync: [receiver] chown "/tmp/src/themes/docsy/userguide/static/images/.version-banner.png.nPONje" failed: Operation not permitted (1)
rsync: [receiver] chown "/tmp/src/themes/docsy/userguide/static/images/.yes.png.iMPnje" failed: Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.3.0]
make: *** [Makefile:80: container-server] Error 23

This issue came up in WSL and I tried a fresh setup in codespaces and the issue is the same. It seems like the issue dooesn't happen if I remove this line

contributor-site/Makefile

Line 83 in f42204c

--cap-drop=ALL \

But running the docker commands manually works as expected and so local container previewing still works.

@sftim From your comment, I dunno how this convo got resolved but yeah, container-server target in the makefile hasn't been updated. In fact, as mentioned in the convo, all the targets needs to be updated to handle the npm dependency. See this message to know my primary concerns.

There are multiple ways this can be approached.

I can make the changes in the Makefile including adding the dependencies and can keep the refactoring for later.

I can add the dependencies and do the refactoring.

I can do the refactoring in a separate PR to be merged first and fix the dependencies here so that you can get it working.

For now, for you to test things, I will push a commit adding the dependant target to container-server.

Also, the rsync permission issue is still there in my system. I have pushed a commit that should get container-server working. If it doesn't try removing lines 87 and 88 (--cap-drop=ALL and --cap-drop=AUDIT_WRITE). Because I can't get things to work without removing them.

I don't know the real issue behind this as I assume it used to work.

We only support container previewing on Linux and macOS host OSs; if you're using another environment, the support may not have been added yet.

Hmm...this permission issue occurs on GitHub Codespaces too. But this PR as of now has a new commit that updates the container-server target. Can you check if that works for your local host?

I will split this PR into 2 tomorrow.

sftim · 2024-09-28T09:30:50Z

hugo.yaml

+        desc: Discussion and help from your fellow users
+      - name: Twitter
+        url: 'https://twitter.com/K8sContributors'
+        icon: fa-brands fa-x-twitter # Updated the bird icon to the X icon


Omit this comment, but once you have, comment on this line in your own PR to call out the change.

I have removed the comment from the file. Where do you want me to recomment? I have already called out this change in the last line of this PR's description.

Also, what about the other properties that I have commented in hugo.yaml. I have kept those properties commented on to get some feedback, should I remove those properties altogether or should I uncomment them.

netlify.toml

sftim · 2024-09-28T09:33:37Z

community: updated to the current docs layout provided by the theme as it was the same before.
(The only reason this layout is needed is because all generated files in content/en/community doesn't contain the type or layout in its front matter and thus will default to type: page if a community layout is not created.)

Could we add that front matter via the generator? Maybe in a separate PR (which I'd prefer to merge ahead of this one)

SayakMukhopadhyay · 2024-09-28T10:22:45Z

community: updated to the current docs layout provided by the theme as it was the same before.
(The only reason this layout is needed is because all generated files in content/en/community doesn't contain the type or layout in its front matter and thus will default to type: page if a community layout is not created.)

Could we add that front matter via the generator? Maybe in a separate PR (which I'd prefer to merge ahead of this one)

I plan to work on it but I was thinking of doing it in a separate PR as the current workaround works for now.

hugo.toml

…gets

SayakMukhopadhyay · 2024-12-23T17:04:39Z

/retitle Upgrade website to Docsy 0.5.1

k8s-ci-robot · 2024-12-23T17:04:48Z

@SayakMukhopadhyay: Re-titling can only be requested by trusted users, like repository collaborators.

In response to this:

/retitle Upgrade website to Docsy 0.5.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

SayakMukhopadhyay · 2024-12-23T18:13:52Z

Side note: the value of $gray-800 used to be #888 but was updated to #797676 to improve accessibility. This means that even though _variables.scss in the project would be different from the _variables.scss, it's only becuase it wasn't updated. See google/docsy#540

…ying over and override only the properties that are required

SayakMukhopadhyay · 2024-12-23T18:23:09Z

@sftim This PR has been modified to upgrade only upto 0.5.1. This PR also has an update to the Makefile to ensure that the docsy dependencies are installed on make commands. Once this PR is merged, I will work on #557 which would make things a bit simpler as far as the Makefile is concerned.

I can squash this PR down to a commit if you want. Let me know.

sftim · 2024-12-23T18:24:36Z

Makefile

 	@echo -e "**** The use of docker-server is deprecated. Use container-server instead. ****" 1>&2
 	$(MAKE) container-server

-container-server: ## Run Hugo locally within a container, available at http://localhost:1313/
+container-server: dependencies ## Run Hugo locally within a container, available at http://localhost:1313/


This is actually:

-container-server: dependencies ## Run Hugo locally within a container, available at http://localhost:1313/ +container-server: dependencies ## Run npm locally on your host computer; then run Hugo locally within a container, available at http://localhost:1313/

I'd prefer to see the approach from k/website where NPM doesn't run locally; one of the benefits of a container is isolating less trusted code, whereas this approach executes it on the host.

I have done it this way as other non-container targets would need to do this on the host anyway. Moreover, even container-server executes git submodule update --init --recursive --depth 1 in the host which too could be done in the container. I am following that style in this PR.

This will need to be reworked anyway when we migrate from submodules to npm modules and I suggest that this change be done then.

We can assume that people trust git, but we should minimize the other code they need to trust.

I'm really reluctant to go the other way and widen what contributors must trust.

@sftim I have made the changes. This PR now also closes #557 and all npm commands run inside the container and not on the host.

My only question now is for the render and server commands. Since these 2 commands run on the host anyway, do you think it's ok to add npm ci to these commands? Or should the fact that npm ci needs to be run manually be added to the docs and we leave it up to the folks who clone this repo to run npm commands?

Yes, those are fine to run on the host.

assets/scss/_variables_project.scss

sftim · 2024-12-24T12:55:28Z

The container image doesn't build for me:

STEP 5/8: RUN npm ci
npm ERR! code EUSAGE
npm ERR!
npm ERR! The `npm ci` command can only install with an existing package-lock.json or
npm ERR! npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm@5 or
npm ERR! later to generate a package-lock.json file, then try again.
npm ERR!
npm ERR! Clean install a project
npm ERR!
npm ERR! Usage:
npm ERR! npm ci
npm ERR!
npm ERR! Options:
npm ERR! [--install-strategy <hoisted|nested|shallow|linked>] [--legacy-bundling]
npm ERR! [--global-style] [--omit <dev|optional|peer> [--omit <dev|optional|peer> ...]]
npm ERR! [--include <prod|dev|optional|peer> [--include <prod|dev|optional|peer> ...]]
npm ERR! [--strict-peer-deps] [--foreground-scripts] [--ignore-scripts] [--no-audit]
npm ERR! [--no-bin-links] [--no-fund] [--dry-run]
npm ERR! [-w|--workspace <workspace-name> [-w|--workspace <workspace-name> ...]]
npm ERR! [-ws|--workspaces] [--include-workspace-root] [--install-links]
npm ERR!
npm ERR! aliases: clean-install, ic, install-clean, isntall-clean
npm ERR!
npm ERR! Run "npm help ci" for more info

npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-12-24T12_50_27_990Z-debug-0.log
Error: building at STEP "RUN npm ci": while running runtime: exit status 1
make: *** [Makefile:55: container-image] Error 1

Try cloning your repo (you can pull from a filesystem clone to save bandwidth) and check that a container image build runs OK from that clean clone @SayakMukhopadhyay

Once it does, feel free to squash to 1 or 2 commits.

SayakMukhopadhyay · 2024-12-24T13:03:03Z

@sftim would it be ok, if I COPY over the package.json and package-lock.json over to the Dockerfile. I could do npm i autoprefixer ... again but I think its redundant to list down the required packages in the Dockerfile when they are already there in the package.json.

sftim · 2024-12-24T13:04:54Z

would it be ok, if I COPY over the package.json and package-lock.json over to the Dockerfile.

Sure, that's a really common technique for this situation.

…e deps

SayakMukhopadhyay · 2024-12-24T13:13:37Z

@sftim It's fixed. Do you think there is any value in checking in a CI step that the container builds and runs?

sftim · 2024-12-24T13:33:32Z

Do you think there is any value in checking in a CI step that the container builds and runs?

One for a follow-up PR, and maybe best if we involve SIG Testing folks.

sftim · 2024-12-24T13:33:35Z

Please squash this down to fewer commits; I think that's tidiest.

sftim

This is clear progress.

Make sure that, once you've built an image, make container-server runs without an error.

Warnings are OK, but it's good to fix those too.

…eplaced --verbose with --logLevel info for hugo

SayakMukhopadhyay · 2024-12-24T15:37:44Z

Alright, the couple of warnings have been addressed. I will squash this PR to 3 commits, 1 containing all the upgrades, 1 containing the submodule -> npm change and the last containing the misc changes.

sftim

A local preview doesn't work for me. Try removing node_modules before you test @SayakMukhopadhyay

sftim · 2024-12-24T15:46:45Z

Dockerfile

-    postcss-cli
+COPY package*.json ./
+
+RUN npm ci


I think this will create /src/node_modules, but when the repository is mounted over that, the node modules will be shadowed / hidden.

Welp....that's right. Maybe I should move the npm ci into hack/gen-content.sh.

Seems like k8s/website too has this issue.

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 21, 2024

k8s-ci-robot requested review from reylejano and sftim September 21, 2024 15:02

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Sep 21, 2024

k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Sep 21, 2024

SayakMukhopadhyay force-pushed the upgrade branch 4 times, most recently from bba8ab8 to dce4971 Compare September 22, 2024 11:30

SayakMukhopadhyay marked this pull request as ready for review September 22, 2024 14:57

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 22, 2024

k8s-ci-robot requested review from jeefy and tokt September 22, 2024 14:57

k8s-ci-robot assigned SayakMukhopadhyay Sep 22, 2024

sftim reviewed Sep 28, 2024

View reviewed changes

assets/scss/_variables_project.scss Show resolved Hide resolved

sftim reviewed Sep 28, 2024

View reviewed changes

netlify.toml Outdated Show resolved Hide resolved

sftim reviewed Nov 4, 2024

View reviewed changes

hugo.toml Outdated Show resolved Hide resolved

SayakMukhopadhyay force-pushed the upgrade branch from 2bf59ca to 381862d Compare December 23, 2024 13:15

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Dec 23, 2024

SayakMukhopadhyay added 2 commits December 23, 2024 20:48

feat: add docsy dependency installations as a dependency of build tar…

6517167

…gets

ci: force to use a compatible node version

28ef958

SayakMukhopadhyay changed the title ~~Upgrade website to Docsy 0.10.0 and Hugo 0.134~~ Upgrade website to Docsy 0.5.1 Dec 23, 2024

Use docsy recommended way of overriding SCSS variables instead of cop…

42b9580

…ying over and override only the properties that are required

sftim reviewed Dec 23, 2024

View reviewed changes

assets/scss/_variables_project.scss Show resolved Hide resolved

feat: migrate docsy from using as a submodule to an npm module

60a420c

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 24, 2024

SayakMukhopadhyay added 2 commits December 24, 2024 18:05

feat: rework Makefile and Dockerfile to exclude git submodule operations

472dff6

feat: remove unnecessary .gitmodules file

6f0fe7d

feat: add npm ci as a dependant command for local execution

7ad66f9

fix: make sure that the package files are copied before installing th…

67237af

…e deps

sftim reviewed Dec 24, 2024

View reviewed changes

chore: remove taxonomyTerm as a disabled Kind as its deprecated and r…

18f03d8

…eplaced --verbose with --logLevel info for hugo

sftim reviewed Dec 24, 2024

View reviewed changes

Upgrade website to Docsy 0.5.1 #531

Are you sure you want to change the base?

Upgrade website to Docsy 0.5.1 #531

Conversation

SayakMukhopadhyay commented Sep 21, 2024 • edited Loading

k8s-ci-robot commented Sep 21, 2024

k8s-ci-robot commented Sep 21, 2024

SayakMukhopadhyay commented Sep 22, 2024 • edited Loading

SayakMukhopadhyay commented Sep 22, 2024

tokt commented Sep 23, 2024

SayakMukhopadhyay commented Sep 23, 2024

chris-short commented Sep 27, 2024

SayakMukhopadhyay commented Sep 28, 2024

sftim commented Sep 28, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SayakMukhopadhyay Sep 28, 2024 • edited Loading

Choose a reason for hiding this comment

sftim commented Sep 28, 2024

SayakMukhopadhyay commented Sep 28, 2024

SayakMukhopadhyay commented Dec 23, 2024

k8s-ci-robot commented Dec 23, 2024

SayakMukhopadhyay commented Dec 23, 2024

SayakMukhopadhyay commented Dec 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sftim commented Dec 24, 2024

SayakMukhopadhyay commented Dec 24, 2024

sftim commented Dec 24, 2024

SayakMukhopadhyay commented Dec 24, 2024

sftim commented Dec 24, 2024

sftim commented Dec 24, 2024

sftim left a comment

Choose a reason for hiding this comment

SayakMukhopadhyay commented Dec 24, 2024

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SayakMukhopadhyay commented Sep 21, 2024 •

edited

Loading

SayakMukhopadhyay commented Sep 22, 2024 •

edited

Loading

sftim commented Sep 28, 2024 •

edited

Loading

SayakMukhopadhyay Sep 28, 2024 •

edited

Loading