Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve UX of the new command #4154

Open
dgzlopes opened this issue Dec 27, 2024 · 4 comments
Open

Improve UX of the new command #4154

dgzlopes opened this issue Dec 27, 2024 · 4 comments
Assignees

Comments

@dgzlopes
Copy link
Member

dgzlopes commented Dec 27, 2024

Feature Description

The k6 new command was introduced to:

  • Simplify the process of creating new k6 scripts.
  • Teach users about k6's capabilities (incl. Cloud integration and browser support).

After seeing it used in many places, I have detected some problems:

  • It doesn't work well with the Cloud flows.
    • On our Cloud onboarding, in some cases, we tell the user to generate the script, open it, and change things manually.
      • Image
    • In others, we ask the user to copy-paste a script directly instead of using this command (e.g., new Test Run page).
      • Image
  • The current example script tries to do too much without being able to be minimal enough or complete enough.
    • It is a protocol test with a massive commented config block that always includes Cloud, Browser, etc.
    • Even if you have Browser in the options block... there is no browser code in the test logic.

Suggested Solution (optional)

  • Having two 'new' commands
    • k6 new
    • k6 cloud new
      • The Cloud block will appear in the config block of the generated test when you use this one.
      • Also, optionally, users can pass --project-id to override the dummy one.
        • This would solve the "manual changes" required step of Cloud onboarding. We can tell users to copy-paste:
          • k6 cloud new --project-id 123145
  • Having templates instead of one massive example (note: I was inspired by how Svelte handles templates).
    • Usage: k6 cloud new --project-id 123145 --template=browser

Minimal (default) - Small and concise script. It is ideal for folks who need a starting point for a new project

import http from 'k6/http';
import { sleep, check } from 'k6';

export const options = {
  vus: 10,
  duration: '30s',
};

export default function() {
  let res = http.get('https://quickpizza.grafana.com');
  check(res, { "status is 200": (res) => res.status === 200 });
  sleep(1);
}

Protocol - More real script. It is ideal for folks learning, as a reference, or for demos.

import http from "k6/http";
import { check, sleep } from "k6";

const BASE_URL = __ENV.BASE_URL || 'https://quickpizza.grafana.com';

export const options = {
  vus: 10,
  stages: [
    { duration: "10s", target: 5 },
    { duration: "20s", target: 10 },
    { duration: "1s", target: 0 },
  ], thresholds: {
    http_req_failed: ["rate<0.01"],
    http_req_duration: ["p(95)<500", "p(99)<1000"],
  },
};

export function setup() {
  let res = http.get(BASE_URL);
  if (res.status !== 200) {
    throw new Error(
      `Got unexpected status code ${res.status} when trying to setup. Exiting.`
    );
  }
}

export default function() {
  let restrictions = {
    maxCaloriesPerSlice: 500,
    mustBeVegetarian: false,
    excludedIngredients: ["pepperoni"],
    excludedTools: ["knife"],
    maxNumberOfToppings: 6,
    minNumberOfToppings: 2
  };

  let res = http.post(BASE_URL + "/api/pizza", JSON.stringify(restrictions), {
    headers: {
      'Content-Type': 'application/json',
      'X-User-ID': 23423,
    },
  });

  check(res, { "status is 200": (res) => res.status === 200 });
  console.log(res.json().pizza.name + " (" + res.json().pizza.ingredients.length + " ingredients)");
  sleep(1);
}

Browser - Same as Protocol but with Browser.

import { browser } from "k6/browser";
import http from "k6/http";
import { sleep, check } from 'k6';

const BASE_URL = __ENV.BASE_URL || "https://quickpizza.grafana.com";

export const options = {
  scenarios: {
    ui: {
      executor: "shared-iterations",
      options: {
        browser: {
          type: "chromium",
        },
      },
    },
  }, 
};

export function setup() {
  let res = http.get(BASE_URL);
  if (res.status !== 200) {
    throw new Error(
      `Got unexpected status code ${res.status} when trying to setup. Exiting.`
    );
  }
}

export default async function() {
  let checkData;
  const page = await browser.newPage();
  try {
    await page.goto(BASE_URL);
    checkData = await page.locator("h1").textContent();
    check(page, {
      header: checkData === "Looking to break out of your pizza routine?",
    });
    await page.locator('//button[. = "Pizza, Please!"]').click();
    await page.waitForTimeout(500);
    await page.screenshot({ path: "screenshot.png" });
    checkData = await page.locator("div#recommendations").textContent();
    check(page, {
      recommendation: checkData !== "",
    });
  } finally {
    await page.close();
  }
  sleep(1);
}

(Future: We could let people leverage external templates)

Already existing or connected issues / PRs (optional)

#4153

@andrewslotin
Copy link
Contributor

andrewslotin commented Dec 27, 2024

I really like the templates idea, but the fact that k6 cloud new does not create a new test in GCk6, but initializes a new test file locally instead might be confusing. The intended purpose of k6 cloud subcommand is to access and manage resources in Grafana Cloud k6. Would it make sense to change the existing k6 new command to accept optional --project-id flag instead and populate the options field in case it has been provided?

@joanlopez
Copy link
Contributor

Generally speaking, I also think that a simple templates mechanism would be needed here. Indeed, I think we even discussed that specifically at time of releasing k6 new, as likely the next step to make that feature more powerful. And, in that case, I would see the "cloud" variant as yet another flavour, as "protocol", "browser", and others could be, either complementarily or mutually exclusively.


That said,

the fact that k6 cloud new does not create a new test in GCk6, but initializes a new test file locally instead might be confusing

I agree with @andrewslotin that k6 cloud new could be confusing, especially now that we have certain plans to start adding more subcommands to k6 cloud, and all of them involve interaction with cloud.

@dgzlopes
Copy link
Member Author

I agree with @andrewslotin that k6 cloud new could be confusing, especially now that we have certain plans to start adding more subcommands to k6 cloud, and all of them involve interaction with cloud.

Yup, makes sense! Thoughts on this @oleiade?

And, in that case, I would see the "cloud" variant as yet another flavour, as "protocol", "browser", and others could be, either complementarily or mutually exclusively.

I'm not really sure about this. It seems a bit redundant to have a Cloud template when any of the existing templates can be converted (rather easily) to a Cloud test.

@joanlopez
Copy link
Contributor

joanlopez commented Dec 30, 2024

I'm not really sure about this. It seems a bit redundant to have a Cloud template when any of the existing templates can be converted (rather easily) to a Cloud test.

Sorry, I wasn't clear enough. I didn't want to mean exactly like browser or protocol, but some option you can set on the templating system. Like with X or Y, where one of these would be with cloud.

I explained it like this because I wanted to point to the fact that I see it as another option of the same templating system, and not a "completely separated" thing (like a different (sub)command).

In essence, how I envision it , I think it's similar to your draft PR, except for the sub-command and its naming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants