More interactive commands

[psionic-k]

Project-based test discovery, completions dispatch of individual tests. The run-at-point command is neat, but completions to run suites / specs while developing would be more useful to avoid needing to get the point back at the test.

[snogge]

One of the problems with the current way of dealing with tests is that every time you evaluate a describemacro, that complete suite with all of its sub-suites and cases is added to the list of suites with no duplication check. The buttercup-run-at-point command avoids this by using a separate list of suites. So if you develop code and tests at the same time, you will accumulate old versions of the tests in the buttercup-suites variable. This is not really helpful.
So unless you need to debug a function I recommend using the compile command to run your tests in a separate Emacs instance.
That's not to say we cannot try to improve the situation, but I'd like to see some real suggestions on how you want these commands should be used. Ideas for implementation are also welcome.
I will not close this issue as I would really like to make this aspect of buttercup better, but it is not an easy thing to do without rethinking parts of the buttercup implementation.

[psionic-k]

I'm diving into seriously elisp development. I lack context. I will endlessly invest in team productivity.

99% of use cases are run all the tests or run just one test. However, running the test at point is a bit cumbersome compared to picking it out of completions. Manual reloading is the other big workflow tax.

I think rebuilding the list of suites is fine. The time spent loading will almost always be less than time saved by automation. To avoid clobbering when using multiple projects, a single per-project buffer with buffer locals could be used. The issue then is just how to find the tests & features and then reload the feature under test and tests themselves.

Figuring out load order exactly would require building a feature DAG by looking at a project. A heuristic + per-project settings via variables could probably do as well, especially with completions for features and using features to discover where they were loaded from.

In addition, since all (requires 'feature) statements within a package should just work without extra load-path configuration, finding the root feature might not even be an issue.. I'm afraid of tricky load order scenarios, but the only dependency I can think of is tests depending on the feature they test. Cases of improper package construction should be caught by compile linting before the user screws it up.

CC @alphapapa no idea where discovery + reload code needs to live eventually, but it seems generally useful. All of these linters and test runners need to be able to discover and reload the project feature / elisp files. I think I will make some contributions to unpackaged and duplicate useful bits to elisp-repo-kit for turnaround time.

I just discovered yesterday that let-binding load-prefer-newer as another way to pick up changes to updates with -compile-and-load commands. Not sure yet how reliable this is compared to explicit reload.

Because the user's own interactive state might be clobbered by reloading features that are in use, (the bootstrap problem for all tooling), it's reasonable to want to run tests in a separate Emacs when true purity or isolation from the user's own use of Emacs is needed. Support for opting out of automatic feature reloading may be useful, again using some project setting like so: https://emacs.stackexchange.com/questions/691/how-can-i-customize-the-compile-command

[psionic-k]

I was destroyed by some issue that only appears with a byte compiled loaded file in alphapapa/makem.sh#36

I don't have much experience debugging .elc so I'm having to put buttercup lower in my priority for now.

[alphapapa]

I was destroyed by some issue that only appears with a byte compiled loaded file in alphapapa/makem.sh#36

I don't have much experience debugging .elc so I'm having to put buttercup lower in my priority for now.

As I pointed out there, that is #218 in Buttercup. I've already made #219 to fix it, which I use locally when testing my packages.

CC @alphapapa no idea where discovery + reload code needs to live eventually, but it seems generally useful. All of these linters and test runners need to be able to discover and reload the project feature / elisp files. I think I will make some contributions to unpackaged and duplicate useful bits to elisp-repo-kit for turnaround time.

The discovery problems I've mostly solved in makem.sh. Reloading is handled by unpackaged/reload-package, but the usual caveats about reloading symbols apply (e.g. I can't reload ement while it's syncing a session, otherwise it would lose all the data in its symbols, which would cause errors; so first I ement-disconnect, then reload ement, then ement-connect again).