[Bugfix] Under the session with a couple of symbol-files(.map etc.), fixes 'Reload all' button to expected behavior (loading entire symbols).

[mb3h]

[How to fix]

--- old/SymbolTable.cpp	2023-07-17 13:55:11.000000000 +0200
+++ src/SymbolTable.cpp	2025-01-16 00:00:00.000000000 +0000
@@ -548,7 +548,7 @@
 
 void SymbolTable::reloadFiles()
 {
-	for (int i = 0; i < symbolFiles.size(); ++i) {
+	for (int i = symbolFiles.size() -1; 0 <= i; --i) {
 		// check if file is newer
 		QFileInfo fi = QFileInfo(symbolFiles[i].fileName);
 		if (fi.lastModified() <= symbolFiles[i].refreshTime) continue;

[Detail of bug]

- SymbolTable::reloadFiles()
  - SymbolTable::unloadFile()
    - QList<SymbolFileRecord>::removeAt()

'SymbolTable::reloadFiles()' scanning the container 'symbolFiles[]' for replacing the old definition with the new one, but it doesn't work expectedly.
Because 'removeAt()' in loop breaks the order of the container.
(classical bug.)

[Note]
The session-file having '' tags and its definitions being older than symbol-file, it causes more wired behavior.

[m9710797]

Hi mb3h,

Thanks a lot for your (3) PRs!

Though before merging them I'd like to check something:
Are you aware that this openmsx-debugger application is abandoned? Meaning we will keep it available for people to study it. But we will no longer improve it or make new releases.
Instead we now have a debugger integrated into openMSX itself. Most of the functionality of this project (the old debugger) is already ported to the new debugger.

So I'd be happy to merge your PR (all 3 of them), though know that your improvements won't end up in new releases.
I would very much appreciate your help with the new debugger. E.g. I think the bug fix you made in this PR is not present in the new debugger, but it would be nice if you could double check that (I did not yet have time to look at your other PRs in detail).

Let me know how you'd like to continue.

[mb3h]

Thank you for early confirming. I'd like to reply your asking.

It's hard to help something about new debugger (probably you said integrated version on openMSX-20.0).
New version having seemed not convenient for me, I can't make the time for testing v20.0 and any motivation.

• layout problem (not comfortable for pivoted 1920 x 1200 monitor)
• collaboration problem (not convenient for my private tools, assembler and stuff)

Respecting openMSX, especially I've thought that it is the great achievement to teach us the accurate VRAM access timing of VDP commands.
But, splitted debugger is more comfortable than integrated it for me, I can't contribute your suggestion, sorry.

And it isn't the problem not to end up in new releases for me.
Because, this pull-request also has the deferent important purpose with contributing openMSX.
It is for adding my resume the reliability of my engineering skill through C++17/Qt (aiming to get more applicable position).
If merged, I'm pleased with you, but the above is my personal purpose, therefore, if it doesn't contribute openMSX, I can't require anything.

I want you to choose to spend your time or not.

[MBilderbeek]

Can you indicate which parts of the built-in debugger should be improved to make it comfortable enough to become your preferred solution?

[mb3h]

If it means "How can we update Debugger without separating from openMSX?", it is hard for me to answer it because it is the cost and risk for me to be integrated itself.
However, I can answer the question "What do you need for contributing not 'openMSX/debugger' but 'openMSX/openMSX' with keeping integrating in the latest openMSX?".
It is to append one configuration about building.

• 1. Identify the modules needed for compiling Debugger not including openMSX (*1)
• 2. Disable the coding with tightly coupled on them or change to it with loosely coupled (probably use something like '#ifdef DEBUGGER_ONLY') (*1)
• 3. Append a bit of files for build-system (Linux: Makefile.am, Windows: makefile, MacOS: ???) (*1)(*2)

I can appreciate that having integrated is the result of choosing and focussing.
The time is limited for all of people, contributers are quite less than users, therefore probably it's important to focus majority for the efficiency.
That's often done even on the product of company with benefit, so I think it would rather tend to cause it on the works without benefit.

However, it is the normal thing that not important stuff for one is integral for another one.
For example on my PR-3, it is just for this that I don't erase the behavior of past version and have left it.
I can appreciate this judgement is difficult and suffers us.

On Open-Source-Software, "Free not to update" exists if it is needed, I think (if the problem of security and stuff doesn't exist).

I expect to be allowed it.

(*1)
If having completed my activity mentioned on the previous reply and catched my breath, probably I can help 1. 2. 3.

(*2)
It is thought that 'GNUmakefile' or '.vcproj' isn't needed.
Because probably this is demand for the limited people who customize openMSX or Debugger themselves (like I am so), therefore, it is hard to think that they can't use 'autotools' or 'nmake'.
(Not having Macintosh, I don't know it about MacOS)

[MBilderbeek]

What I mean is: what is the reason for you to stick to the separate debugger? What is missing or wrong for you in the integrated debugger?
Or is it only about it being separate? If yes, what's the reason you find it so important?

[mb3h]

What's getting unable to do on integrated:

• (time cost) Customize Debugger/openMSX without analyzing again
• (time cost) Keep behavior of customized version without spending anything
• (efficiency) Customize them on ramdisk without disturbing other works on it
• (efficiency) Achieve all of the following (*1)
    - maximizing the speed of building
    - maximizing the speed of resuming (from storage to ramdisk)
    - minimizing the space on storage
    - minimizing the wear and tear of storage
• (risk for future) Connect to other Z80 emulators (including another MSX emulator)

Several years ago, on downloading openMSX as not integrated version, it was almost perfect for my purpose.
Having fixed the bug only once, but I've only customized something else for all of personal purpose.

Confirming the latest version, the coding of Debugger was changed to not on Qt, so the time cost has more increased for analyzing again.
For people not having any motive to use openMSX of the latest version, integrated Debugger is neither convenient nor comfortable, I think.
At least, until getting circumstance enoughly afordable to pay this cost.

(*1)
Per each '-ggdb' and not, it is archived for the middle files(.o, .d, Version.ii, moc_XXX and stuff) matched with source codes(.cpp, .h and stuff) on storage (only base version).
Managing filetime correctly, it is optimized for the efficiencies of development, storage and ramdisk.

[sdsnatcher]

What I mean is: what is the reason for you to stick to the separate debugger? What is missing or wrong for you in the integrated debugger? Or is it only about it being separate? If yes, what's the reason you find it so important?

@MBilderbeek
I have to support his decision. The internal debugger is simply incomplete for heavy MSX development and works only for the "oh, I'm curious and want to play around a bit" use case.

While it's true that the external debugger still have some issues, it "serves it purpose most of the time". And the new internal debugger has some interesting fresh ideas, it's also true that it simply has some nonsensical design decisions:

• I've never seen a debugger where the developer has to manually re-create all the breakpoints/watchpoints/watches etc every single time he runs the code. This is just counter productive and turns it into a toy debugger
• Many features have regressed to the point where the external debugger was some 10 years ago.
  • Data that I could simply copy/paste now require manual typing, or typing console commands
  • The external debugger highlighted in red the data changed in any widget
  • Excessive use of mouse hover. It's not possible to keep that item selected and the hover information is just gone after you move the mouse away from the window
  • The important state of the VDP register latches (command register and palette registers) is not shown anywhere anymore
  • The important state of the VDP VRAM pointer (read or write) is not shown anywhere anymore

And some very old limitations are still present on the new debugger:

• Slot/MegaROM/MemoryMapper breakpoint support is still limited to the PC. It's simply not possible to add a watchpoint for a byte read/write on a specific Slot/MegaROM/MemoryMapper page.
• Adding VRAM write watchpoints is so convoluted that most user simply won't know how to do it. It requires setting "too_fast_vram_writes = broken" and a watchpoint to writes on port 98h.
• Similarly, adding VRAM read watchpoints is convoluted, since not all reads go to the register A.
• At least having an hex window pointing to the current VRAM pointer position is now possible, but it's not user friendly at all. It requires writing a Tcl formula involving the VRAM pointer debuggable and the VDP register R#14. Most users will struggle to do that.
• It's not possible to have watchpoints triggered by register of the other ICs having a certain value. All breakpoints are CPU based, but the MSX has other ICs (VDP, Y8950 etc) that have a live of their own.
• The debugger is still limited to what the CPU can do. IOW, it's not possible for it to directly modify the contents of a ROM. It's even worse for MegaROMs, since values written to the hex viewer will usually trigger page changes

[MBilderbeek]

It would be very helpful if you'd make an issue for any of these points (if they don't have one already) in the openMSX tracker, instead of making a new list here in the old debugger issue tracker.

[m9710797]

While it's true that the external debugger still have some issues, it "serves it purpose most of the time". And the new internal debugger has some interesting fresh ideas, it's also true that it simply has some nonsensical design decisions:

It's hard to reply to such a highly subjective post. I can only say that we've received a lot more "positive" feedback than "negative". Including reports that the new debugger enables some "essential" stuff that wasn't possible before (some examples: visualizing VDP commands, multiple bitmap/tile viewers (to look at multiple pages at the same time), ...). But such reports are also subjective of course.

Now let's try to stay objective for the rest of this message.

Sure, there will be some missing features in the new debugger compared to the old.

• Some are just not implemented yet, or were overlooked. E.g. VDP register latches or VRAM pointer. I overlooked those, but these should be easy to add. Please create tickets for these.
• Some were intentionally left out. But I understand that for specific users for specific use cases exactly that feature was important. Also create (separate) tickets for those items. Then we can work out how to re-add it (maybe in a slightly modified way .. and feedback remains very important). It also helps to include a description of your use case (why you think it's a useful feature). "It was present in the old debugger" by itself is not always a convincing reason.
• The new and old debugger have a vastly different design. E.g. the most requested feature for the old debugger (except for near unusable docking) was the lack of up-to-date (real-time updated) views. But that was near impossible to implement in the old design. In the new design all views show real-time information, in fact showing non-real-time info is now a_lot harder. But this makes highlighting changes harder (not impossible but a lot harder). So different designs, different tradeoffs.

You also list some missing features / limitations present in both versions.

• Most of these are there because of limitations in the openMSX emulation model itself.
• E.g. arbitrary watchpoints in arbitrary devices is technically possible of course, but it comes with a big performance cost.
• All the current openMSX debug features have near zero emulation overhead when not used.
• 20 years ago this was very important. Maybe today less so, but we also get reports that openMSX emulation is too slow. Different use cases require different tradeoffs. It's not always possible to please everyone.

Last, I fully agree with MBilderbeek. Please open separate tickets for the issues you mentioned on the correct project (not this one). E.g. some of the issues you raised should be easy to fix, but were never mention (by anyone) before. So without a tickets we don't know about them, and they won't get fixed.

[sdsnatcher]

While it's true that the external debugger still have some issues, it "serves it purpose most of the time". And the new internal debugger has some interesting fresh ideas, it's also true that it simply has some nonsensical design decisions:

It's hard to reply to such a highly subjective post. I can only say that we've received a lot more "positive" feedback than "negative". Including reports that the new debugger enables some "essential" stuff that wasn't possible before (some examples: visualizing VDP commands, multiple bitmap/tile viewers (to look at multiple pages at the same time), ...). But such reports are also subjective of course.

Please don't get me wrong. The new debugger has many strong points. It's just that @MBilderbeek asked a question about reasons to stick with the old debugger and what was missing, and was very direct on the feedback. No need to get on defensive mode, since I'm not against the new debugger. (ok, maybe I was too direct and might have given this impression).

While the new debugger doesn't have most of the features that I listed here that the old debugger had, I'll inevitably have to stick with the old.

Last, I fully agree with MBilderbeek. Please open separate tickets for the issues you mentioned on the correct project (not this one). E.g. some of the issues you raised should be easy to fix, but were never mention (by anyone) before. So without a tickets we don't know about them, and they won't get fixed.

The feedback also intended to simply discuss what points would be acceptable or not. Because here it's still "pub chatting": less structured than opening issues, since those require a lot more time to detail them in an acceptable form. If a point is not acceptable even in a draft form like listed here, then I wouldn't waste time detailing lots of issues there just for them to be rejected.

[MBilderbeek]

I think it's fine to create issues for all of them. Use a brief description and we'll work it out together as we go.

Some things are already (partly) covered:

The external debugger highlighted in red the data changed in any widget - openMSX/openMSX#1657

The debugger is still limited to what the CPU can do. IOW, it's not possible for it to directly modify the contents of a ROM. It's even worse for MegaROMs, since values written to the hex viewer will usually trigger page changes - openMSX/openMSX#193

Please open issues for all other items, in the openMSX core issue tracker. Perhaps you can already use the information Wouter posted in his reply.

[sdsnatcher]

Nice! Thanks for the feedback.

But right now I'm juggling too many things, so it might take some time until I'm able to create the issues and detail the there.

[MBilderbeek]

Take your time, it's fine to do e.g. one issue per time. (I suggest with the most important!)