Usage feedback

[Kamirus]

Hey thanks for implementing the Motoko debugger! 🎉 I've tried it out and it works quite well!
But I've encountered some limitations that I'd like to ask if they are part of your current scope of this project.

I wanted to test it out on the new-base repo. I've gathered all my experiments in this PR. I wanted to debug one of the test files but the larger ones fail with an error index out of bounds: the len is 27 but the index is 30 and the other ones fail to produce a valid list output (usually File not found). For more details please have a look at the PR.

Overall, on small examples like this one the debugger seems to work really well!

[hapeeeeee]

I wanted to test it out on the new-base repo. I've gathered all my experiments in this PR. I wanted to debug one of the test files but the larger ones fail with an error index out of bounds: the len is 27 but the index is 30 and

Which file triggered this error?

the other ones fail to produce a valid list output (usually File not found). For more details please have a look at the PR.

I compiled the List.mo file and printed all the file entries extracted by motoko_debugger from the DWARF data. The output showed:

"./internals"
"<moc-asset>/prelude"
"<moc-asset>/prim"
"<moc-asset>/rts.wasm"

There are no List.mo file. Clearly, the DWARF information for List.mo has been completely lost. It seems that when the Motoko compiler encounters large files, it might merge the DWARF data of multiple source files into a single internals file, which could explain the error related to the List.mo file. Maybe we should ask the developers from the compiler team.

Another possible cause is that compiling without using absolute paths. Currently, the Motoko compiler’s generated wasm files contain incomplete DWARF information, missing a critical debug_info section. One important function of this section is to convert relative paths into absolute paths so the debugger can locate the files on disk. Therefore, if you compile your files using relative paths but run the debugger from a different directory, the debugger may fail to find the files, causing this error. See caffeinelabs/motoko#4938

Also, I tried compiling List.test.mo, and it showed:

moc -g ./test/List.test.mo 
./test/List.test.mo:1.1-1.33: import error [M0010], package "matchers" not defined
./test/List.test.mo:2.1-2.32: import error [M0010], package "matchers" not defined
./test/List.test.mo:3.1-3.32: import error [M0010], package "matchers" not defined
./test/List.test.mo:4.1-4.22: import error [M0010], package "test" not defined

[Kamirus]

The import errors are because you need to link the mops packages, like so moc $(mops sources). Here is the full command.
Make sure you have mops installed (probably npm i inside the repo is enough) and then execute mops install to get the packages.

[Kamirus]

I wanted to test it out on the new-base repo. I've gathered all my experiments in this PR. I wanted to debug one of the test files but the larger ones fail with an error index out of bounds: the len is 27 but the index is 30 and

Which file triggered this error?

Most of the test files trigger it, here is an example using test/List.test.mo

kamil.listopad ~/motoko-core (main) % moc $(mops sources) -o MoDe.wasm -g test/List.test.mo && ~/ic-sdk/target/debug/dfx debug MoDe.wasm       
thread 'main' panicked at crates/debugger/src/dwarf/mod.rs:476:34:
index out of bounds: the len is 27 but the index is 30
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
exit code: Some(101)
Finished debugging canisters.

Note that recently we've renamed new-motoko-base into motoko-core. Sorry for the confusion!

[Kamirus]

I compiled the List.mo file and printed all the file entries extracted by motoko_debugger from the DWARF data. The output showed:

Maybe the optimizer removed unused definitions? Try compiling another file that imports List, e.g. my provided test file src/SimpleDebuggerTest.mo

[hapeeeeee]

Maybe the optimizer removed unused definitions? Try compiling another file that imports List, e.g. my provided test file src/SimpleDebuggerTest.mo也许优化器删除了未使用的定义？尝试编译另一个导入 List 的文件，例如我提供的测试文件 src/SimpleDebuggerTest.mo。

I successfully compiled SimpleDebuggerTest.mo and run motoko_debugger, and once again encountered a File not found error for ./internals. What’s strange is that there is no internals file in the current directory.

After that, I executed the thread step-in command multiple times, and eventually it stepped into the List.mo file successfully.

At this point, I can't be certain whether the issue lies within the debugger or the DWARF information generated by the Motoko compiler. In fact, the debugger currently uses gimli, which is the most widely used DWARF-related library in the Rust ecosystem. So I believe the issue is unlikely to be caused by gimli itself.

My current hypothesis is that the issue may be triggered by one of the following two scenarios:

1. motoko_debugger's step-in command may not be correctly skipping over the function prologue (which is the setup code automatically inserted at the beginning of a function), and the Motoko function prologue might be associated with the ./internals file.
2. There may be a problem with how the Motoko compiler generates the line table (.debug_line section) in the DWARF data.

I believe resolving this issue will require support from the compiler team.

[hapeeeeee]

The index out of bounds issue occurs at the following line:

let filepath = self.paths[row.file_index() as usize - self.sequence_base_index].clone();

This line uses the file index from the line table entry to look up the corresponding file path. In theory, it should not panic, since both the line table entries (row) and the file entries (files) come directly from the .debug_line section in the DWARF data.

Therefore, it's highly likely that the issue originates from the Motoko compiler.

[Kamirus]

I successfully compiled SimpleDebuggerTest.mo and run motoko_debugger, and once again encountered a File not found error for ./internals. What’s strange is that there is no internals file in the current directory.

internals.mo is located in the motoko compiler, here is the file

[hapeeeeee]

internals.mo is located in the motoko compiler, here is the fileinternals.mo 位于 motoko 编译器中， 这是文件

OK, I think bundling this file directly into motoko_debugger could solve the "file not found" issue.

[Kamirus]

I've verified that indeed the .debug_line section was wrong, referencing too high file indices.
I have a proof-of-concept PR that demonstrates a fix with which it is possible to execute the List.test.mo.

The changes made to the src/wasm-exts/customModuleEncode.ml indeed resolved the exception. But the list command seems to stop working (no output).
However, the changes are not entirely correct. We'll discuss them internally in the motoko language team with @ggreif and @crusso. If you'd like to take a look at them and review I've attached instructions in the PR how I've compiled and run the custom moc.

[hapeeeeee]

internals.mo is located in the motoko compiler, here is the file

we have build internals.mo into motoko debugger. It can list the content in internals and not show file not found.
clone and pull this project, run

cargo install --release
./target/release/debugger /path/to/your.wasm

to see this update

[Kamirus]

internals.mo is located in the motoko compiler, here is the file

we have build internals.mo into motoko debugger. It can list the content in internals and not show file not found. clone and pull this project, run

cargo install --release
./target/release/debugger /path/to/your.wasm
to see this update

Thanks the file not found issue got resolved!

[Kamirus]

FYI all the .mo files in this directory are included in the <moc-asset>. I see that you've bundled these files into the debugger (in this commit).

I'm going to fix internals.mo so that it is included in the <moc-asset> directory not the . local directory (fix here).
So that this line can be replaced with

m.insert("<moc-asset>/internals", include_str!("source_code_in_moc/internals.mo"));

You can also add the timers-api file there as well.
Note: Bundling these files into the debugger is not ideal as when these files change they need to be updated in the debugger as well, but for now this is fine.