Skip to content

Commit 6440a19

Browse files
dblnzdblnz
committed
add documentation about the gdb debugging feature
Signed-off-by: Doru Blânzeanu <[email protected]>
1 parent a0bd86d commit 6440a19

File tree

3 files changed

+276
-0
lines changed

3 files changed

+276
-0
lines changed

‎docs/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ This project is composed internally of several internal components, depicted in
3434

3535
* [Security guidance for developers](./security-guidance-for-developers.md)
3636
* [Paging Development Notes](./paging-development-notes.md)
37+
* [How to debug a Hyperlight guest](./how-to-debug-a-hyperlight-guest.md)
3738
* [How to use Flatbuffers in Hyperlight](./how-to-use-flatbuffers.md)
3839
* [How to make a Hyperlight release](./how-to-make-releases.md)
3940
* [Getting Hyperlight Metrics, Logs, and Traces](./hyperlight-metrics-logs-and-traces.md)

‎docs/how-to-debug-a-hyperlight-guest.md

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
# How to debug a Hyperlight **KVM** guest using gdb
2+
3+
Hyperlight supports gdb debugging of a **KVM** guest running inside a Hyperlight sandbox.
4+
When Hyperlight is compiled with the `gdb` feature enabled, a Hyperlight KVM sandbox can be configured
5+
to start listening for a gdb connection.
6+
7+
## Supported features
8+
9+
The Hyperlight `gdb` feature enables **KVM** guest debugging:
10+
- an entry point breakpoint is automatically set for the guest to stop
11+
- add and remove HW breakpoints (maximum 4 set breakpoints at a time)
12+
- add and remove SW breakpoints
13+
- read and write registers
14+
- read and write addresses
15+
- step/continue
16+
- get code offset from target
17+
18+
## Expected behavior
19+
20+
Below is a list describing some cases of expected behavior from a gdb debug
21+
session of a guest binary running inside a KVM Hyperlight sandbox.
22+
23+
- when the `gdb` feature is enabled and a SandboxConfiguration is provided a
24+
debug port, the created sandbox will wait for a gdb client to connect on the
25+
configured port
26+
- when the gdb client attaches, the guest vCPU is expected to be stopped at the
27+
entry point
28+
- if a gdb client disconnects unexpectedly, the debug session will be closed and
29+
the guest will continue executing disregarding any prior breakpoints
30+
- if multiple sandbox instances are created, each instance will have its own
31+
gdb thread listening on the configured port
32+
- if two sandbox instances are created with the same debug port, the second
33+
instance logs an error and the gdb thread will not be created, but the sandbox
34+
will continue to run without gdb debugging
35+
36+
## Example
37+
38+
### Sandbox configuration
39+
40+
The `guest-debugging` example in Hyperlight demonstrates how to configure a Hyperlight
41+
sandbox to listen for a gdb client on a specific port.
42+
43+
### CLI Gdb configuration
44+
45+
One can use a gdb config file to provide the symbols and desired configuration.
46+
47+
The below contents of the `.gdbinit` file can be used to provide a basic configuration
48+
to gdb startup.
49+
50+
```gdb
51+
# Path to symbols
52+
file path/to/symbols.elf
53+
# The port on which Hyperlight listens for a connection
54+
target remote :8080
55+
set disassembly-flavor intel
56+
set disassemble-next-line on
57+
enable pretty-printer
58+
layout src
59+
```
60+
One can find more information about the `.gdbinit` file at [gdbinit(5)](https://www.man7.org/linux/man-pages/man5/gdbinit.5.html).
61+
62+
### End to end example
63+
64+
Using the example mentioned at [Sandbox configuration](#sandbox-configuration)
65+
one can run the below commands to debug the guest binary:
66+
67+
```bash
68+
# Terminal 1
69+
$ cargo run --example guest-debugging --features gdb
70+
```
71+
72+
```bash
73+
# Terminal 2
74+
$ cat .gdbinit
75+
file src/tests/rust_guests/bin/debug/simpleguest
76+
target remote :8080
77+
set disassembly-flavor intel
78+
set disassemble-next-line on
79+
enable pretty-printer
80+
layout src
81+
82+
$ gdb
83+
```
84+
85+
### Using VSCode to debug a Hyperlight guest
86+
87+
To replicate the above behavior using VSCode follow the below steps:
88+
- install the `gdb` package on the host machine
89+
- install the `C/C++` extension in VSCode to add debugging capabilities
90+
- create a `.vscode/launch.json` file in the project directory with the below content:
91+
```json
92+
{
93+
"version": "0.2.0",
94+
"configurations": [
95+
{
96+
"name": "GDB",
97+
"type": "cppdbg",
98+
"request": "launch",
99+
"program": "${workspaceFolder}/src/tests/rust_guests/bin/debug/simpleguest",
100+
"args": [],
101+
"stopAtEntry": true,
102+
"hardwareBreakpoints": {"require": false, "limit": 4},
103+
"cwd": "${workspaceFolder}",
104+
"environment": [],
105+
"externalConsole": false,
106+
"MIMode": "gdb",
107+
"miDebuggerPath": "/usr/bin/gdb",
108+
"miDebuggerServerAddress": "localhost:8080",
109+
"setupCommands": [
110+
{
111+
"description": "Enable pretty-printing for gdb",
112+
"text": "-enable-pretty-printing",
113+
"ignoreFailures": true
114+
},
115+
{
116+
"description": "Set Disassembly Flavor to Intel",
117+
"text": "-gdb-set disassembly-flavor intel",
118+
"ignoreFailures": true
119+
}
120+
]
121+
}
122+
]
123+
}
124+
```
125+
- in `Run and Debug` tab, select the `GDB` configuration and click on the `Run`
126+
button to start the debugging session.
127+
The gdb client will connect to the Hyperlight sandbox and the guest vCPU will
128+
stop at the entry point.
129+
130+
131+
## How it works
132+
133+
The gdb feature is designed to work like a Request - Response protocol between
134+
a thread that accepts commands from a gdb client and the hypervisor handler over
135+
a communication channel.
136+
137+
All the functionality is implemented on the hypervisor side so it has access to
138+
the shared memory and the vCPU.
139+
140+
The gdb thread uses the `gdbstub` crate to handle the communication with the gdb client.
141+
When the gdb client requests one of the supported features mentioned above, a request
142+
is sent over the communication channel to the hypervisor handler for the sandbox
143+
to resolve.
144+
145+
Below is a sequence diagram that shows the interaction between the entities
146+
involved in the gdb debugging of a Hyperlight guest running inside a KVM sandbox.
147+
148+
```
149+
┌───────────────────────────────────────────────────────────────────────────────────────────────┐
150+
│ Hyperlight Sandbox │
151+
USER │ │
152+
┌────────────┐ │ ┌──────────────┐ ┌───────────────────────────┐ ┌────────┐ │
153+
│ gdb client │ │ │ gdb thread │ │ hypervisor handler thread │ │ vCPU │ │
154+
└────────────┘ │ └──────────────┘ └───────────────────────────┘ └────────┘ │
155+
| │ | create_gdb_thread | | │
156+
| │ |◄─────────────────────────────────────────┌─┐ vcpu stopped ┌─┐ │
157+
| attach │ ┌─┐ │ │◄──────────────────────────────┴─┘ │
158+
┌─┐───────────────────────┼────────►│ │ │ │ entrypoint breakpoint | │
159+
│ │ attach response │ │ │ │ │ | │
160+
│ │◄──────────────────────┼─────────│ │ │ │ | │
161+
│ │ │ │ │ │ │ | │
162+
│ │ add_breakpoint │ │ │ │ │ | │
163+
│ │───────────────────────┼────────►│ │ add_breakpoint │ │ | │
164+
│ │ │ │ │────────────────────────────────────────►│ │ add_breakpoint | │
165+
│ │ │ │ │ │ │────┐ | │
166+
│ │ │ │ │ │ │ │ | │
167+
│ │ │ │ │ │ │◄───┘ | │
168+
│ │ │ │ │ add_breakpoint response │ │ | │
169+
│ │ add_breakpoint response │ │◄────────────────────────────────────────│ │ | │
170+
│ │◄──────────────────────┬─────────│ │ │ │ | │
171+
│ │ continue │ │ │ │ │ | │
172+
│ │───────────────────────┼────────►│ │ continue │ │ | │
173+
│ │ │ │ │────────────────────────────────────────►│ │ resume vcpu | │
174+
│ │ │ │ │ │ │──────────────────────────────►┌─┐ │
175+
│ │ │ │ │ │ │ │ │ │
176+
│ │ │ │ │ │ │ │ │ │
177+
│ │ │ │ │ │ │ │ │ │
178+
│ │ │ │ │ │ │ │ │ │
179+
│ │ │ │ │ │ │ vcpu stopped │ │ │
180+
│ │ │ │ │ notify vcpu stop reason │ │◄──────────────────────────────┴─┘ │
181+
│ │ notify vcpu stop reason │ │◄────────────────────────────────────────│ │ | │
182+
│ │◄──────────────────────┬─────────│ │ │ │ | │
183+
│ │ continue until end │ │ │ │ │ | │
184+
│ │───────────────────────┼────────►│ │ continue │ │ resume vcpu | │
185+
│ │ │ │ │────────────────────────────────────────►│ │──────────────────────────────►┌─┐ │
186+
│ │ │ │ │ │ │ │ │ │
187+
│ │ │ │ │ comm channel disconnected │ │ vcpu halted │ │ │
188+
│ │ target finished exec│ │ │◄────────────────────────────────────────┤ │◄──────────────────────────────┴─┘ │
189+
│ │◄──────────────────────┼─────────┴─┘ target finished exec └─┘ | │
190+
│ │ │ | | | │
191+
└─┘ │ | | | │
192+
| └───────────────────────────────────────────────────────────────────────────────────────────────┘
193+
```

‎src/hyperlight_host/examples/guest-debugging/main.rs

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
/*
2+
Copyright 2024 The Hyperlight Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
use std::sync::{Arc, Mutex};
18+
use std::thread;
19+
20+
use hyperlight_common::flatbuffer_wrappers::function_types::{ParameterValue, ReturnType};
21+
use hyperlight_host::func::HostFunction0;
22+
#[cfg(gdb)]
23+
use hyperlight_host::sandbox::config::DebugInfo;
24+
use hyperlight_host::sandbox::SandboxConfiguration;
25+
use hyperlight_host::sandbox_state::sandbox::EvolvableSandbox;
26+
use hyperlight_host::sandbox_state::transition::Noop;
27+
use hyperlight_host::{MultiUseSandbox, UninitializedSandbox};
28+
29+
/// Build a sandbox configuration that enables GDB debugging when the `gdb` feature is enabled.
30+
fn get_sandbox_cfg() -> Option<SandboxConfiguration> {
31+
#[cfg(gdb)]
32+
{
33+
let mut cfg = SandboxConfiguration::default();
34+
let debug_info = DebugInfo { port: 8080 };
35+
cfg.set_guest_debug_info(debug_info);
36+
37+
Some(cfg)
38+
}
39+
40+
#[cfg(not(gdb))]
41+
None
42+
}
43+
44+
fn main() -> hyperlight_host::Result<()> {
45+
let cfg = get_sandbox_cfg();
46+
47+
// Create an uninitialized sandbox with a guest binary
48+
let mut uninitialized_sandbox = UninitializedSandbox::new(
49+
hyperlight_host::GuestBinary::FilePath(
50+
hyperlight_testing::simple_guest_as_string().unwrap(),
51+
),
52+
cfg, // sandbox configuration
53+
None, // default run options
54+
None, // default host print function
55+
)?;
56+
57+
// Register a host functions
58+
fn sleep_5_secs() -> hyperlight_host::Result<()> {
59+
thread::sleep(std::time::Duration::from_secs(5));
60+
Ok(())
61+
}
62+
63+
let host_function = Arc::new(Mutex::new(sleep_5_secs));
64+
65+
host_function.register(&mut uninitialized_sandbox, "Sleep5Secs")?;
66+
// Note: This function is unused, it's just here for demonstration purposes
67+
68+
// Initialize sandbox to be able to call host functions
69+
let mut multi_use_sandbox: MultiUseSandbox = uninitialized_sandbox.evolve(Noop::default())?;
70+
71+
// Call guest function
72+
let message = "Hello, World! I am executing inside of a VM :)\n".to_string();
73+
let result = multi_use_sandbox.call_guest_function_by_name(
74+
"PrintOutput", // function must be defined in the guest binary
75+
ReturnType::Int,
76+
Some(vec![ParameterValue::String(message.clone())]),
77+
);
78+
79+
assert!(result.is_ok());
80+
81+
Ok(())
82+
}

0 commit comments

Comments
 (0)